9.23 Workaround for "Arguments too long" Error

When the shell matches a wildcard (15.1) in a big directory or with long pathnames (14.2, 15.6), it can sometimes run out of room. You'll get an error like this:

pr	% pr */* | lpr Arguments too long.

Sometimes you can work around that. The trick is to split the command line into pieces with semicolons (8.5)- and use a subshell (13.7) to combine the outputs. For example, I rewrote the previous command like this:

% (pr [a-f]*/*;pr [g-m]*/*;pr [n-z]*/*) | lpr

The first command prints the files in directories whose names start with "a" through "f," and so on.

How did I decide where to split? There's no magic formula. The number of pieces you'll need and the way you divide them will depend on how many directories and files you're trying to match - and your version of UNIX. Do it by experiment. A dummy command like true that ignores its arguments is good for this. In the example above, I first tried splitting the arguments in half. Then I split them more. I did the same for other chunks until the shell was happy with all of them:

% true [a-m]*/* 
Arguments too long.
% true [a-f]*/* 
% true [g-z]*/* 
Arguments too long.
% true [g-m]*/* 
% true [n-z]*/*

This trick works fine for commands like pr that make regular output that is consistent whether you run separate chunks of files or do all at the same time. Some commands start each listing with a separate heading - for instance, ls -l prints total n before it lists a directory. That kind of command won't work as neatly with this trick because you'll get several headings mixed in with the output instead of just one. Still, it might be better than nothing!