Executing commands in parallel in Shell and other fun stuff

Recently, I was required to generate multiple UUID’s and pass onto HTTP request as a paremeter. I thought of using Python or Node.js for this purpose, but thought why not use a plain old shell.

But, is there a way to generate UUID in shell? Turns out, there is using uuidgen command which works on both Linux and MacOS.

Hence, using the power of another command called seq, and xargs we can achieve many powerful things on shell

Sequentially print 1 to 10000:

seq 1 10000

Print and pipe to xargs command for printing the output using single thread

seq 1 10000 | xargs -I {} -P 1 printf '{}\n'

Parallely print 1 to 10000 using 10 threads:

seq 1 10000 | xargs -I {} -P 10 printf '{}\n'

You can test out the parallelism using time command, which will print the output for above 2 commands. For me output is:

Executed in   14.23 secs   fish           external
   usr time    3.88 secs  167.00 micros    3.88 secs
   sys time    9.31 secs  1005.00 micros    9.31 secs
Executed in    5.08 secs   fish           external
   usr time    4.77 secs  229.00 micros    4.77 secs
   sys time   16.05 secs  1027.00 micros   16.05 secs

Another way to confirm the parallelism, is to observe that, with parallelism, numbers are printed slightly jumbled in between.

Using, the above fundamental knowledge, we can generate multiple UUID’s using uuidgen command as follows:

seq 1 10 | xargs -I {} -P 1 uuidgen

Output will be:


How about using this uuid’s in a curl command:

seq 1 10 | xargs -I {}  uuidgen | xargs -I{} -n1 -P 10 curl -X GET 'http://google.com/{}'

Hope you enjoyed little fun tricks in shell