Linux parallel: how to execute a command in parallel and get arguments from a file

Parallel is a command that can help you do your work faster because it can execute your commands in parallel.

Scenario: i had to execute a script vs hundred of devices, the script could take as a parameter the ip address, to run this for each ip address is very slow, so we need a way to run the script for 10 ip addresses in parallel.

How to do this: the script is executed in this form:

$ ./myscript -i <ip_address>

I created a file named ip_addresses.txt which each line is an ip address

10.0.0.1
10.0.0.2
...
10.0.0.100

and i executed parallel:

$ parallel -j10 --xapply ./myscript -i ::: <(cat ip_addresses.txt)

What it does:

  • parallel is the command that allows parallel execution
  • -j10 means run in batches of 10 parallel jobs
  • — xapply: apply for each argument, its like a “ for arg in (arguments)” statement
  • ::: actually means put the argument here
  • <(cat ip_addresses.txt) read file with arguments and pass it to parallel

I hope you enjoyed this article :)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store