Linux parallel: how to execute a command in parallel and get arguments from a file
Parallel is a command that can help you do your work faster because it can execute your commands in parallel.
Scenario: i had to execute a script vs hundred of devices, the script could take as a parameter the ip address, to run this for each ip address is very slow, so we need a way to run the script for 10 ip addresses in parallel.
How to do this: the script is executed in this form:
$ ./myscript -i <ip_address>
I created a file named ip_addresses.txt which each line is an ip address
and i executed parallel:
$ parallel -j10 --xapply ./myscript -i ::: <(cat ip_addresses.txt)
What it does:
- parallel is the command that allows parallel execution
- -j10 means run in batches of 10 parallel jobs
- — xapply: apply for each argument, its like a “ for arg in (arguments)” statement
- ::: actually means put the argument here
- <(cat ip_addresses.txt) read file with arguments and pass it to parallel
I hope you enjoyed this article :)