GNU Parallel Ginsburg Is Released
The latest version of the most comprehensive tool for running terminal commands in parallel has new CPU detection code for GNU/Linux, bug fixes and a few changes to the very long and detailed manual page.
written by 윤채경 (Yoon Chae-kyung) 2020-09-23 - last edited 2020-09-23. © CC BY
The current GNU Kart from SuperTuxKart git. GNU Parallel is a GNU project.
GNU Parallel is a hidden gem useful for doing all kinds of jobs in terminals where you would normally do something like:
for f in *.png; do
convert $f ${f%.*}.webp
done
convert is a single-threaded program, so if you have many cores and you want to use them all you can do:
parallel -j$(nproc) convert {} {.}.webp ::: *.png
or, simply:
parallel convert {} {.}.webp ::: *.png
because GNU Parallel can detect how many CPU cores the machine has and -j$(nproc) is the default. This CPU detection has been improved in the latest GNU Parallel Ginsburg release.
GNU Parallel replaces {} with the input file and replaces {.} with input file names without the file extension. ::: is used to separate between the command to be executed and the files (or other arguments) each process should work on.
You can also pipe a list of files to it:
find recordings -type f -name '*.wav' | parallel ffmpeg -i {} -vn -sn -c:a flac -compression_level 6 {.}.flac
There are plenty of other examples in the very long and extremely detailed manual page. That manual page has seen some minor updates since the last release in May. A close-up inspection of the diff between the old and the new manual page reveal that a few examples have been altered and that's about it.
There is a paper-back book by GNU Parallel author Ole Tange from 2018 for sale at sa store called "Lulu" in case you find that the long detailed manual page isn't enough for you.
The source for the latest GNU Parallel version can be acquired from gnu.mirrors.hoobly.com/parallel/. It's a very small and quick compile with near-zero dependencies.
GNU Parallel really is quite handy if you need to process large amounts of files using some single-threaded program on a regular basis. It gives a bit more options than xargs -P (which can also be used to run jobs in parallel).
Enable comment auto-refresher