ShellCheck
| Original author(s) |  The Norwegians, specifically Vidar Holen |
|---|---|
| Repository | github.com/koalaman/shellcheck |
| Written in | Haskell |
| License | GPLv3 |
| Documentation | github.com/koalaman/shellcheck/blob/master/README.md |
| Website | www.shellcheck.net |
ShellCheck is like a SpellCheck for shell scripts. It will analyze your shell scripts and point out many of the embarrassing mistakes, flaws and potential corner-case problems you made writing them, and propose changes. ShellCheck is not a magic bullet that will detect and point out everything that is wrong with any and all shell scripts, and a shell script may not work as expected, or at all, even if ShellCheck fails to produce any warnings. It does catch a lot, and it is absolutely worth using on every shell script you write or use even if your shell scripts seemingly look and work just fine.
ShellCheck is not for everyone, and understanding it's output does require some computer knowledge. It isn't for you if you have no idea what a shell or a shell script is. You may want to read Bash Guide for Beginners if that's the case. Everyone who has written, or just used, a shell script will find ShellCheck to be a great, and perhaps essential, tool.
Features And Usability[edit]
ShellCheck checking a shell script for errors.
Writing a simple shell script that will run on just about every Linux machine you will encounter seems really easy the first time you encounter the concept. Just write #!/bin/sh into a file and a series of commands you'd like to be executed in sequence, save it and volla, you're done.
#!/bin/sh
echo Hello world!
echo look at all these files!!!
ls
Just chmod +x MyFirstShellScript.sh to make it executable and run it with
./MyFirstShellScript.sh
..and now you're a "bash programmer". That proudness won't last long, you will quickly realize that shell scripting is, in fact, not that easy:
#!/bin/sh
echo Hello world! I can manipulate files!
echo This is my file!! > my file.txt
This code may look super duper at first glance, but there are some minor problems with it. First of all, no file named my file.txt will be created. What you get is a file called my and it will contain:
This is my file!! file.txt
You may wonder what goes wrong with that script and check it with ShellCheck. That won't help, shellcheck MySecondShellScript.sh will not produce any warnings in this case. It's fine as far as ShellCheck is concerned.
ShellCheck will not a magic program that will magically detect everything that is wrong and broken in any and all shell scripts. It is not some kind of artificial intelligence that will deduce what you are trying to do and explain what you are doing wrong and how to do it right in any and all cases.
Now, let's make a script that is somewhat similar to the one above:
#!/bin/sh
echo "Hello world! I can manipulate files!"
text = "This is my file!"
file = "my file.txt"
echo $text > $file
...and run shellcheck ScriptWithVariables.sh on that one - now the potential usefulness of ShellCheck becomes more apparent:
In ScriptWithVariables.sh line 2:
text = "This is my file!"
^-- SC1068: Don't put spaces around the = in assignments (or quote to make it literal).
In ScriptWithVariables.sh line 3:
file = "my file.txt"
^-- SC1068: Don't put spaces around the = in assignments (or quote to make it literal).
In ScriptWithVariables.sh line 4:
echo $text > $file
^---^ SC2086: Double quote to prevent globbing and word splitting.
^---^ SC2086: Double quote to prevent globbing and word splitting.
Did you mean:
echo "$text" > "$file"
For more information:
https://www.shellcheck.net/wiki/SC1068 -- Don't put spaces around the = in ...
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing ...
Now we can use the proposals from ShellCheck and fix our little script so there are no spaces around the = and quotes around the variables (see Bash Guide for Beginners Chapter 3, "Quoting characters"):
#!/bin/sh
echo "Hello world! I can manipulate files!"
text="This is my file!"
file="my file.txt"
echo "$text" > "$file"
ShellCheck does pick up on a lot of things you may not notice or think about when you write a bash script. It won't magically understand the intention of every line and fix every little error for you, but there is a lot of things it will catch and point out.
Shebang Awareness[edit]
ShellCheck cares about the shell you specify with #! in the very first line of your shell scripts.
Most GNU/Linux distributions have a symbolic link pointing from /bin/sh to /bin/bash. You can typically start your scripts with #!/bin/sh and use all kinds of advanced bash extensions since it is bash, not some ancient POSIX compliant Bourne shell binary, that will interpret your shell script.
ShellCheck assumes that you will be using the ancient Bourne shell, or another POSIX compliant shell, if you use the #!/bin/sh shebang. It will point out potential problems, that won't apply to your machine, if you use #!/bin/sh with the assumption that it will always be bash who is interpreting your scripts. Consider this:
#!/bin/sh
function runbenchmark(){
for f in 20-anime-girls-with-questionmarks/*.jpg;do
outputf="output${1}/${f/20-anime-girls-with-questionmarks\//}"
echo -n .
"$HOME/bin/realsr-ncnn-vulkan" \
-m "$HOME/bin/realsr-ncnn-vulkan-models/models-DF2K_JPEG" \
-g "${1}" -i "$f" -o "$outputf" 2>/dev/null
done
echo
}
ShellCheck will, in the case of the above script with a single function and nothing using it, point out that:
In 1.sh line 2:
function runbenchmark(){
^-- SC2112: 'function' keyword is non-standard. Delete it.
In 1.sh line 4:
outputf="output${1}/${f/20-anime-girls-with-questionmarks\//}"
^-- SC2039: In POSIX sh, string replacement is undefined.
In 1.sh line 5:
echo -n .
^-- SC2039: In POSIX sh, echo flags are undefined.
ShellCheck is correct, there would be a problem if that function where ever used in a machine where #!/bin/sh isn't a symbolic link to bash. Replacing #!/bin/sh with #!/bin/bash makes those errors go away.
ShellCheck will identify sh (seen as POSIX), bash, ksh and dash shells. Any support for other shells is not indicated in src/ShellCheck/Checker.hs.
ShellCheck will also warn you if you do not have any shebang. A single one-liner like:
QT_AUTO_SCREEN_SCALE_FACTOR=1 "$HOME/bin/monero-wallet-gui" &
makes ShellCheck tell you that:
In monero line 1: QT_AUTO_SCREEN_SCALE_FACTOR=1 "$HOME/bin/monero-wallet-gui" & ^-- SC2148: Tips depend on target shell and yours is unknown. Add a shebang or a 'shell' directive. For more information: https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and y...
Specifying what shell should be used may not seem like a big deal, and it's probably not if you'll stick to the same machine running the same operating system forever. Problems can and probably will arise the moment you copy a script that doesn't specify what shell should run it from one box to a very different box. That is specially true if the default shells on those boxes differ.
Checking Multiple Files[edit]
ShellCheck accepts long lists of files as arguments. You can use wildcards and check entire folders at once.
Limitations[edit]
ShellCheck won't find every little thing that's wrong with your scripts. It's not magic. And it will even get things wrong. Consider this:
#!/bin/bash
parallel -j$(nproc) convert {} {.}.webp ::: *.png
ShellCheck believes the GNU Parallel statement is flawed and warns that:
In png2webp.sh line 2:
parallel -j$(nproc) convert {} {.}.webp ::: *.png
^------^ SC2046: Quote this to prevent word splitting.
^-- SC1083: This { is literal. Check expression (missing ;/\n?) or quote it.
^-- SC1083: This } is literal. Check expression (missing ;/\n?) or quote it.
There is no reason to check that expression. GNU Parallel works like that. It's fine.
You can't expect that ShellCheck will know what every random program your scripts call with arguments do with those arguments. It may get some things wrong. It is somewhat right about -j$(nproc) even though $(nproc) will always be a whole number of available CPU threads. -j$(nproc) is the default -j option for GNU Parallel, so it doesn't need to be there at all.
Verdict And Conclusion[edit]
ShellCheck is something you should install and use if you write shell scripts - even if they are mostly very small and seemingly trivial. Even something as short and silly as:
#!/bin/bash
for f in *.txt ;do
cat $f
done
can work fine and seem fine - and it's all good until someone drops a file named "my file.txt" into the folder where it runs. ShellCheck will tell you things like
In catthemall.sh line 3:
cat $f
^-- SC2086: Double quote to prevent globbing and word splitting.
and a lot of other useful things you may not notice. Consider this:
#!/bin/bash
if [ ! -f{${1} ];then
exit 0
fi
ffmpeg -i "$1" -c:v libvpx-vp9 \
-b:v 0 -crf 18 \
-threads 4 \
-speed 1 \
-f webm "/tmp/gif2webm.webm"
ffmpeg -i /tmp/gif2webm.webm -map 0:0 -c:v libvpx -b:v 2M "${1}.webm"
rm /tmp/gif2webm.webm
So you see anything wrong with the above? There is something wrong, and ShellCheck would tell you:
In gif2webm.sh line 3:
if [ ! -f{${1} ];then
^-- SC1035: You are missing a required space here.
^-- SC1083: This { is literal. Check expression (missing ;/\n?) or quote it.
^--^ SC2086: Double quote to prevent globbing and word splitting.
The if statement if [ ! -f{${1} ];then will actually always be false since -f{${1} will always be true and we are checking for not true with [ ! ... ].
if [ ! -f "${1}" ];then makes the -f expression do what you would expect it to do: check if the scripts first argument is a file in the current folder.
The erroneous if statement is a pretty good example of why using ShellCheck is a good idea: bash won't tell you that there's anything wrong with -f{${1}, it will just interpret it as true and move along. ShellCheck can help you figure out what's going wrong in cases where there are no apparent errors from bash or other shells yet the script isn't doing what you expect it to do.
We can strong recommend ShellCheck for both Linux novices who just begun reading Bash Guide for Beginners and experienced shell "programmers" who know the Advanced Bash-Scripting Guide by heart.
Installation[edit]
All the major GNU/Linux distributions have it in their repositories, though the name will vary. Some call it shellcheck (Debian, Arch, Gentoo, OpenBSD), some call it ShellCheck (RHEL, Fedora, openSUSE) and FreeBSD calls it hs-ShellCheck. iToddlers can install it as shellcheck using something called "brew".
Links[edit]
- GitHub Page: https://github.com/koalaman/shellcheck
- www.shellcheck.net, online shell script checker
Enable comment auto-refresher
Andy5995
Permalink |