The Linux Philosophy for SysAdmins, Tenet 08 — Always use shell scripts
Author’s note: This article is excerpted in part from chapter 9 of my book, The Linux Philosophy for SysAdmins, with some changes to update the information in it and to better fit this format.
When writing programs to automate — well, everything — always use shell scripts. Because shell scripts are stored in ASCII text format, they can be easily viewed and modified by humans just as easily as they can by computers. You can examine a shell program and see exactly what it does and whether there are any obvious errors in the syntax or logic. This is a powerful example of what it means to be open.
I know a few developers who tend to consider shell scripts something less than true programming. This marginalization of shell scripts and those who write them seems to be predicated on the idea that the only true programming language is one that must be compiled from source code to produce executable code. I can tell you from experience this is categorically untrue.
I have used many languages including BASIC, C, C++, Pascal, Perl, Tcl/Expect, REXX and some of its variations including Object REXX, many shell languages including Korn and Bash, and even some assembly language. Every computer language ever devised has had one purpose – to allow humans to tell computers what to do. When you write a program, regardless of the language you choose, you are giving the computer instructions to perform specific tasks in a specific sequence.
Definition
A shell script or program is an executable file that contains at least one shell command. They usually have more than a single command and some shell scripts have thousands of lines of code. When taken together, these commands are the ones necessary to perform a desired task with a specifically defined result.
Although an executable file containing a single line with a shell command can be run with the current shell, it is good practice to add a line called the “shebang” that defines the shell under which the program is to run. Let’s try it both ways. This experiment should be performed as a non-root user. Create a minimal script in your home directory, make it executable, and run it.
First open a new file in your home directory with vim.
$ vim test1Add one line at the beginning of the file and save the file. Do not exit from vim because we will be making more changes to the test1 script.
echo “Hello world!”In another terminal session do a long listing of the new program.
$ ls -l test1
-rw-rw-r-- 1 student student 20 Dec 31 15:27 test1The file permissions show that it is not executable. Make it executable for the user and the group and list it again.
$ chmod ug+x test1
$ ls -l test1
-rwxrwxr-- 1 student student 20 Dec 31 15:38 test1Now lets run the program. We use ./ before the name of the file to specify that the program file is located in the current directory. Home directories are not part of the path so we must specify the path to the executable file.
$ ./test1
Hello world!Now lets add the shebang line before the echo command. This specifies that no matter which shell is running in the termnal, the program will always run under the Bash shell.
Now our program as two lines and looks like this.
#!/bin/bash
echo "Hello world!"Run the program again. The results should not change. Exit from vim.
For a simple shell script like this one, it does not matter whether we add the shebang line. All of the shells in which I experimented with this script produced the same results. But there are some built-in shell commands that may not exist in other shells, or some commands may be implemented differently and the different results may affect the outcome of the program when run.
Regardless, it is always good practice to include the shebang line.
The SysAdmin context
Context is important and this tenet, “Always use shell scripts,” should be considered in the context of our jobs as SysAdmins.
The SysAdmin’s job differs significantly from those of developers and testers. In addition to resolving both hardware and software problems we manage the day to day operation of the systems under our care. We monitor those systems for potential problems and make all possible efforts to prevent those problems before they impact our users. We install updates and perform full release level upgrades to the operating system. We resolve problems caused by our users. SysAdmins develop code to do all of those things and more; then we test that code; and then we support that code in a production environment.
Many of us also manage and maintain the networks to which our systems are connected. In other cases we tell the network admins where the problems are located and how to fix them because we find and diagnose them first.
We SysAdmins have been DevOps far longer than that term has been around. In fact the SysAdmin job is more like DevTestOpsSecNet than just DevOps. Our knowledge and daily task lists cover all of those areas of expertise.
In this context the requirements for creating shell scripts are complex, inter-related, and many times contradictory. Let’s look at some of the typical factors SysAdmins must consider when writing shell scripts.
Requirements
This means that we need to obtain a set of requirements from the end user who is requesting the script. Even if we happen to be both developer and user, we should sit down and create a set of requirements before we begin to write code.
Even a short list of two or three objectives for the program will suffice as a set of requirements. The minimum I will accept is a description and sample of the input data; any formulas, logic, or other processing required; and a description of the required outputs or functional results. Of course more is better, but with these things as a starting point I can begin work.
Naturally the requirements will become more explicit as the project continues. Things that were not considered initially will arise. Assumptions will be changed.
Development speed
Programs usually must be written quickly to meet time constraints imposed by circumstances or the PHB. Most of the scripts we write are to fix a problem, to clean up the aftermath of a problem, or to deliver a program that must be operational long before a compiled program could be written and tested.
Writing a program quickly requires shell programming because it allows quick response to the needs of the customer whether that be ourselves or someone else. If there are problems with the logic or bugs in the code they can be corrected and retested almost immediately. If the original set of requirements was flawed or incomplete, shell scripts can be altered very quickly to meet the new requirements. So, in general, we can say that the need for speed of development in the SysAdmin’s job overrides the need to make the program run as fast as possible or to use as little as possible in the way of system resources like RAM.
Let’s look at the BASH command line program in Figure 1. It is designed to list each user ID that is currently logged into the system.
echo `who | awk '{print $1}' | sort | uniq` | sed "s/ /, /g" Figure 1: A CLI program to list logged in users.
Because users may be logged in multiple times, this one line program only displays each ID once, and separates the IDs with commas. To program this in the C language would require a significant amount of single-purpose code. The table below shows the number of lines of code in each of the CLI commands used in the above BASH program. These numbers were accurate when I found them several years ago. If they have changed since then it would not be significant.
| Command | Source Lines of Code |
| echo | 177 |
| who | 755 |
| awk | 3,412 |
| sort | 2,614 |
| uniq | 302 |
| sed | 2,093 |
| TOTAL | 9,353 |
You can see that the BASH script above uses programs that together contain 9,353 lines of C code. All of these programs contain far more functionality than that which we actually use in our script. Yet we combine these programs that have already been written and use the parts we need.
It takes far less time to write and test the resulting BASH script than it would to create a new compiled program to do the same thing.
Performance speed
Script performance in terms of speed of execution is much less relevant now than in the past. Today’s CPUs are blazing fast and most computers have multiple processors. Most of my own computers have 4 processors with Hyperthreading and run at 3GHz or higher. My main workstation has an Intel Core i9 with 16 processors and 32 threads. I tend to have a large number of virtual machines open simultaneously while working on various projects including research for my books and articles.
$ echo "The Lazy SysAdmin uses the tools at hand." | cowsay
_____________________________________
/ The Lazy SysAdmin uses the tools at \
\ hand. /
-------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||In general, the only question to ask is whether the job gets done in time. If it does then no worries. If it does not, the time required to write and test the same program in a compiled language would most likely have made it even later. The time saved when the compiled program runs is less than the time saved in development when using a shell program. Remember we are considering the context of the SysAdmin’s job.
Consider the example program in Figure 1 and the amount of C code in Figure 2. The fact is that our example CLI program is still using large amounts of C code that has already been written and extensively tested. As lazy SysAdmins we have lots of C code already available in the form of the Linux Core Utilities and other command line utilities. We should always use that which is already there.
This does not mean that some performance tuning might not be called for on the rare occasion. I have found the need to improve the performance of a shell script. The problems I discovered were usually more about dealing with large amounts of data than about the functional logic of the program.
Besides, the hardware will be faster next week.
Variables
Use variables instead of hard coded values for almost everything. Even if you think you will only use a particular value once, such as a directory name or a file name, create a variable and use the variable where you would have placed the hard-coded name.
Many times I have needed a particular value in more places in the scripts so I am already prepared if it is accessed as a variable. It can take less time to type a variable name than a complete directory name, especially if it is a long one. It is also easier to change a script if the value changes. Fixing the value of the variable in one location is much easier than replacing it in several locations.
I always have a single location in my scripts to set initial values for variables. Keeping the initial variable statements in the same place helps make them easy to find.
Testing
Interactive testing of shell scripts can be accomplished as soon as the most basic code structure is complete, at all stages during development, when the code is complete, and when any needed changes have been made.
The test plan should be created from the requirements statements. The test plan will have lists of the requirements to test, such as, “for input X the output should be Y,” and “for bad input error message X should be displayed.”
Having a test plan enables me to test each new feature as it is added to the program. It helps to ensure that testing is consistent as program development proceeds from start to finish. The importance of testing cannot be understated. Testing must take place right from the very start.
Open and open source
By their very nature shell scripts are open because we can read them. They are written in ASCII text format and are never compiled or altered into a binary or other format that is unreadable to humans. The shell, bash for example, reads the contents of the shell programs and interprets them on the fly. Their existence as ASCII text files also means that shell scripts can be easily modified and run immediately without having to wait through a recompile.
This open access to the code also means that we can explore shell scripts in aid of understanding their functional logic. This can be useful when writing our own scripts because we can easily include this existing code in our own scripts instead of writing our own code to perform the same task.
Of course this code sharing depends upon the open source licensing of the original code. I always include within the code itself an explicit statement of the license under which I share the code I write, usually the GPL V2. Many times I even have an option in the program to display the GPL license statement.
Making all of the code I write Open Source and properly licensed as such is just another basic requirement as far as I am concerned.
Shell scripts as prototypes
I have seen a number of articles and books about the Unix philosophy in which they discuss shell scripts as a tool for prototyping large and complex programs. I think there may be some value in that for application developers rather than SysAdmins. That approach can allow for fast prototyping and early testing to ensure that the program is exactly what the customer wants.
As a SysAdmin I find that shell scripts are perfect for both prototype and the completed program. I mean why take the extra time to translate something that is already working well into another language? Hey – we are trying to be lazy here!
Process
We all have our own processes – ways of working that enable us to work our way through projects to completion. We are all different and our processes are different. And sometimes we have more than one process depending upon our starting point. I want to describe to you a couple methods that work for me.
Quick and dirty
Most of my programming projects start as quick and dirty command line programs that I use to perform a specific task. The doUpdates.sh program is a good example. After all, installing updates is a simple dnf command, right? Not so much.
For a long time, I would login to each host, run the dnf -y update command and then manually reboot if the kernel had been updated. The next step occurred when I determined in advance that the kernel was being updated. I used the compound command dnf -y update && reboot which rebooted the computer if the update was successful. But I was still typing the commands on the command line.
As the number of computers in my home network grew I realized that I was also updating the man database, making a decision and, if there was a kernel update, updating the GRUB configuration file, and running the reboot command. At that point I wrote a simple script with no frills to perform those tasks.
But that script needed to make a couple decisions of its own and some direction from me. I did not want to have the script arbitrarily reboot the host every time it was run. So I added an option to reboot only if the kernel or glibc were updated. That required that I add the case command to interpret the options. I also added a variable that contained the current version of the program and an option to display the version. A bit later I added a “verbose” option so I could get more debugging information if the program encountered problems..
With the addition of options I needed a Help facility, so I added that. Then I added an option to display the GPL statement so others — like you — can use my code and know it’s properly licensed.
Many large programs grow from those little, everyday command line programs and become indispensable to our daily working lives. Sometimes the process is not noticeable until you realize you have a fully working script on your hands.
Planning and foresight
Some programs written by SysAdmins are actually planned in advance. Once again I start with a set of requirements although I try to spend a bit more time formulating them than with the quick and dirty programs.
To start coding I make a copy of the script template and name it appropriately. The template contains all of the standard procedures and basic structure that I need to begin any project. This template includes a skeletal help facility, a procedure for ending the program with an appropriate return code (RC), and a case statement to enable use of options.
So the first thing I do with my template is code the help facility. Then I test to see if that is working and looks as I intend. Coding the Help facility first also begins the process of documentation. It helps me to define the function of the script as well as some of the features.
At this point I like to add comments that define specific functionality and create execution sequences within the script. If I need to write a new procedure, I create a small skeleton for that procedure with comments that contain a short description of its function. By adding these comments first, I have embedded the set of requirements I created earlier into the very fabric of the code. This makes it easy to follow and ensure that I have translated all of those requirements into code.
I then begin to add code to each section of comments. And then I test each new section to ensure that it meets the requirements stated in the comments.
Then I add a bit more and test. And add a bit more and test. Every time I test, I test everything, even the features and code segments I have tested before because new code can break existing code, too. I follow this procedure until the shell script is complete.
Template
I have mentioned a number of times that I have a template from I like to create my programs. Let’s look at that template and experiment with it. You can download BashTemplate.sh directly or from our Downloads page.
The code
Now that you have downloaded the template let’s look at Figure 3 and I will point out some of its key features. Then we’ll do an experiment to see how it works.
Of course all scripts should begin with the shebang and this one is no different. Then I add a couple sections of comments.
The first comment section is the program name and description and a change history. This is a format I learned while working at IBM and it provides a means of documenting the long-term development of the program and any fixes applied to it. This is an important start to documenting your program.
The second comment section is a copyright and license statement. I use the GPL2 and this seems to be a standard statement for programs licensed under the GPL2. If you choose to use a different open source license, that is fine, but I do suggest adding an explicit statement like this to the code in order to eliminate any possible confusion about licensing. I read an interesting article recently, “The source code is the license1,” that helps to explain the reasoning behind this.
The procedures section begins after these two comment sections. This is the required location for procedures in Bash. They must appear before the body of the program. As part of my own need to document everything I place a comment before each procedure that contains a short description of what it is intended to do. I also include comments inside the procedure to provide further elaboration. Your own procedures can be added here.
I won’t dissect the function of each of these procedures. Between the comments and your ability to read the code, they should be understandable. However at the end of Figure 3, I will discuss some other aspects of this template.
#!/bin/bash
################################################################################
# Bash.template.sh #
# #
# This template is a standard starting point for new Bash scripts. #
# #
# Add a description of the program in this space. #
# #
# Change History #
# 2023/02/24 David Both Original code. #
# #
# #
# #
################################################################################
################################################################################
################################################################################
# #
# Copyright (C) 2007, 2023 David Both #
# LinuxGeek46@both.org #
# #
# This program is free software; you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation; either version 2 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program; if not, write to the Free Software #
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #
# #
################################################################################
################################################################################
################################################################################
################################################################################
# Help #
################################################################################
Help()
{
# Display Help
echo " BashTemplate.sh "
echo ""
echo "Add a description of the program"
echo
echo "Syntax: mymotd [-g|h|v|V]"
echo "options:"
echo "g Print the GPL license notification."
echo "h Print this Help."
echo "v Verbose mode."
echo "V Print software version and exit."
echo
}
################################################################################
# Print the GPL license header #
################################################################################
gpl()
{
echo
echo "################################################################################"
echo "# Copyright (C) 2007, 2023 David Both #"
echo "# http://www.both.org #"
echo "# #"
echo "# This program is free software; you can redistribute it and/or modify #"
echo "# it under the terms of the GNU General Public License as published by #"
echo "# the Free Software Foundation; either version 2 of the License, or #"
echo "# (at your option) any later version. #"
echo "# #"
echo "# This program is distributed in the hope that it will be useful, #"
echo "# but WITHOUT ANY WARRANTY; without even the implied warranty of #"
echo "# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #"
echo "# GNU General Public License for more details. #"
echo "# #"
echo "# You should have received a copy of the GNU General Public License #"
echo "# along with this program; if not, write to the Free Software #"
echo "# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA #"
echo "################################################################################"
echo
}
################################################################################
# Quit nicely with messages as appropriate #
################################################################################
Quit()
{
if [ $verbose = 1 ]
then
if [ $error = 0 ]
then
echo "Program terminated normally"
else
echo "Program terminated with error ID $ErrorMsg";
fi
fi
exit $error
}
################################################################################
# Display verbose messages in a common format #
################################################################################
PrintMsg()
{
if [ $verbose = 1 ] && [ -n "$Msg" ]
then
echo "########## $Msg ##########"
# Set the message to null
Msg=""
fi
}
################################################################################
# Convert KB to GB #
################################################################################
kb2gb()
{
# Convert KBytes to Giga using 1024
echo "scale=3;$number/1024/1024" | bc
}
################################################################################
################################################################################
# Main program #
################################################################################
################################################################################
# Set initial variables
badoption=0
error=0
host=""
verbose=0
Version=01.00.04
ErrorMsg="0"
#---------------------------------------------------------------------------
# Check for root. Delete if necessary.
# if [ `id -u` != 0 ]
# then
# echo ""
# echo "You must be root user to run this program"
# echo ""
# Quit 1
# fi
#
# #---------------------------------------------------------------------------
# Check for Linux
if [[ "$(uname -s)" != "Linux" ]]
then
echo ""
echo "This script only runs on Linux -- OS detected: $(uname -s)."
echo ""
Quit 1
fi
---------------------------------------------------------------------------
################################################################################
# Process the input options. Add options as needed. #
################################################################################
# Get the options
while getopts ":ghrvV" option; do
case $option in
g) # display GPL
gpl
Quit;;
h) # display Help
Help
Quit;;
v) # Set verbose mode
verbose=1;;
V) # Print the software version
echo "Version = $Version"
Quit;;
\?) # incorrect option
badoption=1;;
esac
done
if [ $badoption = 1 ]
then
echo "ERROR: Invalid option"
Help
verbose=1
error=1
ErrorMsg="10T"
Quit $error
fi
################################################################################
################################################################################
# The main body of your program goes here.
################################################################################
################################################################################
Quit
################################################################################
# End of program
################################################################################
Figure 3: The BashTemplate.sh template file I use as a starting point for new programs.
The main part of the program begins after the end of the procedures section. I usually start this section with a section to set the initial values of all the variables used in the program. This ensures that all of the variables I use have been set to some default initial value. It also provides a list of all of the variables used in the program.
Next I have a check to see if root is running this program and, if not, display a message and exit. If your program can be run by non-root users, you can delete this section or comment it out.
Then the getops and case statements that check the command line to determine whether any options have been entered. For each option the case statement sets specified variables or calls procedures like Help() and Quit(). If an invalid option is entered, the last case stanza sets a variable to indicate that and the next bit of code throws an error message and quits.
Finally the main body of the program is where most of your code will go. This program is executable as it is without errors. But because there is no functional code all you can do is display the help and the GPL license statement, and generate an error for using an invalid option. Until you add some functional code to the program, it will do nothing else at all.
Set the permissions to executable for user, and group, and set both the user and group ownership to your non-root user. In a terminal session ensure the PWD is your home directory. Before proceeding, let’s do some testing.
Display the help information as the non-root user.
$ ./BashTemplate.sh -h
You must be root user to run this programThat is the bit of code telling you that you must be root. You can bypass that by commenting out those lines of code. Be sure to save the changes you have made. Now run the script again using the -h option to view the help.
$ ./BashTemplate.sh -h
BashTemplate.sh
Add a description of the program
Syntax: mymotd [-g|h|v|V]
options:
g Print the GPL license notification.
h Print this Help.
v Verbose mode.
V Print software version and exit.Let’s see what happens when you give the program an option it does not recognize.
$ ./BashTemplate.sh -a
ERROR: Invalid option
BashTemplate.sh
Add a description of the program
Syntax: mymotd [-g|h|v|V]
options:
g Print the GPL license notification.
h Print this Help.
v Verbose mode.
V Print software version and exit.
Program terminated with error ID 10TThat’s good – it displays the help and terminates with an error message. Most people will not understand the humor of the error message ID – nevertheless, it is not one I would leave in any production script.
So let’s at least make our little test script perform some useful work. Add the free command in the body section.
################################################################################
################################################################################
# The main body of your program goes here.
################################################################################
################################################################################
free
Quit
################################################################################
# End of program
################################################################################Yup – that’s all, just the free command. Save the script and run the it again without any options.
$ ./BashTemplate.sh
total used free shared buff/cache available
Mem: 32691672 5126940 23931664 827744 4929524 27564732
Swap: 8388604 0 8388604So now you have created a working script from a fairly simple template. You have performed some simple tests to verify that the script is performing as expected.
One of the options that I like to have is a “test” mode in which the program runs and describes what it will do or prints some debugging data to STDOUT so that I can visualize how it is working. Add that option yourself and test it to ensure that it works as expected.
Feel free to use this template and alter it to meet your own requirements. Because the template is open source under the GPL2 you can share it and modify it. My intention is for it to be used by you if you so choose. Remember that the Lazy Admin always uses freely available code to prevent having to duplicate the effort of writing code that already does what you need. I hope you find it useful.
Final thoughts
Compiled programs are necessary and fill a very important need. But for SysAdmins there is always a better way. We should always use shell scripts to meet the automation needs of our jobs.
Shell scripts are open; their content and purpose are knowable. They can be readily modified to meet differing requirements. Personally, I have found nothing that I have ever needed to do in my SysAdmin role that could not be accomplished with a shell script.
In the very rare event that you find something a shell script cannot do, don’t write the whole program in a compiled language. Write as much as possible as a shell script. Then, if – and only if – there is no possible way to do that little bit that is left by using a shell command or a series of shell commands in a pipeline, write a little program that does one thing well – that one little bit that cannot be found anywhere else.
- Scott K Peterson, “The source code is the license,” Opensource.com, https://opensource.com/article/17/12/source-code-license ↩︎
More Stories
The Linux Philosophy for SysAdmins, Tenet 07 — Automate Everything
SysAdmins are most productive when creating the shell programs that automate the solutions that they have conceived while appearing to be unproductive. The more automation we have in place the more time we have available to fix real problems when they occur and to contemplate how to automate even more than we already have.
Terminal size and ncurses
When programs like vi first came along, terminals had a predefined size: usually 80 columns and 24 lines (actually 25...
Searching text files from the command line
The find command is a powerful and flexible tool to locate files under a path.
Random numbers from the Linux kernel
Let the kernel generate random numbers for you using ‘getrandom.’
Working with Ollama
Now that I'm working with Ollama, I needed to figure out how to locate the models on my storage medium...
A gentle introduction to ncurses
You can create your own programs that use a text user interface to control the screen.