Your Perfect Kickstart To Shell Scripting

According to me, the best way of learning new frameworks or technologies is to get a hands on experience simultaneously, and here in this article you will be learning the basics of shell scripting by writing the code yourself!
This article contains syntax, basics of shell scripts to intermediate level shell programming. Use this to build your knowledge about shell which provides you an interface with Linux/Unix system :)

Welcome, my friend :)

Introduction

You might have came across the word ‘script’ a lot of times, but what is the meaning of a script? Basically, a script is a program that contains a series of commands to be executed. These commands are executed by an interpreter. Anything you can put into a command line, you can put in a script. And, scripts are great for automating tasks.
If you find yourself repeating some commands frequently, you can, rather you should, create a script for doing it!

The Linux philosophy is ‘Laugh in the face of danger’. Oops. Wrong One. ‘Do it yourself’. Yes, that’s it.
— Linus Torvalds

Let’s write our first script

#!/bin/bash
echo "My First Script!"

To run it,

$ chmod 755 script.sh
$ ./script.sh

Wohooo! you just wrote your first bash script.
I know you did not understand the script, mostly the first line. Don’t worry we will see shell scripts in detail in this article. Before going into any topic, I always recommend to form a roadmap or a proper index of contents in your mind and be clear about what we are going to learn.
So, here are some key points that we are going to discuss in our article.

• Shebang
• Variables
• User Input
• Tests
• Decision Making
• Iterative Statements
• Positional Parameters
• Exit Statuses
• Logic Operations
• Functions
• Wildcards
• Debugging

So, this will be the order of our discussion and at the end of this article, I am sure you will be confident enough to write shell scripts yourself :)
Happy Learning Guys

Shebang

You might have noticed in above script that it starts with #!/bin/bash This is called shebang. It is basically to refer the path to the interpreter. There are many interpreters, some of them are: bash, zsh, csh and ksh, etc.

All the best people in life seem to like LINUX. — Steve Wozniak

To use bash: #!/bin/bash
To use zsh: #!/bin/zsh
To use ksh: #!/bin/ksh
To use csh: #!/bin/csh
and so on…

Why Shebang?
# is often called sharp and ! is called Bang, hence the name sharp bang, but generally people say it shebang instead of sharp bang.

Note* If a script does not contain the shebang, the commands are executed using your shell, so there are chances that the code might run properly, but still, that isn’t the correct way of doing it!

Comments
Comments are started by a # sign, anything after pound sign on that line is ignored.

Variables

Variables are basically storage location that have a name and can store some data which can be changed in future.

VARIABLE_NAME="Value"

You must remember some points while naming variables:

• Variables are case sensitive
• By convention, variables are uppercase
• To use a variable, just write the variable name followed by the $ sign

Example:

#!/bin/bash
MY_NAME="Madhav Bahl"
echo "Hello, I am $MY_NAME"
OR
#!/bin/bash
MY_NAME="Madhav Bahl"
echo "Hello, I am ${MY_NAME}"

Note* We can assign the output of a command to a variable too Example: LIST=$(ls) , another example: SERVER_NAME=$(hostname)

Some Valid Variable Names:

THIS3VARIABLE=”ABC”
THIS_IS_VARIABLE=”ABC”
thisIsVariable=”ABC”

Some Invalid Variable Names:

4Number=”NUM”
This-Is-Var=”VAR”
# No special character apart from underscore is allowed!

User Input

read command accepts STDIN (Standard Input)

read -p "PROMPT MESSAGE" VARIABLE

Here, “Prompt Message” will be prompted to the screen and whatever user enters next will be stored in VARIABLE

#!/bin/bash
read -p "Please Enter You Name: " NAME
echo "Your Name Is: $NAME"

Tests

Tests are used for decision making. [ condition-to-test-for ] Example: `[ -e /etc/passwd ]

1. File Test Operations

-d FILE_NAM  # True if FILE_NAM is a directory
-e FILE_NAM  # True if FILE_NAM exists
-f FILE_NAM  # True if FILE_NAM exists and is a regular file
-r FILE_NAM  # True if FILE_NAM is readable
-s FILE_NAM  # True if FILE_NAM exists and is not empty
-w FILE_NAM  # True if FILE_NAM has write permission
-x FILE_NAM  # True if FILE_NAM is executable

2. String Test Operations

-z STRING  # True if STRING is empty
-n STRING  # True if STRING is not empty
STRING1 = STRIN2 # True if strings are equal
STRING1 != STRIN2 # True if strings are not equal

3. Arithmetic Operators

var1 -eq var2  # True if var1 is equal to var2
var1 -ne var2  # True if var1 not equal to var2
var1 -lt var2  # True if var1 is less than var2
var1 -le var2  # True if var1 is less than or equal to var2
var1 -gt var2  # True if var1 is greater than var2
var1 -ge var2  # True if var1 is greater than or equal to var2

Decision Making

Just like any script, shell scripts can make decisions based on conditions. We can use If-Else (or, If-Elif-Else) and case statements for decision making.

Avoid the Gates of Hell. Use Linux!

1. The if statement

if [ condition-is-true ]
then
  command 1
  command 2
    ...
    ...
  command N
fi

The if-elif Ladder

if [ condition-is-true ]
then
  command 1
elif [ condition-is-true ]
then
  command 2
elif [ condition-is-true ]
then
  command 3
else
  command 4
fi

2. Case Statements
The case statements are an alternative for if statements which are a little easier to read than complex if elif ladder. If you find yourself using an if statement to compare the same variable against some different/discrete values, you can use a case statements instead of if-elif ladder.

Syntax:

case "$VAR" in
  pattern_1)
    # commands when $VAR matches pattern 1
    ;;
  pattern_2)
    # commands when $VAR matches pattern 2
    ;;
  *)
    # This will run if $VAR doesnt match any of the given patterns
    ;;
esac

Example:

#!/bin/bash
read -p "Enter the answer in Y/N: " ANSWER
case "$ANSWER" in
  [yY] | [yY][eE][sS])
    echo "The Answer is Yes :)"
    ;;
  [nN] | [nN][oO])
    echo "The Answer is No :("
    ;;
  *)
    echo "Invalid Answer :/"
    ;;
esac

Iterative Statements: Loops

Loops can execute a block of code a number of times and are basically used for performing iterations.

1. The For Loop

for VARIABLE_NAME in ITEM_1 ITEM_N
do
  command 1
  command 2
    ...
    ...
  command N
done

For example:

#!/bin/bash
COLORS="red green blue"
for COLOR in $COLORS
do
  echo "The Color is: ${COLOR}"
done

Another way of using for loop:

for (( VAR=1;VAR<N;VAR++ ))
do
  command 1
  command 2
    ...
    ...
  command N
done

Here’s an example of how you can make a script to rename each file with .txt format to new-<old-name>.txt

#!/bin/bash
FILES=$(ls *txt)
NEW="new"
for FILE in $FILES
do
  echo "Renaming $FILE to new-$FILE"
  mv $FILE $NEW-$FILE
done

2. The While Loop

While loop repeats a series of commands for as long as the given condition holds true.

while [ CONNDITION_IS_TRUE ]
do
  # Commands will change he entry condition
  command 1
  command 2
    ...
    ...
  command N
done

Condition can be any test or command. If the test/command retrns a 0 exit status, it means that the condition is true and commands will be executed. If the command returns a non-zero exit status, it the loop will stop its iterations. If the condition is false initially, then the commands inside the loop will never get executed.
Don’t worry if you do not know what an exit status is. We will cover that in a bit :)

Example: Reading a file line by line

#!/bin/bash
LINE=1
while read CURRENT_LINE
do
  echo "${LINE}: $CURRENT_LINE"
  ((LINE++))
done < /etc/passwd
# This script loops through the file /etc/passwd line by line

Note:
continue statement is used to take the flow of control to the next iteration. Any statement after continue statement is hit will not be executed and the flow of control will shift to the next iteration.
break statement can be used in while loops (or other loops like for loop) to end the loops. Once the break statement is hit, the flow of control will move outside the loop.

Positional Parameters

Some arguements or parameters can be passed when we call the script.
For Example: $ ./script.sh param1 param2 param3 param4

All the parameters will be stored in various variables:

  $0 -- "script.sh"
  $1 -- "param1"
  $2 -- "param2"
  $3 -- "param3"
  $4 -- "param4"
  $@ -- array of all positional parameters

These variables can be accessed anywhere in the script just like any other global variable.

Exit Statuses

Every command returns an exit status, also called the return code which ranges from 0 to 255. Exit status are used for error checking.

• 0 means success
• Any code other than 0 means an error condition.

To find out what an exit status for a command means, one can look for the documentations or manual using man or info command.
$? contains the return code of previously executed command.

Example: A script which will print Reachable/Unreachable Host/server using ping command.

#!/bin/bash
HOST="google.com"
ping -c 1 $HOST     # -c is used for count, it will send the request, number of times mentioned
RETURN_CODE=$?
if [ "$RETURN_CODE" -eq "0" ]
then
  echo "$HOST reachable"
else
  echo "$HOST unreachable"
fi

Custom Exit Statuses
Exit command is used to expicitly define the return code. If we do not define the exit status of the shell script explicitly, then by default the exit status of the last command executed is taken as the exit status of the script. We can use exit command anywherer in the script, and as soon as exit command is encountered, the shell script will stop executing.

  exit 0
  exit 1
  exit 2
    ...
    ...
  exit 255

Logic Operations

Shell scripts supports logical AND and logical OR.

Logical AND = &&
Logical OR = ||

Example: mkdir tempDir && cd tempDir && mkdir subTempDir
In this example, tempDir is created with mkdir command. If it succeeds, then cd tempDir is executed, and then mkdir subTempDir is executed.

Functions

A set of instructions which can be reused/called any time from the main program whenever the need for those instructions arrives. Consider a situation that you have a particular module that does a particular job, now let’s suppose that job has to be done 20 (say) times in the main program (for example calculating maximum number in an array of numbers). Now, if you write the code for that 20 times, your code would become very large, however, if we write a function for that, and call that function whenever it is required, the code would remain short, easy to read and much more modularised.

Always keep your code DRY!

Therefore, functions are great because they make the code DRY (Don’t Repeat Yourself), we can write once and use that many times, using functions reduces overall length/size of script. Moreover, functions make program easier to maintain, because they make the code divided into modules making particular place to edit and troubleshoot available in case of bugs/errors.

Note:
- Whenever you find yourself repeating a set of instructions, make a function for that. (A function must be defined before use).
- It is a good practise to define all your functions at the top before starting the main program or main instructions.

Syntax:

function function_name() {
    command 1
    command 2
    command 3
      ...
      ...
    command N
}

To call a function: simply write it’s name on a line in the script.

#!/bin/bash
function myFunc () {
    echo "Shell Scripting Is Fun!"
}
myFunc

Positional Parameters In Functions

Just like a shell script, functions can also accept parameters. The first parameter is stored in $1 The second paramaeter is stored in $2 and so on. $@ contains all the parameters.
Note: $0 is still the name of script itself, not the name of function.
To provide parameters, just write them after the function name with a white space in between. myFunc param1 param2 param3 ...

Scope Of Variables

Global Scope: All variables have, by default, global scope. Having a global scope means that the value of that variable can be accessed from anywhere in the script. Variables must be defined before it is used.
Local Scope: Local variables can be accessed only from within the function. Local variables are created using localkeyword and only functions can have local variables. It is a good practise to keep variables inside a function local.

Wildcards

A character or a string patterns that is used to match file and directory names is/are called wildcard(s). The process used to expand wildcard pattern into a list of files and/or directories (or basically paths) is called Globbing.
Wild Cards can be used with most of the commands that require file/dir path as argument. (Example ls,rm,cp etc).

Some Commonly Used Wildcards

* = Matches zero or more characters
Example:*.txt hello.* great*.md

? = matches exactly one character
Example: ?.md Hello?

[ ] = A character class
This wildcard is used to match any of the characters included between the square brackets (Matching exactly one character).
Example: He[loym], [AIEOU]

[!] = maches characters not included within brackets
It matches exactly one character.
Example: To match a consonant: [!aeiou]

Predefined named character classes

• [[:alpha:]]
• [[:alnum:]]
• [[:space:]]
• [[:upper:]]]
• [[:lower:]]
• [[:digit:]]

Matching wildcard characters: In case we have to match wildcard characters themselves like *, or ?, we can go with escape character - \
Example: *\? –> will match all files that end with a question mark.

Debugging

A bug is an error in a computer program/software that causes it to produce an unexpected or an incorrect result. Most of the bugs are cased by errors in the code and it’s design. To fix an error, try to reach to the root of that unexpected behaviour.

The bash shell provides some options that can help you in debugging your script. You can use these options by updating first line of the script.

Some Options:

1. -x option
It prints commands and arguments as they execute. It is called print debugging, tracing or an x-trace. We can use it by modifying the first line #!/bin/bash -x

2. -e option
It stands for “Exit on error”. This will cause your script to exit immediately if a command exits with a non-zero exit status.

3. -v option
It prints shell commands/input lines as they are read.

Note* These options can be combined, and more than one options can be used at a time!

#!/bin/bash-xe
#!/bin/bash-ex
#!/bin/bash-x-e
#!/bin/bash-e-x

Although there are many tools available which help in debugging, still, you can debug the code only if you understand how it works. So, make sure to spend enough time actually practise writing scripts and debugging them yourself so that you get to know places where you can do mistakes, so that you don’t do them again :)

Conclusion

This is it, I hope this article helped you :)
In case of any doubts, feel free to send me an email. We can connect on GitHub or LinkedIn and I would more than happy if you send your feedbacks, suggestions or ask queries. Moreover, I love to make new friends and we can be friends, just drop me a text.

Where to go from here?

This article contained the basics of shell scripting. But, articles have limitations, we can’t provide complete knowledge about shell scripts through one article. Therefore, I would highly recommend going to this website in which I wrote about shell scripting in detail.

Shell Scripting

Also, you can download a complimentary pdf of this guidebook for free here!

After going through this tutorial, I would also recommend you too read this book “The Linux Command Line” By William Shotts, it will explain you how the linux command line/terminal works in detail and how to customise it to make it much better!

Just in case you wish to be a web developer, have a look at this article, I wrote it to provide a clear road map of how to start and proceed with web development.

P.S. I am trying to build a community of developers called CodeToExpress where we learn, share, develop and grow together. If interested, join the slack workspace today.

Invite Link: https://join.slack.com/t/codetoexpress/shared_invite/enQtNDUwNzg0NDI1MzAwLWZiODY3OGZkZjdkZDA2MWNmMjFiNTY2MDlmMDg5ZGVjMzc5NDQ5OWU0NjEyNWZiM2Y4MmZmYmM0ZmQ3NjJmYWI

Teaching Tech, Seamlessly! (@codetoexpress) * Instagram photos and videos

Thanks a lot for keeping your calm and reading till end. All the best and Happy coding! You can contact me in case of any doubts or if you need any assistance:
Email: madhavbahl10@gmail.com
Web: http://madhavbahl.tech/
Github: https://github.com/MadhavBahlMD
LinkedIn: https://www.linkedin.com/in/madhavbahl/

✉️ Subscribe to CodeBurst’s once-weekly Email Blast, 🐦 Follow CodeBurst on Twitter, view 🗺️ The 2018 Web Developer Roadmap, and 🕸️ Learn Full Stack Web Development.