Linux - Shell Scripting

linux

Important:
http://steve-parker.org/sh/sh.shtml - done reading
http://www.grymoire.com/Unix/Quote.html
http://www.grymoire.com/Unix/Regular.html

http://www.freeos.com/guides/lsst/
http://www.howtogeek.com/67469/the-beginners-guide-to-shell-scripting-the-basics/
http://www.tecmint.com/understand-linux-shell-and-basic-shell-scripting-language-tips/
http://linuxconfig.org/bash-scripting-tutorial
http://www.tldp.org/LDP/abs/html/
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html

How can we declare a variable?

variableName=someValue

Note that there must be no spaces around the equal sign.

How can we use the declared variable?

#!/bin/sh
MY_MESSAGE="Hello World"
echo $MY_MESSAGE

How can we do math within a shell scripting environment?

Use the expr command. Read the man page for the expr command.

How can we take input from the user?

#!/bin/sh
echo "What is your name?"
read MY_NAME
echo "Hello $MY_NAME - hope you're well."

How can we separate a variable name from the string next to it?

Use the curly braces. Consider this code:

#!/bin/sh
echo "What is your name?"
read USER_NAME
echo "Hello $USER_NAME"
echo "I will create you a file called $USER_NAME_file"
touch $USER_NAME_file

If you enter "steve" as your USER_NAME, should the script create steve_file? Actually, no. This will cause an error unless there is a variable called USER_NAME_file. The shell does not know where the variable ends and the rest starts. To get around this problem, we enclose the variable itself in curly brackets:

#!/bin/sh
echo "What is your name?"
read USER_NAME
echo "Hello $USER_NAME"
echo "I will create you a file called ${USER_NAME}_file"
touch "${USER_NAME}_file"

In this case, The shell now knows that we are referring to the variable USER_NAME and that we want it suffixed with “_file”.

What is the escape character?

The escape character is the backslash

How does the shell handle interpolation?

You have already seen the variable being used inside a string:

echo "What is your name?"
read USER_NAME
echo "Hello $USER_NAME"
echo "I will create you a file called ${USER_NAME}_file"
touch "${USER_NAME}_file"

When inside a string, most characters (*, ', etc) are not interpreted (ie, they are taken literally). They are taken as is and passed on to the command being called.

However, ", $, ‘, and \ are still interpreted by the shell, even when they’re in double quotes. The backslash (\) character is used to mark these special characters so that they are not interpreted by the shell, but passed on to the command being run

How can we use the for loop construct?

The for loops iterate through a set of values until the list is exhausted:

#!/bin/sh
for i in 1 2 3 4 5
do
  echo "Looping ... number $i"
done

Note that the values can be anything. It can be number or string.

How can we use the while loop construct?

#!/bin/sh
INPUT_STRING=hello
while [ "$INPUT_STRING" != "bye" ]
do
  echo "Please type something in (bye to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"
done
#!/bin/sh
while :
do
  echo "Please type something in (^C to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"
done

The colon always evaluate to true.

What is the "while read" do?

#!/bin/sh
while read f
do
  case $f in
        hello)          echo English    ;;
        howdy)          echo American   ;;
        gday)           echo Australian ;;
        bonjour)        echo French     ;;
        "guten tag")    echo German     ;;
        *)              echo Unknown Language: $f
                ;;
   esac
done < myfile

The above code read a string from the myfile (specified next to the last line), and based on the string, it output the name of the language.

What is the syntax for the if construct?

if [ ... ]
then
  # if-code
else
  # else-code
fi

Note that fi is if backwards! This is used again later with case and esac. Also, be aware of the syntax - the "if [ … ]" and the "then" commands must be on different lines. Alternatively, the semicolon ";" can separate them:

if [ ... ]; then
  # do something
fi

You can also use the elif, like this:

if  [ something ]; then
 echo "Something"
 elif [ something_else ]; then
   echo "Something else"
 else
   echo "None of the above"
fi

How can we do comparison?

#!/bin/sh
if [ "$X" -lt "0" ]
then
  echo "X is less than zero"
fi
if [ "$X" -gt "0" ]; then
  echo "X is more than zero"
fi
[ "$X" -le "0" ] && \
      echo "X is less than or equal to  zero"
[ "$X" -ge "0" ] && \
      echo "X is more than or equal to zero"
[ "$X" = "0" ] && \
      echo "X is the string or number \"0\""
[ "$X" = "hello" ] && \
      echo "X matches the string \"hello\""
[ "$X" != "hello" ] && \
      echo "X is not the string \"hello\""
[ -n "$X" ] && \
      echo "X is of nonzero length"
[ -f "$X" ] && \
      echo "X is the path of a real file" || \
      echo "No such file: $X"
[ -x "$X" ] && \
      echo "X is the path of an executable file"
[ "$X" -nt "/etc/passwd" ] && \
      echo "X is a file which is newer than /etc/passwd"

Read the manual page for the test command:

man test

What does the backslash character do when it is not inside a pair of quotes?

The backslash simply tells the shell that this is not the end of the line, but the two (or more) lines should be treated as one. This is useful for readability. It is customary to indent the following line.

How can we do and or?

&& and ||

What is the purpose of the $? variable?

The contains the exit code of the previous command.

How can we use the case statement?

  case $INPUT_STRING in
        hello)
                echo "Hello yourself!"
                ;;
        bye)
                echo "See you again!"
                break
                ;;
        *)
                echo "Sorry, I don't understand"
                ;;
  esac

What is the purpose of $0?

$0 is the basename of the program as it was called.

What are are the purposes of $1 .. $9?

$1 .. $9 are the first 9 additional parameters the script was called with.

What is the purpose of $@?

$@ is all parameters $1 .. whatever.

The $@ parameters are changed within the function to reflect how the function was called.

What is the purpose of $*?

The variable $@ is all parameters $1 .. whatever. The variable $*, is similar, but does not preserve any whitespace, and quoting, so "File with spaces" becomes "File" "with" "spaces". As a general rule, use $@ and avoid $*.

What is the purpose of $#?

$# is the number of parameters the script was called with. Actually, it represent the length of the $@ array.

What is the purpose of the $$ variable?

The $$ variable is the PID (Process IDentifier) of the currently running process.

What is the purpose of the $! variable?

The $! variable is the PID of the last run background process.

What is the purpose of the IFS variable?

Another interesting variable is IFS. This is the Internal Field Separator. The default value is SPACE TAB NEWLINE, but if you are changing it, it's easier to take a copy, as shown:

#!/bin/sh
old_IFS="$IFS"
IFS=:
echo "Please input three data separated by colons ..."
read x y z
IFS=$old_IFS
echo "x is $x y is $y z is $z"

It is important when dealing with IFS in particular (but any variable not entirely under your control) to realise that it could contain spaces, newlines and other "uncontrollable" characters. It is therefore a very good idea to use double-quotes around it, ie: old_IFS="$IFS" instead of old_IFS=$IFS.

What is the purpose of the :- construct?

echo "Your name is : ${myname:-John Doe}"

The above code check the myname variable. If it is null or undefined, John Doe is displayed. Otherwise, the value of the variable is displayed.

What is the purpose of the := construct?

echo "Your name is : ${myname:=John Doe}"

Sets the variable to the default if it is undefined. This technique means that any subsequent access to the $myname variable will always get a value, either entered by the user, or "John Doe" otherwise.

How can we include a library of shell function?

. ./library.sh

goes at the start of the script.

How can we define a function?

#!/bin/sh
# A simple script with a function...

add_a_user()
{
  USER=$1
  PASSWORD=$2
  shift; shift;
  # Having shifted twice, the rest is now comments ...
  COMMENTS=$@
  echo "Adding user $USER ..."
  echo useradd -c "$COMMENTS" $USER
  echo passwd $USER $PASSWORD
  echo "Added user $USER ($COMMENTS) with pass $PASSWORD"
}

###
# Main body of script starts here
###
echo "Start of script..."
add_a_user bob letmein Bob Holness the presenter
add_a_user fred badpassword Fred Durst the singer
add_a_user bilko worsepassword Sgt. Bilko the role model
echo "End of script..."

How can we pass parameters to our function?

So within that function, $1 is set to bob, regardless of what $1 may be set to outside of the function.

So if we want to refer to the "original" $1 inside the function, we have to assign a name to it - such as: A=$1 before we call the function. Then, within the function, we can refer to $A.

Is there scoping in shell scripting?

No. Basically, there is no scoping, other than the parameters ($1, $2, $@, etc).

Can a shell function be recursive?

Yes:

#!/bin/sh

factorial()
{
  if [ "$1" -gt "1" ]; then
    i=`expr $1 - 1`
    j=`factorial $i`
    k=`expr $1 \* $j`
    echo $k
  else
    echo 1
  fi
}

while :
do
  echo "Enter a number:"
  read x
  factorial $x
done

How can a function return a value?

Use the return statement:

return 1

What is the difference between variableName2="$variableName1" and variableName2=$variableName1?

The first handle it appropriately if variableName1 contains space.

How can we handle exit code?

Exit codes are a number between 0 and 255, which is returned by any Unix command when it returns control to its parent process. Other numbers can be used, but these are treated modulo 256, so exit -10 is equivalent to exit 246, and exit 257 is equivalent to exit 1.

Success is traditionally represented with exit 0; failure is normally indicated with a non-zero exit-code. This value can indicate different reasons for failure.

#!/bin/sh
# First attempt at checking return codes
USERNAME=`grep "^${1}:" /etc/passwd | cut -d":" -f1`
if [ "$?" -ne "0" ]; then
  echo "Sorry, cannot find user ${1} in /etc/passwd"
  exit 1
fi
NAME=`grep "^${1}:" /etc/passwd | cut -d":" -f5`
HOMEDIR=`grep "^${1}:" /etc/passwd | cut -d":" -f6`

echo "USERNAME: $USERNAME"
echo "NAME: $NAME"
echo "HOMEDIR: $HOMEDIR"

The above code show the use of $? variable. This script works fine if you supply a valid username in /etc/passwd. However, if you enter an invalid code, it does not do what you might at first expect - it keeps running, and just shows:

USERNAME: 
NAME: 
HOMEDIR:

Why is this? As mentioned, the $? variable is set to the return code of the last executed command. In this case, that is cut. cut had no problems which it feels like reporting - as far as I can tell from testing it, and reading the documentation, cut returns zero whatever happens! It was fed an empty string, and did its job - returned the first field of its input, which just happened to be the empty string.

So what do we do? If we have an error here, grep will report it, not cut. Therefore, we have to test grep's return code, not cut's.

#!/bin/sh
# Second attempt at checking return codes
grep "^${1}:" /etc/passwd > /dev/null 2>&1
if [ "$?" -ne "0" ]; then
  echo "Sorry, cannot find user ${1} in /etc/passwd"
  exit 1
fi
USERNAME=`grep "^${1}:" /etc/passwd | cut -d":" -f1`
NAME=`grep "^${1}:" /etc/passwd | cut -d":" -f5`
HOMEDIR=`grep "^${1}:" /etc/passwd | cut -d":" -f6`

echo "USERNAME: $USERNAME"
echo "NAME: $NAME"
echo "HOMEDIR: $HOMEDIR"

This fixes the problem for us, though at the expense of slightly longer code. That is the basic way which textbooks might show you, but it is far from being all there is to know about error-checking in shell scripts. This method may not be the most suitable to your particular command-sequence, or may be unmaintainable.

As a second approach, we can tidy this somewhat by putting the test into a separate function, instead of littering the code with lots of 4-line tests:

#!/bin/sh
# A Tidier approach

check_errs()
{
  # Function. Parameter 1 is the return code
  # Para. 2 is text to display on failure.
  if [ "${1}" -ne "0" ]; then
    echo "ERROR # ${1} : ${2}"
    # as a bonus, make our script exit with the right error code.
    exit ${1}
  fi
}

### main script starts here ###

grep "^${1}:" /etc/passwd > /dev/null 2>&1
check_errs $? "User ${1} not found in /etc/passwd"
USERNAME=`grep "^${1}:" /etc/passwd|cut -d":" -f1`
check_errs $? "Cut returned an error"
echo "USERNAME: $USERNAME"
check_errs $? "echo returned an error - very strange!"

This allows us to test for errors 3 times, with customised error messages, without having to write 3 individual tests. By writing the test routine once. we can call it as many times as we wish, creating a more intelligent script, at very little expense to the programmer. Perl programmers will recognise this as being similar to the die command in Perl.

As a third approach, we shall look at a simpler and cruder method. I tend to use this for building Linux kernels - simple automations which, if they go well, should just get on with it, but when things go wrong, tend to require the operator to do something intelligent (ie, that which a script cannot do!):

#!/bin/sh
cd /usr/src/linux && \
make dep && make bzImage && make modules && make modules_install && \
cp arch/i386/boot/bzImage /boot/my-new-kernel && cp System.map /boot && \
echo "Your new kernel awaits, m'lord."

This script runs through the various tasks involved in building a Linux kernel (which can take quite a while), and uses the && operator to check for success. To do this with if would involve:

#!/bin/sh
cd /usr/src/linux
if [ "$?" -eq "0" ]; then
  make dep 
    if [ "$?" -eq "0" ]; then
      make bzImage 
      if [ "$?" -eq "0" ]; then
        make modules 
        if [ "$?" -eq "0" ]; then
          make modules_install
          if [ "$?" -eq "0" ]; then
            cp arch/i386/boot/bzImage /boot/my-new-kernel
            if [ "$?" -eq "0" ]; then
              cp System.map /boot/
              if [ "$?" -eq "0" ]; then
                echo "Your new kernel awaits, m'lord."
              fi
            fi
          fi
        fi
      fi
    fi
  fi
fi

The && and || operators are the shell's equivalent of AND and OR tests. These can be thrown together as above, or:

#!/bin/sh
cp /foo /bar && echo Success || echo Failed

Only one command can be in each part. This method is handy for simple success / fail scenarios, but if you want to check on the status of the echo commands themselves, it is easy to quickly become confused about which && and || applies to which command. It is also very difficult to maintain. Therefore this construct is only recommended for simple sequencing of commands.

What is the syntax of using sub-shell?

( command1 ; command2; command3 )

What is the return code of the sub-shell?

The return code of the subshell is the return code of the final command

What is the purpose of the trap statement?

Trap is a simple, but very useful utility. If your script creates temporary files, such as this simple script which replaces FOO for BAR in all files in the current directory, /tmp is clean when the script exits. If it gets interrupted partway through, though, there could be a file lying around in /tmp:

#!/bin/sh

trap cleanup 1 2 3 6

cleanup()
{
  echo "Caught Signal ... cleaning up."
  rm -rf /tmp/temp_*.$$
  echo "Done cleanup ... quitting."
  exit 1
}

### main script
for i in *
do
  sed s/FOO/BAR/g $i > /tmp/temp_${i}.$$ && mv /tmp/temp_${i}.$$ $i
done

The trap statement tells the script to run cleanup() on signals 1, 2, 3 or 6. The most common one (CTRL-C) is signal 2 (SIGINT). This can also be used for quite interesting purposes:

#!/bin/sh

trap 'increment' 2

increment()
{
  echo "Caught SIGINT ..."
  X=`expr ${X} + 500`
  if [ "${X}" -gt "2000" ]
  then
    echo "Okay, I'll quit ..."
    exit 1
  fi
}

### main script
X=0
while :
do
  echo "X=$X"
  X=`expr ${X} + 1`
  sleep 1
done

The above script is quite fun - it catches a CTRL-C, doesn't exit, but just changes how it's running. How this could be useful for positive and negative effect is left as an exercise to the reader:) This particular example concedes to quit after 4 interrupts (or 2000 seconds). Note that anything will be killed by a kill -9 <PID> without getting the chance to process it.

For a list of signal:

man signal
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License