Online resources

Advanced Bash-Scripting Guide

Changing numbering in filenames

Recently I got a bunch of filenames that looked like this:


These files had to be renamed such that numbering started at zero instead of one. You can do this with the following shell script:

  for i in *; do
    # First remove the prefix
    # Then remove the extension
    # Bash interprets a leading zero as an octal number; force base 10
    # Now subtract one
    # Rename the file
    mv $i Wekker$(printf "%04d" $num).png

Testing for a certain extension

If you want to test whether a file has a certain extension, use the following trick:

  # Test for the extension "plist"
  if [ "${x%%plist}" == "$x" ]
      echo "Does not have the extension"
      echo "Has the extension"

Basically, you try to strip off the extension (in this case, "plist"), and if stripping succeeded, it won't be the same as the original filename.

Disabling or handling CTRL-C

Sometimes you want to temporarily disable signals like Interrupt (CTRL-C). The solution:

  trap '' INT      # Disable interruptions
  # Do whatever you want here which must be uninterruptible
  sleep 10
  trap - INT       # Restore normal function of the interrupt signal

To do a cleanup or similar after the user hits CTRL-C:

  trap "rm the_temporary_file; exit;" INT TERM
  sleep 10             # Do lots of nifty stuff here
  trap - INT TERM

Testing configuration of the webserver

Place test.cgi in a directory which is supposed to be configured to run CGI scripts and test with browser.

Using functions

  function print_error ()
    echo "Parameter 1: $1"
    echo "Parameter 2: $2"
  print_error "First param" "Second param"

Return values

You can't really return values in a function in bash, however you can echo text and then catch output with backticks.

  function return_a_string ()
    echo "This is the return value"

Saving a text file

To save a text file, create the following HTML file:

  <form action="/cgi-bin/" method="post"
    <input name="file" file">
    <input type="submit" value="Submit">

Then, in your cgi-bin, create a shellscript with the following lines:

  while read var
      echo $var >> outputfile
  echo "Content-type: text/plain"
  echo ""
  echo "Thanks!"

Of course, stuff gets pasted by the browser around the text. And this is a security hole to leave it wide open like this. So don't.

Saving a binary file

To save a binary or text file, create the HTML file like mentioned in the previous example. Then, in your cgi-bin, create a shellscript with the following lines:

  boundary=$(export | \
    sed '/CONTENT_TYPE/!d;s/^.*dary=//;s/.$//')
  filename=$(echo "$QUERY_STRING" | \
    sed -n '2!d;s/\(.*filename=\"\)\(.*\)\".*$/\2/;p')
  file=$(echo "$QUERY_STRING" | \
    sed -n "1,/$boundary/p" | sed '1,4d;$d')

The uploaded file is now contained in the $file variable.


Debugging is easiest done by writing to stderr. When the script is run through CGI, this ends up in the web server's error log:

  echo "This is a debugging message" >&2

Setting cookies

Before printing the Content-type, set a cookie with:

  value="some value"
  echo Set-Cookie: name=$value

Useful preset parameters

  SERVER_SOFTWARE = Apache/2.0.52 (Fedora)
  SERVER_NAME = localhost
  SCRIPT_NAME = /~b.kuik/cgi-bin/test.cgi
  QUERY_STRING = name=value1&name2=value2

Parsing parameters and cookies

To parse parameters and cookies, include Bashlib like this:

  . ./bashlib

Then, read cookies and parameters like this:

  sender=`param sender`
  recipient=`cookie recipient`

And test whether they're actually passed with:

  if [ -z $sender ]; then
    echo "Sender param not passed!" >&2

Testing a button pressed

The HTML looks like this :

  <input type="submit" value="Save" name="button1">

Then test like this:

  button1=`param button1`
  if [ "x$default" != "x" ]; then
    # do the default

Starting a job in the background

Problem: you want to start a process in the background, but want your CGI script to finish:

  process_that_takes_a_long_time >/dev/null 2>&1 &
  echo Content-type: text/plain
  echo Finished!

This doesn't work. The reason is that the started process keeps its stdout and stderr open, it's just redirected to the null device. Use the following syntax to close them:

  process_that_takes_a_long_time >&- 2>&- &

Your script will immediately finish.

Passwords and kill catching

  trap "stty echo ; exit" 1 2 15
  stty -echo
  read password
  stty echo
  trap "" 1 2 15

If the user press Ctrl+C in the password prompt, the normal stty mode will be restored

Run commands as another user

To run a single command as another user in a script run by root:

  su mysql -c mysql_install_db

If you want to run multiple commands, a HERE document is very useful:

  su - mysql <<HERE

Be careful though; environment variables are taken from the environment outside of the su statement, not the user that was switched to. Take, for example, the following script:

  su - nobody << HERE
  echo "User is: $USER"
  echo "Output of id: ${ID}"

When running as root, this will print:

  User is: root

Backticks also seem problematic, better not use them inside the su HERE block.

Hiding passwords

Sometimes, you need to put a password in a shell script. There are several methods to hide passwords from the unintentional glance by those who you trust. And you need to trust them, since the password can easily be recovered.

  $ echo secret_password | rot13

In the shell script, do something like:

  PASSWORD=`echo frperg_cnffjbeq | rot13`   # Colleagues, please don't look at this password

Dialog boxes

To bring up a dialog box through a shellscript in X, use the xdialog package:

  Xdialog --msgbox "Don't forget your coffee" 10 50

Creating temporary files and directories

The safest method is to use mktemp:

  echo "Very important data" > $TMPFILE

The mktemp command creates a unique file in /tmp and prints the name. The backticks catch the name and put it in the TMPFILE variable.

If you need a directory, add the -d option.

If you need a prefix, pass a template where the row of capital X'en is replaced by a unique string:

  TMPFILE=`mktemp -t hello_XXXXXXXXXX` || exit 1
  echo "Very important data" > $TMPFILE

Logging logins

  # If we're logging in through SSH, write this down
  if [ -n "$SSH_CLIENT" ]; then
    # The variable SSH_CONNECTION has the form
    if [ -e $LOGFILE ]; then
    # Alternative settings
    FROM=`echo $SSH_CLIENT | cut -f1 -d" "`
    case $FROM in
        export TMOUT=180 #Logout after 3 minutes

Removing prefixes and extensions

The other substring operator is "#" which removes prefix patterns. If you think about it, "#" is used before a number, e.g. #6, and "%" appears afterwards, e.g. "6%". This will help keep it clear which one removes prefixes and which suffixes.

The other thing to note about these operators is that a _single_ # or % means match the shortest substring and that doubling the operator means match the longest substring. i.e. "%%" and "##".

These let you avoid a lot of external programs in shell scripts. e.g. dirname(1) and basename(1) can be more effeciently done within sh as "${file%/*} and "${file##*/}.

FYI, here's the chunk from the "Paremter Expansion" section of the Bourne shell sh(1) manpage. Note the below are all part of the Single Unix Specification standard and have been for years.

             Remove Smallest Suffix Pattern.  The word is expanded to produce
             a pattern.  The parameter expansion then results in parameter,
             with the smallest portion of the suffix matched by the pattern
             Remove Largest Suffix Pattern.  The word is expanded to produce a
             pattern.  The parameter expansion then results in parameter, with
             the largest portion of the suffix matched by the pattern deleted.
             Remove Smallest Prefix Pattern.  The word is expanded to produce
             a pattern.  The parameter expansion then results in parameter,
             with the smallest portion of the prefix matched by the pattern
             Remove Largest Prefix Pattern.  The word is expanded to produce a
             pattern.  The parameter expansion then results in parameter, with
             the largest portion of the prefix matched by the pattern deleted.

-- From a post by Adrian Filipi

Some examples...

To print the current directory name without the leading path:

  $ echo ${PWD##*/}

In a script, to print the script name:

  echo ${0##*/}

To remove the extension from a filename:

  echo ${FILENAME%.tar.gz}

Script displaying feedback

The following bash function can be included in your scripts to provide feedback to the user whenever an action (such as starting a process) takes time. Instead of the while [ true ], you should put a condition there which can be periodically checked.

 function bounce() {
     while [ true ]; do
         sleep 0.5
         if [ $dot -eq 0 ]; then
             echo -n $'\b'
             echo -n "."
             echo -n $'\b'
             echo -n " "