Handling Errors & Debugging in Bash

In programming, error handling is the process of gracefully responding to and recovering from unexpected execution errors in a program.

On the other hand, debugging is the process of identifying and removing programming errors and faulty logic from a program.

These practices ensure that your script behaves predictably and provides meaningful feedback to both developers and users.

Testing the existence and values of variables

In scripting, the first strategy for preventing execution errors consists in verifying the existence and values of the different inputted or computed variables, before they are used by the other components of the script, such as functions and commands.

This helps improve the reliability of the script by ensuring that the data is initialized and in conformity with the expected type, and by avoiding executing functions and commands that are likely to fail due to invalid parameters, provoking unexpected crashes or behaviors.

This is usually done by creating conditions within if and elif statements using comparison operators, such as ==, -eq, or =~.

Since we've already covered strings and numbers comparison, we'll now cover nullity and filepaths validation.

Testing null variables

A null variable typically refers to a variable that hasn't been defined or initialized with a value.

variable
variable=""

To test if a variable is null, has zero length, or is undefined, you can use the -z operator:

[[ -z $variable ]]

On the other hand, to test if a variable is not null, you can use the -n operator

[[ -n $variable ]]

Testing filepaths

To test if a file identified by a path exists, you can use the -e operator (short for exists):

[[ -e $filepath ]]

To test if a file is a regular file, you can use the -f operator (short for file):

[[ -f $filepath ]]

To if a file is a directory, you can use the -d operator (short for directory):

[[ -d $filepath ]]

Example

Let's consider this script, that checks whether the supplied positional argument is a valid regular file:

#!/bin/bash

file=$1

if [[ -z $file ]]; then
  echo "Usage $0 <file>"
elif [[ ! -f $file ]]; then
  echo "Error: $file is not a regular file"
else
  echo "$file is a regular file"
fi

When executed, it will:

1. Declare a variable named file and initialize it with the value of the first positional argument.
2. Evaluate whether the value of the file variable is undefined and output an error message if true.
3. Otherwise, evaluate whether the value of the file variable doesn't point to a valid regular file and output an error message if true.
4. Otherwise, output a validation message.

Which will produce this output:

$ ./script.sh
Usage: ./script.sh <file>
$ ./script.sh .
Error: . is not a regular file
$ ./script.sh script.sh
script.sh is a regular file

Testing the exit status of commands and functions

Another strategy for preventing execution errors consists in verifying the exit status code of commands and functions to conditionally execute (or not) the next set of instructions.

command

if [[ $? -eq 0 ]]; then
  instructions
fi

function

if [[ $? -ne 0 ]]; then
  instructions
fi

This can help, for example, preventing the script from attempting to create a file in a non-existent directory, process incomplete sent by a third-party service, or send unauthenticated requests to a server.

Example

Let's consider the directory named tmp where only the owner has read and execute permission on:

$ ls -l
-rwxr-xr-x  1 razvan  staff  302 Nov  8 19:21 script.sh
dr-x------  2 razvan  staff   64 Nov  8 19:06 tmp

Let's consider this script, that writes its execution date and time into the file named logs located in the directory named tmp:

#!/bin/bash

logger () {
  local file="tmp/logs"

  if [[ ! -e $file ]]; then
    touch $file

    if [[ $? -ne 0 ]]; then
      echo "Error: Cannot create $file"
      return 1
    fi
  fi

  echo $1 >> $file
}

logger "Script executed at $(date +'%H:%M:%S')"

When executed, it will:

1. Declare a function named logger.

   • Define a local variable named file and initialize it with the path to the "tmp/logs" file.
   • Check if the logs file doesn't exist in the tmp directory and create it using the touch command if it evaluates to true.
   • Check if the exit status of the touch command doesn't equal to 0, and output an error message and perform an early return if it evaluates to true.
   • Write the function parameter string into the logs file.

2. Invoke the logger function with a string containing the current date and time as arguments.

Which will produce this output:

$ ./script.sh 2>/dev/null
Error: Cannot create tmp/logs

Terminating a script on error

In some cases, it is preferable to immediately terminate the execution of a script as soon as an error is encountered, rather than trying to gracefully control it with complex error-handling logic.

This can help prevent undesirable side-effects, such as silent error propagation or data corruption, and simplify the debugging process allowing developers to identify the specific command that caused the error.

For example, it can help prevent a backup script from continuing running if the command intended to write the data to a file fails, which could otherwise result in a corrupted backup file with incomplete data.

Exiting on first error

To automatically terminate a script if a command exits with a non-zero status, you can use the set command with the -e flag right at the top of your script:

#!/bin/bash

set -e

# instructions

Example

Let's consider this script, that outputs the result of the division of two integers:

#!/bin/bash

set -e

divide() {
  if ! [[ $1 =~ ^-?[0-9]+$ && $2 =~ ^-?[0-9]+$ ]]; then
    echo "Error: Operands must be integers"
    return 1
  fi

  if [[ $2 -eq 0 ]]; then
    echo "Error: Division by zero"
    return 2
  fi

  echo $(($1 / $2))
}

divide 10 5
divide 4 0
divide 20 a
divide 20 5

When executed, it will:

1. Invoke the set -e command.

2. Define a function named divide.

   • Check if either one of the function's parameters is not an integer and perform an early return if true.
   • Check if the second function's parameter is not equal to 0 and perform an early return if true.
   • Output the result of the division of the first function parameter by the second.

3. Invoke the divide function 4 times, both with valid and invalid parameters.

Which will produce this output:

$ ./script.sh
2
Error: Division by zero

In comparison, if the set -e instruction wasn't declared, the script would have actually invoked the divide function 4 times instead of 2:

$ ./script.sh
2
Error: Division by zero
Error: Operands must be integers
4

Exiting with a status code

To manually terminate a script with a custom exit status code, you can use the exit command:

exit code

Where:

• code is an integer representing an exit status.

Example

Let's consider this file named leads.csv:

marie-anne,jolt,majolt@mail.com
marina,mmore@mail.com
viktor,glitch,glitchyvik@mail.com

Let's consider this script, that checks if each line of a CSV file has the same number of fields:

#!/bin/bash

file=$1
line_number=1
field_count=0

if [[ ! -f "$file" ]]; then
  echo "Error: File \"$file\" does not exist"
  exit 1
fi

while IFS= read -r line;
do
  current_field_count=$(echo $line | awk -F, '{print NF}')

  if [[ $line_number -eq 1 ]]; then
    field_count=$current_field_count
  fi

  if [[ $current_field_count -ne $field_count ]]; then
    echo "Error: Invalid file \"$file\""
    echo "Error: Field count error on line $line_number"
    exit 2
  fi

  ((line_number++))
done < $file

echo "File '$file' is a valid CSV."

When executed, it will:

1. Check if the file doesn't exist and immediately terminate the script.
2. Read the file line by line and store the current line into the line variable.
3. Get the number of fields (or columns) in the current line using the awk command and the ',' as a delimiting character.
4. Update the field_count variable with the value of the current_field_count variable if it is the first line of the file.
5. Check if the field count of the current line doesn't match the initial field count and terminate the script.
6. Increment the line number and re-run the while loop from step 2.

Which will produce this output:

$ ./script.sh leads.csv
Error: Invalid file "leads.csv"
Error: Field count error on line 2

Debugging techniques

In programming, debugging is the process of identifying, isolating, and resolving execution errors in a script or program.

This can be done in a multitude and often complementary ways, such as reproducing the conditions in which the bug appears, inspecting the code logic and flow, using temporary logs to print variables and execution messages, isolating instructions by commenting blocks of code, logging error messages in files, using debugging tools, and so on.

Logging error messages

Logging error messages allows developers to identify and locate issues in a script by providing critical information, such as an error code, an error type, a timestamp, a short description, a variable value, a line number, etc.

Writing these error logs into files is particularly useful when scripts are running on remote servers as it allows developers to monitor the scripts' health and performance in real-time, and set alerts to notify them when critical errors occur.

Example

Let's consider the directory named protected with no permissions for any user:

$ ls -l
-rwxr-xr-x  1 razvan  staff  302 Nov  8 19:21 script.sh
d---------  2 razvan  staff   64 Nov  8 19:06 protected

Let's consider this script, that logs an error when trying to create a file within this directory:

#!/bin/bash

log_error () {
  echo "Line $1 - $(date +'%Y-%m-%d %H:%M:%S') - Error: $2"
}

TMP_DIR="tmp"

mkdir "protected/$TMP_DIR"

if [[ $? -ne 0 ]]; then
  log_error $LINENO "Cannot create $TMP_DIR directory"
  exit 1
fi

When executed, it will:

1. Define a function named log_error that takes a line number and an error message as arguments and outputs them in a formatted manner.
2. Declare a new variable named TMP_DIR and initialize it with the string "tmp".
3. Attempt to create a directory named tmp within the protected directory.
4. Check if the exit status of the mkdir command is different from 0, indicating that the command failed.
5. Invoke the log_error function with the current line number stored in the LINENO variable and a custom error message, and terminate the script if it evaluates to true.

Which will produce this output:

$ ./script.sh 2> /dev/null
Line 12 - 2023-10-30 12:11:07 - Error: Cannot create tmp directory

Tracing script execution

In Bash, the xtrace or debug mode allows you to selectively trace the execution of a set of instructions by printing each command before it is executed, as well as the arguments it is called with, and its output.

To enable this feature, you can use the set -x command:

set -x

To disable this feature, you can use the set +x command:

set +x

Note: It is important to use this feature selectively, as it can generate a lot of output, especially in larger scripts, making it harder to focus on the relevant information.

Example

Let's consider this file named lorem.txt located in the current directory:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Pellentesque fermentum quam tempus arcu fermentum gravida.
Aenean orci magna, fringilla vitae laoreet vel, lobortis vel arcu.
Nam sed pretium lorem, a placerat nibh.
Donec diam ligula, tempor id fringilla at, dictum in turpis.

Let's consider this script, that simulates the Unix grep command:

#!/bin/bash

file=$1
pattern=$2

set -x

if [[ -z $file || -z $pattern ]]; then
  echo "Usage $0 <file> <pattern>"
  exit 1
elif [[ ! -f $file ]]; then
  echo "Error: File \"$file\" not found"
  exit 2
fi

set +x

while IFS= read -r line;
do
  if [[ $line =~ $pattern ]]; then
    echo $line
  fi
done < $file

When executed, it will:

1. Enable the debug mode.
2. Check if the file variable or the pattern variable are undefined and terminate the program if it evaluates to true.
3. Check if the file variable doesn't point to a valid regular file and terminate the program if it evaluates to true.
4. Disable the debug mode.
5. Read the file line by line and store the content of the current line into the line variable.
6. Check if the value of the line variable is equivalent to the value of the pattern variable and output its value.
7. Re-run the loop from step 5.

Which will produce this output:

$ ./script.sh lorem.txt ar
+ [[ -z lorem.txt ]]
+ [[ -z ar ]]
+ [[ ! -f lorem.txt ]]
+ set +x
Pellentesque fermentum quam tempus arcu fermentum gravida.
Aenean orci magna, fringilla vitae laoreet vel, lobortis vel arcu.

Summary

Here's a summary of what you've learned in this lesson:

• The -z operator checks if a variable is null, zero-length, or undefined.
• The -n operator checks if a variable is not null.
• The -e operator checks if a file exists.
• The -f operator checks if a file is a regular file.
• The -d operator checks if a file is a directory.
• The set -e command terminates a script if a command exits with a non-zero status.
• The exit command terminates a script with a custom status code.
• The set -x and set +x commands enable and disable the debug mode.