Reading & Processing Input in Bash

In Bash, reading data from files or the standard input empowers scripts with data processing and user interactivity capabilities.

By reading from files, scripts can process data stored in various formats, such as configuration files, log files, or datasets, allowing for automated sorting, filtering, and analysis.

By reading from the standard input, scripts can adapt their behavior and logic according to the user's needs, enabling them to make choices during its execution, which ultimately improves its flexibility and configurability.

Reading from a file

To load the contents of a file at once and store it into a variable, you can combine the command substitution expansion $() with the cat command:

contents=$(cat filepath)

Where:

• filepath is the filepath to the target file.

Example

Let's consider this file named lorem.txt located in the current directory:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Pellentesque non orci nisi.
Curabitur vitae orci id mauris elementum rutrum.

Let's consider this script, that reads and outputs the contents of this file:

#!/bin/bash

file=lorem.txt
contents=$(cat $file)

echo $contents

When executed, it will:

1. Declare a variable named file and initialize it with the string "lorem.txt".
2. Declare a variable named contents and initialize it with the output of the cat command.
3. Output the value of the contents variable.

Which will produce this output:

$ ./script.sh
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Pellentesque non orci nisi.
Curabitur vitae orci id mauris elementum rutrum.

Reading a file line by line

To read and process a file line by line, you can define the read command as the condition of a while loop, and feed it the file's content using the input redirection operator:

while read line;
do
  # process line
done < filepath

Where:

• line is a variable used to store the current line.
• filepath is the filepath to the target file.

Note: When using this syntax, the while loop will automatically stop when the read command encounters an EOF (end-of-file).

Example

Let's consider this script, that reads and outputs the contents of the lorem.txt file line by line:

#!/bin/bash

file=lorem.txt

while read line;
do
  echo "[$line]"
done < $file

When executed, it will:

1. Declare a variable named file and initialize it with the string "lorem.txt".
2. Start a while loop that stores the current line into the line variable and output its value between square brackets to the standard output.
3. Re-run the while loop until the read command reaches the end of the file.

Which will produce this output:

$ ./script.sh
[Lorem ipsum dolor sit amet, consectetur adipiscing elit.]
[Pellentesque non orci nisi.]
[Curabitur vitae orci id mauris elementum rutrum.]

Splitting the lines with a delimiter

To read a file line by line and split each line into an array of values, you can use the read -a command combined with the IFS (Internal Field Separator) variable used to define a custom delimiting character:

while IFS="delimiter" read -a line;
do
  # process line
done < filepath

Where:

• The -a flag is used to split the line into an array using the IFS delimiter.

Note: By default, the value of the IFS variable is the whitespace character ' '.

Example

Let's consider this file named clients.csv in the CSV (comma-separated values) format:

first_name,last_name,email_address
John,Doe,johndoe@mail.com
Alice,Guthrie,agth03@mail.com
Matthew,Jasper,mattjasp@mail.com

Let's consider this script, that reads the contents of the clients.csv file line by line and splits each line into distinct elements:

#!/bin/bash

file=clients.csv

tail -n +2 $file | while IFS="," read -r -a line;
do
  echo "First name: ${line[0]}"
  echo "Last name: ${line[1]}"
  echo "Email address: ${line[2]}"
  echo "---"
done

When executed, it will:

1. Declare a variable named file and initialize it with the string "clients.csv".
2. Read the file starting at the second line using the tail -n +2 command and pipe it to the read command.
3. Read the current line, split it into multiple elements using ',' as delimiter through the IFS variable, and store it into the line variable.
4. Write each element separately to the standard output.
5. Re-run the while loop until the read command reaches the end of the file.

Which will produce this output:

$ ./script.sh
First name: John
Last name: Doe
Email address: johndoe@mail.com
---
First name: Alice
Last name: Guthrie
Email address: agth03@mail.com
---
First name: Matthew
Last name: Jasper
Email address: mattjasp@mail.com
---

Reading from the standard input

Reading data from the standard input rather than from a file allows scripts to be more interactive by prompting users for input during runtime.

It also allows scripts to process larger amounts of data by bypassing command-line arguments length or memory limitations.

To read user input from the standard input, you can use the read command:

read [-s] [-p "prompt"] variable

Where:

• The -s flag (short for silent) is used to prevent the user's input from being displayed into the terminal.
• The -p flag (short for prompt) is used to prompt the user with a message.

Example

Let's consider this script, that makes the user guess a number:

#!/bin/bash

tries=0

read -s -p "Please enter a number: " number
echo -e "\nGot it! Clearing the screen in 3 seconds..."
sleep 3
clear

while read -p "Guess the number: " guess;
do
  if [[ number -eq guess ]]; then
    echo "Great job! You guessed in $tries tries."
    break
  elif [[ number -gt guess ]]; then
    echo "Too low..."
    ((tries++))
  else
    echo "Too high..."
    ((tries++))
  fi
done

When executed, it will:

1. Declare a variable named tries and initialize it to 0, that will contain the number of attempts made by the player to guess the number.
2. Ask the first player to type a number, hide its input using the -s flag, and store it into the number variable.
3. Pause the execution of the program for 3 seconds using the sleep command.
4. Clear the terminal screen using the clear command.
5. Prompt the second player to type a number and store its input into the guess variable.
6. Check if the value of the guess variable equals the value of the number variable.
7. If it evaluates to true, stop the infinite while loop using the break statement.
8. Otherwise, check if the guess variable is greater than the number variable and increment the tries variable.
9. Otherwise, check if the guess variable is lower than the number variable and increment the tries variable.
10. Repeat the loop from step 5.

Which will produce this output:

$ ./script.sh
Please enter a number:
Got it! Clearing the screen in 3 seconds...
Guess the number: 1
Too low...
Guess the number: 5
Too low...
Guess the number: 12
Too high...
Guess the number: 10
Too high...
Guess the number: 9
Great job! You guessed in 4 tries.

Creating a multiple choice menu

To create a multiple choice menu that allows users to choose from a list of options, you can use the select...in construct:

select element in list;
do
  # process element
done

Where:

• element is a variable used to store the user's choice.
• list is a list of space-separated elements.

Example

Let's consider the recipes directory containing the following files:

$ ls recipes
crepes.txt   pancakes.txt

Let's consider this script, that outputs a recipe stored in the recipes directory based on the user's input:

#!/bin/bash

recipes=("pancakes" "crepes")

echo "Select a recipe to load:"

select recipe in "${recipes[@]}"; do
  file="recipes/$recipe.txt"

  if [[ -f $file ]]; then
    cat "$file"
    break
  else
    echo "Invalid selection. Please try again."
  fi
done

When executed, it will:

1. Declare a variable named recipes and initialize it with an array of recipe names.
2. Output the list of available choices to the user and store the user's answer into the recipe variable.
3. Store the concatenated path to the recipe file into the file variable.
4. Check if the filepath in the file variable points to a valid regular file.
5. Output the contents of the file.
6. Exit the select...in instruction.
7. Otherwise, output an error message.

Which will produce this output:

$ ./script.sh
Select a recipe to load:
1) pancakes
2) crepes
#? 3
Invalid selection. Please try again.
#? 1
--- Pancakes ---

Ingredients:

- 1 cup all-purpose flour
- 2 tbsp sugar
- 1 tsp baking powder
- 1/2 tsp baking soda
- 1/4 tsp salt
- 1 cup buttermilk (or milk with 1 tbsp vinegar added)
- 1 large egg
- 2 tbsp melted butter (plus extra for cooking)
- 1 tsp vanilla extract (optional)

Steps:

1. In a large bowl, whisk together the flour, sugar, baking powder, baking soda, and salt.
2. In another bowl, combine the buttermilk, egg, melted butter, and vanilla extract.
3. Pour the wet ingredients into the dry ingredients and stir until just combined. The batter may be a bit lumpy—do not overmix.
4. Heat a non-stick skillet or griddle over medium heat and add a little melted butter to coat the surface.
5. Pour about 1/4 cup of batter onto the skillet for each pancake. Cook until bubbles form on the surface and the edges look set (about 2-3 minutes).
6. Flip the pancakes and cook for another 1-2 minutes until golden brown.
7. Serve warm with maple syrup, fresh fruit, or your favorite toppings.

Summary

Here's a summary of what you've learned in this lesson:

• To store the contents of a file into a variable, you can use the $(cat file) command.
• To read a file line by line, you can use a while read < file loop.
• To split the content of a line into separate elements, you can use the IFS variable to define a custom separator.
• To read from the standard input, you can use the read [-s] [-p] command.
• To create a multiple choice menu, you can use the select...in construct.