Is there a method to indicate the last iteration in a loop over lines of an input file?

Question

Suppose I have a file direction with the lines

east
north
south
west
south-west

and using a loop and echo in a shell script I want to generate this output:

Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

So in other words I want to do something different with the last line in the script.

Answer 1

bash can't detect the end of a file (without trying to read the next line and failing), but perl can with its eof function:

$ perl -n -e 'print "Last " if eof; print "Direction: $_"' direction 
Direction: east
Direction: north
Direction: south
Direction: west
Last Direction: south-west

note: unlike echo in bash, the print statement in perl doesn't print a newline unless you either 1. explicitly tell it to by including \n in the string you're printing, or 2. are using perl's -l command-line option, or 3. if the string already contains a newline....as is the case with $_ - which is why you often need to chomp() it to get rid of the newline.

BTW, in perl, $_ is the current input line. Or the default iterator (often called "the current thingy" probably because "dollarunderscore" is a bit of a mouthful) in any loop that doesn't specify an actual variable name. Many perl functions and operators use $_ as their default/implicit argument if one isn't provided. See man perlvar and search for $_.

sed can too - the $ address matches the last line of a file:

$ sed -e 's/^/Direction: /; $s/^/Last /' direction 
Direction: east
Direction: north
Direction: south
Direction: west
Last Direction: south-west

The order of the sed rules is important. My first attempt did it the wrong way around (and printed "Direction: Last south-west"). This sed script always adds "Direction: " to the beginning of each line. On the last line ($) it adds "Last " to the beginning of the line already modified by the previous statement.

Answer 2

The loop cannot know when it gets to the end unless it actually gets to the end. So either you process your file twice, once to get the number of lines and another to output, or you print out the previous value on each iteration and the last one on exiting the loop:

prev=""
while read direction; do 
    if [ -n "$prev" ]; then 
        echo "Direction: $prev"
    fi
    prev="$direction"
done < direction
echo "Last Direction: $prev" 

Answer 3

A variation of FelixJN's answer, reading the lines into an array, but using bash builtins instead of bash loops:

mapfile -t DIR < direction
printf "Direction: %s\n" "${DIR[@]::${#DIR[@]}-1}"
printf "Last direction: %s\n" "${DIR[-1]}"

mapfile is a builtin added in Bash 4 which reads standard input into an array with one line per entry. The -t option strips the newlines.

printf will apply its format string repeatedly until it has used up all its input arguments.

Answer 4

You may read in the values into an array and then use the array length:

#!/bin/bash
i=0
while read line ; do
  arr[$i]="$line"
  ((i++))
done <file
for (( i=0 ; i<=${#arr[@]}-2 ; i++ )) ; do
  echo Direction: ${arr[i]}
done
echo Last direction: ${arr[-1]}

Of course this means having all data in RAM.

The simpler arr=( $(cat file) ) would only work if the line entries do not have spaces, thus I used a while read-loop here.

Answer 5

You can use a mix of paste, tail and head:

$ paste -s -d'\n' <(head -n -1 direction | sed 's/^/Direction: /') <(tail -n1 direction | sed 's/^/Last direction: /')
Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

---

• paste -s -d'\n' uses new line as delimiter with the option -d'\n', and the option -s:

  -s, --serial
  paste one file at a time instead of in parallel

• head -n -1 retrieves all lines but the last, then is piped to sed to perform the command you need in those lines only.

• tail -n1 file retrieves the last line of the file, then is piped to sed to perform the command you need in that line only.

---

Or more succintly just using cat instead of paste, as noted by @Mick Matteo's comment:

$ cat <(head -n -1 direction | sed 's/^/Direction: /') <(tail -n1 direction | sed 's/^/Last direction: /')

Answer 6

Expanding on the answers by FelixJN and Nick Matteo: using portable shell features (noting that the question has no shell-specific tags) and assuming you are processing a controlled, "small-enough" set of lines, you can know when the loop reaches its last element if you place the data into the array of positional parameters:

IFS='
'
set -f
set -- $(cat direction)
for dir
do
  if [ $(( i = i + 1 )) -eq "$#" ]
  then
    printf '%s' 'Last direction: '
  else
    printf '%s' 'Direction: '
  fi
  printf '%s\n' "$dir"
done

Keep in mind that you may want to properly restore IFS after setting it.

Note, however, that this will remove any empty input lines from the output. If you are not fine with this, you can load the array of positional parameters in a loop:

set --
while IFS= read -r line
do
  set -- "$@" "$line"
done <direction

(No need to set IFS to a newline and set -f in this case).

---

The approach from the answers by cas (sed part) and terdon — handle the last element "outside" the loop — can also be implemented in AWK using a quite common pattern:

awk '
  NR > 1 { print "Direction: " dir }
  { dir = $0 }
  END { if (NR > 0) print "Last direction: " dir }
' direction

Answer 7

A different strategy: note that the last line of your file is the first line of your reversed file. So, instead of relying on line indices, you can use a basic echo loop if you reverse the order of lines in your file with tac, which is part of coreutils. Then, reverse the order of your loop output to retrieve the original order.

prefix='Last direction:'
while read line; do
   echo "$prefix $line"
   prefix='Direction:'
done < <(tac direction) | tac

Answer 8

col=`awk 'END{print NR}' filename`
awk -v co="$col" '{if (NR<co){print "Direction: "$0}else{print "Last direction: "$0}}' filename

output

Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

Answer 9

One way would be to loop over the lines and delay printing the lines by one iteration: we print the previous line stored from a variable (prev in the code below) and then store the current line into this variable afterwards. Once the loop is finished, we have the last line in prev and can print it with its special prefix:

#! /bin/sh
prev=""
first="yes"
# from https://stackoverflow.com/a/1521498/648741
while IFS="" read -r p || [ -n "$p" ]; do
  if [ "$first" = "yes" ]; then
    first="no"
  else
    printf 'Direction: %s\n' "$prev"
  fi
  prev="$p"
done < directions
if [ "$first" = "no" ]; then
    printf 'Last direction: %s\n' "$prev"
fi

The code here takes some extra care to not damage whitespace and to allow for dollar signs within the lines. If this is not needed, a simpler version also works:

#! /bin/sh
prev=""
while read -r p; do
  if [ -n "$prev" ]; then
    echo "Direction: $prev"
  fi
  prev="$p"
done < directions
if [ -n "$prev" ]; then
    echo "Last direction: $prev"
fi

Answer 10

Using Raku (formerly known as Perl_6)

raku -e 'my $fh = $*ARGFILES.IO.open; $fh.eof ?? put("Last direction: $_") !! put("Direction: $_") for $fh.lines; $fh.close;' file.txt

OR

raku -e 'my $fh = $*ARGFILES.IO.open; for $fh.lines() { if not $fh.eof { put "Direction: $_" } else { put "Last direction: $_" };}; $fh.close;' file.txt

Sample Input:

east
north
south
west
south-west

Sample Output:

Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

Currently, Raku implements a .eof method which operates on either IO::Handle or IO::CatHandle objects and "...returns True if the read operations have exhausted the contents of the handle... ."

The first code example above uses Raku's ternary operator $condition ?? $true !! $false to test whether $fh.eof is True. Note above for both code examples how .eof operates directly on the $fh filehandle scalar, whereas $_ gets loaded with retrieved (line-wise) data, which is printed using put (or say, if desired).

Note, the rather simplistic code examples above only work on a single file at a time at the bash command line. However it's likely that feeding a list of files to $fh (for example via Raku's dir() method) may be the best way to manipulate multiple files at once.

https://docs.raku.org/routine/eof
https://docs.raku.org/language/operators#index-entry-operator_ternary
https://raku.org