Reading multiple lines in Bourne Shell

Question

I'm trying to read two lines into two variables. In Bash I would use something like this:

cat << EOF > myfile
line1
line2
EOF

cat myfile |  {
 read firstline
 echo $firstline # "line1" in bash and sh
 read secondline
}

echo $firstline # "line1" in bash, empty in sh
echo $secondline

In Bourne Shell however $firstline and $secondline are empty outside the command group. How can I do it in sh?

Answer 1

(You most likely have firstline set already when you tested that code in bash, its value should be empty at the end).

When running the pipeline

cat myfile | { read firstline; read secondline; }

the right hand side is running in a subshell. Not because of the { ...; } but because it's part of a pipeline. The subshell environment will contain the two variables firstline and secondline (after both have been read), but will be destroyed when the subshell terminates, discarding both variables.

This holds true for both POSIX sh and bash.

In bash (4.2+), you can work around this by setting the lastpipe shell option. From the bash manual:

Each command in a pipeline is executed as a separate process (i.e., in a subshell). See COMMAND EXECUTION ENVIRONMENT for a description of a subshell environment. If the lastpipe option is enabled using the shopt builtin (see the description of shopt below), the last element of a pipeline may be run by the shell process.

This would work in a script, but not in an interactive shell with job control enabled (it's the job control that makes it not work, not the interactivity).

Example:

$ cat script.sh
cat << EOF > myfile
line1
line2
EOF

cat myfile | { read firstline; read secondline; }
printf 'first=%s\n' "$firstline"
printf 'second=%s\n' "$secondline"

shopt -s lastpipe
cat myfile | { read firstline; read secondline; }
printf 'first=%s\n' "$firstline"
printf 'second=%s\n' "$secondline"

$ bash script.sh
first=
second=
first=line1
second=line2

In your particular case, you could also, in both POSIX sh and bash, do away with cat and the pipe completely and instead redirect into the compound command with the two read calls directly:

{ read firstline; read secondline; } <myfile

On a tangential note, you most likely do not have a real historical Bourne shell on your machine (unless it's a Solaris system before Solaris 11). I'm assuming you mean a modern POSIX sh shell.

Answer 2

When you use the right side of a pipe, there are some limitations.

I changed your code to ensure that the variables were unset before use, and compressed it a little bit.

#! /bin/sh

unset firstline secondline
printf '%s\n%s\n' line1 line2 > myfile

cat myfile | { read firstline; read secondline
               echo "internal 1: $firstline"     # "line1" in most shells
               echo "internal 2: $secondline"    # "line2" in most shells
             }

echo "Outside 1: $firstline"     # "line1" in ksh, zsh (even called as sh).
                                 #  Empty in sh, bash, mksh, dash, ash, yash.
echo "Outside 2: $secondline"    # "line2" in ksh, zsh.
                                 # Same as above for other shells.

Internals to the compound command {...} had values, external (Outside) were empty in many cases (especially POSIX, the portable case).

Only if you set shopt -s lastpipe you get bash to keep the values on the right side of the pipe symbol. Both ksh(93) and zsh keep the values read by default.

So, the problem is to avoid the pipe | symbol, well, in this case it is very simple. Either <file { ... } or {...} <file and you are set to keep the value of variables portably.

#! /bin/sh

unset firstline secondline
printf '%s\n%s\n' line1 line2 > myfile

{ read firstline; read secondline
  printf '%s\n' "internal 1: $firstline"     # "line1" in most shells
  printf '%s\n' "internal 2: $secondline"    # "line2" in most shells
} < myfile

printf '%s\n' "Outside 1: $firstline"          # "line1" in most shells.
printf '%s\n' "Outside 2: $secondline"         # "line2" in most shells.

That worket in all shells tested except the old Bourne shell (it lacks the {...} structure that runs in the present shell, the workaround becomes more difficult, not worth the time, read @schily answer for that).

Answer 3

You cannot do this using read in the Bourne Shell since the Bourne Shell has two limitations:

• The right side of a pipeline is always run in a subshell

  With echo foo | read VAR, the read command is always run in a subshell

• The Bourne Shell places command lists with shell builtin commands in { ...; } into a subshell. So

  { read A; read B; } < file is also run in a subshell.

If you like to verify this, I recommend you to check this with obosh from the schilytools, which is a highly portable version of the Solaris Bourne Shell.

See: http://sourceforge.net/projects/schilytools/files/

Modern shells (starting with ksh93 run the rightmost program of a pipeline inside the main shell process in case this is a shell builtin. So when you use the recent version of the Bourne Shell (bosh), this works for you. bosh also does not create a subshell for { ...; } for performance reasons and this has the side effect that

{ read A; read B; } < file

works with bosh

The following should work with all shells:

exec 3<&0               # save standard input as fd 3
exec < file             # use file as stdin
read A
read B
exec 0<&3               # restore standard input
exec 3<&-               # close file descriptor 3