5

Why can the command { echo err >&2; echo out >&1; } | : print "err", but the second can not?

Some tests in container:

  • Debian: 11.5 (bash/5.1.4)
  • Rocky Linux: 9.0 (bash/5.1.8),
  • CentOS: 7.7.1908 (bash/4.2.46),
  • CentOS: 8.4.2105 (bash/4.4.19).

This also happened in host CentOS 8.0.1905, Debian 11.4.

~# { echo err >&2; echo out >&1; } | :
err
~# { echo out >&1; echo err >&2; } | :
~#

And not using : seems right:

~# { echo out >&1; echo err >&2; } | sed 's/.*/-&-/'
err
-out-
~#

But print in another host CentOS 7.7.1908 (bash/4.2.46):

~# { echo out >&1; echo err >&2; } | :
err
~#
Peter Mortensen
  • 1,025
qyecst
  • 53
  • 1
    I tested in a debian:11.5 container and both commands gave output. – muru Nov 11 '22 at 04:24
  • I had the same result on Ubuntu 18.04. Both commands had the output "err". I'm also scratching my head trying to figure out if this has any use. – Wastrel Nov 11 '22 at 15:12
  • FYI, >&1 doesn't do anything. It's redirecting stdout to itself, which is a no-op. – Barmar Nov 11 '22 at 16:53

2 Answers2

9

The difference between

{ echo err >&2; echo out >&1; } | :

... and

{ echo out >&1; echo err >&2; } | :

... is the order in which the two echo calls are made. The order matters only in terms of how quickly the first echo can execute.

The two sides of the pipeline, { ...; } and :, are started concurrently, and : does not read from its standard input stream. This means the pipe is torn down as soon as : terminates, which will be almost immediately.

If the pipe is torn down, there is no pipe buffer for echo out >&1 to write to. When echo tries to write to the non-existent pipe, the left-hand side of the pipeline dies from receiving a PIPE signal.

If it dies before echo err >&2 has executed, there will be no output.

So:

  • You will always get err as output from { echo err >&2; echo out >&1; } | : since echo err >&2 will not care about the state of the pipe.

  • You will sometimes get err as output from { echo out >&1; echo err >&2; } | : depending on whether : has terminated before echo out >&1 has the chance to run. If : terminates first, there will be no output due to the left-hand side getting killed prematurely.

I'm guessing that most people will not experience the non-deterministic behaviour of the second pipeline on most systems. Still, you have found combinations of operating systems and shell versions that show this. I have also replicated the non-deterministic behaviour under zsh on FreeBSD, even in a single shell session (although the shell seems to tend to one behaviour rather than the other, so I needed 10+ tries to finally see the switch).

Kusalananda
  • 333,661
  • To view evidence of the SIGPIPE killing the left-hand side, you could look at $pipestatus in zsh or ${PIPESTATUS[@]} in Bash. Though for whatever reason, on Bash it seems far more likely for the left-hand side to run first than on zsh. (Based on some quick testing on one system anyway.) Of course, even something like { sleep .01; echo out >&1; echo err >&2; } | : would tip the scales. – ilkkachu Nov 11 '22 at 10:14
  • Interesting fact: echo being a builtin is crucial. Try { sleep 1; echo out >&1; echo err >&2; } | : vs { sleep 1; /bin/echo out >&1; echo err >&2; } | :. You may want to notice this in the answer and explain maybe (not for me particularly; I think I know why this happens). – Kamil Maciorowski Nov 11 '22 at 22:55
  • The key here is that echo is builtin in most shells. If it weren't, only the child process spawned to execute echo would die. Here, it's the subshell process running the left hand side of the pipeline that receives a SIGPIPE when it does a write() to a pipe that has no reader left. – Stéphane Chazelas Nov 12 '22 at 09:34
2

(firstly, if you are experimenting with things and you are not sure how they work and what they do, I highly suggest you stop doing so when logged in as root)

{ echo err >&2; echo out >&1; } | :

This will echo the string err\n, with standard output containing that string redirected to standard error (which is displayed on the terminal), followed by echoing the string out\n to standard output pointlessly redirected to standard output; that standard output is then piped as standard input into :, which outputs nothing.

Further testing:

$ { echo err >&2; echo out >&1; } 1> outputfile 2> errorfile
$ for file in *; do echo $file; cat $file; done
errorfile
err
outputfile
out
$ rm  errorfile outputfile
$ { echo err >&2; echo out >&1; } 1> outputfile 2> errorfile
$ for file in *; do echo $file; cat $file; done
errorfile
err
outputfile
out

All that said and demonstrated, I am unsure why you seem to be losing the output to standard error in some cases. I am not able to replicate this:

$ { echo err >&2; echo out >&1; } | :
err
$ { echo out >&1; echo err >&2; } | :
err
DopeGhoti
  • 76,081