104

I'm pretty comfortable with bash, but recently I ended up in a substitution I didn't know.

What exactly is <(command) in bash? How does it compare to the =(command) in zsh?

I understand that this has something to do with default file descriptors. In my computer

echo <()

returns /proc/self/fd/11, which I found out to be a copy of the script STDOUT, but this still seems pretty confusing to me.

ctrl-alt-delor
  • 2,381
Henrique Barcelos
  • 1,257

3 Answers3

147

This is called process substitution.

The <(list) syntax is supported by both, bash and zsh. It provides a way to pass the output of a command (list) to another command when using a pipe (|) is not possible. For example when a command just does not support input from STDIN or you need the output of multiple commands: 

diff <(ls dirA) <(ls dirB)

<(list) connects the output of list with a file in /dev/fd, if supported by the system, otherwise a named pipe (FIFO) is used (which also depends on support by the system; neither manual says what happens if both mechanisms are not supported, presumably it aborts with an error). The name of the file is then passed as argument on the command line.

zsh additionally supports =(list) as possible replacement for <(list). With =(list) a temporary file is used instead of file in /dev/fd or a FIFO. It can be used as a replacement for <(list) if the program needs to lseek in the output.

According to the ZSH manual there might also be other issues with how <(list) works:

The = form is useful as both the /dev/fd and the named pipe implementation of <(...) have drawbacks. In the former case, some programmes may automatically close the file descriptor in question before examining the file on the command line, particularly if this is necessary for security reasons such as when the programme is running setuid. In the second case, if the programme does not actually open the file, the subshell attempting to read from or write to the pipe will (in a typical implementation, different operating systems may have different behaviour) block for ever and have to be killed explicitly. In both cases, the shell actually supplies the information using a pipe, so that programmes that expect to lseek (see man page lseek(2)) on the file will not work.

Adaephon
  • 5,834
  • This helped me figure out why MacOS pfctl -f <(echo "pf rules") would say bad file descriptor. using zsh and =(echo "pf rules") instead works. – johnnyB Dec 20 '17 at 22:42
  • Anyone know what this is called or how to do this same thing in fish? I get "invalid redirection target" in fish shell with <(command) format. – Elijah Lynn Feb 28 '20 at 22:50
  • 1
    @ElijahLynn It is called "process substitution". (Also, see the first line of the answer ;-) ). In fish this can be achieved with psub. – Adaephon Mar 06 '20 at 08:33
  • I thought process substitution is the (command) part. Or does that include the < prefix as well or is there another name for that part? – Elijah Lynn Mar 06 '20 at 16:38
  • @ElijahLynn: It includes the prefix; <( acts as a different token from lone <. Process substitution doesn't even behave like redirection (it doesn't alter the 'primary' command's stdin or stdout, but expands into a word) -- so it is definitely not the same as just having a < followed by ( ). – u1686_grawity Apr 25 '20 at 14:09
  • Can someone please explain why is the additional < required, and <( is not enough if it already redirects the stout of a command to a /dev/fd file? For example why head -c1 <(echo test) works fine and read x <(echo test) not? – gnom1gnom Dec 09 '20 at 13:56
  • I figured out the answer to my question: head takes a file parameter, while read takes stdin behind the scene, after process substitution, these commands look as follows head -c1 /dev/fd/63 read x < /dev/fd/63 – gnom1gnom Dec 09 '20 at 15:17
  • Is there a way to give a custom filename/extension to the <(...) file – minseong Mar 29 '21 at 22:01
  • You can also use this even for programs that determine input type by file extensions! ZSH lets you specify a suffix for the temp files created by =(); see here – forresthopkinsa Oct 17 '22 at 20:47
31

Note, this is a bash answer, not zsh.

There are cases in bash where you can't use pipes:

some_command | some_other_command

because pipes introduce subshells for each component of the pipeline, when the subshells exit, any side-effects you were relying on would disappear. For example, this contrived example:

cat file | while read line; do ((count++)); done
echo $count

will display a blank line, because the $count variable does not exist in the current shell.

A bash process substitution allows you to avoid this conundrum by allowing you to read from the "some_command" output like you would from a file

while read line; do ((count++)); done < <(cat file)
# ....................................1.2
echo $count   # the variable *does* exist in the current shell

(1) is a normal input redirection. (2) is the start of the <() process substitution syntax.

glenn jackman
  • 26,306
  • 2
    =(cmdlist) in zsh has almost the same effect as <(cmdlist) in bash but it creates (and deletes when ready) a temporary file with the output of cmdlist for the redirection. This is good when seeking is potentially done in the program. <(cmdlist) is known by zsh as well. – user556625 Mar 31 '16 at 19:22
  • In bash 4.2 up (released 2011, though not always used immediately) if shopt lastpipe is on and job control is inactive (usually true for script but not interactive, but can be changed) and the pipeline is not backgrounded, the last/rightmost command (only) runs in main shell not subshell and var settings etc are preserved. Other shells may have this as an option, or even automatic. – dave_thompson_085 Nov 29 '20 at 00:43
  • The contrived example can be done in pure Bourne shell using here docs instead, which is much more portable and simpler and easier to understand. – Steve Wills Jun 30 '21 at 16:33
2

Another difference between =(command) & <(command) in zsh is synchronous vs asynchronous execution:

date & \
diff =(sleep 4; date) =(sleep 5; date) & \
diff <(sleep 4; date) <(sleep 5; date) &

Sun 29 Nov 2020 08:16:01 AEDT
[1]    41717 done       date

$ 1c1
< Sun 29 Nov 2020 08:16:05 AEDT
---
> Sun 29 Nov 2020 08:16:06 AEDT

[3]  + 41719 exit 1     diff <(sleep 4; date) <(sleep 5; date)

$ 1c1
< Sun 29 Nov 2020 08:16:05 AEDT
---
> Sun 29 Nov 2020 08:16:10 AEDT

[2]  + 41718 exit 1     diff =(sleep 4; date) =(sleep 5; date)

... although no idea if that was by design or not, as I can't think of a compelling reason to have =() function synchronously.

MrsFixit
  • 21
  • The difference is possibly that =() needs to write the program output to a temporary file, so it waits for program completion to take ownership of the output file, while <() just hooks up stdout of the sub-program as a pipe/FIFO input, and so must be asynchronous or it will block forever waiting for a reader, which is waiting for it, as described in the zsh manual quote in @Adaephon's answer above. – TBBle May 30 '21 at 16:40