How does the "<( cmd )" pattern work in bash?

Question

In bash, I use arguments that look like

paste <(cat file1 | sort) <(cat file2 | sort)

or

comm <(cat file1 | sort) <(cat file2 | sort)

When I check man comm or man paste, the documentation says the args are indeed FILES.

Question:

Are intermediate temporary files get created (on TEMP filesystem or elsewhere on slower disk) for <(cat file1 | sort) and <(cat file2 | sort)?
What is the name for this <( ) magic? (to lookup its documentation)
Is it specific to bash or does it work across other shells?

And https://unix.stackexchange.com/questions/294635/what-is-the-bash-file-contents-syntax-called — muru, May 25 '18 at 01:45
Unrelated to your question, but cat file1 | sort is a Useless Use of cat, and should be written either sort file1 or <file1 sort. — wchargin, May 25 '18 at 07:02
To look under the hood: echo <(true) Or, even better: ls -l <(true) — Bernd Jendrissek, May 25 '18 at 08:43

score 18 · Accepted Answer · edited Jun 11 '20 at 14:16

This is called process substitution.

3.5.6 Process Substitution

Process substitution allows a process’s input or output to be referred to using a filename.

The process list is run asynchronously, and its input or output appears as a filename. This filename is passed as an argument to the current command as the result of the expansion. If the >(list) form is used, writing to the file will provide input for list. If the <(list) form is used, the file passed as an argument should be read to obtain the output of list. Note that no space may appear between the < or > and the left parenthesis, otherwise the construct would be interpreted as a redirection. Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files.

It is not just a bash thing as it originally appeared in ksh but it's not in the posix standard.

Under the hood, process substitution has two implementations. On systems which support /dev/fd (most Unix-like systems) it works by calling the pipe() system call, which returns a file descriptor $fd for a new anonymous pipe, then creating the string /dev/fd/$fd, and substitutes that on the command line. On systems without /dev/fd support, it calls mkfifo with a new temporary filename to create a named pipe, and substitutes this filename on the command line.

This also appears in zsh. To my surprise, most other shells don't do it despite process substitution being quite useful under a variety of circumstances. — Kevin, May 24 '18 at 20:13

Kusalananda · Answer 2 · 2018-05-24T20:18:06.517

You can think of <( somecommand ) as the filename of a temporary file containing the output of somecommand. In other words,

utility < <( somecommand )

is a shortcut for

somecommand >tempfile
utility <tempfile
rm -f tempfile

And

utility <( somecommand )

is a shortcut for

somecommand >tempfile
utility tempfile  # notice the lack of redirection here (utility expected to read from "tempfile")
rm -f tempfile

Likewise >( somecommand ) may be thought of as the filename of a temporary file that will be fed into somecommand on its standard input. In other words,

utility > >( somecommand )

is a shortcut for

utility >tempfile
somecommand <tempfile
rm -f tempfile

And

utility >( somecommand )

could possibly be a shortcut for

mkfifo temppipe
somecommand <temppipe &
utility temppipe  # utility is expected to write to "temppipe"
rm -f temppipe

(or something similar)

This is true as far as it goes, but you should be clear that pipes are used in all cases, not actual temporary files. This matters because it means that programs that try to seek around in the file won't work on process substitutions. — zwol, May 25 '18 at 12:12
@zwol This is a good point, I will add something to this effect in a while (I'm busy right now). — Kusalananda, May 25 '18 at 12:25

How does the "<( cmd )" pattern work in bash?

2 Answers2