29

I occasionally see things like:

cat file | wc | cat > file2

Why do this?

When will the results (or performance) differ (favourably) from simply:

cat file | wc > file2
OJFord
  • 1,963

5 Answers5

37

Both of those examples are useless uses of cat. Both are equivalent to wc < file1 > file2. There is no reason to use cat in this example, unless you are using cat file as a temporary stand-in for something that dynamically generates output.

larsks
  • 34,737
  • Ha, didn't even catch the first one. Thanks for link :) bookmarked to read the other "useless uses of ~" later. – OJFord Aug 24 '15 at 22:05
  • 7
    The first usage of cat is not necessarily useless here. The command wc file prints the counters followed by the name of the file. The command cat file | wc does not print the name of the file. The second cat is useless. wc file1 file2 prints two lines of counts, one for each file (plus the file names). cat file1 file2 | wc prints one line with the total counts, and no file names. – alephzero Aug 24 '15 at 22:34
  • 13
    @alephzero Reread the answer- cat file | wc is equivalent to wc < file1. – lily Aug 24 '15 at 23:10
  • 3
    @IstvanChung: Interestingly, on my system they're actually not equivalent. cat file |wc separates the line/word/character counts with more spaces than wc <file does. I don't know why. – Nate Eldredge Aug 25 '15 at 02:12
  • 21
    @IstvanChung: They're not equivalent. wc < file1 causes wc to run with stdin being a file descriptor for a regular seekable, mmappable file, file1. cat file1 | wc causes wc to run with a non-seekable pipe on stdin. – R.. GitHub STOP HELPING ICE Aug 25 '15 at 02:19
  • 10
    +1 for the last sentence. Often, a "useless" cat is a handy placeholder to be able to pop other commands in and out without rearranging the pipeline. – Reid Aug 25 '15 at 02:54
  • 1
    @alephzero +1, but, danger danger: If file1 does not end in a linefeed, then cat file1 file2 | wc will count one fewer lines, and potentially one fewer words, than would wc file1 file2 (which sees the "break" between file1 and file2). – Iwillnotexist Idonotexist Aug 26 '15 at 02:52
33
cat file | wc | cat > file2

would usually be two useless uses of cat as that's functionally equivalent to:

< file wc > file2

However, there may be a case for:

cat file | wc -c

over

< file wc -c

That is to disable the optimisation that many wc implementations do for regular files.

For regular files, the number of bytes in the file can be obtained without having to read the whole content of the file, but just doing a stat() system call on it and retrieve the size as stored in the inode.

Now, one may want the file to be read for instance because:

  • the stat() information cannot be trusted (like for some files in /proc or /sys on Linux):

    $ < /sys/class/net/lo/mtu wc -c
4096
$ cat /sys/class/net/lo/mtu | wc -c
6
  • one wants to check how much of the data can be read (like in case of a failing hard drive).
  • one just wants to obtain benchmarks on how fast the data can be read.
  • one wants for the content of the file to be cached in memory.

Of course, those are exceptions. In the general case, you'd rather use < file wc -c for performance reasons.

Now, you can imagine even more far fetched scenarios where one may want to use: cat file | wc | cat > file2:

  • maybe wc has an apparmor profile or other security mechanism that prohibits it from reading or writing to files while it's allowed for cat (that would be unheard of)
  • maybe cat is able to deal with large (as in > 232 bytes) files, but not wc on that system (things like that have been needed for some commands on some systems in the past).
  • maybe one wants wc (and the first cat) to run and read the whole file (and be killed at the very last minute) even if file2 can't be open for writing.
  • maybe one wants to hide the failure (exit status) of opening or reading the content of file. Though wc < file > file2 || : would make more sense.
  • maybe one wants to hide (from the output of lsof (list open files)) the fact that he's getting a word count from file or that he's storing a word count in file2.
Stéphane Chazelas
  • 544,893
17

While I don't disagree with the argument for saying it is a 'useless use of cat', there can be reasons for it:

In many languages (including English) words and sentences are read from left to right, so showing the flow of data in the same way can appear more natural to the reader.

A reason for the second cat could be to mask the return code. Such as:

$ wc < /etc/passw
sh: /etc/passw: Cannot find or open the file.
$ echo $?
1

Whereas with cat:

$ wc < /etc/passw | cat
sh: /etc/passw: Cannot find or open the file.
$ echo $?
0

This can come into play if the shell has set -e set. In the first example, this would abort the shell after wc whereas in the latter example it would continue on. Obviously there are other ways of dealing with this.

Also, the performance difference of the two statements (ie with or without cat) is negligible (esp. on today's machines) and if it was important, shell is the wrong language to use.

  • 1
    You make an interesting point, but it's hard to see when one is blinded by the enigma of wc < /etc/passw giving two completely different messages in contexts that are essentially indistinguishable. – Scott - Слава Україні Aug 25 '15 at 01:48
  • @Scott - That's a good point. It is a quirk of AIX - I'll use a different OS instead. –  Aug 25 '15 at 01:51
  • 6
    Interesting remarks. Just my note to the left to right representation of the flow of data: You can easily achieve this with redirection too: < file1 wc > file2 – pabouk - Ukraine stay strong Aug 25 '15 at 07:15
  • 3
    @pabouk - woah! i did not know you could do that. –  Aug 25 '15 at 07:17
  • 2
    What @pabouk said: redirections can appear anywhere on the command line, at least in GNU bash, so "reading left to right" is not a valid argument for using cat instead of a redirection. – user Aug 25 '15 at 08:12
  • @MichaelKjörling starting the command with '<' looks like you are redirecting to the shell itself. –  Aug 25 '15 at 08:24
  • @DarkHeart Which is then.. "auto-piped" to wc :p Thanks for an answer on "the other side", leaving larsk's accepted as for me - and it seems generally - cat is not necessary here. But yours is a useful reference for behaviour with it, or scripting where it can easily be replaced by something else. (In particular I can see it being useful for say a "variable command" - might want to default, or change it to "nothing" without modifying the rest of the script.) – OJFord Aug 25 '15 at 09:40
  • 6
    This is... not a great idea in practice. If you want to swallow errors, || true is a lot more idiomatic and obvious than | cat. – Kevin Aug 25 '15 at 16:51
  • This also assumes set +o pipefail, which is POSIX, but a common bash recommendation (and good idea IMO) is to set it, i.e. false | true (note pipe not ||) should fail. – OJFord Mar 30 '22 at 09:29
10

Let's suppose prog forks a new subprocess and exits, and the new subprocess writes something to its standard output and then exits.

Then the command

prog

won't wait for the subprocess to exit, and it will display the shell prompt early. But the command

prog | cat

will wait for an EOF on the stdin of cat, which effectively waits for the subprocess to exit. So this is a useful use of cat.

pts
  • 1,061
  • 14
  • 23
  • 1
    Valid point, at least for bash. To test it one may run ( (while ((i<10)); do echo $((i++)); sleep 1; done) & exit ; ) | cat with and without final cat. – jimmij Aug 25 '15 at 19:04
  • On the other hand, with shells like the Bourne shell, AT&T ksh or yash, prog | cat could return before prog has returned (In those shells, it will return as soon as cat returns, which will happen as soon as prog (and its children if any) has closed all its fds to the pipe)). Try for instance with prog being sh -c 'echo A; exec >&-; sleep 2; echo >&2 B'. – Stéphane Chazelas Aug 26 '15 at 16:00
-1

The statement contains two uses of cat.

cat file | wc | cat > file2

Clearly the 2nd cat is of no value, as

cat file | wc > file2

has the same meaning in all shells I have ever used.

However

< file wc > file2

does not work in all shells.

Not everyone is using a modem shell on a modem version of unix. (It can be off benefit to write pipeline in a way that work on all systems that have the commands in the pipe installed, even if some of these commons don't ship as standard with the given OS.)

Ian Ringrose
  • 107
  • 1
    < file wc > file2 is Bourne and POSIX and also works in (t)csh, rc and es. The only shell I could find that doesn't support it is fish (which is the most modern of them all). It even worked in the pre-Bourne sh of Unix V1 in 1970! – Stéphane Chazelas Aug 26 '15 at 15:08
  • @StéphaneChazelas, you are assuming that scripts only have to work on "unix"! – Ian Ringrose Aug 26 '15 at 15:43
  • 1
    As far as this unix.stackexchange.com Q&A site is concerned, that's Unix-like systems yes (though other POSIX systems are also covered). And as far as this question is concerned, you'd expect a system where cat (a typical Unix command) is available. – Stéphane Chazelas Aug 26 '15 at 15:52