Order of commands executed and redirection

Question

For a bash script I'm making, I need to delete all but the newest 2 files in a directory.

I decided to use ls -tUr > somefile.txt and then just tail the newest 2, move them somewhere else, rm *, and move them back.

For some reason, somefile.txt actually shows up (among the rest of the files) inside somefile.txt which screws up using the tail command.

So my question is, logically ls would return the current folder/files and then be redirected into somefile.txt, but clearly this isn't happening since somefile.txt must exist before ls runs; why is this?

Note for those that may be surprised to see a -U where that option means unsorted in GNU ls, the OP is probably using a BSD system where -U means to use the creation time instead of last modification time. — Stéphane Chazelas, May 16 '17 at 13:48
See Warning regarding ‘>’ in the answers of the above question. — muru, May 17 '17 at 07:08
Also duplicate but from another site: https://askubuntu.com/q/728382/295286 Basically the answer is that redirections are always done first. — Sergiy Kolodyazhnyy, May 17 '17 at 08:41
That's very nice, but what I'm working on isn't for Ubuntu, it is a generalized question about Linux Operating Systems concerning redirection, so actually, if you want to remove one, then remove that person's because they clearly posted it on the wrong site. — demcooltricks, May 18 '17 at 21:58

Kusalananda · Accepted Answer · 2017-05-17T06:45:05.180

This answers the question "why does this happen?"

Yes, somefile.txt will actually be created before ls is run.

$ utility >file

The first thing that happens is that the shell notices the redirection, creates file (or truncates it if it already exists), then it executes utility with its standard output stream going into file.

The utility is not generally aware of or concerned with where its standard output is going (some, like ls, do check to see whether it's a TTY or not and changes their behaviour accordingly), so it doesn't matter whether it's writing to a regular file, pipe, device file or socket. It's certainly not concerned with creating this file. It is therefore the job of the shell to make sure that the plumbing is in place before the process is started, which includes creating or truncating the file named file in the example above.

For answers dealing with your issue regarding deleting all but the newest files, see the question "remove oldest files"

score 2 · Answer 2 · answered May 16 '17 at 14:36

While Kusalananda has answered your question about why somefile.txt appears in the output of ls, I'll try and address the question about how to delete all but the two most recently created files as the linked question deals with most recently modified files instead.

First, you don't have to write the output in a file. Commands can interact with each other directly, that's what pipes are for.

Here, you could do:

ls -tU | tail -n +3 | xargs echo rm -f -- # echo for dry-run

But that's flawed in the general case as tail works on lines and xargs works on (possibly quoted) words, and file names are not guaranteed to be either of those. They're not even guaranteed to be valid text. It would only work properly if none of the file names contain blanks, newlines, quotes, backslashes or byte sequences not forming valid characters.

If the system is FreeBSD (as your usage of -U suggests), you could use the json output format that is reliably parsable like:

ls --libxo=json -Ut |
  perl -MJSON::PP -l0 -0777 -ne '
    @files = map {$_->{name}} @{
      decode_json($_)->{"file-information"}->{directory}->[0]->{entry}
    };
    print for @files[2..$#files]' | xargs -0 echo rm -f --

Order of commands executed and redirection

2 Answers2

Linked