Copying files based on input list, special characters in file names

Question

This is based on the question here.

Problem:

• I have a .txt file listing file names I want to copy to the folder newfolder in the shell (Mac Terminal).

Approach, where filelist.txt contains file names (separated by \n), that I want to copy:

# version 1
for file in $(cat filelist.txt); do cp "$file" newfolder; done

Errors:

If I have file names in filelist.txt that contain dashes, whitespace, etc, the names are split up and I correctly get a No such file error.

Here is how I try to address it (adding dbl quotes to the variable):

# version 2
for file in "$(cat filelist.txt)"; do cp "$file" newfolder; done

But this prints out all the file names (no splitting on whitespace) and does not copy anything.

Questions:

• Adding quotes as above addresses the name splitting issue e.g. when I feed it to echo; why does it not work for cp?

• What is the right way of doing this with cat and cp? A comment in the answer linked above suggests the following:

  set -f;IFS=$'\n'

which fixes things but I have not idea what it does.

Any hints are much appreciated!

Answer 1

There are a number of options; this is one:

cat filelist.txt | while read file; do
    cp "$file" newfolder
done

So, first why does

for file in $(cat filelist.txt); do cp "$file" newfolder; done

not work? Assume, that your filelist contains

a
b c

The $(cat filelist.txt) produces a list of filenames. The for-loop sees the output of cat and interprets it as:

for file in a b c ; do

which is clearly not the intention.

This is where the remark about the Input Field Separator helps. Normally, the IFS is any white-space and therefore a space in the filename will be interpreted as a separation between filenames. If you set IFS explicitly to a newline, space is no longer seen as a separator.

for file in "$(cat filelist.txt)"; do cp "$file" newfolder; done

creates a single argument for the for-loop. The result is equivalent of:

for file in "a b c" ; do

and therefore, the loop gets executed only once, with a filename that does not exist.