0

There're some files, folders and dot files in a directory like following

|-folder1
|----folder2
|----file.json
|----file2.php
|----.env
|----.dockerfile
|----.someconfig
|----.dockerignore

I run command mv $(ls | egrep -v '(.dockerfile|.env)') ../new_directory to move all files in directory folder1 to the new_directory exclude .dockerfile and .env, but actually all dot files can't be moved, why?

MC68020
  • 7,981
Bill
  • 1

2 Answers2

3

If you run

ls | egrep -v '(.dockerfile|.env)'

on its own, you’ll get an idea of what’s happening here: the output will be

file2.php
file.json
folder2

This happens because ls doesn’t list dotfiles by default. To include dotfiles, but not . or .., use ls -A:

$ ls -A | egrep -v '(.dockerfile|.env)'
.dockerignore
file2.php
file.json
folder2
.someconfig

You don’t need to use ls here (and you shouldn’t), you can use Bash’s extended pattern matching:

shopt -s extglob dotglob
mv !(.dockerfile|.env|.|..) ../new_directory

This will match all files not named .dockerfile, .env, . or ... Using this approach, you won’t have any issues with file names containing spaces, newlines etc.

shopt -s extglob enabled extended pattern matching, and shopt -s dotglob allows globs to match dot files.

Stephen Kitt
  • 434,908
  • Thank you, I run mv !(.dockerfile|.env|.|..) ../new_directory, but I got the error no matches found: !(.dockerfile|.env|.|..). – Bill Jun 30 '22 at 00:08
  • Ah, I forgot to mention that extglob needs to be enabled, see the update. – Stephen Kitt Jun 30 '22 at 05:57
  • I'd expect !(.dockerfile|.env|.|..) expansion not to include hidden files unless you set the dotglob option though IIRC it was different in older versions of bash. What version of bash are you using? – Stéphane Chazelas Jun 30 '22 at 06:11
  • Looks like it went back and forth, 4.2 didn't include dotfiles, 4.3 and 4.4 did, 5.0 don't any longer again. All include . and .. with dotglob (unless excluded manually like you do here) which I'd consider a bug. – Stéphane Chazelas Jun 30 '22 at 06:15
  • Sorry, not all. 4.2 and 3.2 didn't include . and .. with dotglob and !(.foo) for instance! bash-5.2 has a globskipdots option to always exclude them like pdksh / zsh / fish do. – Stéphane Chazelas Jun 30 '22 at 06:35
  • I tested this on bash 4.4.20, with extglob on and dotglob off. – Stephen Kitt Jun 30 '22 at 07:15
1

You shouldn't be using ls for this. Parsing the output of ls is unreliable at best, dangerous at worst. See Why not parse ls (and what to do instead)?

Try find instead. e.g. with GNU find (and GNU cp):

find ./ -maxdepth 1 -regextype awk \
  ! -regex './(|\.env|\.dockerfile)' \
  -exec echo cp -t ../new_directory/ {} +

I use -regextype awk because I'm very used to the regular expression dialect in awk and I don't want to waste time figuring out the exact dialect used in find-default. -regextype egrep would have worked just as well. You can get a list of dialects available by running find -regextype help (unfortunately, PCRE isn't one of them).

The -regex ... excludes ./, ./env, and ./dockerfile. Note that neither ^ nor $ anchors are required, they're implicit with find's -regex predicate, so you only get exact matches. If you want partial matches, you have to add .* to the beginning and/or end of your regular expression. This is why I had to include the ./ in the regex.

If I were to run this from the parent, directory, it would have to be written as:

find folder1/ -maxdepth 1 -regextype awk \
  ! -regex 'folder1/(|\.env|\.dockerfile)' \
  -exec echo cp -t ../newfolder {} +

or with some other pattern that matched the path, e.g. '.*/(|\.env|\.dockerfile)'.

The echo makes this a dry-run, to show you what would be copied without actually copying them. Remove it when you've confirmed the command will do what you want.

GNU cp's -t option allows you to specify the target directory before the list of files to be copied.

cas
  • 78,579