Use shell's "*" glob, but exclude one file and don't match directories?

Question

I've got a makefile rule that builds a zip/tarbar for distribution. In the recipe, it does some "value added" things, like ensure CR/LF's are correct, and ensures execute bits are correct before packaging.

The project has a buffet of files, but here are the requirements: (1) all files except GNUmakefile need CR/LF, (3) GNUmakefile needs LF only, (3) all files except GNUmakefile needs a-x.

Here's what the recipe looks like:

.PHONY: convert
convert:
    chmod a-x *[^GNUmakefile*] TestData/*.dat TestVectors/*.txt
    unix2dos --keepdate --quiet *[^GNUmakefile*] TestData/*.dat TestVectors/*.txt
    dos2unix --keepdate --quiet GNUmakefile

I'm using * and trying to avoid explicitly listing all the files in the buffet because some are non obvious, like IDE specific files. (*[^<somefile>*] is a neat trick; I got that from Exclude one pattern from glob match).

The problem is I'm matching TestData and TestVectors when performing chmod a-x, so I exclude myself from the directories.

I need to refine things, but I'm not sure how. I want to use the shell's "*" glob, but exclude one file and don't match directories.

How should I proceed?

Answer 1

I'd solve this problem by using GNU Make's filter-out and wildcard functions. The only part of your task that you can't do with them is filter out directories; that has to be done via the shell. Code below is untested and assumes (a) no whitespace or shell metacharacters in any filename, (b) TestData/*.dat and TestVectors/*.txt do not need to be checked for directories.

NORM_TOPLEVEL := $(shell for f in $(filter-out GNUMakefile,$(wildcard *)); \
                   do [ -d "$$f" ] || printf '%s\n' "$$f"; done)
NORM_TESTDIRS := $(wildcard TestData/*.dat) $(wildcard TestVectors/*.txt)

convert:
    chmod a-x $(NORM_TOPLEVEL) $(NORM_TESTDIRS)
    unix2dos --keepdate --quiet $(NORM_TOPLEVEL) $(NORM_TESTDIRS)
    dos2unix --keepdate --quiet GNUmakefile

.PHONY: convert

Answer 2

Only zsh has globs where you can select files by type, so, assuming GNU make, you'd need something like:

SHELL = zsh
.SHELLFLAGS = -o extendedglob -c

test:
        echo ^GNUmakefile(^/)

^GNUmakefile (with extendedglob) is for non-hidden files other than GNUmakefile. (^/) is a glob qualifier that selects files of any type other than directory. See also (.) for files of type regular (excludes directories and all other non-regular types of files like fifos, symlinks, sockets...) which seems more like what you're looking for. Add the D glob qualifier (^GNUmakefile(.D)) to include hidden (Dot) files like .gitignore.

Note that *[^GNUmakefile*] expands to the list of non-hidden file names that end in a character other than G, N, U, m, a, k, e, f, i, l, or *. So it will indeed exclude GNUmakefile (since it ends in e), but also foo.a or file.html or bar.exe.

To do the same without changing the shell, you'd need to resort to a loop like (here for the equivalent of ^GNUmakefile(.)):

test:
        set -- *; \
        for i do \
          [ -f "$$i" ] && \
            [ ! -L "$$i" ] && \
            [ "$$i" != GNUmakefile ] && \
            set -- "$$@" "$$i"; \
          shift; \
        done; \
        [ "$$#" -gt 0 ] && echo "$$@"

(replace set -- * with set -- .* * to include hidden files).

Best there would probably be to resort to find instead of shell globs if you can't guarantee availability of zsh:

test:
        find . ! -name . -prune ! -name '.*' ! -name GNUmakefile \
          -type f -exec echo {} +
        find TestData/. ! -name . -prune -name '*.dat' ! -name '.*' \
          -type f -exec echo {} +

(remove ! -name '.*' to include hidden files).

Answer 3

Using Python makes it easy to expand the expression if you wish to filter for more advanced criteria. Python is installed almost everywhere. It's a bit verbose, but it works.

Example Makefile:

all:
    chmod a-x $(shell python -c "import glob; import os.path; print(' '.join([x for x in glob.glob('TestData/*.dat') + glob.glob('TestVectors/*.txt') if not (os.path.isdir(x) or x == 'GNUmakefile')]))")