Pass each find result as an argument with prefix to exec

Question

I would like to find some files and turn them into a chain of arguments for a command. What I have so far is:

find . -name '*.yml' -exec cmd -f {} \+

which results in

cmd -f ./foo.yml ./bar.yml

but what I really like to have is

cmd -f ./foo.yml -f ./bar.yml

(note the 2nd -f). How can I achive this?

Answer 1

Solution

find cannot do this directly. You need an inner shell to rebuild the array of arguments:

find . -name '*.yml' -exec sh -c '
   marker=x
   for f do
      [ "$marker" ] && { set --; marker=''; }
      set -- "$@" -f "$f"
   done
   exec cmd "$@"
' find-sh {} +

---

Possible problem

I think in some circumstances the above code may fail. The problem originates from the fact a command cannot be arbitrarily long (1, 2). find … -exec foo … {} + should be smart enough not to reach the limit, it should run foo multiple times if needed (xargs should be similarly smart). Our foo is sh, then there is -c, then our shell code, then find-sh and then possibly multiple pathnames.

Our ultimate cmd …, in comparison, is stripped off -c, the shell code and find-sh, this shortens the command; but there is an additional -f per pathname, this extends the command. If the extending "wins" then the command may turn out to be too long.

To mitigate this you could inject lines of spaces to the shell code, so the code takes more bytes, so it compensates for more -fs added later. Without knowing the exact limit it's rather impossible to create a truly robust code. IMO injecting spaces is not a good way.

A better way is to (ab)use the starting point given to find (. in our case). I'm not a programmer and I don't know how much space an additional -f really takes. It's two characters and probably some kind of terminator/separator, I guess no more than four characters total. Therefore I would do this:

find .///. -name '*.yml' -exec sh -c '
   marker=x
   for f do
      [ "$marker" ] && { set --; marker=''; }
      set -- "$@" -f "${f#.///}"
   done
   exec cmd "$@"
' find-sh {} +

Now every pathname should start with .///. instead of . (and still be equivalent to ./. which is equivalent to .). Later in the shell code we remove these additional four characters from the beginning of every pathname ("${f#.///}") to compensate for adding one -f per pathname. This way if the command built by find -exec is within the limit then our cmd … command will also be within the limit.

Again, I'm not a programmer and I'm not really sure if four characters are enough (or if it depends on the architecture etc.)*. Nevertheless I believe this approach may be useful.

Note, however, that using .///. instead of . may in general require you to adjust some tests (e.g. -path or -regex of GNU find). -name '*.yml' you used does not need adjustments.

---

* After publishing the answer, I received a comment claiming that I need to account for at least 11 bytes per -f:

An additional -f argument will add those 3 bytes of the "-f" aka ['-', 'f', '\0'] string, but also an extra pointer (so typically 8 bytes these days), which on most systems is accounted for against the execve() limit, so that would be at least 11 bytes

It's beyond my expertise, I cannot tell if it's true; at the same time I totally assume the author of the comment tries to help, not to deceive us. So yeah, most likely you need more slashes.