Environment variable expansion inside $(command substitution)

Question

I'm running Bash 5.1.4 on Debian.

I'm writing a post-installation script to copy configuration and other files to locations in my home directory. I add the intended destination to each file at the beginning with a prefix; for example: # DEST: $HOME/.config/mousepad/Thunar (of course, in the script the file name will be substituted by a variable, and the hash symbol by the appropiate comment character; this line appears within the first 10 lines, not necessarily at the first, so I don't mess with shebangs).

To get these locations I'm using this command: head Thunar.acs | egrep "DEST:" | awk '{print $3}, which returns literally $HOME/.config/Thunar; I'd like it to expand $HOME. What I mean is when I try ls $(head Thunar.acs | egrep "DEST:" | awk '{print $2}) I get the error ls: cannot access '$HOME/.config/Thunar/': No such file or directory. I read this question and tried all of the combination of double quotes in the selected answer, but I still got the error. How can I solve this?

Enclosing the variable name in braces doesn't work either.

Thanks!

Answer 1

Command substitutions and variable expansions aren't recursive. That is, when the shell expands something like $varname or $(somecommand), it doesn't then scan the result for additional instances of $varname or $(somecommand) to expand. The only thing it does to the result is try to split it into "words" and expand anything that looks like a filename wildcard; these cause more trouble than anything else, so you should almost always double-quote the expansion so it just gets left alone.

If you do want $HOME expanded to the actual home path, the simplest ways I see are to pipe it through envsubst (which will expand anything that looks like an environment variable reference, but note that it only works with environment variables, not unexported shell variables), or have awk do the substitution (which'll only do the specific variable(s) you tell it to replace).

Before I show how to do this, I have to object to using three tools -- head, egrep, and awk -- when awk is perfectly capable of doing the entire job itself:

awk '/DEST:/ {print $3}; (NR>=10) {exit}' Thunar.acs

The /DEST:/ bit does the job egrep did in the original, and (NR>=10) {exit} replaces head -n 10.

So, here's a version using envsubst:

ls "$(awk '/DEST:/ {print $3}; (NR>=10) {exit}' Thunar.acs | envsubst)"

(Note the double-quotes to keep word-splitting and wildcard expansion from doing anything weird.)

Here's a version using awk's sub function to do the replacement:

ls "$(awk -v home="$HOME" '/DEST:/ {sub("\\$HOME",home,$3); print $3}; (NR>=10) {exit}' Thunar.acs)"