In bash, feed output of one command line-by-line into another command

Question

I have the following bash command which fetches and block XML from a single column in mysql and feeds it into xmllint to prettify it.

mysql --format csv --skip-column-names -e "select xmldata from mytable limit 1;" | xmllint --format -

So far so good, but it only works for one row at a time. I would like to adapt this so each line in the SQL query output is fed in to a separate call to xmlint.

I know this can be done with a loop in a bash script -- but I'd like to do it in a one-liner if possible. Supposedly this can be done with xargs but I can't figure it out.

score 6 · Accepted Answer · edited Feb 09 '12 at 00:22

6

mysql --format csv --skip-column-names \
    -e "select xmldata from mytable limit 1;" \
    | while IFS= read -r line; do xmllint --format - <<<"$line"; done

Split for easier reading, obviously you don't need to. The <<<$line is a herestring, which feeds the contents of $line to xmllint as stdin.

edited Feb 09 '12 at 00:22

Gilles 'SO- stop being evil'

829,060

answered Feb 08 '12 at 22:05

Kevin

40,767

+1 for herestring, I've always done this as while read line; do echo $line | xmllint... – Kyle Smith Feb 08 '12 at 22:18
Previously the last line of the answer was written: while read line; do xmllint --format - <<<$line; done ... that worked fine for me... why was it changed and what's the difference? – frankadelic Feb 09 '12 at 03:44
@frankadelic it's almost always best to quote a variable when you use it, so Gilles added quotes here. – Kevin Feb 09 '12 at 03:51
Right, but why was IFS= added? And what is read -r ? – frankadelic Feb 09 '12 at 03:53
@gilles before I posted, I checked and it didn't break on anything I could find without the quotes. Can you give an example that breaks an unquoted herestring? – Kevin Feb 09 '12 at 03:53
@frankadelic -r prevents interpreting backslashes (\) as escapes and prints them like any other character (which probably is what you want). IFS is the input field separator, setting it there makes sure it's not set to something unexpected, without changing it for later commands. – Kevin Feb 09 '12 at 04:12
@frankadelic -r on read is the only change that's necessary here. Setting IFS to the empty string retains initial and trailing whitespace, which is a good habit to get into, even if it doesn't matter here because xmllint doesn't care about these. The double quotes around $line aren't necessary because no shell that supports <<< performs any expansion on it; again, double quotes around variable substitutions are a good habit to get into. See also Why is while IFS= read used so often, instead of IFS=; while read..? – Gilles 'SO- stop being evil' Feb 09 '12 at 09:20

score 0 · Answer 2 · edited Apr 13 '17 at 12:36

This is possible with xargs, but xargs would pass each line as an argument to xmllint, not on its standard input. You would need to have xargs invoke a shell to feed the text to xmllint.

mysql … | xargs -L 1 sh -c 'echo "$0" | xmllint --format -'

A shell loop is the easiest method. The canonical way to loop over lines is while IFS= read -r line; do ….

mysql … |
while IFS= read -r line; do printf '%s\n' "$line" | xmllint --format -; done

In bash, feed output of one command line-by-line into another command

2 Answers2