Redirect stderr to file and prepend every lines with a string

Question

I want to redirect a command stderr to a file while prepending them with a string.

exec >&2 "$log_file"
command

The expected log file:

2021-07-13 09:34:03+07:00 [stderr] error lines ...
...

The timestamp is just an example. I might use something else for my other scripts.

I was thinking to use this:

command | xargs -n1 printf '[%s] %s\n' "`date --rfc-3339=s`" 2>> "$log_file"

But this will override the exit status of the piped command. Plus this is very slow.

I remembered that in bash (or some other shells), I can use exec 2>> "$log_file". But then I have no clue on how to use exec to prepend the command output.

As far as I understand it, the title corresponds to the question body. However, is the timestamp just an example or do you really need a timestamp? If so, change the title to "insert a timestamp at the beginning of each line". — berndbausch, Jul 13 '21 at 04:19
Does this answer your question? How can I redirect all output of a script to a file and replace passwords?. Ignore that it's asking about passwords, focus on the exec redirections and using a fifo to modify the output with sed before writing to a file (also, you'll want the exec to apply to stderr, not stdout, so use exec >&2) — cas, Jul 13 '21 at 05:35
Well, it does answer my question. But using what kind of keyword that I could find that question ;) It's too specific, I think. — annahri, Jul 13 '21 at 06:48
Finding info on obscure topics that hardly anyone ever needs to do can be difficult. "exec" and "redirect" and either "sed" or "modify" would be good starting points for searches. — cas, Jul 13 '21 at 06:55
If you want a more generic solution, you should first provide a working specific approach. — Bananguin, Jul 13 '21 at 08:26

cas · Accepted Answer · 2021-07-14T05:21:35.737

In bash, it's fairly simple:

exec 2> >(perl -MDate::Format -pe 's/^/time2str("%Y-%m-%d %H:%M:%S%z [stderr] ",time)/e' > /var/log/some.log)

You can turn that perl one-liner into a standalone script:

#!/usr/bin/perl
use Date::Format;
while(<>) {
  s/^/time2str("%Y-%m-%d %H:%M:%S%z [stderr] ",time)/e;
  print;
}

Save that as, say, /usr/local/bin/timestamp-text and make it executable with chmod +x /usr/local/bin/timestamp-text. Then you can write the exec as:

exec 2> >(/usr/local/bin/timestamp-text > /var/log/some.log)

In other Bourne-like shells, you can't use exec to redirect to a process substitution, you have to use a fifo. I don't want to rewrite the same answer again, so I'll just point you to another answer I wrote for a similar question: https://unix.stackexchange.com/a/644862/7696

BTW, the timestamp-text script (or the one-liner version) can be written in whatever language you like, as long as it has decent date formatting capabilities (most languages do). It will need to query the current time for every input line it modifies, not just at the time it was first run (otherwise, you could probably just use /bin/date)

NOTE: the Date::Format module is in the libtimedate-perl package on debian. This package also contains the Date::Parse, Date::Language, and Time::Zone modules. It should be available as a package for most other distros too, otherwise install it with cpan.

Update

I just remembered that a program called ts exists to do exactly this kind of time-stamping, so there's no need to write your own perl or whatever script. It's available in the moreutils package.

exec 2> >(ts "%Y-%m-%d %H:%M:%S%z [stderr]" > /var/log/some.log)

It is, however, a perl script itself and requires the Getopt::Long module to be installed, and (depending on which command line options you use) may also require Date::Parse, Time::Duration, and/or Time::HiRes. Time::Hires is a core perl library module and should be included with perl.

Redirect stderr to file and prepend every lines with a string

1 Answers1

Update