5

I am aware that STDOUT is usually buffered by commands like mawk (but not gawk), grep, sed, and so on, unless used with the appropriate options (i.e. mawk --Winteractive, or grep --line-buffered, or sed --unbuffered). But the buffering doesn't happen when STDOUT is a terminal/tty, in which case it is line buffered.

Now, what I don't get is why STDOUT is buffered outside of a loop send to a pipe, even though the final destination is the terminal.

A basic example :

$ while sleep 3; do echo -n "Current Time is ";date +%T; done | mawk '{print $NF}'
^C

Nothing happens for a long time, because mawk seems to be buffering it's output.

I wasn't expecting that. mawk's output is the terminal, so why is its STDOUT buffered ?

Indeed, with the -Winteractive option the output is rendering every 3 seconds :

$ while sleep 3; do echo -n "Current Time is ";date +%T; done | mawk -Winteractive '{print $NF}'
10:57:05
10:57:08
10:57:11
^C

Now, this behavior is clearly mawk related, because it isn't reproduced if I use for example grep. Even without its --line-buffered option, grep doesn't buffer its STDOUT, which is the expected behavior given that grep's STDOUT is the terminal :

$ while sleep 3; do echo -n "Current Time is ";date +%T; done | grep Current
Current Time is 11:01:44
Current Time is 11:01:47
Current Time is 11:01:50
^C
ChennyStar
  • 1,743
  • awk (= gawk) will show time at "3": apt install gawk and do $ while sleep 3; do echo -n "Current Time is ";date +%T; done | awk '{print $NF}' .... Info https://superuser.com/questions/75875/awk-mawk-nawk-gawk-what – Knud Larsen Jun 25 '21 at 12:10
  • I know that, gawk never buffers (I said it in my first paragraph). But that's beside the point, I'm trying to understand why mawk is buffering even when STDOUT is tty (terminal). Or, in other words, why my | mawk example behaves differently from my | grep example in terms of buffering. – ChennyStar Jun 25 '21 at 12:13

1 Answers1

13

It's not that it's buffering its output.

mawk is the only utility that I know that buffers its input.

See also https://github.com/ThomasDickey/original-mawk/issues/41#issuecomment-241070898

In other words, mawk will not start processing its input (let alone print anything if that processing involves printing) until it has accumulated a buffer full of input.

You can verify it by running:

(echo 1; sleep 1; echo 2) | mawk '{system("echo "$1)}'

It can be disabled with the -Winteractive option. Note that with -Winteractive, records are lines regardless of the value of RS.

Stéphane Chazelas
  • 544,893
  • Thanks ! I scratched my head on that one... And yes, the answer is in the manpages : -W interactive sets unbuffered writes to stdout and line buffered reads from stdin. – ChennyStar Jun 25 '21 at 13:06
  • Additional info : stdbuf -i0 doesn't seem to be working with mawk (not that it's important, given that mawk has the -Winteractive option). I discovered it after digging for some more infos on STDIN buffering (and incidentally finding another of your answers, https://unix.stackexchange.com/questions/411263/stdin-unbuffered#answer-411330 - it seems your are the stdio guru on this forum ;-) – ChennyStar Jun 25 '21 at 14:53