1

for example

# this line should output ssh banner
nc localhost 22
# this line should output the same string expect the replacements
nc localhost 22|sed 's/SSH/XXX/g'

nc localhost 22 > foo # after killing the process, I can see the banner in file foo
nc localhost 22|sed 's/SSH/XXX/g' > foo # however I see nothing when sed is used

as mentioned, if I use sed to pipe the nc's output, the sed's output can't be redirected to a file.

And I also tried other commands to pipe the result, like grep, tr, the same result.

However, cat can pipe the result to a file. strange..

nc localhost 22|cat > foo

So why this happen? what is the difference between cat and other commands in this situation? how can I make sed's output redirect to file like cat ?

chrisyue
  • 205

1 Answers1

3
$ cat | sed -e 's/foo/bar/' > output.txt 
foo
^C
$ cat output.txt 
[nothing]

but:

$ cat | sed -e 's/foo/bar/' > output.txt 
foo
^D
$ cat output.txt 
bar

Writes to files --but not to the terminal-- are usually buffered by the C library (the stdio functions) to blocks of some kilobytes. If you hit Ctrl-C to kill the pipeline, sed won't have time to write out the buffers before it dies, so the resulting output file is empty. (You could say it's a bug in the C library that it doesn't catch the termination signal for that, but so it goes.)

Plain cat probably doesn't use stdio, since it doesn't deal with anything complicated like lines. Something like cat -n might be different, or not.

You could arrange so that you don't need to terminate the pipe with Ctrl-C, but have it complete by itself. At least in my nc, the -qN option tells it to exit after N seconds after stdin is closed, so e.g.:

$ printf '' | nc -q0 localhost 22 | sed 's/SSH/FOO/' > output.txt 
$ cat output.txt 
FOO-2.0-OpenSSH_7.4p1 Debian-10+deb9u7

Alternatively, there's at least a few ways to turn of buffering, see e.g. Turn off buffering in pipe and `unbuffer` or `stdbuf` for removing stdout buffering?

Also GNU sed has the -u/ --unbuffered switch to

load minimal amounts of data from the input files and flush the output buffers more often

ilkkachu
  • 138,973