I am working on a process to send data via a pipe from one server to another for processing.
Although this is not the exact command, it might look something like this:
tail -f logfile | grep "abc" | grep "def" | grep -v "ghi" | netcat -q 0 n.n.n.n 7777
I would like to wrap all those greps into a script and more importantly prepend the pipe to netcat with an identifier, so the command would look like this:
tail -f logfile | myscript.sh {id}
The script listening on the other end should receive:
{id}
[Line 1 of the logfile]
[Line 2 of the logfile]
...
Wrapping it in a script is easy:
#!/bin/sh
id=$1
grep "abc" | grep "def" | grep -v "ghi" | netcat -q 0 n.n.n.n 7777
but I cannot figure out how to inject $id at the start.
The receiving end is using
socat -u tcp-l:7777,fork system:/dev/receivePipe
so if there is a different way I could get the id (for example somehow as a parameter to /dev/receivePipe), or via an environment variable, that would work too.
EDIT: The final answer was figured out in the comments of the accepted answer:
#!/bin/sh
{
printf '%s\n' $1
grep "abc" | grep "def" | grep -v "ghi"
} | netcat -q 0 192.168.56.105 7777
socat. Did you mean to pipe tonetcat, or is something else going on? Regardless, I don't want to convert the pipes to other things such asawk. Thegreps were just an example for this post. In reality, it might be other intermediary programs that it's getting piped through, with nogrepsat all. All I want to do is inject a line at the start of what ends up going tonetcat. – Ben Holness Aug 01 '22 at 12:39socat, I replaced yournetcatcall with thesocatequivalent. In any case, the first approach will allow you to insert the header before any command. – Stéphane Chazelas Aug 01 '22 at 13:48netcaton the sending server assocatis not installed there. I'm trying to understand how to use the first approach without awk and with the other pipes. I tried removing theawkline entirely (didn't work) and taking out all of the/abc/parts in theawkline (also didn't work). Here's an example of what I tried (the one withoutawk), with different, perhaps non-sensical, intermediary programs (and squashed because multi-line comments aren't possible):
– Ben Holness Aug 01 '22 at 13:55#!/bin/sh { printf '%s\n' $1 } | grep "abc" | head -n 10 | netcat -q 0 192.168.56.105 7777awk '/abc/ && /def/ && ! /ghi/'(which is the same asgrep abc | grep def | grep -v ghi) with your filtering command (likegrep abc | headorgrep -m10 abc) – Stéphane Chazelas Aug 01 '22 at 14:10#!/bin/sh { printf '%s\n' $1 grep "e" | head -n 3 } | netcat -q 0 192.168.56.105 7777Thanks!
– Ben Holness Aug 01 '22 at 14:15