1

So here is my requirement.

I am tailing a log file and grepping on it.
I want to get some context on every grep result..
But the context should be "till a pattern is matched" and not the number of lines (which is what grep -A/-B/-C offer).

For example
Say here is my log..

[l] is prefixed to every log line. Also there will be prefix of [logTypeA] or [logTypeB]

[l][logTypeA] - Log line 1
[l][logTypeB] - Log line 2 
[l][logTypeA] - Log line 3 
.... 

Random data about Log line 3
....

[l][logTypeB] - Log line 4

Now my if my tail command was tail -f log_file.log | grep "[logTypeA]", I'd get an output of 

[l][logTypeA] - Log line 1
[l][logTypeA] - Log line 3

But I need contextual information for my grep result, and that context is NOT some number of lines, but rather till a particular pattern is matched (in this case [l])..

In the example I want my grep result to be

[l][logTypeA] - Log line 1
[l][logTypeA] - Log line 3 
.... 

Random data about Log line 3
....

From here (How to show lines after each grep match until other specific match?), I tried sed command on my tail like 

tail -f log_file.log | sed '/\[logTypeA\]/,/\[l\]/p'

But that doesn't seem to work.

Any ideas?

SatheeshJM
  • 111
  • 1
    tail -f gives buffered output and can not play with "past" and "future" range – RomanPerekhrest Jun 17 '17 at 14:53
  • 1
    In broad terms, write a simple script (I suggest in Awk) which groups related lines into "records" and then prints those records which match a particular pattern. Your question is too generic for it to make sense to show you a working script, but this is an extremely common task with Awk. Spend 30 minutes on a basic tutorial if you are unfamiliar with it, and you should be all set to solve the problem yourself. – tripleee Jun 17 '17 at 17:04
  • @RomanPerekhrest what about tail -f with grep -A, which gives me 10 lines after.. How is my requirement different from this? – SatheeshJM Jun 17 '17 at 20:00
  • @tripleee Writing an awk script was my last resort.. Was just wondering if there was some existing command. – SatheeshJM Jun 17 '17 at 20:02
  • Your requirements are unclear. Your sed script looks like you want to print lines with [l] and all lines between them ... but without any further conditions, that's simply equivalent to all lines, to my understanding. I guess you want all [l] lines which match a particular additional pattern and all non-[l] lines after a match, but without further details, there are too many variables and guesses to write a useful script. It could probably done in sed instead if that's your preference, though Awk seems particularly suitable here. – tripleee Jun 18 '17 at 12:58
  • @tripleee Sorry I was not clear in the question.. Have updated my question with more details.. – SatheeshJM Jun 18 '17 at 19:53
  • Your grep "[logTypeA]" needs to escape the [ or use grep -F. Currently it will print any line which contains l or o or g or T etc. – tripleee Jun 19 '17 at 02:45

2 Answers2

1

This is an extremely common pattern with Awk. Collect related lines into "records", print the record when you have collected all of it if it matches a particular condition.

tail -f file |
awk '/\[l]/ { if (p && stored) print stored; stored = ""; p=0 }
    /\[logTypeA]/ { p=1 }
    { stored = stored (stored ? ORS : "") $0 }
    END { if (p) print stored }'

The END condition doesn't really make sense with a never-ending stream from tail -f but I include it for good measure, and to avoid pesky test failures when the last record you want to test with should be printed, but won't be without the END clause.

tripleee
  • 7,699
1

I would probably end up doing this with a perl one-liner. You can do it in sed but it feels like it's beginning to hit up against the "unintentional turing-completeness" part of sed.

Perl one-liner (the perl -ne is really the take away here)

echo  '[l] boring\nboring data\n[l] boring\n[l]interesting\ndata\ndata\n[l]boring' |  perl -ne '
    if (/\[l\].*interesting/ ) { print $_; $collect=1 ; }
    elsif (/\[l\]/) {$collect=0 }
    elsif ($collect) {print $_}'

For reference, here is the sed one-liner that does the same thing. (Features of sed used: b to achieve a switch, T to achieve conditional execution).

echo  '[l] boring\nboring data\n[l] boring\n[l\]interesting\ndata\ndata\n[l]boring' |  sed -nE '/\[l\].*interesting/ {
p;
s/.*/collect/ ; x ; # store collect marker in the pattern space
b; # terminate processing
}

/\[l\]/ {
s/.*// ; x; # clear hold flag
b
}

/./ {
x;
s/collect/collect/;
T; x; p # print if we are collecting
}
'
Att Righ
  • 1,196
  • The thing in the square brackets is a lowercase L, not the number 1. – tripleee Jun 19 '17 at 17:02
  • The Perl script assumes that the interesting string occurs on the same line as the [l]. I don't know if that covers the OP's use case; my Awk script copes with matches in the subsequent related lines, too. (It's not very hard to make the Perl script also support this, though.) – tripleee Jun 19 '17 at 17:05
  • @triplee good point. It makes one-liner a lot simpler though :). – Att Righ Jun 19 '17 at 21:38
  • I've fixed the 1 to be a lowercase L (though haven't confirmed that the code runs) – Att Righ Jun 19 '17 at 21:41