Only cat from specific line X (with a pattern) to other specific line Y (with a pattern)

Question

A little extended problem from "cat line x to line y on a huge file":

I have a huge file (2-3 GB). I'd like to cat/print only from the line having "foo:" to the line having "goo:". Assume that "foo:" and "goo:" only appear once in a file; "foo:" proceeds "goo:".

So far this is my approach:

• First, find the line with "foo:" and "goo:": grep -nr "foo:" bigfile
• Returns 123456: foo: hello world! and 654321: goo: good bye!
• Once I know these starting and ending line numbers, and the difference (654321-123456=530865) I can do selective cat:
• tail -n+123456 bigfile | head -n 530865

My question is that how can I effectively substitute the line number constants with expressions (e.g., grep ...)?

I can write a simple Python script but want to achieve it using only combining commands.

Answer 1

sed -n '/foo/,/goo/p;/goo/q' <bigfile

That would print only those lines. If you wanted the line numbers you'd add an =.

sed -n '/foo/=;/goo/=;//q' <bigfile

The q is important because it quits the input when it is called - else sed will continue to read the infile through to the end.

If you don't want to print foo/goo lines you can do instead:

With GNU sed:

sed -n '/foo/,/goo/!d;//!p;/goo/q
' <<\DATA
line1
foo 
line3
line4
line5
goo 
line7
DATA

OUTPUT

line3
line4
line5

And with any other:

sed -n '/foo/G;/\n/,/goo/!d;//q;/\n/!p 
' <<\DATA
line1
foo 
line3
line4
line5
goo 
line7
DATA    

OUTPUT

line3
line4
line5

Either way, though, this also quits its input as soon as it encounters the last line in your search.

Answer 2

If you are okay with abandoning your current approach of using something in subshells to get the line numbers and allowing another utility to print the file, this can be accomplished in pure awk with little difficulty:

If you wish to print the lines between foo: and goo: and not the lines themselves, then you can use the following (picked up from here originally):

awk '/goo:/ { exit }; flag; /foo:/ { flag = 1 }' bigFile

The above exits when it sees the end token (goo:), prints if flag is true, and sets flag to true (1, actually) when it reaches the opening token (foo:).

If, however, you would prefer to include the token lines in the output, the command is actually even simpler, as @jasonwryan mentioned:

awk '/foo:/,/goo:/' bigFile

---

If you are hell-bent on only getting the line numbers and not actually printing the file with the same utility, then you can get the line numbers of the start and end tokens like so:

awk '/foo:|goo:/ { print NR }' bigFile

Answer 3

Alternative sed one:

sed '/foo/,$!d;/goo/q'

Answer 4

To substitute the constants with expressions, you can use command substitution.

To substitute the output of a command into an expression, use $(command)

In this case, the appropriate command line is:

tail -n+$(grep -nr "foo:" bigfile | cut -d':' -f1) bigfile | \
head -n$(($(grep -nr "goo:" bigfile | cut -d':' -f1)-$(grep -nr "foo:" bigfile | cut -d':' -f1)+1))

This will print all lines from the line containing foo: to the line containing goo:, inclusive.