5

It can be done with other tools, but I am interested to know how can I delete all but the last X lines of the file with sed.

Felipe Alvarez
  • 2,104

4 Answers4

11

Basically you are emulating tail. X = 20 in this example. The following example will delete all but the last 20 lines:

sed -i -e :a -e '$q;N;21,$D;ba' filename

Explanation:

  • The -e :a creates a label called a
  • The next -e:
    • $q - quits and prints the pattern space if it is the last line
    • N - next line
    • 21,$D - executes the "D" command if the line# is >= 21 (21,$ = 21st line to $ which is the end of the file)
    • ba - branches to label 'a' which is the beginning of the script.
rubo77
  • 4,920
phiz
  • 346
  • It works as tail, but I was not able to make it work in place (as the in the original question) – Matteo Sep 07 '12 at 06:48
  • 2
    +1... BTW. It works with -i if you condense the two -e expressions into one. You can also use it by passing bash variable $1 into it... sed -i ':a;$q;N;'$(($1+1))',$D;ba' filename ... but asking for 0 lines, will return 1 line. – Peter.O Sep 07 '12 at 07:42
  • @Peter.O That just blew my mind. Could you please explain this bit: '$(($1+1))' ?? – voices Jun 05 '16 at 22:02
  • with $(($i+1)) you can calculate the amount of linesby a given variable $i (bash-arithmetic-expansion – rubo77 Feb 08 '21 at 10:09
  • This command uses a lot of RAM if you try it on a large log-file (many GB)! and you'd need enough free memory or you get an error like: sed: couldn't open temporary file /var/log/sedKrDiYy: Permission denied – rubo77 Feb 08 '21 at 10:14
7

sed is quite complex when it comes to task like this one. tail, grep or awk would make this a lot easier and should be used instead. That being said, it is possible.

The following solution is adapted from sed and Multi-Line Search and Replace.

sed -ni '
    # if the first line copy the pattern to the hold buffer
    1h
    # if not the first line then append the pattern to the hold buffer
    1!H
    # if the last line then ...
    ${
            # copy from the hold to the pattern buffer
            g
            # delete current line if it is followed by at least X more lines
            # replace X-1 with the proper value
            s/.*\n\(\(.*\n\)\{X-1\}\)/\1/
            # print
            p
    }
' filename

Without the comments, it makes a nifty one-liner. If you want to eliminate, e.g., everything but the last ten lines, use this:

sed -ni '1h;1!H;${;g;s/.*\n\(\(.*\n\)\{9\}\)/\1/;p;}' filename
Dennis
  • 49,727
  • On non-GNU seds you need -e after -i. – Matteo Sep 07 '12 at 06:46
  • Just like phiz's answer, this does not handle a request for 0 last lines... However, that is one of the reasons for the recommendation to use another tool. – Peter.O Sep 07 '12 at 07:49
  • This command also uses a lot of RAM if you try it on a large log-file (many GB)! and you'd need enough free memory or you get an error like: sed: couldn't open temporary file /var/log/sedKrDiYy: Permission denied. so you better use F=your.file; X=20; head -n -$X $F > /tmp/tmpfile && mv /tmp/tmpfile $F – rubo77 Feb 08 '21 at 10:20
2

Based on the script in section 4.13 of the sed manual you could do something like this:

n=10

(( n > 1 )) && script='1h; 2,'$n'{H;g;}; $q; 1,'$((n-1))'d; N; D'
(( n > 1 )) || script='$!d'

sed -i "$script" infile
Thor
  • 6,573
1

tac|sed|tac>&&(mv||cat>)

Both of the following command snippets will effectively delete all but the very last 5 lines of ~/file1. If you want to retain the last 10 lines, you can substitute: |sed '1,5!d;' with |sed '1,10!d;', and so on, as you see fit. 

  1. tac ~/"file1" |sed '1,5!d;' |tac >"/tmp/file2" &&mv  "/tmp/file2"  ~/"file1"
  2. tac ~/"file1" |sed '1,5!d;' |tac >"/tmp/file2" &&cat "/tmp/file2" >~/"file1"
voices
  • 2,721