How can I "cat" a file and remove commented lines?

Question

I'd like to know if there is a way that I could cat file like php.ini and remove all lines starting with ;

For example, if the file contained this:

;   - Show all errors, except for notices
;
;error_reporting = E_ALL & ~E_NOTICE
;
;   - Show only errors
;
;error_reporting = E_COMPILE_ERROR|E_ERROR|E_CORE_ERROR
;
;   - Show all errors except for notices
;
error_reporting  =  E_ALL & ~E_NOTICE

and I ran the correct command cat | {remove comments command}, then I would end up with:

error_reporting  =  E_ALL & ~E_NOTICE

Note - I assumed that cat would be the best way to do this but I'm actually fine with the answer using another utility like awk, sed, egrep, etc.

What about something like error_reporting = E_ALL & E_NOTICE ; Show all errors, except for notices? Should the comment be removed in that case as well? — user, Dec 07 '11 at 08:29
@MichaelKjörling - I'm really fine with just lines starting with comments being removed — cwd, Dec 07 '11 at 14:16
cat is the tool to concatenate files. grep is the tool to filter lines based on patterns. sed and awk can also modify those lines. — Stéphane Chazelas, Jan 28 '13 at 12:32

score 44 · Answer 1 · edited Dec 07 '11 at 22:50

44

You don't need to pipe a file thru grep, grep takes filename(s) as command line args.

grep -v '^#' file1 file2 file3

will print all lines EXCEPT those that begin with a # char. you can change the comment char to whatever you wish.

If you have more than one comment char (assuming its at the beginning of a line)

egrep -v '^(;|#|//)' filelist

edited Dec 07 '11 at 22:50

Gilles 'SO- stop being evil'

829,060

answered Dec 07 '11 at 21:26

bsd

11,036

3

Another one I've done that always stuck with me was grep '^[^;]' filename. I can't speak to its portability though! – Jodie C Dec 08 '12 at 13:57
@JodieC, that's portable but also removes empty lines (Which is often desired). The standard equivalent of egrep is grep -E. One can also use grep -ve '^[;#]' -e '^//' – Stéphane Chazelas Jan 28 '13 at 12:30
1

if you want to remove blank lines, and also line with spaces before the '#' you can do: grep -ve '\s*#' -e '^$' filename – Evan Morrison Jun 09 '23 at 18:43

score 37 · Accepted Answer · edited Dec 07 '11 at 07:35

37

You can use:

sed -e '/^;/d' php.ini

edited Dec 07 '11 at 07:35

Mat

52,586

answered Dec 07 '11 at 06:06

favadi

1,123

score 10 · Answer 3 · edited Aug 17 '13 at 20:18

10

egrep can save you the use of cat. In other words, create less processes (egrep vs cat+egrep) and use less buffers (pipe from cat to egrep vs no pipe).

It is generally a good idea to limit the use of cat if you simply want to pass a file to a command that can read it on its own.

With this said, the following command will remove comments, even if they are indented with spaces or tabs:

egrep -v '^[[:blank:]]*;' file.ini

edited Aug 17 '13 at 20:18

Amal Murali

105

answered Dec 07 '11 at 23:20

nrolans

1,297

1

Funny that you combine the newer [[ character class ]] regex format with the use of the egrep command deprecated for at least a decade or so. – mikeserv Mar 13 '14 at 16:25
Interesting... current docs at https://www.gnu.org/software/grep/manual/html_node/Character-Classes-and-Bracket-Expressions.html include POSIX classes. – Jack Wasey Aug 18 '17 at 12:08

score 9 · Answer 4 · edited Jan 11 '18 at 09:33

9

egrep -v '^;|^$' $file

that will exclude lines that begin with the ';', and empty lines.

in regex, ^ indicates the beginning of a line, and $ the end of a line, so ^$ specifies lines where the start of line character and the end of line character are right next to each other.

edited Jan 11 '18 at 09:33

Yokai

223

answered Dec 07 '11 at 05:25

Tim Kennedy

19,697

so if i'm reading this right this can remove commented lines, but also blank lines? – cwd Dec 07 '11 at 05:26
1

@cwd Yes. I'm not sure why he included both, but if you only want to remove commented lines just use egrep -v '^;' – Michael Mrozek Dec 07 '11 at 05:44
4

egrep likes files too (less processes and buffers used), and a little bonus would be to remove indented comments too: egrep -v '^[[:blank:]]*;' file.ini – nrolans Dec 07 '11 at 06:55
10

We need a "useless use of cat" badge. – Simon Richter Dec 07 '11 at 11:02
@nrolans - looks like a popular comment, why not make it an answer? – cwd Dec 07 '11 at 14:13
@MichaelMrozek i included both, just because if you're performing this action on a long file, like a stock httpd.conf, or something, excluding commented lines often leaves large blocks of whitespace lines between the lines with real content. so, generally, i find it more useful to filter both comment lines, and whitespace lines, even though it wasn't exactly to the letter of the question. – Tim Kennedy Dec 08 '11 at 03:37

score 3 · Answer 5 · edited Apr 05 '22 at 13:11

3

grep -Ev ^'(#|$)' file.txt

Strips all comments and empty lines from file.txt

Updated answer per Yokai's comment. "Direct invocation as either egrep or fgrep is deprecated, but is provided to allow historical applications that rely on them to run unmodified." ref: https://www.gnu.org/software/grep/manual/grep.html

edited Apr 05 '22 at 13:11

F1Linux

2,476

answered Jul 26 '14 at 20:51

Nikola

39

2

You should consider expanding your answer with a little more information for the uninitiated (e.g., exactly what the regex you use does). – HalosGhost Jul 26 '14 at 21:01
1

egrep is depreciated. Use grep -Ev instead. – Yokai Jan 11 '18 at 08:20
This is the tightest, most elegant answer. Thanks Nikola! – F1Linux Mar 26 '22 at 10:14

score 2 · Answer 6 · answered Dec 07 '11 at 07:31

2

A simple awk one-liner awk '/^;/{next}1' input_file should do the trick.

[jaypal:~/Temp] cat file
;   - Show all errors, except for notices
;
;error_reporting = E_ALL & ~E_NOTICE
;
;   - Show only errors
;
;error_reporting = E_COMPILE_ERROR|E_ERROR|E_CORE_ERROR
;
;   - Show all errors except for notices
;
error_reporting  =  E_ALL & ~E_NOTICE

[jaypal:~/Temp] awk '/^;/{next}1' file
error_reporting  =  E_ALL & ~E_NOTICE
[jaypal:~/Temp]

answered Dec 07 '11 at 07:31

jaypal singh

1,592

3

Correct, but verbose. Just awk '!/^;/' input_file is enough. – manatwork Dec 08 '11 at 09:32

score 2 · Answer 7 · answered Dec 07 '11 at 15:27

2

As well as Jaypal, I also most probably would use awk for these purposes. It worse to mention that perl is sometimes quite handy for such purposes:

cat data.txt | perl -lne "print unless /^;/"

Perl regexps are more powerful compared to awk's one and sometimes you might need them.

answered Dec 07 '11 at 15:27

shabunc

363

+1 for perl, although the cat and the -l are both redundant, so a simpler invocation is perl -ne 'print unless /^;/' data.txt – Simon Whitaker Dec 08 '11 at 03:59
@Simone Whitaker, yes, you are right - it is just a habit to write it the way i write, and it worth to mention it. – shabunc Dec 08 '11 at 04:18
1

Sure thing. In fact, I think cat works fine in these examples if you consider it as a proxy for the more generic "anything generating text on STDOUT". Unix pipes are the best thing since sliced bread, imho. :) – Simon Whitaker Dec 08 '11 at 09:43

score 2 · Answer 8 · answered Dec 08 '11 at 04:06

An elaboration on @shabunc's answer, this uses Perl to strip comments (including inline comments), then print any lines containing anything other than whitespace.

$ perl -ne 's/;.*//; print if /\S/' data.txt

Explanation:

s/;.*// uses the substitution operator (s/<regex>/<replacement>/) to replace instances of a semi-colon and everything following it on a line with the empty string.
print if /\S/ prints the line if it matches the regexp \S, which is a character class matching all non-whitespace characters.

score 1 · Answer 9 · answered Dec 19 '13 at 10:26

Example show only lines + do not show new lines or emtpy lines:

$ egrep -v '^(;|#|//)' /etc/ssh/sshd_config | tr '\n' ' '

 Protocol 2    SyslogFacility AUTHPRIV      PasswordAuthentication yes  ChallengeResponseAuthentication no   GSSAPIAuthentication yes GSSAPICleanupCredentials yes  UsePAM yes  AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS  X11Forwarding yes   Subsystem sftp    /usr/libexec/openssh/sftp-server

OR

$ egrep -v '^(;|#|//|$)' /etc/ssh/sshd_config    

Protocol 2
SyslogFacility AUTHPRIV
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
UsePAM yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
X11Forwarding yes
Subsystem   sftp    /usr/libexec/openssh/sftp-server

The use of egrep is long deprecated. grep -E is the command you're looking for here. — mikeserv, Mar 13 '14 at 16:23

score 1 · Answer 10 · answered Dec 08 '12 at 10:46

Here's one that I use, just substitute ';' with the comment character (e.g. '#' for many UNIX service configuration files):

grep -Ev '^[[:space:]]*;|^$' chan_dahdi.conf.sample | sed 's/;.*$//'

That gets rid of all whole-line comments (even if they have leading whitespace), and any comments that end non-comment lines, and succinctly removes blank lines from the output as well. This may be possible without the pipeline (my sed- or awk-fu is admittedly not great), but it's so easy for me to understand (and remember), I figured I'd post it here.

score 0 · Answer 11 · answered Oct 23 '14 at 14:55

0

Will strip also empty lines

grep -E -v "^\s*($|;)" php.ini

answered Oct 23 '14 at 14:55

Fedir RYKHTIK

111

\s is not guaranteed to be supported in plain grep – tripleee May 31 '22 at 05:27

score 0 · Answer 12 · answered Apr 10 '16 at 17:01

0

you can use the following command to save the lines, excluding empty lines and lines starting with # in a new file

cat <file to be read> | egrep -v '^#|^$' > <file to be written at>

answered Apr 10 '16 at 17:01

KawaiKx

465

Useless cat pipe and egrep is depreciated and to be replaced with grep -E for extended expressions or just grep -v. – Yokai Jan 11 '18 at 08:19

How can I "cat" a file and remove commented lines?

12 Answers12

Linked

Related