18

I am running the following 2 sed commands. The first one adds newline characters where I want them, the second also adds newline characters where I want them, BUT also adds an extra one at the end of the file where there wasn't one before.

sed -e 's|\<LIST_G_STATEMENT>|&\
|g' ${XMLDIR}/statement_tmp_1.xml > ${XMLDIR}/statement_tmp_2.xml

sed -e 's|\</LIST_G_STATEMENT>|&\
|g' ${XMLDIR}/statement_tmp_2.xml > ${XMLDIR}/statement_tmp_3.xml

Using od -c on all 3 of the files gives the following output.

statement_tmp_1.xml (no \n at end of file)

1314700    T   A   T   E   M   E   N   T   >   <   /   L   I   S   T   _
1314720    G   _   S   T   A   T   E   M   E   N   T   >   <   /   G   _
1314740    S   E   T   U   P   >   <   /   L   I   S   T   _   G   _   S
1314760    E   T   U   P   >   <   /   A   R   X   S   G   P   O   >
1314777

statement_tmp_2.xml (no \n at end of file)

1314700    S   T   A   T   E   M   E   N   T   >   <   /   L   I   S   T
1314720    _   G   _   S   T   A   T   E   M   E   N   T   >   <   /   G
1314740    _   S   E   T   U   P   >   <   /   L   I   S   T   _   G   _
1314760    S   E   T   U   P   >   <   /   A   R   X   S   G   P   O   >
1315000

statement_tmp_3.xml (\n at end of file - where did it come from?)

1314700    S   T   A   T   E   M   E   N   T   >   <   /   L   I   S   T
1314720    _   G   _   S   T   A   T   E   M   E   N   T   >  \n   <   /
1314740    G   _   S   E   T   U   P   >   <   /   L   I   S   T   _   G
1314760    _   S   E   T   U   P   >   <   /   A   R   X   S   G   P   O
1315000    >  \n
1315002

I am running AIX 5.3

Basically, I either want it to stop adding the extra \n, or find a way of removing it.

Anthon
  • 79,293
jonnohudski
  • 181

3 Answers3

13

You should consider yourself lucky that AIX sed added that missing newline characters.

A non-empty file that doesn't end in a newline character is not a text file (at least as per the POSIX definition of a text file) as a text file is meant to contain lines and lines are a (not-too-long) sequence of characters terminated by a newline character, so the behaviour of text utilities like sed on it is unspecified and in practice varies from implementation to implementation.

Some sed implementation would have dismissed those spurious character after the last line.

AFAIK, xml files are meant to be text files, so that means sed just fixed it for you.

If you do need that file not to end in a newline character, then you could use perl or other tools that can cope with non-text data.

perl -pe 's|<LIST_G_STATEMENT>|$&\n|g'
Stéphane Chazelas
  • 544,893
  • 1
    The terminating newline is helpful, if you expect to pipe your sed output into any other standard Unix utility. Honestly, I didn't notice sed did this for years, since Bourne shell command substitutions like $(sed 's/bas/replac/' <<<'basement') furtively trim the final newline, if there is one. But there are times when you definitely don't want it; e.g., manipulating X clipboard text with sed. FYI, GNU sed, if available, does not add a terminating newline if you use p it with the -n option, as described in this SE answer. – Kevin E Dec 13 '19 at 22:23
0

Here's a way to remove the final newline from a file using dd:

printf "" | dd  of='/path/to/file' seek=<filesize_in_bytes - 1> bs=1 count=1

To test whether a file ends with a newline you could use:

tail -c 1 /path/to/file | tr -dc '\n' | wc -c

And to get the file size in bytes use:

wc -c < /path/to/file
chan
  • 1
0

According to this AIX manual IBM's tail does -reverse - which looks pretty cool. So long as your file is under 20KB the following should work:

tail -r <file | dd bs=1 skip=1 | tail -r >file.new
mikeserv
  • 58,310