Changing content format

Question

I have a text file which has 1000 rows/entries. These 1000 entries correspond to the column names for my dataframe. These are the new features that need to be added to the dataframe for model development. My input text file looks like this:

show temperature all#Total Numbers Approved#g2/3
show temperature all#Total Numbers Approved#g2/2
show temperature all#Total Numbers Approved#g2/4
show temperature all#Total Numbers Approved#g0/2
show temperature all#Total Numbers Sent#g1/2
show temperature all#Total Numbers Sent#g1/3
show temperature all#Total Numbers Sent#g1/1
formulastat gpucores all parameter function-frames#formula:1#gpucores:11
formulastat gpucores all parameter function-frames#formula:1#gpucores:10
formulastat gpucores all parameter function-frames#formula:2#gpucores:10
formulastat gpucores all parameter function-frames#formula:2#gpucores:11
formulastat gpucores all parameter function-frames#formula:0#gpucores:8

I was looking to automatically add '' for each line, remove the line break and add a comma after each line entry. Expected output is as follows:

'show temperature all#Total Numbers Approved#g2/3', 'show temperature all#Total Numbers Approved#g2/2', 'show temperature all#Total Numbers Approved#g2/4', 'show temperature all#Total Numbers Approved#g0/2', 'show temperature all#Total Numbers Sent#g1/2', 'show temperature all#Total Numbers Sent#g1/3', 'show temperature all#Total Numbers Sent#g1/1', 'formulastat gpucores all parameter function-frames#formula:1#gpucores:11', 'formulastat gpucores all parameter function-frames#formula:1#gpucores:10', 'formulastat gpucores all parameter function-frames#formula:2#gpucores:10', 'formulastat gpucores all parameter function-frames#formula:2#gpucores:11', 'formulastat gpucores all parameter function-frames#formula:0#gpucores:8'

score 3 · Answer 1 · answered Sep 20 '20 at 04:36

3

# if you need the space after comma
$ seq 10 | perl -pe 's/.+/"$&"/; s/\n/, / if !eof'
"1", "2", "3", "4", "5", "6", "7", "8", "9", "10"

# if space isn't required
$ seq 10 | sed 's/.*/"&"/' | paste -sd,
"1","2","3","4","5","6","7","8","9","10"

Use \x27 instead of " in the above solutions if you need single quote instead of double quotes

answered Sep 20 '20 at 04:36

Sundeep

12,008

Does the advice at http://awk.freeshell.org/PrintASingleQuote to not use hex escape sequences \x27 apply to perl and/or sed? – Ed Morton Sep 20 '20 at 18:03
1

For GNU sed (and possibly many other implementations): \xxx Produces or matches a character whose hexadecimal ASCII value is xx. (it is restricted to two digits)... for perl, it requires \x{} form if you need more than two hex characters.. so, as far as I know, \x27 is robust for both sed/perl... and for GNU awk - As of version 4.2, only two digits are processed. – Sundeep Sep 21 '20 at 02:57
1

good to know, thanks for the info! I tried the octal version \047 with perl and it worked too but with sed - not so much! – Ed Morton Sep 21 '20 at 03:02
1

\047 in sed will be treated as \0 (alias for &) followed by 47.. you'll need \oNNN for octal and \dNNN for decimal formats – Sundeep Sep 21 '20 at 04:21

Gordon Davisson · Answer 2 · 2020-09-20T22:46:45.993

1

You can do this pretty easily with a shell loop. Just read the items, then print them with quotes. The only tricky thing is to print ", " after all but the last... or before all but the first (which turns out to be easier).

{
    read item
    printf "'%s'" "$item"
    while IFS= read -r item; do
        printf ", '%s'" "$item"
    done
    echo
} <infile.txt >outfile.txt

edited Sep 20 '20 at 22:46

answered Sep 20 '20 at 04:30

Gordon Davisson

4,945

1

@EdMorton Those points are true in general, but I don't think relevant here. The biggest thing that slows down shell loops is creating subprocesses, and this one doesn't need to do that (I timed it at 0.05 seconds for 1,000 items on my rather old Mac). And the read without -r will mangle backslashes (although printf won't in this mode), but I'm pretty sure if there aren't any backslashes or quotes in the input (and if there were, they'd need special handling anyway). – Gordon Davisson Sep 20 '20 at 22:10
1

@EdMorton I agree; I've edited my answer accordingly. – Gordon Davisson Sep 20 '20 at 22:47

score 0 · Answer 3 · answered Sep 20 '20 at 18:00

$ awk '{printf "%s\047%s\047", sep, $0; sep=", "} END{print ""}' file
'show temperature all#Total Numbers Approved#g2/3', 'show temperature all#Total Numbers Approved#g2/2', 'show temperature all#Total Numbers Approved#g2/4', 'show temperature all#Total Numbers Approved#g0/2', 'show temperature all#Total Numbers Sent#g1/2', 'show temperature all#Total Numbers Sent#g1/3', 'show temperature all#Total Numbers Sent#g1/1', 'formulastat gpucores all parameter function-frames#formula:1#gpucores:11', 'formulastat gpucores all parameter function-frames#formula:1#gpucores:10', 'formulastat gpucores all parameter function-frames#formula:2#gpucores:10', 'formulastat gpucores all parameter function-frames#formula:2#gpucores:11', 'formulastat gpucores all parameter function-frames#formula:0#gpucores:8'

Changing content format

3 Answers3