Concise and portable “join” on the Unix command-li

Question

How can I join multiple lines into one line, with a separator where the new-line characters were, and avoiding a trailing separator and, optionally, ignoring empty lines?

Example. Consider a text file, foo.txt, with three lines:

foo
bar
baz

The desired output is:

foo,bar,baz

The command I'm using now:

tr '\n' ',' <foo.txt |sed 's/,$//g'

Ideally it would be something like this:

cat foo.txt |join ,

What's:

1. the most portable, concise, readable way.
2. the most concise way using non-standard unix tools.

Of course I could write something, or just use an alias. But I'm interested to know the options.

Answer 1

This sed one-line should work -

sed -e :a -e 'N;s/\n/,/;ba' file

Test:

[jaypal:~/Temp] cat file
foo
bar
baz

[jaypal:~/Temp] sed -e :a -e 'N;s/\n/,/;ba' file
foo,bar,baz

To handle empty lines, you can remove the empty lines and pipe it to the above one-liner.

sed -e '/^$/d' file | sed -e :a -e 'N;s/\n/,/;ba'

Answer 2

I had a log file where some data was broken into multiple lines. When this occurred, the last character of the first line was the semi-colon (;). I joined these lines by using the following commands:

for LINE in 'cat $FILE | tr -s " " "|"'
do
    if [ $(echo $LINE | egrep ";$") ]
    then
        echo "$LINE\c" | tr -s "|" " " >> $MYFILE
    else
        echo "$LINE" | tr -s "|" " " >> $MYFILE
    fi
done

The result is a file where lines that were split in the log file were one line in my new file.

Answer 3

Simple way to join the lines with space in-place using ex (also ignoring blank lines), use:

ex +%j -cwq foo.txt

If you want to print the results to the standard output, try:

ex +%j +%p -scq! foo.txt

To join lines without spaces, use +%j! instead of +%j.

To use different delimiter, it's a bit more tricky:

ex +"g/^$/d" +"%s/\n/_/e" +%p -scq! foo.txt

where g/^$/d (or v/\S/d) removes blank lines and s/\n/_/ is substitution which basically works the same as using sed, but for all lines (%). When parsing is done, print the buffer (%p). And finally -cq! executing vi q! command, which basically quits without saving (-s is to silence the output).

Please note that ex is equivalent to vi -e.

This method is quite portable as most of the Linux/Unix are shipped with ex/vi by default. And it's more compatible than using sed where in-place parameter (-i) is not standard extension and utility it-self is more stream oriented, therefore it's not so portable.