Unix script appends ^M at end of each line

Question

I have a Unix shell script which does the following:

1. creates a backup of a file
2. appends some text to a file

Now in #2 if I insert a text, ^M gets appended on all the lines of the file.

For example:

echo " a" >> /cust/vivek.txt
echo " b" >> /cust/vivek.txt

vi vivek.txt
abc^M
bcd^M
a^M
b^M

Any way to avoid this?

Answer 1

I'm not sure how echo could be producing ^M characters but you can remove them by running dos2unix on your file, like this:

dos2unix /cust/vivek.txt

Answer 2

^M are the meta characters which entered your file when it was used in windows.

the dos2unix command can fix this.

dos2unix <filename>

Answer 3

^M is a carriage return, and is commonly seen when files are copied from Windows. Use:

od -xc filename

that should give a low-level list of what your file looks like. If you file does not come from Windows then another possibility is that your terminal setting are not translating correctly. Check that the TERM environment variable is correct.

If the file has come from Windows, then use dos2unix or sed 's/\r//' file > file.new

Answer 4

Only

sed -e "s/\r//g" file

worked for me

Answer 5

I suspect this may be an artifact of your vi settings, rather than the concatenation.

What does

cat -v -e filename

show ? This command will dump out your file and mark the control characters so it's clear what's really in your file. See also this Superuser question/answer set.