Why would a correct shell script give a wrapped/tr

Question

This question already has an answer here:

• Are shell scripts sensitive to encoding and line endings? 5 answers

I have a shell script with a command that seems like it should work, but instead it fails with an odd wrapped/truncated/corrupted error message. Example:

$ ls -l myfile
-rw-r----- 1 me me 0 Aug  7 12:36 myfile
$ cat myscript 
ls -l myfile
$ bash myscript
: No such file or directory

The file clearly exist, but even if I didn't, this is the kind of error message I would normally get:

$ ls -l idontexist
ls: cannot access idontexist: No such file or directory

Notice how it includes the tool name ls, a message string and the filename while mine does not.

Here's what I get if I try to use mysql instead. The error message looks like it's been wrapped, and now starts with a quote:

Command:  mysql -h myhost.example.com
Expected: ERROR 2005 (HY000): Unknown MySQL server host 'myhost.example.com' (0)
Actual:   ' (0) 2005 (HY000): Unknown MySQL server host 'myhost.example.com

And here's my trivial ssh command that should work, or at least give a normal error message, but which instead is wrapped to start with a colon and ends with strange clobbering:

Command:  ssh myhost
Expected: ssh: Could not resolve hostname myhost: Name or service not known
Actual:   : Name or service not knownname myhost

Why does this happen, and how do I fix it?

Answer 1

TL;DR: Your script or data has Windows style CRLF line endings.

Convert to Unix style by deleting the carriage returns.

---

How do I check if my script or data has carriage returns?

They're detectable as ^M in the output of cat -v yourscript:

$ cat -v myscript
ls -l myfile^M

If your script doesn't have them, your data might -- especially if reading from ini/csv files or curl:

hostname=$(curl https://example.com/loginhost.txt)
ssh "$hostname"            # Shows strange error
echo "$hostname" | cat -v  # Shows myhost^M

How do I remove them?

Set your editor to save the file with Unix line endings, aka "line terminators" or "end-of-line characters", and resave it.

You can also remove them from a command line with dos2unix yourscript or cat yourscript | tr -d '\r' > fixedscript.

If found in your data, you can pipe your source through tr -d '\r':

hostname=$(curl https://example.com/loginhost.txt | tr -d '\r')

Why do carriage returns cause strange error messages?

The "carriage return" character, aka CR or \r, causes the cursor to move to the start of the line, and continue printing from there. In other words, it starts overwriting the line from the start. This is why they wrap strangely:

Intended:     ssh: Could not resolve hostname myhost\r: Name or service not known

Written:      ssh: Could not resolve hostname myhost\r
Overwritten:  : Name or service not known
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                  
Result:       : Name or service not knownname myhost