How to send special characters via mail from a she

Question

I have a script that runs on cron that outputs some text which we send to the 'mail' program. The general line is like this:

./command.sh | mail -s "My Subject" [e-mail] -- -F "Sender Name" -f [e-mail]

The problem is that the text generated by the script has some special characters - é, ã, ç - since it is not in english. When the e-mail is received, each character is replaced by ??.

Now I understand that this is most likely due to the encoding that is not set correctly. What is the easiest way to fix this?

Answer 1

This is probably not a command line issue, but a character set problem. Usually when sending E-Mails, the character set will be iso-8859-1. Most likely the text you are putting into the process is not iso-8859-1 encoded. Check out what the encoding is of whatever data source you are getting the text from.

Obligatory "good reading" link: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

Re your update: In that case, if you enter the special characters manually, your terminal may be using UTF-8 encoding. You should be able to convert the file's character set using iconv for example. The alternative would be to tell mail to use UTF-8 encoding, but IIRC that is not entirely trivial.

Answer 2

You're right in assuming this is a charset issue. You need to set the appropriate environment variables to the beginning of your crontab.

Something like this should work:

LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8

Optionally use LC_ALL in place of LC_CTYPE.

Reference: http://opengroup.org/onlinepubs/007908799/xbd/envvar.html

Edit: The reason it displays fine when you run it in your shell is probably because the above env vars are set in your shell.

To verify, execute 'locale' in your shell, then compare to the output of a cronjob that runs the same command.

Re-Edit: Ok, so it's not an env var problem.

I am assuming you're using mailx, as it is the most common nowdays. It's manpage says:

The character set for outgoing messages is not necessarily the same as the one used on the terminal. If an outgoing text message contains characters not representable in US-ASCII, the character set being used must be declared within its header. Permissible values can be declared using the sendcharsets variable,

So, try and add the following arguments when calling mail:

-S sendcharsets=utf-8,iso-8859-1

Answer 3

Just to give additional information to KumZ answer: if you need to specify more headers with the -a switch, feel free to add them up, like this (note the polyusage of -a).

echo /path/to/file | mail -s "Some subject" [e-mail] -a "From: Human Name <[e-mail]>" -a "Content-Type: text/plain; charset=UTF-8"

Answer 4

i've written a bash function to send an email to recipients. The function send utf-8 encoded mails and work with utf-8 chars in subject and content by doing a base64 encode.

To send a plain text email:

send_email "plain" "[e-mail]" "subject" "contents" "[e-mail]" "[e-mail]" "[e-mail]" ...

To send a HTML email:

send_email "html" "[e-mail]" "subject" "contents" "[e-mail]" "[e-mail]" "[e-mail]" ...

Here is the function code.

# Send a email to recipients.
#
# @param string $content_type Email content mime type: 'html' or 'plain'.
# @param string $from_address Sender email.
# @param string $subject Email subject.
# @param string $contents Email contents.
# @param array $recipients Email recipients.
function send_email() {
  [[ ${#} -lt 5 ]] && exit 1

  local content_type="${1}"
  local from_address="${2}"
  local subject="${3}"
  local contents="${4}"

  # Remove all args but recipients.
  shift 4

  local encoded_contents="$(base64 <<< "${contents}")"
  local encoded_subject="=?utf-8?B?$(base64 --wrap=0 <<< "${subject}")?="

  for recipient in ${@}; do
    if [[ -n "${recipient}" ]]; then
    sendmail -f "${from_address}" "${recipient}" \
        <<< "Subject: ${encoded_subject}
MIME-Version: 1.0
From: ${from_address}
To: ${recipient}
Content-Type: text/${content_type}; charset=\"utf-8\"
Content-Transfer-Encoding: base64
Content-Disposition: inline

${encoded_contents}"
    fi
  done

  return 0
} # send_message()

Answer 5

My /usr/bin/mail is symlinked to /etc/alternatives/mail which is also symlinked to /usr/bin/bsd-mailx

I had to specify myself the encoding in the mail header. (The -S is not supported here.)

cat myutf8-file | mail -a "Content-Type: text/plain; charset=UTF-8" -s "My Subject" [e-mail]

Answer 6

You may use sendmail command directly without mail wrapper/helper.
It would allow you to generate all headers required for "raw" UTF-8 body
(UTF-8 is mentioned in asker's comments),

WARNING-1:
Non 7bit/ASCII characters in headers (e.g. Subject:,From:,To:) require special encoding
WARNING-2:
sendmail may break long lines (>990 bytes).

SENDER_ADDR=[e-mail]
SENDER_NAME="Sender Name"
RECIPIENT_ADDR=[e-mail]
(
# BEGIN of mail generation chain of commands
# "HERE" document with all headers and headers-body separator
cat << END
Subject: My Subject
From: $SENDER_NAME <$SENDER_ADDR>
To: $RECIPIENT_ADDR
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

END
# custom script to generate email body
./command.sh
# END   of mail generation chain of commands
) | /usr/sbin/sendmail -i -f$SENDER_ADDR -F"$SENDER_NAME" $RECIPIENT_ADDR