C String encoding UTF8 without libiconv

Question

I know this question has been answered to most of the languages but i have not found a solution for C or i am misunderstanding something.

I have declared a "char *strSSID". I have a function in a web server that process the POST params of the form page. This function return in "srtSSID" a param named SSID which is used to configure the SSID name of a wireless. But the problem is that special characters in the SSID like "ñ" are seen with Linux properly and with my phone or Windows like strange characters.

I know the problem is the encoding. In my plattform (mips and uclibc library) a special character like 'ñ' is stored in two positions (i mean for example "srtSSID[0]" and "strSSID[1]" to store the first 'ñ' character of the SSID) of strSSID.

I need some help.

Thanks

Answer 1

In order to use UTF-8 characters in an SSID, the 802.11 spec requires that the UTF-8 SSID subfield of the Extended Capabilities element needs to be set. If you're finding that UTF-8 SSIDs don't work on certain devices, it's possible that they haven't set the Extended Capabilities element correctly, or perhaps they just don't support that capability.

The 802.11 spec doesn't say how the characters should be interpreted if the UTF-8 SSID capability isn't enabled, but I suspect that typically means only ASCII characters are valid, in which case your ñ is never going to work.

That said, you might have some success just converting to Latin1. If you don't have access to a library that will perform character set conversions, here's a basic conversion function that should be good enough for your needs.

void utf8tolatin1(char *s) {
  size_t i = 0, j = 0;
  char c;
  do {
    c = s[i++];
    if ((c&0xFC) == 0xC0 && s[i])
      c = (c<<6) + (s[i++]&0x3F);
    s[j++] = c;
  } while(c != 0);
}

Note that this only converts UTF-8 characters that are in the Latin1 range - everything else is left as is. If your SSID contains characters outside that range, there's probably nothing you can do to get it to work.

Answer 2

The only way to ensure characters are displayed correctly in all devices is to use characters that appear in ASCII, because (as I think you've identified) you won't be able to force Windows etc. to interpret them as UTF-8. Your problem is that characters such as 'ñ' don't appear in ASCII, so you can't translate unicode strings with such characters into standard ASCII.

The first 128 UTF-8 chars are the same as ASCII, so no translating of the string needs to occur... but you will need to remove or replace any chars that have values above 127.

If all the devices you are likely to use have the same non-English default language, it may be the case that they all use the same extended ASCII version (strictly, ASCII is a 7-bit code, but most implementations use the top bit to add another 128 characters which usually include accented characters and are often dependent on the device's default language). I'm straying into speculation here though! And your extended ASCII character set is unlikely to be compatible with devices which assume the SSID is UTF-8, of course!

Answer 3

If you're posting this through a webpage, you have to make sure that the encoding declared by the page is indeed utf-8. There are several ways to achieve that:

• Use the Content-Type header in the response: Content-Type: text/html; charset=UTF-8
• Use a meta tag in the HTML page. This is dependent on the HTML version you use. in HTML 4 it's <meta http-equiv="Content-type" content="text/html;charset=UTF-8">

The browsers are configured to assume different defaults for encoding, so a likely explanation is that your windows browser is expecting ISO-8859-1 encoding.

See this w3c page for more explanation on how to declare the encoding: http://www.w3.org/International/questions/qa-html-encoding-declarations