Character encoding from UTF8 JSON to ISO-8859-1

Question

Using getJSON to retrieve some data which I am utf8 encoding on the server-side end...

"title":"new movie \u0091The Tree of Life\u0092 on day 6"

The page that is is displayed on is charset ISO-8859-1 and I am doing this...

$.getJSON('index.php', { q: q }, function(data){
    for (var i = 0; i < data.length; i++) {
        alert(data[i].title + "\n" + utf8_decode(data[i].title));
    }
});

The utf8_decode function comes from here.

The problem is that I am still seeing the magic squares for both versions...

new movie The Tree of Life on day 6

new movie ᔨe Tree of Life⠯n day 6

This leads me to believe that perhaps the character is of neither encoding. However it works if I paste the string onto a page and set the charset to either UTF8 or ISO-8859-1 :-/

Any help would be great!

Answer 1

There is no need to escape or decode any characters in data transmitted in JSON. It's done automatically. It is also independent of the page's encoding. You can easily transmit and display the euro sign (\u20ac) with your code even though ISO-8859-1 does not contain the euro sign.

You problem are the characters \u0091 and \u0092. They aren't valid Unicode characters. They are for private use only.

It rather looks as if you in fact have data that originally used the Windows-1250 character set but was not properly translated to Unicode/JSON. In Windows-1250, these two characters are typographic single quotes.

Answer 2

Did you tried without utf8_decode ?

If the characters in your string exist in ISO-8859-1, this will just work, as Javascript decodes the \u0091 in the encoding of the page.