PHP: Problems converting “’” character from ISO-88

Question

I'm having some issues with using PHP to convert ISO-8859-1 database content to UTF-8. I am running the following code to test:

// Connect to a latin1 charset database 
// and retrieve "Georgia O’Keeffe", which contains a "’" character
$connection = mysql_connect('*****', '*****', '*****');
mysql_select_db('*****', $connection);
mysql_set_charset('latin1', $connection);
$result = mysql_query('SELECT notes FROM categories WHERE id = 16', $connection);
$latin1Str = mysql_result($result, 0);
$latin1Str = substr($latin1Str, strpos($latin1Str, 'Georgia'), 16);

// Try to convert it to UTF-8
$utf8Str = iconv('ISO-8859-1', 'UTF-8', $latin1Str);

// Output both
var_dump($latin1Str);
var_dump($utf8Str);

When I run this in Firefox's source view, making sure Firefox's encoding setting is set to "Western (ISO-8859-1)", I get this:

So far, so good. The first output contains that weird quote and I can see it correctly because it's in ISO-8859-1 and so is Firefox.

After I change Firefox's encoding setting to "UTF-8", it looks like this:

Where did the quote go? Wasn't iconv() supposed to convert it to UTF-8?

Answer 1

U+2019 RIGHT SINGLE QUOTATION MARK is not a character in ISO-8859-1. It is a character in windows-1252, as 0x92. The actual ISO-8859-1 character 0x92 is a rarely-used C1 control character called "Private Use 2".

It is very common to mislabel Windows-1252 text data with the charset label ISO-8859-1. Many web browsers and e-mail clients treat the MIME charset ISO-8859-1 as Windows-1252 characters in order to accommodate such mislabeling but it is not standard behaviour and care should be taken to avoid generating these characters in ISO-8859-1 labeled content.

It appears that this is what's happening here. Change "ISO-8859-1" to "windows-1252".

Answer 2

this will solve your problem, supposing that your page header charset is utf-8:

// Opens a connection to a MySQL server
$connection = mysql_connect ($server, $username, $password);
$charset = mysql_client_encoding($connection);
$flagChange = mysql_set_charset('utf8', $connection);
echo "The character set is: $charset</br>mysql_set_charset result:$flagChange</br>";