Getting data with UTF-8 charset from MSSQL server

Question

I can't seem to get data from MSSQL encoded as UTF-8 using FreeTDS extension.

Connecting:

ini_set('mssql.charset', 'UTF-8');
$this->_resource = mssql_connect($config['servername'], $config['username'], $config['password']);

I have no ability to use any other extension.

I've tried creating ~/.freetds.conf

[global]
client charset = UTF-8

I've tried passing parameters to php:

php -d mssql.charset="UTF-8" index.php

Data is still not in UTF-8.

php -i

mssql

MSSQL Support => enabled
Active Persistent Links => 0
Active Links => 0
Library version => FreeTDS

Directive => Local Value => Master Value
mssql.allow_persistent => On => On
mssql.batchsize => 0 => 0
mssql.charset => no value => no value
mssql.compatability_mode => Off => Off
mssql.connect_timeout => 5 => 5
mssql.datetimeconvert => On => On
mssql.max_links => Unlimited => Unlimited
mssql.max_persistent => Unlimited => Unlimited

Ideas?

Answer 1

MSSQL and UTF-8 are quite a pain in the ... sometimes. I had to convert it manually. The problem: MSSQL doesn't actually know and support UTF-8.

Convert from database value to UTF-8:

mb_detect_encoding($value, mb_detect_order(), true) === 'UTF-8' ? $value : mb_convert_encoding($value, 'UTF-8');

Converting from UTF-8 to database value:

mb_convert_encoding($value, 'UCS-2LE', mb_detect_encoding($value, mb_detect_order(), true));

Fortunately I was using Doctrine so all I had was to create a custom StringType implementation.

Answer 2

You should change your TDS version based on what version of SQL server your using. Check out the installation guide for details.

http://www.freetds.org/userguide/choosingtdsprotocol.htm

Answer 3

You can also solve this issue by adding CharacterSet UTF-8 in the $connectionInfo before connecting to the DB.

$serverName = "MyServer";
$connectionInfo = array( "Database"=>"AdventureWorks", "CharacterSet" => "UTF-8");
$conn = sqlsrv_connect( $serverName, $connectionInfo);

Worked fine NO additional encoding needed.

Answer 4

If you use freeTDS, you should change below lines on /etc/freetds/freetds.conf:

[global]
# TDS protocol version
tds version = 4.2

To this:

[global]
# TDS protocol version
tds version = 8.0
;tds version = 4.2

and finally add this line:

# set charset
client charset = UTF-8

** clinet charset is in global [scope]

In your queries, you should use N character. like this:

$query = "INSERT INTO dbo.SMSOutbox (StationID, Dest, Text) VALUES ";
   $query .= '(';
   $query .= "'" . $this->stationId . "', ";
   $query .= "'" . $this->destination . "', ";
   $query .= "N'" . $this->text . "'";
   $query .= ')';

Answer 5

It seem version 7.0 or great is required. iconv() also seems to work well, but is tedious.

$query = $this->db->query($sql);
$result = $query->fetchAll(PDO::FETCH_OBJ);
foreach ($result as $row) {
    foreach (get_object_vars($row) as $key => $value) {
    $row->$key = (mb_detect_encoding($value, mb_detect_order(), true) === 'UTF-8') 
            ? $value : iconv('iso-8859-1', 'utf-8', $value);
    }
    $results[] = $row;
}

Answer 6

I had this problem and it solved by adding this line to my php script before connecting to MSSQL Server:

ini_set('mssql.charset', 'UTF-8');