Reading Excel files in a locale independent way

Question

I am using the following code to read data from various Excel files:

    // IMEX=1 - to force strings on mixed data
    // HDR=NO - to process all the available data
    // Locale 1033 is en-US. This was my first attempt to force en-US locale.
    string connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Locale Identifier=1033;Extended Properties=\"{1};READONLY=TRUE;HDR=NO;IMEX=1;\"";

    // source type according to the
    // http://www.microsoft.com/en-us/download/details.aspx?id=13255

    // try determining from extension
    bool isOldFormat =
            Path.GetExtension(sourceExcel).Equals(".xls", StringComparison.OrdinalIgnoreCase);

    bool isBinary =
            Path.GetExtension(sourceExcel).Equals(".xlsb", StringComparison.OrdinalIgnoreCase);

    string sourceType = isOldFormat ? "Excel 8.0" : "Excel 12.0";

    if (!isOldFormat)
        sourceType += " Xml"; // for some reason the new binary xlsb files also need Xml

    connectionString = string.Format(connectionString, sourceExcel, sourceType);

    // this was my second attempt to force Excel to use US culture
    var oldCulture = Thread.CurrentThread.CurrentCulture;
    Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture("en-US");

    var dt = new DataTable();
    try
    {
        using (var con = new OleDbConnection(connectionString))
        {
            con.Open();

            // get all the available sheets
            using (DataTable dataSet = con.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null))
            {
                // this was my third attempt to force Excel to use US culture
                dataSet.Locale = CultureInfo.CreateSpecificCulture("en-US");
                // get the sheet name in the file (will throw if out of range)
                string workSheetName = dataSet.Rows[worksheetIndex]["TABLE_NAME"].ToString();//.Trim(new[] { '$' }).Replace("'", "");

                string sql = String.Format("select * from [{0}]", workSheetName);

                var da = new OleDbDataAdapter(sql, con);
                // this was my fourth attempt to force Excel to use US culture
                dt.Locale = CultureInfo.CreateSpecificCulture("en-US");
                da.Fill(dt);
            }

            con.Close();
        }

As you see, I was pretty desperate, trying to force Excel to use en-US compatible locale when importing data. I need this because my code might be executed on servers with various locales, but the data needs some additional processing which assumes that the incoming data is en-US/neutral locale.

I tried also CultureInfo.InvariantCulture instead of CultureInfo.CreateSpecificCulture("en-US").

No matter how I try, when the server locale is set to some other locale which uses . as thousands separator and , as decimal separator, I get wrong results in my dt DataTable.

To compare the result for a currency value -£200000.00 :

When the server regional settings correspond to US locale, I get "-£200,000.00"

When the server regional settings correspond to Latvian locale, I get "-£200 000,00"

I cannot even post-process the data using the current numeric separators from Thread.CurrentThread.CurrentCulture, because OleDb seems to ignore it completely.

Where does OleDb get the current culture from? Ho do I tell the OleDbConnection or Microsoft.ACE.OLEDB.12.0 provider that I need the data formatted according to en-US or Invariant culture?

Answer 1

After lots of trials and errors and after reading this outdated article http://support.microsoft.com/default.aspx?scid=KB;EN-US;Q320744 I found that the current version of OLEDB by default seems to be using the culture from HKEY_CURRENT_USER\Control Panel\International. Unfortunately, I did not found how to call the SetVarConversionLocaleSetting function from my C# code to force OLEDB to use the current thread culture, so I went with the principle - if I cannot adjust OLEDB for my code, then I'll adjust my code to be compatible with OLEDB culture. And after I am done with it, I can convert all the data to the invariant culture.

But there is a tricky part. You cannot just grab the decimal separator from HKEY_CURRENT_USER\Control Panel\International, because OLEDB ignores user customized settings for number formats. OLEDB takes only the default preset values for that culture. So I had to do the following:

var oldCulture = Thread.CurrentThread.CurrentCulture;

using (RegistryKey international = 
        Registry.CurrentUser.OpenSubKey("Control Panel\\International", false))
{
    string userDefaultCulture = international.GetValue("LocaleName").ToString();
    // notice: although the user might have customized his decimal/thousand separators,
    // still OLEDB ignores these customizations. That is why I create a culture with default settings.
    cultureToNormalize = new CultureInfo(userDefaultCulture, false);
}

// force both OLEDB and current thread cultures to match for the next ToString() etc. conversions in my function
Thread.CurrentThread.CurrentCulture = cultureToNormalize;

string decSep = cultureToNormalize.NumberFormat.NumberDecimalSeparator;
string groupSep = cultureToNormalize.NumberFormat.NumberGroupSeparator;

and now I can process the data as needed, and also I can call ToString() safely - the cultures of OLEDB and .NET stringified numbers and currencies will match. And, to be a good boy, I restore the previous culture at the end of my function.

If anybody has a better solution, I'll be really grateful. But for now I'll keep it as is - all my unit tests are green now.