C++ File character encoding

Question

Ok so I'm trying to read a json formatted text file with accents (French), under W8, using C++ (Visual Studio 2012 Express).

This is the file:

    {"products": [{"id": 125, "label": "Billél"}, {"id": 4, "label": "Rùbin"}]}

One line, encoded in UTF-8 (no BOM), saved as D:/p.txt

This is the reading code in C++:

    std::ifstream in("D:/p.txt", std::ios::binary | std::ios::in);
    std::string content( (std::istreambuf_iterator<char>(in) ), (std::istreambuf_iterator<char>()    ) );

The output I get:

    {"products": [{"id": 125, "label": "Bill├®l"}, {"id": 4, "label": "R├╣bin"}]}

Tried using CharToOemA :

   {"products": [{"id": 125, "label": "Billél"}, {"id": 4, "label": "Rùbin"}]}

My codepage should allow me to display accents in the console (I tried echoing such accents which yielded a perfectly good display). Both the input and output codepages for my c++ console is CP850 (IBM Internatinal Latin-1).

How can I get my code to output a correct accent in the console? I would ultimately need a cross-platform solution if possible.

Answer 1

If have UTF-8, and you output to a Window expecting ISO 8859-1, it's not going to work. If you have UTF-8 (which will be the case if the global locale is still the default "C"), then you can either change the window to code page 65001, or you must convert the encoding before outputting.

With regards to portability, there is no real solution; what you have to do depends on how the destination interprets the bytes you output. Under Windows, you can change the code page; under Unix systems (X Windows), it is the encoding of the font the window uses which matters. In both cases, they can be different for different windows on the same machine.

Answer 2

Have you tried using chcp 65001 which should switch the code page to UTF-8, according to MSDN. Also note that the default console font might not be able to display all UTF8 glyphs, I recommend using Lucida Console instead.