How to make sure my created filedownload is UTF-8?

Question

i've made a download function to download messages to a CSV file (code is below). Now when i open it up in notepad or notepad++ i see this:

é NY ø ╬ ║► ░ ê ö

(and that is what is in the database btw)

Now, when i open it up in Ms-Excel it shows this:

é NY ø ╬ ║► ░ ê ö

When i open it up in notepad++, it says it's encoded in 'UTF8 without BOM'. When i encode it (in notepad++) to UTF-8, all goes well (that is, Excel shows the right chars too)

But how can i make sure that the file i create from my code is UTF-8?

This is my code:

public ActionResult DownloadPersonalMessages()
{    
    StringBuilder myCsv = new StringBuilder();
    myCsv.Append(new DownloadService().GetPersonalMessages());

    this.Response.ContentType = "text/csv";
    Response.AddHeader("content-disposition", "attachment; filename=PersonalMessages.csv");
    Response.ContentEncoding = Encoding.UTF8;
    Response.Write(myCsv.ToString());
    Response.Flush();
    Response.HeaderEncoding = Encoding.UTF8;
    return Content("");
}

Edit:

my function now returns a ByteArray with this conversion

UTF8Encoding encoding = new UTF8Encoding();
return encoding.GetBytes(str);

and my download is now this:

Response.AddHeader("Content-Disposition", "attachment; filename=PersonalMessages.csv");
return File(new DownloadService().GetPersonalMessages(), "text/csv");

Answer 1

You might want to try using the UTF8Encoding class. The constructor has a parameter that determines if it should provide the BOM or not. You'll probably have to use the GetBytes-method and write the string as a series of bytes in the response, and not convert it back into a .net string object.

Answer 2

Zareth's answer helped the OP, but it didn't actually answer the question. Here's the correct solution, from this other post:

public ActionResult Download()
{
    var data = Encoding.UTF8.GetBytes("some data");
    var result = Encoding.UTF8.GetPreamble().Concat(data).ToArray();
    return File(result, "application/csv", "foo.csv");
}

The byte-order mark (while not technically required for UTF8) clues certain programs (e.g. Excel >2007) in to the fact that you're using UTF8. You have to manually include it via the GetPreamble() method.

Answer 3

You could simplify your code a little:

public ActionResult DownloadPersonalMessages()
{
    StringBuilder myCsv = new StringBuilder();
    myCsv.Append(new DownloadService().GetPersonalMessages());
    Response.AddHeader("Content-Disposition", "attachment; filename=PersonalMessages.csv");
    return File(Encoding.UTF8.GetBytes(myCsv.ToString()), "text/csv");
}

As far as the UTF-8 encoding is concerned I am afraid the problem might be in this GetPersonalMessages method. You might need to return a stream or byte array which you could directly return as file.