Writing data into CSV file in C#

Question

I am trying to write into a csv file row by row using C# language. Here is my function

string first = reader[0].ToString();
string second=image.ToString();
string csv = string.Format("{0},{1}\n", first, second);
File.WriteAllText(filePath, csv);

The whole function runs inside a loop, and every row should be written to the csv file. In my case next row overwrites the existing row and in the end I am getting only single record in the csv file which is last one. How can I write all the rows in the csv file.

Answer 1

Writing csv files by hand can be difficult because your data might contain commas and newlines. I suggest you use an existing library instead.

This question mentions a few options.

Are there any CSV readers/writer libraries in C#?

Answer 2

I would highly recommend you to go the more tedious route. Especially if your file size is large.

using(var w = new StreamWriter(path))
{
    for( /* your loop */)
    {
        var first = yourFnToGetFirst();
        var second = yourFnToGetSecond();
        var line = string.Format("{0},{1}", first, second);
        w.WriteLine(line);
        w.Flush();
    }
}

File.AppendAllText() opens a new file, writes the content and then closes the file. Opening files is a much resource-heavy operation, than writing data into open stream. Opening\closing a file inside a loop will cause performance drop.

The approach suggested by Johan solves that problem by storing all the output in memory and then writing it once. However (in case of big files) you program will consume a large amount of RAM and even crash with OutOfMemoryException

Another advantage of my solution is that you can implement pausing\resuming by saving current position in input data.

upd. Placed using in the right place

Answer 3

Simply use AppendAllText instead:

File.AppendAllText(filePath, csv);

The only downside of the AppendAllText is that it will throw error when file does not exist, so this must be checked

Sorry, blonde moment before reading the documentation. Anyway, the WriteAllText method overwrites anything that was previously written in the file, if the file exists.

Note that your current code is not using proper new lines, for example in Notepad you'll see it all as one long line. Change the code to this to have proper new lines:

string csv = string.Format("{0},{1}{2}", first, image, Environment.NewLine);

Answer 4

Instead of reinventing the wheel a library could be used. CsvHelper is great for creating and reading csv files. It's read and write operations are stream based and therefore also support operations with a big amount of data.

---

You can write your csv like the following.

using(var textWriter = new StreamWriter(@"C:\mypath\myfile.csv"))
{
    var writer = new CsvWriter(textWriter);
    writer.Configuration.Delimiter = ",";

    foreach (var item in list)
    {
        writer.WriteField( "a" );
        writer.WriteField( 2 );
        writer.WriteField( true );
        writer.NextRecord();
    }
}

---

As the library is using reflection it will take any type and parse it directly.

public class CsvRow
{
    public string Column1 { get; set; }
    public bool Column2 { get; set; }

    public CsvRow(string column1, bool column2)
    {
        Column1 = column1;
        Column2 = column2;
    }
}

IEnumerable<CsvRow> rows = new [] {
    new CsvRow("value1", true),
    new CsvRow("value2", false)
};
using(var textWriter = new StreamWriter(@"C:\mypath\myfile.csv")
{
    var writer = new CsvWriter(textWriter);
    writer.Configuration.Delimiter = ",";
    writer.WriteRecords(rows);
}

value1,true

value2,false

---

If you want to read more about the librarys configurations and possibilities you can do so here.

Answer 5

One simple way to get rid of the overwriting issue is to use File.AppendText to append line at the end of the file as

void Main()
{
    using (System.IO.StreamWriter sw = System.IO.File.AppendText("file.txt"))
    {          
        string first = reader[0].ToString();
        string second=image.ToString();
        string csv = string.Format("{0},{1}\n", first, second);
        sw.WriteLine(csv);
    }
} 

Answer 6

Handling Commas

For handling commas inside of values when using string.Format(...), the following has worked for me:

var newLine = string.Format("\"{0}\",\"{1}\",\"{2}\"",
                              first,
                              second,
                              third                                    
                              );
csv.AppendLine(newLine);

So to combine it with Johan's answer, it'd look like this:

//before your loop
var csv = new StringBuilder();

//in your loop
  var first = reader[0].ToString();
  var second = image.ToString();
  //Suggestion made by KyleMit
  var newLine = string.Format("\"{0}\",\"{1}\"", first, second);
  csv.AppendLine(newLine);  

//after your loop
File.WriteAllText(filePath, csv.ToString());

---

Returning CSV File

If you simply wanted to return the file instead of writing it to a location, this is an example of how I accomplished it:

From a Stored Procedure

public FileContentResults DownloadCSV()
{
  // I have a stored procedure that queries the information I need
  SqlConnection thisConnection = new SqlConnection("Data Source=sv12sql;User ID=UI_Readonly;Password=SuperSecure;Initial Catalog=DB_Name;Integrated Security=false");
  SqlCommand queryCommand = new SqlCommand("spc_GetInfoINeed", thisConnection);
  queryCommand.CommandType = CommandType.StoredProcedure;

  StringBuilder sbRtn = new StringBuilder();

  // If you want headers for your file
  var header = string.Format("\"{0}\",\"{1}\",\"{2}\"",
                             "Name",
                             "Address",
                             "Phone Number"
                            );
  sbRtn.AppendLine(header);

  // Open Database Connection
  thisConnection.Open();
  using (SqlDataReader rdr = queryCommand.ExecuteReader())
  {
    while (rdr.Read())
    {
      // rdr["COLUMN NAME"].ToString();
      var queryResults = string.Format("\"{0}\",\"{1}\",\"{2}\"",
                                        rdr["Name"].ToString(),
                                        rdr["Address"}.ToString(),
                                        rdr["Phone Number"].ToString()
                                       );
      sbRtn.AppendLine(queryResults);
    }
  }
  thisConnection.Close();

  return File(new System.Text.UTF8Encoding().GetBytes(sbRtn.ToString()), "text/csv", "FileName.csv");
}

From a List

/* To help illustrate */
public static List<Person> list = new List<Person>();

/* To help illustrate */
public class Person
{
  public string name;
  public string address;
  public string phoneNumber;
}

/* The important part */
public FileContentResults DownloadCSV()
{
  StringBuilder sbRtn = new StringBuilder();

  // If you want headers for your file
  var header = string.Format("\"{0}\",\"{1}\",\"{2}\"",
                             "Name",
                             "Address",
                             "Phone Number"
                            );
  sbRtn.AppendLine(header);

  foreach (var item in list)
  {
      var listResults = string.Format("\"{0}\",\"{1}\",\"{2}\"",
                                        item.name,
                                        item.address,
                                        item.phoneNumber
                                       );
      sbRtn.AppendLine(listResults);
    }
  }

  return File(new System.Text.UTF8Encoding().GetBytes(sbRtn.ToString()), "text/csv", "FileName.csv");
}

Hopefully this is helpful.