File.ReadAllLines or Stream Reader [duplicate]

Question

This question already has an answer here:

• What's the fastest way to read a text file line-by-line? 8 answers

We can read file either by using StreamReader or by using File.ReadAllLines.

For example I want to load each line into a List or string[] for further manipulation on each line.

string[] lines = File.ReadAllLines(@"C:\\file.txt");

foreach(string line in lines)
{
     //DoSomething(line);
}

or

using (StreamReader reader = new StreamReader("file.txt"))
{
    string line;

    while ((line = reader.ReadLine()) != null)
    {
       //DoSomething(line); or //save line into List<string>
    }
}

//if list is created loop through list here 

Application come across different size of text file. Which could grow from few KBs to MBs occasionally.

My question is that which one is preferred way and why one should be preferred over other?

Answer 1

If you want to process each line of a text file without loading the entire file into memory, the best approach is like this:

foreach (var line in File.ReadLines("Filename"))
{
    // ...process line.
}

This avoids loading the entire file, and uses an existing .Net function to do so.

However, if for some reason you need to store all the strings in an array, you're best off just using File.ReadAllLines() - but if you are only using foreach to access the data in the array, then use File.ReadLines().

Answer 2

The StreamReader read the file line by line, it will consume less memory. whereas, File.ReadAllLines read all lines at once and store it into string[], it will consume more memory. And if that string[] is larger than int.maxvalue then that will produce memory overflow(limit of 32bit OS).

So, for bigger files StreamReader will be more efficient.

Answer 3

Microsoft uses a StreamReader in File.ReadAllLines:

    private static String[] InternalReadAllLines(String path, Encoding encoding)
    {
        Contract.Requires(path != null);
        Contract.Requires(encoding != null);
        Contract.Requires(path.Length != 0);

        String line;
        List<String> lines = new List<String>();

        using (StreamReader sr = new StreamReader(path, encoding))
            while ((line = sr.ReadLine()) != null)
                lines.Add(line);

        return lines.ToArray();
    }