Why Standard Extension Method on IEnumerables

Question

When i use a standard Extension Method on a List such as Where(...)

the result is always IEnumerable, and when you decide to do a list operation such as Foreach()

we need to Cast(not pretty) or use a ToList() extension method that

(maybe) uses a new List that consumes more memory (is that right?):

List<string> myList=new List<string>(){//some data};

(Edit: this cast on't Work)

myList.Where(p=>p.Length>5).Tolist().Foreach(...);

or

(myList.Where(p=>p.Length>5) as List<string>).Foreach(...);

Which is better code or is there a third way?

Edit: Foreach is a sample, Replace that with BinarySerach

myList.Where(p=>p.Length>5).Tolist().Binarysearch(...)

Answer 1

The as is definitely not a good approach, and I'd be surprised if it works.

In terms of what is "best", I would propose foreach instead of ForEach:

foreach(var item in myList.Where(p=>p.Length>5)) {
    ... // do something with item
}

If you desperately want to use list methods, perhaps:

myList.FindAll(p=>p.Length>5).ForEach(...);

or indeed

var result = myList.FindAll(p=>p.Length>5).BinarySearch(...);

but note that this does (unlike the first) require an additional copy of the data, which could be a pain if there are 100,000 items in myList with length above 5.

The reason that LINQ returns IEnumerable<T> is that this (LINQ-to-Objects) is designed to be composable and streaming, which is not possible if you go to a list. For example, a combination of a few where / select etc should not strictly need to create lots of intermediate lists (and indeed, LINQ doesn't).

This is even more important when you consider that not all sequences are bounded; there are infinite sequences, for example:

static IEnumerable<int> GetForever() {
    while(true) yield return 42;
}
var thisWorks = GetForever().Take(10).ToList();

as until the ToList it is composing iterators, not generating an intermediate list. There are a few buffered operations, though, like OrderBy, which need to read all the data first. Most LINQ operations are streaming.

Answer 2

If you want to make a simple foreach over a list, you can do like this:

foreach (var item in myList.Where([Where clause]))
{
    // Do something with each item.
}

Answer 3

One of the design goals for LINQ is to allow composable queries on any supported data type, which is achieved by having return-types specified using generic interfaces rather than concrete classes (such as IEnumerable<T> as you noted). This allows the nuts and bolts to be implemented as needed, either as a concrete class (e.g. WhereEnumerableIterator<T> or hoisted into a SQL query) or using the convenient yield keyword.

Additionally, another design philosophy of LINQ is one of deferred execution. Basically, until you actually use the query, no real work has been done. This allows potentially expensive (or infinite as Mark notes) operations to be completed only exactly as needed.

If List<T>.Where returned another List<T> it would potentially limit composition and would certainly hinder deferred execution (not to mention generate excess memory).

So, looking back at your example, the best way to use the result of the Where operator depends on what you want to do with it!

// This assumes myList has 20,000 entries
// if .Where returned a new list we'd potentially double our memory!
var largeStrings = myList.Where(ss => ss.Length > 100);
foreach (var item in largeStrings)
{
    someContainer.Add(item);
}

// or if we supported an IEnumerable<T>
someContainer.AddRange(myList.Where(ss => ss.Length > 100));

Answer 4

You can't cast (as) IEnumerable<string> to List<string>. IEnumerable evaluates items when you access those. Invoking ToList<string>() will enumerate all items in the collection and returns a new List, which is a bit of memory inefficiency and as well as unnecessary. If you are willing to use ForEach extension method to any collection its better to write a new ForEach extension method that will work on any collection.

public static void ForEach<T>(this IEnumerable<T> enumerableList, Action<T> action)
{
    foreach(T item in enumerableList)
    {
        action(item);
    }
}