Using ToList() on Enumerable LINQ query results for large data sets - Efficiency Issue?
.net-4.0, c#, linq, optimization, type-conversion
Solution
Well, it creates a copy of the data. That could be inefficient - but it depends on what's going on. If you need a `List<T>` at the end, `List<T>` is usually going to be close to as efficient as you'll get. The one exception to that is if you're going to just do a conversion and the source is already a list - then using `ConvertAll` will be more efficient, as it can create the backing array of the right size to start with.
If you only need to stream the data - e.g. you're just going to do a `foreach` on it, and taking actions which don't affect the original data sources - then calling `ToList` is definitely a potential source of inefficiency. It will force the whole of `list1` to be evaluated - and if that's a lazily-evaluated sequence (e.g. "the first 1,000,000 values from a random number generator") then that's not good. Note that as you're doing a join, `list2` will be evaluated anyway as soon as you try to pull the first value from the sequence (whether that's in order to populate a list or not).
You might want to read my Edulinq post on `ToList` to see what's going on - at least in one possible implementation - in the background.
Problem
I've been making a lot of use of LINQ queries in the application I'm currently writing, and one of the situations that I keep running into is having to convert the LINQ query results into lists for further processing (I have my reasons for wanting lists). I'd like to have a better understanding of what happens in this list conversion in case there are inefficiencies since I've used it repeatedly now. So, given I execute a line line like this: ``` var matches = (from x in list1 join y in list2 on x equals y select x).ToList(); ``` Questions: Is there any overhead here aside from the creation of a new list and its population with references to the elements in the Enumerable returned from the query? Would you consider this inefficient? Is there a way to get the LINQ query to directly generate a list to avoid the need for a conversion in this circumstance?