How to parallel process inputs until some condition is true in C#

c#, parallel-processing, while-loop

Solution

Note that "collections" like the one you're showing are typically expose via `IEnumerable<T>`. If you have control over the API itself, I'd use `IEnumerable<T>` instead of the `GetNext()`-based iteration approach. If you don't, however, it's simple to perform the conversion...

I would wrap this API to expose it as an `IEnumerable<T>`. You could then use `Parallel.ForEach`:

private IEnumerable<T> EnumerateWidgets<T>(ICollectionWidget<T> widgets)
{
    T element = widgets.GetNext();
    while (element != null)
    {
        yield return element;
        element = widgets.GetNext();
    }
}

You could then use:

Parallel.ForEach(EnumerateWidgets(widgetCollection), widget =>
{
     // Process widget here
});

This will prevent threading issues while enumerating your widgets (as the enumerator will be single threaded), but allow you to process your collection in parallel.

Problem

I am working with an interface which sits on top of (for example) a `StreamReader` or `SqlDataReader`. The interface exposes a method, `GetNext()`, which returns an object if there are any left, or `null` if there are none left. ``` public interface ICollectionWidget<T> { T GetNext(); // Returns a T if there are any left, or null if there aren't } ``` I need to process each `T` returned by `GetNext()` in parallel, and stop processing when `GetNext()` returns `null`. I'm not quite sure how this is done (using TPL or whatever). I need a kind of parallel `while`! Obviously, I don't want any threads still processing to finish when I get a `null`, I just don't want to add any new processing - and then to drop out of the 'loop' when all the threads finish what they're doing. Can anyone help? Please let me know if my question doesn't make sense.

Original source