Fastest (portable) way to split an array in C#

c#, optimization

Solution

I believe the problem is, that you are doing a lot of complex operations in loop. This code removes all the operations except single addition and comparison inside a loop. Other complex stuff happens only when split is detected or at end of an array.

Also, it is hard to tell what kind of data you run your tests with, so this is only guesswork.

public static unsafe Segment[] Split2(byte[] _src, byte value)
{
    var _ln = _src.Length;

    if (_ln == 0) return new Segment[] { };

    fixed (byte* src = _src)
    {
        var segments = new LinkedList<Segment>(); // Segment[c];

        byte* last = src;
        byte* end = src + _ln - 1;
        byte lastValue = *end;
        *end = value; // value-termination

        var cur = src;
        while (true)
        {
            if (*cur == value)
            {
                int begin = (int) (last - src);
                int length = (int) (cur - last + 1);
                segments.AddLast(new Segment(_src, begin, length));

                last = cur + 1;

                if (cur == end)
                {
                    if (lastValue != value)
                    {
                        *end = lastValue;
                    }
                    break;
                }
            }
            cur++;
        }

        return segments.ToArray();
    }
}

Edit: Fixed code, so it returns correct results.

Problem

I'm writing a fully managed Mercurial library (to be used in a fully managed Mercurial Server for Windows, coming soon), and one of the most severe performance problems I'm coming across is, strangely enough, splitting an array in parts. The idea is as follows: there's a byte array with size ranging from several hundred bytes to up to a megabyte and all I need to do with it is to split it in parts delimited by, in my specific case, `\n` characters. Now what dotTrace shows me is that my "optimized" version of `Split` (the code is correct, here's the naive version I began with) takes up 11 seconds for 2,300 calls (there's an obvious performance hit introduced by the dotTrace itself, but everything's up to scale). Here are the numbers: - `unsafe` version: `11 297` ms for `2 312` calls - managed ("naive") version: `20 001` ms for `2 312` calls So here goes: what will be the fastest (preferably portable, meaning supporting both x86 and x64) way to split an array in C#.

Original source