How to properly create a Map/Reduce Index for RavenDB in C#

c#, mapreduce, ravendb

Solution

You will need to combine several techniques to achieve this, but it is quite doable.

Here is an index that should work well for you.

public class MyIndex : AbstractMultiMapIndexCreationTask<MyIndex.ReduceResult>
{
    public class ReduceResult
    {
        public string Source { get; set; }
        public string Code { get; set; }
        public string Data { get; set; }
        public int Count { get; set; }
    }

    public MyIndex()
    {
        AddMap<MyDoc>(docs => from doc in docs
                              select new
                                     {
                                         Source = "Severity",
                                         doc.Severity.Code,
                                         doc.Severity.Data,
                                         Count = 1
                                     });

        AddMap<MyDoc>(docs => from doc in docs
                              select new
                                     {
                                         Source = "Facility",
                                         doc.Facility.Code,
                                         doc.Facility.Data,
                                         Count = 1
                                     });

        Reduce = results => from result in results
                            group result by new { result.Source, result.Code }
                            into g
                            select new
                            {
                                g.Key.Source,
                                g.Key.Code,
                                g.First().Data,
                                Count = g.Sum(x => x.Count)
                            };

        TransformResults = (database, results) =>
                           from result in results
                           group result by 0
                           into g
                           select new
                           {
                               Severity = g.Where(x => x.Source == "Severity")
                                           .ToDictionary(x => x.Data, x => x.Count),
                               Facility = g.Where(x => x.Source == "Facility")
                                           .ToDictionary(x => x.Data, x => x.Count)
                           };
    }
}

You also need a container class for the transformed result:

public class MyDocCounts
{
    public IDictionary<string, int> Severity { get; set; }
    public IDictionary<string, int> Facility { get; set; }
}

You would query it like this:

var result = session.Query<MyIndex.ReduceResult, MyIndex>()
                    .As<MyDocCounts>()
                    .ToList().First();

The `.ToList()` may seem redundant, but it's necessary because we are grouping in the transform.

A complete unit test is here. The output of which looks like this:

{
  "Severity": {
    "AAA": 20,
    "BBB": 20,
    "CCC": 20,
    "DDD": 20,
    "EEE": 20
  },
  "Facility": {
    "FFF": 20,
    "GGG": 20,
    "HHH": 20,
    "III": 20,
    "JJJ": 20
  }
}

Problem

I'm working on an app that uses RavenDB on the back end. It's my first time using Raven, and I'm struggling with Map/Reduce. I have been reading the doc's, but unfortunately I'm not getting anywhere in the process. Basically I have thousands of documents like this. ``` { ..... "Severity": { "Code": 6, "Data": "Info" }, "Facility": { "Code": 16, "Data": "Local Use 0 (local0)" }, ..... } ``` And out of it, I need to make a single query with output that looks like this. ``` {"Severity": [ {"Emergency":0}, {"Alert":0}, {"Critical":0}, {"Error":0}, {"Warning":0}, {"Notice":0}, {"Info":2711}, {"Debug":410} ], "Facility": [ {"Kernel Messages":0}, {"User-Level Messages":0}, {"Mail System":0}, {"System Daemons":0}, {"Security/Authorization Messages":0}, {"Internal Syslogd Messages":0}, {"Line Printer Subsystem":2711}, {"Network News Subsystem":410}, .... {"Local Use 0 (local0)": 2574}, ... ]} ``` Whereby the "Key" in the Severity/Facility Array is the `Data` portion of the above json data, and the "value" in the Severity/Facility Array is the document `Count`for each `Code` type. Example: Using the above data as a guideline, There are 2711 documents in my database with an `Info` severity. There are 410 documents in my database with a `Debug` severity. There are 2574 documents in my database with a `local0` facility. etc... What I'd like to do is generate the appropriate indexes when the app starts up (or check if they already exist), but I don't even know where to begin. note: the app needs to generate the index, it's not enough to just manually write it into the RavenDB Web UI.

Original source