Pig: Get top n values per group

apache-pig, hadoop, hdfs

Solution

One approach is

records = LOAD '/user/nubes/ncdc/micro-tab/top.txt' AS (user:chararray,value:chararray,counter:int);
grpd = GROUP records BY user;

top3 = foreach grpd {
        sorted = order records by counter desc;
        top    = limit sorted 2;
        generate group, flatten(top);
};

Input is:

Alice   third   5 
Alice   first   11 
Alice   second  10
Alice   fourth  2
Bob second  20
Bob third   18
Bob first   21
Bob fourth  8

Output is:

(Alice,Alice,first,11)
(Alice,Alice,second,10
(Bob,Bob,first,21)
(Bob,Bob,second,20)

Problem

I have data that's already grouped and aggregated, it looks like so: ``` user value count ---- -------- ------ Alice third 5 Alice first 11 Alice second 10 Alice fourth 2 ... Bob second 20 Bob third 18 Bob first 21 Bob fourth 8 ... ``` For every user (Alice and Bob), I want retrieve their top n values (let's say 2), sorted terms of 'count'. So the desired output I want is this: ``` Alice first 11 Alice second 10 Bob first 21 Bob second 20 ``` How can I accomplish that?

Original source