apache Pig trying to get max count in each group

问题:

I have data of format in pig

{(group, productId, count)}.

Now I want to get maximum count in each group and the output might look as follows

{(group, productId, maxCount)}. Here is the sample input data

• (south America,prod1, 45),(south America,prod2, 36), (latin america, prod1, 48),(latin america, prod5,35)

here is the output for this input look like

1. (south america, prod1, 45)
2. (North America, prod2, 36)
3. (latin america, prod1, 48)

can someone help me on this.

回答1:

Based on your sample input data, this should do the trick:

data = load 'sf.csv' using PigStorage(',') as (country:chararray, product:chararray, c:int);
g = group data by country;
result = foreach g {
    prods = order data by c desc;
    top_prods = limit prods 1;
    generate flatten(top_prods);
}
dump result;

This groups the input by first column, then in the nested foreach it orders the products per group by count, then takes the first (highest count).

Output:

(latin america,prod1,48)
(south America,prod1,45)