猪谜题：将涉及的减速器重写为简单的猪脚本？

Question

有多个帐户id，每个帐户都有一个按用户名分组的时间戳。对于这些用户名组中的每一个，我想要所有对（最旧的帐户、其他帐户）。

我有一个java减速器可以做到这一点，我可以将它重写为一个简单的pig脚本吗？

架构：

{group:(username),A: {(id , create_dt)}

输入：

(batman,{(id1,100), (id2,200), (id3,50)})
(lulu  ,{(id7,100), (id9,50)})

所需输出：

(batman,{(id3,id1), (id3,id2)})
(lulu  ,{(id9,id7)})

Answer 1

似乎没人关心，但事情就是这样。你必须创建一个 UDF：

desired =   foreach my_input generate group as n, FIND_PAIRS(A) as pairs_bag;

并且 UDF：

public class FindPairs extends EvalFunc<DataBag> {
@Override
    public DataBag exec(Tuple input) throws IOException {
        Long pivotCreatedDate = Long.MAX_VALUE;
        Long pivot = null;

        DataBag accountsBag = (DataBag) input.get(0);
        for (Tuple account : accountsBag){
            Long accountId = Long.parseLong(account.get(0).toString());
            Long creationDate = Long.parseLong(account.get(4).toString());
            if (creationDate < pivotCreatedDate ) {
                // pivot is the one with the minimal creation_dt
                pivot = accountId;
                pivotCreatedDate = creationDate;
            }
        }

        DataBag allPairs = BagFactory.getInstance().newDefaultBag();
        if (pivot != null){
            for (Tuple account : accountsBag){
                Long accountId = Long.parseLong(account.get(0).toString());
                Long creationDate = Long.parseLong(account.get(4).toString());
                if (!accountId.equals(pivot)) {
                    // we don't want any self-pairs
                    Tuple output = TupleFactory.getInstance().newTuple(2);
                    if (pivot < accountId){
                            output.set(0, pivot.toString());
                            output.set(1, accountId.toString());
                    }
                    else {
                  output.set(0, accountId.toString());
                    output.set(1, pivot.toString());
                    }
                allPairs.add(output);
            }
        }               
        return allPairs;
}

如果你想玩得很好，请添加以下内容：

/**
 * Letting pig know that we emit a bag with tuples, each representing a pair of accounts
 */
@Override
public Schema outputSchema(Schema input) {
    try{
        Schema pairSchema = new Schema();
        pairSchema.add(new FieldSchema(null, DataType.BYTEARRAY));
        pairSchema.add(new FieldSchema(null, DataType.BYTEARRAY));
        return new Schema(
                new FieldSchema(null,
                new Schema(pairSchema), DataType.BAG));         
    }catch (Exception e){
            return null;
    }
}   

}