In Apache Pig how can I serialise columns into row

问题:

In Apache Pig I want to serialise columns held in a variable into rows. More specifically:

The data, loaded into the variable, look (via DUMP) like

(val1a, val2a,.... )
(val1b, val2b,val3b,.... )
(val1c, val2c,.... )
.
.
.

and I want to transform this into

(val1a)
(val2a)
.
.
.
(val1b)
(val2b)
(val3b)
.
.
.
(val1c)
(val2c)
.
.
.

So, each column has to be "serialised" into rows and then these rows are added subsequently. Please note: I do not necessarily know how many columns are in each row.

How can I do this in Pig Latin? It would be easy in, e.g., Python, but I don't know how to do it in Pig. I tried different foreach ... generate constructs, but could not make it work.

回答1:

One way to unfold tuples and create multiple tuples, each containing one field:

$ cat data.txt
val1a,val2a,val3a,val4a,val5a,val6a,val7a
val1b,val2b,val3b
val1c,val2c

A = load 'data.txt' using PigStorage(',');
B = foreach A generate FLATTEN(TOBAG(*));
dump B;

(val1a)
(val2a)
(val3a)
(val4a)
(val5a)
(val6a)
(val7a)
(val1b)
(val2b)
(val3b)
(val1c)
(val2c)

Note: You might also check these similar posts:
Splitting a tuple into multiple tuples in Pig
Pivot table with Apache Pig