PIG - retrieve data from XML using XPATH

2019-08-09 10:13发布

问题:

I have n number of these type of xml files.

<students roll_no=1>
    <name>abc</name>
    <gender>m</gender>
    <maxmarks>
        <marks>
            <year>2014</year>
            <maths>100</maths>
            <english>100</english>
            <spanish>100</spanish>
        <marks>
        <marks>
            <year>2015</year>
            <maths>110</maths>
            <english>110</english>
            <spanish>110</spanish>
        <marks>
    </maxmarks>
    <marksobt>
        <marks>
            <year>2014</year>
            <maths>90</maths>
            <english>95</english>
            <spanish>82</spanish>
        <marks>
        <marks>
            <year>2015</year>
            <maths>94</maths>
            <english>98</english>
            <spanish>02</spanish>
        <marks>
    </marksobt>
</Students>

I need output like

roll_no name    gender  year    eng_max_marks   maths_max_marks spanish_max_marks
1       abc     m       2014    100                 100          100
1       abc     m       2015    110                 110          110

I am able to retrieve marks row wise in single statement but not able to extract roll_no and name with this.

A =  LOAD 'student.xml' using org.apache.pig.piggybank.storage.XMLLoader('marks') as (x:chararray);

B = FOREACH A GENERATE XPath(x, 'marks/year'), XPath(x, 'marks/english'), XPath(x, 'marks/math'), XPath(x, 'marks/spanish');

This return

year    eng_max_marks   maths_max_marks spanish_max_marks
2014    100              100             100
2015    110              110             110

I can extract both the chunks but not getting how to join other fields. I can't use across join because I have n number of other files.

Let's forger attribute name (roll_no) for now. How can I extract the rest of nodes

name    gender  year    eng_max_marks   maths_max_marks spanish_max_marks
abc      m     2014      100             100             100
abc      m     2015      110             110             110

I don't want to use marks(1)/english approach because this nodes can also vary and don't want to adopt any dirty approach.

Any pointers????