I am using the below code to parse xml data in Hive. In my xml data, a few tags are repeating so I am using the brickhouse jar and lateral view to parse the tags and place in Hive tables. But when I am executing my code, I am getting an error. Please help as I am not able to understand what I am doing wrong.
Code:
add jar /home/cloudera/brickhouse-0.5.5.jar;
CREATE TEMPORARY FUNCTION numeric_range AS 'brickhouse.udf.collect.NumericRange';
CREATE TEMPORARY FUNCTION array_index AS 'brickhouse.udf.collect.ArrayIndexUDF';
add jar /home/cloudera/hivexmlserde-1.0.5.3.jar;
set hive.exec.mode.local.auto=false;
DROP TABLE IF EXISTS medinfo2;
create table medinfo2 as
select array_index(statusCode,n) AS statusCode,
array_index(startTime,n) AS startTime,
array_index(endTime,n) AS endTime,
array_index(strengthValue,n) AS strengthValue,
array_index(strengthUnits,n) AS strengthUnits
from medications_info7 lateral view numeric_range(size( statusCode )) n1 as n;
Error:
Caused by: java.lang.IndexOutOfBoundsException: Index: 7, Size: 7 at java.util.ArrayList.rangeCheck(ArrayList.java:635) at java.util.ArrayList.get(ArrayList.java:411) at com.ibm.spss.hive.serde2.xml.objectinspector.XmlListObjectInspector.getListElement(XmlListObjectInspector.java:79) at brickhouse.udf.collect.ArrayIndexUDF.evaluate(ArrayIndexUDF.java:59) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:186) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorHead._evaluate(ExprNodeEvaluatorHead.java:44) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65) at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:77) ... 25 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Stage-Stage-1: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec
Sample:
<document>
<code>10160-0</code>
<entryInfo>
<statusCode>completed</statusCode>
<startTime>20110729</startTime>
<endTime>20110822</endTime>
<strengthValue>24</strengthValue>
<strengthUnits>h</strengthUnits>
</entryInfo>
<entryInfo>
<statusCode>completed</statusCode>
<startTime>20120130</startTime>
<endTime>20120326</endTime>
<strengthValue>12</strengthValue>
<strengthUnits>h</strengthUnits>
</entryInfo>
<entryInfo>
<statusCode>completed</statusCode>
<startTime>20100412</startTime>
<endTime>20110822</endTime>
<strengthValue>8</strengthValue>
<strengthUnits>d</strengthUnits>
</entryInfo>
</document>
My actual sample is huge in size and contains a lot of these tags which are repeated.