How to create an external Hive table with column t

问题:

I want to create an external Hive table from a text file containing epoch in HDFS. Let's say the file is located at the /user/me/test.txt. Here's the file content:

1354183921
1354183922

I have Hive 0.8.1 installed and should be able to use type Timestamp, so I created the table:

hive> CREATE EXTERNAL TABLE test1 (epoch Timestamp)
      LOCATION '/user/me';

Then I queried the table:

SELECT * FROM test1;

and got the following exception:

Failed with exception java.io.IOException:java.lang.IllegalArgumentException: 
Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]

Have I missed anything when creating the external table? I'm confused because Hive (since 0.8) supports Timestamp in the format of Unix epoch in second: https://cwiki.apache.org/Hive/languagemanual-types.html#LanguageManualTypes-Timestamps

回答1:

The Timestamp string must be in the format specified in the error message and link, yyyy-mm-dd hh:mm:ss[.fffffffff]. To use data that contains epoch, you would have to define as a BIGINT and then use the built-in UDF, from_unixtime() to convert to a string representing the date. Then you could use the other built-in UDF date functions to manipulate this date.