Hive: Add partitions for existing folder structure

2020-04-20 08:32发布

问题:

I have a folder structure in HDFS like below. However, no partitions were actually created on the table using the ALTER TABLE ADD PARTITION commands, even though the folder structure was setup as if the table had partitions.

How can I automatically add all the partitions to the Hive table? (Hive 1.0, external table)

/user/frank/clicks.db
  /date=20190401
    /file0004.csv
  /date=20190402
    /file0009.csv
  /date=20190501
    /file0000.csv
    /file0001.csv
...etc

回答1:

Use msck repair table command:

MSCK [REPAIR] TABLE tablename;

or

ALTER TABLE tablename RECOVER PARTITIONS;

if you are running Hive on EMR.

Read more details about both commands here: RECOVER PARTITIONS