Audit hive table

问题:

I have a hive table lets say it as table A. My requirement is to capture all the DML and DDL operations on table A in table B. Is there any way to capture the same?

Thanks in advance..

回答1:

I have not come across any such tool however Cloudera Navigator helps to manage it. Refer the detailed documentation.

Cloudera Navigator

Cloudera Navigator auditing supports tracking access to:

1. HDFS entities accessed by HDFS, Hive, HBase, Impala, and Solr services
2. HBase and Impala
3. Hive metadata
4. Sentry
5. Solr
6. Cloudera Navigator Metadata Server

Alternatively, if you are not using cloudera distribution, you can still access hive-metastore log file under /var/log/hive/hadoop-cmf-hive-HIVEMETASTORE.log.out and check the changes applied to the different table.

回答2:

I haven't used Apache atlas yet, but from the documentation, it looks like they have Audit store and hive bridge. That works for operational events as well.

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/atlas-overview/content/apache_atlas_features.html