How to access Metastore from beeline?

2020-04-18 05:59发布

问题:

I need to do some SQL queries (as here) directly from Metasore.
PS: the commands SHOW/DESCRIBE are not enough.

How to enable access from it as database, or what the database name of Metastore? ... In nowadays (2019) it is possible?


NOTES

  • What is Metastore?
    For me is a very important element of the Hive architecture, final user need some access to it... "All Hive implementation need a metastore service, where it stores metadata. It is implemented using tables in relational database. By default, Hive uses built-in Derby SQL server", 1.
    Of course, you need in your context a "standard" Metastore. On my corporation's Hadoop cluster we are planning to standardize Metastore (local and long term standard), perhaps PostgreSQL and also an (PostgREST API for external consume of some SQL-Views from it).

  • The SQL definitions (table names, etc.) will be stable and Metastore queries will be reliable when Metastore is a long-term local standard.

  • The Metastore it is closely connected to Hive, where it is a Java API, but Metastore is also a standard RDBMS and offers standard connection (by SQL) for the external universe.
    PS: my interest on Metastore is in this external context.

回答1:

Spark-shell solution

Spark access Metastore under the hood, it have a first class metadata method, that returns a dataframe, the schema property, that expose names, types, etc. and offers getComment method.

See https://stackoverflow.com/a/57857021/287948



标签: hive beeline