Parse Strings in HIVE using Shell

问题:

I have a shell script that I use to parse a string variable into Hive, in order to filter my observations. I provide both the script and the hive code below.

In the following script I have a variable which has a string value and I try to parse it into hive, the example below:

Shell Script:

name1='"Maria Nash"' *(I use a single quote first and then a double)*

hive --hiveconf name=${name1} -f t2.hql

Hive code (t2.hql)

create table db.mytable as

SELECT *

FROM db.employees

WHERE emp_name='${hivevar:name}';

Conclusion

To be accurate, the final table is created but it does not contain any observation. The employees table contains observations which has emp_name "Maria Nash" though.

I think that I might not parse the string correctly from shell or I do not follow the correct syntax on how I should handle the parsed variable in the hive query.

I would appreciate your help!

回答1:

Use of the CLI is deprecated

you can use beeline from a shell script

it should look something like

beeline << EOF

!connect jdbc:hive2://host:port/db username password

select * 
from db.employees
where emp_name = "${1}"

EOF

assuming that $1 is the input from the script.

This is an example of how to do it rather than a production implementation. Generally,

1. Kerberos would be enabled so username and password wouldn't be there and a valid token would be available
2. Validate the input parameters.

Given that you can do it in a single line

beeline -u jdbc:hive2://hostname:10000  -f {full Path to Script} --hivevar {variable}={value}