Pass environment variables to Hive Transform or Ma

2019-07-20 16:58发布

I am trying to pass a custom environment variable to an executable (my-mapper.script in the example below) used in a Hive Transform eg:

SELECT
   TRANSFORM(x, y, z)
   USING 'my-mapper.script'
FROM
(
   SELECT
      x, y, z
   FROM
      table
)

I know in Hadoop streaming this can be achieved using

-cmdenv EXAMPLE_DIR=/home/example/dictionaries/

But I do not know how to do this in a Hive Transform/MapReduce.

Any ideas?

2条回答
兄弟一词,经得起流年.
2楼-- · 2019-07-20 17:06

Are you looking for something like this?

% hive -hiveconf CURRENT_DATE='2012-09-16' -f test.hql
查看更多
甜甜的少女心
3楼-- · 2019-07-20 17:25

You can wrap your script with a simple 2 line bash script to setup the environment. e.g

#!/bin/sh
export FOO=boo
my-mapper.script

And then use this script in the query

USING 'wrapper.sh'

my-mapper.script will see FOO (with value "boo") in the environment.

查看更多
登录 后发表回答