Pass environment variables to Hive Transform or Ma

问题:

I am trying to pass a custom environment variable to an executable (my-mapper.script in the example below) used in a Hive Transform eg:

SELECT
   TRANSFORM(x, y, z)
   USING 'my-mapper.script'
FROM
(
   SELECT
      x, y, z
   FROM
      table
)

I know in Hadoop streaming this can be achieved using

-cmdenv EXAMPLE_DIR=/home/example/dictionaries/

But I do not know how to do this in a Hive Transform/MapReduce.

Any ideas?

回答1:

You can wrap your script with a simple 2 line bash script to setup the environment. e.g

#!/bin/sh
export FOO=boo
my-mapper.script

And then use this script in the query

USING 'wrapper.sh'

my-mapper.script will see FOO (with value "boo") in the environment.

回答2:

Are you looking for something like this?

% hive -hiveconf CURRENT_DATE='2012-09-16' -f test.hql