I am trying to pass a custom environment variable to an executable (my-mapper.script in the example below) used in a Hive Transform eg:
SELECT
TRANSFORM(x, y, z)
USING 'my-mapper.script'
FROM
(
SELECT
x, y, z
FROM
table
)
I know in Hadoop streaming this can be achieved using
-cmdenv EXAMPLE_DIR=/home/example/dictionaries/
But I do not know how to do this in a Hive Transform/MapReduce.
Any ideas?
Are you looking for something like this?
You can wrap your script with a simple 2 line bash script to setup the environment. e.g
And then use this script in the query
my-mapper.script will see FOO (with value "boo") in the environment.