passing bash array to python list

Question

I'm trying to pass an array from bash to python using the old getenv method however I keep getting this error:

./crcFiles.sh: line 7: export: `0021': not a valid identifier
Traceback (most recent call last):
  File "/shares/web/vm3618/optiload/prog/legalLitres.py", line 30, in <module>
    for i in mdcArray.split(' '):
AttributeError: 'NoneType' object has no attribute 'split'

could someone please explain why the $mdcNo isn't passing from bash to python successfully?

Code .sh:

#!/bin/bash

mdcNo=('0021' '0022' '0036' '0055' '0057' '0059' '0061' '0062' '0063' '0065' '0066' '0086' '0095' '0098' '0106' '0110' '0113' '0114' '0115' '0121' '0126' '0128' '0135' '0141' '0143' '0153' '0155' '0158')

localDIR=/shares/web/vm3618/optiload/prog

export mdcNo

$localDIR/legalLitres.py


for i in "${mdcNo[@]}"
do
echo $i
cp $localDIR/MDC$i/*/QqTrkRec.txt $localDIR/crccalc/.
cd $localDIR/crccalc
./crccalc.py QqTrkRec.txt
cp $localDIR/crccalc/QqTrkRec.txt $localDIR/MDC$i/.
done

code .py:

#!/usr/bin/python

import glob
import os

mdcArray = os.getenv('mdcNo')

#Legal Litres that hex and decimal
legalLitresHex = "47E0"
legalLitresTxt = '18,400'

# file name and Legal Litres header
legalHeader = ":00F0:"
hexFile = "QqTrkRec.txt"

# insert comment to explain change
comment = "#\n#  2015 Nov 20:  Legal Litres changed to 18,400\n#\n"
commentFlag0 = "#  SetDATA"
commentFlag1 = "# SetDATA"

try:
    for i in mdcArray.split(' '):


        line = ""

        Qqfile = glob.glob("/shares/web/vm3618/optiload/prog/MDC"+i+"/*/"+hexFile) 
        outFile = Qqfile[0]+".new"

        print i

Answer 1

When you export a variable from the shell, what you are really doing is adding it to the POSIX "environment" array that all child processes inherit. But the POSIX environment is a flat array of name=value strings; it cannot itself contain arrays. So Bash doesn't even attempt to put arrays there. It will let you export an array variable, and doing so even sets the "exported" flag on that variable, but the environment is not touched. You can verify this fact by running env or a new copy of bash and looking for the "exported" variable:

$ export myArr=(this is an array)
$ bash -c 'echo "${myArr[@]}"'

$

(Some other array-having shells, notably ksh, will actually export an array variable to the environment, but the exported value will consist of only the first element of the array.)

If you want to pass a shell array to the Python script, your best bet is to do so as command line arguments. If you run the Python script like this:

python code.py "${mdcNo[@]}"

... then the Python code can just loop over sys.argv, which is always a list. (Specifically, the passed-in array will be the slice sys.argv[1:], since sys.argv[0] is always set to the name of the script itself.)

If that's not an option, then you'll have to set the environment variable to a string with some delimiter between elements and split it inside the Python code. Something like this..

Bash:

export mdcList='0021,0022,0036,0055,0057,0059,0061,0062,0063,0065,0066,0086,0095,0098,0106,0110,0113,0114,0115,0121,0126,0128,0135,0141,0143,0153,0155,0158'

Or you can build the string up from the array:

export mdcList=${mdcNo[0]}
for i in "${mdcNo[@]:1}"; do
   mdcList+=,$i
done

Either way, the Python script can recover the array as a list like this:

mdc_no = os.getenv('mdcList').split(',')

If your array elements aren't just numbers, you can replace the comma with something less likely to show up in an element; the traditional choice would be the ASCII Unit Separator (U+001F, $'\x1f' in Bash, '\x1f' in Python).

Answer 2

I think Mark Reed already gave you a very good explanation and solution. Nevertheless, have you considered using python's argparse?

#!/usr/bin/env python
import argparse


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('stuff', nargs='+')
    args = parser.parse_args()
    print args.stuff

if __name__ == '__main__':
    main()

Use:

$ mdcNo=('0021' '0022' '0036' '0055' '0057' '0059' '0061' '0062' '0063' '0065' '0066' '0086' '0095' '0098' '0106' '0110' '0113' '0114' '0115' '0121' '0126' '0128' '0135' '0141' '0143' '0153' '0155' '0158')
$ python argp.py "${mdcNo[@]}"
['0021', '0022', '0036', '0055', '0057', '0059', '0061', '0062', '0063', '0065', '0066', '0086', '0095', '0098', '0106', '0110', '0113', '0114', '0115', '0121', '0126', '0128', '0135', '0141', '0143', '0153', '0155', '0158']