Python: script to make multiple bash scripts

Question

I have a file, called list.txt, which is a list of names of files:

input1.txt
input2.txt
input3.txt

I want to build a python script that will make one file for each of these filenames. More precisely, I need it to print some text, incorporate the name of the file and save it to a unique .sh file. My script is as follows:

import os
os.chdir("/Users/user/Desktop/Folder")

with open('list2.txt','r') as f:    
    lines = f.read().split(' ')

for l in lines:
    print "#!/bin/bash\n#BSUB -J",l+".sh","\n#BSUB -o /scratch/DBC/user/"+l+".sh.out\n#BSUB -e /scratch/DBC/user/"+l+".sh.err\n#BSUB -n 1\n#BSUB -q normal\n#BSUB -P DBCDOBZAK\n#BSUB -W 168:00\n"
    print "cd /scratch/DBC/user\n"
    print 'grep "input"',l+" > result."+l+".txt"

    with open('script{}.sh'.format(l), 'w') as output:
        output.write(l)

I have a few issues:

• The output files just contain the name of the file - not the content I have printed.

To be clear my output files (I should have 3) should look like this:

#!/bin/bash
#BSUB -J input3.sh 
#BSUB -o /scratch/DBC/user/input1.sh.out
#BSUB -e /scratch/DBC/user/input3.sh.err
#BSUB -n 1
#BSUB -q normal
#BSUB -P DBCDOBZAK
#BSUB -W 168:00

cd /scratch/DBC/user
grep "input" input3 > result.input3.txt

I am very new to Python so I am assuming there could be much simpler ways of printing and inserting my text. Any help would be much appreciated.

UPDATE I have now made the following script, which nearly works.

import os
os.chdir("/Users/user/Desktop/Folder")

with open('list.txt','r') as f:
    lines = f.read().split('\n')

for l in lines:
    header = "#!/bin/bash \n#BSUB -J %s.sh \n#BSUB -o /scratch/DBC/user/%s.sh.out \n#BSUB -e /scratch/DBC/user/%s.sh.err \n#BSUB -n 1 \n#BSUB -q normal \n#BSUB -P DBCDOBZAK \n#BSUB -W 168:00\n"%(l,l,l)
    script = "cd /scratch/DBC/user\n"
    script2 = 'grep "input" %s > result.%s.txt\n'%(l,l)
    all= "\n".join([header,script,script2])

    with open('script_{}.sh'.format(l), 'w') as output:
        output.write(all)

The problem I still have is that this creates 4 scripts, not 3 as I expected: script_input1.sh, script_input2.sh, script_input3.sh and script_sh. This last one, script_sh just has the printed text but nothing where the "input" text would be. I think this is because my list.txt file has a "\n" at the end of it? However, I looked and there really isn't. Is there a way around this? Maybe I can use some kind of length function?

Answer 1

So, answering in order:

1) Can you detail this issue? You count 4 txt files bu you got just 3 different scripts generated by your code?

2) Sure, you need to create a var, not just using the print statement 3) Just change permissions

So, to summurize, I'd use this approach:

import os
for i, file in enumerate(os.listdir("/Users/user/Desktop/Folder")):
  if "input" in file:
    with open(file) as f:
        lines = f.readlines()

        for l in lines:
            data = ""
            data += "#!/bin/bash\n#BSUB -J",l+".sh","\n#BSUB -o /scratch/DBC/user/"+l+".sh.out\n#BSUB -e /scratch/DBC/user/"+l+".sh.err\n#BSUB -n 1\n#BSUB -q normal\n#BSUB -P DBCDOBZAK\n#BSUB -W 168:00\n"
            data += "cd /scratch/DBC/user\n"
            data += 'grep "input"'+l+" > result."+l+".txt"

            with open('script%s.sh'%i, 'w') as output:
                output.write(data)
            os.chmod("script%s.sh'%i", 700)

By the way, my code it's just a guess. I think you should be more explicit when stating what's your issue. I didn't understood what you wanna achieve.

Answer 2

Another solution that, instead of creating files

script1.sh
script2.sh
script3.sh

etc. would be to create one file file script.sh

#!/bin/bash
#BSUB -J "$1".sh 
#BSUB -o /scratch/DBC/user/"$1".sh.out
#BSUB -e /scratch/DBC/user/"$1".sh.err
#BSUB -n 1
#BSUB -q normal
#BSUB -P DBCDOBZAK
#BSUB -W 168:00

cd /scratch/DBC/user
grep "input" "$1" > result."$1".txt

and run it using

script.sh input1
script.sh input2
script.sh input3