subprocess.call using cygwin instead of cmd on Win

Question

I'm programming on Windows 7 and in one of my Python projects I need to call bedtools, which only works with Cygwin on Windows. I'm new to Cygwin, installed the default version + everything needed for bedtools and then used Cygwin to install bedtools by using make as described in the installation instructions.

$ tar -zxvf BEDTools.tar.gz
$ cd BEDTools-<version>
$ make

When I use the Cygwin terminal to call it manually like below, it works without problem and the output file contains the correct result.

bedtools_exe_path intersect -a gene_bed_file -b snp_bed_file -wa -wb > output_file

But when I use subprocess.call in my program it seems to use Windows cmd instead of Cygwin, which doesn't work.

arguments = [bedtools_exe_path, 'intersect', '-a', gene_bed_file, '-b',
             snp_bed_file, '-wa', '-wb', '>', output_file]
return_code = suprocess.call(arguments)

Results in no output file and a return code of 3221225781.

---

arguments = [bedtools_exe_path, 'intersect', '-a', gene_bed_file, '-b',
             snp_bed_file, '-wa', '-wb', '>', output_file]
return_code = suprocess.call(arguments, shell=True)

Results in an empty output file and a return code of 3221225781.

---

cygwin_bash_path = 'D:/Cygwin/bin/bash.exe'
arguments = [cygwin_bash_path, bedtools_exe_path, 'intersect', '-a', gene_bed_file, '-b',
             snp_bed_file, '-wa', '-wb', '>', output_file]
return_code = suprocess.call(arguments)

Results in no output file, a return code of 126 and

D:/BEDTools/bin/bedtools.exe: D:/BEDTools/bin/bedtools.exe: cannot execute binary file

---

arguments = [cygwin_bash_path, bedtools_exe_path, 'intersect', '-a', gene_bed_file, '-b',
             snp_bed_file, '-wa', '-wb', '>', output_file]
return_code = suprocess.call(arguments, shell=True)

Results in an empty output file, a return code of 126 and

D:/BEDTools/bin/bedtools.exe: D:/BEDTools/bin/bedtools.exe: cannot execute binary file

---

Any ideas how I can get it to work?

Answer 1

The following works without a problem. The " does not need to be escaped.

argument = 'sh -c \"' + bedtools_exe_path + ' intersect -a ' + gene_bed_file + 
           ' -b ' + snp_bed_file + ' -wa -wb\"'

with open(output_file, 'w') as file:
    subprocess.call(argument, stdout=file)

Using the following works as well:

argument = 'bash -c \"' + bedtools_exe_path + ' intersect -a ' + gene_bed_file + 
           ' -b ' + snp_bed_file + ' -wa -wb\"'

with open(output_file, 'w') as file:
    subprocess.call(argument, stdout=file)

With:

bedtools_exe_path = 'D:/BEDTools/bin/bedtools.exe'
gene_bed_file = 'output/gene.csv'
snp_bed_file = 'output/snps.csv'
output_file = 'output/intersect_gene_snp.bed'

Using the path to the cygwin bash.exe (D:/Cygwin/bin/bash.exe) instead of bash or sh does not work.

Thank you, eryksun, Padraic Cunningham and J.F. Sebastian.

Answer 2

Imagine you want to run a Linux command from Windows. You could install Linux into a VM and run commands via ssh (Putty/plink on Windows):

#!/usr/bin/env python
import subprocess

cmd = [r'C:\path\to\plink.exe', '-ssh', 'user@vm_host', '/path/to/bedtools']
with open('output', 'wb', 0) as file:
    subprocess.check_call(cmd, stdout=file)

Cygwin provides run command that allows to run commands directly:

cmd = [r'C:\cygwin\path\to\run.exe', '-p', '/path/to/', 'bedtools', 
        '-wait', 'arg1', 'arg2']

Note: Python script is run from Windows in both cases. bedtools is Linux or Cygwin (non-Windows) command here and therefore you should provide POSIX paths.