I'm extracting a tarball using the tarfile module of python. I don't want the extracted files to be written on the disk, but rather get piped directly to another program, specifically bgzip. I'm also trying to use StringIO for that matter, but I get stuck even on that stage - the tarball gets extracted on the disk.
#!/usr/bin/env python
import tarfile, StringIO
tar = tarfile.open("6genomes.tgz", "r:gz")
def enafun(members):
for tarkati in tar:
if tarkati.isreg():
yield tarkati
reles = StringIO.StringIO()
reles.write(tar.extractall(members=enafun(tar)))
tar.close()
How then do I pipe correctly the output of tar.extractall?
You cannot use extractall method, but you can use getmembers and extractfile methods instead:
#!/usr/bin/env python
import tarfile, StringIO
reles = StringIO.StringIO()
with tarfile.open("6genomes.tgz", "r:gz") as tar:
for m in tar.members():
if m.isreg():
reles.write(tar.extractfile(m).read())
# do what you want with "reles".
According to the documentation, extractfile() method can take a TarInfo and will return a file-like object. You can then get the content of that file with read().
[EDIT] I add what you asked me in comment as formatting in comment seems not to render properly.
#!/usr/bin/env python
import tarfile
import subprocess
with tarfile.open("6genomes.tgz", "r:gz") as tar:
for m in tar.members():
if m.isreg():
f = tar.extractfile(m)
new_filename = generate_new_filename(f.name)
with open(new_filename, 'wb') as new_file:
proc = subprocess.Popen(['bgzip', '-c'], stdin=subprocess.PIPE, stdout=new_file)
proc.stdin.write(f.read())
proc.stdin.close()
proc.wait()
f.close()
