How to use awk for a compressed file

问题:

How can I change the following command for a compressed file?

awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }' input1.vcf input2.vcf

The command working fine with normal file. I need to change the command for compressed files.

回答1:

You need to read them compressed files like this:

awk '{ ... }' <(gzip -dc input1.vcf.gz) <(gzip -dc input2.vcf.gz)

Try this:

awk 'FNR==NR { sub(/AA=\.;/,""); array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }' <(gzip -dc input1.vcf.gz) <(gzip -dc input2.vcf.gz) | gzip > output.vcf.gz

回答2:

zcat FILE | awk '{ ...}'

I wouldn't be able to tell which of all these methods works best, zcat is at least quicker to type ;)

回答3:

bzip2 -dc input1.vcf.bz2 input2.vcf.bz2 | awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }'

or

gzip -dc input1.vcf.gz input2.vcf.gz | awk 'FNR==NR { array[$1,$2]=$8; next } ($1,$2) in array { print $0 ";" array[$1,$2] }'

EDIT:

To write compressed output just append

| bzip2 >output.vcf.bz2

or

| gzip >output.vcf.gz

This will work with any program that prints results to standard output.

BTW: Editing such large command lines gets tedious very quickly. You should consider writing a small shell script to do the job. This has the additional benefit that you don't have to remember the entire thing and can easily repeat the command or modify it if necessary.

A good starting point for Linux shell programming is the Bash Programming Inroduction by Mike G.