awk matching patterns from row based file and outp

2019-06-14 05:26发布

问题:

I've a file with records in this type of format:

LOCUS       NG_029783              19834 bp    DNA     linear   PRI 03-OCT-2014 DEFINITION  Homo sapiens long intergenic non-protein coding RNA 1546
            (LINC01546), RefSeqGene on chromosome X. ACCESSION   NG_029783 VERSION     NG_029783.1 KEYWORDS    RefSeq; RefSeqGene. SOURCE      Homo sapiens (human)   ORGANISM  Homo sapiens
            Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi;
            Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini;
            Catarrhini; Hominidae; Homo. COMMENT     VALIDATED REFSEQ: This record has undergone validation or
            preliminary review. The reference sequence was derived from
            AC004616.1.
            This sequence is a reference standard in the RefSeqGene project. PRIMARY     REFSEQ_SPAN         PRIMARY_IDENTIFIER PRIMARY_SPAN        COMP
            1-19834             AC004616.1         8636-28469 FEATURES             Location/Qualifiers
     source          1..19834
                     /organism="Homo sapiens"
                     /mol_type="genomic DNA"
                     /db_xref="taxon:9606"
                     /chromosome="X"
                     /map="Xp22.33"
     variation       4
                     /replace="c"
                     /replace="t"
                     /db_xref="dbSNP:1205550"
     variation       17
                     /replace="c"
                     /replace="t"
                     /db_xref="dbSNP:1205551"
     gene            5001..5948
                     /gene="OR6K3"
                     /gene_synonym="OR1-18"
                     /note="olfactory receptor family 6 subfamily K member 3"
                     /db_xref="GeneID:391114"
                     /db_xref="HGNC:HGNC:15030"
     mRNA            5001..5948
                     /gene="OR6K3"
                     /gene_synonym="OR1-18"
                     /product="olfactory receptor family 6 subfamily K member

// 
LOCUS       NG_032962              70171 bp    DNA     linear   PRI 17-JUN-2016 DEFINITION  Homo sapiens death domain containing 1 (DTHD1), RefSeqGene on
                chromosome 4. ACCESSION   NG_032962 VERSION     NG_032962.1 KEYWORDS    RefSeq; RefSeqGene. SOURCE      Homo sapiens (human)   ORGANISM  Homo sapiens
                Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi;
                Mammalia; Eutheria; Euarchontoglires; Primates; Haplorrhini;
                Catarrhini; Hominidae; Homo. COMMENT     REVIEWED REFSEQ: This record has been curated by NCBI staff. The
                reference sequence was derived from AC104078.3.
                This sequence is a reference standard in the RefSeqGene project.

        Summary: This gene encodes a protein which contains a death domain.
        Death domain-containing proteins function in signaling pathways and
        formation of signaling complexes, as well as the apoptosis pathway.
        Alternative splicing results in multiple transcript variants.
        [provided by RefSeq, Oct 2012]. PRIMARY     REFSEQ_SPAN         PRIMARY_IDENTIFIER PRIMARY_SPAN        COMP
        1-70171             AC104078.3         59395-129565 FEATURES             Location/Qualifiers
 source          1..70171
                 /organism="Homo sapiens"
                 /mol_type="genomic DNA"
                 /db_xref="taxon:9606"
                 /chromosome="4"
                 /map="4p14"
 gene            5001..129091
                 /gene="REEP1"
                 /gene_synonym="C2orf23; HMN5B; SPG31; Yip2a"
                 /note="receptor accessory protein 1"
                 /db_xref="GeneID:65055"
                 /db_xref="HGNC:HGNC:25786"
                 /db_xref="MIM:609139"
 mRNA            join(5001..5060,60842..60914,79043..79119,88270..88390,
                 91014..91127,110282..110459,125974..129091)
                 /gene="REEP1"
                 /gene_synonym="C2orf23; HMN5B; SPG31; Yip2a"
                 /product="receptor accessory protein 1, transcript variant
                 1"
                 /transcript_id="NM_001164730.1"

Ive been using this workflow:

  1. remove spaces gawk '{$1=$1}1' raw_file.text > temp_file.txt

  2. match the "Summary" content gawk /Summary/,/\]/{print} temp_file.text > summary_temp.txt

  3. remove new lines gawk 'BEGIN {RS=""}{gsub(/\n/,"",$0); print $0}' summary_temp.text > summary.txt

I've a couple of questions. First, how could I combine those 3 steps. Second, how could I also select one or more additional matches, for example match '/gene="AP3B2"' (which requires matching the first instance of "/gene" after "gene") so that I could output the contents in this form:

gene, summary

回答1:

$ cat tst.awk
BEGIN{RS="//"}
{
  match($0, /\/gene="([^"]+)"/, a)
  print a[1] ", ", 
        gensub(/\s\s+/, "", "g", gensub(/.*Summary:\s([^\[]+).*/, "\\1", "g"))
}

Explanation:

match($0, /\/gene="([^"]+)"/, a)

catches all "\gene" parts in array a. According to your question only the first occurrence is needed, which is a[1] (and is not AP3B2, btw).

gensub(/.*Summary:\s([^\[]+).*/, "\\1", "g")

catches everything after "Summary: " until you find a "[". This last result has spaces and newlines. Let's get rid of them:

gensub(/\s\s+/, "", "g", <<result of 1st gensub>>)

EDIT: Not every record contains a "Summary"

Changed the script to:

$ cat tst.awk
BEGIN{RS="//"}
{
  match($0, /\/gene="([^"]+)"/, a)
  match($0, /Summary:\s([^\[]+)/, b)
  print a[1] ",",
    gensub(/\s\s+/, " ", "g", b[1])
}

Running the script with input provided by OP:

awk -f tst.awk tst.txt
OR6K3,
REEP1,  This gene encodes a protein which contains a death domain.Death 
domain-containing proteins function in signaling pathways andformation 
of signaling complexes, as well as the apoptosis pathway.Alternative 
splicing results in multiple transcript variants.


标签: regex awk gawk