Shell script linux substract parameter grep

Question

I need help to extract coincidences from a file.

I capture network traffic with tcpdump command

tcpdump -Xvv -i eth0 > captureFile.txt

Given any field of IP headers, TCP and Ethernet specify all values ​​found in the captured traffic and count how many times that value for that field. For example if suppose TTL = 128 TTL = 64 then indicate how many packets have that field with each of these values​​.

The content of the file:

09:26:13.245546 IP (tos 0x0, ttl 1, id 3439, offset 0, flags [none], proto UDP (17), length 1018)
10.0.0.226.58935 > 239.255.255.250.3702: UDP, length 990
    0x0000:  4500 03fa 0d6f 0000 0111 ada8 0a00 00e2  E....o..........
    0x0010:  efff fffa e637 0e76 03e6 7ec0 3c3f 786d  .....7.v..~.<?xm
    0x0020:  6c20 7665 7273 696f 6e3d 2231 2e30 2220  l.version="1.0".
    0x0030:  656e 636f 6469 6e67 3d22 7574 662d 3822  encoding="utf-8"
    0x0040:  3f3e 3c73 6f61 703a 456e 7665            ?><soap:Enve
 09:26:13.339173 IP6 (hlim 1, next-header UDP (17) payload length: 998) fe80::21e9:f54b:9ae7:6383.58936 > ff02::c.3702: UDP, length 990
    0x0000:  6000 0000 03e6 1101 fe80 0000 0000 0000  `...............
    0x0010:  21e9 f54b 9ae7 6383 ff02 0000 0000 0000  !..K..c.........
    0x0020:  0000 0000 0000 000c e638 0e76 03e6 666c  .........8.v..fl
    0x0030:  3c3f 786d 6c20 7665 7273 696f 6e3d 2231  <?xml.version="1
    0x0040:  2e30 2220 656e 636f 6469 6e67            .0".encoding
09:26:13.407313 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.3.118 tell 10.0.1.215, length 46
    0x0000:  0001 0800 0604 0001 0009 0fcb 0a0c 0a00  ................
    0x0010:  01d7 0000 0000 0000 0a00 0376 0000 0000  ...........v....
    0x0020:  0000 0000 0000 0000 0000 d9c4 62a8       ............b.
09:26:13.525954 IP (tos 0x0, ttl 128, id 3441, offset 0, flags [none], proto UDP (17), length 161)
10.0.0.226.59131 > 239.255.255.250.1900: UDP, length 133
    0x0000:  4500 00a1 0d71 0000 0111 b0ff 0a00 00e2  E....q..........
    0x0010:  efff fffa e6fb 076c 008d 6fa6 4d2d 5345  .......l..o.M-SE
    0x0020:  4152 4348 202a 2048 5454 502f 312e 310d  ARCH.*.HTTP/1.1.
    0x0030:  0a48 6f73 743a 3233 392e 3235 352e 3235  .Host:239.255.25
    0x0040:  352e 3235 303a 3139 3030 0d0a            5.250:1900..
09:26:13.557002 IP (tos 0x0, ttl 1, id 3442, offset 0, flags [none], proto UDP (17), length 161)
10.0.0.226.59131 > 239.255.255.250.1900: UDP, length 133
    0x0000:  4500 00a1 0d72 0000 0111 b0fe 0a00 00e2  E....r..........
    0x0010:  efff fffa e6fb 076c 008d 6fa6 4d2d 5345  .......l..o.M-SE
    0x0020:  4152 4348 202a 2048 5454 502f 312e 310d  ARCH.*.HTTP/1.1.
    0x0030:  0a48 6f73 743a 3233 392e 3235 352e 3235  .Host:239.255.25
    0x0040:  352e 3235 303a 3139 3030 0d0a            5.250:1900..
09:26:13.642734 IP (tos 0x0, ttl 1, id 21767, offset 0, flags [none], proto UDP (17), length 684)
10.0.0.237.58882 > 239.255.255.250.3702: UDP, length 656
    0x0000:  4500 02ac 5507 0000 0111 6753 0a00 00ed  E...U.....gS....
    0x0010:  efff fffa e602 0e76 0298 5568 3c3f 786d  .......v..Uh<?xm
    0x0020:  6c20 7665 7273 696f 6e3d 2231 2e30 2220  l.version="1.0".
    0x0030:  656e 636f 6469 6e67 3d22 7574 662d 3822  encoding="utf-8"
    0x0040:  3f3e 3c73 6f61 703a 456e 7665            ?><soap:Enve
09:26:13.642960 IP6 (hlim 1, next-header UDP (17) payload length: 664) fe80::b8a2:bd0:4e0b:1bb5.58883 > ff02::c.3702: UDP, length 656
    0x0000:  6000 0000 0298 1101 fe80 0000 0000 0000  `...............
    0x0010:  b8a2 0bd0 4e0b 1bb5 ff02 0000 0000 0000  ....N...........
    0x0020:  0000 0000 0000 000c e603 0e76 0298 248c  ...........v..$.
    0x0030:  3c3f 786d 6c20 7665 7273 696f 6e3d 2231  <?xml.version="
09:26:13.642999 IP (tos 0x0, ttl 64, id 21767, offset 0, flags [none], proto UDP (17), length 684)
10.0.0.237.58882 > 239.255.255.250.3702: UDP, length 656
    0x0000:  4500 02ac 5507 0000 0111 6753 0a00 00ed  E...U.....gS....
    0x0010:  efff fffa e602 0e76 0298 5568 3c3f 786d  .......v..Uh<?xm
    0x0020:  6c20 7665 7273 696f 6e3d 2231 2e30 2220  l.version="1.0".
    0x0030:  656e 636f 6469 6e67 3d22 7574 662d 3822  encoding="utf-8"
    0x0040:  3f3e 3c73 6f61 703a 456e 7665            ?><soap:Enve

The result must be:

ttl 64 - 1 time 
ttl 128 - 1 time
ttl 1 - 3 times 

Answer 1

Simple awk script:

$ awk -F, '/ttl/{u[$2]++}END{for(k in u)print k" - "u[k]" time"(u[k]>1?"s":"")}'
 ttl 128 - 1 time
 ttl 64 - 1 time
 ttl 1 - 3 times

No need to waste sub-process.

Answer 2

I think this would be exactly same as your expected output.

grep -ioP 'ttl \d+' file|awk '{a[$0]++}END{for(x in a)print x" - "a[x]" times"}'

output would be:

ttl 1 - 3 times
ttl 64 - 1 times
ttl 128 - 1 times

well not exactly same, since I didn't check time and times.. do you really need it? it could be done easily..

EDIT

as OP asks, output time/times depends on the count:

 grep -ioP 'ttl \d+' file|awk '{a[$0]++}END{for(x in a)print x" - "a[x]" time"(a[x]>1?"s":"")}'  

output:

ttl 1 - 3 times
ttl 64 - 1 time
ttl 128 - 1 time

Answer 3

It's a bit long and I'm sure it can be refactored quite a lot but it works if you don't|can't have perl installed:

grep ttl captureFile.txt | awk '{print $5,$6}' | sed 's/,//' | sort | uniq -c | awk '{print $2,$3,"-",$1,"times"}'

Answer 4

grep "ttl [0-9]*" captureFile.txt -o

Would get only the relevant parts of the text file.

grep "ttl [0-9]*" captureFile.txt -o |
awk 'NF{ count[ toupper( $0 ) ]++} 
END{ for ( name in count ) { print "*" name " -  " count[ name ] " times"
};
}' 

Would get the formatting you wanted.

Answer 5

Two approaches:

If you have perl,

captureFile.txt | 
perl -ne '/ttl (\d+),/ and $TTL{$1}++; 
  END { for my $ttl (keys %TTL) {print "* ttl $1 - $TTL{$ttl} time\n"}}'

Should do it. But I think uniq -c may also work with grep...

captureFile.txt | egrep -o 'ttl ([0-9]+)' | uniq  -c

And to get the exact output format you asked for, just add this after uniq -c

| awk '{print "* ttl "$3" - "$1" time"}'