How do you extract IP addresses from files using a

Question

How to extract a text part by regexp in linux shell? Lets say, I have a file where in every line is an IP address, but on a different position. What is the simplest way to extract those IP addresses using common unix command-line tools?

Answer 1

cat ip_address.txt | grep '^[0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[,].*$\|^.*[,][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[,].*$\|^.*[,][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}[.][0-9]\{1,3\}$'

Lets assume the file is comma delimited and the position of ip address in the beginning ,end and somewhere in the middle

First regexp looks for the exact match of ip address in the beginning of the line. The second regexp after the or looks for ip address in the middle.we are matching it in such a way that the number that follows ,should be exactly 1 to 3 digits .falsy ips like 12345.12.34.1 can be excluded in this.

The third regexp looks for the ip address at the end of the line

Answer 2

This works fine for me in access logs.

cat access_log | egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}'

Let's break it part by part.

• [0-9]{1,3} means one to three occurrences of the range mentioned in []. In this case it is 0-9. so it matches patterns like 10 or 183.

• Followed by a '.'. We will need to escape this as '.' is a meta character and has special meaning for the shell.

So now we are at patterns like '123.' '12.' etc.

• This pattern repeats itself three times(with the '.'). So we enclose it in brackets. ([0-9]{1,3}\.){3}

• And lastly the pattern repeats itself but this time without the '.'. That is why we kept it separately in the 3rd step. [0-9]{1,3}

If the ips are at the beginning of each line as in my case use:

egrep -o '^([0-9]{1,3}\.){3}[0-9]{1,3}'

where '^' is an anchor that tells to search at the start of a line.

Answer 3

You could use awk, as well. Something like ...

awk '{i=1; if (NF > 0) do {if ($i ~ /regexp/) print $i; i++;} while (i <= NF);}' file

-- may need cleaning. just a quick and dirty response to show basically how to do it with awk

Answer 4

I have tried all answers but all of them had one or many problems that I list a few of them.

1. Some detected 123.456.789.111 as valid IP
2. Some don't detect 127.0.00.1 as valid IP
3. Some don't detect IP that start with zero like 08.8.8.8

So here I post a regex that works on all above conditions.

Note : I have extracted more than 2 millions IP without any problem with following regex.

(?:(?:1\d\d|2[0-5][0-5]|2[0-4]\d|0?[1-9]\d|0?0?\d)\.){3}(?:1\d\d|2[0-5][0-5]|2[0-4]\d|0?[1-9]\d|0?0?\d)

Answer 5

If you are not given a specific file and you need to extract IP address then we need to do it recursively. grep command -> Searches a text or file for matching a given string and displays the matched string .

grep -roE '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}' | grep -oE '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}'

-r -> We can search the entire directory tree i.e. the current directory and all levels of sub-directories. It denotes recursive searching.

-o -> Print only the matching string

-E -> Use extended regular expression

If we would not have used the second grep command after the pipe we would have got the IP address along with the path where it is present