Efficient way to check IP in slice of IP addresses

Question

I'm developing a network application in Golang. I have a slice of IP addresses. Each time a request comes I use net.LookupIP(host) to find out IP address of host which returns a slice of net.IP. What is the best approach to compare these?

By the way in Python we have a set data structure which makes above question so easy to resolve but what about Go?

Answer 1

With a "set"

Building our set

There is no builtin Set type in Go, but you can elegantly use a map[Type]bool as a set, e.g.:

// Create a set with 2 values in it: [1, 2]
m := map[int]bool{1: true, 2: true}

// Test an element:
fmt.Println(m[1]) // true
fmt.Println(m[3]) // false

// Set an element:
m[3] = true
fmt.Println(m[3]) // true

// Delete an element:
delete(m, 1)
fmt.Println(m[1]) // false

Note: we exploited the fact that if a key is not in the map, indexing the map results in the zero value for the value type, which is false in case of bool, properly telling that the element is not in the map (set).

Try it on the Go Playground.

^{Note #2: there are a few tricks to make the code to handle a map as a set shorter, you can check them in this answer: Check if a value is in a list.}

Using net.IP in the set

Now we only need a type representing a net.IP which can be used as the key type in a map (see this question about what constitutes a map key type: How can I prevent a type being used as a map key?).

Unfortunately net.IP itself does not qualify, because it is a slice:

type IP []byte

And slices are not comparable. See this question for details: Hash with key as an array type and this: Why have arrays in Go?

An easy way is to convert it to a canonical string value and we're done. For this we may simply convert the bytes of the IP to a hex string. But an IPv4 address may be presented as IPv6, so we should first convert it to IPv6:

func Key(ip net.IP) string {
    return hex.EncodeToString(ip.To16())
}

Note: bytes of an IP address may not be a valid UTF-8 encoded string (which is how Go stores strings in memory), but string values in Go represent arbitrary byte sequences, so the following also works, is much simpler and is much more efficient:

func Key(ip net.IP) string {
    return string(ip.To16())  // Simple []byte => string conversion
}

We can use such IP strings as the keys. Populate your map with IPs to check against:

// Populate forbidden IPs:
forbIPs := map[string]bool{
    Key(ip1): true,
    Key(ip2): true,
}

// Now check a single IP:
ipToCheck := ...
if forbIPs[Key(ipToCheck)] {
    fmt.Println("Forbidden!")
} else {
    fmt.Println("Allowed.")
}

If you have multiple IPs to check (as returned by net.LookupIP()), it's a single for loop:

ips, err := net.LookupIP(host)
// Check err
for _, ip := range ips {
    if forbIPs[Key(ip)] {
        // FORBIDDEN!
    }
}

Alternative Key type

Note that –as mentioned above– slices are not comparable but arrays are. So we could also use an array as the key. This is how it could look like:

func Key(ip net.IP) (a [16]byte) {
    copy(a[:], ip)
    return
}

// And the IP set:
forbIPs := map[[16]byte]bool{
    // ...
}

Alternatives

Sorted slice

Alternatively we could just simply store the forbidden IPs in a slice []net.IP, and keep it sorted. If it is sorted, we can use binary search to find an IP in it (standard library sort.Search()).

Yes, binary search has O(log2(n)) complexity compared to the O(1) complexity of the (hash)map solution above. But this alternative has another pro:

Enumerating individual IPs is not always practical. Sometimes (often) it is easier to list IP ranges. The first solution is not feasible to handle IP ranges, but this solution may be: you can find ranges that cover an IP address also in O(log2(n)) time.

Answer 2

You may use func (ip IP) Equal(x IP) bool from net package:

Equal reports whether ip and x are the same IP address. An IPv4 address and that same address in IPv6 form are considered to be equal.

Like this working sample:

package main

import (
    "fmt"
    "net"
)

func main() {
    ip := net.ParseIP("127.0.0.1")
    ips, err := net.LookupIP("localhost")
    if err != nil {
        panic(err)
    }
    for _, v := range ips {
        if v.Equal(ip) {
            fmt.Println(v)
        }
    }
}