Why is pcap_datalink() always returning 1 (Etherne

I'm having an issue where by pcap_datalink() is always returning 1. To my understanding this is LINKTYPE_ETHERNET. But, the device I am using is a wireless card and in my case en0.

This is stopping me from putting the card into monitor mode, and stopping my WLAN filters from working. I've tried to run this on both OSX and Linux with the same results. I also run as root.

Here's the part of my code that's causing the problem. For the example, assume dev is set to en0 (wireless device on Mac).

#include <stdio.h>
#include <pcap.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
    pcap_t *pcap_h;
    char *dev, errbuf[PCAP_ERRBUF_SIZE];
    struct bpf_program fp;
    struct pcap_pkthdr header;
    const u_char *packet;

    if(argc < 2)
    {
        printf("Usage: %s device\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    dev = argv[1];

    if((pcap_h = pcap_create(dev, errbuf)) == NULL)
    {
        printf("pcap_create() failed: %s\n", errbuf);
        exit(EXIT_FAILURE);
    }

    if(pcap_can_set_rfmon(pcap_h) == 0)
    {
        printf("Monitor mode can not be set.\n");
    }

    if(pcap_set_rfmon(pcap_h, 1) != 0)
    {
        printf("Failed to set monitor mode.\n");
        exit(EXIT_FAILURE);
    }

    if(pcap_activate(pcap_h) != 0)
    {
        printf("pcap_activate() failed\n");
        exit(EXIT_FAILURE);
    }

    /*
     * Compile a filter to sniff 802.11 probe requests
     * Filter: type mgt subtype probe-req
     */ 
    if(pcap_compile(pcap_h, &fp, "type mgt subtype probe-req", 0, PCAP_NETMASK_UNKNOWN) == -1)
    {
        printf("pcap_compile() failed: %s\n", pcap_geterr(pcap_h));
        exit(EXIT_FAILURE);
    }

    /*
     * Set the compiled filter
     */ 
    if(pcap_setfilter(pcap_h, &fp) == -1)
    {
        printf("pcap_setfilter() failed: %s\n", pcap_geterr(pcap_h));
        exit(EXIT_FAILURE);
    }

    pcap_freecode(&fp);

    packet = pcap_next(pcap_h, &header);

    printf("Header: %d\n", header.len); 
    pcap_close(pcap_h);
    return 0;
}

Any idea's why pcap_datalink() is always returning 1?

Edit

Updated code, and added pcap_set_rfmon() before the call to pcap_activate(). I get an error:

pcap_compile() failed: 802.11 link-layer types supported only on 802.11

Are you shure this is what's is stopping you from putting the card into monitor mode, and stopping your WLAN filters from working, or do you do this call to pcap_datalink() as a check trying to pinpoint the issue?

Be aware that, from PCAP-LINKTYPE(7):

For a live capture or ``savefile'', libpcap supplies, as the return value of the pcap_datalink(3PCAP) routine, a value that indicates the type of link-layer header at the beginning of the packets it provides. This is not necessarily the type of link-layer header that the packets being captured have on the network from which they're being captured; for example, packets from an IEEE 802.11 network might be provided by libpcap with Ethernet headers that the network adapter or the network adapter driver generates from the 802.11 headers.

So I would not take this LINKTYPE_ETHERNET / DLT_EN10MB return value as the sure indication of a problem here.

EDIT: Also, pcap_set_rfmon() is supposed to be call before the handle is activated, which is not visible in your code.

pcap is rather touchy about the order things should be done. Have a look at the man pages for pcap_can_set_rfmon and pcap_set_rfmon.

The order should be: