What would cause UDP packets to be dropped when be

Question

I'm sending very large (64000 bytes) datagrams. I realize that the MTU is much smaller than 64000 bytes (a typical value is around 1500 bytes, from my reading), but I would suspect that one of two things would happen - either no datagrams would make it through (everything greater than 1500 bytes would get silently dropped or cause an error/exception to be thrown) or the 64000 byte datagrams would get chunked into about 43 1500 byte messages and transmitted transparently.

Over a long run (2000+ 64000 byte datagrams), about 1% (which seems abnormally high for even a LAN) of the datagrams get dropped. I might expect this over a network, where datagrams can arrive out of order, get dropped, filtered, and so on. However, I did not expect this when running on localhost.

What is causing the inability to send/receive data locally? I realize UDP is unreliable, but I didn't expect it to be so unreliable on localhost. I'm wondering if it's just a timing issue since both the sending and receiving components are on the same machine.

For completeness, I've included the code to send/receive datagrams.

Sending:

DatagramSocket socket = new DatagramSocket(senderPort);

int valueToSend = 0;

while (valueToSend < valuesToSend || valuesToSend == -1) {
    byte[] intBytes = intToBytes(valueToSend);

    byte[] buffer = new byte[bufferSize - 4];

     //this makes sure that the data is put into an array of the size we want to send
    byte[] bytesToSend = concatAll(intBytes, buffer);

    System.out.println("Sending " + valueToSend + " as " + bytesToSend.length + " bytes");

    DatagramPacket packet = new DatagramPacket(bytesToSend,
                        bufferSize, receiverAddress, receiverPort);

    socket.send(packet);

    Thread.sleep(delay);

    valueToSend++;
}

Receiving:

DatagramSocket socket = new DatagramSocket(receiverPort);

while (true) {
    DatagramPacket packet = new DatagramPacket(
            new byte[bufferSize], bufferSize);

    System.out.println("Waiting for datagram...");
    socket.receive(packet);

    int receivedValue = bytesToInt(packet.getData(), 0);

    System.out.println("Received: " + receivedValue
            + ". Expected: " + expectedValue);

    if (receivedValue == expectedValue) {
        receivedDatagrams++;
        totalDatagrams++;
    }
    else {
        droppedDatagrams++;
        totalDatagrams++;
    }

    expectedValue = receivedValue + 1;
    System.out.println("Expected Datagrams: " + totalDatagrams);
    System.out.println("Received Datagrams: " + receivedDatagrams);
    System.out.println("Dropped Datagrams: " + droppedDatagrams);
    System.out.println("Received: "
            + ((double) receivedDatagrams / totalDatagrams));
    System.out.println("Dropped: "
            + ((double) droppedDatagrams / totalDatagrams));
    System.out.println();
}

Answer 1

UDP pkts scheduling may be handled by multiple threads on OS level. That would explain why you receive them out of order even on 127.0.0.1.

Answer 2

I don't know what makes you expect a percentage less then 1% of dropped packets for UDP.

That being said, based on RFC 1122 (see section 3.3.2), the maximum buffer size guaranteed not to be split into multiple IP datagrams is 576 bytes. Larger UDP datagrams may be transmitted but they will likely be split into multiple IP datagrams to be reassembled at the receiving end point.

I would imagine that a reason contributing to the high rate of dropped packets you're seeing is that if one IP packet that was part of a large UDP datagram is lost, the whole UDP datagram will be lost. And you're counting UDP datagrams - not IP packets.

Answer 3

UDP packets are not guaranteed to reach their destination whereas TCP is!

Answer 4

Your expectations, as expressed in your question and in numerous comments to other answers, are wrong. All the following can happen even in the absence of routers and cables.

1. If you send a packet to any receiver and there is no room in his socket receive buffer it will get dropped.

2. If you send a UDP datagram larger than the path MTU it will get fragmented into smaller packets, which are subject to (1).

3. If all the packets of a datagram don't arrive, the datagram will never get delivered.

4. The TCP/IP stack has no obligation to deliver packets or UDP datagrams in order.

Answer 5

What is causing the inability to send/receive data locally?

Mostly buffer space. Say you're sending a constant 10MB/second , and you're only able to consume 5MB/second, the OS and network stack can't keep up, so it will drop the packets - this should be rather obvious. (Which, naturally is different from TCP which provides flow control and retransmission to handle such a situation).

Even if you're normally keeping up with consuming the data, there might be small time slices where you're not. e.g. a garbage collector kicks in, the OS decided to schedule another process instead of your consumer for 0.5 seconds, etc - and the system will drop packets.

This can be extended to any network devices in between. If you're running on a network instead of just locally, an ethernet switch, router, etc. will also drop packets if its queues are full (e.g. you're sending a 10MB/s stream through an 100MB/s ethernet switch, and a few seconds in the middle of the night someone else tries to cram 100MB/sec through the same path, some packets will lose.)

Try increasing the socket buffers size, often you have to increase this on the OS level as well.

(e.g. on linux, the default socket buffer size is often only 128k or less, which leaves very little room for pausing the data processing , you can try to increase them by setting the sysctl net.core.wmem_max,net.core.wmem_default, net.core.rmem_max, net.core.rmem_default)