I'm trying to find a simple solution for high-performance broadcast/multicast communication between unprivileged processes running on a Linux host. I'm looking for a solution that is 1) simple, 2) unprivileged (no root), 3) language-independent, 4) packet-oriented and 5) efficient (Gbit/s and up).
To put this in context, my existing code simply uses UDP sockets for unicast communication, which neatly matches the above requirements (except being unicast). I've looked into expanding this to multicast by having multiple programs listen to the same UDP port (using SO_REUSEADDR and/or SO_REUSEPORT), but this doesn't actually distribute copies of the packet to all the processes.
I've also looked into using loopback broadcast (127.255.255.255) to reach multiple listening processes, but it seems that I'll need to bind to multiple IP-addresses on the loopback device for this to work, and adding these addresses requires root.
Expanding on Pete's suggestion, I've found the following solution, which is not too complex.
The following Python 2 program implements a simple chat program over multicast/loopback. Tested on Linux 3.13 / Ubuntu 14.04.
import os, socket, sys
# Use an administratively scoped multicast IP (RFC 2365).
mcast_group = '239.0.0.0'
port = 1234
# Communicate over the loopback interface.
ifc = '127.0.0.1'
def send(data):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
# Send over loopback interface.
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF,
socket.inet_aton(ifc))
sock.sendto(data, (mcast_group, port))
def listen():
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
# Join group.
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP,
socket.inet_aton(mcast_group) + socket.inet_aton(ifc))
# Allow multiple subscribers.
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((mcast_group, port))
while True:
data, remote = sock.recvfrom(1024)
print '#%d (%s:%d): %s' % ( (os.getpid(),) + remote + (data,) )
args = ' '.join(sys.argv[1:])
if args:
send(args)
else:
listen()
This expands trivially to communication across the local broadcast segment (LAN) by setting ifc = '0.0.0.0' and unsetting the IP_MULTICAST_IF option.
Receiving one's own traffic
The only problem is that there appears to be no simple way to join a group and send to everyone else in the group without getting your own traffic as well (even by using the same socket for listening and sending).
The IPPROTO_IP socket option IP_MULTICAST_LOOP does not work; it has no effect when using the loopback interface, and when communicating over the network it prevents other local clients from receiving the messages.
I can work around this by filtering on the source address, though.
(As a side note, I just realized that QEMU includes native support for sending virtual network traffic using multicast. Faced with the above problem, the QEMU developers also discussed filtering based on source address, though it appears that they never actually did anything about it. As a result, the QEMU VM receives a copy of its own outgoing traffic, though the traffic is usually rejected in the network stack.)
