From: "Jiayuan Chen" jiayuan.chen@linux.dev Date: Thu, 01 May 2025 06:22:17 +0000
2025/5/1 12:42, "Kuniyuki Iwashima" kuniyu@amazon.com wrote:
From: Jiayuan Chen jiayuan.chen@linux.dev
Date: Thu, 1 May 2025 11:51:08 +0800
For some services we are using "established-over-unconnected" model.
'''
// create unconnected socket and 'listen()'
srv_fd = socket(AF_INET, SOCK_DGRAM)
setsockopt(srv_fd, SO_REUSEPORT)
bind(srv_fd, SERVER_ADDR, SERVER_PORT)
// 'accept()'
data, client_addr = recvmsg(srv_fd)
// create a connected socket for this request
cli_fd = socket(AF_INET, SOCK_DGRAM)
setsockopt(cli_fd, SO_REUSEPORT)
bind(cli_fd, SERVER_ADDR, SERVER_PORT)
connect(cli, client_addr)
...
// do handshake with cli_fd
'''
This programming pattern simulates accept() using UDP, creating a new
socket for each client request. The server can then use separate sockets
to handle client requests, avoiding the need to use a single UDP socket
for I/O transmission.
But there is a race condition between the bind() and connect() of the
connected socket:
We might receive unexpected packets belonging to the unconnected socket
before connect() is executed, which is not what we need.
(Of course, before connect(), the unconnected socket will also receive
packets from the connected socket, which is easily resolved because
upper-layer protocols typically require explicit boundaries, and we
receive a complete packet before creating a connected socket.)
Before this patch, the connected socket had to filter requests at recvmsg
time, acting as a dispatcher to some extent. With this patch, we can
consider the bind and connect operations to be atomic.
SO_ATTACH_REUSEPORT_EBPF is what you want.
The socket won't receive any packets until the socket is added to
the BPF map.
No need to reinvent a subset of BPF functionalities.
I think this feature is for selecting one socket, not filtering out certain sockets.
Does this mean that I need to first capture all sockets bound to the same port, and then if the kernel selects a socket that I don't want to receive packets on, I'll need to implement an algorithm in the BPF program to choose another socket from the ones I've captured, in order to avoid returning that socket?
Right.
If you want a set of sockets to listen on the port, you can implement as such with BPF; register the sockets to the BPF map, and if kernel pick up other sockets and triggers the BPF prog, just return one of the registerd sk.
Even when you have connect()ed sockets on the same port, kernel will fall back to the normal scoring to find the best one, and it's not a problem as the last 'result' is one selected by BPF or a connected sk, and the packet won't be routed to not-yet-registered unconnected sk.
This looks like it completely bypasses the kernel's built-in scoring logic. Or is expanding BPF_PROG_TYPE_SK_REUSEPORT to have filtering capabilities also an acceptable solution?