On Thu, Jul 6, 2023 at 8:33 AM Kuniyuki Iwashima kuniyu@amazon.com wrote:
From: Lorenz Bauer lmb@isovalent.com Date: Thu, 6 Jul 2023 09:11:15 +0100
On Thu, Jul 6, 2023 at 1:41 AM Kuniyuki Iwashima kuniyu@amazon.com wrote:
Sorry for late reply.
What we know about sk before inet6?_lookup_reuseport() are
(1) sk was full socket in bpf_sk_assign() (2) sk had SOCK_RCU_FREE in bpf_sk_assign() (3) sk was TCP_LISTEN here if TCP
Are we looking at the same bpf_sk_assign? Confusingly there are two very similarly named functions. The one we care about is:
BPF_CALL_3(bpf_sk_assign, struct sk_buff *, skb, struct sock *, sk, u64, flags) { if (!sk || flags != 0) return -EINVAL; if (!skb_at_tc_ingress(skb)) return -EOPNOTSUPP; if (unlikely(dev_net(skb->dev) != sock_net(sk))) return -ENETUNREACH; if (sk_is_refcounted(sk) && unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) return -ENOENT;
skb_orphan(skb); skb->sk = sk; skb->destructor = sock_pfree; return 0;}
From this we can't tell what state the socket is in or whether it is RCU freed or not.
But we can in inet6?_steal_sock() by calling sk_is_refcounted() again via skb_steal_sock().
In inet6?_steal_sock(), we call inet6?_lookup_reuseport() only for sk that was a TCP listener or UDP non-connected socket until just before the sk_state checks. Then, we know *refcounted should be false for such sockets even before inet6?_lookup_reuseport().
After the checks, sk might be poped out of the reuseport group before inet6?_lookup_reuseport() and reuse_sk might be NULL, but it's not related because *refcounted is a value for sk, not for reuse_sk.
I was about to apply v5 before I noticed this discussion on v4. Sounds like v6 will be needed. Next time please continue discussion in the latest version.