On 2025/5/1 00:53, Alexei Starovoitov wrote:
On Wed, Apr 30, 2025 at 8:55 AM Leon Hwang leon.hwang@linux.dev wrote:
On 2025/4/30 20:43, Kafai Wan wrote:
On Wed, Apr 30, 2025 at 10:46 AM Alexei Starovoitov alexei.starovoitov@gmail.com wrote:
On Sat, Apr 26, 2025 at 9:00 AM KaFai Wan mannkafai@gmail.com wrote:
[...]
bpf_get_func_arg() will be very helpful for bpfsnoop[1] when tracing tp_btf.
In bpfsnoop, it can generate a small snippet of bpf instructions to use bpf_get_func_arg() for retrieving and filtering arguments. For example, with the netif_receive_skb tracepoint, bpfsnoop can use bpf_get_func_arg() to filter the skb argument using pcap-filter(7)[2] or a custom attribute-based filter. This will allow bpfsnoop to trace multiple tracepoints using a single bpf program code.
I doubt you thought it through end to end. When tracepoint prog attaches we have this check: /* * check that program doesn't access arguments beyond what's * available in this tracepoint */ if (prog->aux->max_ctx_offset > btp->num_args * sizeof(u64)) return -EINVAL;
So you cannot have a single bpf prog attached to many tracepoints to read many arguments as-is. You can hack around that limit with probe_read, but the values won't be trusted and you won't be able to pass such untrusted pointers into skb and other helpers/kfuncs.
I understand that a single bpf program cannot be attached to multiple tracepoints using tp_btf. However, the same bpf code can be reused to create multiple bpf programs, each attached to a different tracepoint.
For example:
SEC("fentry") int BPF_PROG(fentry_fn) { /* ... */ return BPF_OK; }
The above fentry code can be compiled into multiple bpf programs to trace different kernel functions. Each program can then use the bpf_get_func_arg() helper to access the arguments of the traced function.
With this patch, tp_btf will gain similar flexibility. For example:
SEC("tp_btf") int BPF_PROG(tp_btf_fn) { /* ... */ return BPF_OK; }
Here, bpf_get_func_arg() can be used to access tracepoint arguments.
Currently, due to the lack of bpf_get_func_arg() support in tp_btf, bpfsnoop[1] uses bpf_probe_read_kernel() to read tracepoint arguments. This is also used when filtering specific argument attributes.
For instance, to filter the skb argument of the netif_receive_skb tracepoint by 'skb->dev->ifindex == 2', the translated bpf instructions with bpf_probe_read_kernel() would look like this:
bool filter_arg(__u64 * args): ; filter_arg(__u64 *args) 209: (79) r1 = *(u64 *)(r1 +0) /* all tracepoint's argument has been read into args using bpf_probe_read_kernel() */ 210: (bf) r3 = r1 211: (07) r3 += 16 212: (b7) r2 = 8 213: (bf) r1 = r10 214: (07) r1 += -8 215: (85) call bpf_probe_read_kernel#-125280 216: (79) r3 = *(u64 *)(r10 -8) 217: (15) if r3 == 0x0 goto pc+10 218: (07) r3 += 224 219: (b7) r2 = 8 220: (bf) r1 = r10 221: (07) r1 += -8 222: (85) call bpf_probe_read_kernel#-125280 223: (79) r3 = *(u64 *)(r10 -8) 224: (67) r3 <<= 32 225: (77) r3 >>= 32 226: (b7) r0 = 1 227: (15) if r3 == 0x2 goto pc+1 228: (af) r0 ^= r0 229: (95) exit
If bpf_get_func_arg() is supported in tp_btf, the bpf program will instead look like:
static __noinline bool filter_skb(void *ctx) { struct sk_buff *skb;
(void) bpf_get_func_arg(ctx, 0, (__u64 *) &skb); return skb->dev->ifindex == 2; }
This will simplify the generated code and eliminate the need for bpf_probe_read_kernel() calls. However, in my tests (on kernel 6.8.0-35-generic, Ubuntu 24.04 LTS), the pointer returned by bpf_get_func_arg() is marked as a scalar rather than a trusted pointer:
0: R1=ctx() R10=fp0 ; if (!filter_skb(ctx)) 0: (85) call pc+3 caller: R10=fp0 callee: frame1: R1=ctx() R10=fp0 4: frame1: R1=ctx() R10=fp0 ; filter_skb(void *ctx) 4: (bf) r3 = r10 ; frame1: R3_w=fp0 R10=fp0 ; 5: (07) r3 += -8 ; frame1: R3_w=fp-8 ; (void) bpf_get_func_arg(ctx, 0, (__u64 *) &skb); 6: (b7) r2 = 0 ; frame1: R2_w=0 7: (85) call bpf_get_func_arg#183 ; frame1: R0_w=scalar() ; return skb->dev->ifindex == 2; 8: (79) r1 = *(u64 *)(r10 -8) ; frame1: R1_w=scalar() R10=fp0 fp-8=mmmmmmmm ; return skb->dev->ifindex == 2; 9: (79) r1 = *(u64 *)(r1 +16) R1 invalid mem access 'scalar' processed 7 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
If the returned skb is a trusted pointer, the verifier will accept something like:
static __noinline bool filter_skb(struct sk_buff *skb) { return skb->dev->ifindex == 2; }
Which will compile into much simpler and more efficient instructions:
bool filter_skb(struct sk_buff * skb): ; return skb->dev->ifindex == 2; 92: (79) r1 = *(u64 *)(r1 +16) ; return skb->dev->ifindex == 2; 93: (61) r1 = *(u32 *)(r1 +224) 94: (b7) r0 = 1 ; return skb->dev->ifindex == 2; 95: (15) if r1 == 0x2 goto pc+1 96: (b7) r0 = 0 ; return skb->dev->ifindex == 2; 97: (95) exit
In conclusion:
1. It will be better if the pointer returned by bpf_get_func_arg() is trusted, only when the argument index is a known constant. 2. Adding bpf_get_func_arg() support to tp_btf will significantly simplify and improve tools like bpfsnoop.
[1] https://github.com/bpfsnoop/bpfsnoop
Thanks, Leon