From: Puranjay Mohan puranjay@kernel.org
[ Upstream commit 19d3c179a37730caf600a97fed3794feac2b197b ]
When BPF_TRAMP_F_CALL_ORIG is set, the trampoline calls __bpf_tramp_enter() and __bpf_tramp_exit() functions, passing them the struct bpf_tramp_image *im pointer as an argument in R0.
The trampoline generation code uses emit_addr_mov_i64() to emit instructions for moving the bpf_tramp_image address into R0, but emit_addr_mov_i64() assumes the address to be in the vmalloc() space and uses only 48 bits. Because bpf_tramp_image is allocated using kzalloc(), its address can use more than 48-bits, in this case the trampoline will pass an invalid address to __bpf_tramp_enter/exit() causing a kernel crash.
Fix this by using emit_a64_mov_i64() in place of emit_addr_mov_i64() as it can work with addresses that are greater than 48-bits.
Fixes: efc9909fdce0 ("bpf, arm64: Add bpf trampoline for arm64") Signed-off-by: Puranjay Mohan puranjay@kernel.org Signed-off-by: Daniel Borkmann daniel@iogearbox.net Closes: https://lore.kernel.org/all/SJ0PR15MB461564D3F7E7A763498CA6A8CBDB2@SJ0PR15MB... Link: https://lore.kernel.org/bpf/20240711151838.43469-1-puranjay@kernel.org [Minor context change fixed.] Signed-off-by: Bin Lan bin.lan.cn@windriver.com Signed-off-by: He Zhe zhe.he@windriver.com --- Build test passed. --- arch/arm64/net/bpf_jit_comp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 5074bd1d37b5..7c5156e7d31e 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -1952,7 +1952,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, emit(A64_STR64I(A64_R(20), A64_SP, regs_off + 8), ctx);
if (flags & BPF_TRAMP_F_CALL_ORIG) { - emit_addr_mov_i64(A64_R(0), (const u64)im, ctx); + emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); emit_call((const u64)__bpf_tramp_enter, ctx); }
@@ -1996,7 +1996,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
if (flags & BPF_TRAMP_F_CALL_ORIG) { im->ip_epilogue = ctx->image + ctx->idx; - emit_addr_mov_i64(A64_R(0), (const u64)im, ctx); + emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); emit_call((const u64)__bpf_tramp_exit, ctx); }
From: Peter Collingbourne pcc@google.com
[ Upstream commit a552e2ef5fd1a6c78267cd4ec5a9b49aa11bbb1c ]
When BPF_TRAMP_F_CALL_ORIG is enabled, the address of a bpf_tramp_image struct on the stack is passed during the size calculation pass and an address on the heap is passed during code generation. This may cause a heap buffer overflow if the heap address is tagged because emit_a64_mov_i64() will emit longer code than it did during the size calculation pass. The same problem could occur without tag-based KASAN if one of the 16-bit words of the stack address happened to be all-ones during the size calculation pass. Fix the problem by assuming the worst case (4 instructions) when calculating the size of the bpf_tramp_image address emission.
Fixes: 19d3c179a377 ("bpf, arm64: Fix trampoline for BPF_TRAMP_F_CALL_ORIG") Signed-off-by: Peter Collingbourne pcc@google.com Signed-off-by: Daniel Borkmann daniel@iogearbox.net Acked-by: Xu Kuohai xukuohai@huawei.com Link: https://linux-review.googlesource.com/id/I1496f2bc24fba7a1d492e16e2b94cf4371... Link: https://lore.kernel.org/bpf/20241018221644.3240898-1-pcc@google.com [Minor context change fixed.] Signed-off-by: Bin Lan bin.lan.cn@windriver.com Signed-off-by: He Zhe zhe.he@windriver.com --- Build test passed. --- arch/arm64/net/bpf_jit_comp.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 7c5156e7d31e..95724f58399b 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -1952,7 +1952,11 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, emit(A64_STR64I(A64_R(20), A64_SP, regs_off + 8), ctx);
if (flags & BPF_TRAMP_F_CALL_ORIG) { - emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); + /* for the first pass, assume the worst case */ + if (!ctx->image) + ctx->idx += 4; + else + emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); emit_call((const u64)__bpf_tramp_enter, ctx); }
@@ -1996,7 +2000,11 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
if (flags & BPF_TRAMP_F_CALL_ORIG) { im->ip_epilogue = ctx->image + ctx->idx; - emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); + /* for the first pass, assume the worst case */ + if (!ctx->image) + ctx->idx += 4; + else + emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); emit_call((const u64)__bpf_tramp_exit, ctx); }
[ Sasha's backport helper bot ]
Hi,
✅ All tests passed successfully. No issues detected. No action required from the submitter.
The upstream commit SHA1 provided is correct: a552e2ef5fd1a6c78267cd4ec5a9b49aa11bbb1c
WARNING: Author mismatch between patch and upstream commit: Backport author: bin.lan.cn@windriver.com Commit author: Peter Collingbournepcc@google.com
Status in newer kernel trees: 6.14.y | Present (exact SHA1) 6.12.y | Present (exact SHA1)
Note: The patch differs from the upstream commit: --- 1: a552e2ef5fd1a ! 1: c1cc7847f8445 bpf, arm64: Fix address emission with tag-based KASAN enabled @@ Metadata ## Commit message ## bpf, arm64: Fix address emission with tag-based KASAN enabled
+ [ Upstream commit a552e2ef5fd1a6c78267cd4ec5a9b49aa11bbb1c ] + When BPF_TRAMP_F_CALL_ORIG is enabled, the address of a bpf_tramp_image struct on the stack is passed during the size calculation pass and an address on the heap is passed during code generation. This may @@ Commit message Acked-by: Xu Kuohai xukuohai@huawei.com Link: https://linux-review.googlesource.com/id/I1496f2bc24fba7a1d492e16e2b94cf4371... Link: https://lore.kernel.org/bpf/20241018221644.3240898-1-pcc@google.com + [Minor context change fixed.] + Signed-off-by: Bin Lan bin.lan.cn@windriver.com + Signed-off-by: He Zhe zhe.he@windriver.com
## arch/arm64/net/bpf_jit_comp.c ## @@ arch/arm64/net/bpf_jit_comp.c: static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, @@ arch/arm64/net/bpf_jit_comp.c: static int prepare_trampoline(struct jit_ctx *ctx @@ arch/arm64/net/bpf_jit_comp.c: static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
if (flags & BPF_TRAMP_F_CALL_ORIG) { - im->ip_epilogue = ctx->ro_image + ctx->idx; + im->ip_epilogue = ctx->image + ctx->idx; - emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); + /* for the first pass, assume the worst case */ + if (!ctx->image) ---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-6.12.y | Success | Success |
[ Sasha's backport helper bot ]
Hi,
Summary of potential issues: ⚠️ Found follow-up fixes in mainline
The upstream commit SHA1 provided is correct: 19d3c179a37730caf600a97fed3794feac2b197b
WARNING: Author mismatch between patch and upstream commit: Backport author: bin.lan.cn@windriver.com Commit author: Puranjay Mohanpuranjay@kernel.org
Status in newer kernel trees: 6.14.y | Present (exact SHA1) 6.12.y | Present (exact SHA1)
Found fixes commits: a552e2ef5fd1 bpf, arm64: Fix address emission with tag-based KASAN enabled
Note: The patch differs from the upstream commit: --- 1: 19d3c179a3773 ! 1: 5d90fc4a6da2b bpf, arm64: Fix trampoline for BPF_TRAMP_F_CALL_ORIG @@ Metadata ## Commit message ## bpf, arm64: Fix trampoline for BPF_TRAMP_F_CALL_ORIG
+ [ Upstream commit 19d3c179a37730caf600a97fed3794feac2b197b ] + When BPF_TRAMP_F_CALL_ORIG is set, the trampoline calls __bpf_tramp_enter() and __bpf_tramp_exit() functions, passing them the struct bpf_tramp_image *im pointer as an argument in R0. @@ Commit message Signed-off-by: Daniel Borkmann daniel@iogearbox.net Closes: https://lore.kernel.org/all/SJ0PR15MB461564D3F7E7A763498CA6A8CBDB2@SJ0PR15MB... Link: https://lore.kernel.org/bpf/20240711151838.43469-1-puranjay@kernel.org + [Minor context change fixed.] + Signed-off-by: Bin Lan bin.lan.cn@windriver.com + Signed-off-by: He Zhe zhe.he@windriver.com
## arch/arm64/net/bpf_jit_comp.c ## @@ arch/arm64/net/bpf_jit_comp.c: static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, @@ arch/arm64/net/bpf_jit_comp.c: static int prepare_trampoline(struct jit_ctx *ctx @@ arch/arm64/net/bpf_jit_comp.c: static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
if (flags & BPF_TRAMP_F_CALL_ORIG) { - im->ip_epilogue = ctx->ro_image + ctx->idx; + im->ip_epilogue = ctx->image + ctx->idx; - emit_addr_mov_i64(A64_R(0), (const u64)im, ctx); + emit_a64_mov_i64(A64_R(0), (const u64)im, ctx); emit_call((const u64)__bpf_tramp_exit, ctx); ---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-6.6.y | Success | Success |
linux-stable-mirror@lists.linaro.org
-
bin.lan.cn＠windriver.com -
Sasha Levin