From: Peter Zijlstra peterz@infradead.org
[ Upstream commit 517e6a301f34613bff24a8e35b5455884f2d83d8 ]
Per syzbot it is possible for perf_pending_task() to run after the event is free()'d. There are two related but distinct cases:
- the task_work was already queued before destroying the event; - destroying the event itself queues the task_work.
The first cannot be solved using task_work_cancel() since perf_release() itself might be called from a task_work (____fput), which means the current->task_works list is already empty and task_work_cancel() won't be able to find the perf_pending_task() entry.
The simplest alternative is extending the perf_event lifetime to cover the task_work.
The second is just silly, queueing a task_work while you know the event is going away makes no sense and is easily avoided by re-arranging how the event is marked STATE_DEAD and ensuring it goes through STATE_OFF on the way down.
Reported-by: syzbot+9228d6098455bb209ec8@syzkaller.appspotmail.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Tested-by: Marco Elver elver@google.com [ Discard the changes in event_sched_out() due to 5.10 don't have the commit: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") and commit: ca6c21327c6a ("perf: Fix missing SIGTRAPs") ] Signed-off-by: Xiangyu Chen xiangyu.chen@windriver.com Signed-off-by: He Zhe zhe.he@windriver.com --- Verified the build test. --- kernel/events/core.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c index 8f19d6ab039e..798c839a00b3 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -2419,6 +2419,7 @@ group_sched_out(struct perf_event *group_event, }
#define DETACH_GROUP 0x01UL +#define DETACH_DEAD 0x04UL
/* * Cross CPU call to remove a performance event @@ -2439,10 +2440,18 @@ __perf_remove_from_context(struct perf_event *event, update_cgrp_time_from_cpuctx(cpuctx, false); }
+ /* + * Ensure event_sched_out() switches to OFF, at the very least + * this avoids raising perf_pending_task() at this time. + */ + if (flags & DETACH_DEAD) + event->pending_disable = 1; event_sched_out(event, cpuctx, ctx); if (flags & DETACH_GROUP) perf_group_detach(event); list_del_event(event, ctx); + if (flags & DETACH_DEAD) + event->state = PERF_EVENT_STATE_DEAD;
if (!ctx->nr_events && ctx->is_active) { if (ctx == &cpuctx->ctx) @@ -5111,9 +5120,7 @@ int perf_event_release_kernel(struct perf_event *event)
ctx = perf_event_ctx_lock(event); WARN_ON_ONCE(ctx->parent_ctx); - perf_remove_from_context(event, DETACH_GROUP);
- raw_spin_lock_irq(&ctx->lock); /* * Mark this event as STATE_DEAD, there is no external reference to it * anymore. @@ -5125,8 +5132,7 @@ int perf_event_release_kernel(struct perf_event *event) * Thus this guarantees that we will in fact observe and kill _ALL_ * child events. */ - event->state = PERF_EVENT_STATE_DEAD; - raw_spin_unlock_irq(&ctx->lock); + perf_remove_from_context(event, DETACH_GROUP|DETACH_DEAD);
perf_event_ctx_unlock(event, ctx);
@@ -6533,6 +6539,8 @@ static void perf_pending_event(struct irq_work *entry)
if (rctx >= 0) perf_swevent_put_recursion_context(rctx); + + put_event(event); }
/*
[ Sasha's backport helper bot ]
Hi,
Summary of potential issues: ❌ Build failures detected ⚠️ Found follow-up fixes in mainline
The upstream commit SHA1 provided is correct: 517e6a301f34613bff24a8e35b5455884f2d83d8
WARNING: Author mismatch between patch and upstream commit: Backport author: Xiangyu Chenxiangyu.chen@eng.windriver.com Commit author: Peter Zijlstrapeterz@infradead.org
Status in newer kernel trees: 6.14.y | Present (exact SHA1) 6.13.y | Present (exact SHA1) 6.12.y | Present (exact SHA1) 6.6.y | Present (exact SHA1) 6.1.y | Present (exact SHA1) 5.15.y | Present (different SHA1: 8bffa95ac19f)
Found fixes commits: 3a5465418f5f perf: Fix event leak upon exec and file release 2fd5ad3f310d perf: Fix event leak upon exit
Note: The patch differs from the upstream commit: --- 1: 517e6a301f346 ! 1: b2173ec15f3b2 perf: Fix perf_pending_task() UaF @@ Metadata ## Commit message ## perf: Fix perf_pending_task() UaF
+ [ Upstream commit 517e6a301f34613bff24a8e35b5455884f2d83d8 ] + Per syzbot it is possible for perf_pending_task() to run after the event is free()'d. There are two related but distinct cases:
@@ Commit message Reported-by: syzbot+9228d6098455bb209ec8@syzkaller.appspotmail.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Tested-by: Marco Elver elver@google.com + [ Discard the changes in event_sched_out() due to 5.10 don't have the + commit: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") + and commit: ca6c21327c6a ("perf: Fix missing SIGTRAPs") ] + Signed-off-by: Xiangyu Chen xiangyu.chen@windriver.com + Signed-off-by: He Zhe zhe.he@windriver.com
## kernel/events/core.c ## -@@ kernel/events/core.c: event_sched_out(struct perf_event *event, - !event->pending_work) { - event->pending_work = 1; - dec = false; -+ WARN_ON_ONCE(!atomic_long_inc_not_zero(&event->refcount)); - task_work_add(current, &event->pending_task, TWA_RESUME); - } - if (dec) @@ kernel/events/core.c: group_sched_out(struct perf_event *group_event, + }
#define DETACH_GROUP 0x01UL - #define DETACH_CHILD 0x02UL +#define DETACH_DEAD 0x04UL
/* @@ kernel/events/core.c: __perf_remove_from_context(struct perf_event *event, event_sched_out(event, cpuctx, ctx); if (flags & DETACH_GROUP) perf_group_detach(event); - if (flags & DETACH_CHILD) - perf_child_detach(event); list_del_event(event, ctx); + if (flags & DETACH_DEAD) + event->state = PERF_EVENT_STATE_DEAD; @@ kernel/events/core.c: int perf_event_release_kernel(struct perf_event *event)
perf_event_ctx_unlock(event, ctx);
-@@ kernel/events/core.c: static void perf_pending_task(struct callback_head *head) +@@ kernel/events/core.c: static void perf_pending_event(struct irq_work *entry) + if (rctx >= 0) perf_swevent_put_recursion_context(rctx); - preempt_enable_notrace(); + + put_event(event); }
- #ifdef CONFIG_GUEST_PERF_EVENTS + /* ---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-5.10.y | Success | Failed |
Build Errors: Build error for stable/linux-5.10.y: kernel/trace/trace_events_synth.c: In function 'synth_event_reg': kernel/trace/trace_events_synth.c:769:9: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement] 769 | int ret = trace_event_reg(call, type, data); | ^~~ In file included from ./include/linux/kernel.h:15, from ./include/linux/list.h:9, from ./include/linux/kobject.h:19, from ./include/linux/of.h:17, from ./include/linux/clk-provider.h:9, from drivers/clk/qcom/clk-rpmh.c:6: drivers/clk/qcom/clk-rpmh.c: In function 'clk_rpmh_bcm_send_cmd': ./include/linux/minmax.h:20:35: warning: comparison of distinct pointer types lacks a cast [-Wcompare-distinct-pointer-types] 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ ./include/linux/minmax.h:26:18: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ ./include/linux/minmax.h:36:31: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ ./include/linux/minmax.h:45:25: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ drivers/clk/qcom/clk-rpmh.c:273:21: note: in expansion of macro 'min' 273 | cmd_state = min(cmd_state, BCM_TCS_CMD_VOTE_MASK); | ^~~ drivers/firmware/efi/mokvar-table.c: In function 'efi_mokvar_table_init': drivers/firmware/efi/mokvar-table.c:107:23: warning: unused variable 'size' [-Wunused-variable] 107 | unsigned long size; | ^~~~ .tmp_vmlinux.kallsyms2.S:196892:57: internal compiler error: Segmentation fault 196892 | .byte 0x0b, 0x74, 0x77, 0x77, 0x5f, 0xb6, 0x73, 0xfc, 0x6e, 0xbd, 0x6d, 0xed | ^~~~ 0x7f74e8f6cd1f ??? ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 0x7f74e8f56d67 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 0x7f74e8f56e24 __libc_start_main_impl ../csu/libc-start.c:360 Please submit a full bug report, with preprocessed source (by using -freport-bug). Please include the complete backtrace with any bug report. See https://gcc.gnu.org/bugs/ for instructions. make: *** [Makefile:1212: vmlinux] Error 1 make: Target '__all' not remade because of errors.
Hi,
Could you share the config with me so that I can reproduce the issue on my setup?
I have tested with x86_64 default config without error, seems there is something different with my local setup.
Thanks.
Br,
Xiangyu
On 4/10/25 23:53, Sasha Levin wrote:
CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
[ Sasha's backport helper bot ]
Hi,
Summary of potential issues: ❌ Build failures detected ⚠️ Found follow-up fixes in mainline
The upstream commit SHA1 provided is correct: 517e6a301f34613bff24a8e35b5455884f2d83d8
WARNING: Author mismatch between patch and upstream commit: Backport author: Xiangyu Chenxiangyu.chen@eng.windriver.com Commit author: Peter Zijlstrapeterz@infradead.org
Status in newer kernel trees: 6.14.y | Present (exact SHA1) 6.13.y | Present (exact SHA1) 6.12.y | Present (exact SHA1) 6.6.y | Present (exact SHA1) 6.1.y | Present (exact SHA1) 5.15.y | Present (different SHA1: 8bffa95ac19f)
Found fixes commits: 3a5465418f5f perf: Fix event leak upon exec and file release 2fd5ad3f310d perf: Fix event leak upon exit
Note: The patch differs from the upstream commit:
1: 517e6a301f346 ! 1: b2173ec15f3b2 perf: Fix perf_pending_task() UaF @@ Metadata ## Commit message ## perf: Fix perf_pending_task() UaF
+ [ Upstream commit 517e6a301f34613bff24a8e35b5455884f2d83d8 ] + Per syzbot it is possible for perf_pending_task() to run after the event is free()'d. There are two related but distinct cases: @@ Commit message Reported-by: syzbot+9228d6098455bb209ec8@syzkaller.appspotmail.com Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Marco Elver <elver@google.com> + [ Discard the changes in event_sched_out() due to 5.10 don't have the + commit: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") + and commit: ca6c21327c6a ("perf: Fix missing SIGTRAPs") ] + Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com> + Signed-off-by: He Zhe <zhe.he@windriver.com> ## kernel/events/core.c ## -@@ kernel/events/core.c: event_sched_out(struct perf_event *event, - !event->pending_work) { - event->pending_work = 1; - dec = false; -+ WARN_ON_ONCE(!atomic_long_inc_not_zero(&event->refcount)); - task_work_add(current, &event->pending_task, TWA_RESUME); - } - if (dec) @@ kernel/events/core.c: group_sched_out(struct perf_event *group_event, + } #define DETACH_GROUP 0x01UL - #define DETACH_CHILD 0x02UL +#define DETACH_DEAD 0x04UL /* @@ kernel/events/core.c: __perf_remove_from_context(struct perf_event *event, event_sched_out(event, cpuctx, ctx); if (flags & DETACH_GROUP) perf_group_detach(event); - if (flags & DETACH_CHILD) - perf_child_detach(event); list_del_event(event, ctx); + if (flags & DETACH_DEAD) + event->state = PERF_EVENT_STATE_DEAD; @@ kernel/events/core.c: int perf_event_release_kernel(struct perf_event *event) perf_event_ctx_unlock(event, ctx); -@@ kernel/events/core.c: static void perf_pending_task(struct callback_head *head) +@@ kernel/events/core.c: static void perf_pending_event(struct irq_work *entry) + if (rctx >= 0) perf_swevent_put_recursion_context(rctx); - preempt_enable_notrace(); + + put_event(event); } - #ifdef CONFIG_GUEST_PERF_EVENTS + /*
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-5.10.y | Success | Failed |
Build Errors: Build error for stable/linux-5.10.y: kernel/trace/trace_events_synth.c: In function 'synth_event_reg': kernel/trace/trace_events_synth.c:769:9: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement] 769 | int ret = trace_event_reg(call, type, data); | ^~~ In file included from ./include/linux/kernel.h:15, from ./include/linux/list.h:9, from ./include/linux/kobject.h:19, from ./include/linux/of.h:17, from ./include/linux/clk-provider.h:9, from drivers/clk/qcom/clk-rpmh.c:6: drivers/clk/qcom/clk-rpmh.c: In function 'clk_rpmh_bcm_send_cmd': ./include/linux/minmax.h:20:35: warning: comparison of distinct pointer types lacks a cast [-Wcompare-distinct-pointer-types] 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ ./include/linux/minmax.h:26:18: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ ./include/linux/minmax.h:36:31: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ ./include/linux/minmax.h:45:25: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ drivers/clk/qcom/clk-rpmh.c:273:21: note: in expansion of macro 'min' 273 | cmd_state = min(cmd_state, BCM_TCS_CMD_VOTE_MASK); | ^~~ drivers/firmware/efi/mokvar-table.c: In function 'efi_mokvar_table_init': drivers/firmware/efi/mokvar-table.c:107:23: warning: unused variable 'size' [-Wunused-variable] 107 | unsigned long size; | ^~~~ .tmp_vmlinux.kallsyms2.S:196892:57: internal compiler error: Segmentation fault 196892 | .byte 0x0b, 0x74, 0x77, 0x77, 0x5f, 0xb6, 0x73, 0xfc, 0x6e, 0xbd, 0x6d, 0xed | ^~~~ 0x7f74e8f6cd1f ??? ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 0x7f74e8f56d67 __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 0x7f74e8f56e24 __libc_start_main_impl ../csu/libc-start.c:360 Please submit a full bug report, with preprocessed source (by using -freport-bug). Please include the complete backtrace with any bug report. See https://gcc.gnu.org/bugs/ for instructions. make: *** [Makefile:1212: vmlinux] Error 1 make: Target '__all' not remade because of errors.
On Tue, Apr 08, 2025 at 02:10:44PM +0800, Xiangyu Chen wrote:
From: Peter Zijlstra peterz@infradead.org
[ Upstream commit 517e6a301f34613bff24a8e35b5455884f2d83d8 ]
Per syzbot it is possible for perf_pending_task() to run after the event is free()'d. There are two related but distinct cases:
- the task_work was already queued before destroying the event;
- destroying the event itself queues the task_work.
The first cannot be solved using task_work_cancel() since perf_release() itself might be called from a task_work (____fput), which means the current->task_works list is already empty and task_work_cancel() won't be able to find the perf_pending_task() entry.
The simplest alternative is extending the perf_event lifetime to cover the task_work.
The second is just silly, queueing a task_work while you know the event is going away makes no sense and is easily avoided by re-arranging how the event is marked STATE_DEAD and ensuring it goes through STATE_OFF on the way down.
Reported-by: syzbot+9228d6098455bb209ec8@syzkaller.appspotmail.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Tested-by: Marco Elver elver@google.com [ Discard the changes in event_sched_out() due to 5.10 don't have the commit: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") and commit: ca6c21327c6a ("perf: Fix missing SIGTRAPs") ] Signed-off-by: Xiangyu Chen xiangyu.chen@windriver.com Signed-off-by: He Zhe zhe.he@windriver.com
Verified the build test.
You missed all of the fix-up patches for this commit that happened after it, fixing memory leaks and the like. So if we applied this, we would have more bugs added to the tree than fixed :(
ALWAYS check for follow-on fixes.
I'll go drop this.
greg k-h
Hi Greg,
On 4/23/25 22:15, Greg KH wrote:
CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Tue, Apr 08, 2025 at 02:10:44PM +0800, Xiangyu Chen wrote:
From: Peter Zijlstra peterz@infradead.org
[ Upstream commit 517e6a301f34613bff24a8e35b5455884f2d83d8 ]
Per syzbot it is possible for perf_pending_task() to run after the event is free()'d. There are two related but distinct cases:
- the task_work was already queued before destroying the event;
- destroying the event itself queues the task_work.
The first cannot be solved using task_work_cancel() since perf_release() itself might be called from a task_work (____fput), which means the current->task_works list is already empty and task_work_cancel() won't be able to find the perf_pending_task() entry.
The simplest alternative is extending the perf_event lifetime to cover the task_work.
The second is just silly, queueing a task_work while you know the event is going away makes no sense and is easily avoided by re-arranging how the event is marked STATE_DEAD and ensuring it goes through STATE_OFF on the way down.
Reported-by: syzbot+9228d6098455bb209ec8@syzkaller.appspotmail.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Tested-by: Marco Elver elver@google.com [ Discard the changes in event_sched_out() due to 5.10 don't have the commit: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") and commit: ca6c21327c6a ("perf: Fix missing SIGTRAPs") ] Signed-off-by: Xiangyu Chen xiangyu.chen@windriver.com Signed-off-by: He Zhe zhe.he@windriver.com
Verified the build test.
You missed all of the fix-up patches for this commit that happened after it, fixing memory leaks and the like. So if we applied this, we would have more bugs added to the tree than fixed :(
ALWAYS check for follow-on fixes.
I'll go drop this.
Thanks for your info, I have checked the full log and there is another commit to fix current commit,
Please ignore this patch , I will try to backport the fixes to 5.10 and resend the review to list after local testing.
Thanks.
Br,
Xiangyu
greg k-h
Hi Greg,
After investigating the follow-on patch, current commit 517e6a301f34 ("perf: Fix perf_pending_task() UaF") needs
to backport the following commits:
Commit 3a5465418f5f ("perf: Fix event leak upon exec and file release") -- Fix commit 517e6a301f34
Commit 56799bc03565 ("perf: Fix hang while freeing sigtrap event") -- Fix commit 56799bc03565
For the commit 3a5465418f5f ("perf: Fix event leak upon exec and file release"), it depends on commit ca6c21327c6a ("perf: Fix missing SIGTRAPs"),
but commit ca6c21327c6a ("perf: Fix missing SIGTRAPs") based on Commit 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") which is a new
feature intruduced since kernel 5.13-rc1, and the new feature changed the perf_event.h in uapi.
Fully backport the current commit and the follow-on patches too risky for a stable branch, so please ignore current patch, that
might cause more bugs for 5.10, thanks :)
Br,
Xiangyu
On 4/24/25 14:33, Xiangyu Chen wrote:
Hi Greg,
On 4/23/25 22:15, Greg KH wrote:
CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Tue, Apr 08, 2025 at 02:10:44PM +0800, Xiangyu Chen wrote:
From: Peter Zijlstra peterz@infradead.org
[ Upstream commit 517e6a301f34613bff24a8e35b5455884f2d83d8 ]
Per syzbot it is possible for perf_pending_task() to run after the event is free()'d. There are two related but distinct cases:
- the task_work was already queued before destroying the event; - destroying the event itself queues the task_work.
The first cannot be solved using task_work_cancel() since perf_release() itself might be called from a task_work (____fput), which means the current->task_works list is already empty and task_work_cancel() won't be able to find the perf_pending_task() entry.
The simplest alternative is extending the perf_event lifetime to cover the task_work.
The second is just silly, queueing a task_work while you know the event is going away makes no sense and is easily avoided by re-arranging how the event is marked STATE_DEAD and ensuring it goes through STATE_OFF on the way down.
Reported-by: syzbot+9228d6098455bb209ec8@syzkaller.appspotmail.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Tested-by: Marco Elver elver@google.com [ Discard the changes in event_sched_out() due to 5.10 don't have the commit: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events") and commit: ca6c21327c6a ("perf: Fix missing SIGTRAPs") ] Signed-off-by: Xiangyu Chen xiangyu.chen@windriver.com Signed-off-by: He Zhe zhe.he@windriver.com
Verified the build test.
You missed all of the fix-up patches for this commit that happened after it, fixing memory leaks and the like. So if we applied this, we would have more bugs added to the tree than fixed :(
ALWAYS check for follow-on fixes.
I'll go drop this.
Thanks for your info, I have checked the full log and there is another commit to fix current commit,
Please ignore this patch , I will try to backport the fixes to 5.10 and resend the review to list after local testing.
Thanks.
Br,
Xiangyu
greg k-h
linux-stable-mirror@lists.linaro.org