Re: [tip: sched/core] sched/tracing: Don't re-read p->state when emitting sched_switch event
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
Thanks!
-- Qais Yousef
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
Once it hits Linus's tree, let stable@vger.kernel.org know what the git commit id is in Linus's tree.
thanks,
greg k-h
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Not sure about the 2nd patch, but I can try to figure it out.
Once it hits Linus's tree, let stable@vger.kernel.org know what the git commit id is in Linus's tree.
Sure. My bad if I rushed the request. I just wanted to know whether it's okay for this to go into stable. If no one shouts, I'll ping once it lands in Linus'.
Thanks!
-- Qais Yousef
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
Thanks!
-- Qais Yousef
On 04/10/22 00:38, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
commit: fa2c3254d7cfff5f7a916ab928a562d1165f17bb subject: sched/tracing: Don't re-read p->state when emitting sched_switch event
This patch has an impact on Android 5.10 users who experience tooling breakage. Is it possible to include in 5.10 LTS please?
It was already picked up for 5.15+ by AUTOSEL and only 5.10 is missing.
Thanks
-- Qais Yousef
On 2022-04-11 00:06, Qais Yousef wrote:
On 04/10/22 00:38, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
commit: fa2c3254d7cfff5f7a916ab928a562d1165f17bb subject: sched/tracing: Don't re-read p->state when emitting sched_switch event
This patch has an impact on Android 5.10 users who experience tooling breakage. Is it possible to include in 5.10 LTS please?
It was already picked up for 5.15+ by AUTOSEL and only 5.10 is missing.
https://lore.kernel.org/stable/Yk2PQzynOVOzJdPo@kroah.com/
However, since then further investigation (still in progress) has shown that this may have been the fault of the tool in question, so if you can verify that tracing sched still works for you with this patch in 5.15.x then by all means let's merge it.
-h
On 2022-04-11 01:22, Holger Hoffstätte wrote:
On 2022-04-11 00:06, Qais Yousef wrote:
On 04/10/22 00:38, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote: > The following commit has been merged into the sched/core branch of tip: > > Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb > Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb > Author: Valentin Schneider valentin.schneider@arm.com > AuthorDate: Thu, 20 Jan 2022 16:25:19 > Committer: Peter Zijlstra peterz@infradead.org > CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00 > > sched/tracing: Don't re-read p->state when emitting sched_switch event > > As of commit > > c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu") > > the following sequence becomes possible: > > p->__state = TASK_INTERRUPTIBLE; > __schedule() > deactivate_task(p); > ttwu() > READ !p->on_rq > p->__state=TASK_WAKING > trace_sched_switch() > __trace_sched_switch_state() > task_state_index() > return 0; > > TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in > the trace event. > > Prevent this by pushing the value read from __schedule() down the trace > event. > > Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com > Signed-off-by: Valentin Schneider valentin.schneider@arm.com > Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org > Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org > Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
commit: fa2c3254d7cfff5f7a916ab928a562d1165f17bb subject: sched/tracing: Don't re-read p->state when emitting sched_switch event
This patch has an impact on Android 5.10 users who experience tooling breakage. Is it possible to include in 5.10 LTS please?
It was already picked up for 5.15+ by AUTOSEL and only 5.10 is missing.
https://lore.kernel.org/stable/Yk2PQzynOVOzJdPo@kroah.com/
However, since then further investigation (still in progress) has shown that this may have been the fault of the tool in question, so if you can verify that tracing sched still works for you with this patch in 5.15.x then by all means let's merge it.
So it turns out the lockup is indeed the fault of the tool, which contains multiple kernel-version dependent tracepoint definitions and now fails with this patch.
Greg, please re-enqueue this patch where necessary (5.10, 5.15+)
-h
On Mon, Apr 11, 2022 at 09:18:19AM +0200, Holger Hoffstätte wrote:
On 2022-04-11 01:22, Holger Hoffstätte wrote:
On 2022-04-11 00:06, Qais Yousef wrote:
On 04/10/22 00:38, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote: > +CC stable > > On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote: > > The following commit has been merged into the sched/core branch of tip: > > > > Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb > > Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb > > Author: Valentin Schneider valentin.schneider@arm.com > > AuthorDate: Thu, 20 Jan 2022 16:25:19 > > Committer: Peter Zijlstra peterz@infradead.org > > CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00 > > > > sched/tracing: Don't re-read p->state when emitting sched_switch event > > > > As of commit > > > > c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu") > > > > the following sequence becomes possible: > > > > p->__state = TASK_INTERRUPTIBLE; > > __schedule() > > deactivate_task(p); > > ttwu() > > READ !p->on_rq > > p->__state=TASK_WAKING > > trace_sched_switch() > > __trace_sched_switch_state() > > task_state_index() > > return 0; > > > > TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in > > the trace event. > > > > Prevent this by pushing the value read from __schedule() down the trace > > event. > > > > Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com > > Signed-off-by: Valentin Schneider valentin.schneider@arm.com > > Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org > > Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org > > Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com > > Any objection to picking this for stable? I'm interested in this one for some > Android users but prefer if it can be taken by stable rather than backport it > individually. > > I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
commit: fa2c3254d7cfff5f7a916ab928a562d1165f17bb subject: sched/tracing: Don't re-read p->state when emitting sched_switch event
This patch has an impact on Android 5.10 users who experience tooling breakage. Is it possible to include in 5.10 LTS please?
It was already picked up for 5.15+ by AUTOSEL and only 5.10 is missing.
https://lore.kernel.org/stable/Yk2PQzynOVOzJdPo@kroah.com/
However, since then further investigation (still in progress) has shown that this may have been the fault of the tool in question, so if you can verify that tracing sched still works for you with this patch in 5.15.x then by all means let's merge it.
So it turns out the lockup is indeed the fault of the tool, which contains multiple kernel-version dependent tracepoint definitions and now fails with this patch.
What tools is this?
Greg, please re-enqueue this patch where necessary (5.10, 5.15+)
If I queue it up again, will the tools keep breaking?
thanks,
greg k-hj
On 2022-04-11 09:28, Greg KH wrote:
On Mon, Apr 11, 2022 at 09:18:19AM +0200, Holger Hoffstätte wrote:
On 2022-04-11 01:22, Holger Hoffstätte wrote:
On 2022-04-11 00:06, Qais Yousef wrote:
On 04/10/22 00:38, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote: > On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote: >> +CC stable >> >> On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote: >>> The following commit has been merged into the sched/core branch of tip: >>> >>> Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb >>> Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb >>> Author: Valentin Schneider valentin.schneider@arm.com >>> AuthorDate: Thu, 20 Jan 2022 16:25:19 >>> Committer: Peter Zijlstra peterz@infradead.org >>> CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00 >>> >>> sched/tracing: Don't re-read p->state when emitting sched_switch event >>> >>> As of commit >>> >>> c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu") >>> >>> the following sequence becomes possible: >>> >>> p->__state = TASK_INTERRUPTIBLE; >>> __schedule() >>> deactivate_task(p); >>> ttwu() >>> READ !p->on_rq >>> p->__state=TASK_WAKING >>> trace_sched_switch() >>> __trace_sched_switch_state() >>> task_state_index() >>> return 0; >>> >>> TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in >>> the trace event. >>> >>> Prevent this by pushing the value read from __schedule() down the trace >>> event. >>> >>> Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com >>> Signed-off-by: Valentin Schneider valentin.schneider@arm.com >>> Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org >>> Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org >>> Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com >> >> Any objection to picking this for stable? I'm interested in this one for some >> Android users but prefer if it can be taken by stable rather than backport it >> individually. >> >> I think it makes sense to pick the next one in the series too. > > What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
commit: fa2c3254d7cfff5f7a916ab928a562d1165f17bb subject: sched/tracing: Don't re-read p->state when emitting sched_switch event
This patch has an impact on Android 5.10 users who experience tooling breakage. Is it possible to include in 5.10 LTS please?
It was already picked up for 5.15+ by AUTOSEL and only 5.10 is missing.
https://lore.kernel.org/stable/Yk2PQzynOVOzJdPo@kroah.com/
However, since then further investigation (still in progress) has shown that this may have been the fault of the tool in question, so if you can verify that tracing sched still works for you with this patch in 5.15.x then by all means let's merge it.
So it turns out the lockup is indeed the fault of the tool, which contains multiple kernel-version dependent tracepoint definitions and now fails with this patch.
What tools is this?
sysdig - which uses a helper kernel module which accesses tracepoints, but of course (as I just found) with copypasta'd TP definitions, which broke with this patch due to the additional parameter in the function signature. It's been prone to breakage forever because of a lack of a stable kernel ABI.
Took me a while to find/figure out, but IMHO better safe than sorry. We've had autoselected scheduler patches before that looked fine but really were not.
Greg, please re-enqueue this patch where necessary (5.10, 5.15+)
If I queue it up again, will the tools keep breaking?
Yes, but that's their problem with an out-of-tree module; a few more #ifdefs are not going to make a big difference.
thanks Holger
On Mon, Apr 11, 2022 at 10:05:19AM +0200, Holger Hoffstätte wrote:
On 2022-04-11 09:28, Greg KH wrote:
On Mon, Apr 11, 2022 at 09:18:19AM +0200, Holger Hoffstätte wrote:
On 2022-04-11 01:22, Holger Hoffstätte wrote:
On 2022-04-11 00:06, Qais Yousef wrote:
On 04/10/22 00:38, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote: > On 03/08/22 19:10, Greg KH wrote: > > On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote: > > > +CC stable > > > > > > On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote: > > > > The following commit has been merged into the sched/core branch of tip: > > > > > > > > Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb > > > > Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb > > > > Author: Valentin Schneider valentin.schneider@arm.com > > > > AuthorDate: Thu, 20 Jan 2022 16:25:19 > > > > Committer: Peter Zijlstra peterz@infradead.org > > > > CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00 > > > > > > > > sched/tracing: Don't re-read p->state when emitting sched_switch event > > > > > > > > As of commit > > > > > > > > c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu") > > > > > > > > the following sequence becomes possible: > > > > > > > > p->__state = TASK_INTERRUPTIBLE; > > > > __schedule() > > > > deactivate_task(p); > > > > ttwu() > > > > READ !p->on_rq > > > > p->__state=TASK_WAKING > > > > trace_sched_switch() > > > > __trace_sched_switch_state() > > > > task_state_index() > > > > return 0; > > > > > > > > TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in > > > > the trace event. > > > > > > > > Prevent this by pushing the value read from __schedule() down the trace > > > > event. > > > > > > > > Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com > > > > Signed-off-by: Valentin Schneider valentin.schneider@arm.com > > > > Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org > > > > Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org > > > > Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com > > > > > > Any objection to picking this for stable? I'm interested in this one for some > > > Android users but prefer if it can be taken by stable rather than backport it > > > individually. > > > > > > I think it makes sense to pick the next one in the series too. > > > > What commit does this fix in Linus's tree? > > It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
commit: fa2c3254d7cfff5f7a916ab928a562d1165f17bb subject: sched/tracing: Don't re-read p->state when emitting sched_switch event
This patch has an impact on Android 5.10 users who experience tooling breakage. Is it possible to include in 5.10 LTS please?
It was already picked up for 5.15+ by AUTOSEL and only 5.10 is missing.
https://lore.kernel.org/stable/Yk2PQzynOVOzJdPo@kroah.com/
However, since then further investigation (still in progress) has shown that this may have been the fault of the tool in question, so if you can verify that tracing sched still works for you with this patch in 5.15.x then by all means let's merge it.
So it turns out the lockup is indeed the fault of the tool, which contains multiple kernel-version dependent tracepoint definitions and now fails with this patch.
What tools is this?
sysdig - which uses a helper kernel module which accesses tracepoints, but of course (as I just found) with copypasta'd TP definitions, which broke with this patch due to the additional parameter in the function signature. It's been prone to breakage forever because of a lack of a stable kernel ABI.
Took me a while to find/figure out, but IMHO better safe than sorry. We've had autoselected scheduler patches before that looked fine but really were not.
Thanks for the info, sysdig does do some pretty intrusive things but I thought it had moved over to being almost all bpf code.
Anyway, I'll wait for Qais to submit a properly backported and tested set of patches before picking this up again due to all of the problems I've had with this patch.
thanks,
greg k-h
On Sun, Apr 10, 2022 at 12:38:29AM +0100, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
It does not apply to 5.10 at all, how did you test this?
{sigh}
Again, if you want this applied to any stable trees, please test that it works and send the properly backported patches.
thanks,
greg k-h
On 04/11/22 15:22, Greg KH wrote:
On Sun, Apr 10, 2022 at 12:38:29AM +0100, Qais Yousef wrote:
On 03/08/22 18:51, Qais Yousef wrote:
On 03/08/22 19:10, Greg KH wrote:
On Tue, Mar 08, 2022 at 06:02:40PM +0000, Qais Yousef wrote:
+CC stable
On 03/01/22 15:24, tip-bot2 for Valentin Schneider wrote:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: fa2c3254d7cfff5f7a916ab928a562d1165f17bb Gitweb: https://git.kernel.org/tip/fa2c3254d7cfff5f7a916ab928a562d1165f17bb Author: Valentin Schneider valentin.schneider@arm.com AuthorDate: Thu, 20 Jan 2022 16:25:19 Committer: Peter Zijlstra peterz@infradead.org CommitterDate: Tue, 01 Mar 2022 16:18:39 +01:00
sched/tracing: Don't re-read p->state when emitting sched_switch event
As of commit
c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
the following sequence becomes possible:
p->__state = TASK_INTERRUPTIBLE; __schedule() deactivate_task(p);ttwu() READ !p->on_rq p->__state=TASK_WAKING trace_sched_switch() __trace_sched_switch_state() task_state_index() return 0;
TASK_WAKING isn't in TASK_REPORT, so the task appears as TASK_RUNNING in the trace event.
Prevent this by pushing the value read from __schedule() down the trace event.
Reported-by: Abhijeet Dharmapurikar adharmap@quicinc.com Signed-off-by: Valentin Schneider valentin.schneider@arm.com Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Reviewed-by: Steven Rostedt (Google) rostedt@goodmis.org Link: https://lore.kernel.org/r/20220120162520.570782-2-valentin.schneider@arm.com
Any objection to picking this for stable? I'm interested in this one for some Android users but prefer if it can be taken by stable rather than backport it individually.
I think it makes sense to pick the next one in the series too.
What commit does this fix in Linus's tree?
It should be this one: c6e7bd7afaeb ("sched/core: Optimize ttwu() spinning on p->on_cpu")
Should this be okay to be picked up by stable now? I can see AUTOSEL has picked it up for v5.15+, but it impacts v5.10 too.
It does not apply to 5.10 at all, how did you test this?
{sigh}
Again, if you want this applied to any stable trees, please test that it works and send the properly backported patches.
Sorry. I'm picking up pieces after a colleague left and was under the impression that 5.10 Android users already use as-is, but due to GKI just need it formally in stable. Few things got lost in translation, I am very sorry!
Let me sort all of this out properly including the testing part for 5.15 too.
Thanks!
-- Qais Yousef
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Holger Hoffstätte -
Qais Yousef