In tick_cpu_dying(), if the dying CPU is the current timekeeper, it has to pass the job over to another CPU. The current code passes it to another online CPU. However, that CPU may not be a timer tick housekeeping CPU. If that happens, another CPU will have to manually take it over again later. Avoid this unnecessary work by directly assigning an online housekeeping CPU.
Use READ_ONCE/WRITE_ONCE() to access tick_do_timer_cpu in case the non-HK CPUs may not be in stop machine in the future.
Signed-off-by: Waiman Long longman@redhat.com --- kernel/time/tick-common.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 9a3859443c04..6d5ff85281cc 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -17,6 +17,7 @@ #include <linux/profile.h> #include <linux/sched.h> #include <linux/module.h> +#include <linux/sched/isolation.h> #include <trace/events/power.h>
#include <asm/irq_regs.h> @@ -394,12 +395,18 @@ int tick_cpu_dying(unsigned int dying_cpu) { /* * If the current CPU is the timekeeper, it's the only one that can - * safely hand over its duty. Also all online CPUs are in stop - * machine, guaranteed not to be idle, therefore there is no + * safely hand over its duty. Also all online housekeeping CPUs are + * in stop machine, guaranteed not to be idle, therefore there is no * concurrency and it's safe to pick any online successor. */ - if (tick_do_timer_cpu == dying_cpu) - tick_do_timer_cpu = cpumask_first(cpu_online_mask); + if (READ_ONCE(tick_do_timer_cpu) == dying_cpu) { + unsigned int new_cpu; + + new_cpu = cpumask_first_and(cpu_online_mask, housekeeping_cpumask(HK_TYPE_TICK)); + if (WARN_ON_ONCE(new_cpu >= nr_cpu_ids)) + new_cpu = cpumask_first(cpu_online_mask); + WRITE_ONCE(tick_do_timer_cpu, new_cpu); + }
/* Make sure the CPU won't try to retake the timekeeping duty */ tick_sched_timer_dying(dying_cpu);