From: Aaron Thompson dev@aaront.org
[ Upstream commit f31dcb152a3d0816e2f1deab4e64572336da197d ]
Have local_clock() return sched_clock() if sched_clock_init() has not yet run. sched_clock_cpu() has this check but it was not included in the new noinstr implementation of local_clock().
The effect can be seen on x86 with CONFIG_PRINTK_TIME enabled, for instance. scd->clock quickly reaches the value of TICK_NSEC and that value is returned until sched_clock_init() runs.
dmesg without this patch:
[ 0.000000] kvm-clock: ... [ 0.000002] kvm-clock: ... [ 0.000672] clocksource: ... [ 0.001000] tsc: ... [ 0.001000] e820: ... [ 0.001000] e820: ... ... [ 0.001000] ..TIMER: ... [ 0.001000] clocksource: ... [ 0.378956] Calibrating delay loop ... [ 0.379955] pid_max: ...
dmesg with this patch:
[ 0.000000] kvm-clock: ... [ 0.000001] kvm-clock: ... [ 0.000675] clocksource: ... [ 0.002685] tsc: ... [ 0.003331] e820: ... [ 0.004190] e820: ... ... [ 0.421939] ..TIMER: ... [ 0.422842] clocksource: ... [ 0.424582] Calibrating delay loop ... [ 0.425580] pid_max: ...
Fixes: 776f22913b8e ("sched/clock: Make local_clock() noinstr") Signed-off-by: Aaron Thompson dev@aaront.org Signed-off-by: Peter Zijlstra (Intel) peterz@infradead.org Link: https://lkml.kernel.org/r/20230413175012.2201-1-dev@aaront.org Signed-off-by: Sasha Levin sashal@kernel.org --- kernel/sched/clock.c | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c index 5732fa75ebab2..b5cc2b53464de 100644 --- a/kernel/sched/clock.c +++ b/kernel/sched/clock.c @@ -300,6 +300,9 @@ noinstr u64 local_clock(void) if (static_branch_likely(&__sched_clock_stable)) return sched_clock() + __sched_clock_offset;
+ if (!static_branch_likely(&sched_clock_running)) + return sched_clock(); + preempt_disable_notrace(); clock = sched_clock_local(this_scd()); preempt_enable_notrace();