This is a backport of the series that recently went into 5.8. Note that the first patch is more a complete rewriting than a backport, as the vdso implementation in 5.4 doesn't have much in common with mainline. This affects the 32bit arch code in a benign way.
It has seen very little testing, as I don't have the HW that triggers this issue. I have run it in VMs by faking the CPU MIDR, and nothing caught fire. Famous last words.
The original cover letter follows.
M.
The relatively recent introduction of the compat vdso on arm64 has overlooked its interactions with some of the interesting errata workarounds, such as ARM64_ERRATUM_1418040 (and its older 1188873 incarnation).
This erratum requires the 64bit kernel to trap 32bit accesses to the virtual counter and emulate it. When the workaround was introduced, the compat vdso simply wasn't a thing. Now that the patches have landed in mainline, we trap the CVTVCT accesses from the vdso.
This can end-up in a nasty loop in the vdso, where the sequence number changes on each trap, never stabilising, and leaving userspace in a bit of a funny state (which is why we disable the vdso in most similar cases). This erratum mentionned above is a bit special in the sense that in only requires to trap AArch32 accesses, and 64bit tasks can be left alone. Consequently, the vdso is never disabled and AArch32 tasks are affected.
Obviously, we really want to retain the 64bit vdso in this case. To that effect, this series offers a way to disable the 32bit view of the vdso without impacting its 64bit counterpart, by providing a "no-compat" vdso clock_mode, and plugging this feature into the 1418040 detection code.
Marc Zyngier (3): arm64: Introduce a way to disable the 32bit vdso arm64: arch_timer: Allow an workaround descriptor to disable compat vdso arm64: arch_timer: Disable the compat vdso for cores affected by ARM64_WORKAROUND_1418040
arch/arm/include/asm/clocksource.h | 11 ++++++++++- arch/arm/kernel/vdso.c | 2 +- arch/arm64/include/asm/arch_timer.h | 1 + arch/arm64/include/asm/clocksource.h | 5 ++++- arch/arm64/include/asm/vdso/clocksource.h | 14 ++++++++++++++ .../include/asm/vdso/compat_gettimeofday.h | 5 +++-- arch/arm64/include/asm/vdso/gettimeofday.h | 6 ++++-- arch/arm64/include/asm/vdso/vsyscall.h | 4 +--- drivers/clocksource/arm_arch_timer.c | 19 +++++++++++++++---- 9 files changed, 53 insertions(+), 14 deletions(-) create mode 100644 arch/arm64/include/asm/vdso/clocksource.h
Commit 97884ca8c2925d14c32188e865069f21378b4b4f upstream.
[this is a redesign rather than a backport]
We have a class of errata (grouped under the ARM64_WORKAROUND_1418040 banner) that force the trapping of counter access from 32bit EL0.
We would normally disable the whole vdso for such defect, except that it would disable it for 64bit userspace as well, which is a shame.
Instead, add a new vdso_clock_mode, which signals that the vdso isn't usable for compat tasks. This gets checked in the __arch_get_hw_counter() helper.
Signed-off-by: Marc Zyngier maz@kernel.org Acked-by: Mark Rutland mark.rutland@arm.com Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20200706163802.1836732-2-maz@kernel.org Signed-off-by: Will Deacon will@kernel.org Signed-off-by: Marc Zyngier maz@kernel.org --- arch/arm/include/asm/clocksource.h | 11 ++++++++++- arch/arm/kernel/vdso.c | 2 +- arch/arm64/include/asm/clocksource.h | 5 ++++- arch/arm64/include/asm/vdso/clocksource.h | 14 ++++++++++++++ arch/arm64/include/asm/vdso/compat_gettimeofday.h | 5 +++-- arch/arm64/include/asm/vdso/gettimeofday.h | 6 ++++-- arch/arm64/include/asm/vdso/vsyscall.h | 4 +--- drivers/clocksource/arm_arch_timer.c | 8 ++++---- 8 files changed, 41 insertions(+), 14 deletions(-) create mode 100644 arch/arm64/include/asm/vdso/clocksource.h
diff --git a/arch/arm/include/asm/clocksource.h b/arch/arm/include/asm/clocksource.h index 0b350a7e26f3..afb7a59828fe 100644 --- a/arch/arm/include/asm/clocksource.h +++ b/arch/arm/include/asm/clocksource.h @@ -1,8 +1,17 @@ #ifndef _ASM_CLOCKSOURCE_H #define _ASM_CLOCKSOURCE_H
+enum vdso_arch_clockmode { + /* vdso clocksource not usable */ + VDSO_CLOCKMODE_NONE, + /* vdso clocksource usable */ + VDSO_CLOCKMODE_ARCHTIMER, + VDSO_CLOCKMODE_ARCHTIMER_NOCOMPAT = VDSO_CLOCKMODE_ARCHTIMER, +}; + struct arch_clocksource_data { - bool vdso_direct; /* Usable for direct VDSO access? */ + /* Usable for direct VDSO access? */ + enum vdso_arch_clockmode clock_mode; };
#endif diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c index f00e45fa62c4..6c69a5548ba2 100644 --- a/arch/arm/kernel/vdso.c +++ b/arch/arm/kernel/vdso.c @@ -281,7 +281,7 @@ static bool tk_is_cntvct(const struct timekeeper *tk) if (!IS_ENABLED(CONFIG_ARM_ARCH_TIMER)) return false;
- if (!tk->tkr_mono.clock->archdata.vdso_direct) + if (tk->tkr_mono.clock->archdata.clock_mode != VDSO_CLOCKMODE_ARCHTIMER) return false;
return true; diff --git a/arch/arm64/include/asm/clocksource.h b/arch/arm64/include/asm/clocksource.h index 0ece64a26c8c..0c7910447235 100644 --- a/arch/arm64/include/asm/clocksource.h +++ b/arch/arm64/include/asm/clocksource.h @@ -2,8 +2,11 @@ #ifndef _ASM_CLOCKSOURCE_H #define _ASM_CLOCKSOURCE_H
+#include <asm/vdso/clocksource.h> + struct arch_clocksource_data { - bool vdso_direct; /* Usable for direct VDSO access? */ + /* Usable for direct VDSO access? */ + enum vdso_arch_clockmode clock_mode; };
#endif diff --git a/arch/arm64/include/asm/vdso/clocksource.h b/arch/arm64/include/asm/vdso/clocksource.h new file mode 100644 index 000000000000..8019f616e1f7 --- /dev/null +++ b/arch/arm64/include/asm/vdso/clocksource.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_VDSOCLOCKSOURCE_H +#define __ASM_VDSOCLOCKSOURCE_H + +enum vdso_arch_clockmode { + /* vdso clocksource not usable */ + VDSO_CLOCKMODE_NONE, + /* vdso clocksource for both 32 and 64bit tasks */ + VDSO_CLOCKMODE_ARCHTIMER, + /* vdso clocksource for 64bit tasks only */ + VDSO_CLOCKMODE_ARCHTIMER_NOCOMPAT, +}; + +#endif diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h index c50ee1b7d5cd..413d42e197c7 100644 --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h @@ -10,6 +10,7 @@ #include <asm/unistd.h> #include <uapi/linux/time.h>
+#include <asm/vdso/clocksource.h> #include <asm/vdso/compat_barrier.h>
#define __VDSO_USE_SYSCALL ULLONG_MAX @@ -117,10 +118,10 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) u64 res;
/* - * clock_mode == 0 implies that vDSO are enabled otherwise + * clock_mode == ARCHTIMER implies that vDSO are enabled otherwise * fallback on syscall. */ - if (clock_mode) + if (clock_mode != VDSO_CLOCKMODE_ARCHTIMER) return __VDSO_USE_SYSCALL;
/* diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h index b08f476b72b4..ff83b8b574fc 100644 --- a/arch/arm64/include/asm/vdso/gettimeofday.h +++ b/arch/arm64/include/asm/vdso/gettimeofday.h @@ -10,6 +10,8 @@ #include <asm/unistd.h> #include <uapi/linux/time.h>
+#include <asm/vdso/clocksource.h> + #define __VDSO_USE_SYSCALL ULLONG_MAX
#define VDSO_HAS_CLOCK_GETRES 1 @@ -71,10 +73,10 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) u64 res;
/* - * clock_mode == 0 implies that vDSO are enabled otherwise + * clock_mode != NONE implies that vDSO are enabled otherwise * fallback on syscall. */ - if (clock_mode) + if (clock_mode == VDSO_CLOCKMODE_NONE) return __VDSO_USE_SYSCALL;
/* diff --git a/arch/arm64/include/asm/vdso/vsyscall.h b/arch/arm64/include/asm/vdso/vsyscall.h index 0c20a7c1bee5..e50f26741946 100644 --- a/arch/arm64/include/asm/vdso/vsyscall.h +++ b/arch/arm64/include/asm/vdso/vsyscall.h @@ -24,9 +24,7 @@ struct vdso_data *__arm64_get_k_vdso_data(void) static __always_inline int __arm64_get_clock_mode(struct timekeeper *tk) { - u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct; - - return use_syscall; + return tk->tkr_mono.clock->archdata.clock_mode; } #define __arch_get_clock_mode __arm64_get_clock_mode
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index 9a5464c625b4..909fe093249e 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -69,7 +69,7 @@ static enum arch_timer_ppi_nr arch_timer_uses_ppi = ARCH_TIMER_VIRT_PPI; static bool arch_timer_c3stop; static bool arch_timer_mem_use_virtual; static bool arch_counter_suspend_stop; -static bool vdso_default = true; +static enum vdso_arch_clockmode vdso_default = VDSO_CLOCKMODE_ARCHTIMER;
static cpumask_t evtstrm_available = CPU_MASK_NONE; static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM); @@ -560,8 +560,8 @@ void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa * change both the default value and the vdso itself. */ if (wa->read_cntvct_el0) { - clocksource_counter.archdata.vdso_direct = false; - vdso_default = false; + clocksource_counter.archdata.clock_mode = VDSO_CLOCKMODE_NONE; + vdso_default = VDSO_CLOCKMODE_NONE; } }
@@ -979,7 +979,7 @@ static void __init arch_counter_register(unsigned type) }
arch_timer_read_counter = rd; - clocksource_counter.archdata.vdso_direct = vdso_default; + clocksource_counter.archdata.clock_mode = vdso_default; } else { arch_timer_read_counter = arch_counter_get_cntvct_mem; }
Commit c1fbec4ac0d701f350a581941d35643d5a9cd184 upstream.
As we are about to disable the vdso for compat tasks in some circumstances, let's allow a workaround descriptor to express exactly that.
Signed-off-by: Marc Zyngier maz@kernel.org Acked-by: Mark Rutland mark.rutland@arm.com Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20200706163802.1836732-3-maz@kernel.org Signed-off-by: Will Deacon will@kernel.org Signed-off-by: Marc Zyngier maz@kernel.org --- arch/arm64/include/asm/arch_timer.h | 1 + drivers/clocksource/arm_arch_timer.c | 3 +++ 2 files changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index 7ae54d7d333a..9f0ec21d6327 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -58,6 +58,7 @@ struct arch_timer_erratum_workaround { u64 (*read_cntvct_el0)(void); int (*set_next_event_phys)(unsigned long, struct clock_event_device *); int (*set_next_event_virt)(unsigned long, struct clock_event_device *); + bool disable_compat_vdso; };
DECLARE_PER_CPU(const struct arch_timer_erratum_workaround *, diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index 909fe093249e..fd2a75f0af77 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -562,6 +562,9 @@ void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa if (wa->read_cntvct_el0) { clocksource_counter.archdata.clock_mode = VDSO_CLOCKMODE_NONE; vdso_default = VDSO_CLOCKMODE_NONE; + } else if (wa->disable_compat_vdso && vdso_default != VDSO_CLOCKMODE_NONE) { + vdso_default = VDSO_CLOCKMODE_ARCHTIMER_NOCOMPAT; + clocksource_counter.archdata.clock_mode = vdso_default; } }
Commit 4b661d6133c5d3a7c9aca0b4ee5a78c7766eff3f upstream.
ARM64_WORKAROUND_1418040 requires that AArch32 EL0 accesses to the virtual counter register are trapped and emulated by the kernel. This makes the vdso pretty pointless, and in some cases livelock prone.
Provide a workaround entry that limits the vdso to 64bit tasks.
Signed-off-by: Marc Zyngier maz@kernel.org Acked-by: Mark Rutland mark.rutland@arm.com Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20200706163802.1836732-4-maz@kernel.org Signed-off-by: Will Deacon will@kernel.org Signed-off-by: Marc Zyngier maz@kernel.org --- drivers/clocksource/arm_arch_timer.c | 8 ++++++++ 1 file changed, 8 insertions(+)
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index fd2a75f0af77..4be83b4de2a0 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -476,6 +476,14 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = { .set_next_event_virt = erratum_set_next_event_tval_virt, }, #endif +#ifdef CONFIG_ARM64_ERRATUM_1418040 + { + .match_type = ate_match_local_cap_id, + .id = (void *)ARM64_WORKAROUND_1418040, + .desc = "ARM erratum 1418040", + .disable_compat_vdso = true, + }, +#endif };
typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
On Wed, Jul 15, 2020 at 01:56:11PM +0100, Marc Zyngier wrote:
This is a backport of the series that recently went into 5.8. Note that the first patch is more a complete rewriting than a backport, as the vdso implementation in 5.4 doesn't have much in common with mainline. This affects the 32bit arch code in a benign way.
It has seen very little testing, as I don't have the HW that triggers this issue. I have run it in VMs by faking the CPU MIDR, and nothing caught fire. Famous last words.
These are also needed in 5.7.y, right? If so, I need that series before I can take this one as we don't want people moving to a newer kernel and suffer regressions :(
thanks,
greg k-h
Hi Greg,
On 2020-07-16 12:58, Greg KH wrote:
On Wed, Jul 15, 2020 at 01:56:11PM +0100, Marc Zyngier wrote:
This is a backport of the series that recently went into 5.8. Note that the first patch is more a complete rewriting than a backport, as the vdso implementation in 5.4 doesn't have much in common with mainline. This affects the 32bit arch code in a benign way.
It has seen very little testing, as I don't have the HW that triggers this issue. I have run it in VMs by faking the CPU MIDR, and nothing caught fire. Famous last words.
These are also needed in 5.7.y, right? If so, I need that series before I can take this one as we don't want people moving to a newer kernel and suffer regressions :(
The original mainline changes:
4b661d6133c5 arm64: arch_timer: Disable the compat vdso for cores affected by ARM64_WORKAROUND_1418040 c1fbec4ac0d7 arm64: arch_timer: Allow an workaround descriptor to disable compat vdso 97884ca8c292 arm64: Introduce a way to disable the 32bit vdso
do apply cleanly to stable-5.7. Do you want me to resend them separately, or will you pick the patches directly from mainline?
Thanks,
M.
On Fri, Jul 17, 2020 at 09:02:06AM +0100, Marc Zyngier wrote:
Hi Greg,
On 2020-07-16 12:58, Greg KH wrote:
On Wed, Jul 15, 2020 at 01:56:11PM +0100, Marc Zyngier wrote:
This is a backport of the series that recently went into 5.8. Note that the first patch is more a complete rewriting than a backport, as the vdso implementation in 5.4 doesn't have much in common with mainline. This affects the 32bit arch code in a benign way.
It has seen very little testing, as I don't have the HW that triggers this issue. I have run it in VMs by faking the CPU MIDR, and nothing caught fire. Famous last words.
These are also needed in 5.7.y, right? If so, I need that series before I can take this one as we don't want people moving to a newer kernel and suffer regressions :(
The original mainline changes:
4b661d6133c5 arm64: arch_timer: Disable the compat vdso for cores affected by ARM64_WORKAROUND_1418040 c1fbec4ac0d7 arm64: arch_timer: Allow an workaround descriptor to disable compat vdso 97884ca8c292 arm64: Introduce a way to disable the 32bit vdso
do apply cleanly to stable-5.7. Do you want me to resend them separately, or will you pick the patches directly from mainline?
Hm, cherry-pick seems to work due to file renames, let me try this again...
On Fri, Jul 17, 2020 at 11:33:57AM +0200, Greg KH wrote:
On Fri, Jul 17, 2020 at 09:02:06AM +0100, Marc Zyngier wrote:
Hi Greg,
On 2020-07-16 12:58, Greg KH wrote:
On Wed, Jul 15, 2020 at 01:56:11PM +0100, Marc Zyngier wrote:
This is a backport of the series that recently went into 5.8. Note that the first patch is more a complete rewriting than a backport, as the vdso implementation in 5.4 doesn't have much in common with mainline. This affects the 32bit arch code in a benign way.
It has seen very little testing, as I don't have the HW that triggers this issue. I have run it in VMs by faking the CPU MIDR, and nothing caught fire. Famous last words.
These are also needed in 5.7.y, right? If so, I need that series before I can take this one as we don't want people moving to a newer kernel and suffer regressions :(
The original mainline changes:
4b661d6133c5 arm64: arch_timer: Disable the compat vdso for cores affected by ARM64_WORKAROUND_1418040 c1fbec4ac0d7 arm64: arch_timer: Allow an workaround descriptor to disable compat vdso 97884ca8c292 arm64: Introduce a way to disable the 32bit vdso
do apply cleanly to stable-5.7. Do you want me to resend them separately, or will you pick the patches directly from mainline?
Hm, cherry-pick seems to work due to file renames, let me try this again...
Ok, my fault, these are already all in 5.7.9, sorry for the noise. I'll go queue these up now.
greg "I need more coffee..." k-h
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Marc Zyngier