Adds support for detecting and reporting the speed of unaligned vector accesses on RISC-V CPUs. Adds vec_misaligned_speed key to the hwprobe adds Zicclsm to cpufeature and fixes the check for scalar unaligned emulated all CPUs. The vec_misaligned_speed key keeps the same format as the scalar unaligned access speed key.
This set does not emulate unaligned vector accesses on CPUs that do not support them. Only reports if userspace can run them and speed of unaligned vector accesses if supported.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Signed-off-by: Jesse Taube jesse@rivosinc.com --- Changes in V6: Added ("RISC-V: Scalar unaligned access emulated on hotplug CPUs")
Changes in V8: Dropped Zicclsm s/RISCV_HWPROBE_VECTOR_MISALIGNED/RISCV_HWPROBE_MISALIGNED_VECTOR/g to match RISCV_HWPROBE_MISALIGNED_SCALAR_* Rebased onto palmer/fixes (32d5f7add080a936e28ab4142bfeea6b06999789)
Changes in V9: Missed a RISCV_HWPROBE_VECTOR_MISALIGNED...
Changes in V10: - I sent on behalf of Jesse - Remove v0 from clobber args in inline asm and leave comment
--- Jesse Taube (6): RISC-V: Check scalar unaligned access on all CPUs RISC-V: Scalar unaligned access emulated on hotplug CPUs RISC-V: Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED RISC-V: Detect unaligned vector accesses supported RISC-V: Report vector unaligned access speed hwprobe RISC-V: hwprobe: Document unaligned vector perf key
Documentation/arch/riscv/hwprobe.rst | 16 +++ arch/riscv/Kconfig | 58 ++++++++++- arch/riscv/include/asm/cpufeature.h | 10 +- arch/riscv/include/asm/entry-common.h | 11 -- arch/riscv/include/asm/hwprobe.h | 2 +- arch/riscv/include/asm/vector.h | 2 + arch/riscv/include/uapi/asm/hwprobe.h | 5 + arch/riscv/kernel/Makefile | 3 +- arch/riscv/kernel/copy-unaligned.h | 5 + arch/riscv/kernel/fpu.S | 4 +- arch/riscv/kernel/sys_hwprobe.c | 41 ++++++++ arch/riscv/kernel/traps_misaligned.c | 139 +++++++++++++++++++++++-- arch/riscv/kernel/unaligned_access_speed.c | 156 +++++++++++++++++++++++++++-- arch/riscv/kernel/vec-copy-unaligned.S | 58 +++++++++++ arch/riscv/kernel/vector.c | 2 +- 15 files changed, 474 insertions(+), 38 deletions(-) --- base-commit: 98f7e32f20d28ec452afb208f9cffc08448a2652 change-id: 20240920-jesse_unaligned_vector-7083fd28659c
From: Jesse Taube jesse@rivosinc.com
Originally, the check_unaligned_access_emulated_all_cpus function only checked the boot hart. This fixes the function to check all harts.
Fixes: 71c54b3d169d ("riscv: report misaligned accesses emulation to hwprobe") Signed-off-by: Jesse Taube jesse@rivosinc.com Reviewed-by: Charlie Jenkins charlie@rivosinc.com Reviewed-by: Evan Green evan@rivosinc.com Cc: stable@vger.kernel.org --- arch/riscv/include/asm/cpufeature.h | 2 ++ arch/riscv/kernel/traps_misaligned.c | 14 +++++++------- 2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 45f9c1171a48..dfa5cdddd367 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -8,6 +8,7 @@
#include <linux/bitmap.h> #include <linux/jump_label.h> +#include <linux/workqueue.h> #include <asm/hwcap.h> #include <asm/alternative-macros.h> #include <asm/errno.h> @@ -60,6 +61,7 @@ void riscv_user_isa_enable(void);
#if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +void check_unaligned_access_emulated(struct work_struct *work __always_unused); void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); DECLARE_PER_CPU(long, misaligned_access_speed); diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c index d4fd8af7aaf5..d076dde5ad20 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -526,11 +526,11 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; }
-static bool check_unaligned_access_emulated(int cpu) +void check_unaligned_access_emulated(struct work_struct *work __always_unused) { + int cpu = smp_processor_id(); long *mas_ptr = per_cpu_ptr(&misaligned_access_speed, cpu); unsigned long tmp_var, tmp_val; - bool misaligned_emu_detected;
*mas_ptr = RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN;
@@ -538,19 +538,16 @@ static bool check_unaligned_access_emulated(int cpu) " "REG_L" %[tmp], 1(%[ptr])\n" : [tmp] "=r" (tmp_val) : [ptr] "r" (&tmp_var) : "memory");
- misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED); /* * If unaligned_ctl is already set, this means that we detected that all * CPUS uses emulated misaligned access at boot time. If that changed * when hotplugging the new cpu, this is something we don't handle. */ - if (unlikely(unaligned_ctl && !misaligned_emu_detected)) { + if (unlikely(unaligned_ctl && (*mas_ptr != RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED))) { pr_crit("CPU misaligned accesses non homogeneous (expected all emulated)\n"); while (true) cpu_relax(); } - - return misaligned_emu_detected; }
bool check_unaligned_access_emulated_all_cpus(void) @@ -562,8 +559,11 @@ bool check_unaligned_access_emulated_all_cpus(void) * accesses emulated since tasks requesting such control can run on any * CPU. */ + schedule_on_each_cpu(check_unaligned_access_emulated); + for_each_online_cpu(cpu) - if (!check_unaligned_access_emulated(cpu)) + if (per_cpu(misaligned_access_speed, cpu) + != RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED) return false;
unaligned_ctl = true;
From: Jesse Taube jesse@rivosinc.com
The check_unaligned_access_emulated() function should have been called during CPU hotplug to ensure that if all CPUs had emulated unaligned accesses, the new CPU also does.
This patch adds the call to check_unaligned_access_emulated() in the hotplug path.
Fixes: 55e0bf49a0d0 ("RISC-V: Probe misaligned access speed in parallel") Signed-off-by: Jesse Taube jesse@rivosinc.com Reviewed-by: Evan Green evan@rivosinc.com Cc: stable@vger.kernel.org --- arch/riscv/kernel/unaligned_access_speed.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index 160628a2116d..f3508cc54f91 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -191,6 +191,7 @@ static int riscv_online_cpu(unsigned int cpu) if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN) goto exit;
+ check_unaligned_access_emulated(NULL); buf = alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); if (!buf) { pr_warn("Allocation failure, not measuring misaligned performance\n");
Hello:
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt palmer@rivosinc.com:
On Thu, 17 Oct 2024 12:00:17 -0700 you wrote:
Adds support for detecting and reporting the speed of unaligned vector accesses on RISC-V CPUs. Adds vec_misaligned_speed key to the hwprobe adds Zicclsm to cpufeature and fixes the check for scalar unaligned emulated all CPUs. The vec_misaligned_speed key keeps the same format as the scalar unaligned access speed key.
This set does not emulate unaligned vector accesses on CPUs that do not support them. Only reports if userspace can run them and speed of unaligned vector accesses if supported.
[...]
Here is the summary with links: - [v10,1/6] RISC-V: Check scalar unaligned access on all CPUs https://git.kernel.org/riscv/c/8d20a739f17a - [v10,2/6] RISC-V: Scalar unaligned access emulated on hotplug CPUs https://git.kernel.org/riscv/c/9c528b5f7927 - [v10,3/6] RISC-V: Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED https://git.kernel.org/riscv/c/c05a62c92516 - [v10,4/6] RISC-V: Detect unaligned vector accesses supported https://git.kernel.org/riscv/c/d1703dc7bc8e - [v10,5/6] RISC-V: Report vector unaligned access speed hwprobe https://git.kernel.org/riscv/c/e7c9d66e313b - [v10,6/6] RISC-V: hwprobe: Document unaligned vector perf key https://git.kernel.org/riscv/c/40e09ebd791f
You are awesome, thank you!
linux-stable-mirror@lists.linaro.org
-
Charlie Jenkins -
patchwork-bot+linux-riscv＠kernel.org