Hi stable maintainers,
I recently ran into an issue where trying to load a module with jump table entries crashes the system when function tracing is enabled. The crash happens because ftrace is modifying the code and then marking it as read-only too early. ftrace_make_call() calls module_enable_ro(mod, true) before module init is over because ftrace_module_enable() calls __ftrace_replace_code() which does FTRACE_UPDATE_MAKE_CALL. All this code is gone now upstream but is still present on v5.4 stable kernels. I picked this set of patches to v5.4 and it fixed it for me.
fbf6c73c5b26 ftrace: add ftrace_init_nop() a1326b17ac03 module/ftrace: handle patchable-function-entry bd8b21d3dd66 arm64: module: rework special section handling f1a54ae9af0d arm64: module/ftrace: intialize PLT at load time
after doing that I ran into another issue because I'm using clang. Would it be possible to pick two more patches to the stable tree to silence this module warning from sysfs complaining about /module/<modname>/sections/__patchable_function_entries being duplicated?
dd2776222abb kbuild: lto: merge module sections 6a3193cdd5e5 kbuild: lto: Merge module sections if and only if CONFIG_LTO_CLANG is enabled
All of these apply cleanly to v5.4.y stable branch.
Crash below.
Unable to handle kernel write to read-only memory at virtual address ffffffd1b81fc0b0 Mem abort info: ESR = 0x9600004f EC = 0x25: DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 Data abort info: ISV = 0, ISS = 0x0000004f CM = 0, WnR = 1 swapper pgtable: 4k pages, 39-bit VAs, pgdp=0000000081bc1000 [ffffffd1b81fc0b0] pgd=0000000271858003, pud=0000000271858003, pmd=0000000266c05003, pte=00600001f3f4df93 Internal error: Oops: 9600004f [#1] PREEMPT SMP Modules linked in: vsock(+) <a bunch more modules> CPU: 5 PID: 6254 Comm: modprobe Not tainted 5.4.177 #19 Hardware name: Google Lazor (rev3 - 8) with KB Backlight (DT) pstate: 60400009 (nZCv daif +PAN -UAO) pc : jump_label_swap+0x2c/0x68 lr : jump_label_swap+0x18/0x68 sp : ffffffc01946bb10 x29: ffffffc01946bb10 x28: 00000000000000b0 x27: 0000000000000010 x26: 00000000000000b0 x25: 00000000000001a0 x24: ffffffd1b81fc180 x23: ffffffd2036540fc x22: ffffffd1b81fc000 x21: 0000000000000010 x20: ffffffd1b81fc0b0 x19: ffffffd1b81fc180 x18: 0000000000000000 x17: ffffffd204006e08 x16: 0000000000006000 x15: ffffffd1b8209000 x14: 0066ba79a6ffffff x13: 0000000000000004 x12: 000000004c55e4d8 x11: 00000000ffffd9f0 x10: 00000000ffffd754 x9 : ffffffffffffff30 x8 : 00000000ffffe2ec x7 : fefefefefefefefe x6 : 00000000000341d5 x5 : ffffffd203654164 x4 : ffffffd2036540fc x3 : 0000000000000000 x2 : ffffffc01946bb30 x1 : ffffffd203654110 x0 : 00000000fffffff0 Call trace: jump_label_swap+0x2c/0x68 do_swap+0x98/0xa0 sort_r+0x178/0x1a0 sort+0x14/0x1c jump_label_module_notify+0x7c/0x2c0 notifier_call_chain+0x58/0x90 __blocking_notifier_call_chain+0x58/0x84 blocking_notifier_call_chain+0x38/0x48 prepare_coming_module+0x30/0x3c load_module+0xda4/0xf8c __arm64_sys_finit_module+0xa4/0xdc el0_svc_common+0xb4/0x17c el0_svc_compat_handler+0x2c/0x58 el0_svc_compat+0x8/0x2c Code: cb130289 29402e8a f940068c 4b090108 (b9000288)
On Thu, Feb 17, 2022 at 05:27:33PM -0800, Stephen Boyd wrote:
Hi stable maintainers,
I recently ran into an issue where trying to load a module with jump table entries crashes the system when function tracing is enabled. The crash happens because ftrace is modifying the code and then marking it as read-only too early. ftrace_make_call() calls module_enable_ro(mod, true) before module init is over because ftrace_module_enable() calls __ftrace_replace_code() which does FTRACE_UPDATE_MAKE_CALL. All this code is gone now upstream but is still present on v5.4 stable kernels. I picked this set of patches to v5.4 and it fixed it for me.
fbf6c73c5b26 ftrace: add ftrace_init_nop() a1326b17ac03 module/ftrace: handle patchable-function-entry bd8b21d3dd66 arm64: module: rework special section handling f1a54ae9af0d arm64: module/ftrace: intialize PLT at load time
These all apply just fine, thanks.
after doing that I ran into another issue because I'm using clang. Would it be possible to pick two more patches to the stable tree to silence this module warning from sysfs complaining about /module/<modname>/sections/__patchable_function_entries being duplicated?
dd2776222abb kbuild: lto: merge module sections 6a3193cdd5e5 kbuild: lto: Merge module sections if and only if CONFIG_LTO_CLANG is enabled
These two do not apply to the 5.4.y branch, as the file they touch is not present in 5.4.y. They do apply to 5.10.y, so I've queued them up there, but I think you need to provide a working backport please.
thanks,
greg k-h
Quoting Greg KH (2022-02-18 01:01:35)
On Thu, Feb 17, 2022 at 05:27:33PM -0800, Stephen Boyd wrote:
Hi stable maintainers,
I recently ran into an issue where trying to load a module with jump table entries crashes the system when function tracing is enabled. The crash happens because ftrace is modifying the code and then marking it as read-only too early. ftrace_make_call() calls module_enable_ro(mod, true) before module init is over because ftrace_module_enable() calls __ftrace_replace_code() which does FTRACE_UPDATE_MAKE_CALL. All this code is gone now upstream but is still present on v5.4 stable kernels. I picked this set of patches to v5.4 and it fixed it for me.
fbf6c73c5b26 ftrace: add ftrace_init_nop() a1326b17ac03 module/ftrace: handle patchable-function-entry bd8b21d3dd66 arm64: module: rework special section handling f1a54ae9af0d arm64: module/ftrace: intialize PLT at load time
These all apply just fine, thanks.
Cool, thanks!
after doing that I ran into another issue because I'm using clang. Would it be possible to pick two more patches to the stable tree to silence this module warning from sysfs complaining about /module/<modname>/sections/__patchable_function_entries being duplicated?
dd2776222abb kbuild: lto: merge module sections 6a3193cdd5e5 kbuild: lto: Merge module sections if and only if CONFIG_LTO_CLANG is enabled
These two do not apply to the 5.4.y branch, as the file they touch is not present in 5.4.y. They do apply to 5.10.y, so I've queued them up there, but I think you need to provide a working backport please.
Ok. Good news! They're not necessary. I looked further and found that I had originally picked the entire arm64 series, including commit 3b23e4991fb6 ("arm64: implement ftrace with regs"). Then I ran into the problem where __patchable_function_entries wasn't being combined by the linker script. I picked those extra lto patches and it fixed it. Then I peeled away the new ftrace with regs feature because they were big and scary and not necessary. Now I've narrowed it down to only needing the one line from the first lto patch:
__patchable_function_entries : { *(__patchable_function_entries) }
in combination with the ftrace with regs support.
Long story short, they aren't needed unless commit 3b23e4991fb6 ("arm64: implement ftrace with regs") is picked, which isn't needed because we don't need that feature, just the part where we initialize PLT at load time and stop patching the code up too early, mistakenly enabling RO protection before the module is done being formed.
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Stephen Boyd