[bug report] NULL pointer at blk_mq_put_rq_ref+0x20/0xb4 observed with blktests on 5.13.15
Hello
I found this issue with blktests on[1], did we miss some patch on stable? [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git queue/5.13
[ 68.989907] run blktests block/006 at 2021-09-09 04:34:35 [ 69.085724] null_blk: module loaded [ 74.271624] Unable to handle kernel NULL pointer dereference at virtual address 00000000000002b8 [ 74.280414] Mem abort info: [ 74.283195] ESR = 0x96000004 [ 74.286245] EC = 0x25: DABT (current EL), IL = 32 bits [ 74.291545] SET = 0, FnV = 0 [ 74.294587] EA = 0, S1PTW = 0 [ 74.297720] Data abort info: [ 74.300588] ISV = 0, ISS = 0x00000004 [ 74.304411] CM = 0, WnR = 0 [ 74.307368] user pgtable: 4k pages, 48-bit VAs, pgdp=000008004366e000 [ 74.313796] [00000000000002b8] pgd=0000000000000000, p4d=0000000000000000 [ 74.320577] Internal error: Oops: 96000004 [#1] SMP [ 74.325443] Modules linked in: null_blk mlx5_ib ib_uverbs ib_core rfkill sunrpc vfat fat joydev acpi_ipmi ipmi_ssif cdc_ether usbnet mii mlx5_core psample ipmi_devintf mlxfw tls ipmi_msghandler arm_cmn cppc_cpufreq arm_dsu_pmu acpi_tad fuse zram ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper crct10dif_ce syscopyarea ghash_ce sysfillrect uas sysimgblt sbsa_gwdt fb_sys_fops cec drm_ttm_helper ttm nvme usb_storage nvme_core drm xgene_hwmon aes_neon_bs [ 74.366458] CPU: 31 PID: 2511 Comm: fio Not tainted 5.13.15+ #1 [ 74.372367] Hardware name: WIWYNN Mt.Jade Server System B81.030Z1.0007/Mt.Jade Motherboard, BIOS 1.6.20210526 (SCP: 1.06.20210526) 2021/05/26 [ 74.385045] pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--) [ 74.391040] pc : blk_mq_put_rq_ref+0x20/0xb4 [ 74.395301] lr : bt_iter+0x64/0xd0 [ 74.398690] sp : ffff800049153980 [ 74.401992] x29: ffff800049153980 x28: ffff07ff8f694520 x27: 0000000000400cc0 [ 74.409116] x26: ffff07ffbff407f0 x25: 00000000000000c0 x24: 0000000000000010 [ 74.416240] x23: 0000000000000000 x22: 0000000000000001 x21: ffff07ffc50d0000 [ 74.423363] x20: ffff800049153a50 x19: ffff07ffc5188c40 x18: 0000000000000000 [ 74.430486] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 74.437609] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 74.444732] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffaae6737ae164 [ 74.451855] x8 : ffff800049153c00 x7 : ffffffffffffff80 x6 : 0000000000000007 [ 74.458978] x5 : 0000000000000000 x4 : 0000000000000001 x3 : ffff07ffc5188d28 [ 74.466100] x2 : 0000000000000000 x1 : ffff07ffc5188c40 x0 : ffff07ffc5188c40 [ 74.473224] Call trace: [ 74.475657] blk_mq_put_rq_ref+0x20/0xb4 [ 74.479567] bt_iter+0x64/0xd0 [ 74.482609] blk_mq_queue_tag_busy_iter+0x1a0/0x300 [ 74.487475] blk_mq_in_flight+0x30/0x44 [ 74.491298] part_stat_show+0x60/0x160 [ 74.495036] dev_attr_show+0x2c/0x6c [ 74.498599] sysfs_kf_seq_show+0x94/0x140 [ 74.502598] kernfs_seq_show+0x38/0x44 [ 74.506336] seq_read_iter+0x1dc/0x4f0 [ 74.510075] kernfs_fop_read_iter+0x44/0x50 [ 74.514245] new_sync_read+0xdc/0x154 [ 74.517896] vfs_read+0x158/0x1e4 [ 74.521199] ksys_read+0x64/0xf0 [ 74.524414] __arm64_sys_read+0x28/0x34 [ 74.528237] invoke_syscall+0x50/0x120 [ 74.531976] el0_svc_common.constprop.0+0x4c/0x100 [ 74.536755] do_el0_svc+0x34/0xa0 [ 74.540057] el0_svc+0x2c/0x54 [ 74.543100] el0_sync_handler+0xa4/0x130 [ 74.547011] el0_sync+0x19c/0x1c0 [ 74.550320] Code: a9bf7bfd aa0003e1 910003fd f9400802 (f9415c42) [ 74.556503] ---[ end trace 76adda8a4ccf9d09 ]--- [ 74.561107] Kernel panic - not syncing: Oops: Fatal exception [ 74.566897] SMP: stopping secondary CPUs [ 74.570914] Kernel Offset: 0x2ae6630e0000 from 0xffff800010000000 [ 74.576994] PHYS_OFFSET: 0x80000000 [ 74.580469] CPU features: 0x000042c1,a3302e42 [ 74.584813] Memory Limit: none [ 74.587902] ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---
On Thu, Sep 9, 2021 at 4:47 PM Yi Zhang yi.zhang@redhat.com wrote:
Hello
I found this issue with blktests on[1], did we miss some patch on stable? [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git queue/5.13
[ 68.989907] run blktests block/006 at 2021-09-09 04:34:35 [ 69.085724] null_blk: module loaded [ 74.271624] Unable to handle kernel NULL pointer dereference at virtual address 00000000000002b8 [ 74.280414] Mem abort info: [ 74.283195] ESR = 0x96000004 [ 74.286245] EC = 0x25: DABT (current EL), IL = 32 bits [ 74.291545] SET = 0, FnV = 0 [ 74.294587] EA = 0, S1PTW = 0 [ 74.297720] Data abort info: [ 74.300588] ISV = 0, ISS = 0x00000004 [ 74.304411] CM = 0, WnR = 0 [ 74.307368] user pgtable: 4k pages, 48-bit VAs, pgdp=000008004366e000 [ 74.313796] [00000000000002b8] pgd=0000000000000000, p4d=0000000000000000 [ 74.320577] Internal error: Oops: 96000004 [#1] SMP [ 74.325443] Modules linked in: null_blk mlx5_ib ib_uverbs ib_core rfkill sunrpc vfat fat joydev acpi_ipmi ipmi_ssif cdc_ether usbnet mii mlx5_core psample ipmi_devintf mlxfw tls ipmi_msghandler arm_cmn cppc_cpufreq arm_dsu_pmu acpi_tad fuse zram ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper crct10dif_ce syscopyarea ghash_ce sysfillrect uas sysimgblt sbsa_gwdt fb_sys_fops cec drm_ttm_helper ttm nvme usb_storage nvme_core drm xgene_hwmon aes_neon_bs [ 74.366458] CPU: 31 PID: 2511 Comm: fio Not tainted 5.13.15+ #1
Looks the fixes haven't land on linux-5.13.y:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
Thanks,
On Thu, Sep 09, 2021 at 05:14:18PM +0800, Ming Lei wrote:
On Thu, Sep 9, 2021 at 4:47 PM Yi Zhang yi.zhang@redhat.com wrote:
Hello
I found this issue with blktests on[1], did we miss some patch on stable? [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git queue/5.13
[ 68.989907] run blktests block/006 at 2021-09-09 04:34:35 [ 69.085724] null_blk: module loaded [ 74.271624] Unable to handle kernel NULL pointer dereference at virtual address 00000000000002b8 [ 74.280414] Mem abort info: [ 74.283195] ESR = 0x96000004 [ 74.286245] EC = 0x25: DABT (current EL), IL = 32 bits [ 74.291545] SET = 0, FnV = 0 [ 74.294587] EA = 0, S1PTW = 0 [ 74.297720] Data abort info: [ 74.300588] ISV = 0, ISS = 0x00000004 [ 74.304411] CM = 0, WnR = 0 [ 74.307368] user pgtable: 4k pages, 48-bit VAs, pgdp=000008004366e000 [ 74.313796] [00000000000002b8] pgd=0000000000000000, p4d=0000000000000000 [ 74.320577] Internal error: Oops: 96000004 [#1] SMP [ 74.325443] Modules linked in: null_blk mlx5_ib ib_uverbs ib_core rfkill sunrpc vfat fat joydev acpi_ipmi ipmi_ssif cdc_ether usbnet mii mlx5_core psample ipmi_devintf mlxfw tls ipmi_msghandler arm_cmn cppc_cpufreq arm_dsu_pmu acpi_tad fuse zram ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper crct10dif_ce syscopyarea ghash_ce sysfillrect uas sysimgblt sbsa_gwdt fb_sys_fops cec drm_ttm_helper ttm nvme usb_storage nvme_core drm xgene_hwmon aes_neon_bs [ 74.366458] CPU: 31 PID: 2511 Comm: fio Not tainted 5.13.15+ #1
Looks the fixes haven't land on linux-5.13.y:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
Now queued up. Someone could have told us they were needed :)
thanks,
greg k-h
On Thu, Sep 09, 2021 at 12:07:32PM +0200, Greg KH wrote:
On Thu, Sep 09, 2021 at 05:14:18PM +0800, Ming Lei wrote:
On Thu, Sep 9, 2021 at 4:47 PM Yi Zhang yi.zhang@redhat.com wrote:
Hello
I found this issue with blktests on[1], did we miss some patch on stable? [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git queue/5.13
[ 68.989907] run blktests block/006 at 2021-09-09 04:34:35 [ 69.085724] null_blk: module loaded [ 74.271624] Unable to handle kernel NULL pointer dereference at virtual address 00000000000002b8 [ 74.280414] Mem abort info: [ 74.283195] ESR = 0x96000004 [ 74.286245] EC = 0x25: DABT (current EL), IL = 32 bits [ 74.291545] SET = 0, FnV = 0 [ 74.294587] EA = 0, S1PTW = 0 [ 74.297720] Data abort info: [ 74.300588] ISV = 0, ISS = 0x00000004 [ 74.304411] CM = 0, WnR = 0 [ 74.307368] user pgtable: 4k pages, 48-bit VAs, pgdp=000008004366e000 [ 74.313796] [00000000000002b8] pgd=0000000000000000, p4d=0000000000000000 [ 74.320577] Internal error: Oops: 96000004 [#1] SMP [ 74.325443] Modules linked in: null_blk mlx5_ib ib_uverbs ib_core rfkill sunrpc vfat fat joydev acpi_ipmi ipmi_ssif cdc_ether usbnet mii mlx5_core psample ipmi_devintf mlxfw tls ipmi_msghandler arm_cmn cppc_cpufreq arm_dsu_pmu acpi_tad fuse zram ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper crct10dif_ce syscopyarea ghash_ce sysfillrect uas sysimgblt sbsa_gwdt fb_sys_fops cec drm_ttm_helper ttm nvme usb_storage nvme_core drm xgene_hwmon aes_neon_bs [ 74.366458] CPU: 31 PID: 2511 Comm: fio Not tainted 5.13.15+ #1
Looks the fixes haven't land on linux-5.13.y:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
Now queued up. Someone could have told us they were needed :)
Thanks for queuing it up, sorry for not Cc stable.
BTW, the following two patches are missed too in linux-5.13-y:
364b61818f65 blk-mq: clearing flush request reference in tags->rqs[] bd63141d585b blk-mq: clear stale request in tags->rq[] before freeing one request pool
Both can fix request UAF issue.
Thanks, Ming
On Fri, Sep 10, 2021 at 09:43:28AM +0800, Ming Lei wrote:
On Thu, Sep 09, 2021 at 12:07:32PM +0200, Greg KH wrote:
On Thu, Sep 09, 2021 at 05:14:18PM +0800, Ming Lei wrote:
On Thu, Sep 9, 2021 at 4:47 PM Yi Zhang yi.zhang@redhat.com wrote:
Hello
I found this issue with blktests on[1], did we miss some patch on stable? [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git queue/5.13
[ 68.989907] run blktests block/006 at 2021-09-09 04:34:35 [ 69.085724] null_blk: module loaded [ 74.271624] Unable to handle kernel NULL pointer dereference at virtual address 00000000000002b8 [ 74.280414] Mem abort info: [ 74.283195] ESR = 0x96000004 [ 74.286245] EC = 0x25: DABT (current EL), IL = 32 bits [ 74.291545] SET = 0, FnV = 0 [ 74.294587] EA = 0, S1PTW = 0 [ 74.297720] Data abort info: [ 74.300588] ISV = 0, ISS = 0x00000004 [ 74.304411] CM = 0, WnR = 0 [ 74.307368] user pgtable: 4k pages, 48-bit VAs, pgdp=000008004366e000 [ 74.313796] [00000000000002b8] pgd=0000000000000000, p4d=0000000000000000 [ 74.320577] Internal error: Oops: 96000004 [#1] SMP [ 74.325443] Modules linked in: null_blk mlx5_ib ib_uverbs ib_core rfkill sunrpc vfat fat joydev acpi_ipmi ipmi_ssif cdc_ether usbnet mii mlx5_core psample ipmi_devintf mlxfw tls ipmi_msghandler arm_cmn cppc_cpufreq arm_dsu_pmu acpi_tad fuse zram ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper crct10dif_ce syscopyarea ghash_ce sysfillrect uas sysimgblt sbsa_gwdt fb_sys_fops cec drm_ttm_helper ttm nvme usb_storage nvme_core drm xgene_hwmon aes_neon_bs [ 74.366458] CPU: 31 PID: 2511 Comm: fio Not tainted 5.13.15+ #1
Looks the fixes haven't land on linux-5.13.y:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
Now queued up. Someone could have told us they were needed :)
Thanks for queuing it up, sorry for not Cc stable.
BTW, the following two patches are missed too in linux-5.13-y:
364b61818f65 blk-mq: clearing flush request reference in tags->rqs[]
This one applies, but,
bd63141d585b blk-mq: clear stale request in tags->rq[] before freeing one request pool
This one does not.
Please provide working backports for both of these if you want to see them merged into the stable trees. And what about 5.10 for them as well?
thanks,
greg k-h
Hi,
Greg KH gregkh@linuxfoundation.org 于2021年9月10日周五 上午8:51写道：
On Fri, Sep 10, 2021 at 09:43:28AM +0800, Ming Lei wrote:
On Thu, Sep 09, 2021 at 12:07:32PM +0200, Greg KH wrote:
On Thu, Sep 09, 2021 at 05:14:18PM +0800, Ming Lei wrote:
On Thu, Sep 9, 2021 at 4:47 PM Yi Zhang yi.zhang@redhat.com wrote:
Hello
I found this issue with blktests on[1], did we miss some patch on stable? [1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git queue/5.13
[ 68.989907] run blktests block/006 at 2021-09-09 04:34:35 [ 69.085724] null_blk: module loaded [ 74.271624] Unable to handle kernel NULL pointer dereference at virtual address 00000000000002b8 [ 74.280414] Mem abort info: [ 74.283195] ESR = 0x96000004 [ 74.286245] EC = 0x25: DABT (current EL), IL = 32 bits [ 74.291545] SET = 0, FnV = 0 [ 74.294587] EA = 0, S1PTW = 0 [ 74.297720] Data abort info: [ 74.300588] ISV = 0, ISS = 0x00000004 [ 74.304411] CM = 0, WnR = 0 [ 74.307368] user pgtable: 4k pages, 48-bit VAs, pgdp=000008004366e000 [ 74.313796] [00000000000002b8] pgd=0000000000000000, p4d=0000000000000000 [ 74.320577] Internal error: Oops: 96000004 [#1] SMP [ 74.325443] Modules linked in: null_blk mlx5_ib ib_uverbs ib_core rfkill sunrpc vfat fat joydev acpi_ipmi ipmi_ssif cdc_ether usbnet mii mlx5_core psample ipmi_devintf mlxfw tls ipmi_msghandler arm_cmn cppc_cpufreq arm_dsu_pmu acpi_tad fuse zram ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper crct10dif_ce syscopyarea ghash_ce sysfillrect uas sysimgblt sbsa_gwdt fb_sys_fops cec drm_ttm_helper ttm nvme usb_storage nvme_core drm xgene_hwmon aes_neon_bs [ 74.366458] CPU: 31 PID: 2511 Comm: fio Not tainted 5.13.15+ #1
Looks the fixes haven't land on linux-5.13.y:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
Now queued up. Someone could have told us they were needed :)
Thanks for queuing it up, sorry for not Cc stable.
BTW, the following two patches are missed too in linux-5.13-y:
364b61818f65 blk-mq: clearing flush request reference in tags->rqs[]
This one applies, but,
bd63141d585b blk-mq: clear stale request in tags->rq[] before freeing one request pool
This one does not.
this is already included since 5.10.50 747b654e4069 ("blk-mq: clear stale request in tags->rq[] before freeing one request pool")
Please provide working backports for both of these if you want to see them merged into the stable trees. And what about 5.10 for them as well?
thanks,
greg k-h
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Greg KH -
Jack Wang -
Ming Lei -
Yi Zhang