stable-rc 4.14.154-rc1/0d12dcf336c6: regressions detected in project stable v.4.14.y on OE - sanity
Hi Greg,
Regressions detected in project stable v.4.14. arm64, arm, x86_64 boot failed.
Summary ------------------------------------------------------------------------ git branch: linux-4.14.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: a67492b55c53045e9c0b9969f04410723448c1ee git describe: v4.14.153-104-ga67492b55c53 make_kernelversion: 4.14.154-rc1 kernel-config: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab...
arm64 kernel crash log: --------------------------- [ 0.000000] Linux version 4.14.154-rc1 (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP PREEMPT Mon Nov 11 17:15:34 UTC 2019 [ 0.000000] Boot CPU: AArch64 Processor [410fd033] [ 0.000000] Machine model: ARM Juno development board (r2) [ 0.000000] earlycon: pl11 at MMIO 0x000000007ff80000 (options '') [ 0.000000] bootconsole [pl11] enabled [ 0.000000] efi: Getting EFI parameters from FDT: [ 0.000000] efi: UEFI not found. [ 0.000000] cma: Reserved 16 MiB at 0x00000000fe000000 [ 0.000000] NUMA: No NUMA configuration found [ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000009ffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x9fffc6f80-0x9fffc8bff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000080000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x00000009ffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080000000-0x00000000feffffff] [ 0.000000] node 0: [mem 0x0000000880000000-0x00000009ffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000009ffffffff] [ 0.000000] On node 0 totalpages: 2093056 [ 0.000000] DMA zone: 8192 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 520192 pages, LIFO batch:31 [ 0.000000] Normal zone: 24576 pages used for memmap [ 0.000000] Normal zone: 1572864 pages, LIFO batch:31 [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.0 [ 0.000000] percpu: Embedded 25 pages/cpu s62280 r8192 d31928 u102400 [ 0.000000] pcpu-alloc: s62280 r8192 d31928 u102400 alloc=25*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: enabling workaround for ARM erratum 845719 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 2060288 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: console=ttyAMA0,115200n8 root=/dev/nfs rw nfsroot=10.66.16.123:/var/lib/lava/dispatcher/tmp/1004132/extract-nfsrootfs-jn9hl2x_,tcp,hard,intr,vers=3 rootwait earlycon=pl011,0x7ff80000 debug systemd.log_target=null user_debug=31 androidboot.hardware=juno loglevel=9 sky2.mac_address=0x00,0x02,0xF7,0x00,0x68,0x15 ip=dhcp [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] software IO TLB: mapped [mem 0xf9fff000-0xfdfff000] (64MB) [ 0.000000] Memory: 8123672K/8372224K available (12988K kernel code, 1878K rwdata, 5196K rodata, 1728K init, 12356K bss, 232168K reserved, 16384K cma-reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB) [ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB) [ 0.000000] .text : 0xffff000008080000 - 0xffff000008d30000 ( 12992 KB) [ 0.000000] .rodata : 0xffff000008d30000 - 0xffff000009250000 ( 5248 KB) [ 0.000000] .init : 0xffff000009250000 - 0xffff000009400000 ( 1728 KB) [ 0.000000] .data : 0xffff000009400000 - 0xffff0000095d5a00 ( 1879 KB) [ 0.000000] .bss : 0xffff0000095d5a00 - 0xffff00000a1e6dd0 ( 12357 KB) [ 0.000000] fixed : 0xffff7dfffe7f9000 - 0xffff7dfffec00000 ( 4124 KB) [ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB) [ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum) [ 0.000000] 0xffff7e0000000000 - 0xffff7e0026000000 ( 608 MB actual) [ 0.000000] memory : 0xffff800000000000 - 0xffff800980000000 ( 38912 MB) [ 0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00000510 [ 0.000000] Mem abort info: [ 0.000000] Exception class = DABT (current EL), IL = 32 bits [ 0.000000] SET = 0, FnV = 0 [ 0.000000] EA = 0, S1PTW = 0 [ 0.000000] Data abort info: [ 0.000000] ISV = 0, ISS = 0x00000004 [ 0.000000] CM = 0, WnR = 0 [ 0.000000] [0000000000000510] user address but active_mm is swapper [ 0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 0.000000] Modules linked in: [ 0.000000] Process swapper (pid: 0, stack limit = 0xffff000009400000) [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.154-rc1 #1 [ 0.000000] Hardware name: ARM Juno development board (r2) (DT) [ 0.000000] task: ffff000009419780 task.stack: ffff000009400000 [ 0.000000] PC is at __rmqueue+0x3c4/0x908 [ 0.000000] LR is at __rmqueue+0x384/0x908 [ 0.000000] pc : [<ffff0000082594ec>] lr : [<ffff0000082594ac>] pstate: 600000c5 [ 0.000000] sp : ffff000009403b10 [ 0.000000] x29: ffff000009403b10 x28: 0000000000000000 [ 0.000000] x27: 000000000000000a x26: ffff80097ffc7700 [ 0.000000] x25: 00007ff6800387c0 x24: 0000000000000000 [ 0.000000] x23: ffff00000940f000 x22: ffff80097ffc7800 [ 0.000000] x21: ffffffffffffff00 x20: ffff80097ffc7840 [ 0.000000] x19: ffff0000093ef000 x18: 0000000000000001 [ 0.000000] x17: 000000000000001c x16: 000000000000001b [ 0.000000] x15: 000000000000001a x14: ffff0000096833f8 [ 0.000000] x13: ffff0000095e3198 x12: 000000000c67c24e [ 0.000000] x11: ffff000009419fb0 x10: ffff00000a174000 [ 0.000000] x9 : ffff00000988b728 x8 : 0000000000000000 [ 0.000000] x7 : ffff0000092cf3a4 x6 : 0000000000000000 [ 0.000000] x5 : fffffffffffff1ff x4 : ffff7e0025de0020 [ 0.000000] x3 : ffff7e0025dd0020 x2 : 000000000000000a [ 0.000000] x1 : 000000000000001e x0 : 0000000000000410 [ 0.000000] Call trace: [ 0.000000] Exception stack(0xffff0000094039d0 to 0xffff000009403b10) [ 0.000000] 39c0: 0000000000000410 000000000000001e [ 0.000000] 39e0: 000000000000000a ffff7e0025dd0020 ffff7e0025de0020 fffffffffffff1ff [ 0.000000] 3a00: 0000000000000000 ffff0000092cf3a4 0000000000000000 ffff00000988b728 [ 0.000000] 3a20: ffff00000a174000 ffff000009419fb0 000000000c67c24e ffff0000095e3198 [ 0.000000] 3a40: ffff0000096833f8 000000000000001a 000000000000001b 000000000000001c [ 0.000000] 3a60: 0000000000000001 ffff0000093ef000 ffff80097ffc7840 ffffffffffffff00 [ 0.000000] 3a80: ffff80097ffc7800 ffff00000940f000 0000000000000000 00007ff6800387c0 [ 0.000000] 3aa0: ffff80097ffc7700 000000000000000a 0000000000000000 ffff000009403b10 [ 0.000000] 3ac0: ffff0000082594ac ffff000009403b10 ffff0000082594ec 00000000600000c5 [ 0.000000] 3ae0: ffffffffffffff00 ffff80097ffc7800 ffffffffffffffff ffff0000082594ac [ 0.000000] 3b00: ffff000009403b10 ffff0000082594ec [ 0.000000] [<ffff0000082594ec>] __rmqueue+0x3c4/0x908 [ 0.000000] [<ffff000008259e98>] get_page_from_freelist+0x468/0xcb8 [ 0.000000] [<ffff00000825aa3c>] __alloc_pages_nodemask+0x12c/0x1248 [ 0.000000] [<ffff0000082cdba8>] new_slab+0xc0/0x568 [ 0.000000] [<ffff0000082d5328>] __kmem_cache_create+0x2e0/0x628 [ 0.000000] [<ffff000009271dec>] create_boot_cache+0xa8/0xdc [ 0.000000] [<ffff000009275b40>] kmem_cache_init+0x58/0x120 [ 0.000000] [<ffff000009250b3c>] start_kernel+0x200/0x3d8 [ 0.000000] Code: 8b000401 8b010800 d37df000 8b081000 (f9408000) [ 0.000000] random: get_random_bytes called from print_oops_end_marker+0x54/0x70 with crng_init=0 [ 0.000000] ---[ end trace c75a488acdeef932 ]--- [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task! [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task!
Full test log link, https://lkft.validation.linaro.org/scheduler/job/1004132#L482
- Naresh
On Mon, Nov 11, 2019 at 11:45:35PM +0530, Naresh Kamboju wrote:
Hi Greg,
Regressions detected in project stable v.4.14. arm64, arm, x86_64 boot failed.
Summary
git branch: linux-4.14.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: a67492b55c53045e9c0b9969f04410723448c1ee git describe: v4.14.153-104-ga67492b55c53 make_kernelversion: 4.14.154-rc1 kernel-config: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab...
arm64 kernel crash log:
[ 0.000000] Linux version 4.14.154-rc1 (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP PREEMPT Mon Nov 11 17:15:34 UTC 2019 [ 0.000000] Boot CPU: AArch64 Processor [410fd033] [ 0.000000] Machine model: ARM Juno development board (r2) [ 0.000000] earlycon: pl11 at MMIO 0x000000007ff80000 (options '') [ 0.000000] bootconsole [pl11] enabled [ 0.000000] efi: Getting EFI parameters from FDT: [ 0.000000] efi: UEFI not found. [ 0.000000] cma: Reserved 16 MiB at 0x00000000fe000000 [ 0.000000] NUMA: No NUMA configuration found [ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000009ffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x9fffc6f80-0x9fffc8bff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000080000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x00000009ffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080000000-0x00000000feffffff] [ 0.000000] node 0: [mem 0x0000000880000000-0x00000009ffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000009ffffffff] [ 0.000000] On node 0 totalpages: 2093056 [ 0.000000] DMA zone: 8192 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 520192 pages, LIFO batch:31 [ 0.000000] Normal zone: 24576 pages used for memmap [ 0.000000] Normal zone: 1572864 pages, LIFO batch:31 [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.0 [ 0.000000] percpu: Embedded 25 pages/cpu s62280 r8192 d31928 u102400 [ 0.000000] pcpu-alloc: s62280 r8192 d31928 u102400 alloc=25*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: enabling workaround for ARM erratum 845719 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 2060288 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: console=ttyAMA0,115200n8 root=/dev/nfs rw nfsroot=10.66.16.123:/var/lib/lava/dispatcher/tmp/1004132/extract-nfsrootfs-jn9hl2x_,tcp,hard,intr,vers=3 rootwait earlycon=pl011,0x7ff80000 debug systemd.log_target=null user_debug=31 androidboot.hardware=juno loglevel=9 sky2.mac_address=0x00,0x02,0xF7,0x00,0x68,0x15 ip=dhcp [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] software IO TLB: mapped [mem 0xf9fff000-0xfdfff000] (64MB) [ 0.000000] Memory: 8123672K/8372224K available (12988K kernel code, 1878K rwdata, 5196K rodata, 1728K init, 12356K bss, 232168K reserved, 16384K cma-reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB) [ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB) [ 0.000000] .text : 0xffff000008080000 - 0xffff000008d30000 ( 12992 KB) [ 0.000000] .rodata : 0xffff000008d30000 - 0xffff000009250000 ( 5248 KB) [ 0.000000] .init : 0xffff000009250000 - 0xffff000009400000 ( 1728 KB) [ 0.000000] .data : 0xffff000009400000 - 0xffff0000095d5a00 ( 1879 KB) [ 0.000000] .bss : 0xffff0000095d5a00 - 0xffff00000a1e6dd0 ( 12357 KB) [ 0.000000] fixed : 0xffff7dfffe7f9000 - 0xffff7dfffec00000 ( 4124 KB) [ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB) [ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum) [ 0.000000] 0xffff7e0000000000 - 0xffff7e0026000000 ( 608 MB actual) [ 0.000000] memory : 0xffff800000000000 - 0xffff800980000000 ( 38912 MB) [ 0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00000510 [ 0.000000] Mem abort info: [ 0.000000] Exception class = DABT (current EL), IL = 32 bits [ 0.000000] SET = 0, FnV = 0 [ 0.000000] EA = 0, S1PTW = 0 [ 0.000000] Data abort info: [ 0.000000] ISV = 0, ISS = 0x00000004 [ 0.000000] CM = 0, WnR = 0 [ 0.000000] [0000000000000510] user address but active_mm is swapper [ 0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 0.000000] Modules linked in: [ 0.000000] Process swapper (pid: 0, stack limit = 0xffff000009400000) [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.154-rc1 #1 [ 0.000000] Hardware name: ARM Juno development board (r2) (DT) [ 0.000000] task: ffff000009419780 task.stack: ffff000009400000 [ 0.000000] PC is at __rmqueue+0x3c4/0x908 [ 0.000000] LR is at __rmqueue+0x384/0x908 [ 0.000000] pc : [<ffff0000082594ec>] lr : [<ffff0000082594ac>] pstate: 600000c5 [ 0.000000] sp : ffff000009403b10 [ 0.000000] x29: ffff000009403b10 x28: 0000000000000000 [ 0.000000] x27: 000000000000000a x26: ffff80097ffc7700 [ 0.000000] x25: 00007ff6800387c0 x24: 0000000000000000 [ 0.000000] x23: ffff00000940f000 x22: ffff80097ffc7800 [ 0.000000] x21: ffffffffffffff00 x20: ffff80097ffc7840 [ 0.000000] x19: ffff0000093ef000 x18: 0000000000000001 [ 0.000000] x17: 000000000000001c x16: 000000000000001b [ 0.000000] x15: 000000000000001a x14: ffff0000096833f8 [ 0.000000] x13: ffff0000095e3198 x12: 000000000c67c24e [ 0.000000] x11: ffff000009419fb0 x10: ffff00000a174000 [ 0.000000] x9 : ffff00000988b728 x8 : 0000000000000000 [ 0.000000] x7 : ffff0000092cf3a4 x6 : 0000000000000000 [ 0.000000] x5 : fffffffffffff1ff x4 : ffff7e0025de0020 [ 0.000000] x3 : ffff7e0025dd0020 x2 : 000000000000000a [ 0.000000] x1 : 000000000000001e x0 : 0000000000000410 [ 0.000000] Call trace: [ 0.000000] Exception stack(0xffff0000094039d0 to 0xffff000009403b10) [ 0.000000] 39c0: 0000000000000410 000000000000001e [ 0.000000] 39e0: 000000000000000a ffff7e0025dd0020 ffff7e0025de0020 fffffffffffff1ff [ 0.000000] 3a00: 0000000000000000 ffff0000092cf3a4 0000000000000000 ffff00000988b728 [ 0.000000] 3a20: ffff00000a174000 ffff000009419fb0 000000000c67c24e ffff0000095e3198 [ 0.000000] 3a40: ffff0000096833f8 000000000000001a 000000000000001b 000000000000001c [ 0.000000] 3a60: 0000000000000001 ffff0000093ef000 ffff80097ffc7840 ffffffffffffff00 [ 0.000000] 3a80: ffff80097ffc7800 ffff00000940f000 0000000000000000 00007ff6800387c0 [ 0.000000] 3aa0: ffff80097ffc7700 000000000000000a 0000000000000000 ffff000009403b10 [ 0.000000] 3ac0: ffff0000082594ac ffff000009403b10 ffff0000082594ec 00000000600000c5 [ 0.000000] 3ae0: ffffffffffffff00 ffff80097ffc7800 ffffffffffffffff ffff0000082594ac [ 0.000000] 3b00: ffff000009403b10 ffff0000082594ec [ 0.000000] [<ffff0000082594ec>] __rmqueue+0x3c4/0x908 [ 0.000000] [<ffff000008259e98>] get_page_from_freelist+0x468/0xcb8 [ 0.000000] [<ffff00000825aa3c>] __alloc_pages_nodemask+0x12c/0x1248 [ 0.000000] [<ffff0000082cdba8>] new_slab+0xc0/0x568 [ 0.000000] [<ffff0000082d5328>] __kmem_cache_create+0x2e0/0x628 [ 0.000000] [<ffff000009271dec>] create_boot_cache+0xa8/0xdc [ 0.000000] [<ffff000009275b40>] kmem_cache_init+0x58/0x120 [ 0.000000] [<ffff000009250b3c>] start_kernel+0x200/0x3d8 [ 0.000000] Code: 8b000401 8b010800 d37df000 8b081000 (f9408000) [ 0.000000] random: get_random_bytes called from print_oops_end_marker+0x54/0x70 with crng_init=0 [ 0.000000] ---[ end trace c75a488acdeef932 ]--- [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task! [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task!
Full test log link, https://lkft.validation.linaro.org/scheduler/job/1004132#L482
Any chance you can bisect?
thanks,
greg k-h
Hello!
On Mon, 11 Nov 2019 at 12:58, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
On Mon, Nov 11, 2019 at 11:45:35PM +0530, Naresh Kamboju wrote:
Hi Greg,
Regressions detected in project stable v.4.14. arm64, arm, x86_64 boot failed.
Summary
git branch: linux-4.14.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: a67492b55c53045e9c0b9969f04410723448c1ee git describe: v4.14.153-104-ga67492b55c53 make_kernelversion: 4.14.154-rc1 kernel-config: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab...
arm64 kernel crash log:
[ 0.000000] Linux version 4.14.154-rc1 (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP PREEMPT Mon Nov 11 17:15:34 UTC 2019 [ 0.000000] Boot CPU: AArch64 Processor [410fd033] [ 0.000000] Machine model: ARM Juno development board (r2) [ 0.000000] earlycon: pl11 at MMIO 0x000000007ff80000 (options '') [ 0.000000] bootconsole [pl11] enabled [ 0.000000] efi: Getting EFI parameters from FDT: [ 0.000000] efi: UEFI not found. [ 0.000000] cma: Reserved 16 MiB at 0x00000000fe000000 [ 0.000000] NUMA: No NUMA configuration found [ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000009ffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x9fffc6f80-0x9fffc8bff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000080000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x00000009ffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080000000-0x00000000feffffff] [ 0.000000] node 0: [mem 0x0000000880000000-0x00000009ffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000009ffffffff] [ 0.000000] On node 0 totalpages: 2093056 [ 0.000000] DMA zone: 8192 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 520192 pages, LIFO batch:31 [ 0.000000] Normal zone: 24576 pages used for memmap [ 0.000000] Normal zone: 1572864 pages, LIFO batch:31 [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.0 [ 0.000000] percpu: Embedded 25 pages/cpu s62280 r8192 d31928 u102400 [ 0.000000] pcpu-alloc: s62280 r8192 d31928 u102400 alloc=25*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: enabling workaround for ARM erratum 845719 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 2060288 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: console=ttyAMA0,115200n8 root=/dev/nfs rw nfsroot=10.66.16.123:/var/lib/lava/dispatcher/tmp/1004132/extract-nfsrootfs-jn9hl2x_,tcp,hard,intr,vers=3 rootwait earlycon=pl011,0x7ff80000 debug systemd.log_target=null user_debug=31 androidboot.hardware=juno loglevel=9 sky2.mac_address=0x00,0x02,0xF7,0x00,0x68,0x15 ip=dhcp [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] software IO TLB: mapped [mem 0xf9fff000-0xfdfff000] (64MB) [ 0.000000] Memory: 8123672K/8372224K available (12988K kernel code, 1878K rwdata, 5196K rodata, 1728K init, 12356K bss, 232168K reserved, 16384K cma-reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB) [ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB) [ 0.000000] .text : 0xffff000008080000 - 0xffff000008d30000 ( 12992 KB) [ 0.000000] .rodata : 0xffff000008d30000 - 0xffff000009250000 ( 5248 KB) [ 0.000000] .init : 0xffff000009250000 - 0xffff000009400000 ( 1728 KB) [ 0.000000] .data : 0xffff000009400000 - 0xffff0000095d5a00 ( 1879 KB) [ 0.000000] .bss : 0xffff0000095d5a00 - 0xffff00000a1e6dd0 ( 12357 KB) [ 0.000000] fixed : 0xffff7dfffe7f9000 - 0xffff7dfffec00000 ( 4124 KB) [ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB) [ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum) [ 0.000000] 0xffff7e0000000000 - 0xffff7e0026000000 ( 608 MB actual) [ 0.000000] memory : 0xffff800000000000 - 0xffff800980000000 ( 38912 MB) [ 0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00000510 [ 0.000000] Mem abort info: [ 0.000000] Exception class = DABT (current EL), IL = 32 bits [ 0.000000] SET = 0, FnV = 0 [ 0.000000] EA = 0, S1PTW = 0 [ 0.000000] Data abort info: [ 0.000000] ISV = 0, ISS = 0x00000004 [ 0.000000] CM = 0, WnR = 0 [ 0.000000] [0000000000000510] user address but active_mm is swapper [ 0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 0.000000] Modules linked in: [ 0.000000] Process swapper (pid: 0, stack limit = 0xffff000009400000) [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.154-rc1 #1 [ 0.000000] Hardware name: ARM Juno development board (r2) (DT) [ 0.000000] task: ffff000009419780 task.stack: ffff000009400000 [ 0.000000] PC is at __rmqueue+0x3c4/0x908 [ 0.000000] LR is at __rmqueue+0x384/0x908 [ 0.000000] pc : [<ffff0000082594ec>] lr : [<ffff0000082594ac>] pstate: 600000c5 [ 0.000000] sp : ffff000009403b10 [ 0.000000] x29: ffff000009403b10 x28: 0000000000000000 [ 0.000000] x27: 000000000000000a x26: ffff80097ffc7700 [ 0.000000] x25: 00007ff6800387c0 x24: 0000000000000000 [ 0.000000] x23: ffff00000940f000 x22: ffff80097ffc7800 [ 0.000000] x21: ffffffffffffff00 x20: ffff80097ffc7840 [ 0.000000] x19: ffff0000093ef000 x18: 0000000000000001 [ 0.000000] x17: 000000000000001c x16: 000000000000001b [ 0.000000] x15: 000000000000001a x14: ffff0000096833f8 [ 0.000000] x13: ffff0000095e3198 x12: 000000000c67c24e [ 0.000000] x11: ffff000009419fb0 x10: ffff00000a174000 [ 0.000000] x9 : ffff00000988b728 x8 : 0000000000000000 [ 0.000000] x7 : ffff0000092cf3a4 x6 : 0000000000000000 [ 0.000000] x5 : fffffffffffff1ff x4 : ffff7e0025de0020 [ 0.000000] x3 : ffff7e0025dd0020 x2 : 000000000000000a [ 0.000000] x1 : 000000000000001e x0 : 0000000000000410 [ 0.000000] Call trace: [ 0.000000] Exception stack(0xffff0000094039d0 to 0xffff000009403b10) [ 0.000000] 39c0: 0000000000000410 000000000000001e [ 0.000000] 39e0: 000000000000000a ffff7e0025dd0020 ffff7e0025de0020 fffffffffffff1ff [ 0.000000] 3a00: 0000000000000000 ffff0000092cf3a4 0000000000000000 ffff00000988b728 [ 0.000000] 3a20: ffff00000a174000 ffff000009419fb0 000000000c67c24e ffff0000095e3198 [ 0.000000] 3a40: ffff0000096833f8 000000000000001a 000000000000001b 000000000000001c [ 0.000000] 3a60: 0000000000000001 ffff0000093ef000 ffff80097ffc7840 ffffffffffffff00 [ 0.000000] 3a80: ffff80097ffc7800 ffff00000940f000 0000000000000000 00007ff6800387c0 [ 0.000000] 3aa0: ffff80097ffc7700 000000000000000a 0000000000000000 ffff000009403b10 [ 0.000000] 3ac0: ffff0000082594ac ffff000009403b10 ffff0000082594ec 00000000600000c5 [ 0.000000] 3ae0: ffffffffffffff00 ffff80097ffc7800 ffffffffffffffff ffff0000082594ac [ 0.000000] 3b00: ffff000009403b10 ffff0000082594ec [ 0.000000] [<ffff0000082594ec>] __rmqueue+0x3c4/0x908 [ 0.000000] [<ffff000008259e98>] get_page_from_freelist+0x468/0xcb8 [ 0.000000] [<ffff00000825aa3c>] __alloc_pages_nodemask+0x12c/0x1248 [ 0.000000] [<ffff0000082cdba8>] new_slab+0xc0/0x568 [ 0.000000] [<ffff0000082d5328>] __kmem_cache_create+0x2e0/0x628 [ 0.000000] [<ffff000009271dec>] create_boot_cache+0xa8/0xdc [ 0.000000] [<ffff000009275b40>] kmem_cache_init+0x58/0x120 [ 0.000000] [<ffff000009250b3c>] start_kernel+0x200/0x3d8 [ 0.000000] Code: 8b000401 8b010800 d37df000 8b081000 (f9408000) [ 0.000000] random: get_random_bytes called from print_oops_end_marker+0x54/0x70 with crng_init=0 [ 0.000000] ---[ end trace c75a488acdeef932 ]--- [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task! [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task!
Full test log link, https://lkft.validation.linaro.org/scheduler/job/1004132#L482
Any chance you can bisect?
Reverting 61dbb1f20417 ("mm, meminit: recalculate pcpu batch and high limits after init completes") got the system working again.
Greetings!
Daniel Díaz daniel.diaz@linaro.org
On Mon, Nov 11, 2019 at 09:19:40PM -0600, Daniel Díaz wrote:
Hello!
On Mon, 11 Nov 2019 at 12:58, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
On Mon, Nov 11, 2019 at 11:45:35PM +0530, Naresh Kamboju wrote:
Hi Greg,
Regressions detected in project stable v.4.14. arm64, arm, x86_64 boot failed.
Summary
git branch: linux-4.14.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: a67492b55c53045e9c0b9969f04410723448c1ee git describe: v4.14.153-104-ga67492b55c53 make_kernelversion: 4.14.154-rc1 kernel-config: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab...
arm64 kernel crash log:
[ 0.000000] Linux version 4.14.154-rc1 (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP PREEMPT Mon Nov 11 17:15:34 UTC 2019 [ 0.000000] Boot CPU: AArch64 Processor [410fd033] [ 0.000000] Machine model: ARM Juno development board (r2) [ 0.000000] earlycon: pl11 at MMIO 0x000000007ff80000 (options '') [ 0.000000] bootconsole [pl11] enabled [ 0.000000] efi: Getting EFI parameters from FDT: [ 0.000000] efi: UEFI not found. [ 0.000000] cma: Reserved 16 MiB at 0x00000000fe000000 [ 0.000000] NUMA: No NUMA configuration found [ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000009ffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x9fffc6f80-0x9fffc8bff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000080000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x00000009ffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080000000-0x00000000feffffff] [ 0.000000] node 0: [mem 0x0000000880000000-0x00000009ffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000009ffffffff] [ 0.000000] On node 0 totalpages: 2093056 [ 0.000000] DMA zone: 8192 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 520192 pages, LIFO batch:31 [ 0.000000] Normal zone: 24576 pages used for memmap [ 0.000000] Normal zone: 1572864 pages, LIFO batch:31 [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.0 [ 0.000000] percpu: Embedded 25 pages/cpu s62280 r8192 d31928 u102400 [ 0.000000] pcpu-alloc: s62280 r8192 d31928 u102400 alloc=25*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: enabling workaround for ARM erratum 845719 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 2060288 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: console=ttyAMA0,115200n8 root=/dev/nfs rw nfsroot=10.66.16.123:/var/lib/lava/dispatcher/tmp/1004132/extract-nfsrootfs-jn9hl2x_,tcp,hard,intr,vers=3 rootwait earlycon=pl011,0x7ff80000 debug systemd.log_target=null user_debug=31 androidboot.hardware=juno loglevel=9 sky2.mac_address=0x00,0x02,0xF7,0x00,0x68,0x15 ip=dhcp [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] software IO TLB: mapped [mem 0xf9fff000-0xfdfff000] (64MB) [ 0.000000] Memory: 8123672K/8372224K available (12988K kernel code, 1878K rwdata, 5196K rodata, 1728K init, 12356K bss, 232168K reserved, 16384K cma-reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB) [ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB) [ 0.000000] .text : 0xffff000008080000 - 0xffff000008d30000 ( 12992 KB) [ 0.000000] .rodata : 0xffff000008d30000 - 0xffff000009250000 ( 5248 KB) [ 0.000000] .init : 0xffff000009250000 - 0xffff000009400000 ( 1728 KB) [ 0.000000] .data : 0xffff000009400000 - 0xffff0000095d5a00 ( 1879 KB) [ 0.000000] .bss : 0xffff0000095d5a00 - 0xffff00000a1e6dd0 ( 12357 KB) [ 0.000000] fixed : 0xffff7dfffe7f9000 - 0xffff7dfffec00000 ( 4124 KB) [ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB) [ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum) [ 0.000000] 0xffff7e0000000000 - 0xffff7e0026000000 ( 608 MB actual) [ 0.000000] memory : 0xffff800000000000 - 0xffff800980000000 ( 38912 MB) [ 0.000000] Unable to handle kernel NULL pointer dereference at virtual address 00000510 [ 0.000000] Mem abort info: [ 0.000000] Exception class = DABT (current EL), IL = 32 bits [ 0.000000] SET = 0, FnV = 0 [ 0.000000] EA = 0, S1PTW = 0 [ 0.000000] Data abort info: [ 0.000000] ISV = 0, ISS = 0x00000004 [ 0.000000] CM = 0, WnR = 0 [ 0.000000] [0000000000000510] user address but active_mm is swapper [ 0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 0.000000] Modules linked in: [ 0.000000] Process swapper (pid: 0, stack limit = 0xffff000009400000) [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.154-rc1 #1 [ 0.000000] Hardware name: ARM Juno development board (r2) (DT) [ 0.000000] task: ffff000009419780 task.stack: ffff000009400000 [ 0.000000] PC is at __rmqueue+0x3c4/0x908 [ 0.000000] LR is at __rmqueue+0x384/0x908 [ 0.000000] pc : [<ffff0000082594ec>] lr : [<ffff0000082594ac>] pstate: 600000c5 [ 0.000000] sp : ffff000009403b10 [ 0.000000] x29: ffff000009403b10 x28: 0000000000000000 [ 0.000000] x27: 000000000000000a x26: ffff80097ffc7700 [ 0.000000] x25: 00007ff6800387c0 x24: 0000000000000000 [ 0.000000] x23: ffff00000940f000 x22: ffff80097ffc7800 [ 0.000000] x21: ffffffffffffff00 x20: ffff80097ffc7840 [ 0.000000] x19: ffff0000093ef000 x18: 0000000000000001 [ 0.000000] x17: 000000000000001c x16: 000000000000001b [ 0.000000] x15: 000000000000001a x14: ffff0000096833f8 [ 0.000000] x13: ffff0000095e3198 x12: 000000000c67c24e [ 0.000000] x11: ffff000009419fb0 x10: ffff00000a174000 [ 0.000000] x9 : ffff00000988b728 x8 : 0000000000000000 [ 0.000000] x7 : ffff0000092cf3a4 x6 : 0000000000000000 [ 0.000000] x5 : fffffffffffff1ff x4 : ffff7e0025de0020 [ 0.000000] x3 : ffff7e0025dd0020 x2 : 000000000000000a [ 0.000000] x1 : 000000000000001e x0 : 0000000000000410 [ 0.000000] Call trace: [ 0.000000] Exception stack(0xffff0000094039d0 to 0xffff000009403b10) [ 0.000000] 39c0: 0000000000000410 000000000000001e [ 0.000000] 39e0: 000000000000000a ffff7e0025dd0020 ffff7e0025de0020 fffffffffffff1ff [ 0.000000] 3a00: 0000000000000000 ffff0000092cf3a4 0000000000000000 ffff00000988b728 [ 0.000000] 3a20: ffff00000a174000 ffff000009419fb0 000000000c67c24e ffff0000095e3198 [ 0.000000] 3a40: ffff0000096833f8 000000000000001a 000000000000001b 000000000000001c [ 0.000000] 3a60: 0000000000000001 ffff0000093ef000 ffff80097ffc7840 ffffffffffffff00 [ 0.000000] 3a80: ffff80097ffc7800 ffff00000940f000 0000000000000000 00007ff6800387c0 [ 0.000000] 3aa0: ffff80097ffc7700 000000000000000a 0000000000000000 ffff000009403b10 [ 0.000000] 3ac0: ffff0000082594ac ffff000009403b10 ffff0000082594ec 00000000600000c5 [ 0.000000] 3ae0: ffffffffffffff00 ffff80097ffc7800 ffffffffffffffff ffff0000082594ac [ 0.000000] 3b00: ffff000009403b10 ffff0000082594ec [ 0.000000] [<ffff0000082594ec>] __rmqueue+0x3c4/0x908 [ 0.000000] [<ffff000008259e98>] get_page_from_freelist+0x468/0xcb8 [ 0.000000] [<ffff00000825aa3c>] __alloc_pages_nodemask+0x12c/0x1248 [ 0.000000] [<ffff0000082cdba8>] new_slab+0xc0/0x568 [ 0.000000] [<ffff0000082d5328>] __kmem_cache_create+0x2e0/0x628 [ 0.000000] [<ffff000009271dec>] create_boot_cache+0xa8/0xdc [ 0.000000] [<ffff000009275b40>] kmem_cache_init+0x58/0x120 [ 0.000000] [<ffff000009250b3c>] start_kernel+0x200/0x3d8 [ 0.000000] Code: 8b000401 8b010800 d37df000 8b081000 (f9408000) [ 0.000000] random: get_random_bytes called from print_oops_end_marker+0x54/0x70 with crng_init=0 [ 0.000000] ---[ end trace c75a488acdeef932 ]--- [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task! [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task!
Full test log link, https://lkft.validation.linaro.org/scheduler/job/1004132#L482
Any chance you can bisect?
Reverting 61dbb1f20417 ("mm, meminit: recalculate pcpu batch and high limits after init completes") got the system working again.
Yeah, I messed that one up :(
I'm pushing out a -rc2 now to hopefully fix this up, thanks!
greg k-h
On Tue, 12 Nov 2019 at 11:02, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
Any chance you can bisect?
Reverting 61dbb1f20417 ("mm, meminit: recalculate pcpu batch and high limits after init completes") got the system working again.
Yeah, I messed that one up :(
I'm pushing out a -rc2 now to hopefully fix this up, thanks!
The -rc2 boot pass. Full functional testing is in progress.
- Naresh
On Tue, Nov 12, 2019 at 12:50:39PM +0530, Naresh Kamboju wrote:
On Tue, 12 Nov 2019 at 11:02, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
Any chance you can bisect?
Reverting 61dbb1f20417 ("mm, meminit: recalculate pcpu batch and high limits after init completes") got the system working again.
Yeah, I messed that one up :(
I'm pushing out a -rc2 now to hopefully fix this up, thanks!
The -rc2 boot pass.
Great!
linux-stable-mirror@lists.linaro.org
-
Daniel Díaz -
Greg Kroah-Hartman -
Naresh Kamboju