next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Regressions found while running LTP msync04 tests on qemu-arm64 running Linux next-20250721, next-20250722 and next-20250723 with 16K and 64K page size enabled builds.
CONFIG_ARM64_64K_PAGES=y ( kernel warning as below ) CONFIG_ARM64_16K_PAGES=y ( kernel warning as below )
No warning noticed with 4K page size. CONFIG_ARM64_4K_PAGES=y works as expected
First seen on the tag next-20250721. Good: next-20250718 Bad: next-20250721 to next-20250723
Regression Analysis: - New regression? Yes - Reproducibility? Yes
Test regression: next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
## Test log ------------[ cut here ]------------ [ 343.828105] WARNING: fs/fuse/file.c:2146 at fuse_iomap_writeback_range+0x478/0x558 [fuse], CPU#0: msync04/4190 [ 343.830969] Modules linked in: btrfs blake2b_generic xor xor_neon raid6_pq zstd_compress sm3_ce sha3_ce drm fuse backlight ip_tables x_tables [ 343.833830] CPU: 0 UID: 0 PID: 4190 Comm: msync04 Not tainted 6.16.0-rc7-next-20250723 #1 PREEMPT [ 343.834736] Hardware name: linux,dummy-virt (DT) [ 343.835788] pstate: 03402009 (nzcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--) [ 343.836455] pc : fuse_iomap_writeback_range+0x478/0x558 fuse [ 343.837294] lr : iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.838178] sp : ffff80008b26f8d0 [ 343.838668] x29: ffff80008b26f8d0 x28: fff00000e7f8c800 x27: 0000000000000000 [ 343.839391] x26: fff00000d4b30000 x25: 0000000000000000 x24: 0000000000000000 [ 343.840305] x23: 0000000000000000 x22: fffffc1fc0334200 x21: 0000000000001000 [ 343.840928] x20: ffff80008b26fa00 x19: 0000000000000000 x18: 0000000000000000 [ 343.841782] x17: 0000000000000000 x16: ffffb8d3b90c67c8 x15: 0000000000000000 [ 343.842565] x14: ffffb8d3ba91e340 x13: 0000ffff8ff3ffff x12: 0000000000000000 [ 343.843002] x11: 1ffe000004b74a21 x10: fff0000025ba510c x9 : ffffb8d3b90c6308 [ 343.843962] x8 : ffff80008b26f788 x7 : ffffb8d365830b90 x6 : ffffb8d3bb6c9000 [ 343.844718] x5 : 0000000000000000 x4 : 000000000000000a x3 : 0000000000001000 [ 343.845333] x2 : fff00000c0b5ecc0 x1 : 000000000000ffff x0 : 0bfffe000000400b [ 343.846323] Call trace: [ 343.846767] fuse_iomap_writeback_range+0x478/0x558 fuse (P) [ 343.847288] iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.847930] iomap_writepages (fs/iomap/buffered-io.c:1762) [ 343.848494] fuse_writepages+0xa0/0xe8 fuse [ 343.849112] do_writepages (mm/page-writeback.c:2634) [ 343.849614] filemap_fdatawrite_wbc (mm/filemap.c:386 mm/filemap.c:376) [ 343.850202] __filemap_fdatawrite_range (mm/filemap.c:420) [ 343.850791] file_write_and_wait_range (mm/filemap.c:794) [ 343.851108] fuse_fsync+0x6c/0x138 fuse [ 343.851688] vfs_fsync_range (fs/sync.c:188) [ 343.852002] __arm64_sys_msync (mm/msync.c:96 mm/msync.c:32 mm/msync.c:32) [ 343.852197] invoke_syscall.constprop.0 (arch/arm64/include/asm/syscall.h:61 arch/arm64/kernel/syscall.c:54) [ 343.852914] do_el0_svc (include/linux/thread_info.h:135 (discriminator 2) arch/arm64/kernel/syscall.c:140 (discriminator 2) arch/arm64/kernel/syscall.c:151 (discriminator 2)) [ 343.853389] el0_svc (arch/arm64/include/asm/irqflags.h:82 (discriminator 1) arch/arm64/include/asm/irqflags.h:123 (discriminator 1) arch/arm64/include/asm/irqflags.h:136 (discriminator 1) arch/arm64/kernel/entry-common.c:169 (discriminator 1) arch/arm64/kernel/entry-common.c:182 (discriminator 1) arch/arm64/kernel/entry-common.c:880 (discriminator 1)) [ 343.853829] el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:899) [ 343.854350] el0t_64_sync (arch/arm64/kernel/entry.S:596) [ 343.854652] ---[ end trace 0000000000000000 ]---
## Source * Git tree: https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git * Project: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/ * Git sha: a933d3dc1968fcfb0ab72879ec304b1971ed1b9a * Git describe: 6.16.0-rc7-next-20250723 * kernel version: next-20250723 * Architectures: arm64 * Toolchains: gcc-13 * Kconfigs: defconfig + CONFIG_ARM64_64K_PAGES=y * Kconfigs: defconfig + CONFIG_ARM64_16K_PAGES=y
## Test * Test log 1: https://qa-reports.linaro.org/api/testruns/29227309/log_file/ * Test log 2: https://qa-reports.linaro.org/api/testruns/29227074/log_file/ * Test run: https://regressions.linaro.org/lkft/linux-next-master/next-20250723/testruns... * Test history: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/tes... * Test plan: https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/30G3hpJVVdX... * Build link: https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t... * Kernel config: https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
-- Linaro LKFT https://lkft.linaro.org
[cc Joanne]
On Wed, Jul 23, 2025 at 05:14:28PM +0530, Naresh Kamboju wrote:
Regressions found while running LTP msync04 tests on qemu-arm64 running Linux next-20250721, next-20250722 and next-20250723 with 16K and 64K page size enabled builds.
CONFIG_ARM64_64K_PAGES=y ( kernel warning as below ) CONFIG_ARM64_16K_PAGES=y ( kernel warning as below )
No warning noticed with 4K page size. CONFIG_ARM64_4K_PAGES=y works as expected
You might want to cc Joanne since she's been working on large folio support in fuse.
First seen on the tag next-20250721. Good: next-20250718 Bad: next-20250721 to next-20250723
Regression Analysis:
- New regression? Yes
- Reproducibility? Yes
Test regression: next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
## Test log ------------[ cut here ]------------ [ 343.828105] WARNING: fs/fuse/file.c:2146 at fuse_iomap_writeback_range+0x478/0x558 [fuse], CPU#0: msync04/4190
WARN_ON_ONCE(len & (PAGE_SIZE - 1));
/me speculates that this might be triggered by an attempt to write back some 4k fsblock within a 16/64k base page?
--D
[ 343.830969] Modules linked in: btrfs blake2b_generic xor xor_neon raid6_pq zstd_compress sm3_ce sha3_ce drm fuse backlight ip_tables x_tables [ 343.833830] CPU: 0 UID: 0 PID: 4190 Comm: msync04 Not tainted 6.16.0-rc7-next-20250723 #1 PREEMPT [ 343.834736] Hardware name: linux,dummy-virt (DT) [ 343.835788] pstate: 03402009 (nzcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--) [ 343.836455] pc : fuse_iomap_writeback_range+0x478/0x558 fuse [ 343.837294] lr : iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.838178] sp : ffff80008b26f8d0 [ 343.838668] x29: ffff80008b26f8d0 x28: fff00000e7f8c800 x27: 0000000000000000 [ 343.839391] x26: fff00000d4b30000 x25: 0000000000000000 x24: 0000000000000000 [ 343.840305] x23: 0000000000000000 x22: fffffc1fc0334200 x21: 0000000000001000 [ 343.840928] x20: ffff80008b26fa00 x19: 0000000000000000 x18: 0000000000000000 [ 343.841782] x17: 0000000000000000 x16: ffffb8d3b90c67c8 x15: 0000000000000000 [ 343.842565] x14: ffffb8d3ba91e340 x13: 0000ffff8ff3ffff x12: 0000000000000000 [ 343.843002] x11: 1ffe000004b74a21 x10: fff0000025ba510c x9 : ffffb8d3b90c6308 [ 343.843962] x8 : ffff80008b26f788 x7 : ffffb8d365830b90 x6 : ffffb8d3bb6c9000 [ 343.844718] x5 : 0000000000000000 x4 : 000000000000000a x3 : 0000000000001000 [ 343.845333] x2 : fff00000c0b5ecc0 x1 : 000000000000ffff x0 : 0bfffe000000400b [ 343.846323] Call trace: [ 343.846767] fuse_iomap_writeback_range+0x478/0x558 fuse (P) [ 343.847288] iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.847930] iomap_writepages (fs/iomap/buffered-io.c:1762) [ 343.848494] fuse_writepages+0xa0/0xe8 fuse [ 343.849112] do_writepages (mm/page-writeback.c:2634) [ 343.849614] filemap_fdatawrite_wbc (mm/filemap.c:386 mm/filemap.c:376) [ 343.850202] __filemap_fdatawrite_range (mm/filemap.c:420) [ 343.850791] file_write_and_wait_range (mm/filemap.c:794) [ 343.851108] fuse_fsync+0x6c/0x138 fuse [ 343.851688] vfs_fsync_range (fs/sync.c:188) [ 343.852002] __arm64_sys_msync (mm/msync.c:96 mm/msync.c:32 mm/msync.c:32) [ 343.852197] invoke_syscall.constprop.0 (arch/arm64/include/asm/syscall.h:61 arch/arm64/kernel/syscall.c:54) [ 343.852914] do_el0_svc (include/linux/thread_info.h:135 (discriminator 2) arch/arm64/kernel/syscall.c:140 (discriminator 2) arch/arm64/kernel/syscall.c:151 (discriminator 2)) [ 343.853389] el0_svc (arch/arm64/include/asm/irqflags.h:82 (discriminator 1) arch/arm64/include/asm/irqflags.h:123 (discriminator
- arch/arm64/include/asm/irqflags.h:136 (discriminator 1)
arch/arm64/kernel/entry-common.c:169 (discriminator 1) arch/arm64/kernel/entry-common.c:182 (discriminator 1) arch/arm64/kernel/entry-common.c:880 (discriminator 1)) [ 343.853829] el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:899) [ 343.854350] el0t_64_sync (arch/arm64/kernel/entry.S:596) [ 343.854652] ---[ end trace 0000000000000000 ]---
## Source
- Git tree: https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git
- Project: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/
- Git sha: a933d3dc1968fcfb0ab72879ec304b1971ed1b9a
- Git describe: 6.16.0-rc7-next-20250723
- kernel version: next-20250723
- Architectures: arm64
- Toolchains: gcc-13
- Kconfigs: defconfig + CONFIG_ARM64_64K_PAGES=y
- Kconfigs: defconfig + CONFIG_ARM64_16K_PAGES=y
## Test
- Test log 1: https://qa-reports.linaro.org/api/testruns/29227309/log_file/
- Test log 2: https://qa-reports.linaro.org/api/testruns/29227074/log_file/
- Test run: https://regressions.linaro.org/lkft/linux-next-master/next-20250723/testruns...
- Test history:
https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/tes...
- Test plan: https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/30G3hpJVVdX...
- Build link: https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
- Kernel config:
https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
-- Linaro LKFT https://lkft.linaro.org
On Wed, Jul 23, 2025 at 7:46 AM Darrick J. Wong djwong@kernel.org wrote:
[cc Joanne]
On Wed, Jul 23, 2025 at 05:14:28PM +0530, Naresh Kamboju wrote:
Regressions found while running LTP msync04 tests on qemu-arm64 running Linux next-20250721, next-20250722 and next-20250723 with 16K and 64K page size enabled builds.
CONFIG_ARM64_64K_PAGES=y ( kernel warning as below ) CONFIG_ARM64_16K_PAGES=y ( kernel warning as below )
No warning noticed with 4K page size. CONFIG_ARM64_4K_PAGES=y works as expected
You might want to cc Joanne since she's been working on large folio support in fuse.
First seen on the tag next-20250721. Good: next-20250718 Bad: next-20250721 to next-20250723
Thanks for the report. Is there a link to the script that mounts the fuse server for these tests? I'm curious whether this was mounted as a fuseblk filesystem.
Regression Analysis:
- New regression? Yes
- Reproducibility? Yes
Test regression: next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
## Test log ------------[ cut here ]------------ [ 343.828105] WARNING: fs/fuse/file.c:2146 at fuse_iomap_writeback_range+0x478/0x558 [fuse], CPU#0: msync04/4190
WARN_ON_ONCE(len & (PAGE_SIZE - 1));/me speculates that this might be triggered by an attempt to write back some 4k fsblock within a 16/64k base page?
I think this can happen on 4k base pages as well actually. On the iomap side, the length passed is always block-aligned and in fuse, we set blkbits to be PAGE_SHIFT so theoretically block-aligned is always page-aligned, but I missed that if it's a "fuseblk" filesystem, that isn't true and the blocksize is initialized to a default size of 512 or whatever block size is passed in when it's mounted.
I'll send out a patch to remove this line. It doesn't make any difference for fuse_iomap_writeback_range() logic whether len is page-aligned or not; I had added it as a sanity-check against sketchy ranges.
Also, I just noticed that apparently the blocksize can change dynamically for an inode in fuse through getattr replies from the server (see fuse_change_attributes_common()). This is a problem since the iomap uses inode->i_blkbits for reading/writing to the bitmap. I think we will have to cache the inode blkbits in the iomap_folio_state struct unfortunately :( I'll think about this some more and send out a patch for this.
Thanks, Joanne
--D
[ 343.830969] Modules linked in: btrfs blake2b_generic xor xor_neon raid6_pq zstd_compress sm3_ce sha3_ce drm fuse backlight ip_tables x_tables [ 343.833830] CPU: 0 UID: 0 PID: 4190 Comm: msync04 Not tainted 6.16.0-rc7-next-20250723 #1 PREEMPT [ 343.834736] Hardware name: linux,dummy-virt (DT) [ 343.835788] pstate: 03402009 (nzcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--) [ 343.836455] pc : fuse_iomap_writeback_range+0x478/0x558 fuse [ 343.837294] lr : iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.838178] sp : ffff80008b26f8d0 [ 343.838668] x29: ffff80008b26f8d0 x28: fff00000e7f8c800 x27: 0000000000000000 [ 343.839391] x26: fff00000d4b30000 x25: 0000000000000000 x24: 0000000000000000 [ 343.840305] x23: 0000000000000000 x22: fffffc1fc0334200 x21: 0000000000001000 [ 343.840928] x20: ffff80008b26fa00 x19: 0000000000000000 x18: 0000000000000000 [ 343.841782] x17: 0000000000000000 x16: ffffb8d3b90c67c8 x15: 0000000000000000 [ 343.842565] x14: ffffb8d3ba91e340 x13: 0000ffff8ff3ffff x12: 0000000000000000 [ 343.843002] x11: 1ffe000004b74a21 x10: fff0000025ba510c x9 : ffffb8d3b90c6308 [ 343.843962] x8 : ffff80008b26f788 x7 : ffffb8d365830b90 x6 : ffffb8d3bb6c9000 [ 343.844718] x5 : 0000000000000000 x4 : 000000000000000a x3 : 0000000000001000 [ 343.845333] x2 : fff00000c0b5ecc0 x1 : 000000000000ffff x0 : 0bfffe000000400b [ 343.846323] Call trace: [ 343.846767] fuse_iomap_writeback_range+0x478/0x558 fuse (P) [ 343.847288] iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.847930] iomap_writepages (fs/iomap/buffered-io.c:1762) [ 343.848494] fuse_writepages+0xa0/0xe8 fuse [ 343.849112] do_writepages (mm/page-writeback.c:2634) [ 343.849614] filemap_fdatawrite_wbc (mm/filemap.c:386 mm/filemap.c:376) [ 343.850202] __filemap_fdatawrite_range (mm/filemap.c:420) [ 343.850791] file_write_and_wait_range (mm/filemap.c:794) [ 343.851108] fuse_fsync+0x6c/0x138 fuse [ 343.851688] vfs_fsync_range (fs/sync.c:188) [ 343.852002] __arm64_sys_msync (mm/msync.c:96 mm/msync.c:32 mm/msync.c:32) [ 343.852197] invoke_syscall.constprop.0 (arch/arm64/include/asm/syscall.h:61 arch/arm64/kernel/syscall.c:54) [ 343.852914] do_el0_svc (include/linux/thread_info.h:135 (discriminator 2) arch/arm64/kernel/syscall.c:140 (discriminator 2) arch/arm64/kernel/syscall.c:151 (discriminator 2)) [ 343.853389] el0_svc (arch/arm64/include/asm/irqflags.h:82 (discriminator 1) arch/arm64/include/asm/irqflags.h:123 (discriminator
- arch/arm64/include/asm/irqflags.h:136 (discriminator 1)
arch/arm64/kernel/entry-common.c:169 (discriminator 1) arch/arm64/kernel/entry-common.c:182 (discriminator 1) arch/arm64/kernel/entry-common.c:880 (discriminator 1)) [ 343.853829] el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:899) [ 343.854350] el0t_64_sync (arch/arm64/kernel/entry.S:596) [ 343.854652] ---[ end trace 0000000000000000 ]---
## Source
- Git tree: https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git
- Project: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/
- Git sha: a933d3dc1968fcfb0ab72879ec304b1971ed1b9a
- Git describe: 6.16.0-rc7-next-20250723
- kernel version: next-20250723
- Architectures: arm64
- Toolchains: gcc-13
- Kconfigs: defconfig + CONFIG_ARM64_64K_PAGES=y
- Kconfigs: defconfig + CONFIG_ARM64_16K_PAGES=y
## Test
- Test log 1: https://qa-reports.linaro.org/api/testruns/29227309/log_file/
- Test log 2: https://qa-reports.linaro.org/api/testruns/29227074/log_file/
- Test run: https://regressions.linaro.org/lkft/linux-next-master/next-20250723/testruns...
- Test history:
https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/tes...
- Test plan: https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/30G3hpJVVdX...
- Build link: https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
- Kernel config:
https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
-- Linaro LKFT https://lkft.linaro.org
On Wed, Jul 23, 2025 at 11:42:42AM -0700, Joanne Koong wrote:
On Wed, Jul 23, 2025 at 7:46 AM Darrick J. Wong djwong@kernel.org wrote:
[cc Joanne]
On Wed, Jul 23, 2025 at 05:14:28PM +0530, Naresh Kamboju wrote:
Regressions found while running LTP msync04 tests on qemu-arm64 running Linux next-20250721, next-20250722 and next-20250723 with 16K and 64K page size enabled builds.
CONFIG_ARM64_64K_PAGES=y ( kernel warning as below ) CONFIG_ARM64_16K_PAGES=y ( kernel warning as below )
No warning noticed with 4K page size. CONFIG_ARM64_4K_PAGES=y works as expected
You might want to cc Joanne since she's been working on large folio support in fuse.
First seen on the tag next-20250721. Good: next-20250718 Bad: next-20250721 to next-20250723
Thanks for the report. Is there a link to the script that mounts the fuse server for these tests? I'm curious whether this was mounted as a fuseblk filesystem.
Regression Analysis:
- New regression? Yes
- Reproducibility? Yes
Test regression: next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
## Test log ------------[ cut here ]------------ [ 343.828105] WARNING: fs/fuse/file.c:2146 at fuse_iomap_writeback_range+0x478/0x558 [fuse], CPU#0: msync04/4190
WARN_ON_ONCE(len & (PAGE_SIZE - 1));/me speculates that this might be triggered by an attempt to write back some 4k fsblock within a 16/64k base page?
I think this can happen on 4k base pages as well actually. On the iomap side, the length passed is always block-aligned and in fuse, we set blkbits to be PAGE_SHIFT so theoretically block-aligned is always page-aligned, but I missed that if it's a "fuseblk" filesystem, that isn't true and the blocksize is initialized to a default size of 512 or whatever block size is passed in when it's mounted.
<nod> I think you're correct.
I'll send out a patch to remove this line. It doesn't make any difference for fuse_iomap_writeback_range() logic whether len is page-aligned or not; I had added it as a sanity-check against sketchy ranges.
Also, I just noticed that apparently the blocksize can change dynamically for an inode in fuse through getattr replies from the server (see fuse_change_attributes_common()). This is a problem since the iomap uses inode->i_blkbits for reading/writing to the bitmap. I think we will have to cache the inode blkbits in the iomap_folio_state struct unfortunately :( I'll think about this some more and send out a patch for this.
From my understanding of the iomap code, it's possible to do that if you flush and unmap the entire pagecache (whilst holding i_rwsem and mmap_invalidate_lock) before you change i_blkbits. Nobody *does* this so I have no idea if it actually works, however. Note that even I don't implement the flush and unmap bit; I just scream loudly and do nothing:
void fuse_iomap_set_i_blkbits(struct inode *inode, u8 new_blkbits) { trace_fuse_iomap_set_i_blkbits(inode, new_blkbits);
if (inode->i_blkbits == new_blkbits) return;
if (!S_ISREG(inode->i_mode)) goto set_it;
/* * iomap attaches per-block state to each folio, so we cannot allow * the file block size to change if there's anything in the page cache. * In theory, fuse servers should never be doing this. */ if (inode->i_mapping->nrpages > 0) { WARN_ON(inode->i_blkbits != new_blkbits && inode->i_mapping->nrpages > 0); return; }
set_it: inode->i_blkbits = new_blkbits; }
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/commit/...
--D
Thanks, Joanne
--D
[ 343.830969] Modules linked in: btrfs blake2b_generic xor xor_neon raid6_pq zstd_compress sm3_ce sha3_ce drm fuse backlight ip_tables x_tables [ 343.833830] CPU: 0 UID: 0 PID: 4190 Comm: msync04 Not tainted 6.16.0-rc7-next-20250723 #1 PREEMPT [ 343.834736] Hardware name: linux,dummy-virt (DT) [ 343.835788] pstate: 03402009 (nzcv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--) [ 343.836455] pc : fuse_iomap_writeback_range+0x478/0x558 fuse [ 343.837294] lr : iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.838178] sp : ffff80008b26f8d0 [ 343.838668] x29: ffff80008b26f8d0 x28: fff00000e7f8c800 x27: 0000000000000000 [ 343.839391] x26: fff00000d4b30000 x25: 0000000000000000 x24: 0000000000000000 [ 343.840305] x23: 0000000000000000 x22: fffffc1fc0334200 x21: 0000000000001000 [ 343.840928] x20: ffff80008b26fa00 x19: 0000000000000000 x18: 0000000000000000 [ 343.841782] x17: 0000000000000000 x16: ffffb8d3b90c67c8 x15: 0000000000000000 [ 343.842565] x14: ffffb8d3ba91e340 x13: 0000ffff8ff3ffff x12: 0000000000000000 [ 343.843002] x11: 1ffe000004b74a21 x10: fff0000025ba510c x9 : ffffb8d3b90c6308 [ 343.843962] x8 : ffff80008b26f788 x7 : ffffb8d365830b90 x6 : ffffb8d3bb6c9000 [ 343.844718] x5 : 0000000000000000 x4 : 000000000000000a x3 : 0000000000001000 [ 343.845333] x2 : fff00000c0b5ecc0 x1 : 000000000000ffff x0 : 0bfffe000000400b [ 343.846323] Call trace: [ 343.846767] fuse_iomap_writeback_range+0x478/0x558 fuse (P) [ 343.847288] iomap_writeback_folio (fs/iomap/buffered-io.c:1586 fs/iomap/buffered-io.c:1710) [ 343.847930] iomap_writepages (fs/iomap/buffered-io.c:1762) [ 343.848494] fuse_writepages+0xa0/0xe8 fuse [ 343.849112] do_writepages (mm/page-writeback.c:2634) [ 343.849614] filemap_fdatawrite_wbc (mm/filemap.c:386 mm/filemap.c:376) [ 343.850202] __filemap_fdatawrite_range (mm/filemap.c:420) [ 343.850791] file_write_and_wait_range (mm/filemap.c:794) [ 343.851108] fuse_fsync+0x6c/0x138 fuse [ 343.851688] vfs_fsync_range (fs/sync.c:188) [ 343.852002] __arm64_sys_msync (mm/msync.c:96 mm/msync.c:32 mm/msync.c:32) [ 343.852197] invoke_syscall.constprop.0 (arch/arm64/include/asm/syscall.h:61 arch/arm64/kernel/syscall.c:54) [ 343.852914] do_el0_svc (include/linux/thread_info.h:135 (discriminator 2) arch/arm64/kernel/syscall.c:140 (discriminator 2) arch/arm64/kernel/syscall.c:151 (discriminator 2)) [ 343.853389] el0_svc (arch/arm64/include/asm/irqflags.h:82 (discriminator 1) arch/arm64/include/asm/irqflags.h:123 (discriminator
- arch/arm64/include/asm/irqflags.h:136 (discriminator 1)
arch/arm64/kernel/entry-common.c:169 (discriminator 1) arch/arm64/kernel/entry-common.c:182 (discriminator 1) arch/arm64/kernel/entry-common.c:880 (discriminator 1)) [ 343.853829] el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:899) [ 343.854350] el0t_64_sync (arch/arm64/kernel/entry.S:596) [ 343.854652] ---[ end trace 0000000000000000 ]---
## Source
- Git tree: https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git
- Project: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/
- Git sha: a933d3dc1968fcfb0ab72879ec304b1971ed1b9a
- Git describe: 6.16.0-rc7-next-20250723
- kernel version: next-20250723
- Architectures: arm64
- Toolchains: gcc-13
- Kconfigs: defconfig + CONFIG_ARM64_64K_PAGES=y
- Kconfigs: defconfig + CONFIG_ARM64_16K_PAGES=y
## Test
- Test log 1: https://qa-reports.linaro.org/api/testruns/29227309/log_file/
- Test log 2: https://qa-reports.linaro.org/api/testruns/29227074/log_file/
- Test run: https://regressions.linaro.org/lkft/linux-next-master/next-20250723/testruns...
- Test history:
https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20250723/tes...
- Test plan: https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/30G3hpJVVdX...
- Build link: https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
- Kernel config:
https://storage.tuxsuite.com/public/linaro/lkft/builds/30G3dvSFyHHQ3E8CvKH7t...
-- Linaro LKFT https://lkft.linaro.org
On Wed, Jul 23, 2025 at 2:20 PM Darrick J. Wong djwong@kernel.org wrote:
On Wed, Jul 23, 2025 at 11:42:42AM -0700, Joanne Koong wrote:
On Wed, Jul 23, 2025 at 7:46 AM Darrick J. Wong djwong@kernel.org wrote:
[cc Joanne]
On Wed, Jul 23, 2025 at 05:14:28PM +0530, Naresh Kamboju wrote:
Regressions found while running LTP msync04 tests on qemu-arm64 running Linux next-20250721, next-20250722 and next-20250723 with 16K and 64K page size enabled builds.
CONFIG_ARM64_64K_PAGES=y ( kernel warning as below ) CONFIG_ARM64_16K_PAGES=y ( kernel warning as below )
No warning noticed with 4K page size. CONFIG_ARM64_4K_PAGES=y works as expected
You might want to cc Joanne since she's been working on large folio support in fuse.
First seen on the tag next-20250721. Good: next-20250718 Bad: next-20250721 to next-20250723
Thanks for the report. Is there a link to the script that mounts the fuse server for these tests? I'm curious whether this was mounted as a fuseblk filesystem.
Regression Analysis:
- New regression? Yes
- Reproducibility? Yes
Test regression: next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
## Test log ------------[ cut here ]------------ [ 343.828105] WARNING: fs/fuse/file.c:2146 at fuse_iomap_writeback_range+0x478/0x558 [fuse], CPU#0: msync04/4190
WARN_ON_ONCE(len & (PAGE_SIZE - 1));/me speculates that this might be triggered by an attempt to write back some 4k fsblock within a 16/64k base page?
I think this can happen on 4k base pages as well actually. On the iomap side, the length passed is always block-aligned and in fuse, we set blkbits to be PAGE_SHIFT so theoretically block-aligned is always page-aligned, but I missed that if it's a "fuseblk" filesystem, that isn't true and the blocksize is initialized to a default size of 512 or whatever block size is passed in when it's mounted.
<nod> I think you're correct.
I'll send out a patch to remove this line. It doesn't make any difference for fuse_iomap_writeback_range() logic whether len is page-aligned or not; I had added it as a sanity-check against sketchy ranges.
Also, I just noticed that apparently the blocksize can change dynamically for an inode in fuse through getattr replies from the server (see fuse_change_attributes_common()). This is a problem since the iomap uses inode->i_blkbits for reading/writing to the bitmap. I think we will have to cache the inode blkbits in the iomap_folio_state struct unfortunately :( I'll think about this some more and send out a patch for this.
From my understanding of the iomap code, it's possible to do that if you flush and unmap the entire pagecache (whilst holding i_rwsem and mmap_invalidate_lock) before you change i_blkbits. Nobody *does* this so I have no idea if it actually works, however. Note that even I don't implement the flush and unmap bit; I just scream loudly and do nothing:
lol! i wish I could scream loudly and do nothing too for my case.
AFAICT, I think I just need to flush and unmap that file and can leave the rest of the files/folios in the pagecache as is? But then if the file has active refcounts on it or has been pinned into memory, can I still unmap and remove it from the page cache? I see the invalidate_inode_pages2() function but my understanding is that the page still stays in the cache if it has has active references, and if the page gets mmaped and there's a page fault on it, it'll end up using the preexisting old page in the page cache.
I don't think I really need to have it removed from the page cache so much as just have the ifs state for all the folios in the file freed (after flushing the file) so that it can start over with a new ifs. Ideally we could just flush the file, then iterate through all the folios in the mapping in order of ascending index, and kfree their ->private, but I'm not seeing how we can prevent the case of new writes / a new ifs getting allocated for folios at previous indexes while we're trying to do the iteration/kfreeing.
void fuse_iomap_set_i_blkbits(struct inode *inode, u8 new_blkbits) { trace_fuse_iomap_set_i_blkbits(inode, new_blkbits);
if (inode->i_blkbits == new_blkbits) return; if (!S_ISREG(inode->i_mode)) goto set_it; /* * iomap attaches per-block state to each folio, so we cannot allow * the file block size to change if there's anything in the page cache. * In theory, fuse servers should never be doing this. */ if (inode->i_mapping->nrpages > 0) { WARN_ON(inode->i_blkbits != new_blkbits && inode->i_mapping->nrpages > 0); return; }set_it: inode->i_blkbits = new_blkbits; }
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/commit/...
--D
Thanks, Joanne
On Wed, Jul 23, 2025 at 11:42 AM Joanne Koong joannelkoong@gmail.com wrote:
On Wed, Jul 23, 2025 at 7:46 AM Darrick J. Wong djwong@kernel.org wrote:
[cc Joanne]
On Wed, Jul 23, 2025 at 05:14:28PM +0530, Naresh Kamboju wrote:
Regressions found while running LTP msync04 tests on qemu-arm64 running Linux next-20250721, next-20250722 and next-20250723 with 16K and 64K page size enabled builds.
CONFIG_ARM64_64K_PAGES=y ( kernel warning as below ) CONFIG_ARM64_16K_PAGES=y ( kernel warning as below )
No warning noticed with 4K page size. CONFIG_ARM64_4K_PAGES=y works as expected
You might want to cc Joanne since she's been working on large folio support in fuse.
First seen on the tag next-20250721. Good: next-20250718 Bad: next-20250721 to next-20250723
Thanks for the report. Is there a link to the script that mounts the fuse server for these tests? I'm curious whether this was mounted as a fuseblk filesystem.
Regression Analysis:
- New regression? Yes
- Reproducibility? Yes
Test regression: next-20250721 arm64 16K and 64K page size WARNING fs fuse file.c at fuse_iomap_writeback_range
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
## Test log ------------[ cut here ]------------ [ 343.828105] WARNING: fs/fuse/file.c:2146 at fuse_iomap_writeback_range+0x478/0x558 [fuse], CPU#0: msync04/4190
WARN_ON_ONCE(len & (PAGE_SIZE - 1));/me speculates that this might be triggered by an attempt to write back some 4k fsblock within a 16/64k base page?
I think this can happen on 4k base pages as well actually. On the iomap side, the length passed is always block-aligned and in fuse, we set blkbits to be PAGE_SHIFT so theoretically block-aligned is always page-aligned, but I missed that if it's a "fuseblk" filesystem, that isn't true and the blocksize is initialized to a default size of 512 or whatever block size is passed in when it's mounted.
I'll send out a patch to remove this line. It doesn't make any difference for fuse_iomap_writeback_range() logic whether len is page-aligned or not; I had added it as a sanity-check against sketchy ranges.
https://lore.kernel.org/linux-fsdevel/20250723230850.2395561-1-joannelkoong@... is the patch for removing this
-
Darrick J. Wong -
Joanne Koong -
Naresh Kamboju