On Wed, May 14, 2025 at 09:17:45PM -0700, Pawan Gupta wrote:
On Wed, May 14, 2025 at 07:49:29PM +0800, Shung-Hsi Yu wrote:
On Mon, May 12, 2025 at 07:37:30PM +0200, Greg Kroah-Hartman wrote:
This is the start of the stable review cycle for the 6.14.7 release. There are 197 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Running included BPF selftests with a BPF CI fork (i.e. running on GitHub Action x86-64 machines), I observe that that running the BPF selftests now takes about 2x the time (from ~25m to ~50m), and verif_scale_loop3_fail is timing out, taking more than 6 minutes to run compare to the usual single digit second runtime. See [1] for the log.
...
Compare to a day before when such behavior wasn't observed[2], the main difference being these additional patches:
...
Not sure why but this commit seems to related to the failure.
Below is log of bisecting v6.14.6 and v6.14.7-rc2 with the test:
./tools/testing/selftests/bpf/vmtest.sh -i -- timeout 20 ./test_progs -t verif_scale_loop3_fail
# good: [e2d3e1fdb530198317501eb7ded4f3a5fb6c881c] Linux 6.14.6 git bisect good e2d3e1fdb530198317501eb7ded4f3a5fb6c881c
...
git bisect bad 336f780075f36e0d1181ce44d6d4197e4a22babc # bad: [665f26e5de2325e3bca107b632bc2ccac1b9806a] mm: vmalloc: support more granular vrealloc() sizing git bisect bad 665f26e5de2325e3bca107b632bc2ccac1b9806a # first bad commit: [665f26e5de2325e3bca107b632bc2ccac1b9806a] mm: vmalloc: support more granular vrealloc() sizing
Thanks! Just dawn on me after seeing this that I should try 6.15-rc6 as well (which has 665f26e5de23), turns out it also reproduce there. I'll report regression in a separate mail.
...
No patches touch BPF's core component, and while the verif_scale_loop3_fail test did time out, the verifier is still correctly rejecting it, so shouldn't have anything to do with kernel/bpf/. The x86/arm64 BPF patches only affect JIT output, and only for cBPF.
In comparison, with 6.12.29-rc1 I don't observe any timeout or increase in runtime[3]. Below is a diff comparing the applied patches in 6.12.29-rc1 and 6.14.7-rc1. Seems like 6.14.7-rc1 does not have the CALL_NOSPEC patches, but I cannot tell whether that is what makes the difference.
Thats because CALL_NOSPEC patches were already part of v6.14.
Ah yes indeed, sorry about the negligence.