Hi Alexei,
On Wed Oct 22, 2025 at 6:44 PM CEST, Alexei Starovoitov wrote:
On Wed, Oct 22, 2025 at 12:52 AM Alexis Lothoré alexis.lothore@bootlin.com wrote:
[...]
A note about test duration: the overall test duration, in my setup (x86 qemu-based setup, running on x86), is around 13s. Reviews on similar series ([1]) shows that such a duration is not really desirable for CI integration. I checked how to reduce it, and it appears that most of it is due to the fact that for each subtest, we verify that if we insert bpf encapsulation (egress) program, and nothing on server side, we properly fail to connect client to server. This test then relies on timeout connection, and I already reduced it as much as possible, but I guess going below the current value (500ms) will just start to make the whole test flaky.
I took this "check connection failure" from the original script, and kind of like it for its capacity to detect false negatives, but should I eventually get rid of it ?
I vote to get rid of it. I'd rather have test_progs that are quick enough to execute for CI and for all developers then more in depth coverage for the corner case.
ACK. I' ll get rid of it. For the record, I drop down to ~3s in my testing setup instead of ~13s when removing this "ensure connection failure test".
Note that for the verifier range test we randomize the test coverage, since the whole permutation takes hours to run. Instead we randomly pick a couple tests and run only those. Since CI runs for every patch the overall coverage is good enough. Would something like that possible here ? and in the other xsk test?
I see that test_verifier takes some "to" and "from" indexes, selecting the range of tests that we are able to run. Is this the mechanism you are referring to ? (and if so, I guess the rand part is handled by the CI runner ?)
If we want it for some specific test_progs tests like test_tc_tunnel and test_xsk, I guess it is doable, but we need to think about who controls the randomization, and how to still force execution of specific (or whole range of) subtests, when needing to reproduce an issue.
Alexis