On Wed, Jul 16, 2025 at 11:36:02AM -0300, Gustavo Padovan wrote:
On Wed, 2025-07-16 at 13:58 +0100, Mark Brown wrote:
On Wed, Jul 16, 2025 at 12:46:28PM -0000, KernelCI bot wrote:
kselftest.seccomp.seccomp_seccomp_benchmark_per- filter_last_2_diff_per-filter_filters_4 running on bcm2837-rpi-3-b- plus
FWIW the seccomp benchmarks are very unstable on a fairly wide range of hardware. We probably need some filtering on the tests that get reported.
Indeed. However, for the previous 17 executions it passed 12 with 5 infra issues unrelated to the test. That's is why we sent this report.
Yeah, it does work a lot of the time but it fails often enough for me to have excluded it from triggering bisects in my own CI. It is more unstable on some other platforms that this one though.
But to your point, we really need clear understanding of patterns to flag something as regression vs it being an unstable test.
Yup.