On Mon, Jun 02, 2025 at 08:14:08PM +0200, Geert Uytterhoeven wrote:
Hi Kees,
On Mon, 2 Jun 2025 at 20:08, Kees Cook kees@kernel.org wrote:
On Mon, Jun 02, 2025 at 04:49:51PM +0200, Geert Uytterhoeven wrote:
When CONFIG_RANDSTRUCT is not enabled, all randstruct tests are skipped. Move this logic from run-time to config-time, to avoid people building and running tests that do not do anything.
I don't like doing this because it means that looking at CI output means I can't tell if the test was not built or if the config was not included. I want to always have the test available, but skip the test if the config is missing.
So should we drop all dependencies from tests? Do you want Zorro bus, NuBus, ... tests (assumed we have them) to be built on all platforms, and "run" on all CIs?
I can't speak for those authors, but I think they are pretty different from treewide instrumentation tests. As a consumer of CI output, I would like to be able to always see this kind of test output. Sometimes it's Kconfig settings (like here) and sometimes it's architectural capabilities (as seen in LKDTM tests). If the test is always run and explains why it has been skipped if it is skipped, I always get actionable details without needing to do a round-trip with the CI runner, spend time locating .config files, etc, etc.