Hello everyone,
This is an RFC patch series to propose the addition of a test attributes framework to KUnit.
There has been interest in filtering out "slow" KUnit tests. Most notably, a new config, CONFIG_MEMCPY_SLOW_KUNIT_TEST, has been added to exclude a particularly slow memcpy test (https://lore.kernel.org/all/20230118200653.give.574-kees@kernel.org/).
This proposed attributes framework would be used to save and access test associated data, including whether a test is slow. These attributes would be reportable (via KTAP and command line output) and some will be filterable.
This framework is designed to allow for the addition of other attributes in the future. These attributes could include whether the test is flaky, associated test files, etc.
This is the second version of the RFC I have added a few big changes: - Change method for inputting filters to allow for spaces in filtering values - Add option to skip filtered tests instead of not run or show them with the --filter_skip flag - Separate the new feature to list tests and their attributes into both --list_tests (lists just tests) and --list_tests_attr (lists all) - Add new attribute to store module name associated with test - Add Tests to executor_test.c - Add Documentation - A few small changes to code commented on previously
I would love to hear about the new features. If the series seems overall good I will send out the next version as an official patch series.
Thanks! Rae
Rae Moar (9): kunit: Add test attributes API structure kunit: Add speed attribute kunit: Add module attribute kunit: Add ability to filter attributes kunit: tool: Add command line interface to filter and report attributes kunit: memcpy: Mark tests as slow using test attributes kunit: time: Mark test as slow using test attributes kunit: add tests for filtering attributes kunit: Add documentation of KUnit test attributes
.../dev-tools/kunit/running_tips.rst | 163 +++++++ include/kunit/attributes.h | 50 +++ include/kunit/test.h | 68 ++- kernel/time/time_test.c | 2 +- lib/Kconfig.debug | 3 + lib/kunit/Makefile | 3 +- lib/kunit/attributes.c | 406 ++++++++++++++++++ lib/kunit/executor.c | 115 ++++- lib/kunit/executor_test.c | 119 ++++- lib/kunit/kunit-example-test.c | 9 + lib/kunit/test.c | 27 +- lib/memcpy_kunit.c | 8 +- tools/testing/kunit/kunit.py | 80 +++- tools/testing/kunit/kunit_kernel.py | 6 +- tools/testing/kunit/kunit_tool_test.py | 39 +- 15 files changed, 1022 insertions(+), 76 deletions(-) create mode 100644 include/kunit/attributes.h create mode 100644 lib/kunit/attributes.c
base-commit: 2e66833579ed759d7b7da1a8f07eb727ec6e80db