Due to the lack of the SKIP directive in the output, if any of the parameterized test was skipped, the parser could not recognize that correctly and was marking the test as PASSED.
This can easily be seen by running the new subtest from patch 1:
$ ./tools/testing/kunit/kunit.py run \ --kunitconfig ./lib/kunit/.kunitconfig *.example_params*
[ ] Starting KUnit Kernel (1/1)... [ ] ============================================================ [ ] =================== example (1 subtest) ==================== [ ] =================== example_params_test =================== [ ] [PASSED] example value 2 [ ] [PASSED] example value 1 [ ] [PASSED] example value 0 [ ] =============== [PASSED] example_params_test =============== [ ] ===================== [PASSED] example ===================== [ ] ============================================================ [ ] Testing complete. Ran 3 tests: passed: 3
$ ./tools/testing/kunit/kunit.py run \ --kunitconfig ./lib/kunit/.kunitconfig *.example_params* \ --raw_output
[ ] Starting KUnit Kernel (1/1)... KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example 1..1 KTAP version 1 # Subtest: example_params_test # example_params_test: initializing ok 1 example value 2 # example_params_test: initializing ok 2 example value 1 # example_params_test: initializing ok 3 example value 0 # example_params_test: pass:2 fail:0 skip:1 total:3 ok 1 example_params_test # Totals: pass:2 fail:0 skip:1 total:3 ok 1 example
After adding the SKIP directive, the report looks as expected:
[ ] Starting KUnit Kernel (1/1)... [ ] ============================================================ [ ] =================== example (1 subtest) ==================== [ ] =================== example_params_test =================== [ ] [PASSED] example value 2 [ ] [PASSED] example value 1 [ ] [SKIPPED] example value 0 [ ] =============== [PASSED] example_params_test =============== [ ] ===================== [PASSED] example ===================== [ ] ============================================================ [ ] Testing complete. Ran 3 tests: passed: 2, skipped: 1
[ ] Starting KUnit Kernel (1/1)... KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example 1..1 KTAP version 1 # Subtest: example_params_test # example_params_test: initializing ok 1 example value 2 # example_params_test: initializing ok 2 example value 1 # example_params_test: initializing ok 3 example value 0 # SKIP unsupported param value # example_params_test: pass:2 fail:0 skip:1 total:3 ok 1 example_params_test # Totals: pass:2 fail:0 skip:1 total:3 ok 1 example
v2: better align with future support for arbitrary levels of testing v3: rebased on kunit tree [1]
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/lo...
Cc: David Gow davidgow@google.com Cc: Rae Moar rmoar@google.com
Michal Wajdeczko (3): kunit/test: Add example test showing parameterized testing kunit: Fix reporting of the skipped parameterized tests kunit: Update kunit_print_ok_not_ok function
include/kunit/test.h | 1 + lib/kunit/kunit-example-test.c | 34 +++++++++++++++++++++++++++ lib/kunit/test.c | 43 ++++++++++++++++++++++------------ 3 files changed, 63 insertions(+), 15 deletions(-)