Hi Shuah and Usama,
On 11/25/24 09:46, Muhammad Usama Anjum wrote:
Hi Laura,
Thank you for making change.
On 11/22/24 11:19 PM, Shuah wrote:
On 11/22/24 08:55, Laura Nao wrote:
Update the functions that print the test totals at the end of a selftest to include a warning message when skipped tests are detected. The message advises users that skipped tests may indicate missing configuration options and suggests enabling them to improve coverage.
Signed-off-by: Laura Nao laura.nao@collabora.com
This patch follows up on a previous discussion[1] and aims to highlight skipped tests for the user's attention.
[1] https://lore.kernel.org/lkml/2bb2d338-cd00-4ac2-b8bd-5579eae82637@linuxfound...
tools/testing/selftests/kselftest.h | 4 ++++ tools/testing/selftests/kselftest/ksft.py | 3 +++ tools/testing/selftests/kselftest/ktap_helpers.sh | 4 ++++ 3 files changed, 11 insertions(+)
diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 29fedf609611..d3f64b333acd 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -147,6 +147,10 @@ static inline void ksft_set_plan(unsigned int plan) static inline void ksft_print_cnts(void) {
- if (ksft_cnt.ksft_xskip > 0)
printf("# Skipped tests detected. Consider enabling relevant config options to improve coverage.\n"Looks good. Printing the number of skipped tests would be an improvement. I'm thinking about a case where some tests got failed and some skipped. Would this warning be useful in that case?
I believe the warning remains useful, as it helps users identify possible gaps in their configuration - I think that's valuable regardless of the results of other tests.
I like this. How about printing the number of skipped tests in this message also to make it easy to parse.
Same comment on other print messages,
Sure, that makes sense. I'll submit a v2 to include the number of skipped tests.
); if (ksft_plan != ksft_test_num()) printf("# Planned tests != run tests (%u != %u)\n", ksft_plan, ksft_test_num());diff --git a/tools/testing/selftests/kselftest/ksft.py b/tools/testing/selftests/kselftest/ksft.py index bf215790a89d..7675a15a1264 100644 --- a/tools/testing/selftests/kselftest/ksft.py +++ b/tools/testing/selftests/kselftest/ksft.py @@ -27,6 +27,9 @@ def set_plan(num_tests): def print_cnts():
- if ksft_cnt['skip'] > 0:
print("# Skipped tests detected. Consider enabling relevant config options to improve coverage.")print( f"# Totals: pass:{ksft_cnt['pass']} fail:{ksft_cnt['fail']} xfail:0 xpass:0 skip:{ksft_cnt['skip']} error:0" )diff --git a/tools/testing/selftests/kselftest/ktap_helpers.sh b/tools/testing/selftests/kselftest/ktap_helpers.sh index 79a125eb24c2..a4211221ccd6 100644 --- a/tools/testing/selftests/kselftest/ktap_helpers.sh +++ b/tools/testing/selftests/kselftest/ktap_helpers.sh @@ -107,5 +107,9 @@ ktap_finished() { } ktap_print_totals() {
- if [ "$KTAP_CNT_SKIP" -gt 0 ]; then
echo "# Skipped tests detected. " \"Consider enabling relevant config options to improve coverage."- fi echo "# Totals: pass:$KTAP_CNT_PASS fail:$KTAP_CNT_FAIL xfail:0 xpass:0 skip:$KTAP_CNT_SKIP error:0" }
thanks, -- Shuah
Thanks,
Laura