On 6/10/20 11:43 AM, Bird, Tim wrote:
-----Original Message----- From: linux-kselftest-owner@vger.kernel.org linux-kselftest-owner@vger.kernel.org On Behalf Of Shuah Khan
On 6/10/20 9:44 AM, Paolo Bonzini wrote:
According to the TAP specification, a skipped test must be marked as "ok" and annotated with the SKIP directive, for example
ok 23 # skip Insufficient flogiston pressure. (https://testanything.org/tap-specification.html)Fix the runner script to match this.
Signed-off-by: Paolo Bonzini pbonzini@redhat.com
tools/testing/selftests/kselftest/runner.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index 676b3a8b114d..f4815cbcd60f 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -77,7 +77,7 @@ run_one() echo "ok $test_num $TEST_HDR_MSG") || (rc=$?; \ if [ $rc -eq $skip_rc ]; then \
echo "not ok $test_num $TEST_HDR_MSG # SKIP"
echo "ok $test_num $TEST_HDR_MSG # SKIP"This is a pretty big change, and might break upstream CIs that have come to rely on kselftest's existing behavior. I know it's going to break Fuego's parsing of results.
Thanks for chiming in. We don't want to break CI workflow.
kselftest has a few conventions that are different from the TAP spec, and a few items it does that are extensions to the TAP spec. IMHO, the TAP spec got this one wrong, but I could be convinced otherwise. But I think we should discuss this among CI users of kselftest before making the change.
I started work quite a while ago on an effort to document the conventions used by kselftest (particularly where it deviates from the TAP spec), but never submitted it.
I'm going to submit what I've got as an RFC now, for discussion, even though it's not finished. I'll do that in a separate thread.
elif [ $rc -eq $timeout_rc ]; then \ echo "#" echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT"Thanks. I will pull this in for Linux 5.8-rc2
Shuah - can you hold off on this until we discuss it?
Of course. Thanks for getting my attention before I pulled it in.
thanks, -- Shuah