On 3/20/23 14:21, Daniel Latypov wrote:
On Thu, Mar 16, 2023 at 3:59 PM Rae Moar rmoar@google.com wrote:
Add recognition of the test name line ("# Subtest: <name>") to the KTAP v2 spec.
The purpose of this line is to declare the name of a test before its results. This functionality is especially useful when trying to parse test results incrementally and when interpretting results after a crash.
This line is already compliant with KTAP v1 as it is interpretted as a
minor nit for if there's a v2, s/interprett/interpret (here and above)
Also, I want to elaborate on the previous paragraph a bit more, in case the motivation isn't clear. The problem with TAP and KTAP as-is is that the name of a test case is only known *after* it completes.
So the scenario being referred to is
KTAP version 1 1..1
<lots of output> <kernel crash, no more output>
It would be nice if parsers could report "test FOO caused a crash" as opposed to "the first test case caused a crash, good luck figuring out which one that was"
Daniel
It would be useful to take some of the motivation explanation from TAP14, which says (everything to the end of this email):
Commented Subtests are encouraged, as they provide the following benefits:
Easier for humans to read. For example:
TAP version 14 1..1 ok 1 - hmm, what level is this? vs:
TAP version 14 # Subtest: level 1 # Subtest: level 2 # Subtest: level 3 1..1 ok 1 - clearly level 3
Additional strictness around matching the Test Point description to Subtest Name can catch errors and detect accidentally interleaved output.