Hi Nikolai,
On mié, nov 15 2023 at 19:43:49, Nikolai Kondrashov Nikolai.Kondrashov@redhat.com wrote:
Introduce a new tag, 'Tested-with:', documented in the Documentation/process/submitting-patches.rst file. The tag is expected to reference the documented test suites, similarly to the 'V:' field, and to certify that the submitter executed the test suite on the change, and that it passed.
I think the 'V:' field in MAINTAINERS is a good addition to document what developers are supposed to test for every subsystem, but in the case of the per-commit "Tested-with:" tag, I think the real value of it would be in using it for accountability and traceability purposes instead, that is, to link to the actual results of the (automatic) tests that were used to validate a commit.
This would provide two important features:
1. Rather than trusting that the tester did things right and that the test environment used was appropriate, we'd now have proof that the test results are as expected and a way to reproduce the steps.
2. A history of test results for future reference. When a regression is introduced, now we'd have more information about how things worked back when the test was still passing.
This is not trivial because tests vary a lot and we'd first need to define which artifacts to link to, and because whatever is linked (test commands, output log, results summary) would need to be stored forever. But since we're doing that already for basically all kernel mailing lists, I wonder if something like "public-inbox for test results" could be possible some day.
Cheers, Ricardo