On Fri, Nov 14, 2025 at 4:59 AM Guopeng Zhang zhangguopeng@kylinos.cn wrote:
Hi Michal,
Thanks for reviewing and pointing out [1].
Could you please explain more why is the TAP layout beneficial? (I understand selftest are for oneself, i.e. human readable only by default.)
Actually, selftests are no longer just something for developers to view locally; they are now extensively run in CI and stable branch regression testing. Using a standardized layout means that general test runners and CI systems can parse the cgroup test results without any special handling.
I second that. In fact, we do run some of those tests in the CI; i.e. https://openqa.opensuse.org/tests/5453031#external We added this: https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/Parser/Format/K... to our CI but frankly the use of the KTAP across the selftests is very inconsistent, so we need to post-process some of the output files quite a lot. Therefore the more standardized the output, the better for any CI.
Small ask: should we amend the commit message to say KTAP?
That being said - the cgroups tests produce nice output which is easy to parse and gives us no issues in our CI apart from the shell tests, specifically test_cpuset_prs.sh.
We currently run the cgroup tests only internally because some of them tend to fail when crossing resource-usage boundaries and don’t provide clear information about by how much. That ties into my earlier effort Michal linked here:: https://lore.kernel.org/all/rua6ubri67gh3b7atarbm5mggqgjyh6646mzkry2n2547jne...
I’ll try to add the cgroup tests to the public openSUSE CI and will test your patches.
TAP provides a structured format that is both human-readable and machine-readable. The plan/result lines are parsed by tools, while the diagnostic lines can still contain human-readable debug information. Over time, other selftest suites (such as mm, KVM, mptcp, etc.) have also been converted to TAP-style output, so this change just brings the cgroup tests in line with that broader direction.
Or is this part of some tree-wide effort?
This patch is not part of a formal, tree-wide conversion series I am running; it is an incremental step to align the cgroup C tests with the existing TAP usage. I started here because these tests already use ksft_test_result_*() and only require minor changes to generate proper TAP output.
I'm asking to better asses whether also the scripts listed in Makefile:TEST_PROGS should be converted too.
I agree that having them produce TAP output would benefit tooling and CI. I did not want to mix that into this change, but if you and other maintainers think this direction is reasonable, I would be happy to follow up and convert the cgroup shell tests to TAP as well.
Thanks again for your review.
Best regards, Guopeng
On 11/14/25 15:04, Sebastian Chlad wrote:
On Fri, Nov 14, 2025 at 4:59 AM Guopeng Zhang zhangguopeng@kylinos.cn wrote:
Hi Michal,
Thanks for reviewing and pointing out [1].
Could you please explain more why is the TAP layout beneficial? (I understand selftest are for oneself, i.e. human readable only by default.)
Actually, selftests are no longer just something for developers to view locally; they are now extensively run in CI and stable branch regression testing. Using a standardized layout means that general test runners and CI systems can parse the cgroup test results without any special handling.
I second that. In fact, we do run some of those tests in the CI; i.e. https://openqa.opensuse.org/tests/5453031#external We added this: https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/Parser/Format/K... to our CI but frankly the use of the KTAP across the selftests is very inconsistent, so we need to post-process some of the output files quite a lot. Therefore the more standardized the output, the better for any CI.
Hi Sebastian,
Thanks a lot for the details and for sharing the openQA links and the KTAP parser. That context is really helpful.
Small ask: should we amend the commit message to say KTAP?
Good point about the naming – I’ll respin this as v2 and amend the commit message to refer to KTAP rather than just TAP.
That being said - the cgroups tests produce nice output which is easy to parse and gives us no issues in our CI apart from the shell tests, specifically test_cpuset_prs.sh.
We currently run the cgroup tests only internally because some of them tend to fail when crossing resource-usage boundaries and don’t provide clear information about by how much. That ties into my earlier effort Michal linked here:: https://lore.kernel.org/all/rua6ubri67gh3b7atarbm5mggqgjyh6646mzkry2n2547jne...
I’ll try to add the cgroup tests to the public openSUSE CI and will test your patches.
It’s also great to hear that the cgroup C tests already integrate reasonably well with your CI. Once this change settles, I’d be happy to follow up with patches to convert the cgroup shell tests, in particular test_cpuset_prs.sh, to KTAP-style output as well.
Thanks again for trying the patches in the openSUSE CI and for the feedback.
TAP provides a structured format that is both human-readable and machine-readable. The plan/result lines are parsed by tools, while the diagnostic lines can still contain human-readable debug information. Over time, other selftest suites (such as mm, KVM, mptcp, etc.) have also been converted to TAP-style output, so this change just brings the cgroup tests in line with that broader direction.
Or is this part of some tree-wide effort?
This patch is not part of a formal, tree-wide conversion series I am running; it is an incremental step to align the cgroup C tests with the existing TAP usage. I started here because these tests already use ksft_test_result_*() and only require minor changes to generate proper TAP output.
I'm asking to better asses whether also the scripts listed in Makefile:TEST_PROGS should be converted too.
I agree that having them produce TAP output would benefit tooling and CI. I did not want to mix that into this change, but if you and other maintainers think this direction is reasonable, I would be happy to follow up and convert the cgroup shell tests to TAP as well.
Thanks again for your review.
Best regards, Guopeng
linux-kselftest-mirror@lists.linaro.org
-
Guopeng Zhang -
Sebastian Chlad