Re: [PATCH] selftests/cgroup: conform test to TAP format output

[Guopeng Zhang <zhangguopeng@kylinos.cn>]

On 11/14/25 15:04, Sebastian Chlad wrote:
> On Fri, Nov 14, 2025 at 4:59 AM Guopeng Zhang <zhangguopeng@kylinos.cn> wrote:
>>
>> Hi Michal,
>>
>> Thanks for reviewing and pointing out [1].
>>
>>> Could you please explain more why is the TAP layout beneficial?
>>> (I understand selftest are for oneself, i.e. human readable only by default.)
>>
>> Actually, selftests are no longer just something for developers to view locally; they are now extensively
>> run in CI and stable branch regression testing. Using a standardized layout means that general test runners
>> and CI systems can parse the cgroup test results without any special handling.
> 
> I second that.
> In fact, we do run some of those tests in the CI; i.e.
> https://openqa.opensuse.org/tests/5453031#external
> We added this: https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/Parser/Format/KTAP.pm
> to our CI
> but frankly the use of the KTAP across the selftests is very
> inconsistent, so we need to post-process some of the output files
> quite a lot.
> Therefore the more standardized the output, the better for any CI.
> 
Hi Sebastian,

Thanks a lot for the details and for sharing the openQA links and the KTAP parser.
That context is really helpful.
> Small ask: should we amend the commit message to say KTAP?
> 
Good point about the naming – I’ll respin this as v2 and amend the commit
message to refer to KTAP rather than just TAP.
> That being said - the cgroups tests produce nice output which is easy
> to parse and gives us no issues in our CI apart
> from the shell tests, specifically test_cpuset_prs.sh.
> 
> We currently run the cgroup tests only internally because some of them
> tend to fail when crossing resource-usage
> boundaries and don’t provide clear information about by how much.
> That ties into my earlier effort Michal linked here::
> https://lore.kernel.org/all/rua6ubri67gh3b7atarbm5mggqgjyh6646mzkry2n2547jne4s@wvvpr3esi5es/
> 
> I’ll try to add the cgroup tests to the public openSUSE CI and will
> test your patches.
> 
It’s also great to hear that the cgroup C tests already integrate reasonably
well with your CI. Once this change settles, I’d be happy to follow up with
patches to convert the cgroup shell tests, in particular test_cpuset_prs.sh,
to KTAP-style output as well.

Thanks again for trying the patches in the openSUSE CI and for the feedback.
>>
>> TAP provides a structured format that is both human-readable and machine-readable. The plan/result lines are parsed by tools,
>> while the diagnostic lines can still contain human-readable debug information. Over time, other selftest suites (such as mm, KVM, mptcp, etc.)
>> have also been converted to TAP-style output, so this change just brings the cgroup tests in line with that broader direction.
>>
>>> Or is this part of some tree-wide effort?
>>
>> This patch is not part of a formal, tree-wide conversion series I am running; it is an incremental step to align the
>> cgroup C tests with the existing TAP usage. I started here because these tests already use ksft_test_result_*() and
>> only require minor changes to generate proper TAP output.
>>
>>> I'm asking to better asses whether also the scripts listed in
>>> Makefile:TEST_PROGS should be converted too.
>>
>> I agree that having them produce TAP output would benefit tooling and CI. I did not want to mix
>> that into this change, but if you and other maintainers think this direction is reasonable,
>> I would be happy to follow up and convert the cgroup shell tests to TAP as well.
>>
>> Thanks again for your review.
>>
>> Best regards,
>> Guopeng
>>
>>