Re: Converting kselftest test modules to kunit

[Muhammad Usama Anjum <usama.anjum@collabora.com>]

On 7/27/24 12:35 AM, Shuah Khan wrote:
> On 7/15/24 04:09, Muhammad Usama Anjum wrote:
>> Hi Kees and All,
>>
>> There are several tests in kselftest subsystem which load modules to tests
>> the internals of the kernel. Most of these test modules are just loaded by
>> the kselftest, their status isn't read and reported to the user logs. Hence
>> they don't provide benefit of executing those tests.
>>
>> I've found patches from Kees where he has been converting such kselftests
>> to kunit tests [1]. The probable motivation is to move tests output of
>> kselftest subsystem which only triggers tests without correctly reporting
>> the results. On the other hand, kunit is there to test the kernel's
>> internal functions which can't be done by userspace.
>>
>> Kselftest:    Test user facing APIs from userspace
>> Kunit:        Test kernel's internal functions from kernelspace
>>
>> This brings me to conclusion that kselftest which are loading modules to
>> test kernelspace should be converted to kunit tests. I've noted several
>> such kselftests.
>>
>> This is just my understanding. Please mention if I'm correct above or more
>> reasons to support kselftest test modules transformation into kunit test.
>>
>> [1] https://lore.kernel.org/all/20221018082824.never.845-kees@kernel.org/
>>
> 
> Please make sure you aren't taking away the ability to run these tests during
> boot. 
The kunit tests are usually run at boot time. They can be run later as
well. So I'm not trying to remove any functionality. Rather adding a way
where failures would actually be detected.

> It doesn't make sense to convert every single test especially when it
> is intended to be run during boot without dependencies - not as a kunit test
> but a regression test during boot.
I started investigating when these lib kselftests were just loading the
test module without checking if test actually passed/failed (which proves
that this type of kselftests can never detect regression as it doesn't
process the results). It was strange. Hence I found out the conversion of
such kselftests to kunit tests done by Kees and started this thread for
discussion before doing actual work and sending patches.

> 
> bitmap is one example - pay attention to the config help test - bitmap
> one clearly states it runs regression testing during boot. Any test that
> says that isn't a candidate for conversion.
Please can you explain what do you mean by bitmap testing during boot? As
far as I understand, the kselftests are usespace tests and are run after
kernel has booted completely and userspace has started. It cannot be
regarded as testing during boot.

> 
> I am going to nack any such conversions.
> 
> thanks,
> -- Shuah
> 

-- 
BR,
Muhammad Usama Anjum