Re: [LSF/MM/BPF ATTEND][LSF/MM/BPF TOPIC] : blktests: status, expansion plan for the storage stack test framework

[Nilay Shroff <nilay@linux.ibm.com>]

On 2/13/26 4:53 PM, Shinichiro Kawasaki wrote:
> On Feb 12, 2026 / 08:52, Daniel Wagner wrote:
>> On Wed, Feb 11, 2026 at 08:35:30PM +0000, Chaitanya Kulkarni wrote:
>>>    For the storage track at LSFMMBPF2026, I propose a session dedicated to
>>>    blktests to discuss expansion plan and CI integration progress.
>>
>> Thanks for proposing this topic.
> 
> Chaitanya, my thank also goes to you.
> 
Yes thanks for proposing this!

>> Just a few random topics which come to mind we could discuss:
>>
>> - blktests has gain a bit of traction and some folks run on regular
>>   basis these tests. Can we gather feedback from them, what is working
>>   good, what is not? Are there feature wishes?
> 
> Good topic, I also would like to hear about it.
> 
One improvement I’d like to highlight is related to how blktests are executed
today. So far, we’ve been running blktests serially, but if it's possible to 
run tests in parallel to improve test turnaround time and make large-scale or
CI-based testing more efficient? For instance, adding parallel_safe Tags: Marking tests
that don't modify global kernel state so they can be safely offloaded to parallel
workers. Marking parallel_safe tags would allow the runner to distinguish:

Safe Tests: Tests that only perform I/O on a specific, non-shared device or 
check static kernel parameters.

Unsafe Tests: Tests that reload kernel modules, modify global /sys or /proc entries,
or require exclusive access to specific hardware addresses.

Yes adding parallel execution support shall require framework/design changes. 

> FYI, from the past LSFMM sessions and hallway talks, major feedbacks I had
> received are these two:
> 
>  1. blktests CI infra looks missing (other than CKI by Redhat)
>     -> Some activities are ongoing to start blktests CI service.
>        I hope the status are shared at the session.
> 
>  2. blktests are rather difficult to start using for some new users
>     -> I think config example is demanded, so that new users can
>        just copy it to start the first run, and understand the
>        config options easily.
> 
>> - Do we need some sort of configuration tool which allows to setup a
>>   config? I'd still have a TODO to provide a config example with all
>>   knobs which influence blktests, but I wonder if we should go a step
>>   further here, e.g. something like kdevops has?
> 
> Do you mean the "make menuconfig" style? Most of the blktests users are
> familiar with menuconfig, so that would be an idea. If users really want
> it, we can think of it. IMO, blktests still do not have so many options,
> then config.example would be simpler and more appropriate, probably.
> 
>> - Which area do we lack tests? Should we just add an initial simple
>>   tests for the missing areas, so the basic infra is there and thus
>>   lowering the bar for adding new tests?
> 
> To identify the uncovered area, I think code coverage will be useful. A few
> years ago, I measured it and shared in LSFMM, but that measurement was done for
> each source tree directory. The coverage ratio by source file will be more
> helpful to identify the missing area. I don't have time slot to measure it,
> so if anyone can do it and share the result, it will be appreciated. Once we
> know the missing areas, it sounds a good idea to add initial samples for each
> of the areas.
> 
>> - The recent addition of kmemleak shows it's a great idea to enable more
>>   of the kernel test infrastructure when running the tests.
> 
> Completely agreed.
> 
>>   Are there more such things we could/should enable?
> 
> I'm also interested in this question :)
> 
>> - I would like to hear from Shin'ichiro if he is happy how things
>>   are going? :)
> 
> More importantly, I would like to listen to voices from storage sub-system
> developers to see if they are happy or not, especially the maintainers.
> 
> From my view, blktests keep on finding kernel bugs. I think it demonstrates the
> value of this community effort, and I'm happy about it. Said that, I find what
> blktests can improve more, of course. Here I share the list of improvement
> opportunities from my view point (I already mentioned the first three items).
> 
>  1. We can have more CI infra to make the most of blktests
>  2. We can add config examples to help new users
>  3. We can measure code coverage to identify missing test areas
>  4. Long standing failures make test result reports dirty
>     - I feel lockdep WARNs are tend to be left unfixed rather long period.
>       How can we gather effort to fix them?

I agree regarding lockdep; recently we did see quite a few lockdep splats.
That said, I believe the number has dropped significantly and only a small
set remains. From what I can tell, most of the outstanding lockdep issues
are related to fs-reclaim paths recursing into the block layer while the
queue is frozen. We should be able to resolve most of these soon, or at
least before the conference. If anything is still outstanding after that,
we can discuss it during the conference and work toward addressing it as
quickly as possible.

>  5. We can refactor and clean up blktests framework for ease of maintainance
>       (e.g. trap handling)
>  6. Some users run blktests with built-in kernel modules, which makes a number
>     of test cases skipped. We can add more built-in kernel modules support to
>     expand test coverage for such use case.

Thanks,
--Nilay