Re: [RFC 0/8] DRM scheduler kthread_worker for submission latency improvements

[Tvrtko Ursulin <tvrtko.ursulin@igalia.com>]

On 09/07/2026 09:28, Tvrtko Ursulin wrote:
> 
> On 08/07/2026 17:24, Tvrtko Ursulin wrote:
> 
> 8<>
> 
>>> How big a hit is WQ_HIGHPRI vs. RT?
>>
>> I can measure that tomorrow or so. But so far I am not very optimistic 
>> about MIN_NICE ability to make a difference.
>>
>> Today I repeated the benchmarking Chia-I did originally, with the 
>> tweak of background load threads always being of normal priority and 
>> only varying the userspace Vulkan submit thread between normal, nice 
>> -1 and FIFO. Those three against three kernels - stock, this RFC 
>> capped to never go above MIN_NICE for the kthread_worker, and this RFC 
>> as is (so kthread_worker temporarily follows userspace FIFO).
>>
>> The results are that only following to FIFO brings interesting gains.
>>
>>          STOCK           --RFC         Full RFC
>>       .    N   RT    .     N    RT    .     N  RT
>>     M    28   32      35   36    26    23   19    19  17
>>   95%   211  537      69  276   186   426  151   284  30
>>   98%   843  993     804  942  1277  1842  982  1028  34
>>
>>       STOCK = 7.1.0
>>       --RFC = This RFC, but kthread_worker capped and MIN_NICE
>>    Full RFC = RFC as is, kthread_worker goes up to FIFO
>>
>>   M = Median
>>
>>    . = Userspace submit thread SCHED_OTHER
>>    N = -||= nice -1
>>   RT = -||- FIFO 1
>>
>> All numbers are latencies between job passed to DRM scheduler to DRM 
>> scheduler passing it to the GPU. In micro seconds.
>>
>> I'll report back with WQ_HIGHPRI results as soon as possible.
> 
> I made panthor set WQ_HIGHPRI on stock 7.1 and tested normal, nice and 
> FIFO vulkan submit thread. As expected I do not see a clear improvement:
> 
>          .    N    RT
>     M   27   28    32
>   95%  163  246   809
>   98%  924  991  1882
> 
> This makes it seem that the 95-th percentile is perhaps better for 
> normal and re-niced userspace thread cases, but it may also be noise and 
> I would need to do many more and longer runs on each config to be sure. 
> 98-th percentile spikes are still there. Also, even if 95% would be a 
> real improvement they are still an order of magnitude above median so I 
> do not think it is worth spending much more time on benchmarking that 
> option.

To add another data point, today I did a quick WQ_RTPRI experiment.

That allows eliminating both the kthread_worker pool complications and 
priority inheritance from this series by allowing creation of SCHED_FIFO 
workqueues. It reduces the series to just two patches. One to add 
WQ_RTPRI and second to make panthor use it for realtime queues.

In the core code I capped max_active for those to num_online_cpus() / 2, 
although I am not sure Tejun if that is what you had in mind when you 
said it might be acceptable if the number of workers would be bound? I 
am also not sure if this will limit the system number or just for 
individual worker. If only the latter than further improvements will 
obviously be required.

On top of that, for the panthor use case we agreed that two threads is 
good enough so that is what the driver requests as the upper bound.

Overall flow is that when userspace requests 
VK_QUEUE_GLOBAL_PRIORITY_REALTIME (for which CAP_SYS_NICE is required 
btw), that queue goes to the DRM scheduler panthor created a WQ_RTPRI 
workqueue for. Lower priority Vulkan queues go to lower priority 
workqueues so there is no danger of normal priority clients over-using 
the RT queues.

Submit latency wise this is still completely effective. Median latency 
is 14us and 95 and 98-th percentiles are 23us and 25us respectively.

So the question is whether something like this is acceptable to 
workqueue core? If so, I can send it as a proper RFC and go from there.

Regards,

Tvrtko

P.S.
My WIP branch implementing this:
https://cgit.freedesktop.org/~tursulin/drm-intel/log/?h=wqrt-panthor