Re: About new backend for GPU compute ROCm in qemu

["Huang, Honglei" <honghuan@amd.com>]

On 8/17/2026 5:06 PM, Alex Bennée wrote:
> "Huang, Honglei" <honghuan@amd.com> writes:
> 
>> Hi Michael, Alex, Dmitry, Akihiko,
>>
>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
>> over virtio
>> implementation to virglrenderer nine months ago (MR !1568 [1]). The
>> ROCm side has
>> been supportted by ROCm offical.
>>
>> Current implementation is a virtio gpu context type capset handled inside
>> virglrenderer, sharing the display path. That's an awkward fit, many
>> compute GPUs have no display engine at all.
>>
>> Beyond that, sharing the display path is increasingly painful:
>>
>>    - Compute hammers the queues more than graphics, so sharing
>>      virtio gpu's single control queue with display/virgl causes contention
>>      and display stutter.
> 
> Is this just due to iteration? From a layman's point of view I'm curious
> as to what the queues are doing.

Really thanks for the reply.

The load is mostly memory management. Running an AI model allocates and 
frees a large number of blobs. We already did some optimization release 
them asynchronously, but the host processing is a single queue one
process_cmdq, this is where the main bottleneck in my debugging work / 
my understanding so far. I'm not certain it's the whole picture, so 
please correct if I am wrong.

A model load or unload frees a large batch of BOs and allocates another. 
Some of those commands are async in the virtio-gpu guest driver, but 
QEMU still has to work through them on the one queue, which takes time; 
so even though any single command is quick, there are simply too many of 
them, the single queue backs up, and everything behind it, gets delayed.

real work load (a few downstream customisations): loading one 16 GB 
model (gemm4 e4b), drives ~1200 blob creates, a burst of
~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
notifies, caused a 22 s guest soft lockup. And the behavior of memory 
operations are controlled by upper layer like pytorch / HIP / runtime,
we can not control it.

To be honest, a separate backend won't fix this. But the real solution 
maybe is compute specific. That logic is only useful to the compute 
path, and folding it into the shared display device / renderer would 
mean churning code that is mature and stable for graphics, with 
regression risk. Keeping compute on its own instance and backend lets us 
iterate on these compute only optimisations.

> 
>>    - Compute contexts need far more blob / shared memory than a display
>>    one.
> 
> I guess weights and context are long lived blobs compared to rendering assets?

Exactly, weights, KV caches, scratch heaps and the userptr/SVM arenas 
are large and long-lived, unlike per frame rendering assets, so an AI 
context maps far more memory than a display one.

> 
>>    - Maybe needs a wider ROCm / compute stack, cause the render model
>>      fits poorly:
>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth streams)
>>      and ROCgdb (wave control, address watch, async exceptions an
>>      out of band channel that must not block display).
>>    - Events, faults and GPU reset/SMI are async and don't map onto fences.
>>    - All of this is hard to extend cleanly inside a display capset.
>>
>> On the QEMU/host side, would something like this be OK? One step, two parts:
>>
>>    - a dedicated headless virtio gpu instance for compute.
> 
> An oft-asked for VirtIO model is virtio-npu but I wonder if there is
> enough commonality for a virtio-compute device with the same sort native
> context type handling to deal with the those that want to have guests
> targeting specific hardware rather than going through an abstraction
> like Vulkan Computer or OpenGL CL.

Agreed there's likely enough commonality. The idea would be for the 
device to provide mechanism, not hardware policy. And actually the 
current ROCm implementation is doing like this, it can also support 
OPENCL and HIP.

> 
>>    - that instance served by a separate ROCm backend library loaded
>>      in-process by QEMU.
> 
> Is this library a binary blob or open source? The last time I looked at
> ROCm I had to give up as my AMD card was in an Aarch64 AVA machine.

It is open source, we are planing put it into ROCm stack, let it be a 
part of ROCm.

Regards,
Honglei

> 
>> That reuses the existing pluggable backend model, a second virtio gpu + a
>> backend library. It doesn't add dedicated queues for debug/profiling
>> currently.
>>
>> Waiting for reply and  happy to share more detail. Thanks!
>>
>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1568
>>
>> Regards,
>> Honglei
>