Re: About new backend for GPU compute ROCm in qemu

["Huang, Honglei" <honghuan@amd.com>]

On 8/17/2026 10:24 PM, Alex Bennée wrote:
> "Huang, Honglei" <honghuan@amd.com> writes:
> 
>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>
>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>> Hi Honglei,
>>>
>>>>
>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
>>>> over virtio
>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). The
>>>> ROCm side has
>>>> been supportted by ROCm offical.
>>>>
>>>> Current implementation is a virtio gpu context type capset handled inside
>>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>>> compute GPUs have no display engine at all.
> <snip>
>>>>     - that instance served by a separate ROCm backend library loaded
>>>>       in-process by QEMU.
>>> First, I think we need to establish why ROCm cannot or should not
>>> remain
>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>> likely better placed to advise on that boundary. Once the protocol
>>> requirements and performance measurements are clear, we can assess the
>>> appropriate QEMU integration.
>>
>> venus is borned for GFX.
>> virCL not merged.
>>
>> To be clear, I'm not saying virglrenderer can't host a ROCm native
>> context it clearly can. My hesitation is more about fit and direction:
>> virglrenderer has grown up around GL/graphics, and I haven't yet found
>> compute oriented plumbing there to build on, while ROCm moves very
>> fast and I need something I can keep current with low friction.
> 
> I'm unsure what the current development status of virglrenderer is but
> it does see a continuing stream of merges. However the threading model
> does make things tricky for QEMU when we are sharing lifetime of blobs
> between QEMU proper and the virglrenderer thread. Perhaps there is a
> better way to organise things?

Yes, agreed. ROCm has no render thread of its own, it uses user-mode 
queues So, I think there's a better organisation.

> 
> Could we do the marshalling of VirtIO GPU commands into ROCm directly
> inside QEMU rather than going through additional plumbing? 

Yes, by "additional plumbing" I take you to mean routing through a
virglrenderer style renderer ( extra thread and blob coordination).
Yes, I'd drop that. What I'd keep is only a thin in QEMU backend the 
9pfs/cryptodev kind of pluggable backend, not a renderer layer which is 
what lets QEMU own the blob lifetime directly.


> Are the sequences we need to handle more or less complex than your general gfx
> rendering? How might this work with other frameworks?

Simpler, I think. ROCm has no render thread. It uses HSA user-mode 
queues: the application writes AQL packets into an in memory ring and 
rings a doorbell an MMIO page from the GPU's mapped straight into user 
space and the GPU's hardware scheduler picks the work up. Nothing on the 
host translates or dispatches a command stream the way a GL renderer 
does. Completion is an signal: either an interrupt backed event wait, or 
the CPU polling the signal value the GPU writes to memory. So the host 
side is closer to an ioctl proxy (create/destroy queues, 
alloc/map/register memory, signals, submit); the hard parts are the 
memory stuff you raised and the async event/fault channel.

For other frameworks, I think we can keep the device generic, user 
queues, a shared memory, events with the framework specific marshalling 
in a small pluggable backend. AMD is unifying queues in DRM, allowing 
devices to use user queues and event like mechanisms, which will greatly 
simplify device emulation. It also may affect the NPU. I think that 
graphics may use the same mechanism in the future to avoid complex 
synchronization issues.

Regards,
Honglei

> 
>>
>> Regards,
>> Honglei
>>
>>
>>> [2]
>>> https://developers.redhat.com/articles/2025/06/05/how-we-improved-
>>> ai-inference-macos-podman-containers
>>> [3] https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu-
>>> opencl-driver
>>> Regards,
>>> Akihiko Odaki
>>>
>>>>
>>>> That reuses the existing pluggable backend model, a second virtio gpu + a
>>>> backend library. It doesn't add dedicated queues for
>>>> debug/profiling currently.
>>>>
>>>> Waiting for reply and  happy to share more detail. Thanks!
>>>>
>>>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/
>>>> merge_requests/1568
>>>>
>>>> Regards,
>>>> Honglei
>>>
>