About new backend for GPU compute ROCm in qemu

About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

Hi Michael, Alex, Dmitry, Akihiko,

I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm over 
virtio
implementation to virglrenderer nine months ago (MR !1568 [1]). The ROCm 
side has
been supportted by ROCm offical.

Current implementation is a virtio gpu context type capset handled inside
virglrenderer, sharing the display path. That's an awkward fit, many
compute GPUs have no display engine at all.

Beyond that, sharing the display path is increasingly painful:

   - Compute hammers the queues more than graphics, so sharing
     virtio gpu's single control queue with display/virgl causes contention
     and display stutter.
   - Compute contexts need far more blob / shared memory than a display one.
   - Maybe needs a wider ROCm / compute stack, cause the render model 
fits poorly:
     rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth streams)
     and ROCgdb (wave control, address watch, async exceptions an
     out of band channel that must not block display).
   - Events, faults and GPU reset/SMI are async and don't map onto fences.
   - All of this is hard to extend cleanly inside a display capset.

On the QEMU/host side, would something like this be OK? One step, two parts:

   - a dedicated headless virtio gpu instance for compute.
   - that instance served by a separate ROCm backend library loaded
     in-process by QEMU.

That reuses the existing pluggable backend model, a second virtio gpu + a
backend library. It doesn't add dedicated queues for debug/profiling 
currently.

Waiting for reply and  happy to share more detail. Thanks!

[1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1568

Regards,
Honglei

Re: About new backend for GPU compute ROCm in qemu

[Alex Bennée]

"Huang, Honglei" <honghuan@amd.com> writes:

> Hi Michael, Alex, Dmitry, Akihiko,
>
> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
> over virtio
> implementation to virglrenderer nine months ago (MR !1568 [1]). The
> ROCm side has
> been supportted by ROCm offical.
>
> Current implementation is a virtio gpu context type capset handled inside
> virglrenderer, sharing the display path. That's an awkward fit, many
> compute GPUs have no display engine at all.
>
> Beyond that, sharing the display path is increasingly painful:
>
>   - Compute hammers the queues more than graphics, so sharing
>     virtio gpu's single control queue with display/virgl causes contention
>     and display stutter.

Is this just due to iteration? From a layman's point of view I'm curious
as to what the queues are doing.

>   - Compute contexts need far more blob / shared memory than a display
>   one.

I guess weights and context are long lived blobs compared to rendering assets?

>   - Maybe needs a wider ROCm / compute stack, cause the render model
>     fits poorly:
>     rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth streams)
>     and ROCgdb (wave control, address watch, async exceptions an
>     out of band channel that must not block display).
>   - Events, faults and GPU reset/SMI are async and don't map onto fences.
>   - All of this is hard to extend cleanly inside a display capset.
>
> On the QEMU/host side, would something like this be OK? One step, two parts:
>
>   - a dedicated headless virtio gpu instance for compute.

An oft-asked for VirtIO model is virtio-npu but I wonder if there is
enough commonality for a virtio-compute device with the same sort native
context type handling to deal with the those that want to have guests
targeting specific hardware rather than going through an abstraction
like Vulkan Computer or OpenGL CL.

>   - that instance served by a separate ROCm backend library loaded
>     in-process by QEMU.

Is this library a binary blob or open source? The last time I looked at
ROCm I had to give up as my AMD card was in an Aarch64 AVA machine.

> That reuses the existing pluggable backend model, a second virtio gpu + a
> backend library. It doesn't add dedicated queues for debug/profiling
> currently.
>
> Waiting for reply and  happy to share more detail. Thanks!
>
> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1568
>
> Regards,
> Honglei

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: About new backend for GPU compute ROCm in qemu

[Akihiko Odaki]

On 2026/08/17 12:19, Huang, Honglei wrote:
> 
> Hi Michael, Alex, Dmitry, Akihiko,

Hi Honglei,

> 
> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm over 
> virtio
> implementation to virglrenderer nine months ago (MR !1568 [1]). The ROCm 
> side has
> been supportted by ROCm offical.
> 
> Current implementation is a virtio gpu context type capset handled inside
> virglrenderer, sharing the display path. That's an awkward fit, many
> compute GPUs have no display engine at all.

I think "sharing the display path" conflates several layers and makes 
the problem difficult to assess. It would help to identify the concrete 
constraint behind "awkward fit."

End-to-end, there are four relevant layers:

1. Host GPU stack: hardware, host kernel, and host userspace
2. Paravirtualization stack: virglrenderer and QEMU
3. Host/guest interface: virtio and the capset-specific command stream
4. Guest GPU stack: guest kernel and guest userspace

Orthogonally, acceleration is separate from display and scanout. A
physical device may provide both, but acceleration does not require a
display engine. Linux likewise exposes render and compute interfaces
separately from modesetting. The userspace interface virglrenderer uses 
is messy; there is Vulkan, EGL, OpenGL, and now you are adding ROCm. But 
there is one thing I must note is that acceleration and display is 
decoupled, and acceleration does not require display.

At the protocol layer, context command buffers are carried by 
VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no enabled 
displays. At the implementation layer, QEMU handles scanout 
presentation. virgl_cmd_set_scanout() obtains resource information 
through virgl_renderer_resource_get_info() or 
virgl_renderer_resource_get_info_ext(). That does not make scanout a 
virglrenderer-owned display path.

Therefore, if "sharing the display path" means sharing the same device, 
control queue, and QEMU execution context, that identifies a possible 
source of contention. If it means that capsets or virglrenderer are 
inherently tied to display, I do not think that is accurate. Vulkan 
compute is already used through Venus with libkrun [2], and VCL proposes 
OpenCL support through virglrenderer [3].
> Beyond that, sharing the display path is increasingly painful:
> 
>    - Compute hammers the queues more than graphics, so sharing
>      virtio gpu's single control queue with display/virgl causes contention
>      and display stutter.

All non-cursor commands do share one control queue, but a fence avoids 
serialization.

There may still be implementation-level contention, and it is not
necessarily specific to compute. A sufficiently busy graphics workload
could expose the same bottlenecks. Possible contributors in current QEMU
include:

a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
    needs to fence scanout commands.

b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
    queue just to delay one command.

c) QEMU dispatches the control queue and calls into virglrenderer from
    its main-loop thread along with display work and many other things.
    Venus's render server can offload renderer work, but control-queue
    dispatch remains in QEMU's main loop.

In any case, I think you need to do some experiments to track down the 
real cause. a) is easy to check: just comment out all 
qemu_console_hw_gl_block() calls; it may corrupt display but removes the 
blocking. b) can also be tested by leaking the mappings instead of 
blocking the whole queue. Using a different display device like qxl 
tells whether c) is causing contention.

>    - Compute contexts need far more blob / shared memory than a display 
> one.

It is not a problem by itself. Frequent mapping and unmapping might 
amplify the second issue above, but that needs to be measured.

>    - Maybe needs a wider ROCm / compute stack, cause the render model 
> fits poorly:
>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth streams)
>      and ROCgdb (wave control, address watch, async exceptions an
>      out of band channel that must not block display).

virglrenderer does not impose a particular render model. That's why 
Vulkan Compute just works with Venus.

>    - Events, faults and GPU reset/SMI are async and don't map onto fences.>    - All of this is hard to extend cleanly inside a display capset.
Capset is not about display but determines the protocol of the 
VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described events, 
faults and GPU reset/SMI are async don't map onto fences that may be 
associated with VIRTIO_GPU_CMD_SUBMIT_3D which is dictated by capset. An 
additional feature may be necessary, and it may or may not be dictated 
by capset. The other things are irrelevant with the protocol capset 
represents; they are either behavioral or about different commands.

> 
> On the QEMU/host side, would something like this be OK? One step, two 
> parts:
> 
>    - a dedicated headless virtio gpu instance for compute.

A second device would isolate its virtqueues and device-wide
renderer_blocked state. That may be useful if measurements show that
these are the bottlenecks, but it is not yet clear that they are or that
a second device is the appropriate solution.

>    - that instance served by a separate ROCm backend library loaded
>      in-process by QEMU.

First, I think we need to establish why ROCm cannot or should not remain
in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
likely better placed to advise on that boundary. Once the protocol
requirements and performance measurements are clear, we can assess the
appropriate QEMU integration.

[2] 
https://developers.redhat.com/articles/2025/06/05/how-we-improved-ai-inference-macos-podman-containers
[3] 
https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu-opencl-driver

Regards,
Akihiko Odaki

> 
> That reuses the existing pluggable backend model, a second virtio gpu + a
> backend library. It doesn't add dedicated queues for debug/profiling 
> currently.
> 
> Waiting for reply and  happy to share more detail. Thanks!
> 
> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/ 
> merge_requests/1568
> 
> Regards,
> Honglei

Re: About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

On 8/17/2026 5:06 PM, Alex Bennée wrote:
> "Huang, Honglei" <honghuan@amd.com> writes:
> 
>> Hi Michael, Alex, Dmitry, Akihiko,
>>
>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
>> over virtio
>> implementation to virglrenderer nine months ago (MR !1568 [1]). The
>> ROCm side has
>> been supportted by ROCm offical.
>>
>> Current implementation is a virtio gpu context type capset handled inside
>> virglrenderer, sharing the display path. That's an awkward fit, many
>> compute GPUs have no display engine at all.
>>
>> Beyond that, sharing the display path is increasingly painful:
>>
>>    - Compute hammers the queues more than graphics, so sharing
>>      virtio gpu's single control queue with display/virgl causes contention
>>      and display stutter.
> 
> Is this just due to iteration? From a layman's point of view I'm curious
> as to what the queues are doing.

Really thanks for the reply.

The load is mostly memory management. Running an AI model allocates and 
frees a large number of blobs. We already did some optimization release 
them asynchronously, but the host processing is a single queue one
process_cmdq, this is where the main bottleneck in my debugging work / 
my understanding so far. I'm not certain it's the whole picture, so 
please correct if I am wrong.

A model load or unload frees a large batch of BOs and allocates another. 
Some of those commands are async in the virtio-gpu guest driver, but 
QEMU still has to work through them on the one queue, which takes time; 
so even though any single command is quick, there are simply too many of 
them, the single queue backs up, and everything behind it, gets delayed.

real work load (a few downstream customisations): loading one 16 GB 
model (gemm4 e4b), drives ~1200 blob creates, a burst of
~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
notifies, caused a 22 s guest soft lockup. And the behavior of memory 
operations are controlled by upper layer like pytorch / HIP / runtime,
we can not control it.

To be honest, a separate backend won't fix this. But the real solution 
maybe is compute specific. That logic is only useful to the compute 
path, and folding it into the shared display device / renderer would 
mean churning code that is mature and stable for graphics, with 
regression risk. Keeping compute on its own instance and backend lets us 
iterate on these compute only optimisations.

> 
>>    - Compute contexts need far more blob / shared memory than a display
>>    one.
> 
> I guess weights and context are long lived blobs compared to rendering assets?

Exactly, weights, KV caches, scratch heaps and the userptr/SVM arenas 
are large and long-lived, unlike per frame rendering assets, so an AI 
context maps far more memory than a display one.

> 
>>    - Maybe needs a wider ROCm / compute stack, cause the render model
>>      fits poorly:
>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth streams)
>>      and ROCgdb (wave control, address watch, async exceptions an
>>      out of band channel that must not block display).
>>    - Events, faults and GPU reset/SMI are async and don't map onto fences.
>>    - All of this is hard to extend cleanly inside a display capset.
>>
>> On the QEMU/host side, would something like this be OK? One step, two parts:
>>
>>    - a dedicated headless virtio gpu instance for compute.
> 
> An oft-asked for VirtIO model is virtio-npu but I wonder if there is
> enough commonality for a virtio-compute device with the same sort native
> context type handling to deal with the those that want to have guests
> targeting specific hardware rather than going through an abstraction
> like Vulkan Computer or OpenGL CL.

Agreed there's likely enough commonality. The idea would be for the 
device to provide mechanism, not hardware policy. And actually the 
current ROCm implementation is doing like this, it can also support 
OPENCL and HIP.

> 
>>    - that instance served by a separate ROCm backend library loaded
>>      in-process by QEMU.
> 
> Is this library a binary blob or open source? The last time I looked at
> ROCm I had to give up as my AMD card was in an Aarch64 AVA machine.

It is open source, we are planing put it into ROCm stack, let it be a 
part of ROCm.

Regards,
Honglei

> 
>> That reuses the existing pluggable backend model, a second virtio gpu + a
>> backend library. It doesn't add dedicated queues for debug/profiling
>> currently.
>>
>> Waiting for reply and  happy to share more detail. Thanks!
>>
>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1568
>>
>> Regards,
>> Honglei
> 

Re: About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
> On 2026/08/17 12:19, Huang, Honglei wrote:
>>
>> Hi Michael, Alex, Dmitry, Akihiko,
> 
> Hi Honglei,
> 
>>
>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm 
>> over virtio
>> implementation to virglrenderer nine months ago (MR !1568 [1]). The 
>> ROCm side has
>> been supportted by ROCm offical.
>>
>> Current implementation is a virtio gpu context type capset handled inside
>> virglrenderer, sharing the display path. That's an awkward fit, many
>> compute GPUs have no display engine at all.
> 
> I think "sharing the display path" conflates several layers and makes 
> the problem difficult to assess. It would help to identify the concrete 
> constraint behind "awkward fit."
> 
> End-to-end, there are four relevant layers:
> 
> 1. Host GPU stack: hardware, host kernel, and host userspace
> 2. Paravirtualization stack: virglrenderer and QEMU

Yes we are asking can we add a new file like virtio-gpu specific for 
compute, but maybe we can only add a new backend like virglrenderer 
specific for compute.

> 3. Host/guest interface: virtio and the capset-specific command stream

In this plan we may need just add a capset id.

> 4. Guest GPU stack: guest kernel and guest userspace

Won't modify the guest kernel in this plan, this email list.

> 
> Orthogonally, acceleration is separate from display and scanout. A
> physical device may provide both, but acceleration does not require a
> display engine. Linux likewise exposes render and compute interfaces
> separately from modesetting. The userspace interface virglrenderer uses 
> is messy; there is Vulkan, EGL, OpenGL, and now you are adding ROCm. But 
> there is one thing I must note is that acceleration and display is 
> decoupled, and acceleration does not require display.

Yes totally agreed.

> 
> At the protocol layer, context command buffers are carried by 
> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no enabled 
> displays. At the implementation layer, QEMU handles scanout 
> presentation. virgl_cmd_set_scanout() obtains resource information 
> through virgl_renderer_resource_get_info() or 
> virgl_renderer_resource_get_info_ext(). That does not make scanout a 
> virglrenderer-owned display path.

Yes, agreed.

> 
> Therefore, if "sharing the display path" means sharing the same device, 
> control queue, and QEMU execution context, that identifies a possible 
> source of contention. If it means that capsets or virglrenderer are 
> inherently tied to display, I do not think that is accurate. Vulkan 
> compute is already used through Venus with libkrun [2], and VCL proposes 
> OpenCL support through virglrenderer [3].

Yes，but the vulkan is for GFX originally, and for some formal AI frame 
work like pytorch, it's support is limited, and it performance is lower 
than ROCm, and vulkan also lacks many AI infrastructure, like composable 
kernel.
And for virCL, actually it is came from same project with ROCm native 
context, but the original author didn't continue to support it, they 
handed it over to someone else to take over. And in the first version of
virCL, it didn't pass the test of actual projects.

And it seems like virCL didn't upstream into virglrenderer also, correct 
me if I am wrong.

>> Beyond that, sharing the display path is increasingly painful:
>>
>>    - Compute hammers the queues more than graphics, so sharing
>>      virtio gpu's single control queue with display/virgl causes 
>> contention
>>      and display stutter.
> 
> All non-cursor commands do share one control queue, but a fence avoids 
> serialization.

yes, agreed.

> 
> There may still be implementation-level contention, and it is not
> necessarily specific to compute. A sufficiently busy graphics workload
> could expose the same bottlenecks. Possible contributors in current QEMU
> include:
> 
> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>     needs to fence scanout commands.

Yes, agreed.

> 
> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>     queue just to delay one command.

Yes, but it is seems like it is must, someone else in AMD tried to use 
async method to relase blob, but it failed to consistency issue, then
reverted to sync version.

> 
> c) QEMU dispatches the control queue and calls into virglrenderer from
>     its main-loop thread along with display work and many other things.
>     Venus's render server can offload renderer work, but control-queue
>     dispatch remains in QEMU's main loop.

Yes, we did some async optimization in ROCm context, but its 
effectiveness is limited, see bellow.

> 
> In any case, I think you need to do some experiments to track down the 
> real cause. a) is easy to check: just comment out all 
> qemu_console_hw_gl_block() calls; it may corrupt display but removes the 
> blocking. b) can also be tested by leaking the mappings instead of 
> blocking the whole queue. Using a different display device like qxl 
> tells whether c) is causing contention.

Yes, totally agreed. following is my findings. In short words:

Optimization can reduce queue pressure, but it can't withstand absolute 
overload because each command has some overhead. Making all commands 
asynchronous would lead to a debugging hell about asynchronous issues.
And we have high load applications rocmprofiler  that continuously catch 
information need virtio queue to handle. But create a new backend can 
not solve it simply, we are trying to find a way. like shmem between 
guest and host, then use cpu polling, bypass the virtqueue.

The load is mostly memory management. Running an AI model allocates and 
frees a large number of blobs. We already did some optimization release 
them asynchronously, but the host processing is a single queue one
process_cmdq, this is where the main bottleneck in my debugging work / 
my understanding so far. I'm not certain it's the whole picture, so 
please correct if I am wrong.

A model load or unload frees a large batch of BOs and allocates another. 
Some of those commands are async in the virtio-gpu guest driver, but 
QEMU still has to work through them on the one queue, which takes time; 
so even though any single command is quick, there are simply too many of 
them, the single queue backs up, and everything behind it, gets delayed.

real work load (a few downstream customisations): loading one 16 GB 
model (gemm4 e4b), drives ~1200 blob creates, a burst of
~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
notifies, caused a 22 s guest soft lockup. And the behavior of memory 
operations are controlled by upper layer like pytorch / HIP / runtime,
we can not control it.

To be honest, a separate backend won't fix this. But the real solution 
maybe is compute specific. That logic is only useful to the compute 
path, and folding it into the shared display device / renderer would 
mean churning code that is mature and stable for graphics, with 
regression risk. Keeping compute on its own instance and backend lets us 
iterate on these compute only optimisations.

> 
>>    - Compute contexts need far more blob / shared memory than a 
>> display one.
> 
> It is not a problem by itself. Frequent mapping and unmapping might 
> amplify the second issue above, but that needs to be measured.

Yes, agreed. I can give more detailed information.

> 
>>    - Maybe needs a wider ROCm / compute stack, cause the render model 
>> fits poorly:
>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth 
>> streams)
>>      and ROCgdb (wave control, address watch, async exceptions an
>>      out of band channel that must not block display).
> 
> virglrenderer does not impose a particular render model. That's why 
> Vulkan Compute just works with Venus.

Yes but vulkan is used for GFX initally. And can not support many AI 
application.>
>>    - Events, faults and GPU reset/SMI are async and don't map onto 
>> fences.>    - All of this is hard to extend cleanly inside a display 
>> capset.
> Capset is not about display but determines the protocol of the 
> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described events, 
> faults and GPU reset/SMI are async don't map onto fences that may be 
> associated with VIRTIO_GPU_CMD_SUBMIT_3D which is dictated by capset. An 
> additional feature may be necessary, and it may or may not be dictated 
> by capset. The other things are irrelevant with the protocol capset 
> represents; they are either behavioral or about different commands.

A fence is the one shot, but event is stateful and repeatable.
That may or may not be tied to capset. Agreed.

> 
>>
>> On the QEMU/host side, would something like this be OK? One step, two 
>> parts:
>>
>>    - a dedicated headless virtio gpu instance for compute.
> 
> A second device would isolate its virtqueues and device-wide
> renderer_blocked state. That may be useful if measurements show that
> these are the bottlenecks, but it is not yet clear that they are or that
> a second device is the appropriate solution.
> 
>>    - that instance served by a separate ROCm backend library loaded
>>      in-process by QEMU.
> 
> First, I think we need to establish why ROCm cannot or should not remain
> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
> likely better placed to advise on that boundary. Once the protocol
> requirements and performance measurements are clear, we can assess the
> appropriate QEMU integration.

venus is borned for GFX.
virCL not merged.

To be clear, I'm not saying virglrenderer can't host a ROCm native 
context it clearly can. My hesitation is more about fit and direction: 
virglrenderer has grown up around GL/graphics, and I haven't yet found 
compute oriented plumbing there to build on, while ROCm moves very fast 
and I need something I can keep current with low friction.

Regards,
Honglei


> 
> [2] https://developers.redhat.com/articles/2025/06/05/how-we-improved- 
> ai-inference-macos-podman-containers
> [3] https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu- 
> opencl-driver
> 
> Regards,
> Akihiko Odaki
> 
>>
>> That reuses the existing pluggable backend model, a second virtio gpu + a
>> backend library. It doesn't add dedicated queues for debug/profiling 
>> currently.
>>
>> Waiting for reply and  happy to share more detail. Thanks!
>>
>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/ 
>> merge_requests/1568
>>
>> Regards,
>> Honglei
> 

Re: About new backend for GPU compute ROCm in qemu

[Alex Bennée]

"Huang, Honglei" <honghuan@amd.com> writes:

> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>
>>> Hi Michael, Alex, Dmitry, Akihiko,
>> Hi Honglei,
>> 
>>>
>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
>>> over virtio
>>> implementation to virglrenderer nine months ago (MR !1568 [1]). The
>>> ROCm side has
>>> been supportted by ROCm offical.
>>>
>>> Current implementation is a virtio gpu context type capset handled inside
>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>> compute GPUs have no display engine at all.
<snip>
>>>    - that instance served by a separate ROCm backend library loaded
>>>      in-process by QEMU.
>> First, I think we need to establish why ROCm cannot or should not
>> remain
>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>> likely better placed to advise on that boundary. Once the protocol
>> requirements and performance measurements are clear, we can assess the
>> appropriate QEMU integration.
>
> venus is borned for GFX.
> virCL not merged.
>
> To be clear, I'm not saying virglrenderer can't host a ROCm native
> context it clearly can. My hesitation is more about fit and direction:
> virglrenderer has grown up around GL/graphics, and I haven't yet found
> compute oriented plumbing there to build on, while ROCm moves very
> fast and I need something I can keep current with low friction.

I'm unsure what the current development status of virglrenderer is but
it does see a continuing stream of merges. However the threading model
does make things tricky for QEMU when we are sharing lifetime of blobs
between QEMU proper and the virglrenderer thread. Perhaps there is a
better way to organise things?

Could we do the marshalling of VirtIO GPU commands into ROCm directly
inside QEMU rather than going through additional plumbing? Are the
sequences we need to handle more or less complex than your general gfx
rendering? How might this work with other frameworks?

>
> Regards,
> Honglei
>
>
>> [2]
>> https://developers.redhat.com/articles/2025/06/05/how-we-improved-
>> ai-inference-macos-podman-containers
>> [3] https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu-
>> opencl-driver
>> Regards,
>> Akihiko Odaki
>> 
>>>
>>> That reuses the existing pluggable backend model, a second virtio gpu + a
>>> backend library. It doesn't add dedicated queues for
>>> debug/profiling currently.
>>>
>>> Waiting for reply and  happy to share more detail. Thanks!
>>>
>>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/
>>> merge_requests/1568
>>>
>>> Regards,
>>> Honglei
>> 

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: About new backend for GPU compute ROCm in qemu

[Akihiko Odaki]

On 2026/08/17 22:44, Huang, Honglei wrote:
> 
> 
> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>
>>> Hi Michael, Alex, Dmitry, Akihiko,
>>
>> Hi Honglei,
>>
>>>
>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm 
>>> over virtio
>>> implementation to virglrenderer nine months ago (MR !1568 [1]). The 
>>> ROCm side has
>>> been supportted by ROCm offical.
>>>
>>> Current implementation is a virtio gpu context type capset handled 
>>> inside
>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>> compute GPUs have no display engine at all.
>>
>> I think "sharing the display path" conflates several layers and makes 
>> the problem difficult to assess. It would help to identify the 
>> concrete constraint behind "awkward fit."
>>
>> End-to-end, there are four relevant layers:
>>
>> 1. Host GPU stack: hardware, host kernel, and host userspace
>> 2. Paravirtualization stack: virglrenderer and QEMU
> 
> Yes we are asking can we add a new file like virtio-gpu specific for 
> compute, but maybe we can only add a new backend like virglrenderer 
> specific for compute.
> 
>> 3. Host/guest interface: virtio and the capset-specific command stream
> 
> In this plan we may need just add a capset id.
> 
>> 4. Guest GPU stack: guest kernel and guest userspace
> 
> Won't modify the guest kernel in this plan, this email list.
> 
>>
>> Orthogonally, acceleration is separate from display and scanout. A
>> physical device may provide both, but acceleration does not require a
>> display engine. Linux likewise exposes render and compute interfaces
>> separately from modesetting. The userspace interface virglrenderer 
>> uses is messy; there is Vulkan, EGL, OpenGL, and now you are adding 
>> ROCm. But there is one thing I must note is that acceleration and 
>> display is decoupled, and acceleration does not require display.
> 
> Yes totally agreed.
> 
>>
>> At the protocol layer, context command buffers are carried by 
>> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
>> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no enabled 
>> displays. At the implementation layer, QEMU handles scanout 
>> presentation. virgl_cmd_set_scanout() obtains resource information 
>> through virgl_renderer_resource_get_info() or 
>> virgl_renderer_resource_get_info_ext(). That does not make scanout a 
>> virglrenderer-owned display path.
> 
> Yes, agreed.
> 
>>
>> Therefore, if "sharing the display path" means sharing the same 
>> device, control queue, and QEMU execution context, that identifies a 
>> possible source of contention. If it means that capsets or 
>> virglrenderer are inherently tied to display, I do not think that is 
>> accurate. Vulkan compute is already used through Venus with libkrun 
>> [2], and VCL proposes OpenCL support through virglrenderer [3].
> 
> Yes，but the vulkan is for GFX originally, and for some formal AI frame 
> work like pytorch, it's support is limited, and it performance is lower 
> than ROCm, and vulkan also lacks many AI infrastructure, like composable 
> kernel.
> And for virCL, actually it is came from same project with ROCm native 
> context, but the original author didn't continue to support it, they 
> handed it over to someone else to take over. And in the first version of
> virCL, it didn't pass the test of actual projects.
> 
> And it seems like virCL didn't upstream into virglrenderer also, correct 
> me if I am wrong.

I cited Venus and VCL only as examples showing that virtio-gpu and
virglrenderer are not intrinsically tied to display. I did not suggest
either as a substitute for ROCm.

> 
>>> Beyond that, sharing the display path is increasingly painful:
>>>
>>>    - Compute hammers the queues more than graphics, so sharing
>>>      virtio gpu's single control queue with display/virgl causes 
>>> contention
>>>      and display stutter.
>>
>> All non-cursor commands do share one control queue, but a fence avoids 
>> serialization.
> 
> yes, agreed.
> 
>>
>> There may still be implementation-level contention, and it is not
>> necessarily specific to compute. A sufficiently busy graphics workload
>> could expose the same bottlenecks. Possible contributors in current QEMU
>> include:
>>
>> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>>     needs to fence scanout commands.
> 
> Yes, agreed.
> 
>>
>> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>>     queue just to delay one command.
> 
> Yes, but it is seems like it is must, someone else in AMD tried to use 
> async method to relase blob, but it failed to consistency issue, then
> reverted to sync version.

Queue-wide suspension is not inherently required. Commit 4eb0aace85f5 
("virtio-gpu: Support mapping hostmem blobs with map_fixed") added a 
path that avoids per-blob MemoryRegion teardown when 
virgl_renderer_resource_map_fixed() succeeds. The remaining path is also 
being improved with:

https://lore.kernel.org/qemu-devel/20260424-force_rcu-v4-0-feccfaca0568@rsg.ci.i.u-tokyo.ac.jp/
("[PATCH v4 0/6] virtio-gpu: Force RCU when unmapping blob")

> 
>>
>> c) QEMU dispatches the control queue and calls into virglrenderer from
>>     its main-loop thread along with display work and many other things.
>>     Venus's render server can offload renderer work, but control-queue
>>     dispatch remains in QEMU's main loop.
> 
> Yes, we did some async optimization in ROCm context, but its 
> effectiveness is limited, see bellow.
> 
>>
>> In any case, I think you need to do some experiments to track down the 
>> real cause. a) is easy to check: just comment out all 
>> qemu_console_hw_gl_block() calls; it may corrupt display but removes 
>> the blocking. b) can also be tested by leaking the mappings instead of 
>> blocking the whole queue. Using a different display device like qxl 
>> tells whether c) is causing contention.
> 
> Yes, totally agreed. following is my findings. In short words:
> 
> Optimization can reduce queue pressure, but it can't withstand absolute 
> overload because each command has some overhead. Making all commands 
> asynchronous would lead to a debugging hell about asynchronous issues.
> And we have high load applications rocmprofiler  that continuously catch 
> information need virtio queue to handle. But create a new backend can 
> not solve it simply, we are trying to find a way. like shmem between 
> guest and host, then use cpu polling, bypass the virtqueue.

Most commands are fast on the CPU side, while heavy processing
happens asynchronously on the GPU. Cases (a) and (b) are exceptions.

> 
> The load is mostly memory management. Running an AI model allocates and 
> frees a large number of blobs. We already did some optimization release 
> them asynchronously, but the host processing is a single queue one
> process_cmdq, this is where the main bottleneck in my debugging work / 
> my understanding so far. I'm not certain it's the whole picture, so 
> please correct if I am wrong.
> 
> A model load or unload frees a large batch of BOs and allocates another. 
> Some of those commands are async in the virtio-gpu guest driver, but 
> QEMU still has to work through them on the one queue, which takes time; 
> so even though any single command is quick, there are simply too many of 
> them, the single queue backs up, and everything behind it, gets delayed.
> 
> real work load (a few downstream customisations): loading one 16 GB 
> model (gemm4 e4b), drives ~1200 blob creates, a burst of
> ~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
> notifies, caused a 22 s guest soft lockup. And the behavior of memory 
> operations are controlled by upper layer like pytorch / HIP / runtime,
> we can not control it.
> 
> To be honest, a separate backend won't fix this. But the real solution 
> maybe is compute specific. That logic is only useful to the compute 
> path, and folding it into the shared display device / renderer would 
> mean churning code that is mature and stable for graphics, with 
> regression risk. Keeping compute on its own instance and backend lets us 
> iterate on these compute only optimisations.

A 22-second lockup is too long for those command counts.

The most probable explanation I have is that the ROCm integration blocks 
QEMU's main loop thread while synchronously waiting for GPU execution. 
Creating separate devices won't resolve this because the main loop 
thread is shared, and synchronously waiting on the GPU should be avoided 
in the first place.

In any case, profiling is necessary before touching the implementation.

> 
>>
>>>    - Compute contexts need far more blob / shared memory than a 
>>> display one.
>>
>> It is not a problem by itself. Frequent mapping and unmapping might 
>> amplify the second issue above, but that needs to be measured.
> 
> Yes, agreed. I can give more detailed information.
> 
>>
>>>    - Maybe needs a wider ROCm / compute stack, cause the render model 
>>> fits poorly:
>>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth 
>>> streams)
>>>      and ROCgdb (wave control, address watch, async exceptions an
>>>      out of band channel that must not block display).
>>
>> virglrenderer does not impose a particular render model. That's why 
>> Vulkan Compute just works with Venus.
> 
> Yes but vulkan is used for GFX initally. And can not support many AI 
> application.>
>>>    - Events, faults and GPU reset/SMI are async and don't map onto 
>>> fences.>    - All of this is hard to extend cleanly inside a display 
>>> capset.
>> Capset is not about display but determines the protocol of the 
>> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described events, 
>> faults and GPU reset/SMI are async don't map onto fences that may be 
>> associated with VIRTIO_GPU_CMD_SUBMIT_3D which is dictated by capset. 
>> An additional feature may be necessary, and it may or may not be 
>> dictated by capset. The other things are irrelevant with the protocol 
>> capset represents; they are either behavioral or about different 
>> commands.
> 
> A fence is the one shot, but event is stateful and repeatable.
> That may or may not be tied to capset. Agreed.
> 
>>
>>>
>>> On the QEMU/host side, would something like this be OK? One step, two 
>>> parts:
>>>
>>>    - a dedicated headless virtio gpu instance for compute.
>>
>> A second device would isolate its virtqueues and device-wide
>> renderer_blocked state. That may be useful if measurements show that
>> these are the bottlenecks, but it is not yet clear that they are or that
>> a second device is the appropriate solution.
>>
>>>    - that instance served by a separate ROCm backend library loaded
>>>      in-process by QEMU.
>>
>> First, I think we need to establish why ROCm cannot or should not remain
>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>> likely better placed to advise on that boundary. Once the protocol
>> requirements and performance measurements are clear, we can assess the
>> appropriate QEMU integration.
> 
> venus is borned for GFX.
> virCL not merged.
> 
> To be clear, I'm not saying virglrenderer can't host a ROCm native 
> context it clearly can. My hesitation is more about fit and direction: 
> virglrenderer has grown up around GL/graphics, and I haven't yet found 
> compute oriented plumbing there to build on, while ROCm moves very fast 
> and I need something I can keep current with low friction.

Whether keeping ROCm in virglrenderer would create extra friction is 
primarily a question for the virglrenderer maintainers. Its graphics 
origins do not by themselves motivate adding a separate backend 
interface to QEMU.

Regards,
Akihiko Odaki

> 
> Regards,
> Honglei
> 
> 
>>
>> [2] https://developers.redhat.com/articles/2025/06/05/how-we-improved- 
>> ai-inference-macos-podman-containers
>> [3] https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu- 
>> opencl-driver
>>
>> Regards,
>> Akihiko Odaki
>>
>>>
>>> That reuses the existing pluggable backend model, a second virtio gpu 
>>> + a
>>> backend library. It doesn't add dedicated queues for debug/profiling 
>>> currently.
>>>
>>> Waiting for reply and  happy to share more detail. Thanks!
>>>
>>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/ 
>>> merge_requests/1568
>>>
>>> Regards,
>>> Honglei
>>
> 

Re: About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

On 8/17/2026 10:24 PM, Alex Bennée wrote:
> "Huang, Honglei" <honghuan@amd.com> writes:
> 
>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>
>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>> Hi Honglei,
>>>
>>>>
>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm
>>>> over virtio
>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). The
>>>> ROCm side has
>>>> been supportted by ROCm offical.
>>>>
>>>> Current implementation is a virtio gpu context type capset handled inside
>>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>>> compute GPUs have no display engine at all.
> <snip>
>>>>     - that instance served by a separate ROCm backend library loaded
>>>>       in-process by QEMU.
>>> First, I think we need to establish why ROCm cannot or should not
>>> remain
>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>> likely better placed to advise on that boundary. Once the protocol
>>> requirements and performance measurements are clear, we can assess the
>>> appropriate QEMU integration.
>>
>> venus is borned for GFX.
>> virCL not merged.
>>
>> To be clear, I'm not saying virglrenderer can't host a ROCm native
>> context it clearly can. My hesitation is more about fit and direction:
>> virglrenderer has grown up around GL/graphics, and I haven't yet found
>> compute oriented plumbing there to build on, while ROCm moves very
>> fast and I need something I can keep current with low friction.
> 
> I'm unsure what the current development status of virglrenderer is but
> it does see a continuing stream of merges. However the threading model
> does make things tricky for QEMU when we are sharing lifetime of blobs
> between QEMU proper and the virglrenderer thread. Perhaps there is a
> better way to organise things?

Yes, agreed. ROCm has no render thread of its own, it uses user-mode 
queues So, I think there's a better organisation.

> 
> Could we do the marshalling of VirtIO GPU commands into ROCm directly
> inside QEMU rather than going through additional plumbing? 

Yes, by "additional plumbing" I take you to mean routing through a
virglrenderer style renderer ( extra thread and blob coordination).
Yes, I'd drop that. What I'd keep is only a thin in QEMU backend the 
9pfs/cryptodev kind of pluggable backend, not a renderer layer which is 
what lets QEMU own the blob lifetime directly.


> Are the sequences we need to handle more or less complex than your general gfx
> rendering? How might this work with other frameworks?

Simpler, I think. ROCm has no render thread. It uses HSA user-mode 
queues: the application writes AQL packets into an in memory ring and 
rings a doorbell an MMIO page from the GPU's mapped straight into user 
space and the GPU's hardware scheduler picks the work up. Nothing on the 
host translates or dispatches a command stream the way a GL renderer 
does. Completion is an signal: either an interrupt backed event wait, or 
the CPU polling the signal value the GPU writes to memory. So the host 
side is closer to an ioctl proxy (create/destroy queues, 
alloc/map/register memory, signals, submit); the hard parts are the 
memory stuff you raised and the async event/fault channel.

For other frameworks, I think we can keep the device generic, user 
queues, a shared memory, events with the framework specific marshalling 
in a small pluggable backend. AMD is unifying queues in DRM, allowing 
devices to use user queues and event like mechanisms, which will greatly 
simplify device emulation. It also may affect the NPU. I think that 
graphics may use the same mechanism in the future to avoid complex 
synchronization issues.

Regards,
Honglei

> 
>>
>> Regards,
>> Honglei
>>
>>
>>> [2]
>>> https://developers.redhat.com/articles/2025/06/05/how-we-improved-
>>> ai-inference-macos-podman-containers
>>> [3] https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu-
>>> opencl-driver
>>> Regards,
>>> Akihiko Odaki
>>>
>>>>
>>>> That reuses the existing pluggable backend model, a second virtio gpu + a
>>>> backend library. It doesn't add dedicated queues for
>>>> debug/profiling currently.
>>>>
>>>> Waiting for reply and  happy to share more detail. Thanks!
>>>>
>>>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/
>>>> merge_requests/1568
>>>>
>>>> Regards,
>>>> Honglei
>>>
> 

Re: About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

On 8/18/2026 12:29 AM, Akihiko Odaki wrote:
> On 2026/08/17 22:44, Huang, Honglei wrote:
>>
>>
>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>
>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>>
>>> Hi Honglei,
>>>
>>>>
>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm 
>>>> over virtio
>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). The 
>>>> ROCm side has
>>>> been supportted by ROCm offical.
>>>>
>>>> Current implementation is a virtio gpu context type capset handled 
>>>> inside
>>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>>> compute GPUs have no display engine at all.
>>>
>>> I think "sharing the display path" conflates several layers and makes 
>>> the problem difficult to assess. It would help to identify the 
>>> concrete constraint behind "awkward fit."
>>>
>>> End-to-end, there are four relevant layers:
>>>
>>> 1. Host GPU stack: hardware, host kernel, and host userspace
>>> 2. Paravirtualization stack: virglrenderer and QEMU
>>
>> Yes we are asking can we add a new file like virtio-gpu specific for 
>> compute, but maybe we can only add a new backend like virglrenderer 
>> specific for compute.
>>
>>> 3. Host/guest interface: virtio and the capset-specific command stream
>>
>> In this plan we may need just add a capset id.
>>
>>> 4. Guest GPU stack: guest kernel and guest userspace
>>
>> Won't modify the guest kernel in this plan, this email list.
>>
>>>
>>> Orthogonally, acceleration is separate from display and scanout. A
>>> physical device may provide both, but acceleration does not require a
>>> display engine. Linux likewise exposes render and compute interfaces
>>> separately from modesetting. The userspace interface virglrenderer 
>>> uses is messy; there is Vulkan, EGL, OpenGL, and now you are adding 
>>> ROCm. But there is one thing I must note is that acceleration and 
>>> display is decoupled, and acceleration does not require display.
>>
>> Yes totally agreed.
>>
>>>
>>> At the protocol layer, context command buffers are carried by 
>>> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
>>> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no enabled 
>>> displays. At the implementation layer, QEMU handles scanout 
>>> presentation. virgl_cmd_set_scanout() obtains resource information 
>>> through virgl_renderer_resource_get_info() or 
>>> virgl_renderer_resource_get_info_ext(). That does not make scanout a 
>>> virglrenderer-owned display path.
>>
>> Yes, agreed.
>>
>>>
>>> Therefore, if "sharing the display path" means sharing the same 
>>> device, control queue, and QEMU execution context, that identifies a 
>>> possible source of contention. If it means that capsets or 
>>> virglrenderer are inherently tied to display, I do not think that is 
>>> accurate. Vulkan compute is already used through Venus with libkrun 
>>> [2], and VCL proposes OpenCL support through virglrenderer [3].
>>
>> Yes，but the vulkan is for GFX originally, and for some formal AI 
>> frame work like pytorch, it's support is limited, and it performance 
>> is lower than ROCm, and vulkan also lacks many AI infrastructure, like 
>> composable kernel.
>> And for virCL, actually it is came from same project with ROCm native 
>> context, but the original author didn't continue to support it, they 
>> handed it over to someone else to take over. And in the first version of
>> virCL, it didn't pass the test of actual projects.
>>
>> And it seems like virCL didn't upstream into virglrenderer also, 
>> correct me if I am wrong.
> 
> I cited Venus and VCL only as examples showing that virtio-gpu and
> virglrenderer are not intrinsically tied to display. I did not suggest
> either as a substitute for ROCm.
> 
>>
>>>> Beyond that, sharing the display path is increasingly painful:
>>>>
>>>>    - Compute hammers the queues more than graphics, so sharing
>>>>      virtio gpu's single control queue with display/virgl causes 
>>>> contention
>>>>      and display stutter.
>>>
>>> All non-cursor commands do share one control queue, but a fence 
>>> avoids serialization.
>>
>> yes, agreed.
>>
>>>
>>> There may still be implementation-level contention, and it is not
>>> necessarily specific to compute. A sufficiently busy graphics workload
>>> could expose the same bottlenecks. Possible contributors in current QEMU
>>> include:
>>>
>>> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>>>     needs to fence scanout commands.
>>
>> Yes, agreed.
>>
>>>
>>> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>>>     queue just to delay one command.
>>
>> Yes, but it is seems like it is must, someone else in AMD tried to use 
>> async method to relase blob, but it failed to consistency issue, then
>> reverted to sync version.
> 
> Queue-wide suspension is not inherently required. Commit 4eb0aace85f5 
> ("virtio-gpu: Support mapping hostmem blobs with map_fixed") added a 
> path that avoids per-blob MemoryRegion teardown when 
> virgl_renderer_resource_map_fixed() succeeds. The remaining path is also 
> being improved with:
> 
> https://lore.kernel.org/qemu-devel/20260424-force_rcu-v4-0- 
> feccfaca0568@rsg.ci.i.u-tokyo.ac.jp/
> ("[PATCH v4 0/6] virtio-gpu: Force RCU when unmapping blob")

Thanks. force_rcu is a clean fix for the RCU-reclamation part, but it 
still keeps the unmap synchronous and serial.

> 
>>
>>>
>>> c) QEMU dispatches the control queue and calls into virglrenderer from
>>>     its main-loop thread along with display work and many other things.
>>>     Venus's render server can offload renderer work, but control-queue
>>>     dispatch remains in QEMU's main loop.
>>
>> Yes, we did some async optimization in ROCm context, but its 
>> effectiveness is limited, see bellow.
>>
>>>
>>> In any case, I think you need to do some experiments to track down 
>>> the real cause. a) is easy to check: just comment out all 
>>> qemu_console_hw_gl_block() calls; it may corrupt display but removes 
>>> the blocking. b) can also be tested by leaking the mappings instead 
>>> of blocking the whole queue. Using a different display device like 
>>> qxl tells whether c) is causing contention.
>>
>> Yes, totally agreed. following is my findings. In short words:
>>
>> Optimization can reduce queue pressure, but it can't withstand 
>> absolute overload because each command has some overhead. Making all 
>> commands asynchronous would lead to a debugging hell about 
>> asynchronous issues.
>> And we have high load applications rocmprofiler  that continuously 
>> catch information need virtio queue to handle. But create a new 
>> backend can not solve it simply, we are trying to find a way. like 
>> shmem between guest and host, then use cpu polling, bypass the virtqueue.
> 
> Most commands are fast on the CPU side, while heavy processing
> happens asynchronously on the GPU. Cases (a) and (b) are exceptions.
> 
>>
>> The load is mostly memory management. Running an AI model allocates 
>> and frees a large number of blobs. We already did some optimization 
>> release them asynchronously, but the host processing is a single queue 
>> one
>> process_cmdq, this is where the main bottleneck in my debugging work / 
>> my understanding so far. I'm not certain it's the whole picture, so 
>> please correct if I am wrong.
>>
>> A model load or unload frees a large batch of BOs and allocates 
>> another. Some of those commands are async in the virtio-gpu guest 
>> driver, but QEMU still has to work through them on the one queue, 
>> which takes time; so even though any single command is quick, there 
>> are simply too many of them, the single queue backs up, and everything 
>> behind it, gets delayed.
>>
>> real work load (a few downstream customisations): loading one 16 GB 
>> model (gemm4 e4b), drives ~1200 blob creates, a burst of
>> ~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
>> notifies, caused a 22 s guest soft lockup. And the behavior of memory 
>> operations are controlled by upper layer like pytorch / HIP / runtime,
>> we can not control it.
>>
>> To be honest, a separate backend won't fix this. But the real solution 
>> maybe is compute specific. That logic is only useful to the compute 
>> path, and folding it into the shared display device / renderer would 
>> mean churning code that is mature and stable for graphics, with 
>> regression risk. Keeping compute on its own instance and backend lets 
>> us iterate on these compute only optimisations.
> 
> A 22-second lockup is too long for those command counts.
> 
> The most probable explanation I have is that the ROCm integration blocks 
> QEMU's main loop thread while synchronously waiting for GPU execution. 
> Creating separate devices won't resolve this because the main loop 
> thread is shared, and synchronously waiting on the GPU should be avoided 
> in the first place.

No synchronously waiting in ROCm backend, we are using user queue, and 
event waiting, no sync operation in CMD wait. all the resource release 
in ROCm are all async now.
Only the sync thing is memory thing mapping/unmapping in qemu, as long 
as it remains synchronous, it will be overwhelmed by the massive number 
of requests.

> 
> In any case, profiling is necessary before touching the implementation.
> 
>>
>>>
>>>>    - Compute contexts need far more blob / shared memory than a 
>>>> display one.
>>>
>>> It is not a problem by itself. Frequent mapping and unmapping might 
>>> amplify the second issue above, but that needs to be measured.
>>
>> Yes, agreed. I can give more detailed information.
>>
>>>
>>>>    - Maybe needs a wider ROCm / compute stack, cause the render 
>>>> model fits poorly:
>>>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth 
>>>> streams)
>>>>      and ROCgdb (wave control, address watch, async exceptions an
>>>>      out of band channel that must not block display).
>>>
>>> virglrenderer does not impose a particular render model. That's why 
>>> Vulkan Compute just works with Venus.
>>
>> Yes but vulkan is used for GFX initally. And can not support many AI 
>> application.>
>>>>    - Events, faults and GPU reset/SMI are async and don't map onto 
>>>> fences.>    - All of this is hard to extend cleanly inside a display 
>>>> capset.
>>> Capset is not about display but determines the protocol of the 
>>> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described events, 
>>> faults and GPU reset/SMI are async don't map onto fences that may be 
>>> associated with VIRTIO_GPU_CMD_SUBMIT_3D which is dictated by capset. 
>>> An additional feature may be necessary, and it may or may not be 
>>> dictated by capset. The other things are irrelevant with the protocol 
>>> capset represents; they are either behavioral or about different 
>>> commands.
>>
>> A fence is the one shot, but event is stateful and repeatable.
>> That may or may not be tied to capset. Agreed.
>>
>>>
>>>>
>>>> On the QEMU/host side, would something like this be OK? One step, 
>>>> two parts:
>>>>
>>>>    - a dedicated headless virtio gpu instance for compute.
>>>
>>> A second device would isolate its virtqueues and device-wide
>>> renderer_blocked state. That may be useful if measurements show that
>>> these are the bottlenecks, but it is not yet clear that they are or that
>>> a second device is the appropriate solution.
>>>
>>>>    - that instance served by a separate ROCm backend library loaded
>>>>      in-process by QEMU.
>>>
>>> First, I think we need to establish why ROCm cannot or should not remain
>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>> likely better placed to advise on that boundary. Once the protocol
>>> requirements and performance measurements are clear, we can assess the
>>> appropriate QEMU integration.
>>
>> venus is borned for GFX.
>> virCL not merged.
>>
>> To be clear, I'm not saying virglrenderer can't host a ROCm native 
>> context it clearly can. My hesitation is more about fit and direction: 
>> virglrenderer has grown up around GL/graphics, and I haven't yet found 
>> compute oriented plumbing there to build on, while ROCm moves very 
>> fast and I need something I can keep current with low friction.
> 
> Whether keeping ROCm in virglrenderer would create extra friction is 
> primarily a question for the virglrenderer maintainers. Its graphics 
> origins do not by themselves motivate adding a separate backend 
> interface to QEMU.

Fair. The first draft version in virglrenderer was in May 2024, and ROCm 
has gone 5.7 → 7.14 in that window.

Regards,
Honglei

> 
> Regards,
> Akihiko Odaki
> 
>>
>> Regards,
>> Honglei
>>
>>
>>>
>>> [2] https://developers.redhat.com/articles/2025/06/05/how-we- 
>>> improved- ai-inference-macos-podman-containers
>>> [3] https://www.qualcomm.com/developer/blog/2024/10/vcl-virtio-gpu- 
>>> opencl-driver
>>>
>>> Regards,
>>> Akihiko Odaki
>>>
>>>>
>>>> That reuses the existing pluggable backend model, a second virtio 
>>>> gpu + a
>>>> backend library. It doesn't add dedicated queues for debug/profiling 
>>>> currently.
>>>>
>>>> Waiting for reply and  happy to share more detail. Thanks!
>>>>
>>>> [1] https://gitlab.freedesktop.org/virgl/virglrenderer/-/ 
>>>> merge_requests/1568
>>>>
>>>> Regards,
>>>> Honglei
>>>
>>
> 

Re: About new backend for GPU compute ROCm in qemu

[Akihiko Odaki]

On 2026/08/18 11:50, Huang, Honglei wrote:
> 
> 
> On 8/18/2026 12:29 AM, Akihiko Odaki wrote:
>> On 2026/08/17 22:44, Huang, Honglei wrote:
>>>
>>>
>>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>>
>>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>>>
>>>> Hi Honglei,
>>>>
>>>>>
>>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm 
>>>>> over virtio
>>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). The 
>>>>> ROCm side has
>>>>> been supportted by ROCm offical.
>>>>>
>>>>> Current implementation is a virtio gpu context type capset handled 
>>>>> inside
>>>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>>>> compute GPUs have no display engine at all.
>>>>
>>>> I think "sharing the display path" conflates several layers and 
>>>> makes the problem difficult to assess. It would help to identify the 
>>>> concrete constraint behind "awkward fit."
>>>>
>>>> End-to-end, there are four relevant layers:
>>>>
>>>> 1. Host GPU stack: hardware, host kernel, and host userspace
>>>> 2. Paravirtualization stack: virglrenderer and QEMU
>>>
>>> Yes we are asking can we add a new file like virtio-gpu specific for 
>>> compute, but maybe we can only add a new backend like virglrenderer 
>>> specific for compute.
>>>
>>>> 3. Host/guest interface: virtio and the capset-specific command stream
>>>
>>> In this plan we may need just add a capset id.
>>>
>>>> 4. Guest GPU stack: guest kernel and guest userspace
>>>
>>> Won't modify the guest kernel in this plan, this email list.
>>>
>>>>
>>>> Orthogonally, acceleration is separate from display and scanout. A
>>>> physical device may provide both, but acceleration does not require a
>>>> display engine. Linux likewise exposes render and compute interfaces
>>>> separately from modesetting. The userspace interface virglrenderer 
>>>> uses is messy; there is Vulkan, EGL, OpenGL, and now you are adding 
>>>> ROCm. But there is one thing I must note is that acceleration and 
>>>> display is decoupled, and acceleration does not require display.
>>>
>>> Yes totally agreed.
>>>
>>>>
>>>> At the protocol layer, context command buffers are carried by 
>>>> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
>>>> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no enabled 
>>>> displays. At the implementation layer, QEMU handles scanout 
>>>> presentation. virgl_cmd_set_scanout() obtains resource information 
>>>> through virgl_renderer_resource_get_info() or 
>>>> virgl_renderer_resource_get_info_ext(). That does not make scanout a 
>>>> virglrenderer-owned display path.
>>>
>>> Yes, agreed.
>>>
>>>>
>>>> Therefore, if "sharing the display path" means sharing the same 
>>>> device, control queue, and QEMU execution context, that identifies a 
>>>> possible source of contention. If it means that capsets or 
>>>> virglrenderer are inherently tied to display, I do not think that is 
>>>> accurate. Vulkan compute is already used through Venus with libkrun 
>>>> [2], and VCL proposes OpenCL support through virglrenderer [3].
>>>
>>> Yes，but the vulkan is for GFX originally, and for some formal AI 
>>> frame work like pytorch, it's support is limited, and it performance 
>>> is lower than ROCm, and vulkan also lacks many AI infrastructure, 
>>> like composable kernel.
>>> And for virCL, actually it is came from same project with ROCm native 
>>> context, but the original author didn't continue to support it, they 
>>> handed it over to someone else to take over. And in the first version of
>>> virCL, it didn't pass the test of actual projects.
>>>
>>> And it seems like virCL didn't upstream into virglrenderer also, 
>>> correct me if I am wrong.
>>
>> I cited Venus and VCL only as examples showing that virtio-gpu and
>> virglrenderer are not intrinsically tied to display. I did not suggest
>> either as a substitute for ROCm.
>>
>>>
>>>>> Beyond that, sharing the display path is increasingly painful:
>>>>>
>>>>>    - Compute hammers the queues more than graphics, so sharing
>>>>>      virtio gpu's single control queue with display/virgl causes 
>>>>> contention
>>>>>      and display stutter.
>>>>
>>>> All non-cursor commands do share one control queue, but a fence 
>>>> avoids serialization.
>>>
>>> yes, agreed.
>>>
>>>>
>>>> There may still be implementation-level contention, and it is not
>>>> necessarily specific to compute. A sufficiently busy graphics workload
>>>> could expose the same bottlenecks. Possible contributors in current 
>>>> QEMU
>>>> include:
>>>>
>>>> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>>>>     needs to fence scanout commands.
>>>
>>> Yes, agreed.
>>>
>>>>
>>>> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>>>>     queue just to delay one command.
>>>
>>> Yes, but it is seems like it is must, someone else in AMD tried to 
>>> use async method to relase blob, but it failed to consistency issue, 
>>> then
>>> reverted to sync version.
>>
>> Queue-wide suspension is not inherently required. Commit 4eb0aace85f5 
>> ("virtio-gpu: Support mapping hostmem blobs with map_fixed") added a 
>> path that avoids per-blob MemoryRegion teardown when 
>> virgl_renderer_resource_map_fixed() succeeds. The remaining path is 
>> also being improved with:
>>
>> https://lore.kernel.org/qemu-devel/20260424-force_rcu-v4-0- 
>> feccfaca0568@rsg.ci.i.u-tokyo.ac.jp/
>> ("[PATCH v4 0/6] virtio-gpu: Force RCU when unmapping blob")
> 
> Thanks. force_rcu is a clean fix for the RCU-reclamation part, but it 
> still keeps the unmap synchronous and serial.
> 
>>
>>>
>>>>
>>>> c) QEMU dispatches the control queue and calls into virglrenderer from
>>>>     its main-loop thread along with display work and many other things.
>>>>     Venus's render server can offload renderer work, but control-queue
>>>>     dispatch remains in QEMU's main loop.
>>>
>>> Yes, we did some async optimization in ROCm context, but its 
>>> effectiveness is limited, see bellow.
>>>
>>>>
>>>> In any case, I think you need to do some experiments to track down 
>>>> the real cause. a) is easy to check: just comment out all 
>>>> qemu_console_hw_gl_block() calls; it may corrupt display but removes 
>>>> the blocking. b) can also be tested by leaking the mappings instead 
>>>> of blocking the whole queue. Using a different display device like 
>>>> qxl tells whether c) is causing contention.
>>>
>>> Yes, totally agreed. following is my findings. In short words:
>>>
>>> Optimization can reduce queue pressure, but it can't withstand 
>>> absolute overload because each command has some overhead. Making all 
>>> commands asynchronous would lead to a debugging hell about 
>>> asynchronous issues.
>>> And we have high load applications rocmprofiler  that continuously 
>>> catch information need virtio queue to handle. But create a new 
>>> backend can not solve it simply, we are trying to find a way. like 
>>> shmem between guest and host, then use cpu polling, bypass the 
>>> virtqueue.
>>
>> Most commands are fast on the CPU side, while heavy processing
>> happens asynchronously on the GPU. Cases (a) and (b) are exceptions.
>>
>>>
>>> The load is mostly memory management. Running an AI model allocates 
>>> and frees a large number of blobs. We already did some optimization 
>>> release them asynchronously, but the host processing is a single 
>>> queue one
>>> process_cmdq, this is where the main bottleneck in my debugging 
>>> work / my understanding so far. I'm not certain it's the whole 
>>> picture, so please correct if I am wrong.
>>>
>>> A model load or unload frees a large batch of BOs and allocates 
>>> another. Some of those commands are async in the virtio-gpu guest 
>>> driver, but QEMU still has to work through them on the one queue, 
>>> which takes time; so even though any single command is quick, there 
>>> are simply too many of them, the single queue backs up, and 
>>> everything behind it, gets delayed.
>>>
>>> real work load (a few downstream customisations): loading one 16 GB 
>>> model (gemm4 e4b), drives ~1200 blob creates, a burst of
>>> ~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
>>> notifies, caused a 22 s guest soft lockup. And the behavior of memory 
>>> operations are controlled by upper layer like pytorch / HIP / runtime,
>>> we can not control it.
>>>
>>> To be honest, a separate backend won't fix this. But the real 
>>> solution maybe is compute specific. That logic is only useful to the 
>>> compute path, and folding it into the shared display device / 
>>> renderer would mean churning code that is mature and stable for 
>>> graphics, with regression risk. Keeping compute on its own instance 
>>> and backend lets us iterate on these compute only optimisations.
>>
>> A 22-second lockup is too long for those command counts.
>>
>> The most probable explanation I have is that the ROCm integration 
>> blocks QEMU's main loop thread while synchronously waiting for GPU 
>> execution. Creating separate devices won't resolve this because the 
>> main loop thread is shared, and synchronously waiting on the GPU 
>> should be avoided in the first place.
> 
> No synchronously waiting in ROCm backend, we are using user queue, and 
> event waiting, no sync operation in CMD wait. all the resource release 
> in ROCm are all async now.
> Only the sync thing is memory thing mapping/unmapping in qemu, as long 
> as it remains synchronous, it will be overwhelmed by the massive number 
> of requests.

Mapping and unmapping should not block QEMU's main-loop thread for that
long. The command counts you reported are relatively small. That is why
I suspect something else went wrong, such as the main-loop thread being
inadvertently blocked while waiting for the GPU.

> 
>>
>> In any case, profiling is necessary before touching the implementation.
>>
>>>
>>>>
>>>>>    - Compute contexts need far more blob / shared memory than a 
>>>>> display one.
>>>>
>>>> It is not a problem by itself. Frequent mapping and unmapping might 
>>>> amplify the second issue above, but that needs to be measured.
>>>
>>> Yes, agreed. I can give more detailed information.
>>>
>>>>
>>>>>    - Maybe needs a wider ROCm / compute stack, cause the render 
>>>>> model fits poorly:
>>>>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth 
>>>>> streams)
>>>>>      and ROCgdb (wave control, address watch, async exceptions an
>>>>>      out of band channel that must not block display).
>>>>
>>>> virglrenderer does not impose a particular render model. That's why 
>>>> Vulkan Compute just works with Venus.
>>>
>>> Yes but vulkan is used for GFX initally. And can not support many AI 
>>> application.>
>>>>>    - Events, faults and GPU reset/SMI are async and don't map onto 
>>>>> fences.>    - All of this is hard to extend cleanly inside a 
>>>>> display capset.
>>>> Capset is not about display but determines the protocol of the 
>>>> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described events, 
>>>> faults and GPU reset/SMI are async don't map onto fences that may be 
>>>> associated with VIRTIO_GPU_CMD_SUBMIT_3D which is dictated by 
>>>> capset. An additional feature may be necessary, and it may or may 
>>>> not be dictated by capset. The other things are irrelevant with the 
>>>> protocol capset represents; they are either behavioral or about 
>>>> different commands.
>>>
>>> A fence is the one shot, but event is stateful and repeatable.
>>> That may or may not be tied to capset. Agreed.
>>>
>>>>
>>>>>
>>>>> On the QEMU/host side, would something like this be OK? One step, 
>>>>> two parts:
>>>>>
>>>>>    - a dedicated headless virtio gpu instance for compute.
>>>>
>>>> A second device would isolate its virtqueues and device-wide
>>>> renderer_blocked state. That may be useful if measurements show that
>>>> these are the bottlenecks, but it is not yet clear that they are or 
>>>> that
>>>> a second device is the appropriate solution.
>>>>
>>>>>    - that instance served by a separate ROCm backend library loaded
>>>>>      in-process by QEMU.
>>>>
>>>> First, I think we need to establish why ROCm cannot or should not 
>>>> remain
>>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>>> likely better placed to advise on that boundary. Once the protocol
>>>> requirements and performance measurements are clear, we can assess the
>>>> appropriate QEMU integration.
>>>
>>> venus is borned for GFX.
>>> virCL not merged.
>>>
>>> To be clear, I'm not saying virglrenderer can't host a ROCm native 
>>> context it clearly can. My hesitation is more about fit and 
>>> direction: virglrenderer has grown up around GL/graphics, and I 
>>> haven't yet found compute oriented plumbing there to build on, while 
>>> ROCm moves very fast and I need something I can keep current with low 
>>> friction.
>>
>> Whether keeping ROCm in virglrenderer would create extra friction is 
>> primarily a question for the virglrenderer maintainers. Its graphics 
>> origins do not by themselves motivate adding a separate backend 
>> interface to QEMU.
> 
> Fair. The first draft version in virglrenderer was in May 2024, and ROCm 
> has gone 5.7 → 7.14 in that window.

One point to note is that virtio-gpu development in QEMU is somewhat
less active. crosvm is the most active user of virglrenderer, and QEMU
sometimes lags behind it. If you are considering moving the ROCm 
integration from virglrenderer to QEMU solely because ROCm evolves 
rapidly, I do not think that would be a good idea. A rapidly evolving 
component is better kept in virglrenderer unless there is another reason 
to place it in QEMU.

Regards,
Akihiko Odaki

Re: About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

On 8/18/2026 12:05 PM, Akihiko Odaki wrote:
> On 2026/08/18 11:50, Huang, Honglei wrote:
>>
>>
>> On 8/18/2026 12:29 AM, Akihiko Odaki wrote:
>>> On 2026/08/17 22:44, Huang, Honglei wrote:
>>>>
>>>>
>>>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>>>
>>>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>>>>
>>>>> Hi Honglei,
>>>>>
>>>>>>
>>>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a ROCm 
>>>>>> over virtio
>>>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). 
>>>>>> The ROCm side has
>>>>>> been supportted by ROCm offical.
>>>>>>
>>>>>> Current implementation is a virtio gpu context type capset handled 
>>>>>> inside
>>>>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>>>>> compute GPUs have no display engine at all.
>>>>>
>>>>> I think "sharing the display path" conflates several layers and 
>>>>> makes the problem difficult to assess. It would help to identify 
>>>>> the concrete constraint behind "awkward fit."
>>>>>
>>>>> End-to-end, there are four relevant layers:
>>>>>
>>>>> 1. Host GPU stack: hardware, host kernel, and host userspace
>>>>> 2. Paravirtualization stack: virglrenderer and QEMU
>>>>
>>>> Yes we are asking can we add a new file like virtio-gpu specific for 
>>>> compute, but maybe we can only add a new backend like virglrenderer 
>>>> specific for compute.
>>>>
>>>>> 3. Host/guest interface: virtio and the capset-specific command stream
>>>>
>>>> In this plan we may need just add a capset id.
>>>>
>>>>> 4. Guest GPU stack: guest kernel and guest userspace
>>>>
>>>> Won't modify the guest kernel in this plan, this email list.
>>>>
>>>>>
>>>>> Orthogonally, acceleration is separate from display and scanout. A
>>>>> physical device may provide both, but acceleration does not require a
>>>>> display engine. Linux likewise exposes render and compute interfaces
>>>>> separately from modesetting. The userspace interface virglrenderer 
>>>>> uses is messy; there is Vulkan, EGL, OpenGL, and now you are adding 
>>>>> ROCm. But there is one thing I must note is that acceleration and 
>>>>> display is decoupled, and acceleration does not require display.
>>>>
>>>> Yes totally agreed.
>>>>
>>>>>
>>>>> At the protocol layer, context command buffers are carried by 
>>>>> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
>>>>> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no enabled 
>>>>> displays. At the implementation layer, QEMU handles scanout 
>>>>> presentation. virgl_cmd_set_scanout() obtains resource information 
>>>>> through virgl_renderer_resource_get_info() or 
>>>>> virgl_renderer_resource_get_info_ext(). That does not make scanout 
>>>>> a virglrenderer-owned display path.
>>>>
>>>> Yes, agreed.
>>>>
>>>>>
>>>>> Therefore, if "sharing the display path" means sharing the same 
>>>>> device, control queue, and QEMU execution context, that identifies 
>>>>> a possible source of contention. If it means that capsets or 
>>>>> virglrenderer are inherently tied to display, I do not think that 
>>>>> is accurate. Vulkan compute is already used through Venus with 
>>>>> libkrun [2], and VCL proposes OpenCL support through virglrenderer 
>>>>> [3].
>>>>
>>>> Yes，but the vulkan is for GFX originally, and for some formal AI 
>>>> frame work like pytorch, it's support is limited, and it performance 
>>>> is lower than ROCm, and vulkan also lacks many AI infrastructure, 
>>>> like composable kernel.
>>>> And for virCL, actually it is came from same project with ROCm 
>>>> native context, but the original author didn't continue to support 
>>>> it, they handed it over to someone else to take over. And in the 
>>>> first version of
>>>> virCL, it didn't pass the test of actual projects.
>>>>
>>>> And it seems like virCL didn't upstream into virglrenderer also, 
>>>> correct me if I am wrong.
>>>
>>> I cited Venus and VCL only as examples showing that virtio-gpu and
>>> virglrenderer are not intrinsically tied to display. I did not suggest
>>> either as a substitute for ROCm.
>>>
>>>>
>>>>>> Beyond that, sharing the display path is increasingly painful:
>>>>>>
>>>>>>    - Compute hammers the queues more than graphics, so sharing
>>>>>>      virtio gpu's single control queue with display/virgl causes 
>>>>>> contention
>>>>>>      and display stutter.
>>>>>
>>>>> All non-cursor commands do share one control queue, but a fence 
>>>>> avoids serialization.
>>>>
>>>> yes, agreed.
>>>>
>>>>>
>>>>> There may still be implementation-level contention, and it is not
>>>>> necessarily specific to compute. A sufficiently busy graphics workload
>>>>> could expose the same bottlenecks. Possible contributors in current 
>>>>> QEMU
>>>>> include:
>>>>>
>>>>> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>>>>>     needs to fence scanout commands.
>>>>
>>>> Yes, agreed.
>>>>
>>>>>
>>>>> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>>>>>     queue just to delay one command.
>>>>
>>>> Yes, but it is seems like it is must, someone else in AMD tried to 
>>>> use async method to relase blob, but it failed to consistency issue, 
>>>> then
>>>> reverted to sync version.
>>>
>>> Queue-wide suspension is not inherently required. Commit 4eb0aace85f5 
>>> ("virtio-gpu: Support mapping hostmem blobs with map_fixed") added a 
>>> path that avoids per-blob MemoryRegion teardown when 
>>> virgl_renderer_resource_map_fixed() succeeds. The remaining path is 
>>> also being improved with:
>>>
>>> https://lore.kernel.org/qemu-devel/20260424-force_rcu-v4-0- 
>>> feccfaca0568@rsg.ci.i.u-tokyo.ac.jp/
>>> ("[PATCH v4 0/6] virtio-gpu: Force RCU when unmapping blob")
>>
>> Thanks. force_rcu is a clean fix for the RCU-reclamation part, but it 
>> still keeps the unmap synchronous and serial.
>>
>>>
>>>>
>>>>>
>>>>> c) QEMU dispatches the control queue and calls into virglrenderer from
>>>>>     its main-loop thread along with display work and many other 
>>>>> things.
>>>>>     Venus's render server can offload renderer work, but control-queue
>>>>>     dispatch remains in QEMU's main loop.
>>>>
>>>> Yes, we did some async optimization in ROCm context, but its 
>>>> effectiveness is limited, see bellow.
>>>>
>>>>>
>>>>> In any case, I think you need to do some experiments to track down 
>>>>> the real cause. a) is easy to check: just comment out all 
>>>>> qemu_console_hw_gl_block() calls; it may corrupt display but 
>>>>> removes the blocking. b) can also be tested by leaking the mappings 
>>>>> instead of blocking the whole queue. Using a different display 
>>>>> device like qxl tells whether c) is causing contention.
>>>>
>>>> Yes, totally agreed. following is my findings. In short words:
>>>>
>>>> Optimization can reduce queue pressure, but it can't withstand 
>>>> absolute overload because each command has some overhead. Making all 
>>>> commands asynchronous would lead to a debugging hell about 
>>>> asynchronous issues.
>>>> And we have high load applications rocmprofiler  that continuously 
>>>> catch information need virtio queue to handle. But create a new 
>>>> backend can not solve it simply, we are trying to find a way. like 
>>>> shmem between guest and host, then use cpu polling, bypass the 
>>>> virtqueue.
>>>
>>> Most commands are fast on the CPU side, while heavy processing
>>> happens asynchronously on the GPU. Cases (a) and (b) are exceptions.
>>>
>>>>
>>>> The load is mostly memory management. Running an AI model allocates 
>>>> and frees a large number of blobs. We already did some optimization 
>>>> release them asynchronously, but the host processing is a single 
>>>> queue one
>>>> process_cmdq, this is where the main bottleneck in my debugging 
>>>> work / my understanding so far. I'm not certain it's the whole 
>>>> picture, so please correct if I am wrong.
>>>>
>>>> A model load or unload frees a large batch of BOs and allocates 
>>>> another. Some of those commands are async in the virtio-gpu guest 
>>>> driver, but QEMU still has to work through them on the one queue, 
>>>> which takes time; so even though any single command is quick, there 
>>>> are simply too many of them, the single queue backs up, and 
>>>> everything behind it, gets delayed.
>>>>
>>>> real work load (a few downstream customisations): loading one 16 GB 
>>>> model (gemm4 e4b), drives ~1200 blob creates, a burst of
>>>> ~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
>>>> notifies, caused a 22 s guest soft lockup. And the behavior of 
>>>> memory operations are controlled by upper layer like pytorch / HIP / 
>>>> runtime,
>>>> we can not control it.
>>>>
>>>> To be honest, a separate backend won't fix this. But the real 
>>>> solution maybe is compute specific. That logic is only useful to the 
>>>> compute path, and folding it into the shared display device / 
>>>> renderer would mean churning code that is mature and stable for 
>>>> graphics, with regression risk. Keeping compute on its own instance 
>>>> and backend lets us iterate on these compute only optimisations.
>>>
>>> A 22-second lockup is too long for those command counts.
>>>
>>> The most probable explanation I have is that the ROCm integration 
>>> blocks QEMU's main loop thread while synchronously waiting for GPU 
>>> execution. Creating separate devices won't resolve this because the 
>>> main loop thread is shared, and synchronously waiting on the GPU 
>>> should be avoided in the first place.
>>
>> No synchronously waiting in ROCm backend, we are using user queue, and 
>> event waiting, no sync operation in CMD wait. all the resource release 
>> in ROCm are all async now.
>> Only the sync thing is memory thing mapping/unmapping in qemu, as long 
>> as it remains synchronous, it will be overwhelmed by the massive 
>> number of requests.
> 
> Mapping and unmapping should not block QEMU's main-loop thread for that
> long. The command counts you reported are relatively small. That is why
> I suspect something else went wrong, such as the main-loop thread being
> inadvertently blocked while waiting for the GPU.

Will investigate it.

> 
>>
>>>
>>> In any case, profiling is necessary before touching the implementation.
>>>
>>>>
>>>>>
>>>>>>    - Compute contexts need far more blob / shared memory than a 
>>>>>> display one.
>>>>>
>>>>> It is not a problem by itself. Frequent mapping and unmapping might 
>>>>> amplify the second issue above, but that needs to be measured.
>>>>
>>>> Yes, agreed. I can give more detailed information.
>>>>
>>>>>
>>>>>>    - Maybe needs a wider ROCm / compute stack, cause the render 
>>>>>> model fits poorly:
>>>>>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth 
>>>>>> streams)
>>>>>>      and ROCgdb (wave control, address watch, async exceptions an
>>>>>>      out of band channel that must not block display).
>>>>>
>>>>> virglrenderer does not impose a particular render model. That's why 
>>>>> Vulkan Compute just works with Venus.
>>>>
>>>> Yes but vulkan is used for GFX initally. And can not support many AI 
>>>> application.>
>>>>>>    - Events, faults and GPU reset/SMI are async and don't map onto 
>>>>>> fences.>    - All of this is hard to extend cleanly inside a 
>>>>>> display capset.
>>>>> Capset is not about display but determines the protocol of the 
>>>>> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described events, 
>>>>> faults and GPU reset/SMI are async don't map onto fences that may 
>>>>> be associated with VIRTIO_GPU_CMD_SUBMIT_3D which is dictated by 
>>>>> capset. An additional feature may be necessary, and it may or may 
>>>>> not be dictated by capset. The other things are irrelevant with the 
>>>>> protocol capset represents; they are either behavioral or about 
>>>>> different commands.
>>>>
>>>> A fence is the one shot, but event is stateful and repeatable.
>>>> That may or may not be tied to capset. Agreed.
>>>>
>>>>>
>>>>>>
>>>>>> On the QEMU/host side, would something like this be OK? One step, 
>>>>>> two parts:
>>>>>>
>>>>>>    - a dedicated headless virtio gpu instance for compute.
>>>>>
>>>>> A second device would isolate its virtqueues and device-wide
>>>>> renderer_blocked state. That may be useful if measurements show that
>>>>> these are the bottlenecks, but it is not yet clear that they are or 
>>>>> that
>>>>> a second device is the appropriate solution.
>>>>>
>>>>>>    - that instance served by a separate ROCm backend library loaded
>>>>>>      in-process by QEMU.
>>>>>
>>>>> First, I think we need to establish why ROCm cannot or should not 
>>>>> remain
>>>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>>>> likely better placed to advise on that boundary. Once the protocol
>>>>> requirements and performance measurements are clear, we can assess the
>>>>> appropriate QEMU integration.
>>>>
>>>> venus is borned for GFX.
>>>> virCL not merged.
>>>>
>>>> To be clear, I'm not saying virglrenderer can't host a ROCm native 
>>>> context it clearly can. My hesitation is more about fit and 
>>>> direction: virglrenderer has grown up around GL/graphics, and I 
>>>> haven't yet found compute oriented plumbing there to build on, while 
>>>> ROCm moves very fast and I need something I can keep current with 
>>>> low friction.
>>>
>>> Whether keeping ROCm in virglrenderer would create extra friction is 
>>> primarily a question for the virglrenderer maintainers. Its graphics 
>>> origins do not by themselves motivate adding a separate backend 
>>> interface to QEMU.
>>
>> Fair. The first draft version in virglrenderer was in May 2024, and 
>> ROCm has gone 5.7 → 7.14 in that window.
> 
> One point to note is that virtio-gpu development in QEMU is somewhat
> less active. crosvm is the most active user of virglrenderer, and QEMU
> sometimes lags behind it. If you are considering moving the ROCm 
> integration from virglrenderer to QEMU solely because ROCm evolves 
> rapidly, I do not think that would be a good idea. A rapidly evolving 
> component is better kept in virglrenderer unless there is another reason 
> to place it in QEMU.

Actually didn't see something new about compute merged in to 
virglrenderer this recently 2 years.

Regards,
Honglei

> 
> Regards,
> Akihiko Odaki

Re: About new backend for GPU compute ROCm in qemu

[Akihiko Odaki]

On 2026/08/18 13:26, Huang, Honglei wrote:
> 
> 
> On 8/18/2026 12:05 PM, Akihiko Odaki wrote:
>> On 2026/08/18 11:50, Huang, Honglei wrote:
>>>
>>>
>>> On 8/18/2026 12:29 AM, Akihiko Odaki wrote:
>>>> On 2026/08/17 22:44, Huang, Honglei wrote:
>>>>>
>>>>>
>>>>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>>>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>>>>
>>>>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>>>>>
>>>>>> Hi Honglei,
>>>>>>
>>>>>>>
>>>>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a 
>>>>>>> ROCm over virtio
>>>>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). 
>>>>>>> The ROCm side has
>>>>>>> been supportted by ROCm offical.
>>>>>>>
>>>>>>> Current implementation is a virtio gpu context type capset 
>>>>>>> handled inside
>>>>>>> virglrenderer, sharing the display path. That's an awkward fit, many
>>>>>>> compute GPUs have no display engine at all.
>>>>>>
>>>>>> I think "sharing the display path" conflates several layers and 
>>>>>> makes the problem difficult to assess. It would help to identify 
>>>>>> the concrete constraint behind "awkward fit."
>>>>>>
>>>>>> End-to-end, there are four relevant layers:
>>>>>>
>>>>>> 1. Host GPU stack: hardware, host kernel, and host userspace
>>>>>> 2. Paravirtualization stack: virglrenderer and QEMU
>>>>>
>>>>> Yes we are asking can we add a new file like virtio-gpu specific 
>>>>> for compute, but maybe we can only add a new backend like 
>>>>> virglrenderer specific for compute.
>>>>>
>>>>>> 3. Host/guest interface: virtio and the capset-specific command 
>>>>>> stream
>>>>>
>>>>> In this plan we may need just add a capset id.
>>>>>
>>>>>> 4. Guest GPU stack: guest kernel and guest userspace
>>>>>
>>>>> Won't modify the guest kernel in this plan, this email list.
>>>>>
>>>>>>
>>>>>> Orthogonally, acceleration is separate from display and scanout. A
>>>>>> physical device may provide both, but acceleration does not require a
>>>>>> display engine. Linux likewise exposes render and compute interfaces
>>>>>> separately from modesetting. The userspace interface virglrenderer 
>>>>>> uses is messy; there is Vulkan, EGL, OpenGL, and now you are 
>>>>>> adding ROCm. But there is one thing I must note is that 
>>>>>> acceleration and display is decoupled, and acceleration does not 
>>>>>> require display.
>>>>>
>>>>> Yes totally agreed.
>>>>>
>>>>>>
>>>>>> At the protocol layer, context command buffers are carried by 
>>>>>> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
>>>>>> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no 
>>>>>> enabled displays. At the implementation layer, QEMU handles 
>>>>>> scanout presentation. virgl_cmd_set_scanout() obtains resource 
>>>>>> information through virgl_renderer_resource_get_info() or 
>>>>>> virgl_renderer_resource_get_info_ext(). That does not make scanout 
>>>>>> a virglrenderer-owned display path.
>>>>>
>>>>> Yes, agreed.
>>>>>
>>>>>>
>>>>>> Therefore, if "sharing the display path" means sharing the same 
>>>>>> device, control queue, and QEMU execution context, that identifies 
>>>>>> a possible source of contention. If it means that capsets or 
>>>>>> virglrenderer are inherently tied to display, I do not think that 
>>>>>> is accurate. Vulkan compute is already used through Venus with 
>>>>>> libkrun [2], and VCL proposes OpenCL support through virglrenderer 
>>>>>> [3].
>>>>>
>>>>> Yes，but the vulkan is for GFX originally, and for some formal AI 
>>>>> frame work like pytorch, it's support is limited, and it 
>>>>> performance is lower than ROCm, and vulkan also lacks many AI 
>>>>> infrastructure, like composable kernel.
>>>>> And for virCL, actually it is came from same project with ROCm 
>>>>> native context, but the original author didn't continue to support 
>>>>> it, they handed it over to someone else to take over. And in the 
>>>>> first version of
>>>>> virCL, it didn't pass the test of actual projects.
>>>>>
>>>>> And it seems like virCL didn't upstream into virglrenderer also, 
>>>>> correct me if I am wrong.
>>>>
>>>> I cited Venus and VCL only as examples showing that virtio-gpu and
>>>> virglrenderer are not intrinsically tied to display. I did not suggest
>>>> either as a substitute for ROCm.
>>>>
>>>>>
>>>>>>> Beyond that, sharing the display path is increasingly painful:
>>>>>>>
>>>>>>>    - Compute hammers the queues more than graphics, so sharing
>>>>>>>      virtio gpu's single control queue with display/virgl causes 
>>>>>>> contention
>>>>>>>      and display stutter.
>>>>>>
>>>>>> All non-cursor commands do share one control queue, but a fence 
>>>>>> avoids serialization.
>>>>>
>>>>> yes, agreed.
>>>>>
>>>>>>
>>>>>> There may still be implementation-level contention, and it is not
>>>>>> necessarily specific to compute. A sufficiently busy graphics 
>>>>>> workload
>>>>>> could expose the same bottlenecks. Possible contributors in 
>>>>>> current QEMU
>>>>>> include:
>>>>>>
>>>>>> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>>>>>>     needs to fence scanout commands.
>>>>>
>>>>> Yes, agreed.
>>>>>
>>>>>>
>>>>>> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>>>>>>     queue just to delay one command.
>>>>>
>>>>> Yes, but it is seems like it is must, someone else in AMD tried to 
>>>>> use async method to relase blob, but it failed to consistency 
>>>>> issue, then
>>>>> reverted to sync version.
>>>>
>>>> Queue-wide suspension is not inherently required. Commit 
>>>> 4eb0aace85f5 ("virtio-gpu: Support mapping hostmem blobs with 
>>>> map_fixed") added a path that avoids per-blob MemoryRegion teardown 
>>>> when virgl_renderer_resource_map_fixed() succeeds. The remaining 
>>>> path is also being improved with:
>>>>
>>>> https://lore.kernel.org/qemu-devel/20260424-force_rcu-v4-0- 
>>>> feccfaca0568@rsg.ci.i.u-tokyo.ac.jp/
>>>> ("[PATCH v4 0/6] virtio-gpu: Force RCU when unmapping blob")
>>>
>>> Thanks. force_rcu is a clean fix for the RCU-reclamation part, but it 
>>> still keeps the unmap synchronous and serial.
>>>
>>>>
>>>>>
>>>>>>
>>>>>> c) QEMU dispatches the control queue and calls into virglrenderer 
>>>>>> from
>>>>>>     its main-loop thread along with display work and many other 
>>>>>> things.
>>>>>>     Venus's render server can offload renderer work, but control- 
>>>>>> queue
>>>>>>     dispatch remains in QEMU's main loop.
>>>>>
>>>>> Yes, we did some async optimization in ROCm context, but its 
>>>>> effectiveness is limited, see bellow.
>>>>>
>>>>>>
>>>>>> In any case, I think you need to do some experiments to track down 
>>>>>> the real cause. a) is easy to check: just comment out all 
>>>>>> qemu_console_hw_gl_block() calls; it may corrupt display but 
>>>>>> removes the blocking. b) can also be tested by leaking the 
>>>>>> mappings instead of blocking the whole queue. Using a different 
>>>>>> display device like qxl tells whether c) is causing contention.
>>>>>
>>>>> Yes, totally agreed. following is my findings. In short words:
>>>>>
>>>>> Optimization can reduce queue pressure, but it can't withstand 
>>>>> absolute overload because each command has some overhead. Making 
>>>>> all commands asynchronous would lead to a debugging hell about 
>>>>> asynchronous issues.
>>>>> And we have high load applications rocmprofiler  that continuously 
>>>>> catch information need virtio queue to handle. But create a new 
>>>>> backend can not solve it simply, we are trying to find a way. like 
>>>>> shmem between guest and host, then use cpu polling, bypass the 
>>>>> virtqueue.
>>>>
>>>> Most commands are fast on the CPU side, while heavy processing
>>>> happens asynchronously on the GPU. Cases (a) and (b) are exceptions.
>>>>
>>>>>
>>>>> The load is mostly memory management. Running an AI model allocates 
>>>>> and frees a large number of blobs. We already did some optimization 
>>>>> release them asynchronously, but the host processing is a single 
>>>>> queue one
>>>>> process_cmdq, this is where the main bottleneck in my debugging 
>>>>> work / my understanding so far. I'm not certain it's the whole 
>>>>> picture, so please correct if I am wrong.
>>>>>
>>>>> A model load or unload frees a large batch of BOs and allocates 
>>>>> another. Some of those commands are async in the virtio-gpu guest 
>>>>> driver, but QEMU still has to work through them on the one queue, 
>>>>> which takes time; so even though any single command is quick, there 
>>>>> are simply too many of them, the single queue backs up, and 
>>>>> everything behind it, gets delayed.
>>>>>
>>>>> real work load (a few downstream customisations): loading one 16 GB 
>>>>> model (gemm4 e4b), drives ~1200 blob creates, a burst of
>>>>> ~1400 resource frees at teardown, ~3700 submits and ~6000 virtqueue 
>>>>> notifies, caused a 22 s guest soft lockup. And the behavior of 
>>>>> memory operations are controlled by upper layer like pytorch / 
>>>>> HIP / runtime,
>>>>> we can not control it.
>>>>>
>>>>> To be honest, a separate backend won't fix this. But the real 
>>>>> solution maybe is compute specific. That logic is only useful to 
>>>>> the compute path, and folding it into the shared display device / 
>>>>> renderer would mean churning code that is mature and stable for 
>>>>> graphics, with regression risk. Keeping compute on its own instance 
>>>>> and backend lets us iterate on these compute only optimisations.
>>>>
>>>> A 22-second lockup is too long for those command counts.
>>>>
>>>> The most probable explanation I have is that the ROCm integration 
>>>> blocks QEMU's main loop thread while synchronously waiting for GPU 
>>>> execution. Creating separate devices won't resolve this because the 
>>>> main loop thread is shared, and synchronously waiting on the GPU 
>>>> should be avoided in the first place.
>>>
>>> No synchronously waiting in ROCm backend, we are using user queue, 
>>> and event waiting, no sync operation in CMD wait. all the resource 
>>> release in ROCm are all async now.
>>> Only the sync thing is memory thing mapping/unmapping in qemu, as 
>>> long as it remains synchronous, it will be overwhelmed by the massive 
>>> number of requests.
>>
>> Mapping and unmapping should not block QEMU's main-loop thread for that
>> long. The command counts you reported are relatively small. That is why
>> I suspect something else went wrong, such as the main-loop thread being
>> inadvertently blocked while waiting for the GPU.
> 
> Will investigate it.
> 
>>
>>>
>>>>
>>>> In any case, profiling is necessary before touching the implementation.
>>>>
>>>>>
>>>>>>
>>>>>>>    - Compute contexts need far more blob / shared memory than a 
>>>>>>> display one.
>>>>>>
>>>>>> It is not a problem by itself. Frequent mapping and unmapping 
>>>>>> might amplify the second issue above, but that needs to be measured.
>>>>>
>>>>> Yes, agreed. I can give more detailed information.
>>>>>
>>>>>>
>>>>>>>    - Maybe needs a wider ROCm / compute stack, cause the render 
>>>>>>> model fits poorly:
>>>>>>>      rocprofiler (PC sampling, SQTT/SPM, counters, high bandwidth 
>>>>>>> streams)
>>>>>>>      and ROCgdb (wave control, address watch, async exceptions an
>>>>>>>      out of band channel that must not block display).
>>>>>>
>>>>>> virglrenderer does not impose a particular render model. That's 
>>>>>> why Vulkan Compute just works with Venus.
>>>>>
>>>>> Yes but vulkan is used for GFX initally. And can not support many 
>>>>> AI application.>
>>>>>>>    - Events, faults and GPU reset/SMI are async and don't map 
>>>>>>> onto fences.>    - All of this is hard to extend cleanly inside a 
>>>>>>> display capset.
>>>>>> Capset is not about display but determines the protocol of the 
>>>>>> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described 
>>>>>> events, faults and GPU reset/SMI are async don't map onto fences 
>>>>>> that may be associated with VIRTIO_GPU_CMD_SUBMIT_3D which is 
>>>>>> dictated by capset. An additional feature may be necessary, and it 
>>>>>> may or may not be dictated by capset. The other things are 
>>>>>> irrelevant with the protocol capset represents; they are either 
>>>>>> behavioral or about different commands.
>>>>>
>>>>> A fence is the one shot, but event is stateful and repeatable.
>>>>> That may or may not be tied to capset. Agreed.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> On the QEMU/host side, would something like this be OK? One step, 
>>>>>>> two parts:
>>>>>>>
>>>>>>>    - a dedicated headless virtio gpu instance for compute.
>>>>>>
>>>>>> A second device would isolate its virtqueues and device-wide
>>>>>> renderer_blocked state. That may be useful if measurements show that
>>>>>> these are the bottlenecks, but it is not yet clear that they are 
>>>>>> or that
>>>>>> a second device is the appropriate solution.
>>>>>>
>>>>>>>    - that instance served by a separate ROCm backend library loaded
>>>>>>>      in-process by QEMU.
>>>>>>
>>>>>> First, I think we need to establish why ROCm cannot or should not 
>>>>>> remain
>>>>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>>>>> likely better placed to advise on that boundary. Once the protocol
>>>>>> requirements and performance measurements are clear, we can assess 
>>>>>> the
>>>>>> appropriate QEMU integration.
>>>>>
>>>>> venus is borned for GFX.
>>>>> virCL not merged.
>>>>>
>>>>> To be clear, I'm not saying virglrenderer can't host a ROCm native 
>>>>> context it clearly can. My hesitation is more about fit and 
>>>>> direction: virglrenderer has grown up around GL/graphics, and I 
>>>>> haven't yet found compute oriented plumbing there to build on, 
>>>>> while ROCm moves very fast and I need something I can keep current 
>>>>> with low friction.
>>>>
>>>> Whether keeping ROCm in virglrenderer would create extra friction is 
>>>> primarily a question for the virglrenderer maintainers. Its graphics 
>>>> origins do not by themselves motivate adding a separate backend 
>>>> interface to QEMU.
>>>
>>> Fair. The first draft version in virglrenderer was in May 2024, and 
>>> ROCm has gone 5.7 → 7.14 in that window.
>>
>> One point to note is that virtio-gpu development in QEMU is somewhat
>> less active. crosvm is the most active user of virglrenderer, and QEMU
>> sometimes lags behind it. If you are considering moving the ROCm 
>> integration from virglrenderer to QEMU solely because ROCm evolves 
>> rapidly, I do not think that would be a good idea. A rapidly evolving 
>> component is better kept in virglrenderer unless there is another 
>> reason to place it in QEMU.
> 
> Actually didn't see something new about compute merged in to 
> virglrenderer this recently 2 years.

Neither QEMU nor virglrenderer has seen new compute-related additions in 
the past two years.

While you have regularly updated the merge request, initiating 
discussions around it is necessary to move review forward. Open-source 
projects like QEMU and virglrenderer need proactive driving to complete 
reviews. Simply shifting the ROCm integration to QEMU will not resolve 
this bottleneck.

Besides, looking at the "Architecture Components" in the description, 
most of them haven't been merged yet. The virglrenderer code cannot be 
merged in its current state, so focusing on those dependencies first is 
essential.

However, taking a naive approach can lead to a chicken-and-egg problem: 
component maintainers want the virglrenderer side stabilized first, 
while virglrenderer maintainers want the component side stabilized. To 
break this deadlock, I suggest seeking consensus on the interfaces 
before completing the implementation. Once an interface agreement is 
reached, changes to each component can land independently:

- virtio interface: I raised a concern regarding the interface [1][2]
   that needs to be addressed.
- amdkfd patches: There are interface-level concerns [3] that still need
   resolution.
- ROCm runtime: The description lists this as "90% complete," but the
   linked pull requests were closed due to inactivity. They need to be
   reopened and seek for a consensus on its interface.

Once these items are addressed, you can update the merge request and ask 
for a fresh review.

In parallel, you can request review of self-contained parts of 
components that do not depend on those decisions. Once the interfaces 
are agreed, the implementations can be reviewed in parallel and merged 
in dependency order.

These steps can proceed in parallel alongside investigating the 
performance bottleneck.

Regards,
Akihiko Odaki

[1] 
https://lore.kernel.org/lkml/b69439ec-0ebd-4527-873b-85b283e03888@rsg.ci.i.u-tokyo.ac.jp/
[2] 
https://lore.kernel.org/qemu-devel/35a8add7-da49-4833-9e69-d213f52c771a@amd.com/
[3] 
https://lore.kernel.org/lkml/20260104072122.3045656-1-honglei1.huang@amd.com/
[4] https://www.spinics.net/lists/amd-gfx/msg137231.html

Re: About new backend for GPU compute ROCm in qemu

[Huang, Honglei]

On 8/18/2026 3:50 PM, Akihiko Odaki wrote:
> On 2026/08/18 13:26, Huang, Honglei wrote:
>>
>>
>> On 8/18/2026 12:05 PM, Akihiko Odaki wrote:
>>> On 2026/08/18 11:50, Huang, Honglei wrote:
>>>>
>>>>
>>>> On 8/18/2026 12:29 AM, Akihiko Odaki wrote:
>>>>> On 2026/08/17 22:44, Huang, Honglei wrote:
>>>>>>
>>>>>>
>>>>>> On 8/17/2026 7:44 PM, Akihiko Odaki wrote:
>>>>>>> On 2026/08/17 12:19, Huang, Honglei wrote:
>>>>>>>>
>>>>>>>> Hi Michael, Alex, Dmitry, Akihiko,
>>>>>>>
>>>>>>> Hi Honglei,
>>>>>>>
>>>>>>>>
>>>>>>>> I'm bringing AMD GPU compute ROCm based on virtio. I posted a 
>>>>>>>> ROCm over virtio
>>>>>>>> implementation to virglrenderer nine months ago (MR !1568 [1]). 
>>>>>>>> The ROCm side has
>>>>>>>> been supportted by ROCm offical.
>>>>>>>>
>>>>>>>> Current implementation is a virtio gpu context type capset 
>>>>>>>> handled inside
>>>>>>>> virglrenderer, sharing the display path. That's an awkward fit, 
>>>>>>>> many
>>>>>>>> compute GPUs have no display engine at all.
>>>>>>>
>>>>>>> I think "sharing the display path" conflates several layers and 
>>>>>>> makes the problem difficult to assess. It would help to identify 
>>>>>>> the concrete constraint behind "awkward fit."
>>>>>>>
>>>>>>> End-to-end, there are four relevant layers:
>>>>>>>
>>>>>>> 1. Host GPU stack: hardware, host kernel, and host userspace
>>>>>>> 2. Paravirtualization stack: virglrenderer and QEMU
>>>>>>
>>>>>> Yes we are asking can we add a new file like virtio-gpu specific 
>>>>>> for compute, but maybe we can only add a new backend like 
>>>>>> virglrenderer specific for compute.
>>>>>>
>>>>>>> 3. Host/guest interface: virtio and the capset-specific command 
>>>>>>> stream
>>>>>>
>>>>>> In this plan we may need just add a capset id.
>>>>>>
>>>>>>> 4. Guest GPU stack: guest kernel and guest userspace
>>>>>>
>>>>>> Won't modify the guest kernel in this plan, this email list.
>>>>>>
>>>>>>>
>>>>>>> Orthogonally, acceleration is separate from display and scanout. A
>>>>>>> physical device may provide both, but acceleration does not 
>>>>>>> require a
>>>>>>> display engine. Linux likewise exposes render and compute interfaces
>>>>>>> separately from modesetting. The userspace interface 
>>>>>>> virglrenderer uses is messy; there is Vulkan, EGL, OpenGL, and 
>>>>>>> now you are adding ROCm. But there is one thing I must note is 
>>>>>>> that acceleration and display is decoupled, and acceleration does 
>>>>>>> not require display.
>>>>>>
>>>>>> Yes totally agreed.
>>>>>>
>>>>>>>
>>>>>>> At the protocol layer, context command buffers are carried by 
>>>>>>> VIRTIO_GPU_CMD_SUBMIT_3D. Scanout uses separate core virtio-gpu 
>>>>>>> commands, and VIRTIO_GPU_CMD_GET_DISPLAY_INFO may report no 
>>>>>>> enabled displays. At the implementation layer, QEMU handles 
>>>>>>> scanout presentation. virgl_cmd_set_scanout() obtains resource 
>>>>>>> information through virgl_renderer_resource_get_info() or 
>>>>>>> virgl_renderer_resource_get_info_ext(). That does not make 
>>>>>>> scanout a virglrenderer-owned display path.
>>>>>>
>>>>>> Yes, agreed.
>>>>>>
>>>>>>>
>>>>>>> Therefore, if "sharing the display path" means sharing the same 
>>>>>>> device, control queue, and QEMU execution context, that 
>>>>>>> identifies a possible source of contention. If it means that 
>>>>>>> capsets or virglrenderer are inherently tied to display, I do not 
>>>>>>> think that is accurate. Vulkan compute is already used through 
>>>>>>> Venus with libkrun [2], and VCL proposes OpenCL support through 
>>>>>>> virglrenderer [3].
>>>>>>
>>>>>> Yes，but the vulkan is for GFX originally, and for some formal AI 
>>>>>> frame work like pytorch, it's support is limited, and it 
>>>>>> performance is lower than ROCm, and vulkan also lacks many AI 
>>>>>> infrastructure, like composable kernel.
>>>>>> And for virCL, actually it is came from same project with ROCm 
>>>>>> native context, but the original author didn't continue to support 
>>>>>> it, they handed it over to someone else to take over. And in the 
>>>>>> first version of
>>>>>> virCL, it didn't pass the test of actual projects.
>>>>>>
>>>>>> And it seems like virCL didn't upstream into virglrenderer also, 
>>>>>> correct me if I am wrong.
>>>>>
>>>>> I cited Venus and VCL only as examples showing that virtio-gpu and
>>>>> virglrenderer are not intrinsically tied to display. I did not suggest
>>>>> either as a substitute for ROCm.
>>>>>
>>>>>>
>>>>>>>> Beyond that, sharing the display path is increasingly painful:
>>>>>>>>
>>>>>>>>    - Compute hammers the queues more than graphics, so sharing
>>>>>>>>      virtio gpu's single control queue with display/virgl causes 
>>>>>>>> contention
>>>>>>>>      and display stutter.
>>>>>>>
>>>>>>> All non-cursor commands do share one control queue, but a fence 
>>>>>>> avoids serialization.
>>>>>>
>>>>>> yes, agreed.
>>>>>>
>>>>>>>
>>>>>>> There may still be implementation-level contention, and it is not
>>>>>>> necessarily specific to compute. A sufficiently busy graphics 
>>>>>>> workload
>>>>>>> could expose the same bottlenecks. Possible contributors in 
>>>>>>> current QEMU
>>>>>>> include:
>>>>>>>
>>>>>>> a) qemu_console_hw_gl_block() blocks the entire queue when QEMU only
>>>>>>>     needs to fence scanout commands.
>>>>>>
>>>>>> Yes, agreed.
>>>>>>
>>>>>>>
>>>>>>> b) virtio_gpu_virgl_unmap_resource_blob() may also block the entire
>>>>>>>     queue just to delay one command.
>>>>>>
>>>>>> Yes, but it is seems like it is must, someone else in AMD tried to 
>>>>>> use async method to relase blob, but it failed to consistency 
>>>>>> issue, then
>>>>>> reverted to sync version.
>>>>>
>>>>> Queue-wide suspension is not inherently required. Commit 
>>>>> 4eb0aace85f5 ("virtio-gpu: Support mapping hostmem blobs with 
>>>>> map_fixed") added a path that avoids per-blob MemoryRegion teardown 
>>>>> when virgl_renderer_resource_map_fixed() succeeds. The remaining 
>>>>> path is also being improved with:
>>>>>
>>>>> https://lore.kernel.org/qemu-devel/20260424-force_rcu-v4-0- 
>>>>> feccfaca0568@rsg.ci.i.u-tokyo.ac.jp/
>>>>> ("[PATCH v4 0/6] virtio-gpu: Force RCU when unmapping blob")
>>>>
>>>> Thanks. force_rcu is a clean fix for the RCU-reclamation part, but 
>>>> it still keeps the unmap synchronous and serial.
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> c) QEMU dispatches the control queue and calls into virglrenderer 
>>>>>>> from
>>>>>>>     its main-loop thread along with display work and many other 
>>>>>>> things.
>>>>>>>     Venus's render server can offload renderer work, but control- 
>>>>>>> queue
>>>>>>>     dispatch remains in QEMU's main loop.
>>>>>>
>>>>>> Yes, we did some async optimization in ROCm context, but its 
>>>>>> effectiveness is limited, see bellow.
>>>>>>
>>>>>>>
>>>>>>> In any case, I think you need to do some experiments to track 
>>>>>>> down the real cause. a) is easy to check: just comment out all 
>>>>>>> qemu_console_hw_gl_block() calls; it may corrupt display but 
>>>>>>> removes the blocking. b) can also be tested by leaking the 
>>>>>>> mappings instead of blocking the whole queue. Using a different 
>>>>>>> display device like qxl tells whether c) is causing contention.
>>>>>>
>>>>>> Yes, totally agreed. following is my findings. In short words:
>>>>>>
>>>>>> Optimization can reduce queue pressure, but it can't withstand 
>>>>>> absolute overload because each command has some overhead. Making 
>>>>>> all commands asynchronous would lead to a debugging hell about 
>>>>>> asynchronous issues.
>>>>>> And we have high load applications rocmprofiler  that continuously 
>>>>>> catch information need virtio queue to handle. But create a new 
>>>>>> backend can not solve it simply, we are trying to find a way. like 
>>>>>> shmem between guest and host, then use cpu polling, bypass the 
>>>>>> virtqueue.
>>>>>
>>>>> Most commands are fast on the CPU side, while heavy processing
>>>>> happens asynchronously on the GPU. Cases (a) and (b) are exceptions.
>>>>>
>>>>>>
>>>>>> The load is mostly memory management. Running an AI model 
>>>>>> allocates and frees a large number of blobs. We already did some 
>>>>>> optimization release them asynchronously, but the host processing 
>>>>>> is a single queue one
>>>>>> process_cmdq, this is where the main bottleneck in my debugging 
>>>>>> work / my understanding so far. I'm not certain it's the whole 
>>>>>> picture, so please correct if I am wrong.
>>>>>>
>>>>>> A model load or unload frees a large batch of BOs and allocates 
>>>>>> another. Some of those commands are async in the virtio-gpu guest 
>>>>>> driver, but QEMU still has to work through them on the one queue, 
>>>>>> which takes time; so even though any single command is quick, 
>>>>>> there are simply too many of them, the single queue backs up, and 
>>>>>> everything behind it, gets delayed.
>>>>>>
>>>>>> real work load (a few downstream customisations): loading one 16 
>>>>>> GB model (gemm4 e4b), drives ~1200 blob creates, a burst of
>>>>>> ~1400 resource frees at teardown, ~3700 submits and ~6000 
>>>>>> virtqueue notifies, caused a 22 s guest soft lockup. And the 
>>>>>> behavior of memory operations are controlled by upper layer like 
>>>>>> pytorch / HIP / runtime,
>>>>>> we can not control it.
>>>>>>
>>>>>> To be honest, a separate backend won't fix this. But the real 
>>>>>> solution maybe is compute specific. That logic is only useful to 
>>>>>> the compute path, and folding it into the shared display device / 
>>>>>> renderer would mean churning code that is mature and stable for 
>>>>>> graphics, with regression risk. Keeping compute on its own 
>>>>>> instance and backend lets us iterate on these compute only 
>>>>>> optimisations.
>>>>>
>>>>> A 22-second lockup is too long for those command counts.
>>>>>
>>>>> The most probable explanation I have is that the ROCm integration 
>>>>> blocks QEMU's main loop thread while synchronously waiting for GPU 
>>>>> execution. Creating separate devices won't resolve this because the 
>>>>> main loop thread is shared, and synchronously waiting on the GPU 
>>>>> should be avoided in the first place.
>>>>
>>>> No synchronously waiting in ROCm backend, we are using user queue, 
>>>> and event waiting, no sync operation in CMD wait. all the resource 
>>>> release in ROCm are all async now.
>>>> Only the sync thing is memory thing mapping/unmapping in qemu, as 
>>>> long as it remains synchronous, it will be overwhelmed by the 
>>>> massive number of requests.
>>>
>>> Mapping and unmapping should not block QEMU's main-loop thread for that
>>> long. The command counts you reported are relatively small. That is why
>>> I suspect something else went wrong, such as the main-loop thread being
>>> inadvertently blocked while waiting for the GPU.
>>
>> Will investigate it.
>>
>>>
>>>>
>>>>>
>>>>> In any case, profiling is necessary before touching the 
>>>>> implementation.
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>    - Compute contexts need far more blob / shared memory than a 
>>>>>>>> display one.
>>>>>>>
>>>>>>> It is not a problem by itself. Frequent mapping and unmapping 
>>>>>>> might amplify the second issue above, but that needs to be measured.
>>>>>>
>>>>>> Yes, agreed. I can give more detailed information.
>>>>>>
>>>>>>>
>>>>>>>>    - Maybe needs a wider ROCm / compute stack, cause the render 
>>>>>>>> model fits poorly:
>>>>>>>>      rocprofiler (PC sampling, SQTT/SPM, counters, high 
>>>>>>>> bandwidth streams)
>>>>>>>>      and ROCgdb (wave control, address watch, async exceptions an
>>>>>>>>      out of band channel that must not block display).
>>>>>>>
>>>>>>> virglrenderer does not impose a particular render model. That's 
>>>>>>> why Vulkan Compute just works with Venus.
>>>>>>
>>>>>> Yes but vulkan is used for GFX initally. And can not support many 
>>>>>> AI application.>
>>>>>>>>    - Events, faults and GPU reset/SMI are async and don't map 
>>>>>>>> onto fences.>    - All of this is hard to extend cleanly inside 
>>>>>>>> a display capset.
>>>>>>> Capset is not about display but determines the protocol of the 
>>>>>>> VIRTIO_GPU_CMD_SUBMIT_3D command stream. You have described 
>>>>>>> events, faults and GPU reset/SMI are async don't map onto fences 
>>>>>>> that may be associated with VIRTIO_GPU_CMD_SUBMIT_3D which is 
>>>>>>> dictated by capset. An additional feature may be necessary, and 
>>>>>>> it may or may not be dictated by capset. The other things are 
>>>>>>> irrelevant with the protocol capset represents; they are either 
>>>>>>> behavioral or about different commands.
>>>>>>
>>>>>> A fence is the one shot, but event is stateful and repeatable.
>>>>>> That may or may not be tied to capset. Agreed.
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On the QEMU/host side, would something like this be OK? One 
>>>>>>>> step, two parts:
>>>>>>>>
>>>>>>>>    - a dedicated headless virtio gpu instance for compute.
>>>>>>>
>>>>>>> A second device would isolate its virtqueues and device-wide
>>>>>>> renderer_blocked state. That may be useful if measurements show that
>>>>>>> these are the bottlenecks, but it is not yet clear that they are 
>>>>>>> or that
>>>>>>> a second device is the appropriate solution.
>>>>>>>
>>>>>>>>    - that instance served by a separate ROCm backend library loaded
>>>>>>>>      in-process by QEMU.
>>>>>>>
>>>>>>> First, I think we need to establish why ROCm cannot or should not 
>>>>>>> remain
>>>>>>> in virglrenderer. The virglrenderer, Venus, and VCL maintainers are
>>>>>>> likely better placed to advise on that boundary. Once the protocol
>>>>>>> requirements and performance measurements are clear, we can 
>>>>>>> assess the
>>>>>>> appropriate QEMU integration.
>>>>>>
>>>>>> venus is borned for GFX.
>>>>>> virCL not merged.
>>>>>>
>>>>>> To be clear, I'm not saying virglrenderer can't host a ROCm native 
>>>>>> context it clearly can. My hesitation is more about fit and 
>>>>>> direction: virglrenderer has grown up around GL/graphics, and I 
>>>>>> haven't yet found compute oriented plumbing there to build on, 
>>>>>> while ROCm moves very fast and I need something I can keep current 
>>>>>> with low friction.
>>>>>
>>>>> Whether keeping ROCm in virglrenderer would create extra friction 
>>>>> is primarily a question for the virglrenderer maintainers. Its 
>>>>> graphics origins do not by themselves motivate adding a separate 
>>>>> backend interface to QEMU.
>>>>
>>>> Fair. The first draft version in virglrenderer was in May 2024, and 
>>>> ROCm has gone 5.7 → 7.14 in that window.
>>>
>>> One point to note is that virtio-gpu development in QEMU is somewhat
>>> less active. crosvm is the most active user of virglrenderer, and QEMU
>>> sometimes lags behind it. If you are considering moving the ROCm 
>>> integration from virglrenderer to QEMU solely because ROCm evolves 
>>> rapidly, I do not think that would be a good idea. A rapidly evolving 
>>> component is better kept in virglrenderer unless there is another 
>>> reason to place it in QEMU.
>>
>> Actually didn't see something new about compute merged in to 
>> virglrenderer this recently 2 years.
> 
> Neither QEMU nor virglrenderer has seen new compute-related additions in 
> the past two years.

Maybe that is the reason we need a compute specific path? But I think 
the VFIO or vDPA are all can be used for compute, they are really 
active. We only need a small file for compute, providing the basic 
mechanisms, this code will also benefit other computing devices, such as 
NPU, I believe there will be more and more computing devices in the future.


> 
> While you have regularly updated the merge request, initiating 
> discussions around it is necessary to move review forward. Open-source 
> projects like QEMU and virglrenderer need proactive driving to complete 
> reviews. Simply shifting the ROCm integration to QEMU will not resolve 
> this bottleneck.

Yes I have actively promoted it, but I haven't received substantial 
reviews regarding virtio gpu userptr and virglrenderer. Hard to make MR 
move forward without a substantial review. So I am finding a another way.

> 
> Besides, looking at the "Architecture Components" in the description, 
> most of them haven't been merged yet. The virglrenderer code cannot be 
> merged in its current state, so focusing on those dependencies first is 
> essential.

Yes, I must admit that most of them not merged.
But actually for para virtualization those components are need merged 
together because they are closely connected.

> 
> However, taking a naive approach can lead to a chicken-and-egg problem: 
> component maintainers want the virglrenderer side stabilized first, 
> while virglrenderer maintainers want the component side stabilized. To 
> break this deadlock, I suggest seeking consensus on the interfaces 
> before completing the implementation. Once an interface agreement is 
> reached, changes to each component can land independently:

I've seen amdgpu native context (for GFX) and msm native context merged 
quickly. And actually ROCm native context is using the same method. 
That's why I think this is a problem of direction.

> 
> - virtio interface: I raised a concern regarding the interface [1][2]
>    that needs to be addressed.

I'm open to any feedback from the virtio maintainer, but since it's 
really just you and me discussing this, I can immediately modify your 
proposal if the maintainer agrees.

> - amdkfd patches: There are interface-level concerns [3] that still need
>    resolution.

We have a another solution to solve it, it is already done in virglrender.

> - ROCm runtime: The description lists this as "90% complete," but the
>    linked pull requests were closed due to inactivity. They need to be
>    reopened and seek for a consensus on its interface.

It was completed using another PR, so it was closed.

> 
> Once these items are addressed, you can update the merge request and ask 
> for a fresh review.
> 
> In parallel, you can request review of self-contained parts of 
> components that do not depend on those decisions. Once the interfaces 
> are agreed, the implementations can be reviewed in parallel and merged 
> in dependency order.
> 
> These steps can proceed in parallel alongside investigating the 
> performance bottleneck.

Really thanks for the suggestion. I think the compute specific path 
still worth discussing.

Regards,
Honglei

> 
> Regards,
> Akihiko Odaki
> 
> [1] https://lore.kernel.org/lkml/ 
> b69439ec-0ebd-4527-873b-85b283e03888@rsg.ci.i.u-tokyo.ac.jp/
> [2] https://lore.kernel.org/qemu-devel/35a8add7-da49-4833-9e69- 
> d213f52c771a@amd.com/
> [3] https://lore.kernel.org/lkml/20260104072122.3045656-1- 
> honglei1.huang@amd.com/
> [4] https://www.spinics.net/lists/amd-gfx/msg137231.html