[PATCH v2 0/2] virtio_gpu: add F_CREATE_GUEST_HANDLE and F_BLOB_CTX_ID_FIX

[PATCH v2 0/2] virtio_gpu: add F_CREATE_GUEST_HANDLE and F_BLOB_CTX_ID_FIX

[Val Packett]

Hi,

I've been working on actually implementing the "lost" CREATE_GUEST_HANDLE
feature, previously half-implemented and removed in crosvm
(https://crrev.com/c/8064741), which allows the guest to tell the device
backend to create a shareable handle to a guest-only memory blob (using
facilities like udmabuf or gntdev-dmabuf) which can be e.g. sent over
to the Wayland compositor when using the cross-domain capset.

This allows for cross-domain Wayland passthrough to:

1) have a zero-copy fast path for wl_shm (software rendered buffers)
   thanks to udmabuf being able to wrap memfds from regular clients;
2) share buffers coming from a passed-through dGPU;
3) have an alternative way to share memory when host-only buffers
   cannot be mapped due to lack of a host_visible virtio SHM region,
   which might not be available in some setups (e.g. on Xen).

I have a working implementation with the zero-copy fast path:

Backend: https://github.com/magma-gpu/rutabaga_gfx/pull/81
       + https://github.com/libkrun/libkrun/pull/822
Guest: https://codeberg.org/drakulix/wl-cross-domain-proxy/pulls/24
Kernel: want to link here when posting to lkml :) temporarily:
        https://github.com/valpackett/linux-qclaptops/commits/guest-handle
Demo video: https://social.treehouse.systems/@valpackett/117149550756102167

This requires adding two new virtio-gpu feature flags:

Obviously the original CREATE_GUEST_HANDLE itself; but during implementation
it was discovered that we need to change the CREATE_BLOB cmd handling in a
backwards-incompatible way, to ensure that code changes on both sides are
an clean-up rather than a pile-up of odd spaghetti conditionals.

If you're wondering why the explicit indication from the guest must be used
rather than the backend "lazily" creating the handles based on usage:
it is better for the guest userspace to explicitly know which ways of
sharing memory are supported, so that one can be auto-negotiated.
(discussed in https://github.com/magma-gpu/rutabaga_gfx/issues/66)

v2: added F_BLOB_CTX_ID_FIX
v1 has been lost to the moderation queue, but if it gets approved:
https://lore.kernel.org/all/20260827092120.97295-1-val@invisiblethingslab.com/

Thanks,
~val

---

Val Packett (2):
  virtio-gpu: support requesting host handles for guest-only blob
    resources
  virtio-gpu: add flag to negotiate always passing ctx_id to
    RESOURCE_CREATE_BLOB

 device-types/gpu/description.tex | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

-- 
2.55.0

[PATCH v2 1/2] virtio-gpu: support requesting host handles for guest-only blob resources

[Val Packett]

So far, desktop integration over VIRTIO_GPU_CAPSET_CROSS_DOMAIN has
relied on host-only blob resources, which are always backed by a
shareable object on the host side (such as a memfd or a DMA-BUF).

However, to access such resources the guest needs to use the
VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB command which is currently only
available when a VIRTIO_GPU_SHM_ID_HOST_VISIBLE shared memory region
exists, which is not possible in all environments (e.g. the memory
sharing model of the Xen hypervisor does not make it possible).

Also, this necessarily introduces copying for software-rendered client
buffers. The Wayland protocol makes clients responsible for allocating
their SHM buffer pools, which are ordinary guest kernel shared temporary
file objects (memfds). The only way to bridge those and host-only buffers
is a copy on every content update.

Host-only resources with the HOST_VISIBLE region were picked as the
default solution because it's easy to implement with KVM. However with
the introduction of the udmabuf driver, it also became possible to create
shareable handles (namely DMA-BUFs) for specific subranges of various
memfd objects (which can be the backing objects for a KVM guest's RAM).
And on the Xen side, gntdev-dmabuf provides a similar operation for
Xen grant references. With these capabilities, it should be possible
to use guest-only blob resources as well!

The udmabuf driver can also be used in the guest to complete the Wayland
SHM zero-copy fast path. A memfd allocated by an ordinary client,
completely unaware of virtualization, can be turned into a DMA-BUF (as
long as some conditions hold) which can be imported into the virtio-gpu
device. This new flag, VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE,
indicates to the host that the provided memory range entries should be
used to construct a corresponding DMA-BUF on the host side as well,
so that this resource can then be sent over to the host Wayland
compositor using the cross-domain context's send command.

Signed-off-by: Val Packett <val@invisiblethingslab.com>
---
 device-types/gpu/description.tex | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
index ac3b427725f6..64e8c4ddecc5 100644
--- a/device-types/gpu/description.tex
+++ b/device-types/gpu/description.tex
@@ -39,6 +39,9 @@ \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
   synchronization timelines supported.  Requires VIRTIO_GPU_F_VIRGL.
 \item[VIRTIO_GPU_F_BLOB_ALIGNMENT (5)] configuration field
   \field{blob_alignment} is valid. Requires VIRTIO_GPU_F_RESOURCE_BLOB.
+\item[VIRTIO_GPU_F_CREATE_GUEST_HANDLE (6)] guest-only blob resources
+  may be created with the VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE flag.
+  Requires VIRTIO_GPU_F_RESOURCE_BLOB.
 \end{description}
 
 \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
@@ -621,9 +624,10 @@ \subsubsection{Device Operation: controlq}\label{sec:Device Types / GPU Device /
 #define VIRTIO_GPU_BLOB_MEM_HOST3D            0x0002
 #define VIRTIO_GPU_BLOB_MEM_HOST3D_GUEST      0x0003
 
-#define VIRTIO_GPU_BLOB_FLAG_USE_MAPPABLE     0x0001
-#define VIRTIO_GPU_BLOB_FLAG_USE_SHAREABLE    0x0002
-#define VIRTIO_GPU_BLOB_FLAG_USE_CROSS_DEVICE 0x0004
+#define VIRTIO_GPU_BLOB_FLAG_USE_MAPPABLE        0x0001
+#define VIRTIO_GPU_BLOB_FLAG_USE_SHAREABLE       0x0002
+#define VIRTIO_GPU_BLOB_FLAG_USE_CROSS_DEVICE    0x0004
+#define VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE 0x0008
 
 struct virtio_gpu_resource_create_blob {
        struct virtio_gpu_ctrl_hdr hdr;
@@ -673,6 +677,12 @@ \subsubsection{Device Operation: controlq}\label{sec:Device Types / GPU Device /
 memory access, sharing between driver instances and/or sharing with
 other devices. This is done via the \field{blob_flags} field.
 
+If VIRTIO_GPU_F_CREATE_GUEST_HANDLE has been negotiated, the
+VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE flag may be used to request the
+creation of a shareable host kernel handle for a guest-only blob resource.
+Guest-only resources using this flag MAY be created from the rendering context
+local object identified by the \field{blob_id}.
+
 If VIRTIO_GPU_F_VIRGL is set, both VIRTIO_GPU_CMD_TRANSFER_TO_HOST_3D
 and VIRTIO_GPU_CMD_TRANSFER_FROM_HOST_3D may be used to update the
 resource. There is no restriction on the image/buffer view the driver
-- 
2.55.0

[PATCH v2 2/2] virtio-gpu: add flag to negotiate always passing ctx_id to RESOURCE_CREATE_BLOB

[Val Packett]

virtio-gpu device backends typically use ctx_id to route commands to
different modules that implement distinct context types, e.g.
virglrenderer vs. cross-domain. Each module may require the blob
resources intented to use with it to be allocated through it as well.

Currently, this happens for host-only and default blobs which are
allocated from a context-specific local blob_id (e.g. an image
requirements blob in cross-domain). For guest-only resources however,
existing software (both the Linux kernel and the rutabaga-gfx library
used by device backends to implement virtio-gpu) allocates those with
ctx_id == 0, which might be a different "default" context type from the
one the resource is intended to be used with, e.g. when both
virglrenderer and cross-domain are present, the socket buffers used by
cross-domain were processed by virglrenderer which happens to work
but does not actually make sense.

Additionally we would like to support PRIME imported resources (udmabufs
or dGPU passthrough dma-bufs) being passed over cross-domain with the
CREATE_GUEST_HANDLE feature, which due to API constraints must also
perform the creation of a guest-only blob resource with ctx_id but
without blob_id.

Unfortunately changing this without breaking backwards compatibility
requires a flag to negotiate the use of the "fixed" convention. Otherwise
new guest kernel + old rutabaga would result in the command failing due
to old rutabaga not expecting guest resources to be allocated w/ ctx_id.

Signed-off-by: Val Packett <val@invisiblethingslab.com>
---
 device-types/gpu/description.tex | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
index 64e8c4ddecc5..c0ff3f34ec1c 100644
--- a/device-types/gpu/description.tex
+++ b/device-types/gpu/description.tex
@@ -42,6 +42,9 @@ \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
 \item[VIRTIO_GPU_F_CREATE_GUEST_HANDLE (6)] guest-only blob resources
   may be created with the VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE flag.
   Requires VIRTIO_GPU_F_RESOURCE_BLOB.
+\item[VIRTIO_GPU_F_BLOB_CTX_ID_FIX (7)] it is always safe to pass the
+  current context ID to VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB,
+  including for guest-only blobs.
 \end{description}
 
 \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
@@ -673,6 +676,12 @@ \subsubsection{Device Operation: controlq}\label{sec:Device Types / GPU Device /
 identified by the \field{blob_id}. The actual allocation is done via
 VIRTIO_GPU_CMD_SUBMIT_3D.
 
+If VIRTIO_GPU_F_BLOB_CTX_ID_FIX has been negotiated, the \field{ctx_id} of
+the \field{hdr} MUST always be filled in with the ID of the default rendering
+context associated with the current handle, if one exists. Otherwise, it MUST
+only be filled when the \field{blob_id} has been filled to create a resource
+from a rendering context local object.
+
 The driver MUST inform the device if the blob resource is used for
 memory access, sharing between driver instances and/or sharing with
 other devices. This is done via the \field{blob_flags} field.
-- 
2.55.0

Re: [PATCH v2 2/2] virtio-gpu: add flag to negotiate always passing ctx_id to RESOURCE_CREATE_BLOB

[Val Packett]

On 9/3/26 7:20 PM, Gurchetan Singh wrote:
>
>
> On Wed, Sep 2, 2026 at 7:15 PM Val Packett 
> <val@invisiblethingslab.com> wrote:
>
>     virtio-gpu device backends typically use ctx_id to route commands to
>     different modules that implement distinct context types, e.g.
>     virglrenderer vs. cross-domain. Each module may require the blob
>     resources intented to use with it to be allocated through it as well.
>
>     Currently, this happens for host-only and default blobs which are
>     allocated from a context-specific local blob_id (e.g. an image
>     requirements blob in cross-domain). For guest-only resources however,
>     existing software (both the Linux kernel and the rutabaga-gfx library
>     used by device backends to implement virtio-gpu) allocates those with
>     ctx_id == 0, which might be a different "default" context type
>     from the
>     one the resource is intended to be used with, e.g. when both
>     virglrenderer and cross-domain are present, the socket buffers used by
>     cross-domain were processed by virglrenderer which happens to work
>     but does not actually make sense.
>
>     Additionally we would like to support PRIME imported resources
>     (udmabufs
>     or dGPU passthrough dma-bufs) being passed over cross-domain with the
>     CREATE_GUEST_HANDLE feature, which due to API constraints must also
>     perform the creation of a guest-only blob resource with ctx_id but
>     without blob_id.
>
>     Unfortunately changing this without breaking backwards compatibility
>     requires a flag to negotiate the use of the "fixed" convention.
>     Otherwise
>     new guest kernel + old rutabaga would result in the command
>     failing due
>     to old rutabaga not expecting guest resources to be allocated w/
>     ctx_id.
>
>     Signed-off-by: Val Packett <val@invisiblethingslab.com>
>     ---
>      device-types/gpu/description.tex | 9 +++++++++
>      1 file changed, 9 insertions(+)
>
>     diff --git a/device-types/gpu/description.tex
>     b/device-types/gpu/description.tex
>     index 64e8c4ddecc5..c0ff3f34ec1c 100644
>     --- a/device-types/gpu/description.tex
>     +++ b/device-types/gpu/description.tex
>     @@ -42,6 +42,9 @@ \subsection{Feature bits}\label{sec:Device Types
>     / GPU Device / Feature bits}
>      \item[VIRTIO_GPU_F_CREATE_GUEST_HANDLE (6)] guest-only blob resources
>        may be created with the
>     VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE flag.
>        Requires VIRTIO_GPU_F_RESOURCE_BLOB.
>     +\item[VIRTIO_GPU_F_BLOB_CTX_ID_FIX (7)] it is always safe to pass the
>     +  current context ID to VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB,
>     +  including for guest-only blobs.
>      \end{description}
>
>      \subsection{Device configuration layout}\label{sec:Device Types /
>     GPU Device / Device configuration layout}
>     @@ -673,6 +676,12 @@ \subsubsection{Device Operation:
>     controlq}\label{sec:Device Types / GPU Device /
>      identified by the \field{blob_id}. The actual allocation is done via
>      VIRTIO_GPU_CMD_SUBMIT_3D.
>
>     +If VIRTIO_GPU_F_BLOB_CTX_ID_FIX has been negotiated, the
>     \field{ctx_id} of
>     +the \field{hdr} MUST always be filled in with the ID of the
>     default rendering
>     +context associated with the current handle, if one exists.
>     Otherwise, it MUST
>     +only be filled when the \field{blob_id} has been filled to create
>     a resource
>     +from a rendering context local object.
>
>
>
> I can confirm for gfxstream/rutabaga_gfx, it is robust enough to 
> handle ctx_id != 0 + BLOB_MEM for all versions we care about.
> [..]

Umm, as long as "versions we care about" means only current git main 
right now..??

Before the latest fixes (PR #81), this would not work for cross-domain, 
starting with its socket buffers.

With ctx_id != 0 rutabaga dispatches to the context's 
context_create_blob impl. For the guest memory (blob_id == 0) that would 
unconditionally try to get the 0th context item and fail with 
RutabagaError::InvalidCrossDomainItemId. Only component-level 
create_blob (used via ctx_id == 0) would handle guest memory.

The flag means "that won't happen anymore", so VMMs built with rutabaga 
 >0.1.85 can set it, in which case the guest will happily go through the 
(ctx_id != 0, blob_id == 0) code path.

~val

Re: [PATCH v2 2/2] virtio-gpu: add flag to negotiate always passing ctx_id to RESOURCE_CREATE_BLOB

[Demi Marie Obenour]

[-- Attachment #1.1: Type: text/plain, Size: 3549 bytes --]

On 9/2/26 22:14, Val Packett wrote:
> virtio-gpu device backends typically use ctx_id to route commands to
> different modules that implement distinct context types, e.g.
> virglrenderer vs. cross-domain. Each module may require the blob
> resources intented to use with it to be allocated through it as well.
> 
> Currently, this happens for host-only and default blobs which are
> allocated from a context-specific local blob_id (e.g. an image
> requirements blob in cross-domain). For guest-only resources however,
> existing software (both the Linux kernel and the rutabaga-gfx library
> used by device backends to implement virtio-gpu) allocates those with
> ctx_id == 0, which might be a different "default" context type from the
> one the resource is intended to be used with, e.g. when both
> virglrenderer and cross-domain are present, the socket buffers used by
> cross-domain were processed by virglrenderer which happens to work
> but does not actually make sense.
> 
> Additionally we would like to support PRIME imported resources (udmabufs
> or dGPU passthrough dma-bufs) being passed over cross-domain with the
> CREATE_GUEST_HANDLE feature, which due to API constraints must also
> perform the creation of a guest-only blob resource with ctx_id but
> without blob_id.
> 
> Unfortunately changing this without breaking backwards compatibility
> requires a flag to negotiate the use of the "fixed" convention. Otherwise
> new guest kernel + old rutabaga would result in the command failing due
> to old rutabaga not expecting guest resources to be allocated w/ ctx_id.
> 
> Signed-off-by: Val Packett <val@invisiblethingslab.com>
> ---
>  device-types/gpu/description.tex | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
> index 64e8c4ddecc5..c0ff3f34ec1c 100644
> --- a/device-types/gpu/description.tex
> +++ b/device-types/gpu/description.tex
> @@ -42,6 +42,9 @@ \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
>  \item[VIRTIO_GPU_F_CREATE_GUEST_HANDLE (6)] guest-only blob resources
>    may be created with the VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE flag.
>    Requires VIRTIO_GPU_F_RESOURCE_BLOB.
> +\item[VIRTIO_GPU_F_BLOB_CTX_ID_FIX (7)] it is always safe to pass the
> +  current context ID to VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB,
> +  including for guest-only blobs.
>  \end{description}
>  
>  \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
> @@ -673,6 +676,12 @@ \subsubsection{Device Operation: controlq}\label{sec:Device Types / GPU Device /
>  identified by the \field{blob_id}. The actual allocation is done via
>  VIRTIO_GPU_CMD_SUBMIT_3D.
>  
> +If VIRTIO_GPU_F_BLOB_CTX_ID_FIX has been negotiated, the \field{ctx_id} of
> +the \field{hdr} MUST always be filled in with the ID of the default rendering
> +context associated with the current handle, if one exists. Otherwise, it MUST
> +only be filled when the \field{blob_id} has been filled to create a resource
> +from a rendering context local object.
> +
>  The driver MUST inform the device if the blob resource is used for
>  memory access, sharing between driver instances and/or sharing with
>  other devices. This is done via the \field{blob_flags} field.

Should VIRTIO_GPU_F_CREATE_GUEST_HANDLE require VIRTIO_GPU_F_BLOB_CTX_ID_FIX?
That could simplify drivers.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 2/2] virtio-gpu: add flag to negotiate always passing ctx_id to RESOURCE_CREATE_BLOB

[Val Packett]

On 9/4/26 12:35 PM, Demi Marie Obenour wrote:
> On 9/2/26 22:14, Val Packett wrote:
>> virtio-gpu device backends typically use ctx_id to route commands to
>> different modules that implement distinct context types, e.g.
>> virglrenderer vs. cross-domain. Each module may require the blob
>> resources intented to use with it to be allocated through it as well.
>>
>> Currently, this happens for host-only and default blobs which are
>> allocated from a context-specific local blob_id (e.g. an image
>> requirements blob in cross-domain). For guest-only resources however,
>> existing software (both the Linux kernel and the rutabaga-gfx library
>> used by device backends to implement virtio-gpu) allocates those with
>> ctx_id == 0, which might be a different "default" context type from the
>> one the resource is intended to be used with, e.g. when both
>> virglrenderer and cross-domain are present, the socket buffers used by
>> cross-domain were processed by virglrenderer which happens to work
>> but does not actually make sense.
>>
>> Additionally we would like to support PRIME imported resources (udmabufs
>> or dGPU passthrough dma-bufs) being passed over cross-domain with the
>> CREATE_GUEST_HANDLE feature, which due to API constraints must also
>> perform the creation of a guest-only blob resource with ctx_id but
>> without blob_id.
>>
>> Unfortunately changing this without breaking backwards compatibility
>> requires a flag to negotiate the use of the "fixed" convention. Otherwise
>> new guest kernel + old rutabaga would result in the command failing due
>> to old rutabaga not expecting guest resources to be allocated w/ ctx_id.
>>
>> Signed-off-by: Val Packett <val@invisiblethingslab.com>
>> ---
>>   device-types/gpu/description.tex | 9 +++++++++
>>   1 file changed, 9 insertions(+)
>>
>> diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
>> index 64e8c4ddecc5..c0ff3f34ec1c 100644
>> --- a/device-types/gpu/description.tex
>> +++ b/device-types/gpu/description.tex
>> @@ -42,6 +42,9 @@ \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
>>   \item[VIRTIO_GPU_F_CREATE_GUEST_HANDLE (6)] guest-only blob resources
>>     may be created with the VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE flag.
>>     Requires VIRTIO_GPU_F_RESOURCE_BLOB.
>> +\item[VIRTIO_GPU_F_BLOB_CTX_ID_FIX (7)] it is always safe to pass the
>> +  current context ID to VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB,
>> +  including for guest-only blobs.
>>   \end{description}
>>   
>>   \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
>> @@ -673,6 +676,12 @@ \subsubsection{Device Operation: controlq}\label{sec:Device Types / GPU Device /
>>   identified by the \field{blob_id}. The actual allocation is done via
>>   VIRTIO_GPU_CMD_SUBMIT_3D.
>>   
>> +If VIRTIO_GPU_F_BLOB_CTX_ID_FIX has been negotiated, the \field{ctx_id} of
>> +the \field{hdr} MUST always be filled in with the ID of the default rendering
>> +context associated with the current handle, if one exists. Otherwise, it MUST
>> +only be filled when the \field{blob_id} has been filled to create a resource
>> +from a rendering context local object.
>> +
>>   The driver MUST inform the device if the blob resource is used for
>>   memory access, sharing between driver instances and/or sharing with
>>   other devices. This is done via the \field{blob_flags} field.
> Should VIRTIO_GPU_F_CREATE_GUEST_HANDLE require VIRTIO_GPU_F_BLOB_CTX_ID_FIX?
> That could simplify drivers.

Yeah, you're right. They seemed orthogonal to me in terms of driver 
code, but that's just because I've decided to simply not support 
CREATE_GUEST_HANDLE actually working when BLOB_CTX_ID_FIX is not set :)

I'll write this in explicitly in v3, and add a check for it in the driver.

Thanks,
~val