Re: [PATCH 1/5] migration/qapi: Add memfd migration transport

[Markus Armbruster <armbru@redhat.com>]

Dongli Zhang <dongli.zhang@oracle.com> writes:

> On Thu, Aug 20, 2026 12:09:19AM -0700, Markus Armbruster wrote:
>> Dongli Zhang <dongli.zhang@oracle.com> writes:
>> 
>>> Add a memfd migration transport to the MigrationAddress QAPI schema. The
>>> new transport has no user-supplied arguments because QEMU creates and
>>> passes the backing memfd internally.
>>>
>>> The initial user is cpr-exec, where the main VM/device migration stream can
>>> be stored in an inherited memfd instead of an external file.
>>>
>>> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
>>> ---
>>>  qapi/migration.json | 32 +++++++++++++++++++++++++++-----
>>>  1 file changed, 27 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/qapi/migration.json b/qapi/migration.json
>>> index b1eaf7b054..1505c201ca 100644
>>> --- a/qapi/migration.json
>>> +++ b/qapi/migration.json
>>> @@ -702,9 +702,11 @@
>>>  # @cpr-exec: The migrate command stops the VM, saves state to the
>>>  #     migration channel, directly exec's a new version of QEMU on the
>>>  #     same host, replacing the original process while retaining its
>>> -#     PID, and loads state from the channel.  Guest RAM is preserved
>>> -#     in place.  Devices and their pinned pages are also preserved for
>>> -#     VFIO and IOMMUFD.
>>> +#     PID, and loads state from the channel.  With the @memfd
>>> +#     migration transport, QEMU saves VM/device state to an internal
>>> +#     memfd instead of an external main migration channel.  Guest RAM
>>> +#     is preserved in place.  Devices and their pinned pages are also
>>> +#     preserved for VFIO and IOMMUFD.
>>>  #
>>>  #     Old QEMU starts new QEMU by exec'ing the command specified by
>>>  #     the @cpr-exec-command parameter.  The command may be a direct
>>> @@ -716,6 +718,13 @@
>>>  #     as a file, that accepts all data before old QEMU exits.
>>>  #     Otherwise, old QEMU may quietly block writing to the channel.
>>>  #
>>> +#     Alternatively, use the @memfd migration transport to save
>>> +#     VM/device state to an internal memfd inherited by new QEMU.
>>> +#
>> 
>> Any guidance on when to use @memfd?
>
> My objective is to avoid using disks, or running QEMU with writable storage.
>
> Both guest I/O and networking can go through VFIO, and everything remains in memory.
>
> When I was testing these CPR features, my favorite was cpr-exec because it does
> not require creating an additional QEMU instance.
>
> Based on my understanding, the best approach to live update a running binary is
> to replace only the code section while keeping everything else in memory, i.e.
> guest memory, device fds, and device state.
>
> Unfortunately, device state cannot simply remain in memory across exec(), so we
> need to migrate it from the old instance to the new instance. I was thinking:
> why not use memory instead of storage?
>
> Initially, I added the change only for cpr-exec. Later, I reworked the change to
> make memfd a new migration transport. As a side benefit for me, I also practiced
> how to add a new transport to the QEMU migration code :)

This is why you wrote the feature.  Please work it into the commit
message.

We should also help users decide whether to use the feature.  I gather
it's useful with migration mode cpr-exec.  Any other use cases?  Does it
even work with other migration modes?  If yes, which ones, and what
happens when you try with another one?

>>> +#     With @memfd, the new QEMU command may include '-incoming memfd:'
>>> +#     to load the migration stream automatically, or '-incoming defer'
>>> +#     to load it later with `migrate-incoming`.
>>> +#
>> 
>> Isn't that the same for any transport?  I.e. old and new QEMU must use a
>> common transport, and you configure it for the new one with -incoming
>> right away, or later with migrate-incoming.
>
> Yes, it is the same for all transports. I added the explanation only because the
> memfd transport does not need additional data or a filename, for example, the
> URI is just "memfd:".
>
> Will remove it.
>
>> 
>>>  #     Memory-backend objects must have the share=on attribute, but
>>>  #     memory-backend-epc is not supported.  The VM must be started
>>>  #     with the '-machine aux-ram-share=on' option.

[...]