Re: [PATCH vfio 0/6] Add support for PRE_COPY initial bytes re-initialization

[Alex Williamson <alex@shazbot.org>]

+Cédric, +Peter, please see what you think of this approach relative to
QEMU.  The broken uAPI for flags on the PRECOPY_INFO ioctl is
unfortunate, but we need an opt-in for the driver to enable REINIT
reporting anyway.  Thanks,

Alex

On Tue, 24 Feb 2026 10:20:13 +0200
Yishai Hadas <yishaih@nvidia.com> wrote:

> This series introduces support for re-initializing the initial_bytes
> value during the VFIO PRE_COPY migration phase.
> 
> Background
> ==========
> As currently defined, initial_bytes is monotonically decreasing and
> precedes dirty_bytes when reading from the saving file descriptor.
> The transition from initial_bytes to dirty_bytes is unidirectional and
> irreversible.
> 
> The initial_bytes are considered critical data that is highly
> recommended to be transferred to the target as part of PRE_COPY.
> Without this data, the PRE_COPY phase would be ineffective.
> 
> Problem Statement
> =================
> In some cases, a new chunk of critical data may appear during the
> PRE_COPY phase. The current API does not provide a mechanism for the
> driver to report an updated initial_bytes value when this occurs.
> 
> Solution
> ========
> For that, we extend the VFIO_MIG_GET_PRECOPY_INFO ioctl with an output
> flag named VFIO_PRECOPY_INFO_REINIT to allow drivers reporting a new
> initial_bytes value during the PRE_COPY phase.
> 
> However, Currently, existing VFIO_MIG_GET_PRECOPY_INFO implementations
> don't assign info.flags before copy_to_user(), this effectively echoes
> userspace-provided flags back as output, preventing the field from being
> used to report new reliable data from the drivers.
> 
> Reliable use of the new VFIO_PRECOPY_INFO_REINIT flag requires userspace
> to explicitly opt in. For that we introduce a new feature named
> VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2.
> 
> User should opt-in to the above feature with a SET operation, no data is
> required and any supplied data is ignored.
> 
> When the caller opts in:
> - We set info.flags to zero, otherwise we keep v1 behaviour as is for
>   compatibility reasons.
> - The new output flag VFIO_PRECOPY_INFO_REINIT can be used reliably.
> - The VFIO_PRECOPY_INFO_REINIT output flag indicates that new initial
>   data is present on the stream. The initial_bytes value should be
>   re-evaluated relative to the readiness state for transition to
>   STOP_COPY.
> 
> The mlx5 VFIO driver is extended to support this case when the
> underlying firmware also supports the REINIT migration state.
> 
> As part of this series, a core helper function is introduced to provide
> shared functionality for implementing the VFIO_MIG_GET_PRECOPY_INFO
> ioctl, and all drivers have been updated to use it.
> 
> Note:
> We may need to send the net/mlx5 patch to VFIO as a pull request to
> avoid conflicts prior to acceptance.
> 
> Yishai
> 
> Yishai Hadas (6):
>   vfio: Define uAPI for re-init initial bytes during the PRE_COPY phase
>   vfio: Add support for VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2
>   vfio: Adapt drivers to use the core helper vfio_check_precopy_ioctl
>   net/mlx5: Add IFC bits for migration state
>   vfio/mlx5: consider inflight SAVE during PRE_COPY
>   vfio/mlx5: Add REINIT support to VFIO_MIG_GET_PRECOPY_INFO
> 
>  .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c    |  17 +--
>  drivers/vfio/pci/mlx5/cmd.c                   |  25 +++-
>  drivers/vfio/pci/mlx5/cmd.h                   |   6 +-
>  drivers/vfio/pci/mlx5/main.c                  | 118 +++++++++++-------
>  drivers/vfio/pci/qat/main.c                   |  17 +--
>  drivers/vfio/pci/vfio_pci_core.c              |   1 +
>  drivers/vfio/pci/virtio/migrate.c             |  17 +--
>  drivers/vfio/vfio_main.c                      |  20 +++
>  include/linux/mlx5/mlx5_ifc.h                 |  16 ++-
>  include/linux/vfio.h                          |  40 ++++++
>  include/uapi/linux/vfio.h                     |  22 ++++
>  samples/vfio-mdev/mtty.c                      |  16 +--
>  12 files changed, 217 insertions(+), 98 deletions(-)
>