Re: [PATCH v2 2/2] hw/vfio/ap: Callbacks for migration of guests with pass-through access to AP devices

["Cédric Le Goater" <clg@redhat.com>]

On 4/10/26 12:50, Anthony Krowiak wrote:
> 
> 
> On 4/10/26 4:19 AM, Cédric Le Goater wrote:
>> On 4/9/26 16:13, Anthony Krowiak wrote:
>>> From: Anthony  Krowiak <akrowiak@linux.ibm.com>
>>>
>>> Implements the callbacks used by the VFIO migration framework to migrate
>>> guests with pass-through access to s390 crypto devices.
>>>
>>> Signed-off-by: Anthony Krowiak <akrowiak@linux.ibm.com>
>>> ---
>>>   hw/vfio/ap.c | 43 +++++++++++++++++++++++++++++++++++++++----
>>>   1 file changed, 39 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c
>>> index fd68983702..49947b18c3 100644
>>> --- a/hw/vfio/ap.c
>>> +++ b/hw/vfio/ap.c
>>> @@ -32,6 +32,7 @@
>>>   #include "hw/s390x/ap-bridge.h"
>>>   #include "system/address-spaces.h"
>>>   #include "qom/object.h"
>>> +#include "vfio-migration-internal.h"
>>>     #define TYPE_VFIO_AP_DEVICE      "vfio-ap"
>>>   @@ -47,6 +48,7 @@ static const VMStateDescription vmstate_ap_device = {
>>>       .name = "vfio-ap-device",
>>>       .version_id = 1,
>>>       .minimum_version_id = 1,
>>> +    .unmigratable = 0,
>>
>> unneeded.
>>
>>>       .fields = (const VMStateField[]) {
>>>           VMSTATE_END_OF_LIST()
>>>       },
>>> @@ -66,17 +68,44 @@ static void __attribute__((constructor)) vfio_ap_global_init(void)
>>>       qemu_mutex_init(&cfg_chg_events_lock);
>>>   }
>>>   +static int vfio_ap_save_config(VFIODevice *vdev, QEMUFile *f, Error **errp)
>>> +{
>>> +    VFIOAPDevice *vapdev = container_of(vdev, VFIOAPDevice, vdev);
>>> +
>>> +    return vmstate_save_state(f, &vmstate_ap_device, vapdev, NULL, errp);
>>> +}
>>> +
>>> +static int vfio_ap_load_config(VFIODevice *vdev, QEMUFile *f)
>>> +{
>>> +    VFIOAPDevice *vapdev = container_of(vdev, VFIOAPDevice, vdev);
>>> +    Error *err = NULL;
>>> +    int ret;
>>> +
>>> +    ret = vmstate_load_state(f, &vmstate_ap_device, vapdev, 1, &err);
>>> +    if (ret) {
>>> +        error_report_err(err);
>>> +    }
>>> +
>>> +    return ret;
>>> +}
>>> +
>>> +static Object *vfio_ap_get_object(VFIODevice *vbasedev)
>>> +{
>>> +    VFIOAPDevice *vdev = container_of(vbasedev, VFIOAPDevice, vdev);
>>> +
>>> +    return OBJECT(vdev);
>>> +}
>>> +
>>>   static void vfio_ap_compute_needs_reset(VFIODevice *vdev)
>>>   {
>>>       vdev->needs_reset = false;
>>>   }
>>>   -/*
>>> - * We don't need vfio_hot_reset_multi and vfio_eoi operations for
>>> - * vfio-ap device now.
>>> - */
>>>   struct VFIODeviceOps vfio_ap_ops = {
>>>       .vfio_compute_needs_reset = vfio_ap_compute_needs_reset,
>>> +    .vfio_save_config = vfio_ap_save_config,
>>> +    .vfio_load_config = vfio_ap_load_config,
>>> +    .vfio_get_object = vfio_ap_get_object,
>>>   };
>>>     static void vfio_ap_req_notifier_handler(void *opaque)
>>> @@ -251,6 +280,10 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)
>>>           goto error;
>>>       }
>>>   +    if (!vfio_migration_realize(vbasedev, errp)) {
>>> +        goto error;
>>> +    }
>>> +
>>>       if (!vfio_ap_register_irq_notifier(vapdev, VFIO_AP_REQ_IRQ_INDEX, &err)) {
>>>           /*
>>>            * Report this error, but do not make it a failing condition.
>>> @@ -287,6 +320,8 @@ static void vfio_ap_unrealize(DeviceState *dev)
>>>     static const Property vfio_ap_properties[] = {
>>>       DEFINE_PROP_STRING("sysfsdev", VFIOAPDevice, vdev.sysfsdev),
>>> +    DEFINE_PROP_ON_OFF_AUTO("enable-migration", VFIOAPDevice,
>>> +                            vdev.enable_migration, ON_OFF_AUTO_ON),
>>
>> Why not ON_OFF_AUTO_AUTO ? This would enable migration only if kernel
>> has migration support. If not, the VM would still start but migration
>> would be blocked.
> 
> Will this allow the vfio-ap device migration patches to get merged before
> the kernel/KVM patches?

It's possible. Several reasons could explain the lack of migration
support. The most obvious is a lack of kernel support, but it could
also be a firmware support issue, an incompatible hardware state of
the adapter, adapter model, etc.

Do all CEXxS crypto cards support migration ?

Do you expect kernel support [*] to be merged in v7.1 ?

Also, I think the QEMU support lacks vfio_migration_exit().

Thanks,

C.

[*]  https://lore.kernel.org/r/20260407205100.331150-1-akrowiak@linux.ibm.com