Re: [PATCH v13 5/7] vfio-pci/zdev: Add a device feature for error information

Farhan Ali
2026-04-13T23:40:58Z

On 4/13/2026 3:57 PM, Alex Williamson wrote:
> On Mon, 13 Apr 2026 14:06:06 -0700
> Farhan Ali <alifm@linux.ibm.com> wrote:
>
>> For zPCI devices, we have platform specific error information. The platform
>> firmware provides this error information to the operating system in an
>> architecture specific mechanism. To enable recovery from userspace for
>> these devices, we want to expose this error information to userspace. Add a
>> new device feature to expose this information.
>>
>> Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
>> Signed-off-by: Farhan Ali <alifm@linux.ibm.com>
>> ---
>> arch/s390/include/asm/pci.h | 3 +++
>> arch/s390/pci/pci_event.c | 19 +++++++++++++++++++
>> drivers/vfio/pci/vfio_pci_core.c | 2 ++
>> drivers/vfio/pci/vfio_pci_priv.h | 9 +++++++++
>> drivers/vfio/pci/vfio_pci_zdev.c | 31 +++++++++++++++++++++++++++++++
>> include/uapi/linux/vfio.h | 20 ++++++++++++++++++++
>> 6 files changed, 84 insertions(+)
>>
>> diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h
>> index 9a6a4eb9d7c1..9c8ee97d7e8a 100644
>> --- a/arch/s390/include/asm/pci.h
>> +++ b/arch/s390/include/asm/pci.h
>> @@ -360,6 +360,9 @@ int zpci_clear_error_state(struct zpci_dev *zdev);
>> int zpci_reset_load_store_blocked(struct zpci_dev *zdev);
>> void zpci_start_mediated_recovery(struct zpci_dev *zdev);
>> void zpci_stop_mediated_recovery(struct zpci_dev *zdev);
>> +void zpci_get_pending_error_and_count(struct zpci_dev *zdev,
>> + struct zpci_ccdf_err *ccdf,
>> + int *count);
>>
>> #ifdef CONFIG_NUMA
>>
>> diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c
>> index c279a9f50a64..c8714d4a32fa 100644
>> --- a/arch/s390/pci/pci_event.c
>> +++ b/arch/s390/pci/pci_event.c
>> @@ -74,6 +74,25 @@ static void zpci_store_pci_error(struct pci_dev *pdev,
>> zdev->pending_errs.count++;
>> }
>>
>> +void zpci_get_pending_error_and_count(struct zpci_dev *zdev,
>> + struct zpci_ccdf_err *ccdf,
>> + int *count)
>> +{
>> + int head = 0;
> Unnecessary. Should also be a blank line between variable declaration
> and code.
>
>> + *count = 0;
> But why do we zero this and not ccdf?

I had thought of memsetting ccdf to 0, but the only caller right already
initializes to 0. So didn't think it was needed.

>
>> +
>> + guard(mutex)(&zdev->pending_errs_lock);
>> + if (zdev->pending_errs.count) {
>> + head = zdev->pending_errs.head % ZPCI_ERR_PENDING_MAX;
>> + memcpy(ccdf, &zdev->pending_errs.err[head],
>> + sizeof(struct zpci_ccdf_err));
>> + zdev->pending_errs.head++;
>> + zdev->pending_errs.count--;
>> + *count = zdev->pending_errs.count;
>> + }
>> +}
> You've describe in the uAPI now how pec = 0 means no error, but why not
> remove that ambiguity altogether and return -ENOMSG in that case. We
> could start here and pass it through:
>
> {
> int head;
>
> guard(mutex)(&zdev->pending_errs_lock);
>
> if (!zdev->pending_errs.count)
> return -ENOMSG;
>
> head = zdev->pending_errs.head % ZPCI_ERR_PENDING_MAX;
> memcpy(ccdf, &zdev->pending_errs.err[head],
> sizeof(struct zpci_ccdf_err));
> zdev->pending_errs.head++;
> zdev->pending_errs.count--;
> *count = zdev->pending_errs.count;
>
> return 0;
> }

I had thought of returning an error (ENOENT was what I was going with),
but perhaps I misunderstood your comment from v12 about specifying pec =
0 and pending error = 0 as not being an error. My assumption was as this
is not an error we shouldn't return an error to userspace.

>> +EXPORT_SYMBOL_GPL(zpci_get_pending_error_and_count);
>> +
>> void zpci_start_mediated_recovery(struct zpci_dev *zdev)
>> {
>> guard(mutex)(&zdev->pending_errs_lock);
>> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
>> index ad52abc46c04..5403730786a1 100644
>> --- a/drivers/vfio/pci/vfio_pci_core.c
>> +++ b/drivers/vfio/pci/vfio_pci_core.c
>> @@ -1534,6 +1534,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags,
>> return vfio_pci_core_feature_token(vdev, flags, arg, argsz);
>> case VFIO_DEVICE_FEATURE_DMA_BUF:
>> return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz);
>> + case VFIO_DEVICE_FEATURE_ZPCI_ERROR:
>> + return vfio_pci_zdev_feature_err(device, flags, arg, argsz);
>> default:
>> return -ENOTTY;
>> }
>> diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
>> index fca9d0dfac90..4e7162234a2e 100644
>> --- a/drivers/vfio/pci/vfio_pci_priv.h
>> +++ b/drivers/vfio/pci/vfio_pci_priv.h
>> @@ -93,6 +93,8 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev,
>> struct vfio_info_cap *caps);
>> int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev);
>> void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev);
>> +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags,
>> + void __user *arg, size_t argsz);
>> #else
>> static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev,
>> struct vfio_info_cap *caps)
>> @@ -107,6 +109,13 @@ static inline int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)
>>
>> static inline void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev)
>> {}
>> +
>> +static inline int vfio_pci_zdev_feature_err(struct vfio_device *device,
>> + u32 flags, void __user *arg,
>> + size_t argsz)
>> +{
>> + return -ENOTTY;
>> +}
>> #endif
>>
>> static inline bool vfio_pci_is_vga(struct pci_dev *pdev)
>> diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_zdev.c
>> index 0658095ac5b1..ee1647f0ffe6 100644
>> --- a/drivers/vfio/pci/vfio_pci_zdev.c
>> +++ b/drivers/vfio/pci/vfio_pci_zdev.c
>> @@ -141,6 +141,37 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev,
>> return ret;
>> }
>>
>> +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags,
>> + void __user *arg, size_t argsz)
>> +{
>> + struct vfio_device_feature_zpci_err err = {};
>> + struct vfio_pci_core_device *vdev;
>> + struct zpci_ccdf_err ccdf = {};
>> + struct zpci_dev *zdev;
>> + int pending_errors = 0;
>> + int ret;
>> +
>> + vdev = container_of(device, struct vfio_pci_core_device, vdev);
>> + zdev = to_zpci(vdev->pdev);
>> + if (!zdev)
>> + return -ENODEV;
>> +
>> + ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET,
>> + sizeof(err));
>> + if (ret != 1)
>> + return ret;
>> +
>> + zpci_get_pending_error_and_count(zdev, &ccdf, &pending_errors);
>> +
>> + err.version = 1;
>> + err.pec = ccdf.pec;
>> + err.pending_errors = pending_errors;
>> + if (copy_to_user(arg, &err, sizeof(err)))
>> + return -EFAULT;
>> +
>> + return 0;
>> +}
>> +
>> int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev)
>> {
>> struct zpci_dev *zdev = to_zpci(vdev->pdev);
>> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
>> index 5de618a3a5ee..2980ca39dd38 100644
>> --- a/include/uapi/linux/vfio.h
>> +++ b/include/uapi/linux/vfio.h
>> @@ -1534,6 +1534,26 @@ struct vfio_device_feature_dma_buf {
>> */
>> #define VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2 12
>>
>> +/**
>> + * VFIO_DEVICE_FEATURE_ZPCI_ERROR feature provides PCI error information to
>> + * userspace for vfio-pci devices on s390x. On s390x, PCI error recovery
>> + * involves platform firmware and notification to operating system is done
>> + * by architecture specific mechanism. Exposing this information to
>> + * userspace allows it to take appropriate actions to handle an
>> + * error on the device. The pending_errors provide any additional errors
>> + * pending for the device, and userspace should read until zero. A value of
>> + * 0 for pending_errors and pec would indicate no pending errors that need
>> + * to be handled.
>> + */
>> +
>> +struct vfio_device_feature_zpci_err {
>> + __u8 version;
>> + __u8 pending_errors;
>> + __u16 pec;
>> +};
> I assume .version is for compatibility, but we don't define a strategy
> for using it or specify what the version should be for this table. It
> doesn't seem like there's actually an value-add to having it.

Its possible we may need to extend this structure in the future if we
want to report more information to userspace. I at least want the
flexibility to do so. We had some discussion around this [1] in an
earlier version. I was trying to follow similar versioning pattern we
had around vfio-pci/zdev structures.

[1]
https://lore.kernel.org/all/88289f74-3d4f-4dd9-8f2a-8871d150fd50@linux.ibm.com/

>
> I'm also not clear why we need to report .pending_errors. It mostly
> seems like another ambiguous feature of this API. The value seems
> volatile and the suggestion is to read until zero, so why provide the
> value at all, the user can just read until -ENOMSG.

Since we don't explicitly return an error indicating there are no
available pci errors to handle, was why I added it. Is the preference to
just return linux error code here?

Thanks

Farhan

>
> At that point, maybe we don't even need a return structure at all,
> return small positive values for pec or -errno. The internal API could
> match, avoiding the pass by address parameters. Thanks,
>
> Alex
>
>
>> +
>> +#define VFIO_DEVICE_FEATURE_ZPCI_ERROR 13
>> +
>> /* -------- API for Type1 VFIO IOMMU -------- */
>>
>> /**