Re: [PATCH v4 01/15] s390x/pci: implement IOMMU replay

[Matthew Rosato <mjrosato@linux.ibm.com>]

On 7/24/26 3:10 PM, Farhan Ali wrote:
> 
> On 7/23/2026 8:09 PM, Konstantin Shkolnyy wrote:
>> From: Matthew Rosato <mjrosato@linux.ibm.com>
>>
>> There are a few scenarios where IOMMU replay can potentially be needed
>> for zPCI device, namely VFIO device reset scenarios where the guest
>> continues running and expects the contents of its IOMMU to be replayed
>> upon IOAT re-registration and migration scenarios where the destination
>> must reconstruct the IOMMU on the destination.
>>
>> zPCI migration is not supported yet, but the IOMMU replay function is
>> implemented so that it can be called both from IOMMUMemoryRegionClass
>> now and migration post_load later.
>>
>> Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
>> Signed-off-by: Konstantin Shkolnyy <kshk@linux.ibm.com>
>> ---
>>   hw/s390x/s390-pci-bus.c          | 62 ++++++++++++++++++++++++++++----
>>   hw/s390x/s390-pci-inst.c         |  4 +--
>>   include/hw/s390x/s390-pci-inst.h |  1 +
>>   3 files changed, 59 insertions(+), 8 deletions(-)
>>
>> diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
>> index eff980fdfe..511926c151 100644
>> --- a/hw/s390x/s390-pci-bus.c
>> +++ b/hw/s390x/s390-pci-bus.c
>> @@ -593,14 +593,64 @@ err:
>>       return ret;
>>   }
>>   -static void s390_pci_iommu_replay(IOMMUMemoryRegion *iommu,
>> +static void s390_pci_ioat_replay(S390PCIIOMMU *iommu)
>> +{
>> +    S390IOTLBEntry entry;
>> +    uint16_t error = 0;
>> +    uint32_t dma_avail;
>> +    hwaddr curr, end;
>> +
>> +    curr = iommu->pba;
>> +    end = iommu->pal;
>> +
>> +    if (iommu->dm_mr) {
>> +        /* If direct mapping is used, there are no guest tables to
>> replay */
>> +        return;
>> +    }
>> +
>> +    if (iommu->dma_limit) {
>> +        dma_avail = iommu->dma_limit->avail;
>> +    } else {
>> +        dma_avail = 1;
>> +    }
>> +
>> +    while (curr < end) {
>> +        error = s390_guest_io_table_walk(iommu->g_iota, curr, &entry);
>> +        if (error) {
>> +            pbdev->state = ZPCI_FS_ERROR;
> 
> This patch doesn't compile on its own. Trying to compile with just this
> patch gives the error:
> 
> i-bus.c.o -c ../hw/s390x/s390-pci-bus.c
> ../hw/s390x/s390-pci-bus.c: In function ‘s390_pci_ioat_replay’:
> ../hw/s390x/s390-pci-bus.c:620:13: error: ‘pbdev’ undeclared (first use
> in this function)
>   620 |             pbdev->state = ZPCI_FS_ERROR;
>       |             ^~~~~
> ../hw/s390x/s390-pci-bus.c:620:13: note: each undeclared identifier is
> reported only once for each function it appears in
> 
> This should be fixed to avoid breaking git bisectability.
> 
Oh, thanks for pointing this out.  Not sure what happened here, I
thought I wrote the new version and tested it independent of this
series, but maybe I did it on top where we get pbdev as an input to
s390_pci_ioat_replay() and then botched the rebase.  That sounds most
likely, but I can't find my working branch at this point.

In any event, doing a S390PCIBusDevice *pbdev = iommu->pbdev; upfront
would be the most straightforward solution, and then it will go away in
patch 3 to instead be replaced by S390PCIIOMMU *iommu = pbdev->iommu;

Konstantin, I can send you another version or you can do that yourself,
up to you.

Here's a diff against this patch of what I mean, with a little extra
defensive programming:

diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
index 511926c151..21bca4d787 100644
--- a/hw/s390x/s390-pci-bus.c
+++ b/hw/s390x/s390-pci-bus.c
@@ -595,6 +595,7 @@ err:

 static void s390_pci_ioat_replay(S390PCIIOMMU *iommu)
 {
+    S390PCIBusDevice *pbdev = iommu->pbdev;
     S390IOTLBEntry entry;
     uint16_t error = 0;
     uint32_t dma_avail;
@@ -603,7 +604,7 @@ static void s390_pci_ioat_replay(S390PCIIOMMU *iommu)
     curr = iommu->pba;
     end = iommu->pal;

-    if (iommu->dm_mr) {
+    if (iommu->dm_mr || !pbdev) {
         /* If direct mapping is used, there are no guest tables to
replay */
         return;
     }