Re: [PATCH v2 3/9] PCI: Allow per function PCI slots for hypervisor isolated functions

[Niklas Schnelle <schnelle@linux.ibm.com>]

On Mon, 2025-08-25 at 10:12 -0700, Farhan Ali wrote:
> On s390 systems, which use a machine level hypervisor, PCI devices are
> always accessed through a form of PCI pass-through which fundamentally
> operates on a per PCI function granularity. This is also reflected in the
> s390 PCI hotplug driver which creates hotplug slots for individual PCI
> functions. Its reset_slot() function, which is a wrapper for
> zpci_hot_reset_device(), thus also resets individual functions.
> 
> Currently, the kernel's PCI_SLOT() macro assigns the same pci_slot object
> to multifunction devices. This approach worked fine on s390 systems that
> only exposed virtual functions as individual PCI domains to the operating
> system.  Since commit 44510d6fa0c0 ("s390/pci: Handling multifunctions")
> s390 supports exposing the topology of multifunction PCI devices by
> grouping them in a shared PCI domain. When attempting to reset a function
> through the hotplug driver, the shared slot assignment causes the wrong
> function to be reset instead of the intended one. It also leaks memory as
> we do create a pci_slot object for the function, but don't correctly free
> it in pci_slot_release().
> 
> This patch adds a helper function to allow per function PCI slots for
> functions managed through a hypervisor which exposes individual PCI
> functions while retaining the topology.
> 
> Fixes: 44510d6fa0c0 ("s390/pci: Handling multifunctions")
> Co-developed-by: Niklas Schnelle <schnelle@linux.ibm.com>
> Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
> Signed-off-by: Farhan Ali <alifm@linux.ibm.com>
> ---
>  drivers/pci/slot.c | 19 ++++++++++++++++---
>  1 file changed, 16 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/slot.c b/drivers/pci/slot.c
> index 50fb3eb595fe..991526af0ffe 100644
> --- a/drivers/pci/slot.c
> +++ b/drivers/pci/slot.c
> @@ -5,6 +5,7 @@
>   *	Alex Chiang <achiang@hp.com>
>   */
>  
> +#include <linux/hypervisor.h>
>  #include <linux/kobject.h>
>  #include <linux/slab.h>
>  #include <linux/pci.h>
> @@ -73,7 +74,7 @@ static void pci_slot_release(struct kobject *kobj)
>  
>  	down_read(&pci_bus_sem);
>  	list_for_each_entry(dev, &slot->bus->devices, bus_list)
> -		if (PCI_SLOT(dev->devfn) == slot->number)
> +		if (dev->slot == slot->number)
>  			dev->slot = NULL;
>  	up_read(&pci_bus_sem);
>  
> @@ -160,13 +161,25 @@ static int rename_slot(struct pci_slot *slot, const char *name)
>  	return result;
>  }
>  
> +static bool pci_dev_matches_slot(struct pci_dev *dev, struct pci_slot *slot)
> +{
> +	if (hypervisor_isolated_pci_functions()) {
> +		if (dev->devfn == slot->number)
> +			return true;
> +	} else {
> +		if (PCI_SLOT(dev->devfn) == slot->number)
> +			return true;
> +	}
> +	return false;
> +}
> +
>  void pci_dev_assign_slot(struct pci_dev *dev)
>  {
>  	struct pci_slot *slot;
>  
>  	mutex_lock(&pci_slot_mutex);
>  	list_for_each_entry(slot, &dev->bus->slots, list)
> -		if (PCI_SLOT(dev->devfn) == slot->number)
> +		if (pci_dev_matches_slot(dev, slot))
>  			dev->slot = slot;
>  	mutex_unlock(&pci_slot_mutex);
>  }

Doing some more digging, I believe this also needs adjustment in
pci_dev_reset_slot_function(). Since commit 10791141a6cf ("PCI:
Simplify pci_dev_reset_slot_function()") that no longer directly looks
at the struct pci_slot linking but instead assumes that slot resets
don't work on multifunction devices. With per PCI function slots the
slot reset should work with pdev->multifunction set. I think adjusting
pci_dev_reset_slot_function() may be easier if instead of using the
hypervisor_isolated_pci_functions() helper we would set up a struct
pci_slot::per_func flag as we had considered as an option.

Thanks,
Niklas