Re: [PATCH net-next 6/6] mlxsw: pci: Add support for new reset flow

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 30 Mar 2023 11:26:30 +0300
Ido Schimmel <idosch@nvidia.com> wrote:

> On Wed, Mar 29, 2023 at 11:10:01AM -0600, Alex Williamson wrote:
> > I think we don't have it because it's unclear how it's actually
> > different from a secondary bus reset from the bridge control register,
> > which is what "bus" would do when selected for the example above.  Per
> > the spec, both must cause a hot reset.  It seems this device needs a
> > significantly longer delay though.  
> 
> Assuming you are referring to the 2ms sleep in
> pci_reset_secondary_bus(), then yes. In our case, after disabling the
> link on the downstream port we need to wait for 500ms before enabling
> it.
> 
> > Note that hot resets can be generated by a userspace driver with
> > ownership of the device and will make use of the pci-core reset
> > mechanisms.  Therefore if there is not a device specific reset, we'll
> > use the standard delays and the device ought not to get itself wedged
> > if the link becomes active at an unexpected point relative to a
> > firmware update.  This might be a point in favor of a device specific
> > reset solution in pci-core.  Thanks,  
> 
> I assume you referring to something like this:
> 
> # echo 1 > /sys/class/pci_bus/0000:03/device/0000:03:00.0/reset
> 
> Doesn't seem to have any effect (network ports remain up, at least).
> Anyway, this device is completely managed by the kernel, not a user
> space driver. I'm not aware of anyone using this method to reset the
> device.

The pci-sysfs reset attribute is only meant to reset the linked device,
so if this is a single function device then it might be accessing bus
reset, but it also might be using FLR or PM reset.  There's a
reset_method attribute to determine and select.

In any case, if the device is unaffected, that suggests we're dealing
with a device that doesn't comply with PCIe reset standards, which
might suggests it needs a device specific reset or to flag broken reset
methods regardless.

Note that QEMU is a vfio-pci userspace driver, so assigning the device
to a VM, where kernel drivers in the guest are managing the device is a
use case of userspace drivers which should have a functional reset
mechanism to avoid data leakage between userspace sessions.
 
> If I understand Bjorn and you correctly, we have two options:
> 
> 1. Keep the current implementation inside the driver.
> 
> 2. Call __pci_reset_function_locked() from the driver and move the link
> toggling to drivers/pci/quirks.c as a "device_specific" method.
> 
> Personally, I don't see any benefit in 2, but we can try to implement
> it, see if it even works and then decide.

The second option enables use cases like above, where the PCI-core can
perform an effective reset of the device rather than embedding that
into a specific driver.  Even if not intended as a primary use case,
it's a more complete solution and avoids potentially unhappy users that
assume such use cases are available.  Thanks,

Alex