Re: [PATCH v2] PCI: Force PM reset for Qualcomm devices with NoSoftRst+

[Jose Ignacio Tornos Martinez <jtornosm@redhat.com>]

Hello Alex,

Thank you again for your review.                                                                                                                                               
                                                            
> What does reset_methods sysfs attribute report for these devices on an
> unpatched kernel?
The kernel we use doesn't have CONFIG_PCI_RESET_SYSFS enabled,
so reset_methods is not available. However, I can provide the actual
behavior observed through testing and dmesg logs.
 
> I'd tend to expect these are single-function devices where bus reset
> would be available as a function level reset.
Yes, these are single-function devices (PCI header type 00).
For example, here's the ath11k device:                                                                                                                                                                                 lspci -xxx -s 0000:03:00.0 | head -2
    03:00.0 Network controller: Qualcomm Technologies, Inc QCNFA765
    00: cb 17 03 11 06 05 10 00 01 00 80 02 10 00 00 00                                                                                                                                                              
                                                  ^^
                                     Header type: 00 (single-function)
 
> I'm very suspicious that this is just masking an underlying issue
> relative to bus reset for these devices
Yes, you are right, there is an underlying bus reset issue. Let me explain
what I have observed through the testing:
Testing showed no reset is performed at all. During both VM startup and
virsh reset operations, there are no reset-related messages in dmesg.
The reset hierarchy returns -ENOTTY at each step:
  - No FLR (device doesn't advertise it)
  - PM reset returns -ENOTTY (NoSoftRst+ flag)
  - Bus reset apparently not attempted
When testing the suggested quirk_no_flr() approach (which worked for
mt7925e), dmesg shows secondary bus reset is attempted:
  vfio-pci 0000:06:00.0: enabling device (0000 -> 0002)
  vfio-pci 0000:06:00.0: resetting
  pcieport 0000:00:1c.4: unlocked secondary bus reset via: __pci_reset_function_locked
  vfio-pci 0000:06:00.0: reset done
However, the device becomes unresponsive after this:
  lspci -vvvvvvvvvvvv -s 0000:03:00.0
    03:00.0 Network controller: Qualcomm Technologies, Inc (rev ff) (prog-if ff)
        !!! Unknown header type 7f
And all config space reads return 0xFF, indicating the device is not
responding after bus reset.
If we use PM reset (D3hot->D0) succeeds and the device works correctly
through multiple VM lifecycles (startup, virsh reset, shutdown/restart).
                                                                                                                                                                                                                     
> especially if we haven't actually verified the device state is                                                                                                                                                   
> actually reset on transition back to D0                                                                                                                                                                          
The verification is functional: with our patch, the device successfully
initializes in the guest after VM reset operations, and continues working
through multiple reset cycles. Without a working reset (default kernel),
WiFi devices (ath11k, ath12k) cannot be reused after VM termination, and
modem devices (SDX62/SDX65) fail to initialize even on first VM assignment.
                                                                                                                                                                                                                     
Summary:
You're correct that there's a bus reset issue, SBR breaks these devices.
The question is whether we should:
  1. Investigate why SBR breaks these single-function devices
  2. Use PM reset which demonstrably works
Option 1 may involve firmware-level investigation, while the PM reset
approach provides a working solution.
This situation is similar to existing quirks: quirk_no_flr() works around
devices with broken FLR implementations. Here we're working around devices
that incorrectly advertise NoSoftRst+ (preventing PM reset) while SBR doesn't
work properly.
I'm open to your guidance on the best path forward.

Thanks

Best regards
José Ignacio