Re: pcie hotplug driver probe is not getting called

[Maverickk 78 <maverickk1778@gmail.com>]

Update:

Hi Bjorn,

Looks like the pciehp_probe is invoked and the corresponding ISR
(pciehp_isr) is triggered if there is an event, but somehow only
pciehp_isr is traced by ftrace but not the probe function.
Maybe my assumptions of ftrace need correction.

Thanks for the response.

Regards


On Sun, 29 Sept 2024 at 19:29, Maverickk 78 <maverickk1778@gmail.com> wrote:
>
> Hi Bjorn,
>
> I have a switch connecting to the Host bridge, one of the downstream port(02:1.0) on the switch has the slot enabled.
>
> Appended pcie_ports=native along with pciehp.pciehp_force=1 pciehp.pciehp_debug=1  to the cmdline and I see the driver creating symlink to sysfs device node.
>
> Does this mean pciehp can handle the hotplug events? asking this because none of the functions in pciehp_core listed in ftrace?
>
> # uname -a
>
> Linux qemu-01 6.11.0+ #2 SMP PREEMPT_DYNAMIC Sat Sep 28 01:32:57 EEST 2024 x86_64 x86_64 x86_64 GNU/Linux
>
> # cat /proc/cmdline
>
> BOOT_IMAGE=/boot/vmlinuz-6.11.0+ root=UUID=f563804b-1b93-4921-90e1-4114c8111e8f ro ftrace=function_graph ftrace_graph_filter=*pcie* pciehp.pciehp_force=1 pciehp.pciehp_debug=1 pcie_ports=native quite splash crashkernel=512M-:192M vt.handoff=7
>
> # ls -ltr /sys/bus/pci_express/drivers/pciehp
>
> total 0
>
> --w------- 1 root root 4096 Sep 29 16:46 uevent
>
> --w------- 1 root root 4096 Sep 29 16:49 unbind
>
> --w------- 1 root root 4096 Sep 29 16:49 bind
>
> lrwxrwxrwx 1 root root    0 Sep 29 16:49 0000:02:01.0:pcie204 -> ../../../../devices/pci0000:00/0000:00:04.0/0000:01:00.0/0000:02:01.0/0000:02:01.0:pcie204
>
> #
>
>
> # lspci -vv -s 2:1.0
>
> 02:01.0 PCI bridge: Broadcom / LSI Device c040 (rev a0) (prog-if 00 [Normal decode])
>
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
>
>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>
>         Latency: 0
>
>         Interrupt: pin ? routed to IRQ 25
>
>         Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
>
>         I/O behind bridge: 00001000-00001fff [size=4K]
>
>         Memory behind bridge: f8000000-f9ffffff [size=32M]
>
>         Prefetchable memory behind bridge: 00000000fe200000-00000000fe3fffff [size=2M]
>
>         Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>
>         BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
>
>                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>
>         Capabilities: [40] Power Management version 3
>
>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
>
>                 Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
>
>         Capabilities: [48] MSI: Enable+ Count=1/8 Maskable+ 64bit+
>
>                 Address: 00000000fee03000  Data: 0020
>
>                 Masking: 000000fe  Pending: 00000000
>
>         Capabilities: [68] Express (v2) Downstream Port (Slot+), MSI 00
>
>                 DevCap: MaxPayload 512 bytes, PhantFunc 0
>
>                         ExtTag- RBE+
>
>                 DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
>
>                         RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
>
>                         MaxPayload 512 bytes, MaxReadReq 128 bytes
>
>                 DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
>
>                 LnkCap: Port #0, Speed unknown, Width x2, ASPM L1, Exit Latency L1 <32us
>
>                         ClockPM- Surprise+ LLActRep+ BwNot+ ASPMOptComp+
>
>                 LnkCtl: ASPM Disabled; Disabled- CommClk-
>
>                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
>
>                 LnkSta: Speed 32GT/s (downgraded), Width x2 (ok)
>
>                         TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
>
>                 SltCap: AttnBtn+ PwrCtrl+ MRL+ AttnInd+ PwrInd+ HotPlug+ Surprise+
>
>                         Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
>
>                 SltCtl: Enable: AttnBtn+ PwrFlt- MRL- PresDet- CmdCplt+ HPIrq+ LinkChg+
>
>                         Control: AttnInd Off, PwrInd Off, Power+ Interlock-
>
>                 SltSta: Status: AttnBtn- PowerFlt- MRL+ CmdCplt- PresDet- Interlock-
>
>                         Changed: MRL- PresDet- LinkState-
>
>                 DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR+
>
>                          10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 4
>
>                          EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
>
>                          FRS- ARIFwd+
>
>                          AtomicOpsCap: Routing+
>
>                 DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
>
>                          AtomicOpsCtl: EgressBlck-
>
>                 LnkCap2: Supported Link Speeds: RsvdP, Crosslink+ Retimer+ 2Retimers+ DRS+
>
>                 LnkCtl2: Target Link Speed: Unknown, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
>
>                          Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
>
>                          Compliance De-emphasis: -6dB
>
>                 LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
>
>                          EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
>
>                          Retimer- 2Retimers- CrosslinkRes: Downstream Port, DRS-
>
>                          DownstreamComp: Link Up - Present
>
>         Capabilities: [a4] Subsystem: Broadcom / LSI Device 0144
>
>         Capabilities: [100 v1] Extended Capability ID 0x2f
>
>         Capabilities: [294 v3] Advanced Error Reporting
>
>                 UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
>
>                 UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
>
>                 UESvrt: DLP+ SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
>
>                 CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
>
>                 CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
>
>                 AERCap: First Error Pointer: 1f, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
>
>                         MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
>
>                 HeaderLog: 00000000 00000000 00000000 00000000
>
>         Capabilities: [138 v1] Power Budgeting <?>
>
>         Capabilities: [db4 v1] Secondary PCI Express
>
>                 LnkCtl3: LnkEquIntrruptEn- PerformEqu-
>
>                 LaneErrStat: 0
>
>         Capabilities: [148 v1] Virtual Channel
>
>                 Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
>
>                 Arb:    Fixed- WRR32- WRR64- WRR128-
>
>                 Ctrl:   ArbSelect=Fixed
>
>                 Status: InProgress-
>
>                 VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
>
>                         Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
>
>                         Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
>
>                         Status: NegoPending- InProgress-
>
>         Capabilities: [af4 v1] Data Link Feature <?>
>
>         Capabilities: [d00 v1] Physical Layer 16.0 GT/s <?>
>
>         Capabilities: [d40 v1] Lane Margining at the Receiver <?>
>
>         Capabilities: [e40 v1] Extended Capability ID 0x2a
>
>         Capabilities: [e70 v1] Extended Capability ID 0x31
>
>         Capabilities: [ec0 v1] Extended Capability ID 0x32
>
>         Capabilities: [a80 v1] Extended Capability ID 0x34
>
>         Capabilities: [b70 v1] Vendor Specific Information: ID=0001 Rev=0 Len=010 <?>
>
>         Capabilities: [f24 v1] Access Control Services
>
>                 ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
>
>                 ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
>
>         Capabilities: [b60 v1] Downstream Port Containment
>
>                 DpcCap: INT Msg #0, RPExt- PoisonedTLP+ SwTrigger+ RP PIO Log 0, DL_ActiveErr+
>
>                 DpcCtl: Trigger:0 Cmpl- INT- ErrCor- PoisonedTLP- SwTrigger- DL_ActiveErr-
>
>                 DpcSta: Trigger- Reason:00 INT- RPBusy- TriggerExt:00 RP PIO ErrPtr:00
>
>                 Source: 0000
>
>         Capabilities: [b20 v1] Extended Capability ID 0x2c
>
>         Kernel driver in use: pcieport
>
>
> On Fri, 27 Sept 2024 at 22:47, Bjorn Helgaas <helgaas@kernel.org> wrote:
>>
>> On Fri, Sep 27, 2024 at 08:50:41PM +0530, Maverickk 78 wrote:
>> > Hello
>> >
>> > Debugging a downstream port with slot capabilities indicating hotplug
>> > capability is advertised in pci capability(id =0x10) .
>> >
>> > None of the hotplug driver is getting invoked.
>> >
>> > I assume pciehp_probe should've been called because its associated
>> > with ".port_type = PCIE_ANY_PORT," in the pcie_port_service_driver
>> > structure.
>> >
>> > I assumed SHPC shpc_probe function would be called because the pci_id
>> > table has PCI_CLASS_BRIDGE_PCI_NORMAL, but nothing related to hotplug
>> > drivers is seen in the ftrace or dmesg.
>> >
>> > Tried "pciehp.pciehp_force=1 pciehp.pciehp_debug=1" in the command
>> > line but no luck
>> >
>> > As part of port initialization if the hotplug capability is indicated
>> > in the capability register the hotplug drivers should have been
>> > invoked, but looks like its not the case.
>>
>> I would expect pciehp to work in this case, but there is some
>> negotiation between the OS and the firmware to figure out which
>> owns it.
>>
>> I assume you have CONFIG_PCIEPORTBUS and CONFIG_HOTPLUG_PCI_PCIE
>> enabled?  Can you supply the dmesg log and output of "sudo lspci -vv"?
>>
>> Bjorn