Re: [PATCH v3] PCI: j721e: Fix programming sequence of "strap" settings

[Siddharth Vadapalli <s-vadapalli@ti.com>]

On Mon, Sep 01, 2025 at 12:14:51PM +0530, Manivannan Sadhasivam wrote:
> On Mon, Sep 01, 2025 at 11:51:33AM GMT, Siddharth Vadapalli wrote:
> > On Mon, Sep 01, 2025 at 11:18:23AM +0530, Manivannan Sadhasivam wrote:
> > > On Mon, Sep 01, 2025 at 10:27:51AM GMT, Siddharth Vadapalli wrote:
> > > > On Sun, Aug 31, 2025 at 06:15:13PM +0530, Manivannan Sadhasivam wrote:
> > > > 
> > > > Hello Mani,
> > > > 
> > > > > On Fri, Aug 29, 2025 at 02:46:28PM GMT, Siddharth Vadapalli wrote:
> > 
> > [...]
> > 
> > > > > > +	/*
> > > > > > +	 * The PCIe Controller's registers have different "reset-values"
> > > > > > +	 * depending on the "strap" settings programmed into the PCIEn_CTRL
> > > > > > +	 * register within the CTRL_MMR memory-mapped register space.
> > > > > > +	 * The registers latch onto a "reset-value" based on the "strap"
> > > > > > +	 * settings sampled after the PCIe Controller is powered on.
> > > > > > +	 * To ensure that the "reset-values" are sampled accurately, power
> > > > > > +	 * off the PCIe Controller before programming the "strap" settings
> > > > > > +	 * and power it on after that.
> > > > > > +	 */
> > > > > > +	ret = pm_runtime_put_sync(dev);
> > > > > > +	if (ret < 0) {
> > > > > > +		dev_err(dev, "Failed to power off PCIe Controller\n");
> > > > > > +		return ret;
> > > > > > +	}
> > > > > 
> > > > > How does the controller gets powered off after pm_runtime_put_sync() since you
> > > > > do not have runtime PM callbacks? I believe the parent is turning off the power
> > > > > domain?
> > > > 
> > > > By invoking 'pm_runtime_put_sync(dev)', the ref-count is being
> > > > decremented and it results in the device being powered off. I have
> > > > verified it by printing the power domain index within the functions for
> > > > powering off and powering on devices on the J7200 SoC. Logs:
> > > > 
> > > > 	root@j7200-evm:~# modprobe pci_j721e
> > > > 	[   25.231675] [Powering On]: 240
> > > > 	[   25.234848] j721e-pcie 2910000.pcie: host bridge /bus@100000/pcie@2910000 ranges:
> > > > 	[   25.242378] j721e-pcie 2910000.pcie:       IO 0x4100001000..0x4100100fff -> 0x0000001000
> > > > 	[   25.250496] j721e-pcie 2910000.pcie:      MEM 0x4100101000..0x41ffffffff -> 0x0000101000
> > > > 	[   25.258588] j721e-pcie 2910000.pcie:   IB MEM 0x0000000000..0xffffffffffff -> 0x0000000000
> > > > 	[   25.267098] [Powering Off]: 240
> > > > 	[   25.270318] [Powering On]: 240
> > > > 	[   25.273511] [Powering On]: 187
> > > > 	[   26.372361] j721e-pcie 2910000.pcie: PCI host bridge to bus 0000:00
> > > > 	[   26.378666] pci_bus 0000:00: root bus resource [bus 00-ff]
> > > > 	[   26.384156] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0x1000-0x100fff])
> > > > 	[   26.393197] pci_bus 0000:00: root bus resource [mem 0x4100101000-0x41ffffffff] (bus address [0x00101000-0xffffffff])
> > > > 	[   26.403728] pci 0000:00:00.0: [104c:b00f] type 01 class 0x060400 PCIe Root Port
> > > > 	[   26.411044] pci 0000:00:00.0: PCI bridge to [bus 00]
> > > > 	[   26.416009] pci 0000:00:00.0:   bridge window [io  0x0000-0x0fff]
> > > > 	[   26.422091] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
> > > > 	[   26.428874] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
> > > > 	[   26.436676] pci 0000:00:00.0: supports D1
> > > > 	[   26.440699] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
> > > > 	[   26.448064] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > > > 	[   26.456274] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
> > > > 	[   26.462923] pci 0000:00:00.0: PCI bridge to [bus 01]
> > > > 	[   26.467933] pci_bus 0000:00: resource 4 [io  0x0000-0xfffff]
> > > > 	[   26.473595] pci_bus 0000:00: resource 5 [mem 0x4100101000-0x41ffffffff]
> > > > 	[   26.480337] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> > > > 	[   26.487479] pcieport 0000:00:00.0: PME: Signaling with IRQ 701
> > > > 	[   26.493909] pcieport 0000:00:00.0: AER: enabled with IRQ 701
> > > > 
> > > > In the above logs, '240' is the Power Domain Index for the PCIe
> > > > Controller on J7200 SoC. It is powered on initially before the driver is
> > > > probed.
> > > 
> > > In that case, the driver should not call pm_runtime_get_sync() in its probe.
> > > What it should do is:
> > > 
> > > 	pm_runtime_set_active()
> > > 	pm_runtime_enable()
> > 
> > If I understand correctly, are you suggesting the following?
> > 
> > j721e_pcie_probe()
> > 	pm_runtime_set_active()
> > 	pm_runtime_enable()
> > 	ret = j721e_pcie_ctrl_init(pcie);
> > 		/*
> > 		 * PCIe Controller should be powered off here, but is there
> > 		 * a way to ensure that it has been powered off?
> > 		 */
> > 		=> Program the strap settings and return to
> > 		j721e_pcie_probe()
> > 	/* Power on the PCIe Controller now */
> > 	ret = pm_runtime_get_sync(dev);
> 
> This pm_runtime_get_sync() should be part of j721e_pcie_ctrl_init() where you
> do power off, program strap and power on.
> 
> This should not be part of probe() as by that time, controller is already
> powered on. So pm_runtime_set_active() and pm_runtime_enable() should be enough
> to convey the state of the device to PM core.

I have tried out the suggestion in the following manner:

	j721e_pcie_probe()
		...
		pm_runtime_set_active(dev);
		pm_runtime_enable(dev);
		ret = j721e_pcie_ctrl_init(pcie);
			... within j721e_pcie_ctrl_init()
			ret = pm_runtime_put_sync(dev);
			/* Program Strap Settings */
			ret = pm_runtime_get_sync(dev);
			...
		...
		exit probe

Since a 'pm_runtime_get_sync()' hasn't yet been invoked prior to the
section where 'pm_runtime_put_sync()' is invoked, it leads to a ref-count
underflow error at runtime. Please let me know if I am missing
something.

The following is the working implementation from the current patch:

	j721e_pcie_probe()
		...
		/*
		 * The following two lines are identical to the existing
		 * driver code
		 */
		pm_runtime_enable(dev);
		ret = pm_runtime_get_sync(dev);
		...
		ret = j721e_pcie_ctrl_init(pcie);
			... within j721e_pcie_ctrl_init()
			ret = pm_runtime_put_sync(dev);
			...
			/* PCIe Controller is now powered off */
			/* Program the strap settings */
			...
			ret = pm_runtime_get_sync(dev);
			...
		...
		exit probe

Regards,
Siddharth.