[PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[Koen Vandeputte]

The subordinate value indicates the highest bus number which can be
reached downstream though a certain device.

Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
parent")
ensures that downstream devices cannot assign busnumbers higher than the
upstream device subordinate number, which was indeed illogical.

By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
value of 0x01.

Due to this combined with above commit, enumeration stops digging deeper
downstream as soon as bus num 0x01 has been assigned, which is always
the case for a bridge device.

This results in all devices behind a bridge bus to remain undetected, as
these would be connected to bus 0x02 or higher.

Fix this by initializing the RC to a subordinate value of 0xff, meaning
that all busses [0x00-0xff] are reachable through this RC.

Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
parent")
Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
Tested-by: Niklas Cassel <niklas.cassel@axis.com>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
---

Will send separate patches to stable as this file got moved/renamed


 drivers/pci/dwc/pcie-designware-host.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
index bf558df5b7b3..2b5470173196 100644
--- a/drivers/pci/dwc/pcie-designware-host.c
+++ b/drivers/pci/dwc/pcie-designware-host.c
@@ -616,7 +616,7 @@ void dw_pcie_setup_rc(struct pcie_port *pp)
 	/* setup bus numbers */
 	val = dw_pcie_readl_dbi(pci, PCI_PRIMARY_BUS);
 	val &= 0xff000000;
-	val |= 0x00010100;
+	val |= 0x00ff0100;
 	dw_pcie_writel_dbi(pci, PCI_PRIMARY_BUS, val);
 
 	/* setup command register */
-- 
2.7.4

Re: [PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[Lorenzo Pieralisi]

On Tue, Jan 09, 2018 at 03:42:21PM +0100, Koen Vandeputte wrote:
> The subordinate value indicates the highest bus number which can be
> reached downstream though a certain device.
> 
> Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> ensures that downstream devices cannot assign busnumbers higher than the
> upstream device subordinate number, which was indeed illogical.
> 
> By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> value of 0x01.
> 
> Due to this combined with above commit, enumeration stops digging deeper
> downstream as soon as bus num 0x01 has been assigned, which is always
> the case for a bridge device.
> 
> This results in all devices behind a bridge bus to remain undetected, as
> these would be connected to bus 0x02 or higher.
> 
> Fix this by initializing the RC to a subordinate value of 0xff, meaning
> that all busses [0x00-0xff] are reachable through this RC.

This is not a correct description of the problem. AFAICS all busses
are reachable through this RC _regardless_ of whatever subordinate
bus number value you programme into it.

You should extend the CC list to all dwc host submaintainers so
that you can actually get it tested.

> Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> parent")
> Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
> ---
> 
> Will send separate patches to stable as this file got moved/renamed

Fixes: commit appeared at v4.15-rc1 (and v4.15 has not been released
yet) - there is no separate patch to be sent.

Thanks,
Lorenzo

>  drivers/pci/dwc/pcie-designware-host.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/dwc/pcie-designware-host.c b/drivers/pci/dwc/pcie-designware-host.c
> index bf558df5b7b3..2b5470173196 100644
> --- a/drivers/pci/dwc/pcie-designware-host.c
> +++ b/drivers/pci/dwc/pcie-designware-host.c
> @@ -616,7 +616,7 @@ void dw_pcie_setup_rc(struct pcie_port *pp)
>  	/* setup bus numbers */
>  	val = dw_pcie_readl_dbi(pci, PCI_PRIMARY_BUS);
>  	val &= 0xff000000;
> -	val |= 0x00010100;
> +	val |= 0x00ff0100;
>  	dw_pcie_writel_dbi(pci, PCI_PRIMARY_BUS, val);
>  
>  	/* setup command register */
> -- 
> 2.7.4
> 

Re: [PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[Mika Westerberg]

On Tue, Jan 09, 2018 at 03:25:58PM +0000, Lorenzo Pieralisi wrote:
> On Tue, Jan 09, 2018 at 03:42:21PM +0100, Koen Vandeputte wrote:
> > The subordinate value indicates the highest bus number which can be
> > reached downstream though a certain device.
> > 
> > Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> > parent")
> > ensures that downstream devices cannot assign busnumbers higher than the
> > upstream device subordinate number, which was indeed illogical.
> > 
> > By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> > value of 0x01.
> > 
> > Due to this combined with above commit, enumeration stops digging deeper
> > downstream as soon as bus num 0x01 has been assigned, which is always
> > the case for a bridge device.
> > 
> > This results in all devices behind a bridge bus to remain undetected, as
> > these would be connected to bus 0x02 or higher.
> > 
> > Fix this by initializing the RC to a subordinate value of 0xff, meaning
> > that all busses [0x00-0xff] are reachable through this RC.
> 
> This is not a correct description of the problem. AFAICS all busses
> are reachable through this RC _regardless_ of whatever subordinate
> bus number value you programme into it.

Type 1 (the ones directed to the other side of the bridge) configuration
transactions are not forwarded if the bus number in the transaction is
not included in secondary and subordinate numbers of the root bridge. I
think the description here is pretty accurate as the bridges (anything
higher than bus number 1) below are effectively hidden.

Re: [PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[Lorenzo Pieralisi]

On Tue, Jan 09, 2018 at 10:00:25PM +0200, Mika Westerberg wrote:
> On Tue, Jan 09, 2018 at 03:25:58PM +0000, Lorenzo Pieralisi wrote:
> > On Tue, Jan 09, 2018 at 03:42:21PM +0100, Koen Vandeputte wrote:
> > > The subordinate value indicates the highest bus number which can be
> > > reached downstream though a certain device.
> > > 
> > > Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
> > > parent")
> > > ensures that downstream devices cannot assign busnumbers higher than the
> > > upstream device subordinate number, which was indeed illogical.
> > > 
> > > By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
> > > value of 0x01.
> > > 
> > > Due to this combined with above commit, enumeration stops digging deeper
> > > downstream as soon as bus num 0x01 has been assigned, which is always
> > > the case for a bridge device.
> > > 
> > > This results in all devices behind a bridge bus to remain undetected, as
> > > these would be connected to bus 0x02 or higher.
> > > 
> > > Fix this by initializing the RC to a subordinate value of 0xff, meaning
> > > that all busses [0x00-0xff] are reachable through this RC.
> > 
> > This is not a correct description of the problem. AFAICS all busses
> > are reachable through this RC _regardless_ of whatever subordinate
> > bus number value you programme into it.
> 
> Type 1 (the ones directed to the other side of the bridge) configuration
> transactions are not forwarded if the bus number in the transaction is
> not included in secondary and subordinate numbers of the root bridge. I
> think the description here is pretty accurate as the bridges (anything
> higher than bus number 1) below are effectively hidden.

That's correct - but that's not what this patch is fixing which is
what the commit log should describe.

See:

https://marc.info/?l=linux-arm-kernel&m=151540153522730&w=2

Re: [PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[Koen Vandeputte]

Hi Lorenzo,

Apologies for the late reply, It's been really busy over here.


On 2018-01-09 16:25, Lorenzo Pieralisi wrote:
> <snip>
>> Fix this by initializing the RC to a subordinate value of 0xff, meaning
>> that all busses [0x00-0xff] are reachable through this RC.
> This is not a correct description of the problem. AFAICS all busses
> are reachable through this RC _regardless_ of whatever subordinate
> bus number value you programme into it.
Noted.

This was written meaning: "as seen be the probing functions below"
I'll try harder in V2 to actually include the message that it's not a HW 
related problem or influencing HW in any way, but really "tricking" the 
probing functions below
>
> You should extend the CC list to all dwc host submaintainers so
> that you can actually get it tested.
How to figure out who to include? (& please provide who to include)
Reading online manuals on "how to send patches" only demo's 
"get_maintainer.pl" script

>> <snip>
>> ---
>>
>> Will send separate patches to stable as this file got moved/renamed
> Fixes: commit appeared at v4.15-rc1 (and v4.15 has not been released
> yet) - there is no separate patch to be sent.
This is something typical which is hard to learn/understand by just 
reading "how to send patches" docs available everywhere. (like I did 
before sending this one over ;-) )

2 questions basically:
- As the commit causing it was included in 4.15-*RC1*,  do I need to add 
a "Fixes; bla bla" at all?
- As the commit causing it was backported to 4.9, (how) should I send a 
separate patch in order to get it fixed there? [0]

> Thanks,
> Lorenzo

Probably annoying questions triggering a 'sigh' .. but I'm lacking 
experience here ..

Thanks for your time & patience so far,
Highly appreciated,

Koen


[0] 
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.9.76&id=9a4bf05126f42c2632729ab0da503021d74ed454

Re: [PATCH] PCI: dwc: fix enumeration end when reaching root subordinate

[Lorenzo Pieralisi]

On Fri, Jan 12, 2018 at 04:56:33PM +0100, Koen Vandeputte wrote:
> Hi Lorenzo,
> 
> Apologies for the late reply, It's been really busy over here.
> 
> 
> On 2018-01-09 16:25, Lorenzo Pieralisi wrote:
> ><snip>
> >>Fix this by initializing the RC to a subordinate value of 0xff, meaning
> >>that all busses [0x00-0xff] are reachable through this RC.
> >This is not a correct description of the problem. AFAICS all busses
> >are reachable through this RC _regardless_ of whatever subordinate
> >bus number value you programme into it.
> Noted.
> 
> This was written meaning: "as seen be the probing functions below"
> I'll try harder in V2 to actually include the message that it's not
> a HW related problem or influencing HW in any way, but really
> "tricking" the probing functions below

Ok, add it to the log please.

> >You should extend the CC list to all dwc host submaintainers so
> >that you can actually get it tested.
> How to figure out who to include? (& please provide who to include)
> Reading online manuals on "how to send patches" only demo's
> "get_maintainer.pl" script

Well, you can do it manually, by checking all pci/dwc submaintainers
in MAINTAINERS.

There are other ways of course but it is simple enough.

> >><snip>
> >>---
> >>
> >>Will send separate patches to stable as this file got moved/renamed
> >Fixes: commit appeared at v4.15-rc1 (and v4.15 has not been released
> >yet) - there is no separate patch to be sent.
> This is something typical which is hard to learn/understand by just
> reading "how to send patches" docs available everywhere. (like I did
> before sending this one over ;-) )
> 
> 2 questions basically:
> - As the commit causing it was included in 4.15-*RC1*,  do I need to
> add a "Fixes; bla bla" at all?

Yes.

> - As the commit causing it was backported to 4.9, (how) should I
> send a separate patch in order to get it fixed there? [0]

So submitting a stable tag does make sense, sorry I missed that.

Or we can send to stable kernel specific backports.

> Probably annoying questions triggering a 'sigh' .. but I'm lacking
> experience here ..

No way, thank you for fixing it !

> Thanks for your time & patience so far,
> Highly appreciated,

Same here, thank you.

Lorenzo

[PATCH] PCI: dwc: Fix enumeration end when reaching root subordinate

[Fabio Estevam]

From: Koen Vandeputte <koen.vandeputte@ncentric.com>

commit fc110ebdd014dd1368c98e7685b47789c31fab42 upstream.

The subordinate value indicates the highest bus number which can be
reached downstream though a certain device.

Commit a20c7f36bd3d ("PCI: Do not allocate more buses than available in
parent") ensures that downstream devices cannot assign busnumbers higher
than the upstream device subordinate number, which was indeed illogical.

By default, dw_pcie_setup_rc() inits the Root Complex subordinate to a
value of 0x01.

Due to this combined with above commit, enumeration stops digging deeper
downstream as soon as bus num 0x01 has been assigned, which is always the
case for a bridge device.

This results in all devices behind a bridge bus remaining undetected, as
these would be connected to bus 0x02 or higher.

Fix this by initializing the RC to a subordinate value of 0xff, which is
not altering hardware behaviour in any way, but informs probing function
pci_scan_bridge() later on which reads this value back from register.

The following nasty errors during boot are also fixed by this:

  pci_bus 0000:02: busn_res: can not insert [bus 02-ff] under [bus 01] (conflicts with (null) [bus 01])
  ...
  pci_bus 0000:03: [bus 03] partially hidden behind bridge 0000:01 [bus 01]
  ...
  pci_bus 0000:04: [bus 04] partially hidden behind bridge 0000:01 [bus 01]
  ...
  pci_bus 0000:05: [bus 05] partially hidden behind bridge 0000:01 [bus 01]
  pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 05
  pci_bus 0000:02: busn_res: can not insert [bus 02-05] under [bus 01] (conflicts with (null) [bus 01])
  pci_bus 0000:02: [bus 02-05] partially hidden behind bridge 0000:01 [bus 01]

Fixes: a20c7f36bd3d ("PCI: Do not allocate more buses than available in
parent")
Tested-by: Niklas Cassel <niklas.cassel@axis.com>
Tested-by: Fabio Estevam <fabio.estevam@nxp.com>
Tested-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>
Signed-off-by: Koen Vandeputte <koen.vandeputte@ncentric.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: Lucas Stach <l.stach@pengutronix.de>
Cc: stable@vger.kernel.org
Cc: Binghui Wang <wangbinghui@hisilicon.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Jianguo Sun <sunjianguo1@huawei.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
Cc: Minghuan Lian <minghuan.Lian@freescale.com>
Cc: Mingkai Hu <mingkai.hu@freescale.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Roy Zang <tie-fei.zang@freescale.com>
Cc: Shawn Guo <shawn.guo@linaro.org>
Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Xiaowei Song <songxiaowei@hisilicon.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>
[fabio: adapted to the file location of 4.9 kernel]
Signed-off-by: Fabio Estevam <fabio.estevam@nxp.com>
---
 drivers/pci/host/pcie-designware.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index af8f6e9..b3a8715 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -861,7 +861,7 @@ void dw_pcie_setup_rc(struct pcie_port *pp)
 	/* setup bus numbers */
 	val = dw_pcie_readl_rc(pp, PCI_PRIMARY_BUS);
 	val &= 0xff000000;
-	val |= 0x00010100;
+	val |= 0x00ff0100;
 	dw_pcie_writel_rc(pp, PCI_PRIMARY_BUS, val);
 
 	/* setup command register */
-- 
2.7.4