[PATCH] arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep

[PATCH] arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep

[Niklas Cassel]

Support for msi-map in the pci-ep device tree binding was added in
commit a6aed6b9c79e ("dt-bindings: PCI: pci-ep: Add support for iommu-map
and msi-map").

The PCI endpoint doorbell feature was added in commit 1c3b002c6bf6 ("PCI:
endpoint: Add RC-to-EP doorbell support using platform MSI controller").

The PCI endpoint doorbell feature requires:
-An interrupt controller that implements GIC interrupt translation
 services (ITS).
-msi-map being defined in the pcie endpoint device tree node.
-CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.

Add msi-map to the pcie3x4_ep device tree node.

With this, the pci endpoint kselftest doorbell test case passes:
 # pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
 TAP version 13
 1..1
 # Starting 1 tests from 1 test cases.
 #  RUN           pcie_ep_doorbell.DOORBELL_TEST ...
 #            OK  pcie_ep_doorbell.DOORBELL_TEST
 ok 1 pcie_ep_doorbell.DOORBELL_TEST
 # PASSED: 1 / 1 tests passed.

Cc: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Niklas Cassel <cassel@kernel.org>
---
 arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
index 90414486e466f..c0121aea791d7 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
@@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 {
 		interrupt-names = "sys", "pmc", "msg", "legacy", "err",
 				  "dma0", "dma1", "dma2", "dma3";
 		max-link-speed = <3>;
+		msi-map = <0x0000 &its1 0x0000 0x1000>;
 		num-lanes = <4>;
 		phys = <&pcie30phy>;
 		phy-names = "pcie-phy";
-- 
2.51.0


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH] arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep

[Frank Li]

On Mon, Sep 08, 2025 at 06:24:01PM +0200, Niklas Cassel wrote:
> Support for msi-map in the pci-ep device tree binding was added in
> commit a6aed6b9c79e ("dt-bindings: PCI: pci-ep: Add support for iommu-map
> and msi-map").
>
> The PCI endpoint doorbell feature was added in commit 1c3b002c6bf6 ("PCI:
> endpoint: Add RC-to-EP doorbell support using platform MSI controller").
>
> The PCI endpoint doorbell feature requires:
> -An interrupt controller that implements GIC interrupt translation
>  services (ITS).
> -msi-map being defined in the pcie endpoint device tree node.
> -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.
>
> Add msi-map to the pcie3x4_ep device tree node.
>
> With this, the pci endpoint kselftest doorbell test case passes:
>  # pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
>  TAP version 13
>  1..1
>  # Starting 1 tests from 1 test cases.
>  #  RUN           pcie_ep_doorbell.DOORBELL_TEST ...
>  #            OK  pcie_ep_doorbell.DOORBELL_TEST
>  ok 1 pcie_ep_doorbell.DOORBELL_TEST
>  # PASSED: 1 / 1 tests passed.
>
> Cc: Frank Li <Frank.Li@nxp.com>
> Signed-off-by: Niklas Cassel <cassel@kernel.org>
> ---
>  arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> index 90414486e466f..c0121aea791d7 100644
> --- a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> @@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 {
>  		interrupt-names = "sys", "pmc", "msg", "legacy", "err",
>  				  "dma0", "dma1", "dma2", "dma3";
>  		max-link-speed = <3>;
> +		msi-map = <0x0000 &its1 0x0000 0x1000>;

Not sure how your hardware map PCIe rid at EP side. If your hardware direct
use RC's RID, difference host may use differecne RID. I remember you met
similar problem when enable IOMMU.

Generally, Only 1 EP support, you can use msi-mask=<0> map all RID to a
fixed value, But it also needs change glue layer configure to force to
use the same RID.

I suppose this may not work if connected to second PCIe host slot.

Frank

>  		num-lanes = <4>;
>  		phys = <&pcie30phy>;
>  		phy-names = "pcie-phy";
> --
> 2.51.0
>
>
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH] arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep

[Niklas Cassel]

On Mon, Sep 08, 2025 at 05:26:03PM -0400, Frank Li wrote:
> On Mon, Sep 08, 2025 at 06:24:01PM +0200, Niklas Cassel wrote:
> > Support for msi-map in the pci-ep device tree binding was added in
> > commit a6aed6b9c79e ("dt-bindings: PCI: pci-ep: Add support for iommu-map
> > and msi-map").
> >
> > The PCI endpoint doorbell feature was added in commit 1c3b002c6bf6 ("PCI:
> > endpoint: Add RC-to-EP doorbell support using platform MSI controller").
> >
> > The PCI endpoint doorbell feature requires:
> > -An interrupt controller that implements GIC interrupt translation
> >  services (ITS).
> > -msi-map being defined in the pcie endpoint device tree node.
> > -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.
> >
> > Add msi-map to the pcie3x4_ep device tree node.
> >
> > With this, the pci endpoint kselftest doorbell test case passes:
> >  # pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
> >  TAP version 13
> >  1..1
> >  # Starting 1 tests from 1 test cases.
> >  #  RUN           pcie_ep_doorbell.DOORBELL_TEST ...
> >  #            OK  pcie_ep_doorbell.DOORBELL_TEST
> >  ok 1 pcie_ep_doorbell.DOORBELL_TEST
> >  # PASSED: 1 / 1 tests passed.
> >
> > Cc: Frank Li <Frank.Li@nxp.com>
> > Signed-off-by: Niklas Cassel <cassel@kernel.org>
> > ---
> >  arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> > index 90414486e466f..c0121aea791d7 100644
> > --- a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> > +++ b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> > @@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 {
> >  		interrupt-names = "sys", "pmc", "msg", "legacy", "err",
> >  				  "dma0", "dma1", "dma2", "dma3";
> >  		max-link-speed = <3>;
> > +		msi-map = <0x0000 &its1 0x0000 0x1000>;
> 
> Not sure how your hardware map PCIe rid at EP side. If your hardware direct
> use RC's RID, difference host may use differecne RID. I remember you met
> similar problem when enable IOMMU.

Yes, the Requester ID will be the BDF of the Root Complex which is sending
the request to the endpoint.


> 
> Generally, Only 1 EP support, you can use msi-mask=<0> map all RID to a
> fixed value, But it also needs change glue layer configure to force to
> use the same RID.
> 
> I suppose this may not work if connected to second PCIe host slot.

I tested the patch with three different hosts.

On Intel and Rock5b host, this patch both worked fine:
ok 1 pcie_ep_doorbell.DOORBELL_TEST

On another ARM64 board, this patch did not work.
I tried adding msi-map-mask = <0>;

But still no luck.
I'm not sure what is missing in the glue driver, because AFAICT,
the logic parsing the msi-map-mask is already in
drivers/irqchip/irq-gic-its-msi-parent.c

Also, the device tree binding in
Documentation/devicetree/bindings/pci/pci-ep.yaml

Does mention that:
         - Bits [2:0] for the function number (func)
         - Bits [18:3] for the virtual function index (vfunc)

      However, the EP cannot rely on Requester ID (RID) because the RID is
      determined by the PCI topology of the host system. Since the EP may be
      connected to different PCI hosts, the RID can vary between systems and is
      therefore not a reliable identifier.

      The resulting device ID is computed as:

        (func & 0x7) | (vfunc << 3)

      The property is an arbitrary number of tuples of
        (device-id-base, msi, msi-base,length).


Thus, I don't see that the Requester ID is used at all.

Anyway, considering that this patch does not work with all hosts,
it is best to not merge this patch for now.



Kind regards,
Niklas

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH] arm64: dts: rockchip: rk3588: add msi-map for pcie3x4_ep

[Frank Li]

On Fri, Sep 26, 2025 at 01:57:40PM +0200, Niklas Cassel wrote:
> On Mon, Sep 08, 2025 at 05:26:03PM -0400, Frank Li wrote:
> > On Mon, Sep 08, 2025 at 06:24:01PM +0200, Niklas Cassel wrote:
> > > Support for msi-map in the pci-ep device tree binding was added in
> > > commit a6aed6b9c79e ("dt-bindings: PCI: pci-ep: Add support for iommu-map
> > > and msi-map").
> > >
> > > The PCI endpoint doorbell feature was added in commit 1c3b002c6bf6 ("PCI:
> > > endpoint: Add RC-to-EP doorbell support using platform MSI controller").
> > >
> > > The PCI endpoint doorbell feature requires:
> > > -An interrupt controller that implements GIC interrupt translation
> > >  services (ITS).
> > > -msi-map being defined in the pcie endpoint device tree node.
> > > -CONFIG_PCI_ENDPOINT_MSI_DOORBELL being enabled.
> > >
> > > Add msi-map to the pcie3x4_ep device tree node.
> > >
> > > With this, the pci endpoint kselftest doorbell test case passes:
> > >  # pci_endpoint_test -r pcie_ep_doorbell.DOORBELL_TEST
> > >  TAP version 13
> > >  1..1
> > >  # Starting 1 tests from 1 test cases.
> > >  #  RUN           pcie_ep_doorbell.DOORBELL_TEST ...
> > >  #            OK  pcie_ep_doorbell.DOORBELL_TEST
> > >  ok 1 pcie_ep_doorbell.DOORBELL_TEST
> > >  # PASSED: 1 / 1 tests passed.
> > >
> > > Cc: Frank Li <Frank.Li@nxp.com>
> > > Signed-off-by: Niklas Cassel <cassel@kernel.org>
> > > ---
> > >  arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi | 1 +
> > >  1 file changed, 1 insertion(+)
> > >
> > > diff --git a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> > > index 90414486e466f..c0121aea791d7 100644
> > > --- a/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> > > +++ b/arch/arm64/boot/dts/rockchip/rk3588-extra.dtsi
> > > @@ -389,6 +389,7 @@ pcie3x4_ep: pcie-ep@fe150000 {
> > >  		interrupt-names = "sys", "pmc", "msg", "legacy", "err",
> > >  				  "dma0", "dma1", "dma2", "dma3";
> > >  		max-link-speed = <3>;
> > > +		msi-map = <0x0000 &its1 0x0000 0x1000>;
> >
> > Not sure how your hardware map PCIe rid at EP side. If your hardware direct
> > use RC's RID, difference host may use differecne RID. I remember you met
> > similar problem when enable IOMMU.
>
> Yes, the Requester ID will be the BDF of the Root Complex which is sending
> the request to the endpoint.
>
>
> >
> > Generally, Only 1 EP support, you can use msi-mask=<0> map all RID to a
> > fixed value, But it also needs change glue layer configure to force to
> > use the same RID.
> >
> > I suppose this may not work if connected to second PCIe host slot.
>
> I tested the patch with three different hosts.
>
> On Intel and Rock5b host, this patch both worked fine:
> ok 1 pcie_ep_doorbell.DOORBELL_TEST
>
> On another ARM64 board, this patch did not work.
> I tried adding msi-map-mask = <0>;
>
> But still no luck.
> I'm not sure what is missing in the glue driver, because AFAICT,
> the logic parsing the msi-map-mask is already in
> drivers/irqchip/irq-gic-its-msi-parent.c
>

The key problem is we don't know EP's RID (RC send out TLP package),  which
is transparent to software.

So layerscape have internal logic to map RID to fixed (predictable) stream
ID, which is neccessary for EP function if use IOMMU or MSI, or you can
capture RID from first cfg access from HOST.

Maybe there already have it in this SOC, suggest add it in future chips if
not support yet.

Frank

> Also, the device tree binding in
> Documentation/devicetree/bindings/pci/pci-ep.yaml
>
> Does mention that:
>          - Bits [2:0] for the function number (func)
>          - Bits [18:3] for the virtual function index (vfunc)
>
>       However, the EP cannot rely on Requester ID (RID) because the RID is
>       determined by the PCI topology of the host system. Since the EP may be
>       connected to different PCI hosts, the RID can vary between systems and is
>       therefore not a reliable identifier.
>
>       The resulting device ID is computed as:
>
>         (func & 0x7) | (vfunc << 3)
>
>       The property is an arbitrary number of tuples of
>         (device-id-base, msi, msi-base,length).
>
>
> Thus, I don't see that the Requester ID is used at all.
>
> Anyway, considering that this patch does not work with all hosts,
> it is best to not merge this patch for now.
>
>
>
> Kind regards,
> Niklas
>
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip