Re: [PATCH v2 0/4] PCI EP driver support MSI doorbell from host

[Bjorn Helgaas <helgaas@kernel.org>]

On Fri, Jul 15, 2022 at 02:22:15PM -0500, Frank Li wrote:
> 
>                   ┌───────┐          ┌──────────┐
>                   │       │          │          │
> ┌─────────────┐   │       │          │ PCI Host │
> │ MSI         │◄┐ │       │          │          │
> │ Controller  │ │ │       │          │          │
> └─────────────┘ └─┼───────┼──────────┼─Bar0     │
>                   │ PCI   │          │ Bar1     │
>                   │ Func  │          │ Bar2     │
>                   │       │          │ Bar3     │
>                   │       │          │ Bar4     │
>                   │       ├─────────►│          │
>                   └───────┘          └──────────┘

Nice diagram and explanation.  I suggest rewrapping to fit in 75
columns and including in one of the patches, probably the
pci-epf-vntb.c one.  Then it will be more likely to make it to the git
history where it will be useful.

> Many PCI controllers provided Endpoint functions.
> Generally PCI endpoint is hardware, which is not running a rich OS, like linux.
> 
> But Linux also supports endpoint functions.  PCI Host write bar<n> space like
> write to memory. The EP side can't know memory changed by the Host driver. 
>
> PCI Spec has not defined a standard method to do that.  Only define MSI(x) to let
> EP notified RC status change. 
> 
> The basic idea is to trigger an irq when PCI RC writes to a memory address.  That's
> what MSI controller provided.  EP drivers just need to request a platform MSI interrupt, 
> struct msi_msg *msg will pass down a memory address and data.  EP driver will
> map such memory address to one of PCI bar<n>.  Host just writes such an address to
> trigger EP side irq.
> 
> If system have gic-its, only need update PCI EP side driver. But i.MX have not chip
> support gic-ites yet. So we have to use MU to simulate a MSI controller. Although
> only 4 MSI irqs are simulated, it matched vntd network requirmenent.
> 
> After enable MSI, ping delay reduce < 1ms from ~8ms
> 
> irqchip: imx mu worked as msi controller: 
>      let imx mu worked as MSI controllers. Although IP is not design as MSI controller,
> we still can use it if limiated irq number to 4.
> 
> pcie: endpoint: pci-epf-vntb: add endpoint msi support
> 	 Based on ntb-next branch. https://github.com/jonmason/ntb/commits/ntb-next
> 	 Using MSI as door bell registers
> 
> i.MX EP function driver is upstreaming by Richard Zhu.
> Some dts change missed at this patches. below is reference dts change

s/bar/BAR/ (several)
s/irq/IRQ/ (several)
s/irqs/IRQs/
s/msi/MSI/
s/gic-ites/?  (capitalize if it's an acronym)
s/requirmenent/requirement/
s/limiated/limited/

You use both "gic-its" and "gic-ites".  I assume they should be the
same.

Not sure what "vntd" refers to.  Capitalize if it's an acronym.

> --- a/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi
> @@ -160,5 +160,6 @@ pcieb_ep: pcie_ep@5f010000 {
>                 num-ib-windows = <6>;
>                 num-ob-windows = <6>;
>                 status = "disabled";
> +               msi-parent = <&lsio_mu12>;
>         };
> 
> --- a/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi
> @@ -172,6 +172,19 @@ lsio_mu6: mailbox@5d210000 {
>                 status = "disabled";
>         };
> 
> +       lsio_mu12: mailbox@5d270000 {
> +               compatible = "fsl,imx6sx-mu-msi";
> +               msi-controller;
> +               interrupt-controller;
> +               reg = <0x5d270000 0x10000>,     /* A side */
> +                     <0x5d300000 0x10000>;     /* B side */
> +               reg-names = "a", "b";
> +               interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_HIGH>;
> +               power-domains = <&pd IMX_SC_R_MU_12A>,
> +                               <&pd IMX_SC_R_MU_12B>;
> +               power-domain-names = "a", "b";
> +       };
> +
> 
> Change Log
> - from V1 to V2
>   Fixed fsl,mu-msi.yaml's problem
>   Fixed irq-imx-mu-msi.c problem according Marc Zyngier's feeback 
>   Added a new patch to allow pass down .pm by IRQCHIP_PLATFORM_DRIVER_END
> 
> -- 
> 2.35.1
>