Re: [PATCH v2 0/2] PCI: xgene: Restore working PCIe functionnality

[Rob Herring <robh@kernel.org>]

On Mon, Mar 21, 2022 at 04:32:40PM -0600, dann frazier wrote:
> On Mon, Mar 21, 2022 at 11:08:20AM -0500, Rob Herring wrote:
> > On Mon, Mar 21, 2022 at 09:50:24AM -0600, dann frazier wrote:
> > > On Mon, Mar 21, 2022 at 10:17:34AM -0500, Rob Herring wrote:
> > > > On Mon, Mar 21, 2022 at 5:49 AM Marc Zyngier <maz@kernel.org> wrote:
> > > > >
> > > > > Since 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup") was
> > > > > merged in the 5.5 time frame, PCIe on the venerable XGene platform has
> > > > > been unusable: 6dce5aa59e0b broke both XGene-1 (Mustang and m400) and
> > > > > XGene-2 (Merlin), while the addition of c7a75d07827a ("PCI: xgene: Fix
> > > > > IB window setup") fixed XGene-2, but left the rest of the zoo
> > > > > unusable.
> > > > >
> > > > > It is understood that this systems come with "creative" DTs that don't
> > > > > match the expectations of modern kernels. However, there is little to
> > > > > be gained by forcing these changes on users -- the firmware is not
> > > > > upgradable, and the current owner of the IP will deny that these
> > > > > machines have ever existed.
> > > > 
> > > > The gain for fixing this properly is not having drivers do their own
> > > > dma-ranges parsing. We've seen what happens when drivers do their own
> > > > parsing of standard properties (e.g. interrupt-map). Currently, we
> > > > don't have any drivers doing their own parsing:
> > > > 
> > > > $ git grep of_pci_dma_range_parser_init
> > > > drivers/of/address.c:int of_pci_dma_range_parser_init(struct
> > > > of_pci_range_parser *parser,
> > > > drivers/of/address.c:EXPORT_SYMBOL_GPL(of_pci_dma_range_parser_init);
> > > > drivers/of/address.c:#define of_dma_range_parser_init
> > > > of_pci_dma_range_parser_init
> > > > drivers/of/unittest.c:  if (of_pci_dma_range_parser_init(&parser, np)) {
> > > > drivers/pci/of.c:       err = of_pci_dma_range_parser_init(&parser, dev_node);
> > > > include/linux/of_address.h:extern int
> > > > of_pci_dma_range_parser_init(struct of_pci_range_parser *parser,
> > > > include/linux/of_address.h:static inline int
> > > > of_pci_dma_range_parser_init(struct of_pci_range_parser *parser,
> > > > 
> > > > And we can probably further refactor this to be private to drivers/pci/of.c.
> > > > 
> > > > For XGene-2 the issue is simply that the driver depends on the order
> > > > of dma-ranges entries.
> > > > 
> > > > For XGene-1, I'd still like to understand what the issue is. Reverting
> > > > the first fix and fixing 'dma-ranges' should have fixed it. I need a
> > > > dump of how the IB registers are initialized in both cases.
> > > 
> > > Happy to provide that for the m400 if told how :)
> > 
> > Something like the below patch. This should be with the 'dma-ranges' 
> > DT change and only c7a75d07827a reverted.
> 
> https://paste.ubuntu.com/p/RHzBd5jT6v/
> 
> Note that networking does come up with this setup. That surprised me
> because I thought I'd tested this combo before, but apparently what
> I'd tested before was 6dce5aa59e0b reverted + the dtb change:
>   https://lore.kernel.org/linux-pci/YgXG838iMrS1l8SC@xps13.dannf/

That doesn't make sense. I just noticed there's an error in what I 
told you to do for dma-ranges. I fixed the wrong cell as it should be:

- dma-ranges = <0x42000000 0x40 0x00 0x40 0x00 0x40 0x00 0x00 0x00 0x79000000 0x00 0x79000000 0x00 0x800000>;
+ dma-ranges = <0x42000000 0x40 0x00 0x40 0x00 0x40 0x00 0x42000000 0x00 0x79000000 0x00 0x79000000 0x00 0x800000>;

Rob