Re: pci_bus_distribute_available_resources() is wrong?

[Alexander Motin <mav@ixsystems.com>]

On 13.12.2022 00:49, Mika Westerberg wrote:
> On Mon, Dec 12, 2022 at 04:10:16PM -0500, Alexander Motin wrote:
>> On 12.12.2022 15:32, Bjorn Helgaas wrote:
>>> On Mon, Dec 12, 2022 at 1:36 PM Alexander Motin <mav@ixsystems.com> wrote:
>>>> In my AMD NTB case PCI topology looks this way:
>>>>
>>>> +-[0000:80]-+-00.0
>>>> |           +-01.1-[81-83]----00.0-[82-83]----00.0-[83]--+-00.0 Dummy
>>>> |           |                                            \-00.1 NTB
>>>>
>>>> 80:01.1 is the root bridge where the hot-plug happens.  The 81:00.0
>>>> bridge in addition to memory windows has small 16KB BAR.  But since it
>>>> is the only bridge on the bus, the function passes all available
>>>> resources down to its children.  As result, that BAR fails to allocate.
>>>>     And while that BAR seems not really needed, in some cases the
>>>> allocation error makes whole memory window to be disabled, that ends up
>>>> in NTB device driver attach failure.
> 
> Just out of the curiosity, is this PCIe or PCI topology?

All PCIe: hot-plug root <-> upstream <-> downstream <-> two endpoints.

>>> Mika is working on what sounds like the same problem.  His current
>>> patch series is at
>>> https://lore.kernel.org/linux-pci/20221130112221.66612-1-mika.westerberg@linux.intel.com/
>>>
>>> We would appreciate your comments and testing as that series is developed.
>>
>> Thank you, Bjorn.  This definitely looks related, but as you've already
>> noted in your review there, present patch does not handle BARs of the bridge
>> itself, that I have in my case.  I'd be happy to test the updated patch.
>> Please keep me in a loop.
>>
>> I also agree with your comment that the same should be done in case of
>> multiple bridges.  I am generally not sure the cases of single bridge or not
>> having hot-plug on this level should be any specific.
> 
> Yeah, I'm working on a new version of the patch series that should take
> these into consideration. The challenge is that the code has been used
> with the Thunderbolt/USB4 PCIe tunneling for some time already and we
> don't want to break that either.
> 
> I'm also more than happy to test any patches regarding this if someone
> else wants to work on it ;-)

I was kind of ready to dive in, I hate hacks and tunables to workaround 
bugs.  But as I have told, I see this code first time, so my solutions 
may appear not right.  But I'll help as I can, if needed.

-- 
Alexander Motin