* IO Space Allocation Question on ARM64
@ 2017-05-22 17:50 Sinan Kaya
2017-05-22 20:14 ` Alex Williamson
0 siblings, 1 reply; 3+ messages in thread
From: Sinan Kaya @ 2017-05-22 17:50 UTC (permalink / raw)
To: Linux PCI
We are seeing IO space allocation failure on QDF2400 with multiple switch hierarchy
below. This card has 20 PCI bridges with endpoints attached to each one to increase
the number of endpoints supported.
QDF2400 supports a maximum of 64k IO space aka 16 bit IO on the root port.
-+-[0003:00]---00.0-[01-16]----00.0-[02-16]--+-00.0-[03]----00.0
| +-01.0-[04]----00.0
| +-02.0-[05]----00.0
| +-03.0-[06]----00.0
| +-04.0-[07]----00.0
| +-05.0-[08]----00.0
| +-06.0-[09]----00.0
| +-07.0-[0a]----00.0
| +-08.0-[0b]--
| +-09.0-[0c]--
| +-0a.0-[0d]--
| +-0b.0-[0e]--
| +-0c.0-[0f]--
| +-0d.0-[10]--
| +-0e.0-[11]--
| +-0f.0-[12]--
| +-10.0-[13]--
| +-11.0-[14]--
| +-12.0-[15]--
| \-13.0-[16]--
What we so far is that Linux is assigning an IO resource for every single bridge
that it finds even though there is nothing connected behind on some of the bridges.
Do we know the reason for this behavior?
[ 4.288269] pci 0003:00:00.0: BAR 13: failed to assign [io size 0x15000]
[ 4.317175] pci 0003:01:00.0: BAR 13: failed to assign [io size 0x14000]
[ 4.644518] pci 0003:02:00.0: BAR 13: failed to assign [io size 0x1000]
[ 4.657539] pci 0003:02:01.0: BAR 13: failed to assign [io size 0x1000]
[ 4.670560] pci 0003:02:02.0: BAR 13: failed to assign [io size 0x1000]
[ 4.683581] pci 0003:02:03.0: BAR 13: failed to assign [io size 0x1000]
[ 4.696602] pci 0003:02:04.0: BAR 13: failed to assign [io size 0x1000]
[ 4.709623] pci 0003:02:05.0: BAR 13: failed to assign [io size 0x1000]
[ 4.722644] pci 0003:02:06.0: BAR 13: failed to assign [io size 0x1000]
[ 4.735664] pci 0003:02:07.0: BAR 13: failed to assign [io size 0x1000]
[ 4.748685] pci 0003:02:08.0: BAR 13: failed to assign [io size 0x1000]
[ 4.761706] pci 0003:02:09.0: BAR 13: failed to assign [io size 0x1000]
[ 4.774727] pci 0003:02:0a.0: BAR 13: failed to assign [io size 0x1000]
[ 4.787748] pci 0003:02:0b.0: BAR 13: failed to assign [io size 0x1000]
[ 4.800768] pci 0003:02:0c.0: BAR 13: failed to assign [io size 0x1000]
[ 4.813789] pci 0003:02:0d.0: BAR 13: failed to assign [io size 0x1000]
[ 4.826810] pci 0003:02:0e.0: BAR 13: failed to assign [io size 0x1000]
[ 4.839831] pci 0003:02:0f.0: BAR 13: failed to assign [io size 0x1000]
[ 4.852852] pci 0003:02:10.0: BAR 13: failed to assign [io size 0x1000]
[ 4.865873] pci 0003:02:11.0: BAR 13: failed to assign [io size 0x1000]
[ 4.878893] pci 0003:02:12.0: BAR 13: failed to assign [io size 0x1000]
[ 4.891914] pci 0003:02:13.0: BAR 13: failed to assign [io size 0x1000]
UEFI seems to be assigning an IO resource to bridge only if finds an endpoint
behind it with an actual IO BAR space.
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: IO Space Allocation Question on ARM64
2017-05-22 17:50 IO Space Allocation Question on ARM64 Sinan Kaya
@ 2017-05-22 20:14 ` Alex Williamson
2017-05-22 21:20 ` Sinan Kaya
0 siblings, 1 reply; 3+ messages in thread
From: Alex Williamson @ 2017-05-22 20:14 UTC (permalink / raw)
To: Sinan Kaya; +Cc: Linux PCI
On Mon, 22 May 2017 13:50:31 -0400
Sinan Kaya <okaya@codeaurora.org> wrote:
> We are seeing IO space allocation failure on QDF2400 with multiple switch hierarchy
> below. This card has 20 PCI bridges with endpoints attached to each one to increase
> the number of endpoints supported.
>
> QDF2400 supports a maximum of 64k IO space aka 16 bit IO on the root port.
>
> -+-[0003:00]---00.0-[01-16]----00.0-[02-16]--+-00.0-[03]----00.0
> | +-01.0-[04]----00.0
> | +-02.0-[05]----00.0
> | +-03.0-[06]----00.0
> | +-04.0-[07]----00.0
> | +-05.0-[08]----00.0
> | +-06.0-[09]----00.0
> | +-07.0-[0a]----00.0
> | +-08.0-[0b]--
> | +-09.0-[0c]--
> | +-0a.0-[0d]--
> | +-0b.0-[0e]--
> | +-0c.0-[0f]--
> | +-0d.0-[10]--
> | +-0e.0-[11]--
> | +-0f.0-[12]--
> | +-10.0-[13]--
> | +-11.0-[14]--
> | +-12.0-[15]--
> | \-13.0-[16]--
>
> What we so far is that Linux is assigning an IO resource for every single bridge
> that it finds even though there is nothing connected behind on some of the bridges.
> Do we know the reason for this behavior?
>
> [ 4.288269] pci 0003:00:00.0: BAR 13: failed to assign [io size 0x15000]
> [ 4.317175] pci 0003:01:00.0: BAR 13: failed to assign [io size 0x14000]
> [ 4.644518] pci 0003:02:00.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.657539] pci 0003:02:01.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.670560] pci 0003:02:02.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.683581] pci 0003:02:03.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.696602] pci 0003:02:04.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.709623] pci 0003:02:05.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.722644] pci 0003:02:06.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.735664] pci 0003:02:07.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.748685] pci 0003:02:08.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.761706] pci 0003:02:09.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.774727] pci 0003:02:0a.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.787748] pci 0003:02:0b.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.800768] pci 0003:02:0c.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.813789] pci 0003:02:0d.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.826810] pci 0003:02:0e.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.839831] pci 0003:02:0f.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.852852] pci 0003:02:10.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.865873] pci 0003:02:11.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.878893] pci 0003:02:12.0: BAR 13: failed to assign [io size 0x1000]
> [ 4.891914] pci 0003:02:13.0: BAR 13: failed to assign [io size 0x1000]
>
> UEFI seems to be assigning an IO resource to bridge only if finds an endpoint
> behind it with an actual IO BAR space.
If it's anything like we saw with similar issues on QEMU, it might be
because the slots report themselves as hotplug capable and Linux tries
to allocate minimum apertures for any hotplug slot. See hpiosize
and hpmemsize options for pci=. Thanks,
Alex
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: IO Space Allocation Question on ARM64
2017-05-22 20:14 ` Alex Williamson
@ 2017-05-22 21:20 ` Sinan Kaya
0 siblings, 0 replies; 3+ messages in thread
From: Sinan Kaya @ 2017-05-22 21:20 UTC (permalink / raw)
To: Alex Williamson; +Cc: Linux PCI
Hi Alex,
On 5/22/2017 4:14 PM, Alex Williamson wrote:
>> UEFI seems to be assigning an IO resource to bridge only if finds an endpoint
>> behind it with an actual IO BAR space.
> If it's anything like we saw with similar issues on QEMU, it might be
> because the slots report themselves as hotplug capable and Linux tries
> to allocate minimum apertures for any hotplug slot. See hpiosize
> and hpmemsize options for pci=. Thanks,
That was it. I was suspicious of this but didn't realize that we were allocating
0x1000 as a minimum due to hotplug.
Thanks for the suggestion. I didn't know the hpmemsize and hpiosize arguments.
Sinan
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-05-22 21:20 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-05-22 17:50 IO Space Allocation Question on ARM64 Sinan Kaya
2017-05-22 20:14 ` Alex Williamson
2017-05-22 21:20 ` Sinan Kaya
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).