* [PATCH] pci-scan: Fix setting the limit
@ 2014-05-13 10:48 Alexey Kardashevskiy
2014-05-13 11:16 ` Alexey Kardashevskiy
2014-05-14 9:52 ` Nikunj A Dadhania
0 siblings, 2 replies; 3+ messages in thread
From: Alexey Kardashevskiy @ 2014-05-13 10:48 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alexey Kardashevskiy, Paul Mackerras, Thomas Huth,
Nikunj A Dadhania
PCI spec says that lower 20 bits are assumed 0xFFFFF. The existing code
seems to get it right in pci-bridge-set-mem-limit.
However pci-bridge-set-mem-base does not account 0xFFFFF and poison
the limit. Since the limit is not stored anywhere in SLOF and only
besides in the config space, it remains broken.
This fixes pci-bridge-set-mem-base.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
I have doubts this is the right fix as I tried to "fix"
pci-bridge-set-mmio-base (while I am here) and it broke the guest.
The problem I am fixing by this is that QEMU started as below is
unable to initialize virtio-net device because there are overlapping
virtio's BAR and bridge's "ranges" property. Note that virtio-net is
attached to the PHB, not that additional bridge.
/home/aik/qemu-system-ppc64 \
-enable-kvm \
-m 1024 \
-machine pseries \
-nographic \
-vga none \
-device pci-bridge,id=id0,bus=pci.0,addr=5.0,chassis_nr=7 \
-netdev tap,id=id1,ifname=tap1,script=ifup.sh,downscript=ifdown.sh \
-device virtio-net-pci,id=id2,netdev=id1 \
-initrd 1.cpio \
-kernel vml315rc3 \
This is from the guest:
PCI host bridge /pci@800000020000000 ranges:
IO 0x0000010080000000..0x000001008000ffff -> 0x0000000000000000
MEM 0x00000100a0000000..0x00000100bfffffff -> 0x0000000080000000
PCI:0000:00:00.0 Resource 0 0000000000010020-000000000001003f [40101]
PCI:0000:00:00.0 Resource 1 00000100b0000000-00000100b0000fff [40200]
PCI:0000:00:00.0 Resource 6 00000100b0040000-00000100b007ffff [4c200]
PCI:0000:00:05.0 Bus rsrc 1 0000000090100000-00000000a00fffff [40200]
PCI:0000:00:05.0 Bus rsrc 2 00000100a0000000-00000100b00fffff [42208]
PCI: PHB (bus 0) bridge rsrc 4: 0000000000010000-000000000001ffff [0x100], parent c000000000f765b8 (PCI IO)
PCI: PHB (bus 0) bridge rsrc 5: 00000100a0000000-00000100bfffffff [0x200], parent c000000000f76580 (PCI mem)
PCI: Allocating 0000:00:00.0: Resource 0: 0000000000010020..000000000001003f [40101]
PCI: Allocating 0000:00:00.0: Resource 1: 00000100b0000000..00000100b0000fff [40200]
PCI: Cannot allocate resource region 1 of device 0000:00:00.0, will remap
This are PHB and bridge "ranges":
[root@erif_root ~]# hexdump -e '7/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/ranges
01000000 00000000 00000000 00000100 80000000 00000000 00010000
02000000 00000000 80000000 00000100 a0000000 00000000 20000000
[root@erif_root ~]# hexdump -e '8/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/pci@5/ranges
02000000 00000000 90100000 02000000 00000000 90100000 00000000 10000000
42000000 00000000 80000000 42000000 00000000 80000000 00000000 10100000
And virtio-net BARs:
[root@erif_root ~]# hexdump -e '5/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/ethernet@0/reg
00000000 00000000 00000000 00000000 00000000
01000010 00000000 00000000 00000000 00000020
02000014 00000000 00000000 00000000 00001000
02000030 00000000 00000000 00000000 00040000
[root@erif_root ~]# hexdump -e '5/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/ethernet@0/assigned-addres
es
82000030 00000000 90040000 00000000 00040000
81000010 00000000 00000020 00000000 00000020
82000014 00000000 90000000 00000000 00001000
---
slof/fs/pci-scan.fs | 1 +
1 file changed, 1 insertion(+)
diff --git a/slof/fs/pci-scan.fs b/slof/fs/pci-scan.fs
index ec9bd27..53b3b2c 100644
--- a/slof/fs/pci-scan.fs
+++ b/slof/fs/pci-scan.fs
@@ -115,6 +115,7 @@ here 100 allot CONSTANT pci-device-vec
THEN \ FI
10 rshift \ keep upper 16 bits
pci-max-mem @ FFFF0000 and or \ and Insert mmem Limit (set it to max)
+ 1-
swap 24 + rtas-config-l! \ and write it into the bridge
;
--
1.9.rc0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] pci-scan: Fix setting the limit
2014-05-13 10:48 [PATCH] pci-scan: Fix setting the limit Alexey Kardashevskiy
@ 2014-05-13 11:16 ` Alexey Kardashevskiy
2014-05-14 9:52 ` Nikunj A Dadhania
1 sibling, 0 replies; 3+ messages in thread
From: Alexey Kardashevskiy @ 2014-05-13 11:16 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Paul Mackerras, Thomas Huth, Nikunj A Dadhania
On 05/13/2014 08:48 PM, Alexey Kardashevskiy wrote:
> PCI spec says that lower 20 bits are assumed 0xFFFFF. The existing code
> seems to get it right in pci-bridge-set-mem-limit.
>
> However pci-bridge-set-mem-base does not account 0xFFFFF and poison
> the limit. Since the limit is not stored anywhere in SLOF and only
> besides in the config space, it remains broken.
>
> This fixes pci-bridge-set-mem-base.
>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
>
> I have doubts this is the right fix as I tried to "fix"
> pci-bridge-set-mmio-base (while I am here) and it broke the guest.
>
> The problem I am fixing by this is that QEMU started as below is
> unable to initialize virtio-net device because there are overlapping
> virtio's BAR and bridge's "ranges" property. Note that virtio-net is
> attached to the PHB, not that additional bridge.
>
> /home/aik/qemu-system-ppc64 \
> -enable-kvm \
> -m 1024 \
> -machine pseries \
> -nographic \
> -vga none \
> -device pci-bridge,id=id0,bus=pci.0,addr=5.0,chassis_nr=7 \
> -netdev tap,id=id1,ifname=tap1,script=ifup.sh,downscript=ifdown.sh \
> -device virtio-net-pci,id=id2,netdev=id1 \
> -initrd 1.cpio \
> -kernel vml315rc3 \
>
> This is from the guest:
>
> PCI host bridge /pci@800000020000000 ranges:
> IO 0x0000010080000000..0x000001008000ffff -> 0x0000000000000000
> MEM 0x00000100a0000000..0x00000100bfffffff -> 0x0000000080000000
>
> PCI:0000:00:00.0 Resource 0 0000000000010020-000000000001003f [40101]
> PCI:0000:00:00.0 Resource 1 00000100b0000000-00000100b0000fff [40200]
> PCI:0000:00:00.0 Resource 6 00000100b0040000-00000100b007ffff [4c200]
>
> PCI:0000:00:05.0 Bus rsrc 1 0000000090100000-00000000a00fffff [40200]
> PCI:0000:00:05.0 Bus rsrc 2 00000100a0000000-00000100b00fffff [42208]
> PCI: PHB (bus 0) bridge rsrc 4: 0000000000010000-000000000001ffff [0x100], parent c000000000f765b8 (PCI IO)
> PCI: PHB (bus 0) bridge rsrc 5: 00000100a0000000-00000100bfffffff [0x200], parent c000000000f76580 (PCI mem)
> PCI: Allocating 0000:00:00.0: Resource 0: 0000000000010020..000000000001003f [40101]
> PCI: Allocating 0000:00:00.0: Resource 1: 00000100b0000000..00000100b0000fff [40200]
> PCI: Cannot allocate resource region 1 of device 0000:00:00.0, will remap
>
> This are PHB and bridge "ranges":
> [root@erif_root ~]# hexdump -e '7/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/ranges
> 01000000 00000000 00000000 00000100 80000000 00000000 00010000
> 02000000 00000000 80000000 00000100 a0000000 00000000 20000000
> [root@erif_root ~]# hexdump -e '8/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/pci@5/ranges
> 02000000 00000000 90100000 02000000 00000000 90100000 00000000 10000000
> 42000000 00000000 80000000 42000000 00000000 80000000 00000000 10100000
>
> And virtio-net BARs:
> [root@erif_root ~]# hexdump -e '5/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/ethernet@0/reg
> 00000000 00000000 00000000 00000000 00000000
> 01000010 00000000 00000000 00000000 00000020
> 02000014 00000000 00000000 00000000 00001000
> 02000030 00000000 00000000 00000000 00040000
> [root@erif_root ~]# hexdump -e '5/4 "%08x "' -e '"\n"' /proc/device-tree/pci@800000020000000/ethernet@0/assigned-addres
> es
> 82000030 00000000 90040000 00000000 00040000
> 81000010 00000000 00000020 00000000 00000020
> 82000014 00000000 90000000 00000000 00001000
Hm. The patch seems to remove non-prefetch range from the PCI bridge
(pci@5/ranges) and made weird start address. So it is wrong. Oh...
[root@erif_root ~]# hexdump -e '7/4 "%08x "' -e '"\n"'
/proc/device-tree/pci@800000020000000/ranges
01000000 00000000 00000000 00000100 80000000 00000000 00010000
02000000 00000000 80000000 00000100 a0000000 00000000 20000000
[root@erif_root ~]# hexdump -e '8/4 "%08x "' -e '"\n"'
/proc/device-tree/pci@800000020000000/pci@5/ranges
42000000 00000000 7ff00000 42000000 00000000 7ff00000 00000000 00100000
[root@erif_root ~]# hexdump -e '5/4 "%08x "' -e '"\n"'
/proc/device-tree/pci@800000020000000/ethernet@0/reg
00000000 00000000 00000000 00000000 00000000
01000010 00000000 00000000 00000000 00000020
02000014 00000000 00000000 00000000 00001000
02000030 00000000 00000000 00000000 00040000
es ot@erif_root ~]# hexdump -e '5/4 "%08x "' -e '"\n"'
/proc/device-tree/pci@800000020000000/ethernet@0/assigned-addresse
82000030 00000000 90040000 00000000 00040000
81000010 00000000 00000020 00000000 00000020
82000014 00000000 90000000 00000000 00001000
> ---
> slof/fs/pci-scan.fs | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/slof/fs/pci-scan.fs b/slof/fs/pci-scan.fs
> index ec9bd27..53b3b2c 100644
> --- a/slof/fs/pci-scan.fs
> +++ b/slof/fs/pci-scan.fs
> @@ -115,6 +115,7 @@ here 100 allot CONSTANT pci-device-vec
> THEN \ FI
> 10 rshift \ keep upper 16 bits
> pci-max-mem @ FFFF0000 and or \ and Insert mmem Limit (set it to max)
> + 1-
> swap 24 + rtas-config-l! \ and write it into the bridge
> ;
>
>
--
Alexey
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] pci-scan: Fix setting the limit
2014-05-13 10:48 [PATCH] pci-scan: Fix setting the limit Alexey Kardashevskiy
2014-05-13 11:16 ` Alexey Kardashevskiy
@ 2014-05-14 9:52 ` Nikunj A Dadhania
1 sibling, 0 replies; 3+ messages in thread
From: Nikunj A Dadhania @ 2014-05-14 9:52 UTC (permalink / raw)
To: Alexey Kardashevskiy, linuxppc-dev
Cc: Alexey Kardashevskiy, Paul Mackerras, Thomas Huth
Alexey Kardashevskiy <aik@ozlabs.ru> writes:
> PCI spec says that lower 20 bits are assumed 0xFFFFF. The existing code
> seems to get it right in pci-bridge-set-mem-limit.
>
> However pci-bridge-set-mem-base does not account 0xFFFFF and poison
> the limit. Since the limit is not stored anywhere in SLOF and only
> besides in the config space, it remains broken.
>
> This fixes pci-bridge-set-mem-base.
>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
>
> I have doubts this is the right fix as I tried to "fix"
> pci-bridge-set-mmio-base (while I am here) and it broke the guest.
>
> The problem I am fixing by this is that QEMU started as below is
> unable to initialize virtio-net device because there are overlapping
> virtio's BAR and bridge's "ranges" property. Note that virtio-net is
> attached to the PHB, not that additional bridge.
>
> /home/aik/qemu-system-ppc64 \
> -enable-kvm \
> -m 1024 \
> -machine pseries \
> -nographic \
> -vga none \
> -device pci-bridge,id=id0,bus=pci.0,addr=5.0,chassis_nr=7 \
> -netdev tap,id=id1,ifname=tap1,script=ifup.sh,downscript=ifdown.sh \
> -device virtio-net-pci,id=id2,netdev=id1 \
> -initrd 1.cpio \
> -kernel vml315rc3 \
The problem that I saw here is the brigde device does not have a
downstream pci device. Before probing, we are setting the ranges
property to the max limit, and the probe is done. Once the probe is over
we would update the ranges property.
In this particular case, when we come to update the " ranges" property,
we do not have any range and we skip updating it. Because of this the
old max range property remains there, which is not correct.
Something like the below solves this particular problem. But what would
happen when someone tries to hotplug a pci device to this bridge, will
the pci-hotplug code take care of updating the ranges property?
diff --git a/slof/fs/pci-properties.fs b/slof/fs/pci-properties.fs
index f88a571..f5e934d 100644
--- a/slof/fs/pci-properties.fs
+++ b/slof/fs/pci-properties.fs
@@ -410,6 +410,7 @@
dup IF \ IF any space present (propsize>0)
s" ranges" property \ | write it into the device tree
ELSE \ ELSE
+ s" " s" ranges" property
2drop \ | forget the properties
THEN \ FI
drop \ forget the address
So I do not see the problem when there is a device allocated downstream
to the pci-bridge.
Regards
Nikunj
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2014-05-14 9:52 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-05-13 10:48 [PATCH] pci-scan: Fix setting the limit Alexey Kardashevskiy
2014-05-13 11:16 ` Alexey Kardashevskiy
2014-05-14 9:52 ` Nikunj A Dadhania
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).