[Qemu-devel] [PATCH] map 64-bit PCI BARs at location provided by emulator

[Qemu-devel] [PATCH] map 64-bit PCI BARs at location provided by emulator

[Igor Mammedov]

Currently 64-bit PCI BARs are unconditionally mapped by BIOS right
over 4G + RamSizeOver4G location, which doesn't allow to reserve
extra space before 64-bit PCI window. For memory hotplug an extra
RAM space might be reserved after present 64-bit RAM end and BIOS
should map 64-bit PCI BARs after it.

Introduce "etc/pcimem64-start" romfile to provide BIOS a hint
where it should start mapping of 64-bit PCI BARs. If romfile is
missing BIOS reverts to legacy behavior and starts mapping right
after high memory.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 src/fw/pciinit.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
index b29db99..f056633 100644
--- a/src/fw/pciinit.c
+++ b/src/fw/pciinit.c
@@ -18,6 +18,8 @@
 #include "paravirt.h" // RamSize
 #include "string.h" // memset
 #include "util.h" // pci_setup
+#include "byteorder.h" // le64_to_cpu
+#include "romfile.h" // romfile_loadint
 
 #define PCI_DEVICE_MEM_MIN     0x1000
 #define PCI_BRIDGE_IO_MIN      0x1000
@@ -764,6 +766,8 @@ static void pci_bios_map_devices(struct pci_bus *busses)
 {
     if (pci_bios_init_root_regions(busses)) {
         struct pci_region r64_mem, r64_pref;
+        u64 base64 = le64_to_cpu(romfile_loadint("etc/pcimem64-start",
+                                 0x100000000ULL + RamSizeOver4G));
         r64_mem.list.first = NULL;
         r64_pref.list.first = NULL;
         pci_region_migrate_64bit_entries(&busses[0].r[PCI_REGION_TYPE_MEM],
@@ -779,7 +783,7 @@ static void pci_bios_map_devices(struct pci_bus *busses)
         u64 align_mem = pci_region_align(&r64_mem);
         u64 align_pref = pci_region_align(&r64_pref);
 
-        r64_mem.base = ALIGN(0x100000000LL + RamSizeOver4G, align_mem);
+        r64_mem.base = ALIGN(base64, align_mem);
         r64_pref.base = ALIGN(r64_mem.base + sum_mem, align_pref);
         pcimem64_start = r64_mem.base;
         pcimem64_end = r64_pref.base + sum_pref;
-- 
1.7.1

Re: [Qemu-devel] [PATCH] map 64-bit PCI BARs at location provided by emulator

[Gerd Hoffmann]

On Fr, 2013-10-11 at 12:19 +0200, Igor Mammedov wrote:
> Currently 64-bit PCI BARs are unconditionally mapped by BIOS right
> over 4G + RamSizeOver4G location, which doesn't allow to reserve
> extra space before 64-bit PCI window. For memory hotplug an extra
> RAM space might be reserved after present 64-bit RAM end and BIOS
> should map 64-bit PCI BARs after it.
> 
> Introduce "etc/pcimem64-start" romfile to provide BIOS a hint
> where it should start mapping of 64-bit PCI BARs. If romfile is
> missing BIOS reverts to legacy behavior and starts mapping right
> after high memory.

Looks good overall.


>          struct pci_region r64_mem, r64_pref;
> +        u64 base64 = le64_to_cpu(romfile_loadint("etc/pcimem64-start",
> +                                 0x100000000ULL + RamSizeOver4G));

Should we sanity-check base64 maybe?
Make sure it isn't smaller than 0x100000000ULL + RamSizeOver4G?

cheers,
  Gerd
       

Re: [Qemu-devel] [PATCH] map 64-bit PCI BARs at location provided by emulator

[Igor Mammedov]

On Fri, 11 Oct 2013 14:35:50 +0200
Gerd Hoffmann <kraxel@redhat.com> wrote:

> On Fr, 2013-10-11 at 12:19 +0200, Igor Mammedov wrote:
> > Currently 64-bit PCI BARs are unconditionally mapped by BIOS right
> > over 4G + RamSizeOver4G location, which doesn't allow to reserve
> > extra space before 64-bit PCI window. For memory hotplug an extra
> > RAM space might be reserved after present 64-bit RAM end and BIOS
> > should map 64-bit PCI BARs after it.
> > 
> > Introduce "etc/pcimem64-start" romfile to provide BIOS a hint
> > where it should start mapping of 64-bit PCI BARs. If romfile is
> > missing BIOS reverts to legacy behavior and starts mapping right
> > after high memory.
> 
> Looks good overall.
> 
> 
> >          struct pci_region r64_mem, r64_pref;
> > +        u64 base64 = le64_to_cpu(romfile_loadint("etc/pcimem64-start",
> > +                                 0x100000000ULL + RamSizeOver4G));
> 
> Should we sanity-check base64 maybe?
> Make sure it isn't smaller than 0x100000000ULL + RamSizeOver4G?
If it would be less than 0x100000000ULL + RamSizeOver4G then it would mean
that QEMU for some reason decided to overshadow present RAM with PCI hole.

It would add a couple of extra bytes to BIOS and point where it should fail
or at least print a warning that nobody will read at runtime.
I think it's better to assert on QEMU side and abort even before guest
started if it's considered as error.

> 
> cheers,
>   Gerd
>        
> 
> 

[Qemu-devel] [PATCH] map 64-bit PCI BARs at location provided by emulator

[Igor Mammedov]

Currently 64-bit PCI BARs are unconditionally mapped by BIOS right
over 4G + RamSizeOver4G location, which doesn't allow to reserve
extra space before 64-bit PCI window. For memory hotplug an extra
RAM space might be reserved after present 64-bit RAM end and BIOS
should map 64-bit PCI BARs after it.

Introduce "etc/pcimem64-minimum-addres" romfile to provide BIOS
a hint where it should start mapping of 64-bit PCI BARs.
If romfile is missing BIOS reverts to legacy behavior and starts
mapping right after high memory.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
v3:
  * rename "etc/pcimem64-start" to "etc/pcimem64-minimum-addres"
v2:
  * place 64-bit window behind high RAM end if "etc/pcimem64-start"
    points below it.
---
 src/fw/pciinit.c |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
index b29db99..64a37c3 100644
--- a/src/fw/pciinit.c
+++ b/src/fw/pciinit.c
@@ -18,6 +18,8 @@
 #include "paravirt.h" // RamSize
 #include "string.h" // memset
 #include "util.h" // pci_setup
+#include "byteorder.h" // le64_to_cpu
+#include "romfile.h" // romfile_loadint
 
 #define PCI_DEVICE_MEM_MIN     0x1000
 #define PCI_BRIDGE_IO_MIN      0x1000
@@ -764,6 +766,15 @@ static void pci_bios_map_devices(struct pci_bus *busses)
 {
     if (pci_bios_init_root_regions(busses)) {
         struct pci_region r64_mem, r64_pref;
+        u64 ram64_end = 0x100000000ULL + RamSizeOver4G;
+        u64 base64 = le64_to_cpu(romfile_loadint("etc/pcimem64-minimum-address",
+                                 ram64_end));
+        if (base64 < ram64_end) {
+            dprintf(1, "ignorig etc/pcimem64-minimum-address [0x%llx] below "
+                       "present RAM, placing 64-bit PCI window behind RAM end: "
+                       "0x%llx", base64, ram64_end);
+            base64 = ram64_end;
+        }
         r64_mem.list.first = NULL;
         r64_pref.list.first = NULL;
         pci_region_migrate_64bit_entries(&busses[0].r[PCI_REGION_TYPE_MEM],
@@ -779,7 +790,7 @@ static void pci_bios_map_devices(struct pci_bus *busses)
         u64 align_mem = pci_region_align(&r64_mem);
         u64 align_pref = pci_region_align(&r64_pref);
 
-        r64_mem.base = ALIGN(0x100000000LL + RamSizeOver4G, align_mem);
+        r64_mem.base = ALIGN(base64, align_mem);
         r64_pref.base = ALIGN(r64_mem.base + sum_mem, align_pref);
         pcimem64_start = r64_mem.base;
         pcimem64_end = r64_pref.base + sum_pref;
-- 
1.7.1