[Qemu-devel] [PATCH V2 0/3] Enable QEMU to handle more than 2GB with Xen.

[Qemu-devel] [PATCH V2 0/3] Enable QEMU to handle more than 2GB with Xen.

[Anthony PERARD]

Hi all,

Update on this series:
  - Use a RAM address of 64bits only on 64bits targets when Xen is enable.
  - Add some comment on the memory registration done for Xen.


Xen is not limited by the QEMU's virtual address space for the allocation of
the guest RAM. So even with a QEMU 32bits, a Xen guest can have more than 4 GB
of RAM.

With this serie, we will be able to run a guest with more than 4GB. The main
point is to change ram_addr_t from ulong to uin64 when QEMU is configure with
Xen. The second point is better register the memory in QEMU.

Regards,

Anthony PERARD (3):
  cpu-common: Have a ram_addr_t of uint64 with Xen.
  xen: Fix the memory registration to reflect of what is done by Xen.
  vl.c: Check the asked ram_size later.

 cpu-common.h |    8 ++++++++
 exec.c       |    9 +++++----
 vl.c         |   14 ++++++++------
 xen-all.c    |   29 +++++++++++++++++++++--------
 4 files changed, 42 insertions(+), 18 deletions(-)

-- 
Anthony PERARD

[Qemu-devel] [PATCH V2 1/3] cpu-common: Have a ram_addr_t of uint64 with Xen.

[Anthony PERARD]

In Xen case, memory can be bigger than the host memory. that mean a
32bits host (and QEMU) should be able to handle a RAM address of 64bits.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 cpu-common.h |    8 ++++++++
 exec.c       |    9 +++++----
 xen-all.c    |    2 +-
 3 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/cpu-common.h b/cpu-common.h
index a5b80e1..06818d2 100644
--- a/cpu-common.h
+++ b/cpu-common.h
@@ -27,7 +27,15 @@ enum device_endian {
 };
 
 /* address in the RAM (different from a physical address) */
+#if defined(CONFIG_XEN_BACKEND) && TARGET_PHYS_ADDR_BITS == 64
+typedef uint64_t ram_addr_t;
+#  define RAM_ADDR_MAX UINT64_MAX
+#  define RAM_ADDR_FMT "%" PRIx64
+#else
 typedef unsigned long ram_addr_t;
+#  define RAM_ADDR_MAX ULONG_MAX
+#  define RAM_ADDR_FMT "%lx"
+#endif
 
 /* memory API */
 
diff --git a/exec.c b/exec.c
index 8ef290e..11c7a3f 100644
--- a/exec.c
+++ b/exec.c
@@ -2863,13 +2863,13 @@ static void *file_ram_alloc(RAMBlock *block,
 static ram_addr_t find_ram_offset(ram_addr_t size)
 {
     RAMBlock *block, *next_block;
-    ram_addr_t offset = 0, mingap = ULONG_MAX;
+    ram_addr_t offset = 0, mingap = RAM_ADDR_MAX;
 
     if (QLIST_EMPTY(&ram_list.blocks))
         return 0;
 
     QLIST_FOREACH(block, &ram_list.blocks, next) {
-        ram_addr_t end, next = ULONG_MAX;
+        ram_addr_t end, next = RAM_ADDR_MAX;
 
         end = block->offset + block->length;
 
@@ -3081,7 +3081,8 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length)
 #endif
                 }
                 if (area != vaddr) {
-                    fprintf(stderr, "Could not remap addr: %lx@%lx\n",
+                    fprintf(stderr, "Could not remap addr: "
+                            RAM_ADDR_FMT "@" RAM_ADDR_FMT "\n",
                             length, addr);
                     exit(1);
                 }
@@ -4052,7 +4053,7 @@ void *cpu_physical_memory_map(target_phys_addr_t addr,
     target_phys_addr_t page;
     unsigned long pd;
     PhysPageDesc *p;
-    ram_addr_t raddr = ULONG_MAX;
+    ram_addr_t raddr = RAM_ADDR_MAX;
     ram_addr_t rlen;
     void *ret;
 
diff --git a/xen-all.c b/xen-all.c
index 5fa92ae..b19da37 100644
--- a/xen-all.c
+++ b/xen-all.c
@@ -184,7 +184,7 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size)
     }
 
     if (xc_domain_populate_physmap_exact(xen_xc, xen_domid, nr_pfn, 0, 0, pfn_list)) {
-        hw_error("xen: failed to populate ram at %lx", ram_addr);
+        hw_error("xen: failed to populate ram at " RAM_ADDR_FMT, ram_addr);
     }
 
     qemu_free(pfn_list);
-- 
Anthony PERARD

[Qemu-devel] [PATCH V2 2/3] xen: Fix the memory registration to reflect of what is done by Xen.

[Anthony PERARD]

A Xen guest memory is allocated by libxc. But this memory is not
allocated continuously, instead, it leaves the VGA IO memory space not
allocated, same for the MMIO space (at HVM_BELOW_4G_MMIO_START of size
HVM_BELOW_4G_MMIO_LENGTH).

So to reflect that, we do not register the physical memory for this two
holes. But we still keep only one RAMBlock for the all RAM as it is more
easier than have two separate blocks (1 above 4G). Also this prevent QEMU
from use the MMIO space for a ROM.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen-all.c |   27 ++++++++++++++++++++-------
 1 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/xen-all.c b/xen-all.c
index b19da37..d535f90 100644
--- a/xen-all.c
+++ b/xen-all.c
@@ -19,6 +19,7 @@
 
 #include <xen/hvm/ioreq.h>
 #include <xen/hvm/params.h>
+#include <xen/hvm/e820.h>
 
 //#define DEBUG_XEN
 
@@ -144,6 +145,12 @@ static void xen_ram_init(ram_addr_t ram_size)
     new_block->host = NULL;
     new_block->offset = 0;
     new_block->length = ram_size;
+    if (ram_size >= HVM_BELOW_4G_RAM_END) {
+        /* Xen does not allocate the memory continuously, and keep a hole at
+         * HVM_BELOW_4G_MMIO_START of HVM_BELOW_4G_MMIO_LENGTH
+         */
+        new_block->length += HVM_BELOW_4G_MMIO_LENGTH;
+    }
 
     QLIST_INSERT_HEAD(&ram_list.blocks, new_block, next);
 
@@ -152,20 +159,26 @@ static void xen_ram_init(ram_addr_t ram_size)
     memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
            0xff, new_block->length >> TARGET_PAGE_BITS);
 
-    if (ram_size >= 0xe0000000 ) {
-        above_4g_mem_size = ram_size - 0xe0000000;
-        below_4g_mem_size = 0xe0000000;
+    if (ram_size >= HVM_BELOW_4G_RAM_END) {
+        above_4g_mem_size = ram_size - HVM_BELOW_4G_RAM_END;
+        below_4g_mem_size = HVM_BELOW_4G_RAM_END;
     } else {
         below_4g_mem_size = ram_size;
     }
 
-    cpu_register_physical_memory(0, below_4g_mem_size, new_block->offset);
-#if TARGET_PHYS_ADDR_BITS > 32
+    cpu_register_physical_memory(0, 0xa0000, 0);
+    /* Skip of the VGA IO memory space, it will be registered later by the VGA
+     * emulated device.
+     *
+     * The area between 0xc0000 and 0x100000 will be used by SeaBIOS to load
+     * the Options ROM, so it is registered here as RAM.
+     */
+    cpu_register_physical_memory(0xc0000, below_4g_mem_size - 0xc0000,
+                                 0xc0000);
     if (above_4g_mem_size > 0) {
         cpu_register_physical_memory(0x100000000ULL, above_4g_mem_size,
-                                     new_block->offset + below_4g_mem_size);
+                                     0x100000000ULL);
     }
-#endif
 }
 
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size)
-- 
Anthony PERARD

[Qemu-devel] [PATCH V2 3/3] vl.c: Check the asked ram_size later.

[Anthony PERARD]

As a Xen guest can have more than 2GB of RAM on a 32bit host, we move
the conditions after than we now if we run one Xen or not.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 vl.c |   14 ++++++++------
 1 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/vl.c b/vl.c
index fcd7395..c2efedf 100644
--- a/vl.c
+++ b/vl.c
@@ -2433,11 +2433,6 @@ int main(int argc, char **argv, char **envp)
                     exit(1);
                 }
 
-                /* On 32-bit hosts, QEMU is limited by virtual address space */
-                if (value > (2047 << 20) && HOST_LONG_BITS == 32) {
-                    fprintf(stderr, "qemu: at most 2047 MB RAM can be simulated\n");
-                    exit(1);
-                }
                 if (value != (uint64_t)(ram_addr_t)value) {
                     fprintf(stderr, "qemu: ram size too large\n");
                     exit(1);
@@ -3091,8 +3086,15 @@ int main(int argc, char **argv, char **envp)
         exit(1);
 
     /* init the memory */
-    if (ram_size == 0)
+    if (ram_size == 0) {
         ram_size = DEFAULT_RAM_SIZE * 1024 * 1024;
+    } else if (!xen_enabled()) {
+        /* On 32-bit hosts, QEMU is limited by virtual address space */
+        if (ram_size > (2047 << 20) && HOST_LONG_BITS == 32) {
+            fprintf(stderr, "qemu: at most 2047 MB RAM can be simulated\n");
+            exit(1);
+        }
+    }
 
     /* init the dynamic translator */
     cpu_exec_init_all(tb_size * 1024 * 1024);
-- 
Anthony PERARD

Re: [Qemu-devel] [PATCH V2 3/3] vl.c: Check the asked ram_size later.

[Alexander Graf]

On 07/20/2011 08:17 PM, Anthony PERARD wrote:
> As a Xen guest can have more than 2GB of RAM on a 32bit host, we move
> the conditions after than we now if we run one Xen or not.
>
> Signed-off-by: Anthony PERARD<anthony.perard@citrix.com>
> ---
>   vl.c |   14 ++++++++------
>   1 files changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/vl.c b/vl.c
> index fcd7395..c2efedf 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -2433,11 +2433,6 @@ int main(int argc, char **argv, char **envp)
>                       exit(1);
>                   }
>
> -                /* On 32-bit hosts, QEMU is limited by virtual address space */
> -                if (value>  (2047<<  20)&&  HOST_LONG_BITS == 32) {
> -                    fprintf(stderr, "qemu: at most 2047 MB RAM can be simulated\n");
> -                    exit(1);
> -                }
>                   if (value != (uint64_t)(ram_addr_t)value) {
>                       fprintf(stderr, "qemu: ram size too large\n");
>                       exit(1);
> @@ -3091,8 +3086,15 @@ int main(int argc, char **argv, char **envp)
>           exit(1);
>
>       /* init the memory */
> -    if (ram_size == 0)
> +    if (ram_size == 0) {
>           ram_size = DEFAULT_RAM_SIZE * 1024 * 1024;
> +    } else if (!xen_enabled()) {

I don't see why those are mutually exclusive.

> +        /* On 32-bit hosts, QEMU is limited by virtual address space */
> +        if (ram_size>  (2047<<  20)&&  HOST_LONG_BITS == 32) {
> +            fprintf(stderr, "qemu: at most 2047 MB RAM can be simulated\n");
> +            exit(1);
> +        }
> +    }
>
>       /* init the dynamic translator */
>       cpu_exec_init_all(tb_size * 1024 * 1024);

Alex

Re: [Qemu-devel] [PATCH V2 0/3] Enable QEMU to handle more than 2GB with Xen.

[Alexander Graf]

On 07/20/2011 08:17 PM, Anthony PERARD wrote:
> Hi all,
>
> Update on this series:
>    - Use a RAM address of 64bits only on 64bits targets when Xen is enable.
>    - Add some comment on the memory registration done for Xen.
>
>
> Xen is not limited by the QEMU's virtual address space for the allocation of
> the guest RAM. So even with a QEMU 32bits, a Xen guest can have more than 4 GB
> of RAM.
>
> With this serie, we will be able to run a guest with more than 4GB. The main
> point is to change ram_addr_t from ulong to uin64 when QEMU is configure with
> Xen. The second point is better register the memory in QEMU.
>
> Regards,
>
> Anthony PERARD (3):
>    cpu-common: Have a ram_addr_t of uint64 with Xen.
>    xen: Fix the memory registration to reflect of what is done by Xen.
>    vl.c: Check the asked ram_size later.
>
>   cpu-common.h |    8 ++++++++
>   exec.c       |    9 +++++----
>   vl.c         |   14 ++++++++------
>   xen-all.c    |   29 +++++++++++++++++++++--------
>   4 files changed, 42 insertions(+), 18 deletions(-)

Thanks, applied all to xen-next. I also squashed the following patch 
into 3/3:

diff --git a/vl.c b/vl.c
index 24df37f..d8c7c01 100644
--- a/vl.c
+++ b/vl.c
@@ -3096,7 +3096,9 @@ int main(int argc, char **argv, char **envp)
      /* init the memory */
      if (ram_size == 0) {
          ram_size = DEFAULT_RAM_SIZE * 1024 * 1024;
-    } else if (!xen_enabled()) {
+    }
+
+    if (!xen_enabled()) {
          /* On 32-bit hosts, QEMU is limited by virtual address space */
          if (ram_size > (2047 << 20) && HOST_LONG_BITS == 32) {
              fprintf(stderr, "qemu: at most 2047 MB RAM can be 
simulated\n");


Alex