[Qemu-devel] [PATCH v3 0/8] acpi: make ROMs resizeable

[Qemu-devel] [PATCH v3 0/8] acpi: make ROMs resizeable

[Michael S. Tsirkin]

This is v3 of the patchset.
Changes since v2:
    - Address Paolo's comments: drop an unused function, fix up
      comment.

Changes since v1:
    - Any RAM can now be resizeable - there's no requirement
      that it's device RAM any longer.
    - For simplicity, max_size RAM is always pre-allocated
    - Added memory_region_set_size, to keep MR size consistent
      in case MR is guest visible (even though for current users,
      it never is)

At the moment we migrate ROMs which reside in fw cfg, which allows
changing ROM code at will, and supports migrating largish blocks early,
with good performance.
However, we are running into a problem: changing size breaks
migration every time.
This already requires somewhat messy compatibility support in
acpi generation code, and it looks like there'll be more to come.

While recent patches by Paolo and others might make it easier
to keep table size static for specific machine types, having
a safety net in case we do need to change it sounds like
a good idea.

Rather than try to guess the correct size once and for all,
this patchset tries to make code future-proof, by
adding support for resizeable ram blocks.

A (possibly very high) amount of space in ram_addr_t space is reserved
and allocated in host for each block, but never used by fw cfg.
If incoming block size differs from current size, block is
reallocated. FW CFG is also notified and updated accordingly.

As reviewers felt that making all RAM "resizeable" in this
way might make debugging migration harder, these patches
set a per-block flag and only allow resizing for blocks
where this was explicitly requested.

Note: migration stream is unaffected by these patches.
This makes it possible to enable this functionality
unconditionally, for all machine types.

In the future, this API might be handy for other things, besides ROMs.

Michael S. Tsirkin (8):
  memory: add memory_region_set_size
  exec: cpu_physical_memory_set/clear_dirty_range
  exec: split length -> used_length/max_length
  exec: qemu_ram_alloc_resizeable, qemu_ram_resize
  arch_init: support resizing on incoming migration
  memory: API to allocate resizeable RAM MR
  acpi-build: make ROMs RAM blocks resizeable
  qemu_ram_resize: document assumptions

 hw/lm32/lm32_hwsetup.h  |   3 +-
 include/exec/cpu-all.h  |  12 +++--
 include/exec/memory.h   |  34 ++++++++++++
 include/exec/ram_addr.h |  22 ++++++--
 include/hw/loader.h     |   4 +-
 arch_init.c             |  28 +++++-----
 exec.c                  | 136 +++++++++++++++++++++++++++++++++++++-----------
 hw/core/loader.c        |  18 +++++--
 hw/i386/acpi-build.c    |  19 +++++--
 memory.c                |  33 ++++++++++++
 10 files changed, 249 insertions(+), 60 deletions(-)

-- 
MST

[Qemu-devel] [PATCH v3 1/8] memory: add memory_region_set_size

[Michael S. Tsirkin]

Add API to change MR size.
Will be used internally for RAM resize.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/exec/memory.h | 10 ++++++++++
 memory.c              | 16 ++++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index f64ab5e..0882221 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -878,6 +878,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_set_size: dynamically update the size of a region.
+ *
+ * Dynamically updates the size of a region.
+ *
+ * @mr: the region to be updated
+ * @size: used size of the region.
+ */
+void memory_region_set_size(MemoryRegion *mr, uint64_t size);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
diff --git a/memory.c b/memory.c
index 15cf9eb..618470b 100644
--- a/memory.c
+++ b/memory.c
@@ -1707,6 +1707,22 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled)
     memory_region_transaction_commit();
 }
 
+void memory_region_set_size(MemoryRegion *mr, uint64_t size)
+{
+    Int128 s = int128_make64(size);
+
+    if (size == UINT64_MAX) {
+        s = int128_2_64();
+    }
+    if (int128_eq(s, mr->size)) {
+        return;
+    }
+    memory_region_transaction_begin();
+    mr->size = s;
+    memory_region_update_pending = true;
+    memory_region_transaction_commit();
+}
+
 static void memory_region_readd_subregion(MemoryRegion *mr)
 {
     MemoryRegion *container = mr->container;
-- 
MST

[Qemu-devel] [PATCH v3 2/8] exec: cpu_physical_memory_set/clear_dirty_range

[Michael S. Tsirkin]

Make cpu_physical_memory_set/clear_dirty_range
behave symmetrically.

To clear range for a given client type only, add
cpu_physical_memory_clear_dirty_range_type.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/exec/ram_addr.h | 15 ++++++++++++---
 exec.c                  |  2 +-
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index 8fc75cd..254931c 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -172,9 +172,9 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
 }
 #endif /* not _WIN32 */
 
-static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
-                                                         ram_addr_t length,
-                                                         unsigned client)
+static inline void cpu_physical_memory_clear_dirty_range_type(ram_addr_t start,
+                                                              ram_addr_t length,
+                                                              unsigned client)
 {
     unsigned long end, page;
 
@@ -184,6 +184,15 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
     bitmap_clear(ram_list.dirty_memory[client], page, end - page);
 }
 
+static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
+                                                         ram_addr_t length)
+{
+    cpu_physical_memory_clear_dirty_range_type(start, length, DIRTY_MEMORY_MIGRATION);
+    cpu_physical_memory_clear_dirty_range_type(start, length, DIRTY_MEMORY_VGA);
+    cpu_physical_memory_clear_dirty_range_type(start, length, DIRTY_MEMORY_CODE);
+}
+
+
 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
                                      unsigned client);
 
diff --git a/exec.c b/exec.c
index 963481a..a89aa6c 100644
--- a/exec.c
+++ b/exec.c
@@ -850,7 +850,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
 {
     if (length == 0)
         return;
-    cpu_physical_memory_clear_dirty_range(start, length, client);
+    cpu_physical_memory_clear_dirty_range_type(start, length, client);
 
     if (tcg_enabled()) {
         tlb_reset_dirty_range_all(start, length);
-- 
MST

[Qemu-devel] [PATCH v3 3/8] exec: split length -> used_length/max_length

[Michael S. Tsirkin]

This patch allows us to distinguish between two
length values for each block:
    max_length - length of memory block that was allocated
    used_length - length of block used by QEMU/guest

Currently, we set used_length - max_length, unconditionally.
Follow-up patches allow used_length <= max_length.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/exec/cpu-all.h |  3 ++-
 arch_init.c            | 19 +++++++++---------
 exec.c                 | 52 +++++++++++++++++++++++++++-----------------------
 3 files changed, 40 insertions(+), 34 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 62f5581..6f2130e 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -303,7 +303,8 @@ typedef struct RAMBlock {
     struct MemoryRegion *mr;
     uint8_t *host;
     ram_addr_t offset;
-    ram_addr_t length;
+    ram_addr_t used_length;
+    ram_addr_t max_length;
     uint32_t flags;
     char idstr[256];
     /* Reads can take either the iothread or the ramlist lock.
diff --git a/arch_init.c b/arch_init.c
index 7680d28..106f46e 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -522,7 +522,7 @@ static void migration_bitmap_sync(void)
     address_space_sync_dirty_bitmap(&address_space_memory);
 
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-        migration_bitmap_sync_range(block->mr->ram_addr, block->length);
+        migration_bitmap_sync_range(block->mr->ram_addr, block->used_length);
     }
     trace_migration_bitmap_sync_end(migration_dirty_pages
                                     - num_dirty_pages_init);
@@ -668,7 +668,7 @@ static int ram_find_and_save_block(QEMUFile *f, bool last_stage)
             offset >= last_offset) {
             break;
         }
-        if (offset >= block->length) {
+        if (offset >= block->used_length) {
             offset = 0;
             block = QTAILQ_NEXT(block, next);
             if (!block) {
@@ -727,7 +727,7 @@ uint64_t ram_bytes_total(void)
     uint64_t total = 0;
 
     QTAILQ_FOREACH(block, &ram_list.blocks, next)
-        total += block->length;
+        total += block->used_length;
 
     return total;
 }
@@ -831,7 +831,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
         uint64_t block_pages;
 
-        block_pages = block->length >> TARGET_PAGE_BITS;
+        block_pages = block->used_length >> TARGET_PAGE_BITS;
         migration_dirty_pages += block_pages;
     }
 
@@ -844,7 +844,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
         qemu_put_byte(f, strlen(block->idstr));
         qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idstr));
-        qemu_put_be64(f, block->length);
+        qemu_put_be64(f, block->used_length);
     }
 
     qemu_mutex_unlock_ramlist();
@@ -1015,7 +1015,7 @@ static inline void *host_from_stream_offset(QEMUFile *f,
     uint8_t len;
 
     if (flags & RAM_SAVE_FLAG_CONTINUE) {
-        if (!block || block->length <= offset) {
+        if (!block || block->max_length <= offset) {
             error_report("Ack, bad migration stream!");
             return NULL;
         }
@@ -1028,7 +1028,8 @@ static inline void *host_from_stream_offset(QEMUFile *f,
     id[len] = 0;
 
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-        if (!strncmp(id, block->idstr, sizeof(id)) && block->length > offset) {
+        if (!strncmp(id, block->idstr, sizeof(id)) &&
+            block->max_length > offset) {
             return memory_region_get_ram_ptr(block->mr) + offset;
         }
     }
@@ -1085,10 +1086,10 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
 
                 QTAILQ_FOREACH(block, &ram_list.blocks, next) {
                     if (!strncmp(id, block->idstr, sizeof(id))) {
-                        if (block->length != length) {
+                        if (block->used_length != length) {
                             error_report("Length mismatch: %s: 0x" RAM_ADDR_FMT
                                          " in != 0x" RAM_ADDR_FMT, id, length,
-                                         block->length);
+                                         block->used_length);
                             ret =  -EINVAL;
                         }
                         break;
diff --git a/exec.c b/exec.c
index a89aa6c..b69216a 100644
--- a/exec.c
+++ b/exec.c
@@ -812,11 +812,11 @@ static RAMBlock *qemu_get_ram_block(ram_addr_t addr)
 
     /* The list is protected by the iothread lock here.  */
     block = ram_list.mru_block;
-    if (block && addr - block->offset < block->length) {
+    if (block && addr - block->offset < block->max_length) {
         goto found;
     }
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-        if (addr - block->offset < block->length) {
+        if (addr - block->offset < block->max_length) {
             goto found;
         }
     }
@@ -1305,13 +1305,14 @@ static ram_addr_t ram_block_add(RAMBlock *new_block, Error **errp)
 
     /* This assumes the iothread lock is taken here too.  */
     qemu_mutex_lock_ramlist();
-    new_block->offset = find_ram_offset(new_block->length);
+    new_block->offset = find_ram_offset(new_block->max_length);
 
     if (!new_block->host) {
         if (xen_enabled()) {
-            xen_ram_alloc(new_block->offset, new_block->length, new_block->mr);
+            xen_ram_alloc(new_block->offset, new_block->max_length,
+                          new_block->mr);
         } else {
-            new_block->host = phys_mem_alloc(new_block->length,
+            new_block->host = phys_mem_alloc(new_block->max_length,
                                              &new_block->mr->align);
             if (!new_block->host) {
                 error_setg_errno(errp, errno,
@@ -1320,13 +1321,13 @@ static ram_addr_t ram_block_add(RAMBlock *new_block, Error **errp)
                 qemu_mutex_unlock_ramlist();
                 return -1;
             }
-            memory_try_enable_merging(new_block->host, new_block->length);
+            memory_try_enable_merging(new_block->host, new_block->max_length);
         }
     }
 
     /* Keep the list sorted from biggest to smallest block.  */
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-        if (block->length < new_block->length) {
+        if (block->max_length < new_block->max_length) {
             break;
         }
     }
@@ -1350,14 +1351,15 @@ static ram_addr_t ram_block_add(RAMBlock *new_block, Error **errp)
                                    old_ram_size, new_ram_size);
        }
     }
-    cpu_physical_memory_set_dirty_range(new_block->offset, new_block->length);
+    cpu_physical_memory_set_dirty_range(new_block->offset,
+                                        new_block->used_length);
 
-    qemu_ram_setup_dump(new_block->host, new_block->length);
-    qemu_madvise(new_block->host, new_block->length, QEMU_MADV_HUGEPAGE);
-    qemu_madvise(new_block->host, new_block->length, QEMU_MADV_DONTFORK);
+    qemu_ram_setup_dump(new_block->host, new_block->max_length);
+    qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_HUGEPAGE);
+    qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_DONTFORK);
 
     if (kvm_enabled()) {
-        kvm_setup_guest_memory(new_block->host, new_block->length);
+        kvm_setup_guest_memory(new_block->host, new_block->max_length);
     }
 
     return new_block->offset;
@@ -1391,7 +1393,8 @@ ram_addr_t qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr,
     size = TARGET_PAGE_ALIGN(size);
     new_block = g_malloc0(sizeof(*new_block));
     new_block->mr = mr;
-    new_block->length = size;
+    new_block->used_length = size;
+    new_block->max_length = size;
     new_block->flags = share ? RAM_SHARED : 0;
     new_block->host = file_ram_alloc(new_block, size,
                                      mem_path, errp);
@@ -1420,7 +1423,8 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     size = TARGET_PAGE_ALIGN(size);
     new_block = g_malloc0(sizeof(*new_block));
     new_block->mr = mr;
-    new_block->length = size;
+    new_block->used_length = size;
+    new_block->max_length = max_size;
     new_block->fd = -1;
     new_block->host = host;
     if (host) {
@@ -1475,11 +1479,11 @@ void qemu_ram_free(ram_addr_t addr)
                 xen_invalidate_map_cache_entry(block->host);
 #ifndef _WIN32
             } else if (block->fd >= 0) {
-                munmap(block->host, block->length);
+                munmap(block->host, block->max_length);
                 close(block->fd);
 #endif
             } else {
-                qemu_anon_ram_free(block->host, block->length);
+                qemu_anon_ram_free(block->host, block->max_length);
             }
             g_free(block);
             break;
@@ -1499,7 +1503,7 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length)
 
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
         offset = addr - block->offset;
-        if (offset < block->length) {
+        if (offset < block->max_length) {
             vaddr = ramblock_ptr(block, offset);
             if (block->flags & RAM_PREALLOC) {
                 ;
@@ -1575,7 +1579,7 @@ void *qemu_get_ram_ptr(ram_addr_t addr)
             return xen_map_cache(addr, 0, 0);
         } else if (block->host == NULL) {
             block->host =
-                xen_map_cache(block->offset, block->length, 1);
+                xen_map_cache(block->offset, block->max_length, 1);
         }
     }
     return ramblock_ptr(block, addr - block->offset);
@@ -1594,9 +1598,9 @@ static void *qemu_ram_ptr_length(ram_addr_t addr, hwaddr *size)
         RAMBlock *block;
 
         QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-            if (addr - block->offset < block->length) {
-                if (addr - block->offset + *size > block->length)
-                    *size = block->length - addr + block->offset;
+            if (addr - block->offset < block->max_length) {
+                if (addr - block->offset + *size > block->max_length)
+                    *size = block->max_length - addr + block->offset;
                 return ramblock_ptr(block, addr - block->offset);
             }
         }
@@ -1619,7 +1623,7 @@ MemoryRegion *qemu_ram_addr_from_host(void *ptr, ram_addr_t *ram_addr)
     }
 
     block = ram_list.mru_block;
-    if (block && block->host && host - block->host < block->length) {
+    if (block && block->host && host - block->host < block->max_length) {
         goto found;
     }
 
@@ -1628,7 +1632,7 @@ MemoryRegion *qemu_ram_addr_from_host(void *ptr, ram_addr_t *ram_addr)
         if (block->host == NULL) {
             continue;
         }
-        if (host - block->host < block->length) {
+        if (host - block->host < block->max_length) {
             goto found;
         }
     }
@@ -2873,7 +2877,7 @@ void qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque)
     RAMBlock *block;
 
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-        func(block->host, block->offset, block->length, opaque);
+        func(block->host, block->offset, block->used_length, opaque);
     }
 }
 #endif
-- 
MST

[Qemu-devel] [PATCH v3 4/8] exec: qemu_ram_alloc_resizeable, qemu_ram_resize

[Michael S. Tsirkin]

Add API to allocate "resizeable" RAM.
This looks just like regular RAM generally, but
has a special property that only a portion of it
(used_length) is actually used, and migrated.

This used_length size can change across reboots.

Follow up patches will change used_length for such blocks at migration,
making it easier to extend devices using such RAM (notably ACPI,
but in the future thinkably other ROMs) without breaking migration
compatibility or wasting ROM (guest) memory.

Device is notified on resize, so it can adjust if necessary.

qemu_ram_alloc_resizeable allocates this memory, qemu_ram_resize resizes
it.

Note: nothing prevents making all RAM resizeable in this way.
However, reviewers felt that only enabling this selectively will
make some class of errors easier to detect.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/exec/cpu-all.h  |  9 ++++--
 include/exec/ram_addr.h |  7 +++++
 exec.c                  | 75 ++++++++++++++++++++++++++++++++++++++++++++++---
 3 files changed, 84 insertions(+), 7 deletions(-)

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 6f2130e..7ced147 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -299,12 +299,15 @@ CPUArchState *cpu_copy(CPUArchState *env);
 
 /* memory API */
 
-typedef struct RAMBlock {
+typedef struct RAMBlock RAMBlock;
+
+struct RAMBlock {
     struct MemoryRegion *mr;
     uint8_t *host;
     ram_addr_t offset;
     ram_addr_t used_length;
     ram_addr_t max_length;
+    void (*resized)(const char*, uint64_t length, void *host);
     uint32_t flags;
     char idstr[256];
     /* Reads can take either the iothread or the ramlist lock.
@@ -312,11 +315,11 @@ typedef struct RAMBlock {
      */
     QTAILQ_ENTRY(RAMBlock) next;
     int fd;
-} RAMBlock;
+};
 
 static inline void *ramblock_ptr(RAMBlock *block, ram_addr_t offset)
 {
-    assert(offset < block->length);
+    assert(offset < block->used_length);
     assert(block->host);
     return (char *)block->host + offset;
 }
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index 254931c..ff558a4 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -28,12 +28,19 @@ ram_addr_t qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr,
 ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                    MemoryRegion *mr, Error **errp);
 ram_addr_t qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr, Error **errp);
+ram_addr_t qemu_ram_alloc_resizeable(ram_addr_t size, ram_addr_t max_size,
+                                     void (*resized)(const char*,
+                                                     uint64_t length,
+                                                     void *host),
+                                     MemoryRegion *mr, Error **errp);
 int qemu_get_ram_fd(ram_addr_t addr);
 void *qemu_get_ram_block_host_ptr(ram_addr_t addr);
 void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);
 
+int qemu_ram_resize(ram_addr_t base, ram_addr_t newsize, Error **errp);
+
 static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
                                                  ram_addr_t length,
                                                  unsigned client)
diff --git a/exec.c b/exec.c
index b69216a..71af138 100644
--- a/exec.c
+++ b/exec.c
@@ -75,6 +75,11 @@ static MemoryRegion io_mem_unassigned;
 /* RAM is mmap-ed with MAP_SHARED */
 #define RAM_SHARED     (1 << 1)
 
+/* Only a portion of RAM (used_length) is actually used, and migrated.
+ * This used_length size can change across reboots.
+ */
+#define RAM_RESIZEABLE (1 << 2)
+
 #endif
 
 struct CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
@@ -1186,7 +1191,7 @@ static ram_addr_t find_ram_offset(ram_addr_t size)
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
         ram_addr_t end, next = RAM_ADDR_MAX;
 
-        end = block->offset + block->length;
+        end = block->offset + block->max_length;
 
         QTAILQ_FOREACH(next_block, &ram_list.blocks, next) {
             if (next_block->offset >= end) {
@@ -1214,7 +1219,7 @@ ram_addr_t last_ram_offset(void)
     ram_addr_t last = 0;
 
     QTAILQ_FOREACH(block, &ram_list.blocks, next)
-        last = MAX(last, block->offset + block->length);
+        last = MAX(last, block->offset + block->max_length);
 
     return last;
 }
@@ -1296,6 +1301,42 @@ static int memory_try_enable_merging(void *addr, size_t len)
     return qemu_madvise(addr, len, QEMU_MADV_MERGEABLE);
 }
 
+int qemu_ram_resize(ram_addr_t base, ram_addr_t newsize, Error **errp)
+{
+    RAMBlock *block = find_ram_block(base);
+
+    assert(block);
+
+    if (block->used_length == newsize) {
+        return 0;
+    }
+
+    if (!(block->flags & RAM_RESIZEABLE)) {
+        error_setg_errno(errp, EINVAL,
+                         "Length mismatch: %s: 0x" RAM_ADDR_FMT
+                         " in != 0x" RAM_ADDR_FMT, block->idstr,
+                         newsize, block->used_length);
+        return -EINVAL;
+    }
+
+    if (block->max_length < newsize) {
+        error_setg_errno(errp, EINVAL,
+                         "Length too large: %s: 0x" RAM_ADDR_FMT
+                         " > 0x" RAM_ADDR_FMT, block->idstr,
+                         newsize, block->max_length);
+        return -EINVAL;
+    }
+
+    cpu_physical_memory_clear_dirty_range(block->offset, block->used_length);
+    block->used_length = newsize;
+    cpu_physical_memory_set_dirty_range(block->offset, block->used_length);
+    memory_region_set_size(block->mr, newsize);
+    if (block->resized) {
+        block->resized(block->idstr, newsize, block->host);
+    }
+    return 0;
+}
+
 static ram_addr_t ram_block_add(RAMBlock *new_block, Error **errp)
 {
     RAMBlock *block;
@@ -1413,7 +1454,12 @@ ram_addr_t qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr,
 }
 #endif
 
-ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
+static
+ram_addr_t qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
+                                   void (*resized)(const char*,
+                                                   uint64_t length,
+                                                   void *host),
+                                   void *host, bool resizeable,
                                    MemoryRegion *mr, Error **errp)
 {
     RAMBlock *new_block;
@@ -1421,15 +1467,21 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     Error *local_err = NULL;
 
     size = TARGET_PAGE_ALIGN(size);
+    max_size = TARGET_PAGE_ALIGN(max_size);
     new_block = g_malloc0(sizeof(*new_block));
     new_block->mr = mr;
+    new_block->resized = resized;
     new_block->used_length = size;
     new_block->max_length = max_size;
+    assert(max_size >= size);
     new_block->fd = -1;
     new_block->host = host;
     if (host) {
         new_block->flags |= RAM_PREALLOC;
     }
+    if (resizeable) {
+        new_block->flags |= RAM_RESIZEABLE;
+    }
     addr = ram_block_add(new_block, &local_err);
     if (local_err) {
         g_free(new_block);
@@ -1439,9 +1491,24 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     return addr;
 }
 
+ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
+                                   MemoryRegion *mr, Error **errp)
+{
+    return qemu_ram_alloc_internal(size, size, NULL, host, false, mr, errp);
+}
+
 ram_addr_t qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr, Error **errp)
 {
-    return qemu_ram_alloc_from_ptr(size, NULL, mr, errp);
+    return qemu_ram_alloc_internal(size, size, NULL, NULL, false, mr, errp);
+}
+
+ram_addr_t qemu_ram_alloc_resizeable(ram_addr_t size, ram_addr_t maxsz,
+                                     void (*resized)(const char*,
+                                                     uint64_t length,
+                                                     void *host),
+                                     MemoryRegion *mr, Error **errp)
+{
+    return qemu_ram_alloc_internal(size, maxsz, resized, NULL, true, mr, errp);
 }
 
 void qemu_ram_free_from_ptr(ram_addr_t addr)
-- 
MST

[Qemu-devel] [PATCH v3 5/8] arch_init: support resizing on incoming migration

[Michael S. Tsirkin]

If block used_length does not match, try to resize it.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 arch_init.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index 106f46e..cfedbf0 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -1086,11 +1086,14 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
 
                 QTAILQ_FOREACH(block, &ram_list.blocks, next) {
                     if (!strncmp(id, block->idstr, sizeof(id))) {
-                        if (block->used_length != length) {
-                            error_report("Length mismatch: %s: 0x" RAM_ADDR_FMT
-                                         " in != 0x" RAM_ADDR_FMT, id, length,
-                                         block->used_length);
-                            ret =  -EINVAL;
+                        if (length != block->used_length) {
+                            Error *local_err = NULL;
+
+                            ret = qemu_ram_resize(block->offset, length, &local_err);
+                            if (local_err) {
+                                error_report("%s", error_get_pretty(local_err));
+                                error_free(local_err);
+                            }
                         }
                         break;
                     }
-- 
MST

[Qemu-devel] [PATCH v3 6/8] memory: API to allocate resizeable RAM MR

[Michael S. Tsirkin]

Add API to allocate resizeable RAM MR.

This looks just like regular RAM generally, but
has a special property that only a portion of it
(used_length) is actually used, and migrated.

This used_length size can change across reboots.

Follow up patches will change used_length for such blocks at migration,
making it easier to extend devices using such RAM (notably ACPI,
but in the future thinkably other ROMs) without breaking migration
compatibility or wasting ROM (guest) memory.

Device is notified on resize, so it can adjust if necessary.

Note: nothing prevents making all RAM resizeable in this way.
However, reviewers felt that only enabling this selectively will
make some class of errors easier to detect.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 include/exec/memory.h | 24 ++++++++++++++++++++++++
 memory.c              | 17 +++++++++++++++++
 2 files changed, 41 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 0882221..0cd96b1 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -321,6 +321,30 @@ void memory_region_init_ram(MemoryRegion *mr,
                             uint64_t size,
                             Error **errp);
 
+/**
+ * memory_region_init_resizeable_ram:  Initialize memory region with resizeable
+ *                                     RAM.  Accesses into the region will
+ *                                     modify memory directly.  Only an initial
+ *                                     portion of this RAM is actually used.
+ *                                     The used size can change across reboots.
+ *
+ * @mr: the #MemoryRegion to be initialized.
+ * @owner: the object that tracks the region's reference count
+ * @name: the name of the region.
+ * @size: used size of the region.
+ * @max_size: max size of the region.
+ * @resized: callback to notify owner about used size change.
+ * @errp: pointer to Error*, to store an error if it happens.
+ */
+void memory_region_init_resizeable_ram(MemoryRegion *mr,
+                                       struct Object *owner,
+                                       const char *name,
+                                       uint64_t size,
+                                       uint64_t max_size,
+                                       void (*resized)(const char*,
+                                                       uint64_t length,
+                                                       void *host),
+                                       Error **errp);
 #ifdef __linux__
 /**
  * memory_region_init_ram_from_file:  Initialize RAM memory region with a
diff --git a/memory.c b/memory.c
index 618470b..c343bf3 100644
--- a/memory.c
+++ b/memory.c
@@ -1152,6 +1152,23 @@ void memory_region_init_ram(MemoryRegion *mr,
     mr->ram_addr = qemu_ram_alloc(size, mr, errp);
 }
 
+void memory_region_init_resizeable_ram(MemoryRegion *mr,
+                                       Object *owner,
+                                       const char *name,
+                                       uint64_t size,
+                                       uint64_t max_size,
+                                       void (*resized)(const char*,
+                                                       uint64_t length,
+                                                       void *host),
+                                       Error **errp)
+{
+    memory_region_init(mr, owner, name, size);
+    mr->ram = true;
+    mr->terminates = true;
+    mr->destructor = memory_region_destructor_ram;
+    mr->ram_addr = qemu_ram_alloc_resizeable(size, max_size, resized, mr, errp);
+}
+
 #ifdef __linux__
 void memory_region_init_ram_from_file(MemoryRegion *mr,
                                       struct Object *owner,
-- 
MST

[Qemu-devel] [PATCH v3 7/8] acpi-build: make ROMs RAM blocks resizeable

[Michael S. Tsirkin]

Use resizeable ram API so we can painlessly extend ROMs in the
future.  Note: migration is not affected, as we are
not actually changing the used length for RAM, which
is the part that's migrated.

Use this in acpi: reserve x16 more RAM space.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/lm32/lm32_hwsetup.h |  3 ++-
 include/hw/loader.h    |  4 ++--
 hw/core/loader.c       | 18 ++++++++++++++----
 hw/i386/acpi-build.c   | 19 ++++++++++++++-----
 4 files changed, 32 insertions(+), 12 deletions(-)

diff --git a/hw/lm32/lm32_hwsetup.h b/hw/lm32/lm32_hwsetup.h
index 9fd5e69..838754d 100644
--- a/hw/lm32/lm32_hwsetup.h
+++ b/hw/lm32/lm32_hwsetup.h
@@ -73,7 +73,8 @@ static inline void hwsetup_free(HWSetup *hw)
 static inline void hwsetup_create_rom(HWSetup *hw,
         hwaddr base)
 {
-    rom_add_blob("hwsetup", hw->data, TARGET_PAGE_SIZE, base, NULL, NULL, NULL);
+    rom_add_blob("hwsetup", hw->data, TARGET_PAGE_SIZE,
+                 TARGET_PAGE_SIZE, base, NULL, NULL, NULL);
 }
 
 static inline void hwsetup_add_u8(HWSetup *hw, uint8_t u)
diff --git a/include/hw/loader.h b/include/hw/loader.h
index 6481639..1d76108 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -60,7 +60,7 @@ int rom_add_file(const char *file, const char *fw_dir,
                  hwaddr addr, int32_t bootindex,
                  bool option_rom);
 ram_addr_t rom_add_blob(const char *name, const void *blob, size_t len,
-                   hwaddr addr, const char *fw_file_name,
+                   size_t max_len, hwaddr addr, const char *fw_file_name,
                    FWCfgReadCallback fw_callback, void *callback_opaque);
 int rom_add_elf_program(const char *name, void *data, size_t datasize,
                         size_t romsize, hwaddr addr);
@@ -74,7 +74,7 @@ void do_info_roms(Monitor *mon, const QDict *qdict);
 #define rom_add_file_fixed(_f, _a, _i)          \
     rom_add_file(_f, NULL, _a, _i, false)
 #define rom_add_blob_fixed(_f, _b, _l, _a)      \
-    rom_add_blob(_f, _b, _l, _a, NULL, NULL, NULL)
+    rom_add_blob(_f, _b, _l, _l, _a, NULL, NULL, NULL)
 
 #define PC_ROM_MIN_VGA     0xc0000
 #define PC_ROM_MIN_OPTION  0xc8000
diff --git a/hw/core/loader.c b/hw/core/loader.c
index 7527fd3..d3f8501 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -712,12 +712,22 @@ static void rom_insert(Rom *rom)
     QTAILQ_INSERT_TAIL(&roms, rom, next);
 }
 
+static void fw_cfg_resized(const char *id, uint64_t length, void *host)
+{
+    if (fw_cfg) {
+        fw_cfg_modify_file(fw_cfg, id + strlen("/rom@"), host, length);
+    }
+}
+
 static void *rom_set_mr(Rom *rom, Object *owner, const char *name)
 {
     void *data;
 
     rom->mr = g_malloc(sizeof(*rom->mr));
-    memory_region_init_ram(rom->mr, owner, name, rom->datasize, &error_abort);
+    memory_region_init_resizeable_ram(rom->mr, owner, name,
+                                      rom->datasize, rom->romsize,
+                                      fw_cfg_resized,
+                                      &error_abort);
     memory_region_set_readonly(rom->mr, true);
     vmstate_register_ram_global(rom->mr);
 
@@ -812,7 +822,7 @@ err:
 }
 
 ram_addr_t rom_add_blob(const char *name, const void *blob, size_t len,
-                   hwaddr addr, const char *fw_file_name,
+                   size_t max_len, hwaddr addr, const char *fw_file_name,
                    FWCfgReadCallback fw_callback, void *callback_opaque)
 {
     Rom *rom;
@@ -821,7 +831,7 @@ ram_addr_t rom_add_blob(const char *name, const void *blob, size_t len,
     rom           = g_malloc0(sizeof(*rom));
     rom->name     = g_strdup(name);
     rom->addr     = addr;
-    rom->romsize  = len;
+    rom->romsize  = max_len ? max_len : len;
     rom->datasize = len;
     rom->data     = g_malloc0(rom->datasize);
     memcpy(rom->data, blob, len);
@@ -841,7 +851,7 @@ ram_addr_t rom_add_blob(const char *name, const void *blob, size_t len,
 
         fw_cfg_add_file_callback(fw_cfg, fw_file_name,
                                  fw_callback, callback_opaque,
-                                 data, rom->romsize);
+                                 data, rom->datasize);
     }
     return ret;
 }
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index a4d0c0c..6a2e9c5 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -68,6 +68,9 @@
 
 #define ACPI_BUILD_TABLE_SIZE             0x20000
 
+/* Reserve RAM space for tables: add another order of magnitude. */
+#define ACPI_BUILD_TABLE_MAX_SIZE         0x200000
+
 /* #define DEBUG_ACPI_BUILD */
 #ifdef DEBUG_ACPI_BUILD
 #define ACPI_BUILD_DPRINTF(fmt, ...)        \
@@ -1718,6 +1721,11 @@ static void acpi_build_update(void *build_opaque, uint32_t offset)
     acpi_build(build_state->guest_info, &tables);
 
     assert(acpi_data_len(tables.table_data) == build_state->table_size);
+
+    /* Make sure RAM size is correct - in case it got changed by migration */
+    qemu_ram_resize(build_state->table_ram, build_state->table_size,
+                    &error_abort);
+
     memcpy(qemu_get_ram_ptr(build_state->table_ram), tables.table_data->data,
            build_state->table_size);
 
@@ -1734,10 +1742,10 @@ static void acpi_build_reset(void *build_opaque)
 }
 
 static ram_addr_t acpi_add_rom_blob(AcpiBuildState *build_state, GArray *blob,
-                               const char *name)
+                               const char *name, uint64_t max_size)
 {
-    return rom_add_blob(name, blob->data, acpi_data_len(blob), -1, name,
-                        acpi_build_update, build_state);
+    return rom_add_blob(name, blob->data, acpi_data_len(blob), max_size, -1,
+                        name, acpi_build_update, build_state);
 }
 
 static const VMStateDescription vmstate_acpi_build = {
@@ -1781,11 +1789,12 @@ void acpi_setup(PcGuestInfo *guest_info)
 
     /* Now expose it all to Guest */
     build_state->table_ram = acpi_add_rom_blob(build_state, tables.table_data,
-                                               ACPI_BUILD_TABLE_FILE);
+                                               ACPI_BUILD_TABLE_FILE,
+                                               ACPI_BUILD_TABLE_MAX_SIZE);
     assert(build_state->table_ram != RAM_ADDR_MAX);
     build_state->table_size = acpi_data_len(tables.table_data);
 
-    acpi_add_rom_blob(NULL, tables.linker, "etc/table-loader");
+    acpi_add_rom_blob(NULL, tables.linker, "etc/table-loader", 0);
 
     fw_cfg_add_file(guest_info->fw_cfg, ACPI_BUILD_TPMLOG_FILE,
                     tables.tcpalog->data, acpi_data_len(tables.tcpalog));
-- 
MST

[Qemu-devel] [PATCH v3 8/8] qemu_ram_resize: document assumptions

[Michael S. Tsirkin]

Document that when using qemu_ram_resize for anything mapped into guest
address space, it's the job of the resize function to update guest
visible state.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 exec.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/exec.c b/exec.c
index 71af138..58ac6d6 100644
--- a/exec.c
+++ b/exec.c
@@ -1301,6 +1301,13 @@ static int memory_try_enable_merging(void *addr, size_t len)
     return qemu_madvise(addr, len, QEMU_MADV_MERGEABLE);
 }
 
+/* Only legal before guest might have detected the memory size: e.g. on
+ * incoming migration, or right after reset.
+ *
+ * As memory core doesn't know how is memory accessed, it is up to
+ * resize callback to update device state and/or add assertions to detect
+ * misuse, if necessary.
+ */
 int qemu_ram_resize(ram_addr_t base, ram_addr_t newsize, Error **errp)
 {
     RAMBlock *block = find_ram_block(base);
-- 
MST

Re: [Qemu-devel] [PATCH v3 0/8] acpi: make ROMs resizeable

[Paolo Bonzini]

On 07/01/2015 20:48, Michael S. Tsirkin wrote:
> This is v3 of the patchset.
> Changes since v2:
>     - Address Paolo's comments: drop an unused function, fix up
>       comment.
> 
> Changes since v1:
>     - Any RAM can now be resizeable - there's no requirement
>       that it's device RAM any longer.
>     - For simplicity, max_size RAM is always pre-allocated
>     - Added memory_region_set_size, to keep MR size consistent
>       in case MR is guest visible (even though for current users,
>       it never is)
> 
> At the moment we migrate ROMs which reside in fw cfg, which allows
> changing ROM code at will, and supports migrating largish blocks early,
> with good performance.
> However, we are running into a problem: changing size breaks
> migration every time.
> This already requires somewhat messy compatibility support in
> acpi generation code, and it looks like there'll be more to come.
> 
> While recent patches by Paolo and others might make it easier
> to keep table size static for specific machine types, having
> a safety net in case we do need to change it sounds like
> a good idea.
> 
> Rather than try to guess the correct size once and for all,
> this patchset tries to make code future-proof, by
> adding support for resizeable ram blocks.
> 
> A (possibly very high) amount of space in ram_addr_t space is reserved
> and allocated in host for each block, but never used by fw cfg.
> If incoming block size differs from current size, block is
> reallocated. FW CFG is also notified and updated accordingly.
> 
> As reviewers felt that making all RAM "resizeable" in this
> way might make debugging migration harder, these patches
> set a per-block flag and only allow resizing for blocks
> where this was explicitly requested.
> 
> Note: migration stream is unaffected by these patches.
> This makes it possible to enable this functionality
> unconditionally, for all machine types.
> 
> In the future, this API might be handy for other things, besides ROMs.

Ok for 2.3, just squash 4 and 8 together.  No need to repost.

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>