[Qemu-devel] [RFC 00/28] bitmap handling optimization

[Qemu-devel] [RFC 00/28] bitmap handling optimization

[Juan Quintela]

Hi

This series split the dirty bitmap (8 bits per page, only three used)
into 3 individual bitmaps.  Once the conversion is done, operations
are handled by bitmap operations, not bit by bit.

- *_DIRTY_FLAG flags are gone, now we use memory.h DIRTY_MEMORY_*
   everywhere.

- We set/reset each flag individually
  (set_dirty_flags(0xff&~CODE_DIRTY_FLAG)) are gone.

- Rename several functions to clarify/make consistent things.

- I know it dont't pass checkpatch for long lines, propper submission
  should pass it. We have to have long lines, short variable names, or
  ugly line splitting :p

- DIRTY_MEMORY_NUM: how can one include exec/memory.h into cpu-all.h?
  #include it don't work, as a workaround, I have copied its value, but
  any better idea?  I can always create "exec/migration-flags.h", though.

- The meat of the code is patch 19.  Rest of patches are quite easy
(even that one is not too complex).

Only optimizations done so far are
set_dirty_range()/clear_dirty_range() that now operates with
bitmap_set/clear.

Note for Xen: cpu_physical_memory_set_dirty_range() was wrong for xen,
see comment on patch.

It passes virt-test migration tests, so it should be perfect.

I post it to ask for comments.

ToDo list:

- create a lock for the bitmaps and fold migration bitmap into this
  one.  This would avoid a copy and make things easier?

- As this code uses/abuses bitmaps, we need to change the type of the
  index from int to long.  With an int index, we can only access a
  maximum of 8TB guest (yes, this is not urgent, we have a couple of
  years to do it).

- merging KVM <-> QEMU bitmap as a bitmap and not bit-by-bit.

- spliting the KVM bitmap synchronization into chunks, i.e. not
  synchronize all memory, just enough to continue with migration.

Any further ideas/needs?

Thanks, Juan.

PD.  Why it took so long?

     Because I was trying to integrate the bitmap on the MemoryRegion
     abstraction.  Would have make the code cleaner, but hit dead-end
     after dead-end.  As practical terms, TCG don't know about
     MemoryRegions, it has been ported to run on top of them, but
     don't use them effectively.

The following changes since commit a684f3cf9b9b9c3cb82be87aafc463de8974610c:

  Merge remote-tracking branch 'kraxel/seabios-1.7.3.2' into staging (2013-09-30 17:15:27 -0500)

are available in the git repository at:


  git://github.com/juanquintela/qemu.git bitmap.next

for you to fetch changes up to a90a941c24589d618b3d8461675d9b5afb3288cb:

  memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations (2013-10-09 12:30:35 +0200)

----------------------------------------------------------------
Juan Quintela (28):
      Move prototypes to memory.h
      memory: cpu_physical_memory_set_dirty_flags() result is never used
      memory: cpu_physical_memory_set_dirty_range() return void
      exec: use accessor function to know if memory is dirty
      memory: create function to set a single dirty bit
      exec: create function to get a single dirty bit
      memory: make cpu_physical_memory_is_dirty return bool
      exec: simplify notdirty_mem_write()
      memory: all users of cpu_physical_memory_get_dirty used only one flag
      memory: set single dirty flags when possible
      memory: cpu_physical_memory_set_dirty_range() allways dirty all flags
      memory: cpu_physical_memory_mask_dirty_range() allways clear a single flag
      memory: use DIRTY_MEMORY_* instead of *_DIRTY_FLAG
      memory: use bit 2 for migration
      memory: make sure that client is always inside range
      memory: only resize dirty bitmap when memory size increases
      memory: cpu_physical_memory_clear_dirty_flag() result is never used
      bitmap: Add bitmap_zero_extend operation
      memory: split dirty bitmap into three
      memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user
      memory: unfold cpu_physical_memory_set_dirty() in its only user
      memory: unfold cpu_physical_memory_set_dirty_flag()
      memory: make cpu_physical_memory_get_dirty() the main function
      memory: cpu_physical_memory_get_dirty() is used as returning a bool
      memory: s/mask/clear/ cpu_physical_memory_mask_dirty_range
      memory: use find_next_bit() to find dirty bits
      memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations
      memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations

 cputlb.c                       |  4 +-
 exec.c                         | 41 ++++++++++--------
 include/exec/cpu-all.h         |  4 +-
 include/exec/cpu-common.h      |  4 --
 include/exec/memory-internal.h | 94 ++++++++++++++++++------------------------
 include/exec/memory.h          | 10 ++---
 include/qemu/bitmap.h          |  9 ++++
 memory.c                       | 12 +++---
 8 files changed, 88 insertions(+), 90 deletions(-)

[Qemu-devel] [PATCH 01/28] Move prototypes to memory.h

[Juan Quintela]

As the comment says, it should only be used on "core" memory files.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/cpu-common.h | 4 ----
 include/exec/memory.h     | 4 +++-
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index e4996e1..8110ef0 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -51,10 +51,6 @@ typedef void CPUWriteMemoryFunc(void *opaque, hwaddr addr, uint32_t value);
 typedef uint32_t CPUReadMemoryFunc(void *opaque, hwaddr addr);

 void qemu_ram_remap(ram_addr_t addr, ram_addr_t length);
-/* This should not be used by devices.  */
-MemoryRegion *qemu_ram_addr_from_host(void *ptr, ram_addr_t *ram_addr);
-void qemu_ram_set_idstr(ram_addr_t addr, const char *name, DeviceState *dev);
-
 void cpu_physical_memory_rw(hwaddr addr, uint8_t *buf,
                             int len, int is_write);
 static inline void cpu_physical_memory_read(hwaddr addr,
diff --git a/include/exec/memory.h b/include/exec/memory.h
index ebe0d24..04e69cf 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -1055,7 +1055,9 @@ void *address_space_map(AddressSpace *as, hwaddr addr,
 void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
                          int is_write, hwaddr access_len);

-
+/* This should not be used by devices.  */
+MemoryRegion *qemu_ram_addr_from_host(void *ptr, ram_addr_t *ram_addr);
+void qemu_ram_set_idstr(ram_addr_t addr, const char *name, DeviceState *dev);
 #endif

 #endif
-- 
1.8.3.1

[Qemu-devel] [PATCH 02/28] memory: cpu_physical_memory_set_dirty_flags() result is never used

[Juan Quintela]

So return void.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index d0e0633..c71a5e6 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -70,10 +70,10 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     return ret;
 }

-static inline int cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
+static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
                                                       int dirty_flags)
 {
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
 }

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
-- 
1.8.3.1

[Qemu-devel] [PATCH 03/28] memory: cpu_physical_memory_set_dirty_range() return void

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/memory.c b/memory.c
index 5a10fd0..1485871 100644
--- a/memory.c
+++ b/memory.c
@@ -1182,7 +1182,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
                              hwaddr size)
 {
     assert(mr->terminates);
-    return cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
+    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
 }

 bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 04/28] exec: use accessor function to know if memory is dirty

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index 26681ce..adbcef0 100644
--- a/exec.c
+++ b/exec.c
@@ -1470,7 +1470,7 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
     /* we remove the notdirty callback only if the code has been
        flushed */
-    if (dirty_flags == 0xff) {
+    if (cpu_physical_memory_is_dirty(ram_addr)) {
         CPUArchState *env = current_cpu->env_ptr;
         tlb_set_dirty(env, env->mem_io_vaddr);
     }
-- 
1.8.3.1

[Qemu-devel] [PATCH 05/28] memory: create function to set a single dirty bit

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 cputlb.c                       | 2 +-
 include/exec/memory-internal.h | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/cputlb.c b/cputlb.c
index 19ecf60..3aaa016 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -137,7 +137,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
 void tlb_unprotect_code_phys(CPUArchState *env, ram_addr_t ram_addr,
                              target_ulong vaddr)
 {
-    cpu_physical_memory_set_dirty_flags(ram_addr, CODE_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, CODE_DIRTY_FLAG);
 }

 static bool tlb_is_dirty_ram(CPUTLBEntry *tlbe)
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index c71a5e6..4ebab80 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -76,6 +76,12 @@ static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
     ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
 }

+static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
+                                                      int dirty_flag)
+{
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flag;
+}
+
 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 {
     cpu_physical_memory_set_dirty_flags(addr, 0xff);
-- 
1.8.3.1

[Qemu-devel] [PATCH 06/28] exec: create function to get a single dirty bit

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c                         | 3 ++-
 include/exec/memory-internal.h | 6 ++++++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index adbcef0..07b625f 100644
--- a/exec.c
+++ b/exec.c
@@ -1447,9 +1447,10 @@ found:
 static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
                                uint64_t val, unsigned size)
 {
+
     int dirty_flags;
     dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
-    if (!(dirty_flags & CODE_DIRTY_FLAG)) {
+    if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
         tb_invalidate_phys_page_fast(ram_addr, size);
         dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
     }
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 4ebab80..9cd2f53 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -49,6 +49,12 @@ static inline int cpu_physical_memory_get_dirty_flags(ram_addr_t addr)
     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS];
 }

+static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
+                                                     int dirty_flag)
+{
+    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & dirty_flag;
+}
+
 /* read dirty bit (return 0 or 1) */
 static inline int cpu_physical_memory_is_dirty(ram_addr_t addr)
 {
-- 
1.8.3.1

[Qemu-devel] [PATCH 07/28] memory: make cpu_physical_memory_is_dirty return bool

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 9cd2f53..eefe501 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -56,9 +56,12 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
 }

 /* read dirty bit (return 0 or 1) */
-static inline int cpu_physical_memory_is_dirty(ram_addr_t addr)
+static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
 {
-    return cpu_physical_memory_get_dirty_flags(addr) == 0xff;
+    bool vga = cpu_physical_memory_get_dirty_flag(addr, VGA_DIRTY_FLAG);
+    bool code = cpu_physical_memory_get_dirty_flag(addr, CODE_DIRTY_FLAG);
+    bool migration = cpu_physical_memory_get_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+    return vga && code && migration;
 }

 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 08/28] exec: simplify notdirty_mem_write()

[Juan Quintela]

We don't need to make special things for CODE, just set the other two bits

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index 07b625f..b9f2825 100644
--- a/exec.c
+++ b/exec.c
@@ -1447,12 +1447,8 @@ found:
 static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
                                uint64_t val, unsigned size)
 {
-
-    int dirty_flags;
-    dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
     if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
         tb_invalidate_phys_page_fast(ram_addr, size);
-        dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
     }
     switch (size) {
     case 1:
@@ -1467,8 +1463,8 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     default:
         abort();
     }
-    dirty_flags |= (0xff & ~CODE_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
+    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
     /* we remove the notdirty callback only if the code has been
        flushed */
     if (cpu_physical_memory_is_dirty(ram_addr)) {
-- 
1.8.3.1

[Qemu-devel] [PATCH 09/28] memory: all users of cpu_physical_memory_get_dirty used only one flag

[Juan Quintela]

So cpu_physical_memory_get_dirty_flags is not needed anymore

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index eefe501..b72c14a 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -44,11 +44,6 @@ void qemu_ram_free_from_ptr(ram_addr_t addr);
 #define CODE_DIRTY_FLAG      0x02
 #define MIGRATION_DIRTY_FLAG 0x08

-static inline int cpu_physical_memory_get_dirty_flags(ram_addr_t addr)
-{
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS];
-}
-
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                      int dirty_flag)
 {
@@ -66,7 +61,7 @@ static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)

 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
                                                 ram_addr_t length,
-                                                int dirty_flags)
+                                                int dirty_flag)
 {
     int ret = 0;
     ram_addr_t addr, end;
@@ -74,7 +69,7 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= cpu_physical_memory_get_dirty_flags(addr) & dirty_flags;
+        ret |= cpu_physical_memory_get_dirty_flag(addr, dirty_flag);
     }
     return ret;
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 10/28] memory: set single dirty flags when possible

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c                         | 7 ++++---
 include/exec/memory-internal.h | 4 +++-
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/exec.c b/exec.c
index b9f2825..0fd9c58 100644
--- a/exec.c
+++ b/exec.c
@@ -1887,7 +1887,8 @@ static void invalidate_and_set_dirty(hwaddr addr,
         /* invalidate code */
         tb_invalidate_phys_page_range(addr, addr + length, 0);
         /* set dirty bit */
-        cpu_physical_memory_set_dirty_flags(addr, (0xff & ~CODE_DIRTY_FLAG));
+        cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
+        cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
     }
     xen_modified_memory(addr, length);
 }
@@ -2467,8 +2468,8 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
                 /* invalidate code */
                 tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
                 /* set dirty bit */
-                cpu_physical_memory_set_dirty_flags(
-                    addr1, (0xff & ~CODE_DIRTY_FLAG));
+                cpu_physical_memory_set_dirty_flag(addr1, MIGRATION_DIRTY_FLAG);
+                cpu_physical_memory_set_dirty_flag(addr1, VGA_DIRTY_FLAG);
             }
         }
     }
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index b72c14a..bfbfc48 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -88,7 +88,9 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 {
-    cpu_physical_memory_set_dirty_flags(addr, 0xff);
+    cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
 }

 static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 11/28] memory: cpu_physical_memory_set_dirty_range() allways dirty all flags

[Juan Quintela]

So remove the flag argument and do it directly.  After this change, there is nothing else using cpu_physical_memory_set_dirty_flags() so remove it.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c                         |  2 +-
 include/exec/memory-internal.h | 11 ++---------
 memory.c                       |  2 +-
 3 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/exec.c b/exec.c
index 0fd9c58..b1b4da2 100644
--- a/exec.c
+++ b/exec.c
@@ -1180,7 +1180,7 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                        last_ram_offset() >> TARGET_PAGE_BITS);
     memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
            0, size >> TARGET_PAGE_BITS);
-    cpu_physical_memory_set_dirty_range(new_block->offset, size, 0xff);
+    cpu_physical_memory_set_dirty_range(new_block->offset, size);

     qemu_ram_setup_dump(new_block->host, size);
     qemu_madvise(new_block->host, size, QEMU_MADV_HUGEPAGE);
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index bfbfc48..e3b5834 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -74,12 +74,6 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     return ret;
 }

-static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
-                                                      int dirty_flags)
-{
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
-}
-
 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       int dirty_flag)
 {
@@ -102,15 +96,14 @@ static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
 }

 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
-                                                       ram_addr_t length,
-                                                       int dirty_flags)
+                                                       ram_addr_t length)
 {
     ram_addr_t addr, end;

     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_set_dirty_flags(addr, dirty_flags);
+        cpu_physical_memory_set_dirty(addr);
     }
     xen_modified_memory(addr, length);
 }
diff --git a/memory.c b/memory.c
index 1485871..41c1b5d 100644
--- a/memory.c
+++ b/memory.c
@@ -1182,7 +1182,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
                              hwaddr size)
 {
     assert(mr->terminates);
-    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
+    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size);
 }

 bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 12/28] memory: cpu_physical_memory_mask_dirty_range() allways clear a single flag

[Juan Quintela]

Document it

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c                         |  4 ++--
 include/exec/memory-internal.h | 12 ++++++------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/exec.c b/exec.c
index b1b4da2..f7691a0 100644
--- a/exec.c
+++ b/exec.c
@@ -678,7 +678,7 @@ static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,

 /* Note: start and end must be within the same ram block.  */
 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-                                     int dirty_flags)
+                                     int dirty_flag)
 {
     uintptr_t length;

@@ -688,7 +688,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
     length = end - start;
     if (length == 0)
         return;
-    cpu_physical_memory_mask_dirty_range(start, length, dirty_flags);
+    cpu_physical_memory_mask_dirty_range(start, length, dirty_flag);

     if (tcg_enabled()) {
         tlb_reset_dirty_range_all(start, end, length);
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index e3b5834..cfe1a24 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -87,10 +87,10 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
     cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
 }

-static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
-                                                        int dirty_flags)
+static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
+                                                       int dirty_flag)
 {
-    int mask = ~dirty_flags;
+    int mask = ~dirty_flag;

     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }
@@ -110,19 +110,19 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,

 static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
                                                         ram_addr_t length,
-                                                        int dirty_flags)
+                                                        int dirty_flag)
 {
     ram_addr_t addr, end;

     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_clear_dirty_flags(addr, dirty_flags);
+        cpu_physical_memory_clear_dirty_flag(addr, dirty_flag);
     }
 }

 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-                                     int dirty_flags);
+                                     int dirty_flag);

 #endif

-- 
1.8.3.1

[Qemu-devel] [PATCH 13/28] memory: use DIRTY_MEMORY_* instead of *_DIRTY_FLAG

[Juan Quintela]

Instead of the bitmap, we use the bitmap number.  Once done this, we
change all names from dirty_flag to memory regions naming of client.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 cputlb.c                       |  4 ++--
 exec.c                         | 18 +++++++++---------
 include/exec/memory-internal.h | 38 +++++++++++++++++---------------------
 include/exec/memory.h          |  3 ---
 memory.c                       | 10 ++++------
 5 files changed, 32 insertions(+), 41 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index 3aaa016..0c2ad48 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -129,7 +129,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
 {
     cpu_physical_memory_reset_dirty(ram_addr,
                                     ram_addr + TARGET_PAGE_SIZE,
-                                    CODE_DIRTY_FLAG);
+                                    DIRTY_MEMORY_CODE);
 }

 /* update the TLB so that writes in physical page 'phys_addr' are no longer
@@ -137,7 +137,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
 void tlb_unprotect_code_phys(CPUArchState *env, ram_addr_t ram_addr,
                              target_ulong vaddr)
 {
-    cpu_physical_memory_set_dirty_flag(ram_addr, CODE_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_CODE);
 }

 static bool tlb_is_dirty_ram(CPUTLBEntry *tlbe)
diff --git a/exec.c b/exec.c
index f7691a0..4f8f8a2 100644
--- a/exec.c
+++ b/exec.c
@@ -678,7 +678,7 @@ static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,

 /* Note: start and end must be within the same ram block.  */
 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-                                     int dirty_flag)
+                                     unsigned client)
 {
     uintptr_t length;

@@ -688,7 +688,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
     length = end - start;
     if (length == 0)
         return;
-    cpu_physical_memory_mask_dirty_range(start, length, dirty_flag);
+    cpu_physical_memory_mask_dirty_range(start, length, client);

     if (tcg_enabled()) {
         tlb_reset_dirty_range_all(start, end, length);
@@ -1447,7 +1447,7 @@ found:
 static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
                                uint64_t val, unsigned size)
 {
-    if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
+    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
         tb_invalidate_phys_page_fast(ram_addr, size);
     }
     switch (size) {
@@ -1463,8 +1463,8 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     default:
         abort();
     }
-    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_MIGRATION);
+    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_VGA);
     /* we remove the notdirty callback only if the code has been
        flushed */
     if (cpu_physical_memory_is_dirty(ram_addr)) {
@@ -1887,8 +1887,8 @@ static void invalidate_and_set_dirty(hwaddr addr,
         /* invalidate code */
         tb_invalidate_phys_page_range(addr, addr + length, 0);
         /* set dirty bit */
-        cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
-        cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
     }
     xen_modified_memory(addr, length);
 }
@@ -2468,8 +2468,8 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
                 /* invalidate code */
                 tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
                 /* set dirty bit */
-                cpu_physical_memory_set_dirty_flag(addr1, MIGRATION_DIRTY_FLAG);
-                cpu_physical_memory_set_dirty_flag(addr1, VGA_DIRTY_FLAG);
+                cpu_physical_memory_set_dirty_flag(addr1, DIRTY_MEMORY_MIGRATION);
+                cpu_physical_memory_set_dirty_flag(addr1, DIRTY_MEMORY_VGA);
             }
         }
     }
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index cfe1a24..3947caa 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -40,28 +40,24 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);

-#define VGA_DIRTY_FLAG       0x01
-#define CODE_DIRTY_FLAG      0x02
-#define MIGRATION_DIRTY_FLAG 0x08
-
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-                                                     int dirty_flag)
+                                                      unsigned client)
 {
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & dirty_flag;
+    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
 }

 /* read dirty bit (return 0 or 1) */
 static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
 {
-    bool vga = cpu_physical_memory_get_dirty_flag(addr, VGA_DIRTY_FLAG);
-    bool code = cpu_physical_memory_get_dirty_flag(addr, CODE_DIRTY_FLAG);
-    bool migration = cpu_physical_memory_get_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
+    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
+    bool migration = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
     return vga && code && migration;
 }

 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
                                                 ram_addr_t length,
-                                                int dirty_flag)
+                                                unsigned client)
 {
     int ret = 0;
     ram_addr_t addr, end;
@@ -69,28 +65,28 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= cpu_physical_memory_get_dirty_flag(addr, dirty_flag);
+        ret |= cpu_physical_memory_get_dirty_flag(addr, client);
     }
     return ret;
 }

 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
-                                                      int dirty_flag)
+                                                      unsigned client)
 {
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flag;
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
 }

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 {
-    cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
+    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
+    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
 }

 static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
-                                                       int dirty_flag)
+                                                       unsigned client)
 {
-    int mask = ~dirty_flag;
+    int mask = ~(1 << client);

     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }
@@ -110,19 +106,19 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,

 static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
                                                         ram_addr_t length,
-                                                        int dirty_flag)
+                                                        unsigned client)
 {
     ram_addr_t addr, end;

     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_clear_dirty_flag(addr, dirty_flag);
+        cpu_physical_memory_clear_dirty_flag(addr, client);
     }
 }

 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-                                     int dirty_flag);
+                                     unsigned client);

 #endif

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 04e69cf..4bb20d0 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -33,9 +33,6 @@
 typedef struct MemoryRegionOps MemoryRegionOps;
 typedef struct MemoryRegionMmio MemoryRegionMmio;

-/* Must match *_DIRTY_FLAGS in cpu-all.h.  To be replaced with dynamic
- * registration.
- */
 #define DIRTY_MEMORY_VGA       0
 #define DIRTY_MEMORY_CODE      1
 #define DIRTY_MEMORY_MIGRATION 3
diff --git a/memory.c b/memory.c
index 41c1b5d..d01a9c6 100644
--- a/memory.c
+++ b/memory.c
@@ -1174,8 +1174,7 @@ bool memory_region_get_dirty(MemoryRegion *mr, hwaddr addr,
                              hwaddr size, unsigned client)
 {
     assert(mr->terminates);
-    return cpu_physical_memory_get_dirty(mr->ram_addr + addr, size,
-                                         1 << client);
+    return cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
 }

 void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
@@ -1190,12 +1189,11 @@ bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
 {
     bool ret;
     assert(mr->terminates);
-    ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size,
-                                        1 << client);
+    ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
     if (ret) {
         cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
                                         mr->ram_addr + addr + size,
-                                        1 << client);
+                                        client);
     }
     return ret;
 }
@@ -1243,7 +1241,7 @@ void memory_region_reset_dirty(MemoryRegion *mr, hwaddr addr,
     assert(mr->terminates);
     cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
                                     mr->ram_addr + addr + size,
-                                    1 << client);
+                                    client);
 }

 void *memory_region_get_ram_ptr(MemoryRegion *mr)
-- 
1.8.3.1

[Qemu-devel] [PATCH 14/28] memory: use bit 2 for migration

[Juan Quintela]

For historical reasons it was bit 3.  One there create a constant to
know the number of clients.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 4bb20d0..a28c6bd 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -35,7 +35,8 @@ typedef struct MemoryRegionMmio MemoryRegionMmio;

 #define DIRTY_MEMORY_VGA       0
 #define DIRTY_MEMORY_CODE      1
-#define DIRTY_MEMORY_MIGRATION 3
+#define DIRTY_MEMORY_MIGRATION 2
+#define DIRTY_MEMORY_NUM       3	/* num of dirty bits */

 struct MemoryRegionMmio {
     CPUReadMemoryFunc *read[3];
-- 
1.8.3.1

[Qemu-devel] [PATCH 15/28] memory: make sure that client is always inside range

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 3947caa..e08ac42 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -43,6 +43,7 @@ void qemu_ram_free_from_ptr(ram_addr_t addr);
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
+    assert(client < DIRTY_MEMORY_NUM);
     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
 }

@@ -73,6 +74,7 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
+    assert(client < DIRTY_MEMORY_NUM);
     ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
 }

@@ -88,6 +90,8 @@ static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
 {
     int mask = ~(1 << client);

+    assert(client < DIRTY_MEMORY_NUM);
+
     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }

-- 
1.8.3.1

[Qemu-devel] [PATCH 16/28] memory: only resize dirty bitmap when memory size increases

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/exec.c b/exec.c
index 4f8f8a2..f037473 100644
--- a/exec.c
+++ b/exec.c
@@ -1116,6 +1116,9 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                    MemoryRegion *mr)
 {
     RAMBlock *block, *new_block;
+    ram_addr_t old_ram_size, new_ram_size;
+
+    old_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;

     size = TARGET_PAGE_ALIGN(size);
     new_block = g_malloc0(sizeof(*new_block));
@@ -1176,10 +1179,13 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     ram_list.version++;
     qemu_mutex_unlock_ramlist();

-    ram_list.phys_dirty = g_realloc(ram_list.phys_dirty,
-                                       last_ram_offset() >> TARGET_PAGE_BITS);
-    memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
+    new_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;
+
+    if (new_ram_size > old_ram_size) {
+        ram_list.phys_dirty = g_realloc(ram_list.phys_dirty, new_ram_size);
+        memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
            0, size >> TARGET_PAGE_BITS);
+    }
     cpu_physical_memory_set_dirty_range(new_block->offset, size);

     qemu_ram_setup_dump(new_block->host, size);
-- 
1.8.3.1

[Qemu-devel] [PATCH 17/28] memory: cpu_physical_memory_clear_dirty_flag() result is never used

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index e08ac42..3f885a6 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -85,14 +85,14 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
     cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
 }

-static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
+static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
                                                        unsigned client)
 {
     int mask = ~(1 << client);

     assert(client < DIRTY_MEMORY_NUM);

-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }

 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 18/28] bitmap: Add bitmap_zero_extend operation

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/qemu/bitmap.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
index 308bbb7..53f5f1f 100644
--- a/include/qemu/bitmap.h
+++ b/include/qemu/bitmap.h
@@ -219,4 +219,13 @@ unsigned long bitmap_find_next_zero_area(unsigned long *map,
 					 unsigned int nr,
 					 unsigned long align_mask);

+static inline unsigned long *bitmap_zero_extend(unsigned long *old,
+                                                int old_nbits, int new_nbits)
+{
+    int new_len = BITS_TO_LONGS(new_nbits) * sizeof(unsigned long);
+    unsigned long *new = g_realloc(old, new_len);
+    bitmap_clear(new, old_nbits, new_nbits - old_nbits);
+    return new;
+}
+
 #endif /* BITMAP_H */
-- 
1.8.3.1

[Qemu-devel] [PATCH 19/28] memory: split dirty bitmap into three

[Juan Quintela]

After all the previous patches, spliting the bitmap gets direct.

ToDo: Why can't i include "exec/memory.h" into cpu-all.h?  This is the
      reason that I have duplicated DIRTY_MEMORY_NUM.

ToDo2: current bitmaps have one int as index, this limit us to 8TB RAM
       guest, Should we move to longs?

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c                         |  9 ++++++---
 include/exec/cpu-all.h         |  4 +++-
 include/exec/memory-internal.h | 11 ++++-------
 3 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/exec.c b/exec.c
index f037473..6fd83c9 100644
--- a/exec.c
+++ b/exec.c
@@ -1182,9 +1182,12 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     new_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;

     if (new_ram_size > old_ram_size) {
-        ram_list.phys_dirty = g_realloc(ram_list.phys_dirty, new_ram_size);
-        memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
-           0, size >> TARGET_PAGE_BITS);
+        int i;
+        for(i = 0; i < DIRTY_MEMORY_NUM; i++) {
+            ram_list.dirty_memory[i] =
+                bitmap_zero_extend(ram_list.dirty_memory[i],
+                                   old_ram_size, new_ram_size);
+       }
     }
     cpu_physical_memory_set_dirty_range(new_block->offset, size);

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index b6998f0..019dc20 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -456,10 +456,12 @@ typedef struct RAMBlock {
     int fd;
 } RAMBlock;

+#define DIRTY_MEMORY_NUM       3
+
 typedef struct RAMList {
     QemuMutex mutex;
     /* Protected by the iothread lock.  */
-    uint8_t *phys_dirty;
+    unsigned long *dirty_memory[DIRTY_MEMORY_NUM];
     RAMBlock *mru_block;
     /* Protected by the ramlist lock.  */
     QTAILQ_HEAD(, RAMBlock) blocks;
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 3f885a6..71f198e 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -44,7 +44,7 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
     assert(client < DIRTY_MEMORY_NUM);
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
+    return test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

 /* read dirty bit (return 0 or 1) */
@@ -75,7 +75,8 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
     assert(client < DIRTY_MEMORY_NUM);
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
+
+    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
@@ -88,11 +89,7 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
                                                        unsigned client)
 {
-    int mask = ~(1 << client);
-
-    assert(client < DIRTY_MEMORY_NUM);
-
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
+    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 20/28] memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 71f198e..d6d3537 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -86,12 +86,6 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
     cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
 }

-static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
-                                                       unsigned client)
-{
-    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
-}
-
 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
                                                        ram_addr_t length)
 {
@@ -114,7 +108,7 @@ static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_clear_dirty_flag(addr, client);
+        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
     }
 }

-- 
1.8.3.1

[Qemu-devel] [PATCH 21/28] memory: unfold cpu_physical_memory_set_dirty() in its only user

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index d6d3537..5fc4eb6 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -79,13 +79,6 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
     set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

-static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
-{
-    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
-    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
-}
-
 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
                                                        ram_addr_t length)
 {
@@ -94,7 +87,9 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_set_dirty(addr);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
     }
     xen_modified_memory(addr, length);
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 22/28] memory: unfold cpu_physical_memory_set_dirty_flag()

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 5fc4eb6..e56f43b 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -87,9 +87,12 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
-        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
+        set_bit(addr >> TARGET_PAGE_BITS,
+                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
+        set_bit(addr >> TARGET_PAGE_BITS,
+                ram_list.dirty_memory[DIRTY_MEMORY_VGA]);
+        set_bit(addr >> TARGET_PAGE_BITS,
+                ram_list.dirty_memory[DIRTY_MEMORY_CODE]);
     }
     xen_modified_memory(addr, length);
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 23/28] memory: make cpu_physical_memory_get_dirty() the main function

[Juan Quintela]

And make cpu_physical_memory_get_dirty_flag() to use it.  It used to
be the other way around.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 35 ++++++++++++++++++-----------------
 1 file changed, 18 insertions(+), 17 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index e56f43b..f66d2ce 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -40,22 +40,6 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);

-static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-                                                      unsigned client)
-{
-    assert(client < DIRTY_MEMORY_NUM);
-    return test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
-}
-
-/* read dirty bit (return 0 or 1) */
-static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
-{
-    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
-    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
-    bool migration = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-    return vga && code && migration;
-}
-
 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
                                                 ram_addr_t length,
                                                 unsigned client)
@@ -63,14 +47,31 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     int ret = 0;
     ram_addr_t addr, end;

+    assert(client < DIRTY_MEMORY_NUM);
+
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= cpu_physical_memory_get_dirty_flag(addr, client);
+        ret |= test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
     }
     return ret;
 }

+static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
+                                                      unsigned client)
+{
+    return cpu_physical_memory_get_dirty(addr, 1, client);
+}
+
+/* read dirty bit (return 0 or 1) */
+static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
+{
+    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
+    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
+    bool migration = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
+    return vga && code && migration;
+}
+
 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
-- 
1.8.3.1

[Qemu-devel] [PATCH 24/28] memory: cpu_physical_memory_get_dirty() is used as returning a bool

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index f66d2ce..de8f279 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -40,11 +40,10 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);

-static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
-                                                ram_addr_t length,
-                                                unsigned client)
+static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
+                                                 ram_addr_t length,
+                                                 unsigned client)
 {
-    int ret = 0;
     ram_addr_t addr, end;

     assert(client < DIRTY_MEMORY_NUM);
@@ -52,9 +51,11 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
+        if (test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client])) {
+            return true;
+        }
     }
-    return ret;
+    return false;
 }

 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 25/28] memory: s/mask/clear/ cpu_physical_memory_mask_dirty_range

[Juan Quintela]

Now all functions use the same wording that bitops/bitmap operations

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c                         | 2 +-
 include/exec/memory-internal.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/exec.c b/exec.c
index 6fd83c9..fcdcc56 100644
--- a/exec.c
+++ b/exec.c
@@ -688,7 +688,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
     length = end - start;
     if (length == 0)
         return;
-    cpu_physical_memory_mask_dirty_range(start, length, client);
+    cpu_physical_memory_clear_dirty_range(start, length, client);

     if (tcg_enabled()) {
         tlb_reset_dirty_range_all(start, end, length);
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index de8f279..0c1dbfa 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -99,9 +99,9 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     xen_modified_memory(addr, length);
 }

-static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
-                                                        ram_addr_t length,
-                                                        unsigned client)
+static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
+                                                         ram_addr_t length,
+                                                         unsigned client)
 {
     ram_addr_t addr, end;

-- 
1.8.3.1

[Qemu-devel] [PATCH 26/28] memory: use find_next_bit() to find dirty bits

[Juan Quintela]

This operation is way faster that doing it bit by bit.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 0c1dbfa..5a5bc0d 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -44,18 +44,15 @@ static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
                                                  ram_addr_t length,
                                                  unsigned client)
 {
-    ram_addr_t addr, end;
+    unsigned long end, page, next;

     assert(client < DIRTY_MEMORY_NUM);

-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        if (test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client])) {
-            return true;
-        }
-    }
-    return false;
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    next = find_next_bit(ram_list.dirty_memory[client], end, page);
+
+    return next < end;
 }

 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 27/28] memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations

[Juan Quintela]

We were setting a range of bits, so use bitmap_set().

Note: xen has always been wrong, and should have used start insntead
of addr from the beggining.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 5a5bc0d..2f704e8 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -81,19 +81,14 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
                                                        ram_addr_t length)
 {
-    ram_addr_t addr, end;
+    unsigned long end, page;

-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        set_bit(addr >> TARGET_PAGE_BITS,
-                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
-        set_bit(addr >> TARGET_PAGE_BITS,
-                ram_list.dirty_memory[DIRTY_MEMORY_VGA]);
-        set_bit(addr >> TARGET_PAGE_BITS,
-                ram_list.dirty_memory[DIRTY_MEMORY_CODE]);
-    }
-    xen_modified_memory(addr, length);
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
+    xen_modified_memory(start, length);
 }

 static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 28/28] memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations

[Juan Quintela]

We were clearing a range of bits, so use bitmap_set().

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/memory-internal.h | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 2f704e8..d46570e 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -95,13 +95,11 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
                                                          ram_addr_t length,
                                                          unsigned client)
 {
-    ram_addr_t addr, end;
+    unsigned long end, page;

-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
-    }
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
 }

 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-- 
1.8.3.1

Re: [Qemu-devel] [RFC 00/28] bitmap handling optimization

[Paolo Bonzini]

Il 09/10/2013 13:28, Juan Quintela ha scritto:
> Hi
> 
> This series split the dirty bitmap (8 bits per page, only three used)
> into 3 individual bitmaps.  Once the conversion is done, operations
> are handled by bitmap operations, not bit by bit.
> 
> - *_DIRTY_FLAG flags are gone, now we use memory.h DIRTY_MEMORY_*
>    everywhere.
> 
> - We set/reset each flag individually
>   (set_dirty_flags(0xff&~CODE_DIRTY_FLAG)) are gone.
> 
> - Rename several functions to clarify/make consistent things.
> 
> - I know it dont't pass checkpatch for long lines, propper submission
>   should pass it. We have to have long lines, short variable names, or
>   ugly line splitting :p
> 
> - DIRTY_MEMORY_NUM: how can one include exec/memory.h into cpu-all.h?
>   #include it don't work, as a workaround, I have copied its value, but
>   any better idea?  I can always create "exec/migration-flags.h", though.

I think both files are too "central" and you get some sort of circular dependency.

The solution could be to move RAM definitions from cpu-all.h to
memory-internal.h.  For example:

diff --git a/arch_init.c b/arch_init.c
index 7545d96..8752e27 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -47,7 +47,7 @@
 #include "qemu/config-file.h"
 #include "qmp-commands.h"
 #include "trace.h"
-#include "exec/cpu-all.h"
+#include "exec/memory-internal.h"
 #include "hw/acpi/acpi.h"
 
 #ifdef DEBUG_ARCH_INIT
diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 019dc20..4cfde66 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -435,57 +435,11 @@ void cpu_watchpoint_remove_all(CPUArchState *env, int mask);
 
 #if !defined(CONFIG_USER_ONLY)
 
-/* memory API */
-
 extern ram_addr_t ram_size;
-
-/* RAM is pre-allocated and passed into qemu_ram_alloc_from_ptr */
-#define RAM_PREALLOC_MASK   (1 << 0)
-
-typedef struct RAMBlock {
-    struct MemoryRegion *mr;
-    uint8_t *host;
-    ram_addr_t offset;
-    ram_addr_t length;
-    uint32_t flags;
-    char idstr[256];
-    /* Reads can take either the iothread or the ramlist lock.
-     * Writes must take both locks.
-     */
-    QTAILQ_ENTRY(RAMBlock) next;
-    int fd;
-} RAMBlock;
-
-#define DIRTY_MEMORY_NUM       3
-
-typedef struct RAMList {
-    QemuMutex mutex;
-    /* Protected by the iothread lock.  */
-    unsigned long *dirty_memory[DIRTY_MEMORY_NUM];
-    RAMBlock *mru_block;
-    /* Protected by the ramlist lock.  */
-    QTAILQ_HEAD(, RAMBlock) blocks;
-    uint32_t version;
-} RAMList;
-extern RAMList ram_list;
-
 extern const char *mem_path;
 extern int mem_prealloc;
 
-/* Flags stored in the low bits of the TLB virtual address.  These are
-   defined so that fast path ram access is all zeros.  */
-/* Zero if TLB entry is valid.  */
-#define TLB_INVALID_MASK   (1 << 3)
-/* Set if TLB entry references a clean RAM page.  The iotlb entry will
-   contain the page physical address.  */
-#define TLB_NOTDIRTY    (1 << 4)
-/* Set if TLB entry is an IO callback.  */
-#define TLB_MMIO        (1 << 5)
-
 void dump_exec_info(FILE *f, fprintf_function cpu_fprintf);
-ram_addr_t last_ram_offset(void);
-void qemu_mutex_lock_ramlist(void);
-void qemu_mutex_unlock_ramlist(void);
 #endif /* !CONFIG_USER_ONLY */
 
 int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
diff --git a/include/exec/cputlb.h b/include/exec/cputlb.h
index e21cb60..719fa27 100644
--- a/include/exec/cputlb.h
+++ b/include/exec/cputlb.h
@@ -20,6 +20,17 @@
 #define CPUTLB_H
 
 #if !defined(CONFIG_USER_ONLY)
+
+/* Flags stored in the low bits of the TLB virtual address.  These are
+   defined so that fast path ram access is all zeros.  */
+/* Zero if TLB entry is valid.  */
+#define TLB_INVALID_MASK   (1 << 3)
+/* Set if TLB entry references a clean RAM page.  The iotlb entry will
+   contain the page physical address.  */
+#define TLB_NOTDIRTY    (1 << 4)
+/* Set if TLB entry is an IO callback.  */
+#define TLB_MMIO        (1 << 5)
+
 /* cputlb.c */
 void tlb_protect_code(ram_addr_t ram_addr);
 void tlb_unprotect_code_phys(CPUArchState *env, ram_addr_t ram_addr,
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index d46570e..aaf76c2 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -22,6 +22,35 @@
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 
+/* RAM is pre-allocated and passed into qemu_ram_alloc_from_ptr */
+#define RAM_PREALLOC_MASK   (1 << 0)
+
+typedef struct RAMBlock {
+    struct MemoryRegion *mr;
+    uint8_t *host;
+    ram_addr_t offset;
+    ram_addr_t length;
+    uint32_t flags;
+    char idstr[256];
+    /* Reads can take either the iothread or the ramlist lock.
+     * Writes must take both locks.
+     */
+    QTAILQ_ENTRY(RAMBlock) next;
+    int fd;
+} RAMBlock;
+
+#define DIRTY_MEMORY_NUM       3
+
+typedef struct RAMList {
+    QemuMutex mutex;
+    /* Protected by the iothread lock.  */
+    unsigned long *dirty_memory[DIRTY_MEMORY_NUM];
+    RAMBlock *mru_block;
+    /* Protected by the ramlist lock.  */
+    QTAILQ_HEAD(, RAMBlock) blocks;
+    uint32_t version;
+} RAMList;
+extern RAMList ram_list;
 
 typedef struct AddressSpaceDispatch AddressSpaceDispatch;
 
@@ -105,6 +134,10 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
                                      unsigned client);
 
+ram_addr_t last_ram_offset(void);
+void qemu_mutex_lock_ramlist(void);
+void qemu_mutex_unlock_ramlist(void);
+
 #endif
 
 #endif
diff --git a/include/exec/softmmu_template.h b/include/exec/softmmu_template.h
index 5bbc56a..cc27058 100644
--- a/include/exec/softmmu_template.h
+++ b/include/exec/softmmu_template.h
@@ -23,6 +23,7 @@
  */
 #include "qemu/timer.h"
 #include "exec/memory.h"
+#include "exec/cputlb.h"
 
 #define DATA_SIZE (1 << SHIFT)
 

Bonus points for splitting RAM save out of arch_init.c. :)

> - create a lock for the bitmaps and fold migration bitmap into this
>   one.  This would avoid a copy and make things easier?

I think this is less important and not that easy.  For example,
how fine-grained should the locking be without bogging down TCG?

You can speed up migration_bitmap_sync by a factor of 64 or more
if you copy word-by-word instead of memory_region_test_and_clear_dirty
and migration_bitmap_set_dirty.

Once you do that and also merge KVM and vhost bitmaps one word at a
time, it's likely that dirty bitmaps get almost out of the
profile.

> - As this code uses/abuses bitmaps, we need to change the type of the
>   index from int to long.  With an int index, we can only access a
>   maximum of 8TB guest (yes, this is not urgent, we have a couple of
>   years to do it).

Yes.

> - merging KVM <-> QEMU bitmap as a bitmap and not bit-by-bit.

Right.  All of vhost_dev_sync_region, kvm_get_dirty_pages_log_range,
xen_sync_dirty_bitmap are really working a word at a time.
So it should be easy to optimize the log_sync implementations to
work with a word-at-a-time API instead of memory_region_set_dirty.

> - spliting the KVM bitmap synchronization into chunks, i.e. not
>   synchronize all memory, just enough to continue with migration.

That can also help.  However, it's not easy to do it without
making ram_save_pending's computations too optimistic.

So I'd just focus on speeding up migration_bitmap_sync first.  It's
easy and should "almost" do the entirety of the work.

Paolo

Re: [Qemu-devel] [PATCH 28/28] memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations

[Richard Henderson]

On 10/09/2013 04:28 AM, Juan Quintela wrote:
> We were clearing a range of bits, so use bitmap_set().

Comment is slightly wrong.  ;-)

> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  include/exec/memory-internal.h | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 2f704e8..d46570e 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -95,13 +95,11 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
>                                                           ram_addr_t length,
>                                                           unsigned client)
>  {
> -    ram_addr_t addr, end;
> +    unsigned long end, page;
> 
> -    end = TARGET_PAGE_ALIGN(start + length);
> -    start &= TARGET_PAGE_MASK;
> -    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> -    }
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    bitmap_clear(ram_list.dirty_memory[client], page, end - page);

Re: [Qemu-devel] [PATCH 08/28] exec: simplify notdirty_mem_write()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> We don't need to make special things for CODE, just set the other two bits
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  exec.c | 8 ++------
>  1 file changed, 2 insertions(+), 6 deletions(-)
> 

> -    dirty_flags |= (0xff & ~CODE_DIRTY_FLAG);
> -    cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);

Worth writing as a single call:

cpu_physical_memory_set_dirty_flag(ram_addr,
MIGRATION_DIRTY_FLAG|VGA_DIRTY_FLAG);

or will that just get in the way of refactoring later in the series?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 12/28] memory: cpu_physical_memory_mask_dirty_range() allways clear a single flag

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 210 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> Document it

s/allways/always/ in the subject


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 08/28] exec: simplify notdirty_mem_write()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

On 10/09/2013 01:10 PM, Eric Blake wrote:
> On 10/09/2013 05:28 AM, Juan Quintela wrote:
>> We don't need to make special things for CODE, just set the other two bits
>>
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  exec.c | 8 ++------
>>  1 file changed, 2 insertions(+), 6 deletions(-)
>>
> 
>> -    dirty_flags |= (0xff & ~CODE_DIRTY_FLAG);
>> -    cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
>> +    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
>> +    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
> 
> Worth writing as a single call:
> 
> cpu_physical_memory_set_dirty_flag(ram_addr,
> MIGRATION_DIRTY_FLAG|VGA_DIRTY_FLAG);
> 
> or will that just get in the way of refactoring later in the series?

Answering myself - it gets in the way. Doing things explicitly one flag
at a time makes it easier to redirect flags to separate tables.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 13/28] memory: use DIRTY_MEMORY_* instead of *_DIRTY_FLAG

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> Instead of the bitmap, we use the bitmap number.  Once done this, we

s/done this/this is done/

> change all names from dirty_flag to memory regions naming of client.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 14/28] memory: use bit 2 for migration

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> For historical reasons it was bit 3.  One there create a constant to

s/One there/Once there,/

> know the number of clients.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 19/28] memory: split dirty bitmap into three

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> After all the previous patches, spliting the bitmap gets direct.
> 
> ToDo: Why can't i include "exec/memory.h" into cpu-all.h?  This is the
>       reason that I have duplicated DIRTY_MEMORY_NUM.
> 
> ToDo2: current bitmaps have one int as index, this limit us to 8TB RAM
>        guest, Should we move to longs?
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---

> +        for(i = 0; i < DIRTY_MEMORY_NUM; i++) {

Not your typical formatting.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 26/28] memory: use find_next_bit() to find dirty bits

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> This operation is way faster that doing it bit by bit.

s/that/than/

> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  include/exec/memory-internal.h | 15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 27/28] memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

On 10/09/2013 05:28 AM, Juan Quintela wrote:
> We were setting a range of bits, so use bitmap_set().
> 
> Note: xen has always been wrong, and should have used start insntead

s/insntead/instead/

> of addr from the beggining.

s/beggining/beginning/

> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  include/exec/memory-internal.h | 19 +++++++------------
>  1 file changed, 7 insertions(+), 12 deletions(-)
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]