[Qemu-devel] [PATCH v3 00/38] bitmap queue

[Qemu-devel] [PATCH v3 00/38] bitmap queue

[Juan Quintela]

Hi

This is the last version of the bitmap patches, changes from last submission:
- cleanups: now it passes checkpatch
- all coments from Eric & Paolo addressed
- fixed problem with DIRTY_MEMORY_NUM
- tested by Vinod and numbers are very good
- move bitmap operations to take long as index
- round ram_addr's of blocks as suggesetd by Paolo.

If there is no negative comments, will sent for inclusion.

Thanks, Juan.


[v2]
In this version:
- fixed all the comments from last versions (thanks Eric)
- kvm migration bitmap is synchronized using bitmap operations
- qemu bitmap -> migration bitmap is synchronized using bitmap operations
If bitmaps are not properly aligned, we fall back to old code.
Code survives virt-tests, so should be in quite good shape.

ToDo list:

- vga ram by default is not aligned in a page number multiple of 64,

  it could be optimized.  Kraxel?  It syncs the kvm bitmap at least 1
  a second or so? bitmap is only 2048 pages (16MB by default).
  We need to change the ram_addr only

- vga: still more, after we finish migration, vga code continues
  synchronizing the kvm bitmap on source machine.  Notice that there
  is no graphics client connected to the VGA.  Worth investigating?

- I haven't yet meassure speed differences on big hosts.  Vinod?

- Depending of performance, more optimizations to do.

- debugging printf's still on the code, just to see if we are taking
  (or not) the optimized paths.

And that is all.  Please test & comment.

Thanks, Juan.

The following changes since commit f46e720a82ccdf1a521cf459448f3f96ed895d43:

  qemu_opts_parse(): always check return value (2013-12-16 15:33:48 -0800)

are available in the git repository at:

  git://github.com/juanquintela/qemu.git bitmap.next

for you to fetch changes up to 197a77684c35f75e43d937220ffac72101783384:

  ram: align ram_addr_t's regions in multiples of 64 (2013-12-17 15:56:04 +0100)

----------------------------------------------------------------
Juan Quintela (38):
      bitmap: use long as index
      memory: cpu_physical_memory_set_dirty_flags() result is never used
      memory: cpu_physical_memory_set_dirty_range() return void
      exec: use accessor function to know if memory is dirty
      memory: create function to set a single dirty bit
      exec: create function to get a single dirty bit
      memory: make cpu_physical_memory_is_dirty return bool
      memory: all users of cpu_physical_memory_get_dirty used only one flag
      memory: set single dirty flags when possible
      memory: cpu_physical_memory_set_dirty_range() always dirty all flags
      memory: cpu_physical_memory_mask_dirty_range() always clears a single flag
      memory: use bit 2 for migration
      memory: make sure that client is always inside range
      memory: only resize dirty bitmap when memory size increases
      memory: cpu_physical_memory_clear_dirty_flag() result is never used
      bitmap: Add bitmap_zero_extend operation
      memory: split dirty bitmap into three
      memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user
      memory: unfold cpu_physical_memory_set_dirty() in its only user
      memory: unfold cpu_physical_memory_set_dirty_flag()
      memory: make cpu_physical_memory_get_dirty() the main function
      memory: cpu_physical_memory_get_dirty() is used as returning a bool
      memory: s/mask/clear/ cpu_physical_memory_mask_dirty_range
      memory: use find_next_bit() to find dirty bits
      memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations
      memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations
      memory: s/dirty/clean/ in cpu_physical_memory_is_dirty()
      memory: make cpu_physical_memory_reset_dirty() take a length parameter
      memory: cpu_physical_memory_set_dirty_tracking() should return void
      memory: split cpu_physical_memory_* functions to its own include
      memory: unfold memory_region_test_and_clear()
      kvm: use directly cpu_physical_memory_* api for tracking dirty pages
      kvm: refactor start address calculation
      memory: move bitmap synchronization to its own function
      memory: syncronize kvm bitmap using bitmaps operations
      ram: split function that synchronizes a range
      migration: synchronize memory bitmap 64bits at a time
      ram: align ram_addr_t's regions in multiples of 64

 arch_init.c                    |  52 ++++++++++++---
 cputlb.c                       |  11 +--
 exec.c                         |  78 +++++++++++-----------
 include/exec/cpu-all.h         |   3 +-
 include/exec/memory-internal.h |  90 -------------------------
 include/exec/memory.h          |  12 ++--
 include/exec/ram_addr.h        | 147 +++++++++++++++++++++++++++++++++++++++++
 include/qemu/bitmap.h          |  86 +++++++++++++-----------
 include/qemu/bitops.h          |  14 ++--
 kvm-all.c                      |  28 ++------
 memory.c                       |  17 ++---
 util/bitmap.c                  |  60 ++++++++---------
 12 files changed, 337 insertions(+), 261 deletions(-)
 create mode 100644 include/exec/ram_addr.h

[Qemu-devel] [PATCH 01/38] bitmap: use long as index

[Juan Quintela]

Move index and size fields from int to long.  We need that for
migration.  long is 64 bits on sane architectures, and 32bits should
be enough on all the 32bits architectures.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/qemu/bitmap.h | 77 ++++++++++++++++++++++++++-------------------------
 include/qemu/bitops.h | 14 +++++-----
 util/bitmap.c         | 60 +++++++++++++++++++--------------------
 3 files changed, 76 insertions(+), 75 deletions(-)

diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
index 308bbb7..afdd257 100644
--- a/include/qemu/bitmap.h
+++ b/include/qemu/bitmap.h
@@ -31,7 +31,7 @@
  * bitmap_andnot(dst, src1, src2, nbits)	*dst = *src1 & ~(*src2)
  * bitmap_complement(dst, src, nbits)		*dst = ~(*src)
  * bitmap_equal(src1, src2, nbits)		Are *src1 and *src2 equal?
- * bitmap_intersects(src1, src2, nbits) 	Do *src1 and *src2 overlap?
+ * bitmap_intersects(src1, src2, nbits)         Do *src1 and *src2 overlap?
  * bitmap_empty(src, nbits)			Are all bits zero in *src?
  * bitmap_full(src, nbits)			Are all bits set in *src?
  * bitmap_set(dst, pos, nbits)			Set specified bit area
@@ -62,71 +62,71 @@
         )

 #define DECLARE_BITMAP(name,bits)                  \
-	unsigned long name[BITS_TO_LONGS(bits)]
+        unsigned long name[BITS_TO_LONGS(bits)]

 #define small_nbits(nbits)                      \
-	((nbits) <= BITS_PER_LONG)
+        ((nbits) <= BITS_PER_LONG)

-int slow_bitmap_empty(const unsigned long *bitmap, int bits);
-int slow_bitmap_full(const unsigned long *bitmap, int bits);
+int slow_bitmap_empty(const unsigned long *bitmap, long bits);
+int slow_bitmap_full(const unsigned long *bitmap, long bits);
 int slow_bitmap_equal(const unsigned long *bitmap1,
-                   const unsigned long *bitmap2, int bits);
+                      const unsigned long *bitmap2, long bits);
 void slow_bitmap_complement(unsigned long *dst, const unsigned long *src,
-                         int bits);
+                            long bits);
 void slow_bitmap_shift_right(unsigned long *dst,
-                          const unsigned long *src, int shift, int bits);
+                             const unsigned long *src, int shift, long bits);
 void slow_bitmap_shift_left(unsigned long *dst,
-                         const unsigned long *src, int shift, int bits);
+                            const unsigned long *src, int shift, long bits);
 int slow_bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
-                 const unsigned long *bitmap2, int bits);
+                    const unsigned long *bitmap2, long bits);
 void slow_bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
-                 const unsigned long *bitmap2, int bits);
+                    const unsigned long *bitmap2, long bits);
 void slow_bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
-                  const unsigned long *bitmap2, int bits);
+                     const unsigned long *bitmap2, long bits);
 int slow_bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
-                    const unsigned long *bitmap2, int bits);
+                       const unsigned long *bitmap2, long bits);
 int slow_bitmap_intersects(const unsigned long *bitmap1,
-			const unsigned long *bitmap2, int bits);
+                           const unsigned long *bitmap2, long bits);

-static inline unsigned long *bitmap_new(int nbits)
+static inline unsigned long *bitmap_new(long nbits)
 {
-    int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
+    long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
     return g_malloc0(len);
 }

-static inline void bitmap_zero(unsigned long *dst, int nbits)
+static inline void bitmap_zero(unsigned long *dst, long nbits)
 {
     if (small_nbits(nbits)) {
         *dst = 0UL;
     } else {
-        int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
+        long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
         memset(dst, 0, len);
     }
 }

-static inline void bitmap_fill(unsigned long *dst, int nbits)
+static inline void bitmap_fill(unsigned long *dst, long nbits)
 {
     size_t nlongs = BITS_TO_LONGS(nbits);
     if (!small_nbits(nbits)) {
-        int len = (nlongs - 1) * sizeof(unsigned long);
+        long len = (nlongs - 1) * sizeof(unsigned long);
         memset(dst, 0xff,  len);
     }
     dst[nlongs - 1] = BITMAP_LAST_WORD_MASK(nbits);
 }

 static inline void bitmap_copy(unsigned long *dst, const unsigned long *src,
-                               int nbits)
+                               long nbits)
 {
     if (small_nbits(nbits)) {
         *dst = *src;
     } else {
-        int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
+        long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
         memcpy(dst, src, len);
     }
 }

 static inline int bitmap_and(unsigned long *dst, const unsigned long *src1,
-                             const unsigned long *src2, int nbits)
+                             const unsigned long *src2, long nbits)
 {
     if (small_nbits(nbits)) {
         return (*dst = *src1 & *src2) != 0;
@@ -135,7 +135,7 @@ static inline int bitmap_and(unsigned long *dst, const unsigned long *src1,
 }

 static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
-			const unsigned long *src2, int nbits)
+                             const unsigned long *src2, long nbits)
 {
     if (small_nbits(nbits)) {
         *dst = *src1 | *src2;
@@ -145,7 +145,7 @@ static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
 }

 static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1,
-			const unsigned long *src2, int nbits)
+                              const unsigned long *src2, long nbits)
 {
     if (small_nbits(nbits)) {
         *dst = *src1 ^ *src2;
@@ -155,7 +155,7 @@ static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1,
 }

 static inline int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
-			const unsigned long *src2, int nbits)
+                                const unsigned long *src2, long nbits)
 {
     if (small_nbits(nbits)) {
         return (*dst = *src1 & ~(*src2)) != 0;
@@ -163,8 +163,9 @@ static inline int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
     return slow_bitmap_andnot(dst, src1, src2, nbits);
 }

-static inline void bitmap_complement(unsigned long *dst, const unsigned long *src,
-			int nbits)
+static inline void bitmap_complement(unsigned long *dst,
+                                     const unsigned long *src,
+                                     long nbits)
 {
     if (small_nbits(nbits)) {
         *dst = ~(*src) & BITMAP_LAST_WORD_MASK(nbits);
@@ -174,7 +175,7 @@ static inline void bitmap_complement(unsigned long *dst, const unsigned long *sr
 }

 static inline int bitmap_equal(const unsigned long *src1,
-			const unsigned long *src2, int nbits)
+                               const unsigned long *src2, long nbits)
 {
     if (small_nbits(nbits)) {
         return ! ((*src1 ^ *src2) & BITMAP_LAST_WORD_MASK(nbits));
@@ -183,7 +184,7 @@ static inline int bitmap_equal(const unsigned long *src1,
     }
 }

-static inline int bitmap_empty(const unsigned long *src, int nbits)
+static inline int bitmap_empty(const unsigned long *src, long nbits)
 {
     if (small_nbits(nbits)) {
         return ! (*src & BITMAP_LAST_WORD_MASK(nbits));
@@ -192,7 +193,7 @@ static inline int bitmap_empty(const unsigned long *src, int nbits)
     }
 }

-static inline int bitmap_full(const unsigned long *src, int nbits)
+static inline int bitmap_full(const unsigned long *src, long nbits)
 {
     if (small_nbits(nbits)) {
         return ! (~(*src) & BITMAP_LAST_WORD_MASK(nbits));
@@ -202,7 +203,7 @@ static inline int bitmap_full(const unsigned long *src, int nbits)
 }

 static inline int bitmap_intersects(const unsigned long *src1,
-			const unsigned long *src2, int nbits)
+                                    const unsigned long *src2, long nbits)
 {
     if (small_nbits(nbits)) {
         return ((*src1 & *src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0;
@@ -211,12 +212,12 @@ static inline int bitmap_intersects(const unsigned long *src1,
     }
 }

-void bitmap_set(unsigned long *map, int i, int len);
-void bitmap_clear(unsigned long *map, int start, int nr);
+void bitmap_set(unsigned long *map, long i, long len);
+void bitmap_clear(unsigned long *map, long start, long nr);
 unsigned long bitmap_find_next_zero_area(unsigned long *map,
-					 unsigned long size,
-					 unsigned long start,
-					 unsigned int nr,
-					 unsigned long align_mask);
+                                         unsigned long size,
+                                         unsigned long start,
+                                         unsigned long nr,
+                                         unsigned long align_mask);

 #endif /* BITMAP_H */
diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h
index 304c90c..340b1e7 100644
--- a/include/qemu/bitops.h
+++ b/include/qemu/bitops.h
@@ -28,7 +28,7 @@
  * @nr: the bit to set
  * @addr: the address to start counting from
  */
-static inline void set_bit(int nr, unsigned long *addr)
+static inline void set_bit(long nr, unsigned long *addr)
 {
 	unsigned long mask = BIT_MASK(nr);
         unsigned long *p = addr + BIT_WORD(nr);
@@ -41,7 +41,7 @@ static inline void set_bit(int nr, unsigned long *addr)
  * @nr: Bit to clear
  * @addr: Address to start counting from
  */
-static inline void clear_bit(int nr, unsigned long *addr)
+static inline void clear_bit(long nr, unsigned long *addr)
 {
 	unsigned long mask = BIT_MASK(nr);
         unsigned long *p = addr + BIT_WORD(nr);
@@ -54,7 +54,7 @@ static inline void clear_bit(int nr, unsigned long *addr)
  * @nr: Bit to change
  * @addr: Address to start counting from
  */
-static inline void change_bit(int nr, unsigned long *addr)
+static inline void change_bit(long nr, unsigned long *addr)
 {
 	unsigned long mask = BIT_MASK(nr);
         unsigned long *p = addr + BIT_WORD(nr);
@@ -67,7 +67,7 @@ static inline void change_bit(int nr, unsigned long *addr)
  * @nr: Bit to set
  * @addr: Address to count from
  */
-static inline int test_and_set_bit(int nr, unsigned long *addr)
+static inline int test_and_set_bit(long nr, unsigned long *addr)
 {
 	unsigned long mask = BIT_MASK(nr);
         unsigned long *p = addr + BIT_WORD(nr);
@@ -82,7 +82,7 @@ static inline int test_and_set_bit(int nr, unsigned long *addr)
  * @nr: Bit to clear
  * @addr: Address to count from
  */
-static inline int test_and_clear_bit(int nr, unsigned long *addr)
+static inline int test_and_clear_bit(long nr, unsigned long *addr)
 {
 	unsigned long mask = BIT_MASK(nr);
         unsigned long *p = addr + BIT_WORD(nr);
@@ -97,7 +97,7 @@ static inline int test_and_clear_bit(int nr, unsigned long *addr)
  * @nr: Bit to change
  * @addr: Address to count from
  */
-static inline int test_and_change_bit(int nr, unsigned long *addr)
+static inline int test_and_change_bit(long nr, unsigned long *addr)
 {
 	unsigned long mask = BIT_MASK(nr);
         unsigned long *p = addr + BIT_WORD(nr);
@@ -112,7 +112,7 @@ static inline int test_and_change_bit(int nr, unsigned long *addr)
  * @nr: bit number to test
  * @addr: Address to start counting from
  */
-static inline int test_bit(int nr, const unsigned long *addr)
+static inline int test_bit(long nr, const unsigned long *addr)
 {
 	return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1)));
 }
diff --git a/util/bitmap.c b/util/bitmap.c
index 687841d..9c6bb52 100644
--- a/util/bitmap.c
+++ b/util/bitmap.c
@@ -36,9 +36,9 @@
  * endian architectures.
  */

-int slow_bitmap_empty(const unsigned long *bitmap, int bits)
+int slow_bitmap_empty(const unsigned long *bitmap, long bits)
 {
-    int k, lim = bits/BITS_PER_LONG;
+    long k, lim = bits/BITS_PER_LONG;

     for (k = 0; k < lim; ++k) {
         if (bitmap[k]) {
@@ -54,9 +54,9 @@ int slow_bitmap_empty(const unsigned long *bitmap, int bits)
     return 1;
 }

-int slow_bitmap_full(const unsigned long *bitmap, int bits)
+int slow_bitmap_full(const unsigned long *bitmap, long bits)
 {
-    int k, lim = bits/BITS_PER_LONG;
+    long k, lim = bits/BITS_PER_LONG;

     for (k = 0; k < lim; ++k) {
         if (~bitmap[k]) {
@@ -74,9 +74,9 @@ int slow_bitmap_full(const unsigned long *bitmap, int bits)
 }

 int slow_bitmap_equal(const unsigned long *bitmap1,
-                      const unsigned long *bitmap2, int bits)
+                      const unsigned long *bitmap2, long bits)
 {
-    int k, lim = bits/BITS_PER_LONG;
+    long k, lim = bits/BITS_PER_LONG;

     for (k = 0; k < lim; ++k) {
         if (bitmap1[k] != bitmap2[k]) {
@@ -94,9 +94,9 @@ int slow_bitmap_equal(const unsigned long *bitmap1,
 }

 void slow_bitmap_complement(unsigned long *dst, const unsigned long *src,
-                            int bits)
+                            long bits)
 {
-    int k, lim = bits/BITS_PER_LONG;
+    long k, lim = bits/BITS_PER_LONG;

     for (k = 0; k < lim; ++k) {
         dst[k] = ~src[k];
@@ -108,10 +108,10 @@ void slow_bitmap_complement(unsigned long *dst, const unsigned long *src,
 }

 int slow_bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
-                    const unsigned long *bitmap2, int bits)
+                    const unsigned long *bitmap2, long bits)
 {
-    int k;
-    int nr = BITS_TO_LONGS(bits);
+    long k;
+    long nr = BITS_TO_LONGS(bits);
     unsigned long result = 0;

     for (k = 0; k < nr; k++) {
@@ -121,10 +121,10 @@ int slow_bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
 }

 void slow_bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
-                    const unsigned long *bitmap2, int bits)
+                    const unsigned long *bitmap2, long bits)
 {
-    int k;
-    int nr = BITS_TO_LONGS(bits);
+    long k;
+    long nr = BITS_TO_LONGS(bits);

     for (k = 0; k < nr; k++) {
         dst[k] = bitmap1[k] | bitmap2[k];
@@ -132,10 +132,10 @@ void slow_bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
 }

 void slow_bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
-                     const unsigned long *bitmap2, int bits)
+                     const unsigned long *bitmap2, long bits)
 {
-    int k;
-    int nr = BITS_TO_LONGS(bits);
+    long k;
+    long nr = BITS_TO_LONGS(bits);

     for (k = 0; k < nr; k++) {
         dst[k] = bitmap1[k] ^ bitmap2[k];
@@ -143,10 +143,10 @@ void slow_bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
 }

 int slow_bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
-                       const unsigned long *bitmap2, int bits)
+                       const unsigned long *bitmap2, long bits)
 {
-    int k;
-    int nr = BITS_TO_LONGS(bits);
+    long k;
+    long nr = BITS_TO_LONGS(bits);
     unsigned long result = 0;

     for (k = 0; k < nr; k++) {
@@ -157,10 +157,10 @@ int slow_bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,

 #define BITMAP_FIRST_WORD_MASK(start) (~0UL << ((start) % BITS_PER_LONG))

-void bitmap_set(unsigned long *map, int start, int nr)
+void bitmap_set(unsigned long *map, long start, long nr)
 {
     unsigned long *p = map + BIT_WORD(start);
-    const int size = start + nr;
+    const long size = start + nr;
     int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
     unsigned long mask_to_set = BITMAP_FIRST_WORD_MASK(start);

@@ -177,10 +177,10 @@ void bitmap_set(unsigned long *map, int start, int nr)
     }
 }

-void bitmap_clear(unsigned long *map, int start, int nr)
+void bitmap_clear(unsigned long *map, long start, long nr)
 {
     unsigned long *p = map + BIT_WORD(start);
-    const int size = start + nr;
+    const long size = start + nr;
     int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
     unsigned long mask_to_clear = BITMAP_FIRST_WORD_MASK(start);

@@ -212,10 +212,10 @@ void bitmap_clear(unsigned long *map, int start, int nr)
  * power of 2. A @align_mask of 0 means no alignment is required.
  */
 unsigned long bitmap_find_next_zero_area(unsigned long *map,
-					 unsigned long size,
-					 unsigned long start,
-					 unsigned int nr,
-					 unsigned long align_mask)
+                                         unsigned long size,
+                                         unsigned long start,
+                                         unsigned long nr,
+                                         unsigned long align_mask)
 {
     unsigned long index, end, i;
 again:
@@ -237,9 +237,9 @@ again:
 }

 int slow_bitmap_intersects(const unsigned long *bitmap1,
-                           const unsigned long *bitmap2, int bits)
+                           const unsigned long *bitmap2, long bits)
 {
-    int k, lim = bits/BITS_PER_LONG;
+    long k, lim = bits/BITS_PER_LONG;

     for (k = 0; k < lim; ++k) {
         if (bitmap1[k] & bitmap2[k]) {
-- 
1.8.3.1

[Qemu-devel] [PATCH 02/38] memory: cpu_physical_memory_set_dirty_flags() result is never used

[Juan Quintela]

So return void.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index d0e0633..c71a5e6 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -70,10 +70,10 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     return ret;
 }

-static inline int cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
+static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
                                                       int dirty_flags)
 {
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
 }

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
-- 
1.8.3.1

[Qemu-devel] [PATCH 03/38] memory: cpu_physical_memory_set_dirty_range() return void

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/memory.c b/memory.c
index 7764314..e497f99 100644
--- a/memory.c
+++ b/memory.c
@@ -1182,7 +1182,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
                              hwaddr size)
 {
     assert(mr->terminates);
-    return cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
+    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
 }

 bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 04/38] exec: use accessor function to know if memory is dirty

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index 00526d1..8800f8e 100644
--- a/exec.c
+++ b/exec.c
@@ -1507,7 +1507,7 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
     /* we remove the notdirty callback only if the code has been
        flushed */
-    if (dirty_flags == 0xff) {
+    if (cpu_physical_memory_is_dirty(ram_addr)) {
         CPUArchState *env = current_cpu->env_ptr;
         tlb_set_dirty(env, env->mem_io_vaddr);
     }
-- 
1.8.3.1

[Qemu-devel] [PATCH 05/38] memory: create function to set a single dirty bit

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 cputlb.c                       | 2 +-
 include/exec/memory-internal.h | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/cputlb.c b/cputlb.c
index fff0afb..72452e5 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -137,7 +137,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
 void tlb_unprotect_code_phys(CPUArchState *env, ram_addr_t ram_addr,
                              target_ulong vaddr)
 {
-    cpu_physical_memory_set_dirty_flags(ram_addr, CODE_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, CODE_DIRTY_FLAG);
 }

 static bool tlb_is_dirty_ram(CPUTLBEntry *tlbe)
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index c71a5e6..4ebab80 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -76,6 +76,12 @@ static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
     ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
 }

+static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
+                                                      int dirty_flag)
+{
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flag;
+}
+
 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 {
     cpu_physical_memory_set_dirty_flags(addr, 0xff);
-- 
1.8.3.1

[Qemu-devel] [PATCH 06/38] exec: create function to get a single dirty bit

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
---
 exec.c                         | 2 +-
 include/exec/memory-internal.h | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index 8800f8e..95bcf70 100644
--- a/exec.c
+++ b/exec.c
@@ -1486,7 +1486,7 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
 {
     int dirty_flags;
     dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
-    if (!(dirty_flags & CODE_DIRTY_FLAG)) {
+    if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
         tb_invalidate_phys_page_fast(ram_addr, size);
         dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
     }
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 4ebab80..136198c 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -49,6 +49,12 @@ static inline int cpu_physical_memory_get_dirty_flags(ram_addr_t addr)
     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS];
 }

+static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
+                                                      int dirty_flag)
+{
+    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & dirty_flag;
+}
+
 /* read dirty bit (return 0 or 1) */
 static inline int cpu_physical_memory_is_dirty(ram_addr_t addr)
 {
-- 
1.8.3.1

[Qemu-devel] [PATCH 07/38] memory: make cpu_physical_memory_is_dirty return bool

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c                         | 7 ++-----
 include/exec/memory-internal.h | 8 ++++++--
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/exec.c b/exec.c
index 95bcf70..a1fc280 100644
--- a/exec.c
+++ b/exec.c
@@ -1484,11 +1484,8 @@ found:
 static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
                                uint64_t val, unsigned size)
 {
-    int dirty_flags;
-    dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
     if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
         tb_invalidate_phys_page_fast(ram_addr, size);
-        dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
     }
     switch (size) {
     case 1:
@@ -1503,8 +1500,8 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     default:
         abort();
     }
-    dirty_flags |= (0xff & ~CODE_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
+    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
     /* we remove the notdirty callback only if the code has been
        flushed */
     if (cpu_physical_memory_is_dirty(ram_addr)) {
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 136198c..0b25c3f 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -56,9 +56,13 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
 }

 /* read dirty bit (return 0 or 1) */
-static inline int cpu_physical_memory_is_dirty(ram_addr_t addr)
+static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
 {
-    return cpu_physical_memory_get_dirty_flags(addr) == 0xff;
+    bool vga = cpu_physical_memory_get_dirty_flag(addr, VGA_DIRTY_FLAG);
+    bool code = cpu_physical_memory_get_dirty_flag(addr, CODE_DIRTY_FLAG);
+    bool migration =
+        cpu_physical_memory_get_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+    return vga && code && migration;
 }

 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 08/38] memory: all users of cpu_physical_memory_get_dirty used only one flag

[Juan Quintela]

So cpu_physical_memory_get_dirty_flags is not needed anymore

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 0b25c3f..53cfe83 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -44,11 +44,6 @@ void qemu_ram_free_from_ptr(ram_addr_t addr);
 #define CODE_DIRTY_FLAG      0x02
 #define MIGRATION_DIRTY_FLAG 0x08

-static inline int cpu_physical_memory_get_dirty_flags(ram_addr_t addr)
-{
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS];
-}
-
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                       int dirty_flag)
 {
@@ -67,7 +62,7 @@ static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)

 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
                                                 ram_addr_t length,
-                                                int dirty_flags)
+                                                int dirty_flag)
 {
     int ret = 0;
     ram_addr_t addr, end;
@@ -75,7 +70,7 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= cpu_physical_memory_get_dirty_flags(addr) & dirty_flags;
+        ret |= cpu_physical_memory_get_dirty_flag(addr, dirty_flag);
     }
     return ret;
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 09/38] memory: set single dirty flags when possible

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c                         | 7 ++++---
 include/exec/memory-internal.h | 4 +++-
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/exec.c b/exec.c
index a1fc280..6981f73 100644
--- a/exec.c
+++ b/exec.c
@@ -1911,7 +1911,8 @@ static void invalidate_and_set_dirty(hwaddr addr,
         /* invalidate code */
         tb_invalidate_phys_page_range(addr, addr + length, 0);
         /* set dirty bit */
-        cpu_physical_memory_set_dirty_flags(addr, (0xff & ~CODE_DIRTY_FLAG));
+        cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
+        cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
     }
     xen_modified_memory(addr, length);
 }
@@ -2493,8 +2494,8 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
                 /* invalidate code */
                 tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
                 /* set dirty bit */
-                cpu_physical_memory_set_dirty_flags(
-                    addr1, (0xff & ~CODE_DIRTY_FLAG));
+                cpu_physical_memory_set_dirty_flag(addr1, MIGRATION_DIRTY_FLAG);
+                cpu_physical_memory_set_dirty_flag(addr1, VGA_DIRTY_FLAG);
             }
         }
     }
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 53cfe83..9f4ad69 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -89,7 +89,9 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 {
-    cpu_physical_memory_set_dirty_flags(addr, 0xff);
+    cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
 }

 static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 10/38] memory: cpu_physical_memory_set_dirty_range() always dirty all flags

[Juan Quintela]

So remove the flag argument and do it directly.  After this change,
there is nothing else using cpu_physical_memory_set_dirty_flags() so
remove it.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c                         |  2 +-
 include/exec/memory-internal.h | 11 ++---------
 memory.c                       |  2 +-
 3 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/exec.c b/exec.c
index 6981f73..45fdb84 100644
--- a/exec.c
+++ b/exec.c
@@ -1274,7 +1274,7 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                        last_ram_offset() >> TARGET_PAGE_BITS);
     memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
            0, size >> TARGET_PAGE_BITS);
-    cpu_physical_memory_set_dirty_range(new_block->offset, size, 0xff);
+    cpu_physical_memory_set_dirty_range(new_block->offset, size);

     qemu_ram_setup_dump(new_block->host, size);
     qemu_madvise(new_block->host, size, QEMU_MADV_HUGEPAGE);
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 9f4ad69..681d63b 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -75,12 +75,6 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     return ret;
 }

-static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
-                                                      int dirty_flags)
-{
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
-}
-
 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       int dirty_flag)
 {
@@ -103,15 +97,14 @@ static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
 }

 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
-                                                       ram_addr_t length,
-                                                       int dirty_flags)
+                                                       ram_addr_t length)
 {
     ram_addr_t addr, end;

     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_set_dirty_flags(addr, dirty_flags);
+        cpu_physical_memory_set_dirty(addr);
     }
     xen_modified_memory(addr, length);
 }
diff --git a/memory.c b/memory.c
index e497f99..fb52e1c 100644
--- a/memory.c
+++ b/memory.c
@@ -1182,7 +1182,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
                              hwaddr size)
 {
     assert(mr->terminates);
-    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
+    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size);
 }

 bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 11/38] memory: cpu_physical_memory_mask_dirty_range() always clears a single flag

[Juan Quintela]

Document it

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 cputlb.c                       |  4 ++--
 exec.c                         | 19 ++++++++++---------
 include/exec/memory-internal.h | 40 ++++++++++++++++++----------------------
 include/exec/memory.h          |  3 ---
 memory.c                       | 10 ++++------
 5 files changed, 34 insertions(+), 42 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index 72452e5..dfd747a 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -129,7 +129,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
 {
     cpu_physical_memory_reset_dirty(ram_addr,
                                     ram_addr + TARGET_PAGE_SIZE,
-                                    CODE_DIRTY_FLAG);
+                                    DIRTY_MEMORY_CODE);
 }

 /* update the TLB so that writes in physical page 'phys_addr' are no longer
@@ -137,7 +137,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
 void tlb_unprotect_code_phys(CPUArchState *env, ram_addr_t ram_addr,
                              target_ulong vaddr)
 {
-    cpu_physical_memory_set_dirty_flag(ram_addr, CODE_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_CODE);
 }

 static bool tlb_is_dirty_ram(CPUTLBEntry *tlbe)
diff --git a/exec.c b/exec.c
index 45fdb84..9996da2 100644
--- a/exec.c
+++ b/exec.c
@@ -737,7 +737,7 @@ static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,

 /* Note: start and end must be within the same ram block.  */
 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-                                     int dirty_flags)
+                                     unsigned client)
 {
     uintptr_t length;

@@ -747,7 +747,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
     length = end - start;
     if (length == 0)
         return;
-    cpu_physical_memory_mask_dirty_range(start, length, dirty_flags);
+    cpu_physical_memory_mask_dirty_range(start, length, client);

     if (tcg_enabled()) {
         tlb_reset_dirty_range_all(start, end, length);
@@ -1484,7 +1484,7 @@ found:
 static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
                                uint64_t val, unsigned size)
 {
-    if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
+    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
         tb_invalidate_phys_page_fast(ram_addr, size);
     }
     switch (size) {
@@ -1500,8 +1500,8 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     default:
         abort();
     }
-    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_MIGRATION);
+    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_VGA);
     /* we remove the notdirty callback only if the code has been
        flushed */
     if (cpu_physical_memory_is_dirty(ram_addr)) {
@@ -1911,8 +1911,8 @@ static void invalidate_and_set_dirty(hwaddr addr,
         /* invalidate code */
         tb_invalidate_phys_page_range(addr, addr + length, 0);
         /* set dirty bit */
-        cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
-        cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
     }
     xen_modified_memory(addr, length);
 }
@@ -2494,8 +2494,9 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
                 /* invalidate code */
                 tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
                 /* set dirty bit */
-                cpu_physical_memory_set_dirty_flag(addr1, MIGRATION_DIRTY_FLAG);
-                cpu_physical_memory_set_dirty_flag(addr1, VGA_DIRTY_FLAG);
+                cpu_physical_memory_set_dirty_flag(addr1,
+                                                   DIRTY_MEMORY_MIGRATION);
+                cpu_physical_memory_set_dirty_flag(addr1, DIRTY_MEMORY_VGA);
             }
         }
     }
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 681d63b..b58010f 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -40,29 +40,25 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);

-#define VGA_DIRTY_FLAG       0x01
-#define CODE_DIRTY_FLAG      0x02
-#define MIGRATION_DIRTY_FLAG 0x08
-
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-                                                      int dirty_flag)
+                                                      unsigned client)
 {
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & dirty_flag;
+    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
 }

 /* read dirty bit (return 0 or 1) */
 static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
 {
-    bool vga = cpu_physical_memory_get_dirty_flag(addr, VGA_DIRTY_FLAG);
-    bool code = cpu_physical_memory_get_dirty_flag(addr, CODE_DIRTY_FLAG);
+    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
+    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
     bool migration =
-        cpu_physical_memory_get_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
+        cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
     return vga && code && migration;
 }

 static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
                                                 ram_addr_t length,
-                                                int dirty_flag)
+                                                unsigned client)
 {
     int ret = 0;
     ram_addr_t addr, end;
@@ -70,28 +66,28 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= cpu_physical_memory_get_dirty_flag(addr, dirty_flag);
+        ret |= cpu_physical_memory_get_dirty_flag(addr, client);
     }
     return ret;
 }

 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
-                                                      int dirty_flag)
+                                                      unsigned client)
 {
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flag;
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
 }

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 {
-    cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
-    cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
+    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
+    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
+    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
 }

-static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
-                                                        int dirty_flags)
+static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
+                                                       unsigned client)
 {
-    int mask = ~dirty_flags;
+    int mask = ~(1 << client);

     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }
@@ -111,19 +107,19 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,

 static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
                                                         ram_addr_t length,
-                                                        int dirty_flags)
+                                                        unsigned client)
 {
     ram_addr_t addr, end;

     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_clear_dirty_flags(addr, dirty_flags);
+        cpu_physical_memory_clear_dirty_flag(addr, client);
     }
 }

 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-                                     int dirty_flags);
+                                     unsigned client);

 #endif

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 480dfbf..b8e76f4 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -33,9 +33,6 @@
 typedef struct MemoryRegionOps MemoryRegionOps;
 typedef struct MemoryRegionMmio MemoryRegionMmio;

-/* Must match *_DIRTY_FLAGS in cpu-all.h.  To be replaced with dynamic
- * registration.
- */
 #define DIRTY_MEMORY_VGA       0
 #define DIRTY_MEMORY_CODE      1
 #define DIRTY_MEMORY_MIGRATION 3
diff --git a/memory.c b/memory.c
index fb52e1c..a490cbd 100644
--- a/memory.c
+++ b/memory.c
@@ -1174,8 +1174,7 @@ bool memory_region_get_dirty(MemoryRegion *mr, hwaddr addr,
                              hwaddr size, unsigned client)
 {
     assert(mr->terminates);
-    return cpu_physical_memory_get_dirty(mr->ram_addr + addr, size,
-                                         1 << client);
+    return cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
 }

 void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
@@ -1190,12 +1189,11 @@ bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
 {
     bool ret;
     assert(mr->terminates);
-    ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size,
-                                        1 << client);
+    ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
     if (ret) {
         cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
                                         mr->ram_addr + addr + size,
-                                        1 << client);
+                                        client);
     }
     return ret;
 }
@@ -1243,7 +1241,7 @@ void memory_region_reset_dirty(MemoryRegion *mr, hwaddr addr,
     assert(mr->terminates);
     cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
                                     mr->ram_addr + addr + size,
-                                    1 << client);
+                                    client);
 }

 void *memory_region_get_ram_ptr(MemoryRegion *mr)
-- 
1.8.3.1

[Qemu-devel] [PATCH 12/38] memory: use bit 2 for migration

[Juan Quintela]

For historical reasons it was bit 3.  Once there, create a constant to
know the number of clients.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index b8e76f4..d5e9d58 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -35,7 +35,8 @@ typedef struct MemoryRegionMmio MemoryRegionMmio;

 #define DIRTY_MEMORY_VGA       0
 #define DIRTY_MEMORY_CODE      1
-#define DIRTY_MEMORY_MIGRATION 3
+#define DIRTY_MEMORY_MIGRATION 2
+#define DIRTY_MEMORY_NUM       3        /* num of dirty bits */

 struct MemoryRegionMmio {
     CPUReadMemoryFunc *read[3];
-- 
1.8.3.1

[Qemu-devel] [PATCH 13/38] memory: make sure that client is always inside range

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index b58010f..d09d6d8 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -43,6 +43,7 @@ void qemu_ram_free_from_ptr(ram_addr_t addr);
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
+    assert(client < DIRTY_MEMORY_NUM);
     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
 }

@@ -74,6 +75,7 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
+    assert(client < DIRTY_MEMORY_NUM);
     ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
 }

@@ -89,6 +91,8 @@ static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
 {
     int mask = ~(1 << client);

+    assert(client < DIRTY_MEMORY_NUM);
+
     return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }

-- 
1.8.3.1

[Qemu-devel] [PATCH 14/38] memory: only resize dirty bitmap when memory size increases

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/exec.c b/exec.c
index 9996da2..bed5c07 100644
--- a/exec.c
+++ b/exec.c
@@ -1210,6 +1210,9 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
                                    MemoryRegion *mr)
 {
     RAMBlock *block, *new_block;
+    ram_addr_t old_ram_size, new_ram_size;
+
+    old_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;

     size = TARGET_PAGE_ALIGN(size);
     new_block = g_malloc0(sizeof(*new_block));
@@ -1270,10 +1273,13 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     ram_list.version++;
     qemu_mutex_unlock_ramlist();

-    ram_list.phys_dirty = g_realloc(ram_list.phys_dirty,
-                                       last_ram_offset() >> TARGET_PAGE_BITS);
-    memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
+    new_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;
+
+    if (new_ram_size > old_ram_size) {
+        ram_list.phys_dirty = g_realloc(ram_list.phys_dirty, new_ram_size);
+        memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
            0, size >> TARGET_PAGE_BITS);
+    }
     cpu_physical_memory_set_dirty_range(new_block->offset, size);

     qemu_ram_setup_dump(new_block->host, size);
-- 
1.8.3.1

[Qemu-devel] [PATCH 15/38] memory: cpu_physical_memory_clear_dirty_flag() result is never used

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index d09d6d8..666490c 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -86,14 +86,14 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
     cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
 }

-static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
+static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
                                                        unsigned client)
 {
     int mask = ~(1 << client);

     assert(client < DIRTY_MEMORY_NUM);

-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
+    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
 }

 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 16/38] bitmap: Add bitmap_zero_extend operation

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/qemu/bitmap.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
index afdd257..1babd5d 100644
--- a/include/qemu/bitmap.h
+++ b/include/qemu/bitmap.h
@@ -220,4 +220,13 @@ unsigned long bitmap_find_next_zero_area(unsigned long *map,
                                          unsigned long nr,
                                          unsigned long align_mask);

+static inline unsigned long *bitmap_zero_extend(unsigned long *old,
+                                                long old_nbits, long new_nbits)
+{
+    long new_len = BITS_TO_LONGS(new_nbits) * sizeof(unsigned long);
+    unsigned long *new = g_realloc(old, new_len);
+    bitmap_clear(new, old_nbits, new_nbits - old_nbits);
+    return new;
+}
+
 #endif /* BITMAP_H */
-- 
1.8.3.1

[Qemu-devel] [PATCH 17/38] memory: split dirty bitmap into three

[Juan Quintela]

After all the previous patches, spliting the bitmap gets direct.

Note: For some reason, I have to move DIRTY_MEMORY_* definitions to
the beginning of memory.h to make compilation work.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c                         |  9 ++++++---
 include/exec/cpu-all.h         |  3 ++-
 include/exec/memory-internal.h |  9 +++------
 include/exec/memory.h          | 10 +++++-----
 4 files changed, 16 insertions(+), 15 deletions(-)

diff --git a/exec.c b/exec.c
index bed5c07..ad9866c 100644
--- a/exec.c
+++ b/exec.c
@@ -1276,9 +1276,12 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
     new_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;

     if (new_ram_size > old_ram_size) {
-        ram_list.phys_dirty = g_realloc(ram_list.phys_dirty, new_ram_size);
-        memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
-           0, size >> TARGET_PAGE_BITS);
+        int i;
+        for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
+            ram_list.dirty_memory[i] =
+                bitmap_zero_extend(ram_list.dirty_memory[i],
+                                   old_ram_size, new_ram_size);
+       }
     }
     cpu_physical_memory_set_dirty_range(new_block->offset, size);

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index b6998f0..4cb4b4a 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -21,6 +21,7 @@

 #include "qemu-common.h"
 #include "exec/cpu-common.h"
+#include "exec/memory.h"
 #include "qemu/thread.h"
 #include "qom/cpu.h"

@@ -459,7 +460,7 @@ typedef struct RAMBlock {
 typedef struct RAMList {
     QemuMutex mutex;
     /* Protected by the iothread lock.  */
-    uint8_t *phys_dirty;
+    unsigned long *dirty_memory[DIRTY_MEMORY_NUM];
     RAMBlock *mru_block;
     /* Protected by the ramlist lock.  */
     QTAILQ_HEAD(, RAMBlock) blocks;
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 666490c..6fb1b64 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -44,7 +44,7 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
     assert(client < DIRTY_MEMORY_NUM);
-    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
+    return test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

 /* read dirty bit (return 0 or 1) */
@@ -76,7 +76,7 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
     assert(client < DIRTY_MEMORY_NUM);
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
+    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

 static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
@@ -89,11 +89,8 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
 static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
                                                        unsigned client)
 {
-    int mask = ~(1 << client);
-
     assert(client < DIRTY_MEMORY_NUM);
-
-    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
+    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
diff --git a/include/exec/memory.h b/include/exec/memory.h
index d5e9d58..296d6ab 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -16,6 +16,11 @@

 #ifndef CONFIG_USER_ONLY

+#define DIRTY_MEMORY_VGA       0
+#define DIRTY_MEMORY_CODE      1
+#define DIRTY_MEMORY_MIGRATION 2
+#define DIRTY_MEMORY_NUM       3        /* num of dirty bits */
+
 #include <stdint.h>
 #include <stdbool.h>
 #include "qemu-common.h"
@@ -33,11 +38,6 @@
 typedef struct MemoryRegionOps MemoryRegionOps;
 typedef struct MemoryRegionMmio MemoryRegionMmio;

-#define DIRTY_MEMORY_VGA       0
-#define DIRTY_MEMORY_CODE      1
-#define DIRTY_MEMORY_MIGRATION 2
-#define DIRTY_MEMORY_NUM       3        /* num of dirty bits */
-
 struct MemoryRegionMmio {
     CPUReadMemoryFunc *read[3];
     CPUWriteMemoryFunc *write[3];
-- 
1.8.3.1

[Qemu-devel] [PATCH 18/38] memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 6fb1b64..2f6610a 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -86,13 +86,6 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
     cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
 }

-static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
-                                                       unsigned client)
-{
-    assert(client < DIRTY_MEMORY_NUM);
-    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
-}
-
 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
                                                        ram_addr_t length)
 {
@@ -112,10 +105,11 @@ static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
 {
     ram_addr_t addr, end;

+    assert(client < DIRTY_MEMORY_NUM);
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_clear_dirty_flag(addr, client);
+        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
     }
 }

-- 
1.8.3.1

[Qemu-devel] [PATCH 19/38] memory: unfold cpu_physical_memory_set_dirty() in its only user

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 2f6610a..9d34434 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -79,13 +79,6 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
     set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
 }

-static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
-{
-    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
-    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
-}
-
 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
                                                        ram_addr_t length)
 {
@@ -94,7 +87,9 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_set_dirty(addr);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
+        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
     }
     xen_modified_memory(addr, length);
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 20/38] memory: unfold cpu_physical_memory_set_dirty_flag()

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 9d34434..b99617a 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -87,9 +87,12 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
-        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
+        set_bit(addr >> TARGET_PAGE_BITS,
+                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
+        set_bit(addr >> TARGET_PAGE_BITS,
+                ram_list.dirty_memory[DIRTY_MEMORY_VGA]);
+        set_bit(addr >> TARGET_PAGE_BITS,
+                ram_list.dirty_memory[DIRTY_MEMORY_CODE]);
     }
     xen_modified_memory(addr, length);
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 21/38] memory: make cpu_physical_memory_get_dirty() the main function

[Juan Quintela]

And make cpu_physical_memory_get_dirty_flag() to use it.  It used to
be the other way around.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 36 +++++++++++++++++++-----------------
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index b99617a..edca8a8 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -40,11 +40,28 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);

+static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
+                                                ram_addr_t length,
+                                                unsigned client)
+{
+    int ret = 0;
+    ram_addr_t addr, end;
+
+    assert(client < DIRTY_MEMORY_NUM);
+
+    end = TARGET_PAGE_ALIGN(start + length);
+    start &= TARGET_PAGE_MASK;
+    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
+        ret |= test_bit(addr >> TARGET_PAGE_BITS,
+                        ram_list.dirty_memory[client]);
+    }
+    return ret;
+}
+
 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
-    assert(client < DIRTY_MEMORY_NUM);
-    return test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
+    return cpu_physical_memory_get_dirty(addr, 1, client);
 }

 /* read dirty bit (return 0 or 1) */
@@ -57,21 +74,6 @@ static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
     return vga && code && migration;
 }

-static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
-                                                ram_addr_t length,
-                                                unsigned client)
-{
-    int ret = 0;
-    ram_addr_t addr, end;
-
-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= cpu_physical_memory_get_dirty_flag(addr, client);
-    }
-    return ret;
-}
-
 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
                                                       unsigned client)
 {
-- 
1.8.3.1

[Qemu-devel] [PATCH 22/38] memory: cpu_physical_memory_get_dirty() is used as returning a bool

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index edca8a8..fa28fc6 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -40,11 +40,10 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
 void qemu_ram_free(ram_addr_t addr);
 void qemu_ram_free_from_ptr(ram_addr_t addr);

-static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
-                                                ram_addr_t length,
-                                                unsigned client)
+static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
+                                                 ram_addr_t length,
+                                                 unsigned client)
 {
-    int ret = 0;
     ram_addr_t addr, end;

     assert(client < DIRTY_MEMORY_NUM);
@@ -52,10 +51,12 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
     end = TARGET_PAGE_ALIGN(start + length);
     start &= TARGET_PAGE_MASK;
     for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        ret |= test_bit(addr >> TARGET_PAGE_BITS,
-                        ram_list.dirty_memory[client]);
+        if (test_bit(addr >> TARGET_PAGE_BITS,
+                     ram_list.dirty_memory[client])) {
+            return true;
+        }
     }
-    return ret;
+    return false;
 }

 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 23/38] memory: s/mask/clear/ cpu_physical_memory_mask_dirty_range

[Juan Quintela]

Now all functions use the same wording that bitops/bitmap operations

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c                         | 2 +-
 include/exec/memory-internal.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/exec.c b/exec.c
index ad9866c..6bef3e5 100644
--- a/exec.c
+++ b/exec.c
@@ -747,7 +747,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
     length = end - start;
     if (length == 0)
         return;
-    cpu_physical_memory_mask_dirty_range(start, length, client);
+    cpu_physical_memory_clear_dirty_range(start, length, client);

     if (tcg_enabled()) {
         tlb_reset_dirty_range_all(start, end, length);
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index fa28fc6..c04a92a 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -100,9 +100,9 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     xen_modified_memory(addr, length);
 }

-static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
-                                                        ram_addr_t length,
-                                                        unsigned client)
+static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
+                                                         ram_addr_t length,
+                                                         unsigned client)
 {
     ram_addr_t addr, end;

-- 
1.8.3.1

[Qemu-devel] [PATCH 24/38] memory: use find_next_bit() to find dirty bits

[Juan Quintela]

This operation is way faster than doing it bit by bit.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index c04a92a..b017c2e 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -44,19 +44,15 @@ static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
                                                  ram_addr_t length,
                                                  unsigned client)
 {
-    ram_addr_t addr, end;
+    unsigned long end, page, next;

     assert(client < DIRTY_MEMORY_NUM);

-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        if (test_bit(addr >> TARGET_PAGE_BITS,
-                     ram_list.dirty_memory[client])) {
-            return true;
-        }
-    }
-    return false;
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    next = find_next_bit(ram_list.dirty_memory[client], end, page);
+
+    return next < end;
 }

 static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 25/38] memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations

[Juan Quintela]

We were setting a range of bits, so use bitmap_set().

Note: xen has always been wrong, and should have used start instead
of addr from the beginning.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index b017c2e..4906cdf 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -81,19 +81,14 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
 static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
                                                        ram_addr_t length)
 {
-    ram_addr_t addr, end;
+    unsigned long end, page;

-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        set_bit(addr >> TARGET_PAGE_BITS,
-                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
-        set_bit(addr >> TARGET_PAGE_BITS,
-                ram_list.dirty_memory[DIRTY_MEMORY_VGA]);
-        set_bit(addr >> TARGET_PAGE_BITS,
-                ram_list.dirty_memory[DIRTY_MEMORY_CODE]);
-    }
-    xen_modified_memory(addr, length);
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
+    xen_modified_memory(start, length);
 }

 static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
-- 
1.8.3.1

[Qemu-devel] [PATCH 26/38] memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations

[Juan Quintela]

We were clearing a range of bits, so use bitmap_clear().

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 include/exec/memory-internal.h | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 4906cdf..e2f55ea 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -95,14 +95,12 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
                                                          ram_addr_t length,
                                                          unsigned client)
 {
-    ram_addr_t addr, end;
+    unsigned long end, page;

     assert(client < DIRTY_MEMORY_NUM);
-    end = TARGET_PAGE_ALIGN(start + length);
-    start &= TARGET_PAGE_MASK;
-    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
-        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
-    }
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
 }

 void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
-- 
1.8.3.1

[Qemu-devel] [PATCH 27/38] memory: s/dirty/clean/ in cpu_physical_memory_is_dirty()

[Juan Quintela]

All uses except one really want the other meaning.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 cputlb.c                       | 3 ++-
 exec.c                         | 6 +++---
 include/exec/memory-internal.h | 5 ++---
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index dfd747a..911d764 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -299,7 +299,8 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
             /* Write access calls the I/O callback.  */
             te->addr_write = address | TLB_MMIO;
         } else if (memory_region_is_ram(section->mr)
-                   && !cpu_physical_memory_is_dirty(section->mr->ram_addr + xlat)) {
+                   && cpu_physical_memory_is_clean(section->mr->ram_addr
+                                                   + xlat)) {
             te->addr_write = address | TLB_NOTDIRTY;
         } else {
             te->addr_write = address;
diff --git a/exec.c b/exec.c
index 6bef3e5..a2f89eb 100644
--- a/exec.c
+++ b/exec.c
@@ -1513,7 +1513,7 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
     cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_VGA);
     /* we remove the notdirty callback only if the code has been
        flushed */
-    if (cpu_physical_memory_is_dirty(ram_addr)) {
+    if (!cpu_physical_memory_is_clean(ram_addr)) {
         CPUArchState *env = current_cpu->env_ptr;
         tlb_set_dirty(env, env->mem_io_vaddr);
     }
@@ -1916,7 +1916,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
 static void invalidate_and_set_dirty(hwaddr addr,
                                      hwaddr length)
 {
-    if (!cpu_physical_memory_is_dirty(addr)) {
+    if (cpu_physical_memory_is_clean(addr)) {
         /* invalidate code */
         tb_invalidate_phys_page_range(addr, addr + length, 0);
         /* set dirty bit */
@@ -2499,7 +2499,7 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
         stl_p(ptr, val);

         if (unlikely(in_migration)) {
-            if (!cpu_physical_memory_is_dirty(addr1)) {
+            if (cpu_physical_memory_is_clean(addr1)) {
                 /* invalidate code */
                 tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
                 /* set dirty bit */
diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index e2f55ea..771b23f 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -61,14 +61,13 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
     return cpu_physical_memory_get_dirty(addr, 1, client);
 }

-/* read dirty bit (return 0 or 1) */
-static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
+static inline bool cpu_physical_memory_is_clean(ram_addr_t addr)
 {
     bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
     bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
     bool migration =
         cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-    return vga && code && migration;
+    return !(vga && code && migration);
 }

 static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
-- 
1.8.3.1

[Qemu-devel] [PATCH 28/38] memory: make cpu_physical_memory_reset_dirty() take a length parameter

[Juan Quintela]

We have an end parameter in all the callers, and this make it coherent
with the rest of cpu_physical_memory_* functions, that also take a
length parameter.

Once here, move the start/end calculation to
tlb_reset_dirty_range_all() as we don't need it here anymore.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 cputlb.c                       |  3 +--
 exec.c                         | 19 ++++++++-----------
 include/exec/memory-internal.h |  2 +-
 memory.c                       |  8 ++------
 4 files changed, 12 insertions(+), 20 deletions(-)

diff --git a/cputlb.c b/cputlb.c
index 911d764..a5805e1 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -127,8 +127,7 @@ void tlb_flush_page(CPUArchState *env, target_ulong addr)
    can be detected */
 void tlb_protect_code(ram_addr_t ram_addr)
 {
-    cpu_physical_memory_reset_dirty(ram_addr,
-                                    ram_addr + TARGET_PAGE_SIZE,
+    cpu_physical_memory_reset_dirty(ram_addr, TARGET_PAGE_SIZE,
                                     DIRTY_MEMORY_CODE);
 }

diff --git a/exec.c b/exec.c
index a2f89eb..ba5989c 100644
--- a/exec.c
+++ b/exec.c
@@ -723,11 +723,14 @@ found:
     return block;
 }

-static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,
-                                      uintptr_t length)
+static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t length)
 {
-    RAMBlock *block;
     ram_addr_t start1;
+    RAMBlock *block;
+    ram_addr_t end;
+
+    end = TARGET_PAGE_ALIGN(start + length);
+    start &= TARGET_PAGE_MASK;

     block = qemu_get_ram_block(start);
     assert(block == qemu_get_ram_block(end - 1));
@@ -736,21 +739,15 @@ static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,
 }

 /* Note: start and end must be within the same ram block.  */
-void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
+void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
                                      unsigned client)
 {
-    uintptr_t length;
-
-    start &= TARGET_PAGE_MASK;
-    end = TARGET_PAGE_ALIGN(end);
-
-    length = end - start;
     if (length == 0)
         return;
     cpu_physical_memory_clear_dirty_range(start, length, client);

     if (tcg_enabled()) {
-        tlb_reset_dirty_range_all(start, end, length);
+        tlb_reset_dirty_range_all(start, length);
     }
 }

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index 771b23f..cb2249f 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -102,7 +102,7 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
     bitmap_clear(ram_list.dirty_memory[client], page, end - page);
 }

-void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
+void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
                                      unsigned client);

 #endif
diff --git a/memory.c b/memory.c
index a490cbd..c010296 100644
--- a/memory.c
+++ b/memory.c
@@ -1191,9 +1191,7 @@ bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
     assert(mr->terminates);
     ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
     if (ret) {
-        cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
-                                        mr->ram_addr + addr + size,
-                                        client);
+        cpu_physical_memory_reset_dirty(mr->ram_addr + addr, size, client);
     }
     return ret;
 }
@@ -1239,9 +1237,7 @@ void memory_region_reset_dirty(MemoryRegion *mr, hwaddr addr,
                                hwaddr size, unsigned client)
 {
     assert(mr->terminates);
-    cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
-                                    mr->ram_addr + addr + size,
-                                    client);
+    cpu_physical_memory_reset_dirty(mr->ram_addr + addr, size, client);
 }

 void *memory_region_get_ram_ptr(MemoryRegion *mr)
-- 
1.8.3.1

[Qemu-devel] [PATCH 29/38] memory: cpu_physical_memory_set_dirty_tracking() should return void

[Juan Quintela]

Result was always 0, and not used anywhere.  Once there, use bool type
for the parameter.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 exec.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index ba5989c..d165fd8 100644
--- a/exec.c
+++ b/exec.c
@@ -56,7 +56,7 @@
 //#define DEBUG_SUBPAGE

 #if !defined(CONFIG_USER_ONLY)
-static int in_migration;
+static bool in_migration;

 RAMList ram_list = { .blocks = QTAILQ_HEAD_INITIALIZER(ram_list.blocks) };

@@ -751,11 +751,9 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
     }
 }

-static int cpu_physical_memory_set_dirty_tracking(int enable)
+static void cpu_physical_memory_set_dirty_tracking(bool enable)
 {
-    int ret = 0;
     in_migration = enable;
-    return ret;
 }

 hwaddr memory_region_section_get_iotlb(CPUArchState *env,
@@ -1797,12 +1795,12 @@ static void tcg_commit(MemoryListener *listener)

 static void core_log_global_start(MemoryListener *listener)
 {
-    cpu_physical_memory_set_dirty_tracking(1);
+    cpu_physical_memory_set_dirty_tracking(true);
 }

 static void core_log_global_stop(MemoryListener *listener)
 {
-    cpu_physical_memory_set_dirty_tracking(0);
+    cpu_physical_memory_set_dirty_tracking(false);
 }

 static MemoryListener core_memory_listener = {
-- 
1.8.3.1

[Qemu-devel] [PATCH 30/38] memory: split cpu_physical_memory_* functions to its own include

[Juan Quintela]

All the functions that use ram_addr_t should be here.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 cputlb.c                       |  1 +
 exec.c                         |  1 +
 include/exec/memory-internal.h | 76 --------------------------------
 include/exec/ram_addr.h        | 98 ++++++++++++++++++++++++++++++++++++++++++
 memory.c                       |  1 +
 5 files changed, 101 insertions(+), 76 deletions(-)
 create mode 100644 include/exec/ram_addr.h

diff --git a/cputlb.c b/cputlb.c
index a5805e1..a1e8421 100644
--- a/cputlb.c
+++ b/cputlb.c
@@ -26,6 +26,7 @@
 #include "exec/cputlb.h"

 #include "exec/memory-internal.h"
+#include "exec/ram_addr.h"

 //#define DEBUG_TLB
 //#define DEBUG_TLB_CHECK
diff --git a/exec.c b/exec.c
index d165fd8..be421e5 100644
--- a/exec.c
+++ b/exec.c
@@ -50,6 +50,7 @@
 #include "translate-all.h"

 #include "exec/memory-internal.h"
+#include "exec/ram_addr.h"

 #include "qemu/range.h"

diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
index cb2249f..25c43c0 100644
--- a/include/exec/memory-internal.h
+++ b/include/exec/memory-internal.h
@@ -20,9 +20,6 @@
 #define MEMORY_INTERNAL_H

 #ifndef CONFIG_USER_ONLY
-#include "hw/xen/xen.h"
-
-
 typedef struct AddressSpaceDispatch AddressSpaceDispatch;

 void address_space_init_dispatch(AddressSpace *as);
@@ -33,78 +30,5 @@ extern const MemoryRegionOps unassigned_mem_ops;
 bool memory_region_access_valid(MemoryRegion *mr, hwaddr addr,
                                 unsigned size, bool is_write);

-ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
-                                   MemoryRegion *mr);
-ram_addr_t qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr);
-void *qemu_get_ram_ptr(ram_addr_t addr);
-void qemu_ram_free(ram_addr_t addr);
-void qemu_ram_free_from_ptr(ram_addr_t addr);
-
-static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
-                                                 ram_addr_t length,
-                                                 unsigned client)
-{
-    unsigned long end, page, next;
-
-    assert(client < DIRTY_MEMORY_NUM);
-
-    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
-    page = start >> TARGET_PAGE_BITS;
-    next = find_next_bit(ram_list.dirty_memory[client], end, page);
-
-    return next < end;
-}
-
-static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
-                                                      unsigned client)
-{
-    return cpu_physical_memory_get_dirty(addr, 1, client);
-}
-
-static inline bool cpu_physical_memory_is_clean(ram_addr_t addr)
-{
-    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
-    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
-    bool migration =
-        cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
-    return !(vga && code && migration);
-}
-
-static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
-                                                      unsigned client)
-{
-    assert(client < DIRTY_MEMORY_NUM);
-    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
-}
-
-static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
-                                                       ram_addr_t length)
-{
-    unsigned long end, page;
-
-    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
-    page = start >> TARGET_PAGE_BITS;
-    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
-    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
-    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
-    xen_modified_memory(start, length);
-}
-
-static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
-                                                         ram_addr_t length,
-                                                         unsigned client)
-{
-    unsigned long end, page;
-
-    assert(client < DIRTY_MEMORY_NUM);
-    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
-    page = start >> TARGET_PAGE_BITS;
-    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
-}
-
-void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
-                                     unsigned client);
-
 #endif
-
 #endif
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
new file mode 100644
index 0000000..db977fb
--- /dev/null
+++ b/include/exec/ram_addr.h
@@ -0,0 +1,98 @@
+/*
+ * Declarations for cpu physical memory functions
+ *
+ * Copyright 2011 Red Hat, Inc. and/or its affiliates
+ *
+ * Authors:
+ *  Avi Kivity <avi@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ *
+ */
+
+/*
+ * This header is for use by exec.c and memory.c ONLY.  Do not include it.
+ * The functions declared here will be removed soon.
+ */
+
+#ifndef RAM_ADDR_H
+#define RAM_ADDR_H
+
+#ifndef CONFIG_USER_ONLY
+#include "hw/xen/xen.h"
+
+ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
+                                   MemoryRegion *mr);
+ram_addr_t qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr);
+void *qemu_get_ram_ptr(ram_addr_t addr);
+void qemu_ram_free(ram_addr_t addr);
+void qemu_ram_free_from_ptr(ram_addr_t addr);
+
+static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
+                                                 ram_addr_t length,
+                                                 unsigned client)
+{
+    unsigned long end, page, next;
+
+    assert(client < DIRTY_MEMORY_NUM);
+
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    next = find_next_bit(ram_list.dirty_memory[client], end, page);
+
+    return next < end;
+}
+
+static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
+                                                      unsigned client)
+{
+    return cpu_physical_memory_get_dirty(addr, 1, client);
+}
+
+static inline bool cpu_physical_memory_is_clean(ram_addr_t addr)
+{
+    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
+    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
+    bool migration =
+        cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
+    return !(vga && code && migration);
+}
+
+static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
+                                                      unsigned client)
+{
+    assert(client < DIRTY_MEMORY_NUM);
+    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
+}
+
+static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
+                                                       ram_addr_t length)
+{
+    unsigned long end, page;
+
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
+    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
+    xen_modified_memory(start, length);
+}
+
+static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
+                                                         ram_addr_t length,
+                                                         unsigned client)
+{
+    unsigned long end, page;
+
+    assert(client < DIRTY_MEMORY_NUM);
+    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
+    page = start >> TARGET_PAGE_BITS;
+    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
+}
+
+void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
+                                     unsigned client);
+
+#endif
+#endif
diff --git a/memory.c b/memory.c
index c010296..59ecc28 100644
--- a/memory.c
+++ b/memory.c
@@ -22,6 +22,7 @@
 #include <assert.h>

 #include "exec/memory-internal.h"
+#include "exec/ram_addr.h"

 //#define DEBUG_UNASSIGNED

-- 
1.8.3.1

[Qemu-devel] [PATCH 31/38] memory: unfold memory_region_test_and_clear()

[Juan Quintela]

We are going to update the bitmap directly

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 arch_init.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index e0acbc5..0e8c8b5 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -48,6 +48,7 @@
 #include "qmp-commands.h"
 #include "trace.h"
 #include "exec/cpu-all.h"
+#include "exec/ram_addr.h"
 #include "hw/acpi/acpi.h"

 #ifdef DEBUG_ARCH_INIT
@@ -400,9 +401,12 @@ static void migration_bitmap_sync(void)

     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
         for (addr = 0; addr < block->length; addr += TARGET_PAGE_SIZE) {
-            if (memory_region_test_and_clear_dirty(block->mr,
-                                                   addr, TARGET_PAGE_SIZE,
-                                                   DIRTY_MEMORY_MIGRATION)) {
+            if (cpu_physical_memory_get_dirty(block->mr->ram_addr + addr,
+                                              TARGET_PAGE_SIZE,
+                                              DIRTY_MEMORY_MIGRATION)) {
+                cpu_physical_memory_reset_dirty(block->mr->ram_addr + addr,
+                                                TARGET_PAGE_SIZE,
+                                                DIRTY_MEMORY_MIGRATION);
                 migration_bitmap_set_dirty(block->mr, addr);
             }
         }
-- 
1.8.3.1

[Qemu-devel] [PATCH 32/38] kvm: use directly cpu_physical_memory_* api for tracking dirty pages

[Juan Quintela]

Performance is important in this function, and we want to optimize even further.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 kvm-all.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 3937754..308dfba 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -31,6 +31,7 @@
 #include "sysemu/kvm.h"
 #include "qemu/bswap.h"
 #include "exec/memory.h"
+#include "exec/ram_addr.h"
 #include "exec/address-spaces.h"
 #include "qemu/event_notifier.h"
 #include "trace.h"
@@ -382,6 +383,7 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
     unsigned int i, j;
     unsigned long page_number, c;
     hwaddr addr, addr1;
+    ram_addr_t ram_addr;
     unsigned int pages = int128_get64(section->size) / getpagesize();
     unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
     unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
@@ -399,8 +401,9 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
                 page_number = (i * HOST_LONG_BITS + j) * hpratio;
                 addr1 = page_number * TARGET_PAGE_SIZE;
                 addr = section->offset_within_region + addr1;
-                memory_region_set_dirty(section->mr, addr,
-                                        TARGET_PAGE_SIZE * hpratio);
+                ram_addr = section->mr->ram_addr + addr;
+                cpu_physical_memory_set_dirty_range(ram_addr,
+                                                    TARGET_PAGE_SIZE * hpratio);
             } while (c != 0);
         }
     }
-- 
1.8.3.1

[Qemu-devel] [PATCH 33/38] kvm: refactor start address calculation

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 kvm-all.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 308dfba..cb62ba4 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -382,7 +382,8 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
 {
     unsigned int i, j;
     unsigned long page_number, c;
-    hwaddr addr, addr1;
+    hwaddr addr;
+    ram_addr_t start = section->offset_within_region + section->mr->ram_addr;
     ram_addr_t ram_addr;
     unsigned int pages = int128_get64(section->size) / getpagesize();
     unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
@@ -399,9 +400,8 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
                 j = ffsl(c) - 1;
                 c &= ~(1ul << j);
                 page_number = (i * HOST_LONG_BITS + j) * hpratio;
-                addr1 = page_number * TARGET_PAGE_SIZE;
-                addr = section->offset_within_region + addr1;
-                ram_addr = section->mr->ram_addr + addr;
+                addr = page_number * TARGET_PAGE_SIZE;
+                ram_addr = start + addr;
                 cpu_physical_memory_set_dirty_range(ram_addr,
                                                     TARGET_PAGE_SIZE * hpratio);
             } while (c != 0);
-- 
1.8.3.1

[Qemu-devel] [PATCH 34/38] memory: move bitmap synchronization to its own function

[Juan Quintela]

We want to have all the functions that handle directly the dirty
bitmap near.  We will change it later.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/ram_addr.h | 31 +++++++++++++++++++++++++++++++
 kvm-all.c               | 27 ++-------------------------
 2 files changed, 33 insertions(+), 25 deletions(-)

diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index db977fb..c6736ed 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -79,6 +79,37 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
     xen_modified_memory(start, length);
 }

+static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
+                                                          ram_addr_t start,
+                                                          ram_addr_t pages)
+{
+    unsigned int i, j;
+    unsigned long page_number, c;
+    hwaddr addr;
+    ram_addr_t ram_addr;
+    unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
+    unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
+
+    /*
+     * bitmap-traveling is faster than memory-traveling (for addr...)
+     * especially when most of the memory is not dirty.
+     */
+    for (i = 0; i < len; i++) {
+        if (bitmap[i] != 0) {
+            c = leul_to_cpu(bitmap[i]);
+            do {
+                j = ffsl(c) - 1;
+                c &= ~(1ul << j);
+                page_number = (i * HOST_LONG_BITS + j) * hpratio;
+                addr = page_number * TARGET_PAGE_SIZE;
+                ram_addr = start + addr;
+                cpu_physical_memory_set_dirty_range(ram_addr,
+                                                    TARGET_PAGE_SIZE * hpratio);
+            } while (c != 0);
+        }
+    }
+}
+
 static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
                                                          ram_addr_t length,
                                                          unsigned client)
diff --git a/kvm-all.c b/kvm-all.c
index cb62ba4..0bfb060 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -380,33 +380,10 @@ static int kvm_set_migration_log(int enable)
 static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
                                          unsigned long *bitmap)
 {
-    unsigned int i, j;
-    unsigned long page_number, c;
-    hwaddr addr;
     ram_addr_t start = section->offset_within_region + section->mr->ram_addr;
-    ram_addr_t ram_addr;
-    unsigned int pages = int128_get64(section->size) / getpagesize();
-    unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
-    unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
+    ram_addr_t pages = int128_get64(section->size) / getpagesize();

-    /*
-     * bitmap-traveling is faster than memory-traveling (for addr...)
-     * especially when most of the memory is not dirty.
-     */
-    for (i = 0; i < len; i++) {
-        if (bitmap[i] != 0) {
-            c = leul_to_cpu(bitmap[i]);
-            do {
-                j = ffsl(c) - 1;
-                c &= ~(1ul << j);
-                page_number = (i * HOST_LONG_BITS + j) * hpratio;
-                addr = page_number * TARGET_PAGE_SIZE;
-                ram_addr = start + addr;
-                cpu_physical_memory_set_dirty_range(ram_addr,
-                                                    TARGET_PAGE_SIZE * hpratio);
-            } while (c != 0);
-        }
-    }
+    cpu_physical_memory_set_dirty_lebitmap(bitmap, start, pages);
     return 0;
 }

-- 
1.8.3.1

[Qemu-devel] [PATCH 35/38] memory: syncronize kvm bitmap using bitmaps operations

[Juan Quintela]

If bitmaps are aligned properly, use bitmap operations.  If they are
not, just use old bit at a time code.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 include/exec/ram_addr.h | 54 ++++++++++++++++++++++++++++++++-----------------
 1 file changed, 36 insertions(+), 18 deletions(-)

diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index c6736ed..33c8acc 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -83,29 +83,47 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
                                                           ram_addr_t start,
                                                           ram_addr_t pages)
 {
-    unsigned int i, j;
+    unsigned long i, j;
     unsigned long page_number, c;
     hwaddr addr;
     ram_addr_t ram_addr;
-    unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
+    unsigned long len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
     unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
+    unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);

-    /*
-     * bitmap-traveling is faster than memory-traveling (for addr...)
-     * especially when most of the memory is not dirty.
-     */
-    for (i = 0; i < len; i++) {
-        if (bitmap[i] != 0) {
-            c = leul_to_cpu(bitmap[i]);
-            do {
-                j = ffsl(c) - 1;
-                c &= ~(1ul << j);
-                page_number = (i * HOST_LONG_BITS + j) * hpratio;
-                addr = page_number * TARGET_PAGE_SIZE;
-                ram_addr = start + addr;
-                cpu_physical_memory_set_dirty_range(ram_addr,
-                                                    TARGET_PAGE_SIZE * hpratio);
-            } while (c != 0);
+    /* start address is aligned at the start of a word? */
+    if (((page * BITS_PER_LONG) << TARGET_PAGE_BITS) == start) {
+        long k;
+        long nr = BITS_TO_LONGS(pages);
+
+        for (k = 0; k < nr; k++) {
+            if (bitmap[k]) {
+                unsigned long temp = leul_to_cpu(bitmap[k]);
+
+                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION][page + k] |= temp;
+                ram_list.dirty_memory[DIRTY_MEMORY_VGA][page + k] |= temp;
+                ram_list.dirty_memory[DIRTY_MEMORY_CODE][page + k] |= temp;
+            }
+        }
+        xen_modified_memory(start, pages);
+    } else {
+        /*
+         * bitmap-traveling is faster than memory-traveling (for addr...)
+         * especially when most of the memory is not dirty.
+         */
+        for (i = 0; i < len; i++) {
+            if (bitmap[i] != 0) {
+                c = leul_to_cpu(bitmap[i]);
+                do {
+                    j = ffsl(c) - 1;
+                    c &= ~(1ul << j);
+                    page_number = (i * HOST_LONG_BITS + j) * hpratio;
+                    addr = page_number * TARGET_PAGE_SIZE;
+                    ram_addr = start + addr;
+                    cpu_physical_memory_set_dirty_range(ram_addr,
+                                       TARGET_PAGE_SIZE * hpratio);
+                } while (c != 0);
+            }
         }
     }
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 36/38] ram: split function that synchronizes a range

[Juan Quintela]

This function is the only bit where we care about speed.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 arch_init.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index 0e8c8b5..2cd3d00 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -360,11 +360,10 @@ ram_addr_t migration_bitmap_find_and_reset_dirty(MemoryRegion *mr,
     return (next - base) << TARGET_PAGE_BITS;
 }

-static inline bool migration_bitmap_set_dirty(MemoryRegion *mr,
-                                              ram_addr_t offset)
+static inline bool migration_bitmap_set_dirty(ram_addr_t addr)
 {
     bool ret;
-    int nr = (mr->ram_addr + offset) >> TARGET_PAGE_BITS;
+    int nr = addr >> TARGET_PAGE_BITS;

     ret = test_and_set_bit(nr, migration_bitmap);

@@ -374,12 +373,28 @@ static inline bool migration_bitmap_set_dirty(MemoryRegion *mr,
     return ret;
 }

+static void migration_bitmap_sync_range(ram_addr_t start, ram_addr_t length)
+{
+    ram_addr_t addr;
+
+    for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
+        if (cpu_physical_memory_get_dirty(start + addr,
+                                          TARGET_PAGE_SIZE,
+                                          DIRTY_MEMORY_MIGRATION)) {
+            cpu_physical_memory_reset_dirty(start + addr,
+                                            TARGET_PAGE_SIZE,
+                                            DIRTY_MEMORY_MIGRATION);
+            migration_bitmap_set_dirty(start + addr);
+        }
+    }
+}
+
+
 /* Needs iothread lock! */

 static void migration_bitmap_sync(void)
 {
     RAMBlock *block;
-    ram_addr_t addr;
     uint64_t num_dirty_pages_init = migration_dirty_pages;
     MigrationState *s = migrate_get_current();
     static int64_t start_time;
@@ -400,16 +415,7 @@ static void migration_bitmap_sync(void)
     address_space_sync_dirty_bitmap(&address_space_memory);

     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
-        for (addr = 0; addr < block->length; addr += TARGET_PAGE_SIZE) {
-            if (cpu_physical_memory_get_dirty(block->mr->ram_addr + addr,
-                                              TARGET_PAGE_SIZE,
-                                              DIRTY_MEMORY_MIGRATION)) {
-                cpu_physical_memory_reset_dirty(block->mr->ram_addr + addr,
-                                                TARGET_PAGE_SIZE,
-                                                DIRTY_MEMORY_MIGRATION);
-                migration_bitmap_set_dirty(block->mr, addr);
-            }
-        }
+        migration_bitmap_sync_range(block->mr->ram_addr, block->length);
     }
     trace_migration_bitmap_sync_end(migration_dirty_pages
                                     - num_dirty_pages_init);
-- 
1.8.3.1

[Qemu-devel] [PATCH 37/38] migration: synchronize memory bitmap 64bits at a time

[Juan Quintela]

We use the old code if the bitmaps are not aligned

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 arch_init.c | 38 +++++++++++++++++++++++++++++---------
 1 file changed, 29 insertions(+), 9 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index 2cd3d00..77912e7 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -50,6 +50,7 @@
 #include "exec/cpu-all.h"
 #include "exec/ram_addr.h"
 #include "hw/acpi/acpi.h"
+#include "qemu/host-utils.h"

 #ifdef DEBUG_ARCH_INIT
 #define DPRINTF(fmt, ...) \
@@ -376,15 +377,34 @@ static inline bool migration_bitmap_set_dirty(ram_addr_t addr)
 static void migration_bitmap_sync_range(ram_addr_t start, ram_addr_t length)
 {
     ram_addr_t addr;
-
-    for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
-        if (cpu_physical_memory_get_dirty(start + addr,
-                                          TARGET_PAGE_SIZE,
-                                          DIRTY_MEMORY_MIGRATION)) {
-            cpu_physical_memory_reset_dirty(start + addr,
-                                            TARGET_PAGE_SIZE,
-                                            DIRTY_MEMORY_MIGRATION);
-            migration_bitmap_set_dirty(start + addr);
+    unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
+
+    /* start address is aligned at the start of a word? */
+    if (((page * BITS_PER_LONG) << TARGET_PAGE_BITS) == start) {
+        int k;
+        int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
+        unsigned long *src = ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION];
+
+        for (k = page; k < page + nr; k++) {
+            if (src[k]) {
+                unsigned long new_dirty;
+                new_dirty = ~migration_bitmap[k];
+                migration_bitmap[k] |= src[k];
+                new_dirty &= src[k];
+                migration_dirty_pages += ctpopl(new_dirty);
+                src[k] = 0;
+            }
+        }
+    } else {
+        for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
+            if (cpu_physical_memory_get_dirty(start + addr,
+                                              TARGET_PAGE_SIZE,
+                                              DIRTY_MEMORY_MIGRATION)) {
+                cpu_physical_memory_reset_dirty(start + addr,
+                                                TARGET_PAGE_SIZE,
+                                                DIRTY_MEMORY_MIGRATION);
+                migration_bitmap_set_dirty(start + addr);
+            }
         }
     }
 }
-- 
1.8.3.1

[Qemu-devel] [PATCH 38/38] ram: align ram_addr_t's regions in multiples of 64

[Juan Quintela]

This allows us to use fastest path for bitmap operations.
This was based on Paolo idea/code.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 exec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index be421e5..faf3f59 100644
--- a/exec.c
+++ b/exec.c
@@ -1108,7 +1108,7 @@ static ram_addr_t find_ram_offset(ram_addr_t size)
     QTAILQ_FOREACH(block, &ram_list.blocks, next) {
         ram_addr_t end, next = RAM_ADDR_MAX;

-        end = block->offset + block->length;
+        end = ROUND_UP(block->offset + block->length, TARGET_PAGE_SIZE * 64);

         QTAILQ_FOREACH(next_block, &ram_list.blocks, next) {
             if (next_block->offset >= end) {
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 38/38] ram: align ram_addr_t's regions in multiples of 64

[Juan Quintela]

Juan Quintela <quintela@redhat.com> wrote:
> This allows us to use fastest path for bitmap operations.
> This was based on Paolo idea/code.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  exec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/exec.c b/exec.c
> index be421e5..faf3f59 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1108,7 +1108,7 @@ static ram_addr_t find_ram_offset(ram_addr_t size)
>      QTAILQ_FOREACH(block, &ram_list.blocks, next) {
>          ram_addr_t end, next = RAM_ADDR_MAX;
>
> -        end = block->offset + block->length;
> +        end = ROUND_UP(block->offset + block->length, TARGET_PAGE_SIZE * 64);
>
>          QTAILQ_FOREACH(next_block, &ram_list.blocks, next) {
>              if (next_block->offset >= end) {

NAK

This patch is giving me trouble.  iht stays at 868kbytes forever,
removing this patch makes things work again.

Paolo, any idea?

will drop it until I found what/where the problem is.

Later, Juan.

16:51:10 DEBUG| (monitor hmp1) Response to 'info migrate'
16:51:10 DEBUG| (monitor hmp1)    capabilities: xbzrle: off x-rdma-pin-all: off auto-converge: off zero-blocks: off 
16:51:10 DEBUG| (monitor hmp1)    Migration status: active
16:51:10 DEBUG| (monitor hmp1)    total time: 1515704 milliseconds
16:51:10 DEBUG| (monitor hmp1)    expected downtime: 30 milliseconds
16:51:10 DEBUG| (monitor hmp1)    setup: 6 milliseconds
16:51:10 DEBUG| (monitor hmp1)    transferred ram: 5168004 kbytes
16:51:10 DEBUG| (monitor hmp1)    throughput: 44.17 mbps
16:51:10 DEBUG| (monitor hmp1)    remaining ram: 868 kbytes
16:51:10 DEBUG| (monitor hmp1)    total ram: 1065688 kbytes
16:51:10 DEBUG| (monitor hmp1)    duplicate: 228523 pages
16:51:10 DEBUG| (monitor hmp1)    skipped: 0 pages
16:51:10 DEBUG| (monitor hmp1)    normal: 37899 pages
16:51:10 DEBUG| (monitor hmp1)    normal bytes: 151596 kbytes
16:51:11 DEBUG| (monitor hmp1) Sending command 'cont' 
16:51:11 DEBUG| Send command: cont
16:51:11 DEBUG| Host does not support 

Re: [Qemu-devel] [PATCH 01/38] bitmap: use long as index

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2307 bytes --]

On 12/17/2013 08:25 AM, Juan Quintela wrote:
> Move index and size fields from int to long.  We need that for
> migration.  long is 64 bits on sane architectures, and 32bits should
> be enough on all the 32bits architectures.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  include/qemu/bitmap.h | 77 ++++++++++++++++++++++++++-------------------------
>  include/qemu/bitops.h | 14 +++++-----
>  util/bitmap.c         | 60 +++++++++++++++++++--------------------
>  3 files changed, 76 insertions(+), 75 deletions(-)
> 
> diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
> index 308bbb7..afdd257 100644
> --- a/include/qemu/bitmap.h
> +++ b/include/qemu/bitmap.h
> @@ -31,7 +31,7 @@
>   * bitmap_andnot(dst, src1, src2, nbits)	*dst = *src1 & ~(*src2)
>   * bitmap_complement(dst, src, nbits)		*dst = ~(*src)
>   * bitmap_equal(src1, src2, nbits)		Are *src1 and *src2 equal?
> - * bitmap_intersects(src1, src2, nbits) 	Do *src1 and *src2 overlap?
> + * bitmap_intersects(src1, src2, nbits)         Do *src1 and *src2 overlap?

Spurious whitespace change?

>   * bitmap_empty(src, nbits)			Are all bits zero in *src?
>   * bitmap_full(src, nbits)			Are all bits set in *src?
>   * bitmap_set(dst, pos, nbits)			Set specified bit area
> @@ -62,71 +62,71 @@
>          )
> 
>  #define DECLARE_BITMAP(name,bits)                  \
> -	unsigned long name[BITS_TO_LONGS(bits)]
> +        unsigned long name[BITS_TO_LONGS(bits)]
> 
>  #define small_nbits(nbits)                      \
> -	((nbits) <= BITS_PER_LONG)
> +        ((nbits) <= BITS_PER_LONG)

Whitespace change, but in same hunk as real changes, so ok for
checkpatch.pl reasons.

> +++ b/include/qemu/bitops.h
> @@ -28,7 +28,7 @@
>   * @nr: the bit to set
>   * @addr: the address to start counting from
>   */
> -static inline void set_bit(int nr, unsigned long *addr)
> +static inline void set_bit(long nr, unsigned long *addr)
>  {
>  	unsigned long mask = BIT_MASK(nr);
>          unsigned long *p = addr + BIT_WORD(nr);

Worth cleaning up this whitespace while in the area?

Content changes seem sane to me:
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 01/38] bitmap: use long as index

[Stefan Weil]

Am 17.12.2013 19:05, schrieb Eric Blake:
> On 12/17/2013 08:25 AM, Juan Quintela wrote:
>> Move index and size fields from int to long.  We need that for
>> migration.  long is 64 bits on sane architectures, and 32bits should
>> be enough on all the 32bits architectures.


Does this also work for "insane" architectures like Windows (64 bit) where
long is only 32 bit? Wouldn't uintptr_t or intptr_t be better (also for
x32)?


>>
>>
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  include/qemu/bitmap.h | 77
++++++++++++++++++++++++++-------------------------
>>  include/qemu/bitops.h | 14 +++++-----
>>  util/bitmap.c         | 60 +++++++++++++++++++--------------------
>>  3 files changed, 76 insertions(+), 75 deletions(-)
>>
>> diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
>> index 308bbb7..afdd257 100644
>> --- a/include/qemu/bitmap.h
>> +++ b/include/qemu/bitmap.h
>> @@ -31,7 +31,7 @@
>>   * bitmap_andnot(dst, src1, src2, nbits)    *dst = *src1 & ~(*src2)
>>   * bitmap_complement(dst, src, nbits)        *dst = ~(*src)
>>   * bitmap_equal(src1, src2, nbits)        Are *src1 and *src2 equal?
>> - * bitmap_intersects(src1, src2, nbits)     Do *src1 and *src2 overlap?
>> + * bitmap_intersects(src1, src2, nbits)         Do *src1 and *src2
overlap?
>
> Spurious whitespace change?
>
>>   * bitmap_empty(src, nbits)            Are all bits zero in *src?
>>   * bitmap_full(src, nbits)            Are all bits set in *src?
>>   * bitmap_set(dst, pos, nbits)            Set specified bit area
>> @@ -62,71 +62,71 @@
>>          )
>>
>>  #define DECLARE_BITMAP(name,bits)                  \
>> -    unsigned long name[BITS_TO_LONGS(bits)]
>> +        unsigned long name[BITS_TO_LONGS(bits)]
>>
>>  #define small_nbits(nbits)                      \
>> -    ((nbits) <= BITS_PER_LONG)
>> +        ((nbits) <= BITS_PER_LONG)
>
> Whitespace change, but in same hunk as real changes, so ok for
> checkpatch.pl reasons.
>
>> +++ b/include/qemu/bitops.h
>> @@ -28,7 +28,7 @@
>>   * @nr: the bit to set
>>   * @addr: the address to start counting from
>>   */
>> -static inline void set_bit(int nr, unsigned long *addr)
>> +static inline void set_bit(long nr, unsigned long *addr)
>>  {
>>      unsigned long mask = BIT_MASK(nr);
>>          unsigned long *p = addr + BIT_WORD(nr);
>
> Worth cleaning up this whitespace while in the area?
>
> Content changes seem sane to me:
> Reviewed-by: Eric Blake <eblake@redhat.com>
>

Cheers,
Stefan

Re: [Qemu-devel] [PATCH 01/38] bitmap: use long as index

[Juan Quintela]

Eric Blake <eblake@redhat.com> wrote:
> On 12/17/2013 08:25 AM, Juan Quintela wrote:
>> Move index and size fields from int to long.  We need that for
>> migration.  long is 64 bits on sane architectures, and 32bits should
>> be enough on all the 32bits architectures.
>> 
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  include/qemu/bitmap.h | 77 ++++++++++++++++++++++++++-------------------------
>>  include/qemu/bitops.h | 14 +++++-----
>>  util/bitmap.c         | 60 +++++++++++++++++++--------------------
>>  3 files changed, 76 insertions(+), 75 deletions(-)
>> 
>> diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
>> index 308bbb7..afdd257 100644
>> --- a/include/qemu/bitmap.h
>> +++ b/include/qemu/bitmap.h
>> @@ -31,7 +31,7 @@
>>   * bitmap_andnot(dst, src1, src2, nbits)	*dst = *src1 & ~(*src2)
>>   * bitmap_complement(dst, src, nbits)		*dst = ~(*src)
>>   * bitmap_equal(src1, src2, nbits)		Are *src1 and *src2 equal?
>> - * bitmap_intersects(src1, src2, nbits) 	Do *src1 and *src2 overlap?
>> + * bitmap_intersects(src1, src2, nbits)         Do *src1 and *src2 overlap?
>
> Spurious whitespace change?
>
> Worth cleaning up this whitespace while in the area?

Yeap.  After passing checkpatch, it complained about lots of lines.  And
after that only three tabs or so were left, so I fixed them.

And forget to fix the comment.

>
> Content changes seem sane to me:
> Reviewed-by: Eric Blake <eblake@redhat.com>

Thanks, Juan.

Re: [Qemu-devel] [PATCH 07/38] memory: make cpu_physical_memory_is_dirty return bool

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c                         | 7 ++-----
>   include/exec/memory-internal.h | 8 ++++++--
>   2 files changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index 95bcf70..a1fc280 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1484,11 +1484,8 @@ found:
>   static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
>                                  uint64_t val, unsigned size)
>   {
> -    int dirty_flags;
> -    dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
>       if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
>           tb_invalidate_phys_page_fast(ram_addr, size);
> -        dirty_flags = cpu_physical_memory_get_dirty_flags(ram_addr);
>       }
>       switch (size) {
>       case 1:
> @@ -1503,8 +1500,8 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
>       default:
>           abort();
>       }
> -    dirty_flags |= (0xff & ~CODE_DIRTY_FLAG);
> -    cpu_physical_memory_set_dirty_flags(ram_addr, dirty_flags);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
>       /* we remove the notdirty callback only if the code has been
>          flushed */
>       if (cpu_physical_memory_is_dirty(ram_addr)) {
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 136198c..0b25c3f 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -56,9 +56,13 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>   }
>
>   /* read dirty bit (return 0 or 1) */
> -static inline int cpu_physical_memory_is_dirty(ram_addr_t addr)
> +static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
>   {
> -    return cpu_physical_memory_get_dirty_flags(addr) == 0xff;
> +    bool vga = cpu_physical_memory_get_dirty_flag(addr, VGA_DIRTY_FLAG);
> +    bool code = cpu_physical_memory_get_dirty_flag(addr, CODE_DIRTY_FLAG);
> +    bool migration =
> +        cpu_physical_memory_get_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
> +    return vga && code && migration;
>   }
>
>   static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 08/38] memory: all users of cpu_physical_memory_get_dirty used only one flag

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> So cpu_physical_memory_get_dirty_flags is not needed anymore
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 9 ++-------
>   1 file changed, 2 insertions(+), 7 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 0b25c3f..53cfe83 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -44,11 +44,6 @@ void qemu_ram_free_from_ptr(ram_addr_t addr);
>   #define CODE_DIRTY_FLAG      0x02
>   #define MIGRATION_DIRTY_FLAG 0x08
>
> -static inline int cpu_physical_memory_get_dirty_flags(ram_addr_t addr)
> -{
> -    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS];
> -}
> -
>   static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>                                                         int dirty_flag)
>   {
> @@ -67,7 +62,7 @@ static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
>
>   static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>                                                   ram_addr_t length,
> -                                                int dirty_flags)
> +                                                int dirty_flag)
>   {
>       int ret = 0;
>       ram_addr_t addr, end;
> @@ -75,7 +70,7 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        ret |= cpu_physical_memory_get_dirty_flags(addr) & dirty_flags;
> +        ret |= cpu_physical_memory_get_dirty_flag(addr, dirty_flag);
>       }
>       return ret;
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 09/38] memory: set single dirty flags when possible

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c                         | 7 ++++---
>   include/exec/memory-internal.h | 4 +++-
>   2 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index a1fc280..6981f73 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1911,7 +1911,8 @@ static void invalidate_and_set_dirty(hwaddr addr,
>           /* invalidate code */
>           tb_invalidate_phys_page_range(addr, addr + length, 0);
>           /* set dirty bit */
> -        cpu_physical_memory_set_dirty_flags(addr, (0xff & ~CODE_DIRTY_FLAG));
> +        cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
> +        cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
>       }
>       xen_modified_memory(addr, length);
>   }
> @@ -2493,8 +2494,8 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
>                   /* invalidate code */
>                   tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
>                   /* set dirty bit */
> -                cpu_physical_memory_set_dirty_flags(
> -                    addr1, (0xff & ~CODE_DIRTY_FLAG));
> +                cpu_physical_memory_set_dirty_flag(addr1, MIGRATION_DIRTY_FLAG);
> +                cpu_physical_memory_set_dirty_flag(addr1, VGA_DIRTY_FLAG);
>               }
>           }
>       }
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 53cfe83..9f4ad69 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -89,7 +89,9 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>
>   static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
>   {
> -    cpu_physical_memory_set_dirty_flags(addr, 0xff);
> +    cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
>   }
>
>   static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 10/38] memory: cpu_physical_memory_set_dirty_range() always dirty all flags

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> So remove the flag argument and do it directly.  After this change,
> there is nothing else using cpu_physical_memory_set_dirty_flags() so
> remove it.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c                         |  2 +-
>   include/exec/memory-internal.h | 11 ++---------
>   memory.c                       |  2 +-
>   3 files changed, 4 insertions(+), 11 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index 6981f73..45fdb84 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1274,7 +1274,7 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
>                                          last_ram_offset() >> TARGET_PAGE_BITS);
>       memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
>              0, size >> TARGET_PAGE_BITS);
> -    cpu_physical_memory_set_dirty_range(new_block->offset, size, 0xff);
> +    cpu_physical_memory_set_dirty_range(new_block->offset, size);
>
>       qemu_ram_setup_dump(new_block->host, size);
>       qemu_madvise(new_block->host, size, QEMU_MADV_HUGEPAGE);
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 9f4ad69..681d63b 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -75,12 +75,6 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>       return ret;
>   }
>
> -static inline void cpu_physical_memory_set_dirty_flags(ram_addr_t addr,
> -                                                      int dirty_flags)
> -{
> -    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flags;
> -}
> -
>   static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>                                                         int dirty_flag)
>   {
> @@ -103,15 +97,14 @@ static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
>   }
>
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
> -                                                       ram_addr_t length,
> -                                                       int dirty_flags)
> +                                                       ram_addr_t length)
>   {
>       ram_addr_t addr, end;
>
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        cpu_physical_memory_set_dirty_flags(addr, dirty_flags);
> +        cpu_physical_memory_set_dirty(addr);
>       }
>       xen_modified_memory(addr, length);
>   }
> diff --git a/memory.c b/memory.c
> index e497f99..fb52e1c 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1182,7 +1182,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
>                                hwaddr size)
>   {
>       assert(mr->terminates);
> -    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size, -1);
> +    cpu_physical_memory_set_dirty_range(mr->ram_addr + addr, size);
>   }
>
>   bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 11/38] memory: cpu_physical_memory_mask_dirty_range() always clears a single flag

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Document it
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   cputlb.c                       |  4 ++--
>   exec.c                         | 19 ++++++++++---------
>   include/exec/memory-internal.h | 40 ++++++++++++++++++----------------------
>   include/exec/memory.h          |  3 ---
>   memory.c                       | 10 ++++------
>   5 files changed, 34 insertions(+), 42 deletions(-)
>
> diff --git a/cputlb.c b/cputlb.c
> index 72452e5..dfd747a 100644
> --- a/cputlb.c
> +++ b/cputlb.c
> @@ -129,7 +129,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
>   {
>       cpu_physical_memory_reset_dirty(ram_addr,
>                                       ram_addr + TARGET_PAGE_SIZE,
> -                                    CODE_DIRTY_FLAG);
> +                                    DIRTY_MEMORY_CODE);
>   }
>
>   /* update the TLB so that writes in physical page 'phys_addr' are no longer
> @@ -137,7 +137,7 @@ void tlb_protect_code(ram_addr_t ram_addr)
>   void tlb_unprotect_code_phys(CPUArchState *env, ram_addr_t ram_addr,
>                                target_ulong vaddr)
>   {
> -    cpu_physical_memory_set_dirty_flag(ram_addr, CODE_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_CODE);
>   }
>
>   static bool tlb_is_dirty_ram(CPUTLBEntry *tlbe)
> diff --git a/exec.c b/exec.c
> index 45fdb84..9996da2 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -737,7 +737,7 @@ static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,
>
>   /* Note: start and end must be within the same ram block.  */
>   void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
> -                                     int dirty_flags)
> +                                     unsigned client)
>   {
>       uintptr_t length;
>
> @@ -747,7 +747,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
>       length = end - start;
>       if (length == 0)
>           return;
> -    cpu_physical_memory_mask_dirty_range(start, length, dirty_flags);
> +    cpu_physical_memory_mask_dirty_range(start, length, client);
>
>       if (tcg_enabled()) {
>           tlb_reset_dirty_range_all(start, end, length);
> @@ -1484,7 +1484,7 @@ found:
>   static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
>                                  uint64_t val, unsigned size)
>   {
> -    if (!cpu_physical_memory_get_dirty_flag(ram_addr, CODE_DIRTY_FLAG)) {
> +    if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) {
>           tb_invalidate_phys_page_fast(ram_addr, size);
>       }
>       switch (size) {
> @@ -1500,8 +1500,8 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
>       default:
>           abort();
>       }
> -    cpu_physical_memory_set_dirty_flag(ram_addr, MIGRATION_DIRTY_FLAG);
> -    cpu_physical_memory_set_dirty_flag(ram_addr, VGA_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_MIGRATION);
> +    cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_VGA);
>       /* we remove the notdirty callback only if the code has been
>          flushed */
>       if (cpu_physical_memory_is_dirty(ram_addr)) {
> @@ -1911,8 +1911,8 @@ static void invalidate_and_set_dirty(hwaddr addr,
>           /* invalidate code */
>           tb_invalidate_phys_page_range(addr, addr + length, 0);
>           /* set dirty bit */
> -        cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
> -        cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
> +        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
> +        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
>       }
>       xen_modified_memory(addr, length);
>   }
> @@ -2494,8 +2494,9 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
>                   /* invalidate code */
>                   tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
>                   /* set dirty bit */
> -                cpu_physical_memory_set_dirty_flag(addr1, MIGRATION_DIRTY_FLAG);
> -                cpu_physical_memory_set_dirty_flag(addr1, VGA_DIRTY_FLAG);
> +                cpu_physical_memory_set_dirty_flag(addr1,
> +                                                   DIRTY_MEMORY_MIGRATION);
> +                cpu_physical_memory_set_dirty_flag(addr1, DIRTY_MEMORY_VGA);
>               }
>           }
>       }
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 681d63b..b58010f 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -40,29 +40,25 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
>   void qemu_ram_free(ram_addr_t addr);
>   void qemu_ram_free_from_ptr(ram_addr_t addr);
>
> -#define VGA_DIRTY_FLAG       0x01
> -#define CODE_DIRTY_FLAG      0x02
> -#define MIGRATION_DIRTY_FLAG 0x08
> -
>   static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
> -                                                      int dirty_flag)
> +                                                      unsigned client)
>   {
> -    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & dirty_flag;
> +    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
>   }
>
>   /* read dirty bit (return 0 or 1) */
>   static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
>   {
> -    bool vga = cpu_physical_memory_get_dirty_flag(addr, VGA_DIRTY_FLAG);
> -    bool code = cpu_physical_memory_get_dirty_flag(addr, CODE_DIRTY_FLAG);
> +    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
> +    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
>       bool migration =
> -        cpu_physical_memory_get_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
> +        cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
>       return vga && code && migration;
>   }
>
>   static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>                                                   ram_addr_t length,
> -                                                int dirty_flag)
> +                                                unsigned client)
>   {
>       int ret = 0;
>       ram_addr_t addr, end;
> @@ -70,28 +66,28 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        ret |= cpu_physical_memory_get_dirty_flag(addr, dirty_flag);
> +        ret |= cpu_physical_memory_get_dirty_flag(addr, client);
>       }
>       return ret;
>   }
>
>   static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
> -                                                      int dirty_flag)
> +                                                      unsigned client)
>   {
> -    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= dirty_flag;
> +    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
>   }
>
>   static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
>   {
> -    cpu_physical_memory_set_dirty_flag(addr, MIGRATION_DIRTY_FLAG);
> -    cpu_physical_memory_set_dirty_flag(addr, VGA_DIRTY_FLAG);
> -    cpu_physical_memory_set_dirty_flag(addr, CODE_DIRTY_FLAG);
> +    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> +    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
> +    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
>   }
>
> -static inline int cpu_physical_memory_clear_dirty_flags(ram_addr_t addr,
> -                                                        int dirty_flags)
> +static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
> +                                                       unsigned client)
>   {
> -    int mask = ~dirty_flags;
> +    int mask = ~(1 << client);
>
>       return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
>   }
> @@ -111,19 +107,19 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>
>   static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
>                                                           ram_addr_t length,
> -                                                        int dirty_flags)
> +                                                        unsigned client)
>   {
>       ram_addr_t addr, end;
>
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        cpu_physical_memory_clear_dirty_flags(addr, dirty_flags);
> +        cpu_physical_memory_clear_dirty_flag(addr, client);
>       }
>   }
>
>   void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
> -                                     int dirty_flags);
> +                                     unsigned client);
>
>   #endif
>
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index 480dfbf..b8e76f4 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -33,9 +33,6 @@
>   typedef struct MemoryRegionOps MemoryRegionOps;
>   typedef struct MemoryRegionMmio MemoryRegionMmio;
>
> -/* Must match *_DIRTY_FLAGS in cpu-all.h.  To be replaced with dynamic
> - * registration.
> - */
>   #define DIRTY_MEMORY_VGA       0
>   #define DIRTY_MEMORY_CODE      1
>   #define DIRTY_MEMORY_MIGRATION 3
> diff --git a/memory.c b/memory.c
> index fb52e1c..a490cbd 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1174,8 +1174,7 @@ bool memory_region_get_dirty(MemoryRegion *mr, hwaddr addr,
>                                hwaddr size, unsigned client)
>   {
>       assert(mr->terminates);
> -    return cpu_physical_memory_get_dirty(mr->ram_addr + addr, size,
> -                                         1 << client);
> +    return cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
>   }
>
>   void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
> @@ -1190,12 +1189,11 @@ bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
>   {
>       bool ret;
>       assert(mr->terminates);
> -    ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size,
> -                                        1 << client);
> +    ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
>       if (ret) {
>           cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
>                                           mr->ram_addr + addr + size,
> -                                        1 << client);
> +                                        client);
>       }
>       return ret;
>   }
> @@ -1243,7 +1241,7 @@ void memory_region_reset_dirty(MemoryRegion *mr, hwaddr addr,
>       assert(mr->terminates);
>       cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
>                                       mr->ram_addr + addr + size,
> -                                    1 << client);
> +                                    client);
>   }
>
>   void *memory_region_get_ram_ptr(MemoryRegion *mr)
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 12/38] memory: use bit 2 for migration

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> For historical reasons it was bit 3.  Once there, create a constant to
> know the number of clients.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory.h | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index b8e76f4..d5e9d58 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -35,7 +35,8 @@ typedef struct MemoryRegionMmio MemoryRegionMmio;
>
>   #define DIRTY_MEMORY_VGA       0
>   #define DIRTY_MEMORY_CODE      1
> -#define DIRTY_MEMORY_MIGRATION 3
> +#define DIRTY_MEMORY_MIGRATION 2
> +#define DIRTY_MEMORY_NUM       3        /* num of dirty bits */
>
>   struct MemoryRegionMmio {
>       CPUReadMemoryFunc *read[3];
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 13/38] memory: make sure that client is always inside range

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index b58010f..d09d6d8 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -43,6 +43,7 @@ void qemu_ram_free_from_ptr(ram_addr_t addr);
>   static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>                                                         unsigned client)
>   {
> +    assert(client < DIRTY_MEMORY_NUM);
>       return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
>   }
>
> @@ -74,6 +75,7 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>   static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>                                                         unsigned client)
>   {
> +    assert(client < DIRTY_MEMORY_NUM);
>       ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
>   }
>
> @@ -89,6 +91,8 @@ static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
>   {
>       int mask = ~(1 << client);
>
> +    assert(client < DIRTY_MEMORY_NUM);
> +
>       return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 14/38] memory: only resize dirty bitmap when memory size increases

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c | 12 +++++++++---
>   1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index 9996da2..bed5c07 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1210,6 +1210,9 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
>                                      MemoryRegion *mr)
>   {
>       RAMBlock *block, *new_block;
> +    ram_addr_t old_ram_size, new_ram_size;
> +
> +    old_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;
>
>       size = TARGET_PAGE_ALIGN(size);
>       new_block = g_malloc0(sizeof(*new_block));
> @@ -1270,10 +1273,13 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
>       ram_list.version++;
>       qemu_mutex_unlock_ramlist();
>
> -    ram_list.phys_dirty = g_realloc(ram_list.phys_dirty,
> -                                       last_ram_offset() >> TARGET_PAGE_BITS);
> -    memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
> +    new_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;
> +
> +    if (new_ram_size > old_ram_size) {
> +        ram_list.phys_dirty = g_realloc(ram_list.phys_dirty, new_ram_size);
> +        memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
>              0, size >> TARGET_PAGE_BITS);
> +    }
>       cpu_physical_memory_set_dirty_range(new_block->offset, size);
>
>       qemu_ram_setup_dump(new_block->host, size);
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 15/38] memory: cpu_physical_memory_clear_dirty_flag() result is never used

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index d09d6d8..666490c 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -86,14 +86,14 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
>       cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
>   }
>
> -static inline int cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
> +static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
>                                                          unsigned client)
>   {
>       int mask = ~(1 << client);
>
>       assert(client < DIRTY_MEMORY_NUM);
>
> -    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
> +    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
>   }
>
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 16/38] bitmap: Add bitmap_zero_extend operation

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/qemu/bitmap.h | 9 +++++++++
>   1 file changed, 9 insertions(+)
>
> diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
> index afdd257..1babd5d 100644
> --- a/include/qemu/bitmap.h
> +++ b/include/qemu/bitmap.h
> @@ -220,4 +220,13 @@ unsigned long bitmap_find_next_zero_area(unsigned long *map,
>                                            unsigned long nr,
>                                            unsigned long align_mask);
>
> +static inline unsigned long *bitmap_zero_extend(unsigned long *old,
> +                                                long old_nbits, long new_nbits)
> +{
> +    long new_len = BITS_TO_LONGS(new_nbits) * sizeof(unsigned long);
> +    unsigned long *new = g_realloc(old, new_len);
> +    bitmap_clear(new, old_nbits, new_nbits - old_nbits);
> +    return new;
> +}
> +
>   #endif /* BITMAP_H */
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 17/38] memory: split dirty bitmap into three

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> After all the previous patches, spliting the bitmap gets direct.
>
> Note: For some reason, I have to move DIRTY_MEMORY_* definitions to
> the beginning of memory.h to make compilation work.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c                         |  9 ++++++---
>   include/exec/cpu-all.h         |  3 ++-
>   include/exec/memory-internal.h |  9 +++------
>   include/exec/memory.h          | 10 +++++-----
>   4 files changed, 16 insertions(+), 15 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index bed5c07..ad9866c 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1276,9 +1276,12 @@ ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
>       new_ram_size = last_ram_offset() >> TARGET_PAGE_BITS;
>
>       if (new_ram_size > old_ram_size) {
> -        ram_list.phys_dirty = g_realloc(ram_list.phys_dirty, new_ram_size);
> -        memset(ram_list.phys_dirty + (new_block->offset >> TARGET_PAGE_BITS),
> -           0, size >> TARGET_PAGE_BITS);
> +        int i;
> +        for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
> +            ram_list.dirty_memory[i] =
> +                bitmap_zero_extend(ram_list.dirty_memory[i],
> +                                   old_ram_size, new_ram_size);
> +       }
>       }
>       cpu_physical_memory_set_dirty_range(new_block->offset, size);
>
> diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
> index b6998f0..4cb4b4a 100644
> --- a/include/exec/cpu-all.h
> +++ b/include/exec/cpu-all.h
> @@ -21,6 +21,7 @@
>
>   #include "qemu-common.h"
>   #include "exec/cpu-common.h"
> +#include "exec/memory.h"
>   #include "qemu/thread.h"
>   #include "qom/cpu.h"
>
> @@ -459,7 +460,7 @@ typedef struct RAMBlock {
>   typedef struct RAMList {
>       QemuMutex mutex;
>       /* Protected by the iothread lock.  */
> -    uint8_t *phys_dirty;
> +    unsigned long *dirty_memory[DIRTY_MEMORY_NUM];
>       RAMBlock *mru_block;
>       /* Protected by the ramlist lock.  */
>       QTAILQ_HEAD(, RAMBlock) blocks;
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 666490c..6fb1b64 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -44,7 +44,7 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>                                                         unsigned client)
>   {
>       assert(client < DIRTY_MEMORY_NUM);
> -    return ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] & (1 << client);
> +    return test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
>   }
>
>   /* read dirty bit (return 0 or 1) */
> @@ -76,7 +76,7 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>                                                         unsigned client)
>   {
>       assert(client < DIRTY_MEMORY_NUM);
> -    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] |= (1 << client);
> +    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
>   }
>
>   static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
> @@ -89,11 +89,8 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
>   static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
>                                                          unsigned client)
>   {
> -    int mask = ~(1 << client);
> -
>       assert(client < DIRTY_MEMORY_NUM);
> -
> -    ram_list.phys_dirty[addr >> TARGET_PAGE_BITS] &= mask;
> +    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
>   }
>
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index d5e9d58..296d6ab 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -16,6 +16,11 @@
>
>   #ifndef CONFIG_USER_ONLY
>
> +#define DIRTY_MEMORY_VGA       0
> +#define DIRTY_MEMORY_CODE      1
> +#define DIRTY_MEMORY_MIGRATION 2
> +#define DIRTY_MEMORY_NUM       3        /* num of dirty bits */
> +
>   #include <stdint.h>
>   #include <stdbool.h>
>   #include "qemu-common.h"
> @@ -33,11 +38,6 @@
>   typedef struct MemoryRegionOps MemoryRegionOps;
>   typedef struct MemoryRegionMmio MemoryRegionMmio;
>
> -#define DIRTY_MEMORY_VGA       0
> -#define DIRTY_MEMORY_CODE      1
> -#define DIRTY_MEMORY_MIGRATION 2
> -#define DIRTY_MEMORY_NUM       3        /* num of dirty bits */
> -
>   struct MemoryRegionMmio {
>       CPUReadMemoryFunc *read[3];
>       CPUWriteMemoryFunc *write[3];
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 18/38] memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 10 ++--------
>   1 file changed, 2 insertions(+), 8 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 6fb1b64..2f6610a 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -86,13 +86,6 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
>       cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
>   }
>
> -static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
> -                                                       unsigned client)
> -{
> -    assert(client < DIRTY_MEMORY_NUM);
> -    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> -}
> -
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>                                                          ram_addr_t length)
>   {
> @@ -112,10 +105,11 @@ static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
>   {
>       ram_addr_t addr, end;
>
> +    assert(client < DIRTY_MEMORY_NUM);
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        cpu_physical_memory_clear_dirty_flag(addr, client);
> +        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
>       }
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 19/38] memory: unfold cpu_physical_memory_set_dirty() in its only user

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 11 +++--------
>   1 file changed, 3 insertions(+), 8 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 2f6610a..9d34434 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -79,13 +79,6 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>       set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
>   }
>
> -static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
> -{
> -    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> -    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
> -    cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
> -}
> -
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>                                                          ram_addr_t length)
>   {
> @@ -94,7 +87,9 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        cpu_physical_memory_set_dirty(addr);
> +        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> +        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
> +        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
>       }
>       xen_modified_memory(addr, length);
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 20/38] memory: unfold cpu_physical_memory_set_dirty_flag()

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 9 ++++++---
>   1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 9d34434..b99617a 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -87,9 +87,12 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> -        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_VGA);
> -        cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
> +        set_bit(addr >> TARGET_PAGE_BITS,
> +                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
> +        set_bit(addr >> TARGET_PAGE_BITS,
> +                ram_list.dirty_memory[DIRTY_MEMORY_VGA]);
> +        set_bit(addr >> TARGET_PAGE_BITS,
> +                ram_list.dirty_memory[DIRTY_MEMORY_CODE]);
>       }
>       xen_modified_memory(addr, length);
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 21/38] memory: make cpu_physical_memory_get_dirty() the main function

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> And make cpu_physical_memory_get_dirty_flag() to use it.  It used to
> be the other way around.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 36 +++++++++++++++++++-----------------
>   1 file changed, 19 insertions(+), 17 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index b99617a..edca8a8 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -40,11 +40,28 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
>   void qemu_ram_free(ram_addr_t addr);
>   void qemu_ram_free_from_ptr(ram_addr_t addr);
>
> +static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
> +                                                ram_addr_t length,
> +                                                unsigned client)
> +{
> +    int ret = 0;
> +    ram_addr_t addr, end;
> +
> +    assert(client < DIRTY_MEMORY_NUM);
> +
> +    end = TARGET_PAGE_ALIGN(start + length);
> +    start &= TARGET_PAGE_MASK;
> +    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> +        ret |= test_bit(addr >> TARGET_PAGE_BITS,
> +                        ram_list.dirty_memory[client]);
> +    }
> +    return ret;
> +}
> +
>   static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>                                                         unsigned client)
>   {
> -    assert(client < DIRTY_MEMORY_NUM);
> -    return test_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> +    return cpu_physical_memory_get_dirty(addr, 1, client);
>   }
>
>   /* read dirty bit (return 0 or 1) */
> @@ -57,21 +74,6 @@ static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
>       return vga && code && migration;
>   }
>
> -static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
> -                                                ram_addr_t length,
> -                                                unsigned client)
> -{
> -    int ret = 0;
> -    ram_addr_t addr, end;
> -
> -    end = TARGET_PAGE_ALIGN(start + length);
> -    start &= TARGET_PAGE_MASK;
> -    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        ret |= cpu_physical_memory_get_dirty_flag(addr, client);
> -    }
> -    return ret;
> -}
> -
>   static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>                                                         unsigned client)
>   {
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 22/38] memory: cpu_physical_memory_get_dirty() is used as returning a bool

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 15 ++++++++-------
>   1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index edca8a8..fa28fc6 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -40,11 +40,10 @@ void *qemu_get_ram_ptr(ram_addr_t addr);
>   void qemu_ram_free(ram_addr_t addr);
>   void qemu_ram_free_from_ptr(ram_addr_t addr);
>
> -static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
> -                                                ram_addr_t length,
> -                                                unsigned client)
> +static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
> +                                                 ram_addr_t length,
> +                                                 unsigned client)
>   {
> -    int ret = 0;
>       ram_addr_t addr, end;
>
>       assert(client < DIRTY_MEMORY_NUM);
> @@ -52,10 +51,12 @@ static inline int cpu_physical_memory_get_dirty(ram_addr_t start,
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        ret |= test_bit(addr >> TARGET_PAGE_BITS,
> -                        ram_list.dirty_memory[client]);
> +        if (test_bit(addr >> TARGET_PAGE_BITS,
> +                     ram_list.dirty_memory[client])) {
> +            return true;
> +        }
>       }
> -    return ret;
> +    return false;
>   }
>
>   static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 18/38] memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 10 ++--------
>   1 file changed, 2 insertions(+), 8 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 6fb1b64..2f6610a 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -86,13 +86,6 @@ static inline void cpu_physical_memory_set_dirty(ram_addr_t addr)
>       cpu_physical_memory_set_dirty_flag(addr, DIRTY_MEMORY_CODE);
>   }
>
> -static inline void cpu_physical_memory_clear_dirty_flag(ram_addr_t addr,
> -                                                       unsigned client)
> -{
> -    assert(client < DIRTY_MEMORY_NUM);
> -    clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> -}
> -
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>                                                          ram_addr_t length)
>   {
> @@ -112,10 +105,11 @@ static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
>   {
>       ram_addr_t addr, end;
>
> +    assert(client < DIRTY_MEMORY_NUM);
>       end = TARGET_PAGE_ALIGN(start + length);
>       start &= TARGET_PAGE_MASK;
>       for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        cpu_physical_memory_clear_dirty_flag(addr, client);
> +        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
>       }
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 23/38] memory: s/mask/clear/ cpu_physical_memory_mask_dirty_range

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Now all functions use the same wording that bitops/bitmap operations
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c                         | 2 +-
>   include/exec/memory-internal.h | 6 +++---
>   2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index ad9866c..6bef3e5 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -747,7 +747,7 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
>       length = end - start;
>       if (length == 0)
>           return;
> -    cpu_physical_memory_mask_dirty_range(start, length, client);
> +    cpu_physical_memory_clear_dirty_range(start, length, client);
>
>       if (tcg_enabled()) {
>           tlb_reset_dirty_range_all(start, end, length);
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index fa28fc6..c04a92a 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -100,9 +100,9 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>       xen_modified_memory(addr, length);
>   }
>
> -static inline void cpu_physical_memory_mask_dirty_range(ram_addr_t start,
> -                                                        ram_addr_t length,
> -                                                        unsigned client)
> +static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
> +                                                         ram_addr_t length,
> +                                                         unsigned client)
>   {
>       ram_addr_t addr, end;
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 24/38] memory: use find_next_bit() to find dirty bits

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> This operation is way faster than doing it bit by bit.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 16 ++++++----------
>   1 file changed, 6 insertions(+), 10 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index c04a92a..b017c2e 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -44,19 +44,15 @@ static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
>                                                    ram_addr_t length,
>                                                    unsigned client)
>   {
> -    ram_addr_t addr, end;
> +    unsigned long end, page, next;
>
>       assert(client < DIRTY_MEMORY_NUM);
>
> -    end = TARGET_PAGE_ALIGN(start + length);
> -    start &= TARGET_PAGE_MASK;
> -    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        if (test_bit(addr >> TARGET_PAGE_BITS,
> -                     ram_list.dirty_memory[client])) {
> -            return true;
> -        }
> -    }
> -    return false;
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    next = find_next_bit(ram_list.dirty_memory[client], end, page);
> +
> +    return next < end;
>   }
>
>   static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 25/38] memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> We were setting a range of bits, so use bitmap_set().
>
> Note: xen has always been wrong, and should have used start instead
> of addr from the beginning.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 19 +++++++------------
>   1 file changed, 7 insertions(+), 12 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index b017c2e..4906cdf 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -81,19 +81,14 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>   static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>                                                          ram_addr_t length)
>   {
> -    ram_addr_t addr, end;
> +    unsigned long end, page;
>
> -    end = TARGET_PAGE_ALIGN(start + length);
> -    start &= TARGET_PAGE_MASK;
> -    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        set_bit(addr >> TARGET_PAGE_BITS,
> -                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
> -        set_bit(addr >> TARGET_PAGE_BITS,
> -                ram_list.dirty_memory[DIRTY_MEMORY_VGA]);
> -        set_bit(addr >> TARGET_PAGE_BITS,
> -                ram_list.dirty_memory[DIRTY_MEMORY_CODE]);
> -    }
> -    xen_modified_memory(addr, length);
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
> +    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
> +    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
> +    xen_modified_memory(start, length);
>   }
>
>   static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 26/38] memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> We were clearing a range of bits, so use bitmap_clear().
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   include/exec/memory-internal.h | 10 ++++------
>   1 file changed, 4 insertions(+), 6 deletions(-)
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 4906cdf..e2f55ea 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -95,14 +95,12 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
>                                                            ram_addr_t length,
>                                                            unsigned client)
>   {
> -    ram_addr_t addr, end;
> +    unsigned long end, page;
>
>       assert(client < DIRTY_MEMORY_NUM);
> -    end = TARGET_PAGE_ALIGN(start + length);
> -    start &= TARGET_PAGE_MASK;
> -    for (addr = start; addr < end; addr += TARGET_PAGE_SIZE) {
> -        clear_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> -    }
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
>   }
>
>   void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 27/38] memory: s/dirty/clean/ in cpu_physical_memory_is_dirty()

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> All uses except one really want the other meaning.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   cputlb.c                       | 3 ++-
>   exec.c                         | 6 +++---
>   include/exec/memory-internal.h | 5 ++---
>   3 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/cputlb.c b/cputlb.c
> index dfd747a..911d764 100644
> --- a/cputlb.c
> +++ b/cputlb.c
> @@ -299,7 +299,8 @@ void tlb_set_page(CPUArchState *env, target_ulong vaddr,
>               /* Write access calls the I/O callback.  */
>               te->addr_write = address | TLB_MMIO;
>           } else if (memory_region_is_ram(section->mr)
> -                   && !cpu_physical_memory_is_dirty(section->mr->ram_addr + xlat)) {
> +                   && cpu_physical_memory_is_clean(section->mr->ram_addr
> +                                                   + xlat)) {
>               te->addr_write = address | TLB_NOTDIRTY;
>           } else {
>               te->addr_write = address;
> diff --git a/exec.c b/exec.c
> index 6bef3e5..a2f89eb 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1513,7 +1513,7 @@ static void notdirty_mem_write(void *opaque, hwaddr ram_addr,
>       cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_VGA);
>       /* we remove the notdirty callback only if the code has been
>          flushed */
> -    if (cpu_physical_memory_is_dirty(ram_addr)) {
> +    if (!cpu_physical_memory_is_clean(ram_addr)) {
>           CPUArchState *env = current_cpu->env_ptr;
>           tlb_set_dirty(env, env->mem_io_vaddr);
>       }
> @@ -1916,7 +1916,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
>   static void invalidate_and_set_dirty(hwaddr addr,
>                                        hwaddr length)
>   {
> -    if (!cpu_physical_memory_is_dirty(addr)) {
> +    if (cpu_physical_memory_is_clean(addr)) {
>           /* invalidate code */
>           tb_invalidate_phys_page_range(addr, addr + length, 0);
>           /* set dirty bit */
> @@ -2499,7 +2499,7 @@ void stl_phys_notdirty(hwaddr addr, uint32_t val)
>           stl_p(ptr, val);
>
>           if (unlikely(in_migration)) {
> -            if (!cpu_physical_memory_is_dirty(addr1)) {
> +            if (cpu_physical_memory_is_clean(addr1)) {
>                   /* invalidate code */
>                   tb_invalidate_phys_page_range(addr1, addr1 + 4, 0);
>                   /* set dirty bit */
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index e2f55ea..771b23f 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -61,14 +61,13 @@ static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
>       return cpu_physical_memory_get_dirty(addr, 1, client);
>   }
>
> -/* read dirty bit (return 0 or 1) */
> -static inline bool cpu_physical_memory_is_dirty(ram_addr_t addr)
> +static inline bool cpu_physical_memory_is_clean(ram_addr_t addr)
>   {
>       bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
>       bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
>       bool migration =
>           cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> -    return vga && code && migration;
> +    return !(vga && code && migration);
>   }
>
>   static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 28/38] memory: make cpu_physical_memory_reset_dirty() take a length parameter

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> We have an end parameter in all the callers, and this make it coherent
> with the rest of cpu_physical_memory_* functions, that also take a
> length parameter.
>
> Once here, move the start/end calculation to
> tlb_reset_dirty_range_all() as we don't need it here anymore.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   cputlb.c                       |  3 +--
>   exec.c                         | 19 ++++++++-----------
>   include/exec/memory-internal.h |  2 +-
>   memory.c                       |  8 ++------
>   4 files changed, 12 insertions(+), 20 deletions(-)
>
> diff --git a/cputlb.c b/cputlb.c
> index 911d764..a5805e1 100644
> --- a/cputlb.c
> +++ b/cputlb.c
> @@ -127,8 +127,7 @@ void tlb_flush_page(CPUArchState *env, target_ulong addr)
>      can be detected */
>   void tlb_protect_code(ram_addr_t ram_addr)
>   {
> -    cpu_physical_memory_reset_dirty(ram_addr,
> -                                    ram_addr + TARGET_PAGE_SIZE,
> +    cpu_physical_memory_reset_dirty(ram_addr, TARGET_PAGE_SIZE,
>                                       DIRTY_MEMORY_CODE);
>   }
>
> diff --git a/exec.c b/exec.c
> index a2f89eb..ba5989c 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -723,11 +723,14 @@ found:
>       return block;
>   }
>
> -static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,
> -                                      uintptr_t length)
> +static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t length)
>   {
> -    RAMBlock *block;
>       ram_addr_t start1;
> +    RAMBlock *block;
> +    ram_addr_t end;
> +
> +    end = TARGET_PAGE_ALIGN(start + length);
> +    start &= TARGET_PAGE_MASK;
>
>       block = qemu_get_ram_block(start);
>       assert(block == qemu_get_ram_block(end - 1));
> @@ -736,21 +739,15 @@ static void tlb_reset_dirty_range_all(ram_addr_t start, ram_addr_t end,
>   }
>
>   /* Note: start and end must be within the same ram block.  */
> -void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
> +void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
>                                        unsigned client)
>   {
> -    uintptr_t length;
> -
> -    start &= TARGET_PAGE_MASK;
> -    end = TARGET_PAGE_ALIGN(end);
> -
> -    length = end - start;
>       if (length == 0)
>           return;
>       cpu_physical_memory_clear_dirty_range(start, length, client);
>
>       if (tcg_enabled()) {
> -        tlb_reset_dirty_range_all(start, end, length);
> +        tlb_reset_dirty_range_all(start, length);
>       }
>   }
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index 771b23f..cb2249f 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -102,7 +102,7 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
>       bitmap_clear(ram_list.dirty_memory[client], page, end - page);
>   }
>
> -void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t end,
> +void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
>                                        unsigned client);
>
>   #endif
> diff --git a/memory.c b/memory.c
> index a490cbd..c010296 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1191,9 +1191,7 @@ bool memory_region_test_and_clear_dirty(MemoryRegion *mr, hwaddr addr,
>       assert(mr->terminates);
>       ret = cpu_physical_memory_get_dirty(mr->ram_addr + addr, size, client);
>       if (ret) {
> -        cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
> -                                        mr->ram_addr + addr + size,
> -                                        client);
> +        cpu_physical_memory_reset_dirty(mr->ram_addr + addr, size, client);
>       }
>       return ret;
>   }
> @@ -1239,9 +1237,7 @@ void memory_region_reset_dirty(MemoryRegion *mr, hwaddr addr,
>                                  hwaddr size, unsigned client)
>   {
>       assert(mr->terminates);
> -    cpu_physical_memory_reset_dirty(mr->ram_addr + addr,
> -                                    mr->ram_addr + addr + size,
> -                                    client);
> +    cpu_physical_memory_reset_dirty(mr->ram_addr + addr, size, client);
>   }
>
>   void *memory_region_get_ram_ptr(MemoryRegion *mr)
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 29/38] memory: cpu_physical_memory_set_dirty_tracking() should return void

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> Result was always 0, and not used anywhere.  Once there, use bool type
> for the parameter.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
>   exec.c | 10 ++++------
>   1 file changed, 4 insertions(+), 6 deletions(-)
>
> diff --git a/exec.c b/exec.c
> index ba5989c..d165fd8 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -56,7 +56,7 @@
>   //#define DEBUG_SUBPAGE
>
>   #if !defined(CONFIG_USER_ONLY)
> -static int in_migration;
> +static bool in_migration;
>
>   RAMList ram_list = { .blocks = QTAILQ_HEAD_INITIALIZER(ram_list.blocks) };
>
> @@ -751,11 +751,9 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
>       }
>   }
>
> -static int cpu_physical_memory_set_dirty_tracking(int enable)
> +static void cpu_physical_memory_set_dirty_tracking(bool enable)
>   {
> -    int ret = 0;
>       in_migration = enable;
> -    return ret;
>   }
>
>   hwaddr memory_region_section_get_iotlb(CPUArchState *env,
> @@ -1797,12 +1795,12 @@ static void tcg_commit(MemoryListener *listener)
>
>   static void core_log_global_start(MemoryListener *listener)
>   {
> -    cpu_physical_memory_set_dirty_tracking(1);
> +    cpu_physical_memory_set_dirty_tracking(true);
>   }
>
>   static void core_log_global_stop(MemoryListener *listener)
>   {
> -    cpu_physical_memory_set_dirty_tracking(0);
> +    cpu_physical_memory_set_dirty_tracking(false);
>   }
>
>   static MemoryListener core_memory_listener = {
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 30/38] memory: split cpu_physical_memory_* functions to its own include

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> All the functions that use ram_addr_t should be here.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   cputlb.c                       |  1 +
>   exec.c                         |  1 +
>   include/exec/memory-internal.h | 76 --------------------------------
>   include/exec/ram_addr.h        | 98 ++++++++++++++++++++++++++++++++++++++++++
>   memory.c                       |  1 +
>   5 files changed, 101 insertions(+), 76 deletions(-)
>   create mode 100644 include/exec/ram_addr.h
>
> diff --git a/cputlb.c b/cputlb.c
> index a5805e1..a1e8421 100644
> --- a/cputlb.c
> +++ b/cputlb.c
> @@ -26,6 +26,7 @@
>   #include "exec/cputlb.h"
>
>   #include "exec/memory-internal.h"
> +#include "exec/ram_addr.h"
>
>   //#define DEBUG_TLB
>   //#define DEBUG_TLB_CHECK
> diff --git a/exec.c b/exec.c
> index d165fd8..be421e5 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -50,6 +50,7 @@
>   #include "translate-all.h"
>
>   #include "exec/memory-internal.h"
> +#include "exec/ram_addr.h"
>
>   #include "qemu/range.h"
>
> diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h
> index cb2249f..25c43c0 100644
> --- a/include/exec/memory-internal.h
> +++ b/include/exec/memory-internal.h
> @@ -20,9 +20,6 @@
>   #define MEMORY_INTERNAL_H
>
>   #ifndef CONFIG_USER_ONLY
> -#include "hw/xen/xen.h"
> -
> -
>   typedef struct AddressSpaceDispatch AddressSpaceDispatch;
>
>   void address_space_init_dispatch(AddressSpace *as);
> @@ -33,78 +30,5 @@ extern const MemoryRegionOps unassigned_mem_ops;
>   bool memory_region_access_valid(MemoryRegion *mr, hwaddr addr,
>                                   unsigned size, bool is_write);
>
> -ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
> -                                   MemoryRegion *mr);
> -ram_addr_t qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr);
> -void *qemu_get_ram_ptr(ram_addr_t addr);
> -void qemu_ram_free(ram_addr_t addr);
> -void qemu_ram_free_from_ptr(ram_addr_t addr);
> -
> -static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
> -                                                 ram_addr_t length,
> -                                                 unsigned client)
> -{
> -    unsigned long end, page, next;
> -
> -    assert(client < DIRTY_MEMORY_NUM);
> -
> -    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> -    page = start >> TARGET_PAGE_BITS;
> -    next = find_next_bit(ram_list.dirty_memory[client], end, page);
> -
> -    return next < end;
> -}
> -
> -static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
> -                                                      unsigned client)
> -{
> -    return cpu_physical_memory_get_dirty(addr, 1, client);
> -}
> -
> -static inline bool cpu_physical_memory_is_clean(ram_addr_t addr)
> -{
> -    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
> -    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
> -    bool migration =
> -        cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> -    return !(vga && code && migration);
> -}
> -
> -static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
> -                                                      unsigned client)
> -{
> -    assert(client < DIRTY_MEMORY_NUM);
> -    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> -}
> -
> -static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
> -                                                       ram_addr_t length)
> -{
> -    unsigned long end, page;
> -
> -    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> -    page = start >> TARGET_PAGE_BITS;
> -    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
> -    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
> -    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
> -    xen_modified_memory(start, length);
> -}
> -
> -static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
> -                                                         ram_addr_t length,
> -                                                         unsigned client)
> -{
> -    unsigned long end, page;
> -
> -    assert(client < DIRTY_MEMORY_NUM);
> -    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> -    page = start >> TARGET_PAGE_BITS;
> -    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
> -}
> -
> -void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
> -                                     unsigned client);
> -
>   #endif
> -
>   #endif
> diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
> new file mode 100644
> index 0000000..db977fb
> --- /dev/null
> +++ b/include/exec/ram_addr.h
> @@ -0,0 +1,98 @@
> +/*
> + * Declarations for cpu physical memory functions
> + *
> + * Copyright 2011 Red Hat, Inc. and/or its affiliates
> + *
> + * Authors:
> + *  Avi Kivity <avi@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + *
> + */
> +
> +/*
> + * This header is for use by exec.c and memory.c ONLY.  Do not include it.
> + * The functions declared here will be removed soon.
> + */
> +
> +#ifndef RAM_ADDR_H
> +#define RAM_ADDR_H
> +
> +#ifndef CONFIG_USER_ONLY
> +#include "hw/xen/xen.h"
> +
> +ram_addr_t qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
> +                                   MemoryRegion *mr);
> +ram_addr_t qemu_ram_alloc(ram_addr_t size, MemoryRegion *mr);
> +void *qemu_get_ram_ptr(ram_addr_t addr);
> +void qemu_ram_free(ram_addr_t addr);
> +void qemu_ram_free_from_ptr(ram_addr_t addr);
> +
> +static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
> +                                                 ram_addr_t length,
> +                                                 unsigned client)
> +{
> +    unsigned long end, page, next;
> +
> +    assert(client < DIRTY_MEMORY_NUM);
> +
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    next = find_next_bit(ram_list.dirty_memory[client], end, page);
> +
> +    return next < end;
> +}
> +
> +static inline bool cpu_physical_memory_get_dirty_flag(ram_addr_t addr,
> +                                                      unsigned client)
> +{
> +    return cpu_physical_memory_get_dirty(addr, 1, client);
> +}
> +
> +static inline bool cpu_physical_memory_is_clean(ram_addr_t addr)
> +{
> +    bool vga = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_VGA);
> +    bool code = cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_CODE);
> +    bool migration =
> +        cpu_physical_memory_get_dirty_flag(addr, DIRTY_MEMORY_MIGRATION);
> +    return !(vga && code && migration);
> +}
> +
> +static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
> +                                                      unsigned client)
> +{
> +    assert(client < DIRTY_MEMORY_NUM);
> +    set_bit(addr >> TARGET_PAGE_BITS, ram_list.dirty_memory[client]);
> +}
> +
> +static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
> +                                                       ram_addr_t length)
> +{
> +    unsigned long end, page;
> +
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION], page, end - page);
> +    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_VGA], page, end - page);
> +    bitmap_set(ram_list.dirty_memory[DIRTY_MEMORY_CODE], page, end - page);
> +    xen_modified_memory(start, length);
> +}
> +
> +static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
> +                                                         ram_addr_t length,
> +                                                         unsigned client)
> +{
> +    unsigned long end, page;
> +
> +    assert(client < DIRTY_MEMORY_NUM);
> +    end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS;
> +    page = start >> TARGET_PAGE_BITS;
> +    bitmap_clear(ram_list.dirty_memory[client], page, end - page);
> +}
> +
> +void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length,
> +                                     unsigned client);
> +
> +#endif
> +#endif
> diff --git a/memory.c b/memory.c
> index c010296..59ecc28 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -22,6 +22,7 @@
>   #include <assert.h>
>
>   #include "exec/memory-internal.h"
> +#include "exec/ram_addr.h"
>
>   //#define DEBUG_UNASSIGNED
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 31/38] memory: unfold memory_region_test_and_clear()

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> We are going to update the bitmap directly
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   arch_init.c | 10 +++++++---
>   1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/arch_init.c b/arch_init.c
> index e0acbc5..0e8c8b5 100644
> --- a/arch_init.c
> +++ b/arch_init.c
> @@ -48,6 +48,7 @@
>   #include "qmp-commands.h"
>   #include "trace.h"
>   #include "exec/cpu-all.h"
> +#include "exec/ram_addr.h"
>   #include "hw/acpi/acpi.h"
>
>   #ifdef DEBUG_ARCH_INIT
> @@ -400,9 +401,12 @@ static void migration_bitmap_sync(void)
>
>       QTAILQ_FOREACH(block, &ram_list.blocks, next) {
>           for (addr = 0; addr < block->length; addr += TARGET_PAGE_SIZE) {
> -            if (memory_region_test_and_clear_dirty(block->mr,
> -                                                   addr, TARGET_PAGE_SIZE,
> -                                                   DIRTY_MEMORY_MIGRATION)) {
> +            if (cpu_physical_memory_get_dirty(block->mr->ram_addr + addr,
> +                                              TARGET_PAGE_SIZE,
> +                                              DIRTY_MEMORY_MIGRATION)) {
> +                cpu_physical_memory_reset_dirty(block->mr->ram_addr + addr,
> +                                                TARGET_PAGE_SIZE,
> +                                                DIRTY_MEMORY_MIGRATION);
>                   migration_bitmap_set_dirty(block->mr, addr);
>               }
>           }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 32/38] kvm: use directly cpu_physical_memory_* api for tracking dirty pages

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> Performance is important in this function, and we want to optimize even further.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   kvm-all.c | 7 +++++--
>   1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/kvm-all.c b/kvm-all.c
> index 3937754..308dfba 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -31,6 +31,7 @@
>   #include "sysemu/kvm.h"
>   #include "qemu/bswap.h"
>   #include "exec/memory.h"
> +#include "exec/ram_addr.h"
>   #include "exec/address-spaces.h"
>   #include "qemu/event_notifier.h"
>   #include "trace.h"
> @@ -382,6 +383,7 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>       unsigned int i, j;
>       unsigned long page_number, c;
>       hwaddr addr, addr1;
> +    ram_addr_t ram_addr;
>       unsigned int pages = int128_get64(section->size) / getpagesize();
>       unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
>       unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
> @@ -399,8 +401,9 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>                   page_number = (i * HOST_LONG_BITS + j) * hpratio;
>                   addr1 = page_number * TARGET_PAGE_SIZE;
>                   addr = section->offset_within_region + addr1;
> -                memory_region_set_dirty(section->mr, addr,
> -                                        TARGET_PAGE_SIZE * hpratio);
> +                ram_addr = section->mr->ram_addr + addr;
> +                cpu_physical_memory_set_dirty_range(ram_addr,
> +                                                    TARGET_PAGE_SIZE * hpratio);
>               } while (c != 0);
>           }
>       }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 33/38] kvm: refactor start address calculation

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   kvm-all.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/kvm-all.c b/kvm-all.c
> index 308dfba..cb62ba4 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -382,7 +382,8 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>   {
>       unsigned int i, j;
>       unsigned long page_number, c;
> -    hwaddr addr, addr1;
> +    hwaddr addr;
> +    ram_addr_t start = section->offset_within_region + section->mr->ram_addr;
>       ram_addr_t ram_addr;
>       unsigned int pages = int128_get64(section->size) / getpagesize();
>       unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
> @@ -399,9 +400,8 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>                   j = ffsl(c) - 1;
>                   c &= ~(1ul << j);
>                   page_number = (i * HOST_LONG_BITS + j) * hpratio;
> -                addr1 = page_number * TARGET_PAGE_SIZE;
> -                addr = section->offset_within_region + addr1;
> -                ram_addr = section->mr->ram_addr + addr;
> +                addr = page_number * TARGET_PAGE_SIZE;
> +                ram_addr = start + addr;
>                   cpu_physical_memory_set_dirty_range(ram_addr,
>                                                       TARGET_PAGE_SIZE * hpratio);
>               } while (c != 0);
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 32/38] kvm: use directly cpu_physical_memory_* api for tracking dirty pages

[Peter Maydell]

On 17 December 2013 15:26, Juan Quintela <quintela@redhat.com> wrote:
> @@ -399,8 +401,9 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>                  page_number = (i * HOST_LONG_BITS + j) * hpratio;
>                  addr1 = page_number * TARGET_PAGE_SIZE;
>                  addr = section->offset_within_region + addr1;
> -                memory_region_set_dirty(section->mr, addr,
> -                                        TARGET_PAGE_SIZE * hpratio);
> +                ram_addr = section->mr->ram_addr + addr;

struct MemoryRegion says:
    /* All fields are private - violators will be prosecuted */

so fishing around in it for mr->ram_addr seems like a bad idea.

Perhaps we could make memory_region_set_dirty() an inline
function in memory.h instead?

thanks
-- PMM

Re: [Qemu-devel] [PATCH 34/38] memory: move bitmap synchronization to its own function

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> We want to have all the functions that handle directly the dirty
> bitmap near.  We will change it later.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   include/exec/ram_addr.h | 31 +++++++++++++++++++++++++++++++
>   kvm-all.c               | 27 ++-------------------------
>   2 files changed, 33 insertions(+), 25 deletions(-)
>
> diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
> index db977fb..c6736ed 100644
> --- a/include/exec/ram_addr.h
> +++ b/include/exec/ram_addr.h
> @@ -79,6 +79,37 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
>       xen_modified_memory(start, length);
>   }
>
> +static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
> +                                                          ram_addr_t start,
> +                                                          ram_addr_t pages)
> +{
> +    unsigned int i, j;
> +    unsigned long page_number, c;
> +    hwaddr addr;
> +    ram_addr_t ram_addr;
> +    unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
> +    unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
> +
> +    /*
> +     * bitmap-traveling is faster than memory-traveling (for addr...)
> +     * especially when most of the memory is not dirty.
> +     */
> +    for (i = 0; i < len; i++) {
> +        if (bitmap[i] != 0) {
> +            c = leul_to_cpu(bitmap[i]);
> +            do {
> +                j = ffsl(c) - 1;
> +                c &= ~(1ul << j);
> +                page_number = (i * HOST_LONG_BITS + j) * hpratio;
> +                addr = page_number * TARGET_PAGE_SIZE;
> +                ram_addr = start + addr;
> +                cpu_physical_memory_set_dirty_range(ram_addr,
> +                                                    TARGET_PAGE_SIZE * hpratio);
> +            } while (c != 0);
> +        }
> +    }
> +}
> +
>   static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
>                                                            ram_addr_t length,
>                                                            unsigned client)
> diff --git a/kvm-all.c b/kvm-all.c
> index cb62ba4..0bfb060 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -380,33 +380,10 @@ static int kvm_set_migration_log(int enable)
>   static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>                                            unsigned long *bitmap)
>   {
> -    unsigned int i, j;
> -    unsigned long page_number, c;
> -    hwaddr addr;
>       ram_addr_t start = section->offset_within_region + section->mr->ram_addr;
> -    ram_addr_t ram_addr;
> -    unsigned int pages = int128_get64(section->size) / getpagesize();
> -    unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
> -    unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
> +    ram_addr_t pages = int128_get64(section->size) / getpagesize();
>
> -    /*
> -     * bitmap-traveling is faster than memory-traveling (for addr...)
> -     * especially when most of the memory is not dirty.
> -     */
> -    for (i = 0; i < len; i++) {
> -        if (bitmap[i] != 0) {
> -            c = leul_to_cpu(bitmap[i]);
> -            do {
> -                j = ffsl(c) - 1;
> -                c &= ~(1ul << j);
> -                page_number = (i * HOST_LONG_BITS + j) * hpratio;
> -                addr = page_number * TARGET_PAGE_SIZE;
> -                ram_addr = start + addr;
> -                cpu_physical_memory_set_dirty_range(ram_addr,
> -                                                    TARGET_PAGE_SIZE * hpratio);
> -            } while (c != 0);
> -        }
> -    }
> +    cpu_physical_memory_set_dirty_lebitmap(bitmap, start, pages);
>       return 0;
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 35/38] memory: syncronize kvm bitmap using bitmaps operations

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> If bitmaps are aligned properly, use bitmap operations.  If they are
> not, just use old bit at a time code.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   include/exec/ram_addr.h | 54 ++++++++++++++++++++++++++++++++-----------------
>   1 file changed, 36 insertions(+), 18 deletions(-)
>
> diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
> index c6736ed..33c8acc 100644
> --- a/include/exec/ram_addr.h
> +++ b/include/exec/ram_addr.h
> @@ -83,29 +83,47 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
>                                                             ram_addr_t start,
>                                                             ram_addr_t pages)
>   {
> -    unsigned int i, j;
> +    unsigned long i, j;
>       unsigned long page_number, c;
>       hwaddr addr;
>       ram_addr_t ram_addr;
> -    unsigned int len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
> +    unsigned long len = (pages + HOST_LONG_BITS - 1) / HOST_LONG_BITS;
>       unsigned long hpratio = getpagesize() / TARGET_PAGE_SIZE;
> +    unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
>
> -    /*
> -     * bitmap-traveling is faster than memory-traveling (for addr...)
> -     * especially when most of the memory is not dirty.
> -     */
> -    for (i = 0; i < len; i++) {
> -        if (bitmap[i] != 0) {
> -            c = leul_to_cpu(bitmap[i]);
> -            do {
> -                j = ffsl(c) - 1;
> -                c &= ~(1ul << j);
> -                page_number = (i * HOST_LONG_BITS + j) * hpratio;
> -                addr = page_number * TARGET_PAGE_SIZE;
> -                ram_addr = start + addr;
> -                cpu_physical_memory_set_dirty_range(ram_addr,
> -                                                    TARGET_PAGE_SIZE * hpratio);
> -            } while (c != 0);
> +    /* start address is aligned at the start of a word? */
> +    if (((page * BITS_PER_LONG) << TARGET_PAGE_BITS) == start) {
> +        long k;
> +        long nr = BITS_TO_LONGS(pages);
> +
> +        for (k = 0; k < nr; k++) {
> +            if (bitmap[k]) {
> +                unsigned long temp = leul_to_cpu(bitmap[k]);
> +
> +                ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION][page + k] |= temp;
> +                ram_list.dirty_memory[DIRTY_MEMORY_VGA][page + k] |= temp;
> +                ram_list.dirty_memory[DIRTY_MEMORY_CODE][page + k] |= temp;
> +            }
> +        }
> +        xen_modified_memory(start, pages);
> +    } else {
> +        /*
> +         * bitmap-traveling is faster than memory-traveling (for addr...)
> +         * especially when most of the memory is not dirty.
> +         */
> +        for (i = 0; i < len; i++) {
> +            if (bitmap[i] != 0) {
> +                c = leul_to_cpu(bitmap[i]);
> +                do {
> +                    j = ffsl(c) - 1;
> +                    c &= ~(1ul << j);
> +                    page_number = (i * HOST_LONG_BITS + j) * hpratio;
> +                    addr = page_number * TARGET_PAGE_SIZE;
> +                    ram_addr = start + addr;
> +                    cpu_physical_memory_set_dirty_range(ram_addr,
> +                                       TARGET_PAGE_SIZE * hpratio);
> +                } while (c != 0);
> +            }
>           }
>       }
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 36/38] ram: split function that synchronizes a range

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> This function is the only bit where we care about speed.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   arch_init.c | 34 ++++++++++++++++++++--------------
>   1 file changed, 20 insertions(+), 14 deletions(-)
>
> diff --git a/arch_init.c b/arch_init.c
> index 0e8c8b5..2cd3d00 100644
> --- a/arch_init.c
> +++ b/arch_init.c
> @@ -360,11 +360,10 @@ ram_addr_t migration_bitmap_find_and_reset_dirty(MemoryRegion *mr,
>       return (next - base) << TARGET_PAGE_BITS;
>   }
>
> -static inline bool migration_bitmap_set_dirty(MemoryRegion *mr,
> -                                              ram_addr_t offset)
> +static inline bool migration_bitmap_set_dirty(ram_addr_t addr)
>   {
>       bool ret;
> -    int nr = (mr->ram_addr + offset) >> TARGET_PAGE_BITS;
> +    int nr = addr >> TARGET_PAGE_BITS;
>
>       ret = test_and_set_bit(nr, migration_bitmap);
>
> @@ -374,12 +373,28 @@ static inline bool migration_bitmap_set_dirty(MemoryRegion *mr,
>       return ret;
>   }
>
> +static void migration_bitmap_sync_range(ram_addr_t start, ram_addr_t length)
> +{
> +    ram_addr_t addr;
> +
> +    for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
> +        if (cpu_physical_memory_get_dirty(start + addr,
> +                                          TARGET_PAGE_SIZE,
> +                                          DIRTY_MEMORY_MIGRATION)) {
> +            cpu_physical_memory_reset_dirty(start + addr,
> +                                            TARGET_PAGE_SIZE,
> +                                            DIRTY_MEMORY_MIGRATION);
> +            migration_bitmap_set_dirty(start + addr);
> +        }
> +    }
> +}
> +
> +
>   /* Needs iothread lock! */
>
>   static void migration_bitmap_sync(void)
>   {
>       RAMBlock *block;
> -    ram_addr_t addr;
>       uint64_t num_dirty_pages_init = migration_dirty_pages;
>       MigrationState *s = migrate_get_current();
>       static int64_t start_time;
> @@ -400,16 +415,7 @@ static void migration_bitmap_sync(void)
>       address_space_sync_dirty_bitmap(&address_space_memory);
>
>       QTAILQ_FOREACH(block, &ram_list.blocks, next) {
> -        for (addr = 0; addr < block->length; addr += TARGET_PAGE_SIZE) {
> -            if (cpu_physical_memory_get_dirty(block->mr->ram_addr + addr,
> -                                              TARGET_PAGE_SIZE,
> -                                              DIRTY_MEMORY_MIGRATION)) {
> -                cpu_physical_memory_reset_dirty(block->mr->ram_addr + addr,
> -                                                TARGET_PAGE_SIZE,
> -                                                DIRTY_MEMORY_MIGRATION);
> -                migration_bitmap_set_dirty(block->mr, addr);
> -            }
> -        }
> +        migration_bitmap_sync_range(block->mr->ram_addr, block->length);
>       }
>       trace_migration_bitmap_sync_end(migration_dirty_pages
>                                       - num_dirty_pages_init);
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 37/38] migration: synchronize memory bitmap 64bits at a time

[Orit Wasserman]

On 12/17/2013 05:26 PM, Juan Quintela wrote:
> We use the old code if the bitmaps are not aligned
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   arch_init.c | 38 +++++++++++++++++++++++++++++---------
>   1 file changed, 29 insertions(+), 9 deletions(-)
>
> diff --git a/arch_init.c b/arch_init.c
> index 2cd3d00..77912e7 100644
> --- a/arch_init.c
> +++ b/arch_init.c
> @@ -50,6 +50,7 @@
>   #include "exec/cpu-all.h"
>   #include "exec/ram_addr.h"
>   #include "hw/acpi/acpi.h"
> +#include "qemu/host-utils.h"
>
>   #ifdef DEBUG_ARCH_INIT
>   #define DPRINTF(fmt, ...) \
> @@ -376,15 +377,34 @@ static inline bool migration_bitmap_set_dirty(ram_addr_t addr)
>   static void migration_bitmap_sync_range(ram_addr_t start, ram_addr_t length)
>   {
>       ram_addr_t addr;
> -
> -    for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
> -        if (cpu_physical_memory_get_dirty(start + addr,
> -                                          TARGET_PAGE_SIZE,
> -                                          DIRTY_MEMORY_MIGRATION)) {
> -            cpu_physical_memory_reset_dirty(start + addr,
> -                                            TARGET_PAGE_SIZE,
> -                                            DIRTY_MEMORY_MIGRATION);
> -            migration_bitmap_set_dirty(start + addr);
> +    unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
> +
> +    /* start address is aligned at the start of a word? */
> +    if (((page * BITS_PER_LONG) << TARGET_PAGE_BITS) == start) {
> +        int k;
> +        int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
> +        unsigned long *src = ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION];
> +
> +        for (k = page; k < page + nr; k++) {
> +            if (src[k]) {
> +                unsigned long new_dirty;
> +                new_dirty = ~migration_bitmap[k];
> +                migration_bitmap[k] |= src[k];
> +                new_dirty &= src[k];
> +                migration_dirty_pages += ctpopl(new_dirty);
> +                src[k] = 0;
> +            }
> +        }
> +    } else {
> +        for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
> +            if (cpu_physical_memory_get_dirty(start + addr,
> +                                              TARGET_PAGE_SIZE,
> +                                              DIRTY_MEMORY_MIGRATION)) {
> +                cpu_physical_memory_reset_dirty(start + addr,
> +                                                TARGET_PAGE_SIZE,
> +                                                DIRTY_MEMORY_MIGRATION);
> +                migration_bitmap_set_dirty(start + addr);
> +            }
>           }
>       }
>   }
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Re: [Qemu-devel] [PATCH 32/38] kvm: use directly cpu_physical_memory_* api for tracking dirty pages

[Juan Quintela]

Peter Maydell <peter.maydell@linaro.org> wrote:
> On 17 December 2013 15:26, Juan Quintela <quintela@redhat.com> wrote:
>> @@ -399,8 +401,9 @@ static int
>> kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
>>                  page_number = (i * HOST_LONG_BITS + j) * hpratio;
>>                  addr1 = page_number * TARGET_PAGE_SIZE;
>>                  addr = section->offset_within_region + addr1;
>> -                memory_region_set_dirty(section->mr, addr,
>> -                                        TARGET_PAGE_SIZE * hpratio);
>> +                ram_addr = section->mr->ram_addr + addr;
>
> struct MemoryRegion says:
>     /* All fields are private - violators will be prosecuted */

good spot

> so fishing around in it for mr->ram_addr seems like a bad idea.
>
> Perhaps we could make memory_region_set_dirty() an inline
> function in memory.h instead?


But we already use it on two places on arch_init.c.  For instance:


static inline
ram_addr_t migration_bitmap_find_and_reset_dirty(MemoryRegion *mr,
                                                 ram_addr_t start)
{
    unsigned long base = mr->ram_addr >> TARGET_PAGE_BITS;


I tried to do the bitmap optimizations using memory_regions, and it got
me nowhere.  Basically we have:
- TCG: _knows_ how memory bitmap/regions/etc work (actually, never got
  ported to MemoryRegions)
- Migration:  We are weird, and we _know_ about ram_addr_t, and we only
  use it, we don't care about memory regions.

On this series we basicall went the other way around: we created
include/exec/ram_addr.h, and we made _both_ migration and memory_region
work on top of it.

Later, Juan.

Re: [Qemu-devel] [PATCH 01/38] bitmap: use long as index

[Orit Wasserman]

On 12/17/2013 05:25 PM, Juan Quintela wrote:
> Move index and size fields from int to long.  We need that for
> migration.  long is 64 bits on sane architectures, and 32bits should
> be enough on all the 32bits architectures.
>
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>   include/qemu/bitmap.h | 77 ++++++++++++++++++++++++++-------------------------
>   include/qemu/bitops.h | 14 +++++-----
>   util/bitmap.c         | 60 +++++++++++++++++++--------------------
>   3 files changed, 76 insertions(+), 75 deletions(-)
>
> diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
> index 308bbb7..afdd257 100644
> --- a/include/qemu/bitmap.h
> +++ b/include/qemu/bitmap.h
> @@ -31,7 +31,7 @@
>    * bitmap_andnot(dst, src1, src2, nbits)	*dst = *src1 & ~(*src2)
>    * bitmap_complement(dst, src, nbits)		*dst = ~(*src)
>    * bitmap_equal(src1, src2, nbits)		Are *src1 and *src2 equal?
> - * bitmap_intersects(src1, src2, nbits) 	Do *src1 and *src2 overlap?
> + * bitmap_intersects(src1, src2, nbits)         Do *src1 and *src2 overlap?
>    * bitmap_empty(src, nbits)			Are all bits zero in *src?
>    * bitmap_full(src, nbits)			Are all bits set in *src?
>    * bitmap_set(dst, pos, nbits)			Set specified bit area
> @@ -62,71 +62,71 @@
>           )
>
>   #define DECLARE_BITMAP(name,bits)                  \
> -	unsigned long name[BITS_TO_LONGS(bits)]
> +        unsigned long name[BITS_TO_LONGS(bits)]
>
>   #define small_nbits(nbits)                      \
> -	((nbits) <= BITS_PER_LONG)
> +        ((nbits) <= BITS_PER_LONG)
>
> -int slow_bitmap_empty(const unsigned long *bitmap, int bits);
> -int slow_bitmap_full(const unsigned long *bitmap, int bits);
> +int slow_bitmap_empty(const unsigned long *bitmap, long bits);
> +int slow_bitmap_full(const unsigned long *bitmap, long bits);
>   int slow_bitmap_equal(const unsigned long *bitmap1,
> -                   const unsigned long *bitmap2, int bits);
> +                      const unsigned long *bitmap2, long bits);
>   void slow_bitmap_complement(unsigned long *dst, const unsigned long *src,
> -                         int bits);
> +                            long bits);
>   void slow_bitmap_shift_right(unsigned long *dst,
> -                          const unsigned long *src, int shift, int bits);
> +                             const unsigned long *src, int shift, long bits);
>   void slow_bitmap_shift_left(unsigned long *dst,
> -                         const unsigned long *src, int shift, int bits);
> +                            const unsigned long *src, int shift, long bits);
>   int slow_bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
> -                 const unsigned long *bitmap2, int bits);
> +                    const unsigned long *bitmap2, long bits);
>   void slow_bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
> -                 const unsigned long *bitmap2, int bits);
> +                    const unsigned long *bitmap2, long bits);
>   void slow_bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
> -                  const unsigned long *bitmap2, int bits);
> +                     const unsigned long *bitmap2, long bits);
>   int slow_bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
> -                    const unsigned long *bitmap2, int bits);
> +                       const unsigned long *bitmap2, long bits);
>   int slow_bitmap_intersects(const unsigned long *bitmap1,
> -			const unsigned long *bitmap2, int bits);
> +                           const unsigned long *bitmap2, long bits);
>
> -static inline unsigned long *bitmap_new(int nbits)
> +static inline unsigned long *bitmap_new(long nbits)
>   {
> -    int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
> +    long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
>       return g_malloc0(len);
>   }
>
> -static inline void bitmap_zero(unsigned long *dst, int nbits)
> +static inline void bitmap_zero(unsigned long *dst, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           *dst = 0UL;
>       } else {
> -        int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
> +        long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
>           memset(dst, 0, len);
>       }
>   }
>
> -static inline void bitmap_fill(unsigned long *dst, int nbits)
> +static inline void bitmap_fill(unsigned long *dst, long nbits)
>   {
>       size_t nlongs = BITS_TO_LONGS(nbits);
>       if (!small_nbits(nbits)) {
> -        int len = (nlongs - 1) * sizeof(unsigned long);
> +        long len = (nlongs - 1) * sizeof(unsigned long);
>           memset(dst, 0xff,  len);
>       }
>       dst[nlongs - 1] = BITMAP_LAST_WORD_MASK(nbits);
>   }
>
>   static inline void bitmap_copy(unsigned long *dst, const unsigned long *src,
> -                               int nbits)
> +                               long nbits)
>   {
>       if (small_nbits(nbits)) {
>           *dst = *src;
>       } else {
> -        int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
> +        long len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
>           memcpy(dst, src, len);
>       }
>   }
>
>   static inline int bitmap_and(unsigned long *dst, const unsigned long *src1,
> -                             const unsigned long *src2, int nbits)
> +                             const unsigned long *src2, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           return (*dst = *src1 & *src2) != 0;
> @@ -135,7 +135,7 @@ static inline int bitmap_and(unsigned long *dst, const unsigned long *src1,
>   }
>
>   static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
> -			const unsigned long *src2, int nbits)
> +                             const unsigned long *src2, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           *dst = *src1 | *src2;
> @@ -145,7 +145,7 @@ static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
>   }
>
>   static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1,
> -			const unsigned long *src2, int nbits)
> +                              const unsigned long *src2, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           *dst = *src1 ^ *src2;
> @@ -155,7 +155,7 @@ static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1,
>   }
>
>   static inline int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
> -			const unsigned long *src2, int nbits)
> +                                const unsigned long *src2, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           return (*dst = *src1 & ~(*src2)) != 0;
> @@ -163,8 +163,9 @@ static inline int bitmap_andnot(unsigned long *dst, const unsigned long *src1,
>       return slow_bitmap_andnot(dst, src1, src2, nbits);
>   }
>
> -static inline void bitmap_complement(unsigned long *dst, const unsigned long *src,
> -			int nbits)
> +static inline void bitmap_complement(unsigned long *dst,
> +                                     const unsigned long *src,
> +                                     long nbits)
>   {
>       if (small_nbits(nbits)) {
>           *dst = ~(*src) & BITMAP_LAST_WORD_MASK(nbits);
> @@ -174,7 +175,7 @@ static inline void bitmap_complement(unsigned long *dst, const unsigned long *sr
>   }
>
>   static inline int bitmap_equal(const unsigned long *src1,
> -			const unsigned long *src2, int nbits)
> +                               const unsigned long *src2, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           return ! ((*src1 ^ *src2) & BITMAP_LAST_WORD_MASK(nbits));
> @@ -183,7 +184,7 @@ static inline int bitmap_equal(const unsigned long *src1,
>       }
>   }
>
> -static inline int bitmap_empty(const unsigned long *src, int nbits)
> +static inline int bitmap_empty(const unsigned long *src, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           return ! (*src & BITMAP_LAST_WORD_MASK(nbits));
> @@ -192,7 +193,7 @@ static inline int bitmap_empty(const unsigned long *src, int nbits)
>       }
>   }
>
> -static inline int bitmap_full(const unsigned long *src, int nbits)
> +static inline int bitmap_full(const unsigned long *src, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           return ! (~(*src) & BITMAP_LAST_WORD_MASK(nbits));
> @@ -202,7 +203,7 @@ static inline int bitmap_full(const unsigned long *src, int nbits)
>   }
>
>   static inline int bitmap_intersects(const unsigned long *src1,
> -			const unsigned long *src2, int nbits)
> +                                    const unsigned long *src2, long nbits)
>   {
>       if (small_nbits(nbits)) {
>           return ((*src1 & *src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0;
> @@ -211,12 +212,12 @@ static inline int bitmap_intersects(const unsigned long *src1,
>       }
>   }
>
> -void bitmap_set(unsigned long *map, int i, int len);
> -void bitmap_clear(unsigned long *map, int start, int nr);
> +void bitmap_set(unsigned long *map, long i, long len);
> +void bitmap_clear(unsigned long *map, long start, long nr);
>   unsigned long bitmap_find_next_zero_area(unsigned long *map,
> -					 unsigned long size,
> -					 unsigned long start,
> -					 unsigned int nr,
> -					 unsigned long align_mask);
> +                                         unsigned long size,
> +                                         unsigned long start,
> +                                         unsigned long nr,
> +                                         unsigned long align_mask);
>
>   #endif /* BITMAP_H */
> diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h
> index 304c90c..340b1e7 100644
> --- a/include/qemu/bitops.h
> +++ b/include/qemu/bitops.h
> @@ -28,7 +28,7 @@
>    * @nr: the bit to set
>    * @addr: the address to start counting from
>    */
> -static inline void set_bit(int nr, unsigned long *addr)
> +static inline void set_bit(long nr, unsigned long *addr)
>   {
>   	unsigned long mask = BIT_MASK(nr);
>           unsigned long *p = addr + BIT_WORD(nr);
> @@ -41,7 +41,7 @@ static inline void set_bit(int nr, unsigned long *addr)
>    * @nr: Bit to clear
>    * @addr: Address to start counting from
>    */
> -static inline void clear_bit(int nr, unsigned long *addr)
> +static inline void clear_bit(long nr, unsigned long *addr)
>   {
>   	unsigned long mask = BIT_MASK(nr);
>           unsigned long *p = addr + BIT_WORD(nr);
> @@ -54,7 +54,7 @@ static inline void clear_bit(int nr, unsigned long *addr)
>    * @nr: Bit to change
>    * @addr: Address to start counting from
>    */
> -static inline void change_bit(int nr, unsigned long *addr)
> +static inline void change_bit(long nr, unsigned long *addr)
>   {
>   	unsigned long mask = BIT_MASK(nr);
>           unsigned long *p = addr + BIT_WORD(nr);
> @@ -67,7 +67,7 @@ static inline void change_bit(int nr, unsigned long *addr)
>    * @nr: Bit to set
>    * @addr: Address to count from
>    */
> -static inline int test_and_set_bit(int nr, unsigned long *addr)
> +static inline int test_and_set_bit(long nr, unsigned long *addr)
>   {
>   	unsigned long mask = BIT_MASK(nr);
>           unsigned long *p = addr + BIT_WORD(nr);
> @@ -82,7 +82,7 @@ static inline int test_and_set_bit(int nr, unsigned long *addr)
>    * @nr: Bit to clear
>    * @addr: Address to count from
>    */
> -static inline int test_and_clear_bit(int nr, unsigned long *addr)
> +static inline int test_and_clear_bit(long nr, unsigned long *addr)
>   {
>   	unsigned long mask = BIT_MASK(nr);
>           unsigned long *p = addr + BIT_WORD(nr);
> @@ -97,7 +97,7 @@ static inline int test_and_clear_bit(int nr, unsigned long *addr)
>    * @nr: Bit to change
>    * @addr: Address to count from
>    */
> -static inline int test_and_change_bit(int nr, unsigned long *addr)
> +static inline int test_and_change_bit(long nr, unsigned long *addr)
>   {
>   	unsigned long mask = BIT_MASK(nr);
>           unsigned long *p = addr + BIT_WORD(nr);
> @@ -112,7 +112,7 @@ static inline int test_and_change_bit(int nr, unsigned long *addr)
>    * @nr: bit number to test
>    * @addr: Address to start counting from
>    */
> -static inline int test_bit(int nr, const unsigned long *addr)
> +static inline int test_bit(long nr, const unsigned long *addr)
>   {
>   	return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1)));
>   }
> diff --git a/util/bitmap.c b/util/bitmap.c
> index 687841d..9c6bb52 100644
> --- a/util/bitmap.c
> +++ b/util/bitmap.c
> @@ -36,9 +36,9 @@
>    * endian architectures.
>    */
>
> -int slow_bitmap_empty(const unsigned long *bitmap, int bits)
> +int slow_bitmap_empty(const unsigned long *bitmap, long bits)
>   {
> -    int k, lim = bits/BITS_PER_LONG;
> +    long k, lim = bits/BITS_PER_LONG;
>
>       for (k = 0; k < lim; ++k) {
>           if (bitmap[k]) {
> @@ -54,9 +54,9 @@ int slow_bitmap_empty(const unsigned long *bitmap, int bits)
>       return 1;
>   }
>
> -int slow_bitmap_full(const unsigned long *bitmap, int bits)
> +int slow_bitmap_full(const unsigned long *bitmap, long bits)
>   {
> -    int k, lim = bits/BITS_PER_LONG;
> +    long k, lim = bits/BITS_PER_LONG;
>
>       for (k = 0; k < lim; ++k) {
>           if (~bitmap[k]) {
> @@ -74,9 +74,9 @@ int slow_bitmap_full(const unsigned long *bitmap, int bits)
>   }
>
>   int slow_bitmap_equal(const unsigned long *bitmap1,
> -                      const unsigned long *bitmap2, int bits)
> +                      const unsigned long *bitmap2, long bits)
>   {
> -    int k, lim = bits/BITS_PER_LONG;
> +    long k, lim = bits/BITS_PER_LONG;
>
>       for (k = 0; k < lim; ++k) {
>           if (bitmap1[k] != bitmap2[k]) {
> @@ -94,9 +94,9 @@ int slow_bitmap_equal(const unsigned long *bitmap1,
>   }
>
>   void slow_bitmap_complement(unsigned long *dst, const unsigned long *src,
> -                            int bits)
> +                            long bits)
>   {
> -    int k, lim = bits/BITS_PER_LONG;
> +    long k, lim = bits/BITS_PER_LONG;
>
>       for (k = 0; k < lim; ++k) {
>           dst[k] = ~src[k];
> @@ -108,10 +108,10 @@ void slow_bitmap_complement(unsigned long *dst, const unsigned long *src,
>   }
>
>   int slow_bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
> -                    const unsigned long *bitmap2, int bits)
> +                    const unsigned long *bitmap2, long bits)
>   {
> -    int k;
> -    int nr = BITS_TO_LONGS(bits);
> +    long k;
> +    long nr = BITS_TO_LONGS(bits);
>       unsigned long result = 0;
>
>       for (k = 0; k < nr; k++) {
> @@ -121,10 +121,10 @@ int slow_bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
>   }
>
>   void slow_bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
> -                    const unsigned long *bitmap2, int bits)
> +                    const unsigned long *bitmap2, long bits)
>   {
> -    int k;
> -    int nr = BITS_TO_LONGS(bits);
> +    long k;
> +    long nr = BITS_TO_LONGS(bits);
>
>       for (k = 0; k < nr; k++) {
>           dst[k] = bitmap1[k] | bitmap2[k];
> @@ -132,10 +132,10 @@ void slow_bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
>   }
>
>   void slow_bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
> -                     const unsigned long *bitmap2, int bits)
> +                     const unsigned long *bitmap2, long bits)
>   {
> -    int k;
> -    int nr = BITS_TO_LONGS(bits);
> +    long k;
> +    long nr = BITS_TO_LONGS(bits);
>
>       for (k = 0; k < nr; k++) {
>           dst[k] = bitmap1[k] ^ bitmap2[k];
> @@ -143,10 +143,10 @@ void slow_bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
>   }
>
>   int slow_bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
> -                       const unsigned long *bitmap2, int bits)
> +                       const unsigned long *bitmap2, long bits)
>   {
> -    int k;
> -    int nr = BITS_TO_LONGS(bits);
> +    long k;
> +    long nr = BITS_TO_LONGS(bits);
>       unsigned long result = 0;
>
>       for (k = 0; k < nr; k++) {
> @@ -157,10 +157,10 @@ int slow_bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
>
>   #define BITMAP_FIRST_WORD_MASK(start) (~0UL << ((start) % BITS_PER_LONG))
>
> -void bitmap_set(unsigned long *map, int start, int nr)
> +void bitmap_set(unsigned long *map, long start, long nr)
>   {
>       unsigned long *p = map + BIT_WORD(start);
> -    const int size = start + nr;
> +    const long size = start + nr;
>       int bits_to_set = BITS_PER_LONG - (start % BITS_PER_LONG);
>       unsigned long mask_to_set = BITMAP_FIRST_WORD_MASK(start);
>
> @@ -177,10 +177,10 @@ void bitmap_set(unsigned long *map, int start, int nr)
>       }
>   }
>
> -void bitmap_clear(unsigned long *map, int start, int nr)
> +void bitmap_clear(unsigned long *map, long start, long nr)
>   {
>       unsigned long *p = map + BIT_WORD(start);
> -    const int size = start + nr;
> +    const long size = start + nr;
>       int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG);
>       unsigned long mask_to_clear = BITMAP_FIRST_WORD_MASK(start);
>
> @@ -212,10 +212,10 @@ void bitmap_clear(unsigned long *map, int start, int nr)
>    * power of 2. A @align_mask of 0 means no alignment is required.
>    */
>   unsigned long bitmap_find_next_zero_area(unsigned long *map,
> -					 unsigned long size,
> -					 unsigned long start,
> -					 unsigned int nr,
> -					 unsigned long align_mask)
> +                                         unsigned long size,
> +                                         unsigned long start,
> +                                         unsigned long nr,
> +                                         unsigned long align_mask)
>   {
>       unsigned long index, end, i;
>   again:
> @@ -237,9 +237,9 @@ again:
>   }
>
>   int slow_bitmap_intersects(const unsigned long *bitmap1,
> -                           const unsigned long *bitmap2, int bits)
> +                           const unsigned long *bitmap2, long bits)
>   {
> -    int k, lim = bits/BITS_PER_LONG;
> +    long k, lim = bits/BITS_PER_LONG;
>
>       for (k = 0; k < lim; ++k) {
>           if (bitmap1[k] & bitmap2[k]) {
>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>