[Qemu-devel] [PATCH v3 00/38] bitmap queue

[Juan Quintela <quintela@redhat.com>]

Hi

This is the last version of the bitmap patches, changes from last submission:
- cleanups: now it passes checkpatch
- all coments from Eric & Paolo addressed
- fixed problem with DIRTY_MEMORY_NUM
- tested by Vinod and numbers are very good
- move bitmap operations to take long as index
- round ram_addr's of blocks as suggesetd by Paolo.

If there is no negative comments, will sent for inclusion.

Thanks, Juan.


[v2]
In this version:
- fixed all the comments from last versions (thanks Eric)
- kvm migration bitmap is synchronized using bitmap operations
- qemu bitmap -> migration bitmap is synchronized using bitmap operations
If bitmaps are not properly aligned, we fall back to old code.
Code survives virt-tests, so should be in quite good shape.

ToDo list:

- vga ram by default is not aligned in a page number multiple of 64,

  it could be optimized.  Kraxel?  It syncs the kvm bitmap at least 1
  a second or so? bitmap is only 2048 pages (16MB by default).
  We need to change the ram_addr only

- vga: still more, after we finish migration, vga code continues
  synchronizing the kvm bitmap on source machine.  Notice that there
  is no graphics client connected to the VGA.  Worth investigating?

- I haven't yet meassure speed differences on big hosts.  Vinod?

- Depending of performance, more optimizations to do.

- debugging printf's still on the code, just to see if we are taking
  (or not) the optimized paths.

And that is all.  Please test & comment.

Thanks, Juan.

The following changes since commit f46e720a82ccdf1a521cf459448f3f96ed895d43:

  qemu_opts_parse(): always check return value (2013-12-16 15:33:48 -0800)

are available in the git repository at:

  git://github.com/juanquintela/qemu.git bitmap.next

for you to fetch changes up to 197a77684c35f75e43d937220ffac72101783384:

  ram: align ram_addr_t's regions in multiples of 64 (2013-12-17 15:56:04 +0100)

----------------------------------------------------------------
Juan Quintela (38):
      bitmap: use long as index
      memory: cpu_physical_memory_set_dirty_flags() result is never used
      memory: cpu_physical_memory_set_dirty_range() return void
      exec: use accessor function to know if memory is dirty
      memory: create function to set a single dirty bit
      exec: create function to get a single dirty bit
      memory: make cpu_physical_memory_is_dirty return bool
      memory: all users of cpu_physical_memory_get_dirty used only one flag
      memory: set single dirty flags when possible
      memory: cpu_physical_memory_set_dirty_range() always dirty all flags
      memory: cpu_physical_memory_mask_dirty_range() always clears a single flag
      memory: use bit 2 for migration
      memory: make sure that client is always inside range
      memory: only resize dirty bitmap when memory size increases
      memory: cpu_physical_memory_clear_dirty_flag() result is never used
      bitmap: Add bitmap_zero_extend operation
      memory: split dirty bitmap into three
      memory: unfold cpu_physical_memory_clear_dirty_flag() in its only user
      memory: unfold cpu_physical_memory_set_dirty() in its only user
      memory: unfold cpu_physical_memory_set_dirty_flag()
      memory: make cpu_physical_memory_get_dirty() the main function
      memory: cpu_physical_memory_get_dirty() is used as returning a bool
      memory: s/mask/clear/ cpu_physical_memory_mask_dirty_range
      memory: use find_next_bit() to find dirty bits
      memory: cpu_physical_memory_set_dirty_range() now uses bitmap operations
      memory: cpu_physical_memory_clear_dirty_range() now uses bitmap operations
      memory: s/dirty/clean/ in cpu_physical_memory_is_dirty()
      memory: make cpu_physical_memory_reset_dirty() take a length parameter
      memory: cpu_physical_memory_set_dirty_tracking() should return void
      memory: split cpu_physical_memory_* functions to its own include
      memory: unfold memory_region_test_and_clear()
      kvm: use directly cpu_physical_memory_* api for tracking dirty pages
      kvm: refactor start address calculation
      memory: move bitmap synchronization to its own function
      memory: syncronize kvm bitmap using bitmaps operations
      ram: split function that synchronizes a range
      migration: synchronize memory bitmap 64bits at a time
      ram: align ram_addr_t's regions in multiples of 64

 arch_init.c                    |  52 ++++++++++++---
 cputlb.c                       |  11 +--
 exec.c                         |  78 +++++++++++-----------
 include/exec/cpu-all.h         |   3 +-
 include/exec/memory-internal.h |  90 -------------------------
 include/exec/memory.h          |  12 ++--
 include/exec/ram_addr.h        | 147 +++++++++++++++++++++++++++++++++++++++++
 include/qemu/bitmap.h          |  86 +++++++++++++-----------
 include/qemu/bitops.h          |  14 ++--
 kvm-all.c                      |  28 ++------
 memory.c                       |  17 ++---
 util/bitmap.c                  |  60 ++++++++---------
 12 files changed, 337 insertions(+), 261 deletions(-)
 create mode 100644 include/exec/ram_addr.h