[Qemu-devel] [RFC v2 0/6] memory: make dirty_memory[] accesses atomic

[Stefan Hajnoczi <stefanha@redhat.com>]

v2:
 * Make bitmap_set_atomic() and bitmap_test_and_clear_atomic() faster by
   replacing the inner loop with cheaper operations.  I did use smp_wmb() in
   bitmap_set_atomic() so that the function is always a write barrier, no
   matter which code path is taken. [Paolo]

The dirty_memory[] bitmap is used for live migration, TCG self-modifying code
detection, and VGA emulation.  Up until now the bitmap was always accessed
under the QEMU global mutex.  This series makes all dirty_memory[] accesses
atomic to prepare the way for threads writing to guest memory without holding
the global mutex.

In particular, this series converts non-atomic dirty_memory[] accesses to
atomic_or, atomic_xchg, and atomic_fetch_and so that race conditions are
avoided when two threads manipulate the bitmap at the same time.

There are two pieces remaining before the dirty_memory[] bitmap is truly
thread-safe:

1. Convert all cpu_physical_memory_*_dirty() callers to use the API atomically.
   There are TCG callers who things along the lines of:

     if (!cpu_physical_memory_get_dirty(addr)) {
         cpu_physical_memory_set_dirty(addr);  /* not atomic! */
     }

   At first I considered ignoring TCG completely since it is currently not
   multi-threaded, but we might as well tackle this so that
   virtio-blk/virtio-scsi dataplane can be used with TCG eventually (they still
   need ioeventfd/irqfd emulation before they can support TCG).

2. Use array RCU to safely resize dirty_memory[] for memory hotplug.  Currently
   ram_block_add() grows the bitmap in a way that is not thread-safe.

   Paolo has a QEMU userspace RCU implementation which I'd like to bring in for
   this.

Stefan Hajnoczi (6):
  bitmap: add atomic set functions
  bitmap: add atomic test and clear
  memory: use atomic ops for setting dirty memory bits
  migration: move dirty bitmap sync to ram_addr.h
  memory: replace cpu_physical_memory_reset_dirty() with test-and-clear
  memory: make cpu_physical_memory_sync_dirty_bitmap() fully atomic

 arch_init.c             | 46 ++----------------------------
 cputlb.c                |  4 +--
 exec.c                  | 23 +++++++++++----
 include/exec/ram_addr.h | 74 ++++++++++++++++++++++++++++++++++++-------------
 include/qemu/bitmap.h   |  4 +++
 include/qemu/bitops.h   | 14 ++++++++++
 memory.c                | 11 +++-----
 util/bitmap.c           | 73 ++++++++++++++++++++++++++++++++++++++++++++++++
 8 files changed, 170 insertions(+), 79 deletions(-)

-- 
2.1.0