[Qemu-devel] [PATCH 00/18] Migration thread lite (20121029)

[Qemu-devel] [PATCH 00/18] Migration thread lite (20121029)

[Juan Quintela]

Hi

After discussing with Anthony and Paolo, this is the minimal migration
thread support that we can do for 1.3.

- fixed all the comments (thanks eric, paolo and orit).
- buffered_file.c remains until 1.4.
- testing for vinod showed very nice results, that is why the bitmap
  handling optimizations remain.

Note: Writes has become blocking, and I have to change the "remove"
the feature now in qemu-sockets.c.  Checked that migration was the
only user of that feature.  If new users appear, they just need to add
the socket_set_nonblock() by hand.

Please, review.

Thanks, Juan.


The following changes since commit 50cd72148211c5e5f22ea2519d19ce024226e61f:

  hw/xtensa_sim: get rid of intermediate xtensa_sim_init (2012-10-27 15:04:00 +0000)

are available in the git repository at:

  http://repo.or.cz/r/qemu/quintela.git migration-thread-20121029

for you to fetch changes up to 2c74654f19efc7db35117a87c0d9db4776931e1b:

  ram: optimize migration bitmap walking (2012-10-29 14:14:28 +0100)

Juan Quintela (15):
  buffered_file: Move from using a timer to use a thread
  migration: make qemu_fopen_ops_buffered() return void
  migration: stop all cpus correctly
  migration: make writes blocking
  migration: remove unfreeze logic
  migration: take finer locking
  buffered_file: Unfold the trick to restart generating migration data
  buffered_file: don't flush on put buffer
  buffered_file: unfold buffered_append in buffered_put_buffer
  savevm: New save live migration method: pending
  migration: include qemu-file.h
  migration-fd: remove duplicate include
  memory: introduce memory_region_test_and_clear_dirty
  ram: Use memory_region_test_and_clear_dirty
  ram: optimize migration bitmap walking

Paolo Bonzini (1):
  split MRU ram list

Umesh Deshpande (2):
  add a version number to ram_list
  protect the ramlist with a separate mutex

 arch_init.c       | 115 ++++++++++++++++++++++++-----------------------
 block-migration.c |  49 +++++---------------
 buffered_file.c   | 130 +++++++++++++++++++++++++-----------------------------
 buffered_file.h   |   2 +-
 cpu-all.h         |  15 ++++++-
 exec.c            |  44 +++++++++++++++---
 memory.c          |  16 +++++++
 memory.h          |  18 ++++++++
 migration-exec.c  |   3 +-
 migration-fd.c    |   4 +-
 migration-tcp.c   |   2 +-
 migration-unix.c  |   2 +-
 migration.c       | 100 ++++++++++++++++-------------------------
 migration.h       |   4 +-
 qemu-file.h       |   5 ---
 qemu-sockets.c    |   4 --
 savevm.c          |  24 +++++++---
 sysemu.h          |   1 +
 vmstate.h         |   1 +
 19 files changed, 280 insertions(+), 259 deletions(-)

-- 
1.7.11.7

Re: [Qemu-devel] [PATCH 00/18] Migration thread lite (20121029)

[Chegu Vinod]

On 10/29/2012 9:21 AM, Vinod, Chegu wrote:
> Date: Mon, 29 Oct 2012 15:11:25 +0100
> From: Juan Quintela <quintela@redhat.com>
> To: qemu-devel@nongnu.org
> Cc: owasserm@redhat.com, mtosatti@redhat.com, avi@redhat.com,
> 	pbonzini@redhat.com
> Subject: [Qemu-devel] [PATCH 00/18] Migration thread lite (20121029)
>
> Hi
>
> After discussing with Anthony and Paolo, this is the minimal migration thread support that we can do for 1.3.
>
> - fixed all the comments (thanks eric, paolo and orit).
> - buffered_file.c remains until 1.4.
> - testing for vinod showed very nice results, that is why the bitmap
>    handling optimizations remain.

Hi Juan,

Is there any value in calling the migration_bitmap_synch() routine in 
the ram_save_setup() . All the
pages were marked as dirty to begin with...so can't we just assume that 
all pages need to be sent
and proceed ?

migration_bitmap_synch () - still remains an expensive call. and the 
very first call to seems to be ~3x times
  more expensive than the subsequent calls. For large guests (>128G 
guests) this is multiple seconds...and
it freezes the OS instance.

Thanks
Vinod


> Note: Writes has become blocking, and I have to change the "remove"
> the feature now in qemu-sockets.c.  Checked that migration was the only user of that feature.  If new users appear, they just need to add the socket_set_nonblock() by hand.
>
> Please, review.
>
> Thanks, Juan.
>
>
> The following changes since commit 50cd72148211c5e5f22ea2519d19ce024226e61f:
>
>    hw/xtensa_sim: get rid of intermediate xtensa_sim_init (2012-10-27 15:04:00 +0000)
>
> are available in the git repository at:
>
>    http://repo.or.cz/r/qemu/quintela.git migration-thread-20121029
>
> for you to fetch changes up to 2c74654f19efc7db35117a87c0d9db4776931e1b:
>
>    ram: optimize migration bitmap walking (2012-10-29 14:14:28 +0100)
>
> Juan Quintela (15):
>    buffered_file: Move from using a timer to use a thread
>    migration: make qemu_fopen_ops_buffered() return void
>    migration: stop all cpus correctly
>    migration: make writes blocking
>    migration: remove unfreeze logic
>    migration: take finer locking
>    buffered_file: Unfold the trick to restart generating migration data
>    buffered_file: don't flush on put buffer
>    buffered_file: unfold buffered_append in buffered_put_buffer
>    savevm: New save live migration method: pending
>    migration: include qemu-file.h
>    migration-fd: remove duplicate include
>    memory: introduce memory_region_test_and_clear_dirty
>    ram: Use memory_region_test_and_clear_dirty
>    ram: optimize migration bitmap walking
>
> Paolo Bonzini (1):
>    split MRU ram list
>
> Umesh Deshpande (2):
>    add a version number to ram_list
>    protect the ramlist with a separate mutex
>
>   arch_init.c       | 115 ++++++++++++++++++++++++-----------------------
>   block-migration.c |  49 +++++---------------
>   buffered_file.c   | 130 +++++++++++++++++++++++++-----------------------------
>   buffered_file.h   |   2 +-
>   cpu-all.h         |  15 ++++++-
>   exec.c            |  44 +++++++++++++++---
>   memory.c          |  16 +++++++
>   memory.h          |  18 ++++++++
>   migration-exec.c  |   3 +-
>   migration-fd.c    |   4 +-
>   migration-tcp.c   |   2 +-
>   migration-unix.c  |   2 +-
>   migration.c       | 100 ++++++++++++++++-------------------------
>   migration.h       |   4 +-
>   qemu-file.h       |   5 ---
>   qemu-sockets.c    |   4 --
>   savevm.c          |  24 +++++++---
>   sysemu.h          |   1 +
>   vmstate.h         |   1 +
>   19 files changed, 280 insertions(+), 259 deletions(-)
>
> --
> 1.7.11.7
>
>
>
>
>
> .
>

Re: [Qemu-devel] [PATCH 00/18] Migration thread lite (20121029)

[Juan Quintela]

Chegu Vinod <chegu_vinod@hp.com> wrote:
> On 10/29/2012 9:21 AM, Vinod, Chegu wrote:
>> Date: Mon, 29 Oct 2012 15:11:25 +0100
>> From: Juan Quintela <quintela@redhat.com>
>> To: qemu-devel@nongnu.org
>> Cc: owasserm@redhat.com, mtosatti@redhat.com, avi@redhat.com,
>> 	pbonzini@redhat.com
>> Subject: [Qemu-devel] [PATCH 00/18] Migration thread lite (20121029)
>>
>> Hi
>>
>> After discussing with Anthony and Paolo, this is the minimal migration thread support that we can do for 1.3.
>>
>> - fixed all the comments (thanks eric, paolo and orit).
>> - buffered_file.c remains until 1.4.
>> - testing for vinod showed very nice results, that is why the bitmap
>>    handling optimizations remain.
>
> Hi Juan,
>
> Is there any value in calling the migration_bitmap_synch() routine in
> the ram_save_setup() . All the
> pages were marked as dirty to begin with...so can't we just assume
> that all pages need to be sent
> and proceed ?

I haven't investigated it fully, but if we don't do it, we end sending
each page twice.  As said that needs more investigation.

>
> migration_bitmap_synch () - still remains an expensive call. and the
> very first call to seems to be ~3x times
>  more expensive than the subsequent calls. For large guests (>128G
> guests) this is multiple seconds...and
> it freezes the OS instance.

Will try to implement the other optimizaton, but that is 1.4 material
:-(

Later, Juan.)