Re: [Qemu-devel] [PATCH v4 4/4] migration: use the free page hint feature from balloon

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Mar 14, 2018 at 02:50:44PM +0800, Wei Wang wrote:
> On 03/14/2018 10:51 AM, Michael S. Tsirkin wrote:
> > On Wed, Mar 14, 2018 at 10:41:36AM +0800, Wei Wang wrote:
> > > On 03/14/2018 12:35 AM, Michael S. Tsirkin wrote:
> > > > On Wed, Mar 07, 2018 at 08:34:25PM +0800, Wei Wang wrote:
> > > > > Start the free page optimization after the migration bitmap is
> > > > > synchronized. This can't be used in the stop&copy phase since the guest
> > > > > is paused. Make sure the guest reporting has stopped before
> > > > > synchronizing the migration dirty bitmap. Currently, the optimization is
> > > > > added to precopy only.
> > > > > 
> > > > > Signed-off-by: Wei Wang <wei.w.wang@intel.com>
> > > > > CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > > > > CC: Juan Quintela <quintela@redhat.com>
> > > > > CC: Michael S. Tsirkin <mst@redhat.com>
> > > > > ---
> > > > >    migration/ram.c | 19 ++++++++++++++++++-
> > > > >    1 file changed, 18 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/migration/ram.c b/migration/ram.c
> > > > > index e172798..7b4c9b1 100644
> > > > > --- a/migration/ram.c
> > > > > +++ b/migration/ram.c
> > > > > @@ -51,6 +51,8 @@
> > > > >    #include "qemu/rcu_queue.h"
> > > > >    #include "migration/colo.h"
> > > > >    #include "migration/block.h"
> > > > > +#include "sysemu/balloon.h"
> > > > > +#include "sysemu/sysemu.h"
> > > > >    /***********************************************************/
> > > > >    /* ram save/restore */
> > > > > @@ -208,6 +210,8 @@ struct RAMState {
> > > > >        uint32_t last_version;
> > > > >        /* We are in the first round */
> > > > >        bool ram_bulk_stage;
> > > > > +    /* The free pages optimization feature is supported */
> > > > > +    bool free_page_support;
> > > > >        /* How many times we have dirty too many pages */
> > > > >        int dirty_rate_high_cnt;
> > > > >        /* these variables are used for bitmap sync */
> > > > > @@ -775,7 +779,7 @@ unsigned long migration_bitmap_find_dirty(RAMState *rs, RAMBlock *rb,
> > > > >        unsigned long *bitmap = rb->bmap;
> > > > >        unsigned long next;
> > > > > -    if (rs->ram_bulk_stage && start > 0) {
> > > > > +    if (rs->ram_bulk_stage && start > 0 && !rs->free_page_support) {
> > > > >            next = start + 1;
> > > > >        } else {
> > > > >            next = find_next_bit(bitmap, size, start);
> > > > > @@ -833,6 +837,10 @@ static void migration_bitmap_sync(RAMState *rs)
> > > > >        int64_t end_time;
> > > > >        uint64_t bytes_xfer_now;
> > > > > +    if (rs->free_page_support) {
> > > > > +        balloon_free_page_stop();
> > > > > +    }
> > > > > +
> > > > >        ram_counters.dirty_sync_count++;
> > > > >        if (!rs->time_last_bitmap_sync) {
> > > > > @@ -899,6 +907,10 @@ static void migration_bitmap_sync(RAMState *rs)
> > > > >        if (migrate_use_events()) {
> > > > >            qapi_event_send_migration_pass(ram_counters.dirty_sync_count, NULL);
> > > > >        }
> > > > > +
> > > > > +    if (rs->free_page_support && runstate_is_running()) {
> > > > > +        balloon_free_page_start();
> > > > > +    }
> > > > >    }
> > > > I think some of these conditions should go into
> > > > balloon_free_page_start/stop.
> > > > 
> > > > Checking runstate is generally problematic unless you
> > > > also handle run state change notifiers as it can
> > > > be manipulated from QMP.
> > > How about moving the check of runstate to
> > > virtio_balloon_poll_free_page_hints:
> > > 
> > > while (dev->free_page_report_status < FREE_PAGE_REPORT_S_STOP &&
> > > runstate_is_running()) {
> > > ...
> > > }
> > Hard to tell on the outset. E.g. why is just stop affected?  Pls add
> > comments explaining what happens if VM is not running when start or stop
> > is called.
> > 
> > 
> > > In this case, I think we won't need a notifier - if the run state is changed
> > > by qmp, the optimization thread will just exist.
> > But you need to wake it up and notify the guest presumably?
> > 
> 
> 
> I think it's not necessary to wake it up, because when the VM is not
> running, there will be no hints reported to the vq, and the optimization
> thread exits. (there is no issue in that case)
> Probably we can add a notifier which calls virtio_balloon_free_page_stop()
> when qmp wakes up the VM.
> 
> Best,
> Wei

set_status callback is invoked so you can use that maybe.

Might be a good idea to handle a bunch of other corner
cases e.g. if guest driver is loaded when migration
is already in progress.

-- 
MST