[Qemu-devel] Question regarding two variables in qemu migration code

[Qemu-devel] Question regarding two variables in qemu migration code

[Jidong Xiao]

Hi,

I am looking at the qemu source code, and trying to understand the
migration part. In arch_init.c, there are two variables which seems
quite confusing to me,

They are:

static uint64_t migration_dirty_pages;

static int64_t num_dirty_pages_period; // defined in function
migration_bitmap_sync()

Can anyone kindly explain that what does these two variables mean? Thanks.

-Jidong

Re: [Qemu-devel] Question regarding two variables in qemu migration code

[Dr. David Alan Gilbert]

* Jidong Xiao (jidong.xiao@gmail.com) wrote:
> Hi,

Hi,

> I am looking at the qemu source code, and trying to understand the
> migration part. In arch_init.c, there are two variables which seems
> quite confusing to me,
> 
> They are:
> 
> static uint64_t migration_dirty_pages;

'migration_dirty_pages' is the number of pages that are currently known
that need to be sent to the destination; it goes down whenever we send
a page, but goes up when we sync the dirty bitmap that tells us that
something changed the data in the page (see migration_bitmap_sync_range )

> static int64_t num_dirty_pages_period; // defined in function
> migration_bitmap_sync()

This is looking how many pages we've noticed are now dirty within
a particular time - to try and get an estimate of how fast memory is changing
If you see migration_bitmap_sync  has an:
       if (end_time > start_time + 1000) {

and inside there it uses num_dirty_pages_period to update dirty_pages_rate.

> 
> Can anyone kindly explain that what does these two variables mean? Thanks.
> 
> -Jidong

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Question regarding two variables in qemu migration code

[Jidong Xiao]

On Tue, Jan 13, 2015 at 1:38 AM, Dr. David Alan Gilbert
<dgilbert@redhat.com> wrote:
> * Jidong Xiao (jidong.xiao@gmail.com) wrote:
>> Hi,
>
> Hi,
>
>> I am looking at the qemu source code, and trying to understand the
>> migration part. In arch_init.c, there are two variables which seems
>> quite confusing to me,
>>
>> They are:
>>
>> static uint64_t migration_dirty_pages;
>
> 'migration_dirty_pages' is the number of pages that are currently known
> that need to be sent to the destination; it goes down whenever we send
> a page, but goes up when we sync the dirty bitmap that tells us that
> something changed the data in the page (see migration_bitmap_sync_range )
>
>> static int64_t num_dirty_pages_period; // defined in function
>> migration_bitmap_sync()
>
> This is looking how many pages we've noticed are now dirty within
> a particular time - to try and get an estimate of how fast memory is changing
> If you see migration_bitmap_sync  has an:
>        if (end_time > start_time + 1000) {
>
> and inside there it uses num_dirty_pages_period to update dirty_pages_rate.
>
>>
>> Can anyone kindly explain that what does these two variables mean? Thanks.
>>
>> -Jidong
>
> Dave
> --
Thanks Dave, your explanation is really really helpful.

But in function migration_bitmap_sync(), I see this:

    num_dirty_pages_period += migration_dirty_pages - num_dirty_pages_init;

If as you said, num_dirty_pages_period refers to the pages get dirty
within a particular time, then why it is "+=“, instead of "="? i.e.,
something like this:

    num_dirty_pages_period = migration_dirty_pages - num_dirty_pages_init;

I just don't see why num_dirty_pages_period has to be accumulated with
its previous value.

-Jidong

Re: [Qemu-devel] Question regarding two variables in qemu migration code

[Dr. David Alan Gilbert]

* Jidong Xiao (jidong.xiao@gmail.com) wrote:
> On Tue, Jan 13, 2015 at 1:38 AM, Dr. David Alan Gilbert
> <dgilbert@redhat.com> wrote:
> > * Jidong Xiao (jidong.xiao@gmail.com) wrote:
> >> Hi,
> >
> > Hi,
> >
> >> I am looking at the qemu source code, and trying to understand the
> >> migration part. In arch_init.c, there are two variables which seems
> >> quite confusing to me,
> >>
> >> They are:
> >>
> >> static uint64_t migration_dirty_pages;
> >
> > 'migration_dirty_pages' is the number of pages that are currently known
> > that need to be sent to the destination; it goes down whenever we send
> > a page, but goes up when we sync the dirty bitmap that tells us that
> > something changed the data in the page (see migration_bitmap_sync_range )
> >
> >> static int64_t num_dirty_pages_period; // defined in function
> >> migration_bitmap_sync()
> >
> > This is looking how many pages we've noticed are now dirty within
> > a particular time - to try and get an estimate of how fast memory is changing
> > If you see migration_bitmap_sync  has an:
> >        if (end_time > start_time + 1000) {
> >
> > and inside there it uses num_dirty_pages_period to update dirty_pages_rate.
> >
> >>
> >> Can anyone kindly explain that what does these two variables mean? Thanks.
> >>
> >> -Jidong
> >
> > Dave
> > --
> Thanks Dave, your explanation is really really helpful.
> 
> But in function migration_bitmap_sync(), I see this:
> 
>     num_dirty_pages_period += migration_dirty_pages - num_dirty_pages_init;
> 
> If as you said, num_dirty_pages_period refers to the pages get dirty
> within a particular time, then why it is "+=???, instead of "="? i.e.,
> something like this:
> 
>     num_dirty_pages_period = migration_dirty_pages - num_dirty_pages_init;
> 
> I just don't see why num_dirty_pages_period has to be accumulated with
> its previous value.

It's because there are two periods:
   migration_dirty_pages - num_dirty_pages_init
             is the number of pages found dirty in this call to migration_bitmap_sync

   num_dirty_pages_period
             is the number of pages found dirty in a ~1 second periods, as tested by
             the code in migration_bitmap_sync after the:
                 if (end_time > start_time + 1000) {

migration_bitmap_sync is called multiple times within that ~1 second period, and
num_dirty_pages_period accumulates across the calls.

Dave

> 
> -Jidong
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK