[Qemu-devel] Question about dirty page statistics for live migration

[Qemu-devel] Question about dirty page statistics for live migration

[Li Qiang]

Hello Dave, Juan and all,

It is useful to get the dirty page rates in guest to evaluate the guest
loads
so that we can make a decide to live migrate it or not. So I think we can
add a on-demand qmp for showing the dirty page rates.

I found someone has done this work in here:
-->https://github.com/grivon/yabusame-qemu-dpt
and here:
https://github.com/btrplace/qemu-patch

But seems not go to the upstream.

I want to know your opinions about adding this qmp.

Thanks,
Li Qiang

Re: [Qemu-devel] Question about dirty page statistics for live migration

[Dr. David Alan Gilbert]

* Li Qiang (liq3ea@gmail.com) wrote:
> Hello Dave, Juan and all,
> 
> It is useful to get the dirty page rates in guest to evaluate the guest
> loads
> so that we can make a decide to live migrate it or not. So I think we can
> add a on-demand qmp for showing the dirty page rates.
> 
> I found someone has done this work in here:
> -->https://github.com/grivon/yabusame-qemu-dpt
> and here:
> https://github.com/btrplace/qemu-patch
> 
> But seems not go to the upstream.
> 
> I want to know your opinions about adding this qmp.

Something like that could be good;  one easy idea we had was
a 'migrate null:' uri and then you would use most of the existing
migration code to do the measurement; you would only have to
add a dummy file backend, and something to stop the migration ever
terminating (maybe just set the downtime very low).


Dave


> Thanks,
> Li Qiang
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Question about dirty page statistics for live migration

[Li Qiang]

2018-08-13 17:35 GMT+08:00 Dr. David Alan Gilbert <dgilbert@redhat.com>:

> * Li Qiang (liq3ea@gmail.com) wrote:
> > Hello Dave, Juan and all,
> >
> > It is useful to get the dirty page rates in guest to evaluate the guest
> > loads
> > so that we can make a decide to live migrate it or not. So I think we can
> > add a on-demand qmp for showing the dirty page rates.
> >
> > I found someone has done this work in here:
> > -->https://github.com/grivon/yabusame-qemu-dpt
> > and here:
> > https://github.com/btrplace/qemu-patch
> >
> > But seems not go to the upstream.
> >
> > I want to know your opinions about adding this qmp.
>
> Something like that could be good;

one easy idea we had was
> a 'migrate null:' uri and then you would use most of the existing
> migration code to do the measurement; you would only have to
> add a dummy file backend,


As far as I understand, here dummy file backend just means the file migrate
to, right?

Thanks,
Li Qiang



> and something to stop the migration ever
> terminating (maybe just set the downtime very low).
>
>
> Dave
>
>
> > Thanks,
> > Li Qiang
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>

Re: [Qemu-devel] Question about dirty page statistics for live migration

[Dr. David Alan Gilbert]

* Li Qiang (liq3ea@gmail.com) wrote:
> 2018-08-13 17:35 GMT+08:00 Dr. David Alan Gilbert <dgilbert@redhat.com>:
> 
> > * Li Qiang (liq3ea@gmail.com) wrote:
> > > Hello Dave, Juan and all,
> > >
> > > It is useful to get the dirty page rates in guest to evaluate the guest
> > > loads
> > > so that we can make a decide to live migrate it or not. So I think we can
> > > add a on-demand qmp for showing the dirty page rates.
> > >
> > > I found someone has done this work in here:
> > > -->https://github.com/grivon/yabusame-qemu-dpt
> > > and here:
> > > https://github.com/btrplace/qemu-patch
> > >
> > > But seems not go to the upstream.
> > >
> > > I want to know your opinions about adding this qmp.
> >
> > Something like that could be good;
> 
> one easy idea we had was
> > a 'migrate null:' uri and then you would use most of the existing
> > migration code to do the measurement; you would only have to
> > add a dummy file backend,
> 
> 
> As far as I understand, here dummy file backend just means the file migrate
> to, right?

Yes; because then if you have a dummy migration destination, the rest of
the migration code will run, sync the bitmap and clear the bits, and
give you an approximation of the dirty rate.

It might still be worth having the separate code to measure it without
the overhead of the migration code; but then that's more complex - and
if you're trying to measure it to know how hard it is to migrate then
perhaps it's better to use the migration code anyway.

Dave

> Thanks,
> Li Qiang
> 
> 
> 
> > and something to stop the migration ever
> > terminating (maybe just set the downtime very low).
> >
> >
> > Dave
> >
> >
> > > Thanks,
> > > Li Qiang
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> >
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Question about dirty page statistics for live migration

[Li Qiang]

2018-08-13 19:30 GMT+08:00 Dr. David Alan Gilbert <dgilbert@redhat.com>:

> * Li Qiang (liq3ea@gmail.com) wrote:
> > 2018-08-13 17:35 GMT+08:00 Dr. David Alan Gilbert <dgilbert@redhat.com>:
> >
> > > * Li Qiang (liq3ea@gmail.com) wrote:
> > > > Hello Dave, Juan and all,
> > > >
> > > > It is useful to get the dirty page rates in guest to evaluate the
> guest
> > > > loads
> > > > so that we can make a decide to live migrate it or not. So I think
> we can
> > > > add a on-demand qmp for showing the dirty page rates.
> > > >
> > > > I found someone has done this work in here:
> > > > -->https://github.com/grivon/yabusame-qemu-dpt
> > > > and here:
> > > > https://github.com/btrplace/qemu-patch
> > > >
> > > > But seems not go to the upstream.
> > > >
> > > > I want to know your opinions about adding this qmp.
> > >
> > > Something like that could be good;
> >
> > one easy idea we had was
> > > a 'migrate null:' uri and then you would use most of the existing
> > > migration code to do the measurement; you would only have to
> > > add a dummy file backend,
> >
> >
> > As far as I understand, here dummy file backend just means the file
> migrate
> > to, right?
>
> Yes; because then if you have a dummy migration destination, the rest of
> the migration code will run, sync the bitmap and clear the bits, and
> give you an approximation of the dirty rate.
>
> It might still be worth having the separate code to measure it without
> the overhead of the migration code; but then that's more complex - and
> if you're trying to measure it to know how hard it is to migrate then
> perhaps it's better to use the migration code anyway.
>

Agree

But here is another issue, if we choice 'migrate null', then we will break
the 'migrate' interface.
the migrate doesn't return the info, so we need to uses 'info migrate'.
However only we want
is the dirty pages rates. It's too heavy for the user. Anyway we just want
to get the only dirty page rates
using one command.

I will first try to implement a separate qmp.

Thanks,
Li Qiang



> Dave
>
> > Thanks,
> > Li Qiang
> >
> >
> >
> > > and something to stop the migration ever
> > > terminating (maybe just set the downtime very low).
> > >
> > >
> > > Dave
> > >
> > >
> > > > Thanks,
> > > > Li Qiang
> > > --
> > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > >
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>