Re: [Qemu-devel] QEMU MicroCheckpointing Pause & Resume Latency

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* FENG, Jiasheng (nikofeng@connect.hku.hk) wrote:
> Dear QEMU Development Team,
> 
> 
> It is my honor to contact with you.
> 
> 
> 
> I am a postgraduate student from University of Hong Kong. Currently I am
> working on a project related to QEMU MicroCheckpointing and I have
> encountered a performance issue during checkpoint pause & resume.

The microcheckpointing code hasn't been maintained for a long time;
most of the current checkpointing work is based on the COLO work which is
still under development.

> Please kindly refer to migration/checkpoint.c file, in function
> capture_checkpoint, I proceeded a test to see the time consumption between
> vm_stop_force_state and vm_start. I found out that even if the system is
> idle, there are still 12-20ms latency recorded ( mem=2G, vCPU=4 ).
> Moreover, latency will be increased while more cpus equipped by my virtual
> machine. I have done some research on that and I realized that it is
> related to the Memory Barrier in KVM kernel. Each cpu will proceed a
> smp_wmb() request during pause & resume and it takes about  3-5ms to finish
> the request ( mem=2G, vCPU=4 ).
> 
> 
> 
> Therefore, I would like to ask 3 questions regarding on the above issue:
> 
> 
> 1. What is your consideration with calling smp_wmb() in checkpoint period;
> 
> 2. Is it any other solution to minimize the latency to improve the
> performance in checkpoint period;
> 
> 3. Is smp_wmb() able to be safely disabled during the checkpoint period

Well you'd have to understand where it's used; but for example, when taking
a checkpoint you'd want to be sure that the checkpoint data contained
a consistent copy of the last write data from all of the vCPUs; so I think
a wmb would be needed to make sure it's consistent.

I'm surprised that the smp_wmb is such a big chunk of your total checkpoint
time, and that it's quite so long.  
Are the vCPUs idle or are they busy - does it make difference?

Dave

> Really appreciate your help with my problems and hope to receive your
> feedback soon.
> 
> 
> Thanks again for your contribution to QEMU and it is such a masterpiece.

Dave

> 
> 
> 
> Thanks and best regards,
> 
> Niko Jiasheng Feng
> 
> University of Hong Kong
> 
> -- 
> *Niko Jiasheng *
> *Feng **Computer Science(General Stream), Faculty of Engineering, The
> University of Hong Kong*
> Contact:  （852）97908620
> Address: Pokfulam Road, The University of Hong Kong
> Email:      nikofeng@hku.hk / niko_jiasheng@163.com
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK