Re: [RFC] use event channel to improve suspend speed

[Brendan Cully <brendan@cs.ubc.ca>]

On Friday, 25 May 2007 at 07:46, Keir Fraser wrote:
> On 25/5/07 01:06, "Brendan Cully" <brendan@cs.ubc.ca> wrote:
> 
> > In case there's interest, I'll attach the two patches I'm using to
> > speed up checkpointing (and live migration downtime). As I mentioned
> > earlier, the first patch should be semantically equivalent to existing
> > code, and cuts downtime to about 30-35ms. The second notifies xend
> > that the domain has been suspended asynchronously, so that final round
> > memory copying may begin before device migration stage 2. This is a
> > semantic change, but I can't think of a concrete drawback. It's a
> > little rough-and-ready -- suggestions for improvement are welcome.
> 
> Can patch 2 be used without patch 1? The fact it doesn't need to change the
> guest interface again is a big advantage. And it seems to provide by far the
> larger proportional speedup.

Yes, it's possible to separate them out. But they work best together,
since that's the only case where xenstore is completely removed from
the suspend path. I've done another version of patch 1 in which
xc_save triggers the suspend by writing to xenstore (instead of asking
xend to do it), with a vanilla guest kernel. With patch 2 also
applied, I get this:

avg: 23.17 ms, min: 9.08, max: 545.51, median: 13.66

implying the base penalty for using xenstore is about 5-10ms, and we
still suffer some pretty serious jitter.

The same test with /var/lib/xenstored mounted on tmpfs is much less
jittery:

avg: 14.62 ms, min: 8.27, max: 32.67, median: 11.71

but there's still a 5-20ms xenstore penalty.