Re: slow live magration / xc_restore on xen4 pvops

[Brendan Cully <brendan@cs.ubc.ca>]

On Wednesday, 02 June 2010 at 17:18, Ian Jackson wrote:
> Andreas Olsowski writes ("[Xen-devel] slow live magration / xc_restore on xen4 pvops"):
> > [2010-06-01 21:20:57 5211] INFO (XendCheckpoint:423) ERROR Internal 
> > error: Error when reading batch size
> > [2010-06-01 21:20:57 5211] INFO (XendCheckpoint:423) ERROR Internal 
> > error: error when buffering batch, finishing
> 
> These errors, and the slowness of migrations, are caused by changes
> made to support Remus.  Previously, a migration would be regarded as
> complete as soon as the final information including CPU states was
> received at the migration target.  xc_domain_restore would return
> immediately at that point.
> 
> Since the Remus patches, xc_domain_restore waits until it gets an IO
> error, and also has a very short timeout which induces IO errors if
> nothing is received if there is no timeout.  This is correct in the
> Remus case but wrong in the normal case.
> 
> The code should be changed so that xc_domain_restore
>  (a) takes an explicit parameter for the IO timeout, which
>      should default to something much longer than the 100ms or so of
>      the Remus case, and
>  (b) gets told whether
>     (i) it should return immediately after receiving the "tail"
>         which contains the CPU state; or
>     (ii) it should attempt to keep reading after receiving the "tail"
>         and only return when the connection fails.

I'm going to have a look at this today, but the way the code was
originally written I don't believe this should have been a problem:

1. reads are only supposed to be able to time out after the entire
first checkpoint has been received (IOW this wouldn't kick in until
normal migration had already completed)

2. in normal migration, the sender should close the fd after sending
all data, immediately triggering an IO error on the receiver and
completing the restore.

I did try to avoid disturbing regular live migration as much as
possible when I wrote the code. I suspect some other regression has
crept in, and I'll investigate.