Re: [PATCH 5 of 5] xen: Avoid allocations causing swap activity on the resume path

[Jeremy Fitzhardinge <jeremy@goop.org>]

Andrew Morton wrote:
> On Sat, 31 May 2008 11:10:18 +0100 Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>
>   
>>> Whether you should also add __GFP_HIGH to cause the page allocation to
>>> bite harder into the page reserves is unclear to me, sorry.
>>>   
>>>       
>> Well, if the allocation fails the machine is dead in the water, because 
>> it probably can't go on without its devices.  I think it's worth eating 
>> to the reserves to avoid that.  (I'll have a close look to only add it 
>> to allocations which are really unrecoverable if they fail.)
>>     
>
> OK.
>
> I don't actually have a clue what you're doing here.  Restoring a
> virtual machine from disk/network or something like that?
>   

More or less.  When you suspend the VM it gets disconnected from all its 
virtual devices.  When you resume - which may be after a migration to 
another machine - it reconnects to all its devices again.  These 
allocations are happening on the reconnection path, and so may happen 
before there's any underlying device to do IO to (in fact, they've been 
triggered by something trying to do IO, and the driver notices it has 
become disconnected and tries to reconnect itself).

> It might be appropriate to do a big memory-reclaim before starting the
> operation, along the lines of suspend-to-disk - go off and allocate
> (and possible pin) sufficient memroy for the entire operation before
> actually starting it?
>   

Not really.  Live migration - which is one instance of when this comes 
into play - is a very lightweight process from the guest VM's 
perspective, and may cause only ~100ms service interruption.  Doing a 
mass of reclaim/IO would be expensive by comparison.

Also, in general the drivers have to be prepared to deal with a 
disconnection from their devices at any moment (ie, they may 
spontaneously disconnect for no apparent reason), so there's no 
opportunity to prepare for it.

Maybe it would be possible to scurry away some preallocated memory for 
use if the main allocation fails, but that seems logically equivalent to 
dipping into the global emergency pool.

    J