Re: Prepping a bugfix push

[Jeremy Fitzhardinge <jeremy@goop.org>]

On 12/04/09 08:56, Ian Campbell wrote:
> On Fri, 2009-12-04 at 16:37 +0000, Jeremy Fitzhardinge wrote:
>    
>> On 12/04/09 07:50, Ian Campbell wrote:
>>      
>>> On Fri, 2009-12-04 at 07:46 +0000, Ian Campbell wrote:
>>>
>>>        
>>>> I've been doing regular suspend/resumes not checkpoint ones as Brendan
>>>> is doing, I did try a couple of checkpointed ones yesterday and they
>>>> failed, IIRC with a similar softlockup to this one.
>>>>
>>>>          
>>> So what is happening is that the device event channels are getting torn
>>> down by the resume handler and never completely reinstated in the
>>> cancelled suspend (aka checkpoint) case.
>>>
>>>        
>> Hm.
>>
>>      
>>> In 2.6.18 there was a separate ->suspend_cancel() callback for each
>>> driver, called instead of the ->resume() callback in exactly these
>>> circumstances. The cancel callback doesn't do any of the teardown, in
>>> fact for blkfront it doesn't even exist.
>>>
>>> (As a proof of concept, commenting out the entire contents of
>>> blkfront_resume and netfront_resume makes checkpointing work OK for me,
>>> at the cost of breaking regular resume, of course)
>>>
>>> pv-ops uses the generic power management infrastructure which does not
>>> have a concept of cancelling a suspend. Perhaps it should? Otherwise a
>>> different solution will be required, I'm not sure what that might be yet
>>> yet.
>>>
>>>        
>> Well, the obvious one is to treat it as a full suspend followed by
>> immediate resume.  That is, just remove all the special case handling
>> for checkpoint, and let it do the normal resume stuff when the hypercall
>> returns.
>>      
> I'm not sure how much that will help, some of the resume stuff relies on
> the domain actually changing underneath, i.e. the backends are torn down
> and resetup by the tools and therefore expect a fresh reconnection, the
> hypervisor side of event channels is implicitly reset (the kernel just
> resets its own state) etc. None of these things happen during a
> checkpoint. Presumably those who are interested in checkpointing would
> prefer them not to happen in order to remain fast.
>    

Yes, that's certainly all possible with some biggish performance hit...

How about this: if its a checkpoint, then don't bother calling all the 
resume functions.  We may need to call the device model resume just to 
keep everyone sane and happy, but at the xenbus nodes, filter out the 
calls to the drivers.

>> I think the PM core can fail to suspend; it just resumes anything that
>> has been suspended so far.
>>      
> An optional separate hook for that case (called in preference to
> ->resume) might be acceptable upstream? Adding a parameter to the
> ->resume handler itself might also be acceptable but would involve more
> churn.
>    

The whole area is so fragile and fraught, I don't really want to get 
into it.

     J