Hanging save/restore with 2.6.31.6 domU

Hanging save/restore with 2.6.31.6 domU

[Major Hayden]

Hello,

I have a peculiar issue with xm save / xm restore when using vanilla 2.6.31.6 kernel from kernel.org in a domU on a server running Xen 3.1.2-rc1.

When I run 'xm save' against a particular domU, the command will hang indefinitely.  If I ctrl-c out and run xm list, the name of the vm has 'migrating-' appended to the beginning.  Also, the network and console of the domU are unresponsive at this point.  The only way to bring the vm back is to use xm destroy/xm create, which isn't practical.

This is the log output in dom0's xend.log during the hang:

(__init__:1072) [xc_save]: /usr/lib64/xen/bin/xc_save 23 495 0 0 0
(__init__:1072) suspend
(__init__:1072) In saveInputHandler suspend
(__init__:1072) Suspending 495 ...
(__init__:1072) XendDomainInfo.shutdown(suspend)
(__init__:1072) XendDomainInfo.handleShutdownWatch
(__init__:1072) XendDomainInfo.handleShutdownWatch

The last line (XendDomainInfo.handleShutdownWatch) always seems to repeat twice, and then never repeat again.  Inside the domU itself, nothing is logged during the hang.  Since I cannot restore the vm, I can't see what is logged in its dmesg output.

I've compiled the vanilla kernel for the domU following steps outlined here:

	http://wiki.xensource.com/xenwiki/XenParavirtOps

Also, I've reviewed both of these threads to ensure that my issue is unrelated:

	http://lists.xensource.com/archives/html/xen-devel/2009-11/msg00334.html
	http://lists.xensource.com/archives/html/xen-devel/2009-11/msg01057.html

Thanks in advance for any assistance you can provide.

--
Major Hayden

Re: Hanging save/restore with 2.6.31.6 domU

[Jeremy Fitzhardinge]

On 12/08/09 13:43, Major Hayden wrote:
> I have a peculiar issue with xm save / xm restore when using vanilla 2.6.31.6 kernel from kernel.org in a domU on a server running Xen 3.1.2-rc1.
>    

There are a number of bugs with save/restore in vanilla Linux at the 
moment.  I've submitted fixes for them, but I don't think they've been 
picked up yet.  In the mean time, I recommend using the xen.git kernel.

     J

Re: Hanging save/restore with 2.6.31.6 domU

[Major Hayden]

Hey Jeremy,

Thanks for the reply.  I checked out the 2.6.31.4-rt14 tag from your git repo, built it, and I'm running into the same hanging suspend issue on Xen 3.0.3 and 3.1.2.  Xen 3.3 is able to save the VM without a problem (it could with the vanilla 2.6.31 tree as well).

Xen logs are still pretty quiet during the 'xm save':

DEBUG (__init__:1072) [xc_save]: /usr/lib64/xen/bin/xc_save 28 500 0 0 0
DEBUG (__init__:1072) suspend
DEBUG (__init__:1072) In saveInputHandler suspend
DEBUG (__init__:1072) Suspending 500 ...
DEBUG (__init__:1072) XendDomainInfo.shutdown(suspend)
DEBUG (__init__:1072) XendDomainInfo.handleShutdownWatch
DEBUG (__init__:1072) XendDomainInfo.handleShutdownWatch

Do you have a working .config you'd recommend to use as a failsafe?

-- Major

On Dec 8, 2009, at 3:54 PM, Jeremy Fitzhardinge wrote:

> On 12/08/09 13:43, Major Hayden wrote:
>> I have a peculiar issue with xm save / xm restore when using vanilla 2.6.31.6 kernel from kernel.org in a domU on a server running Xen 3.1.2-rc1.
>>   
> 
> There are a number of bugs with save/restore in vanilla Linux at the moment.  I've submitted fixes for them, but I don't think they've been picked up yet.  In the mean time, I recommend using the xen.git kernel.
> 
>    J