Re: questions of vm save/restore on arm64

[Julien Grall <julien.grall@arm.com>]

Hello,

On 04/06/16 03:37, Chenxiao Zhao wrote:
>
>
> On 6/3/2016 4:02 AM, Julien Grall wrote:
>> Hello,
>>
>> First thing, the time in the mail headers seems to be wrong. Maybe
>> because of a wrong timezone?
>>
>> I got: 04/06/16 02:32 however we are still the 3rd in my timezone.
>>
>> On 04/06/16 02:32, Chenxiao Zhao wrote:
>>>
>>>
>>> On 6/3/2016 3:16 AM, Julien Grall wrote:
>>>> Hello,
>>>>
>>>> On 03/06/16 18:05, Chenxiao Zhao wrote:
>>>>> I finally found out that the problem is that the toolstack did not get
>>>>> corret p2m_size while sending all pages on save(always be zero).
>>>>> After I
>>>>> fixed that, the guest could be restored but guest kernel caught
>>>>> handle_mm_fault().
>>>>>
>>>>> where do you think I'm going to investigate, guest kernel hibernation
>>>>> restore or xen?
>>>>
>>>> The hibernation support for ARM64 has only been merged recently in the
>>>> kernel. Which kernel are you using?
>>>
>>> Hi Julien,
>>>
>>> I'm using a linaro ported Linux kernel 4.1 for hikey from this link.
>>>
>>> https://github.com/96boards/linux/tree/android-hikey-linaro-4.1
>>>
>>> I also applied following patches to make the kernel support hibernation.
>>
>> This looks the wrong way to do it as this series may requires some
>> patches which have been upstreamed before hand.
>>
>> Linux upstream seems support to the hikey board [1]. Any reason to not
>> using it?
>
> I tried a newer version of kernel 4.4, but got no luck to start dom0
> with xen. so I decide to stay in 4.1 for now.

The current upstream is 4.7-rc1 not 4.4.

However, the kernel for the guest does not require any support for your 
board so you can use upstream (i.e linus/master).

[...]

>>
>>>
>>>  From my understanding, a kernel hibernation will cause kernel to save
>>> memories to disk(swap partition). But on guest save progress, the
>>> hibernation for domU does not make the guest save memories to disk. it's
>>> more like suspend all processes in guest, and memors actually depends on
>>> xen toolstack to save the pages to file. Am I correct?
>>
>> You are using an older tree with a patch series based on a newer tree.
>>
>> So I would recommend you to move to a newer tree. If it is not possible,
>> please test that hibernation works on baremetal.
>
> I think the suspend/resume in guest is working, cause I can use
> pause/unpause command in toolstack to suspend/resume guest without
> problem. I can also see the suspend/resume kernel messages from guest's
> console. The only problem is it's can not resume from restore.

The commands pause/unpause do not require any kind of cooperation with 
the kernel. They are only request to the hypervisor to put the vCPUs in 
sleep or to wake them up.

You can look at the implementation of libxl_domain_{,un}pause.

> One thing that confused me is that the kernel's hibernation means the
> guest kernel will save the memory state to disk and power off VM at
> last. The guest will also take care of the memory restore itself. But I
> do not see the save/restore on xen works that way. So my question is why
> it requires hibernation (aka. suspend to disk) instead of the real
> suspend (aka. suspend to RAM and standby)?

I am not an expert in the suspend/resume of Xen. However by looking at 
the code, Xen has a specific path to suspend (see drivers/xen/manage.c). 
I guess, this code requires features which are only present when 
CONFIG_HIBERNATION is selected.

In any case, please use upstream Linux for the development in the guest. 
If there is still a bug, then we know that it is not because you are 
using a 4.5 based patch series in a 4.1 kernel.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel