Re: POD: soft lockups in dom0 kernel

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 16/01/14 11:10, Jan Beulich wrote:
>>>> On 05.12.13 at 14:55, Dietmar Hahn <dietmar.hahn@ts.fujitsu.com> wrote:
>> when creating a bigger (> 50 GB) HVM guest with maxmem > memory we get
>> softlockups from time to time.
>>
>> kernel: [  802.084335] BUG: soft lockup - CPU#1 stuck for 22s! [xend:31351]
>>
>> I tracked this down to the call of xc_domain_set_pod_target() and further
>> p2m_pod_set_mem_target().
>>
>> Unfortunately I can this check only with xen-4.2.2 as I don't have a machine
>> with enough memory for current hypervisors. But it seems the code is nearly
>> the same.
> While I still didn't see a formal report of this against SLE11 yet,
> attached a draft patch against the SP3 code base adding manual
> preemption to the hypercall path of privcmd. This is only lightly
> tested, and therefore has a little bit of debugging code still left in
> there. Mind giving this an try (perhaps together with the patch
> David had sent for the other issue - there may still be a need for
> further preemption points in the IOCTL_PRIVCMD_MMAP*
> handling, but without knowing for sure whether that matters to
> you I didn't want to add this right away)?
>
> Jan
>

With my 4.4-rc2 testing, these softlockups are becoming more of a
problem, especially with construction/migration of 128GB guests.

I have been looking at doing a similar patch against mainline.

Having talked it through with David, it seems more sensible to have a
second hypercall page, at which point in_hypercall() becomes
in_preemptable_hypercall().

Any task (which could even be kernel tasks) could use the preemptable
page, rather than the main hypercall page, and the asm code doesn't need
to care whether the task was in privcmd.

This would avoid having to maintain extra state to identify whether the
hypercall was preemptable, and would avoid modification to
evtchn_do_upcall().

I shall see about hacking up a patch to this effect.

~Andrew