Re: [Xen-devel] [PATCH] xen: make sure stop_machine_run() is always called in a tasklet

From: "Jürgen Groß" <jgross@suse.com>
To: Igor Druzhinin <igor.druzhinin@citrix.com>,
	xen-devel@lists.xenproject.org
Cc: "Stefano Stabellini" <sstabellini@kernel.org>,
	"Julien Grall" <julien@xen.org>, "Wei Liu" <wl@xen.org>,
	"Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>,
	"George Dunlap" <George.Dunlap@eu.citrix.com>,
	"Andrew Cooper" <andrew.cooper3@citrix.com>,
	"Ian Jackson" <ian.jackson@eu.citrix.com>,
	"Jan Beulich" <jbeulich@suse.com>,
	"Roger Pau Monné" <roger.pau@citrix.com>
Subject: Re: [Xen-devel] [PATCH] xen: make sure stop_machine_run() is always called in a tasklet
Date: Sat, 15 Feb 2020 08:06:13 +0100	[thread overview]
Message-ID: <bd4b8fcc-6c70-4894-b356-3e97dae61470@suse.com> (raw)
In-Reply-To: <74d0b36a-4d80-39e1-aaf2-cf5209781e1e@citrix.com>

On 14.02.20 18:34, Igor Druzhinin wrote:
> On 14/02/2020 16:39, Jürgen Groß wrote:
>> On 14.02.20 15:06, Igor Druzhinin wrote:
>>> On 11/02/2020 09:35, Juergen Gross wrote:
>>>> With core scheduling active it is mandatory for stop_machine_run() to
>>>> be called in a tasklet only, as otherwise a scheduling deadlock would
>>>> occur: stop_machine_run() does a cpu rendezvous by activating a tasklet
>>>> on all other cpus. In case stop_machine_run() was not called in an idle
>>>> vcpu it would block scheduling the idle vcpu on its siblings with core
>>>> scheduling being active, resulting in a hang.
>>>
>>> I suppose rcu_barrier() is fine due to process_pending_softirqs() being
>>> called inside? I'm a little concerned by imposing is_vcpu_idle() restriction
>>> in that case as rcu_barrier() could be technically called from a non-tasklet
>>> context.
>>
>> No, stop_machine_run() with core scheduling active can only work when
>> called in an idle vcpu.
>>
>> OTOH it would be fairly easy to add another softirq for a similar
>> purpose and have a sync_machine_run() using that instead of tasklets.
>> This could be used for ucode loading, too.
>>
>> stop_machine_run() and sync_machine_run() could use a common main
>> function. The patch should be rather simple.
>>
>> Thoughts?
> 
> I have a patch on the list (which I was planning to send a v2 for) that
> fixes another issue with rcu_barrier():
> https://lists.xenproject.org/archives/html/xen-devel/2020-01/msg02273.html
> 
> As I understand it now that wouldn't work with core-scheduling. Do you think
> it's possible to synchronously wait for tasklets to finish in non-tasklet
> context (because that's what the purpose of rcu_barrier() is)?

No, won't work, unless we add preemption (basically would need per-vcpu
stacks instead of per-pcpu ones).

What might work IMO would be to do rcu_process_callbacks() no longer
during idle, but to have a specific softirq for that purpose. This would
remove the need to involve scheduling for rcu_barrier(). A brief check
of process_pending_softirqs() callers seems to allow that, but I'd like
to have a second opinion from someone having more rcu knowledge than me.
Single problematic users of process_pending_softirqs() could still be
switched to a variant not allowing the new rcu softirq.

Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel