Re: [PATCH] xen/arm: Fix rtds scheduler for arm

[Ian Campbell <Ian.Campbell@citrix.com>]

On Mon, 2015-02-02 at 12:59 +0000, Julien Grall wrote:
> On 02/02/15 12:16, Ian Campbell wrote:
> > On Mon, 2015-02-02 at 11:40 +0000, Jan Beulich wrote:
> >>>>> On 02.02.15 at 12:14, <Ian.Campbell@citrix.com> wrote:
> >>> On Mon, 2015-02-02 at 12:49 +0200, Denys Drozdov wrote:
> >>>> The issue observed on credit2 scheduler is similar to the rt scheduler
> >>>> on arm platform. The root cause is that interrupts are disabled in the
> >>>> beginning of arm context_switch, thus spin_lock_irq is failing in
> >>>> ASSERT(local_irq_is_enabled()). I propose to change both credit2 and
> >>>> rt scheduler to run on arm platform as well and re-run the regression
> >>>> with scheduler patches.
> >>>
> >>> I'd like to hear from the scheduler and other $arch folks regarding
> >>> whether we think the rtds and credit2 schedulers are wrong or whether it
> >>> is actually the ARM arch code which needs fixing before considering any
> >>> change.
> >>
> >> The aspect to be understood first is why ARM needs IRQs disabled
> >> past __context_switch() into schedule_tail() (and there until after
> >> ctxt_switch_from() and ctxt_switch_to()). If that's indeed necessary,
> >> there's no question that the schedulers need to be adjusted to
> >> accommodate for this.
> > 
> > I don't think it's *necessary*, but we do seem to have ended up with the
> > context switch having that requirement today (and relying on it in
> > several places in switch from/to (mostly to).
> 
> > I'm pretty sure we could rework things more along the lines of how x86
> > does it if needed, but it would be a non-trivial refactoring I think.
> 
> We have some part of the code which may inject an interrupt during
> context switch.
> For instance the timer may inject an IRQ as long as it has not been
> disabled. Same when the timer is restored.
> 
> The former may result to inject an interrupt to the wrong vCPU.

I'm pretty sure we could structure things, with appropriate use of
locks, smaller critical sections with IRQs disabled, better quiescing of
subsystems on save and ordering of operations etc such that this would
work without the big hammer of disabling IRQs for the entire context
switch.

In fact I rather expect that eventually some embedded RTOS type person
will want to do exactly that to minimize IRQ latency.

Ian.