Using IRQ_GUEST|IRQ_PER_CPU to signal "delivery to current vcpu"?

Using IRQ_GUEST|IRQ_PER_CPU to signal "delivery to current vcpu"?

[Ian Campbell]

Hi arch maintainers,

The ARM hardware has a concept called a "Peripheral Private Interrupt"
(PPI), an IRQ which is per-core as opposed to a "Shared Peripheral
Interrupt" (SPI) which is per-SOC (routable to all cores).

(For completeness the other classes of IRQ are "Software Generated
Interrupt" (SGI) AKA "IPI" and "Locality-specific Peripheral Interrupt"
(LPI) which is, approximately, an MSI).

PPIs are typically generated by per-core resources, such as the timer
block, or PMU blocks etc.

I would like to arrange for a new (I think) class of interrupt within Xen,
which is an interrupt (necessarily a PPI) which is always delivered to
whichever VCPU is currently resident on a given PCPU and use this to
arrange to deliver vtimer interrupts direct to the current VCPU, taking
care of context switching the status (is it pending? active? etc) of the
PPI while context switching the vtimer (something which the h/w supports
quite easily and which then avoids races wrt guests unmasking the timer
while it is still active in the interrupt virtual controller but not the
real one).

My first cut ([0], from January this year) keyed this behaviour on an ARM
specific struct irq_guest field ->d (domain) being NULL, which I wasn't
terribly comfortable with. In my WIP v2 I have current added a new boolean
field to struct irq_guest, which seems rather wasteful due to rounding up
of the size etc and so I'm not really happy with that either (I also
considered and rejected other tricks like stealing a bit from an existing
field, e.g. the unsigned virq).

So, I was considering using a new bit in the common desc->status bit. Upon
checking xen/irq.h I found there was already IRQ_PER_CPU which appears to
be completely unused in the current code base (it looks to have been used
by IA64 and PPC, but never by x86 and not currently used AFAICT by ARM, x86
or common code.)

It seems like IRQ_GUEST|IRQ_PER_CPU is a pretty good fit for the semantics
I want, but I think it would be confusing to define it as meaning such only
on ARM and having either common code or other architectures ascribe a
different meaning (other than "meaningless on this arch") to it.

I think the other possible use case of this scenario (routing a PPI to a
specific guest) seems rather unlikely (since it would imply pinning in
various odd ways). I think that use case can also be dealt with by the
context switching code which can reroute the interrupt as needed (i.e.
unmask only when the relevant vcpu is running).

Any thoughts?

Arguably all PPIs on ARM (even "normal" ones) ought to have IRQ_PER_CPU set
too but that is something of an aside.

Cheers,
Ian.

[0] http://lists.xen.org/archives/html/xen-devel/2015-01/msg03161.html

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Using IRQ_GUEST|IRQ_PER_CPU to signal "delivery to current vcpu"?

[Jan Beulich]

>>> On 10.11.15 at 11:48, <ian.campbell@citrix.com> wrote:
> So, I was considering using a new bit in the common desc->status bit. Upon
> checking xen/irq.h I found there was already IRQ_PER_CPU which appears to
> be completely unused in the current code base (it looks to have been used
> by IA64 and PPC, but never by x86 and not currently used AFAICT by ARM, x86
> or common code.)
> 
> It seems like IRQ_GUEST|IRQ_PER_CPU is a pretty good fit for the semantics
> I want, but I think it would be confusing to define it as meaning such only
> on ARM and having either common code or other architectures ascribe a
> different meaning (other than "meaningless on this arch") to it.
> 
> I think the other possible use case of this scenario (routing a PPI to a
> specific guest) seems rather unlikely (since it would imply pinning in
> various odd ways). I think that use case can also be dealt with by the
> context switching code which can reroute the interrupt as needed (i.e.
> unmask only when the relevant vcpu is running).
> 
> Any thoughts?

Sounds reasonable.

Jan

Re: Using IRQ_GUEST|IRQ_PER_CPU to signal "delivery to current vcpu"?

[Stefano Stabellini]

On Tue, 10 Nov 2015, Ian Campbell wrote:
> Hi arch maintainers,
> 
> The ARM hardware has a concept called a "Peripheral Private Interrupt"
> (PPI), an IRQ which is per-core as opposed to a "Shared Peripheral
> Interrupt" (SPI) which is per-SOC (routable to all cores).
> 
> (For completeness the other classes of IRQ are "Software Generated
> Interrupt" (SGI) AKA "IPI" and "Locality-specific Peripheral Interrupt"
> (LPI) which is, approximately, an MSI).
> 
> PPIs are typically generated by per-core resources, such as the timer
> block, or PMU blocks etc.
> 
> I would like to arrange for a new (I think) class of interrupt within Xen,
> which is an interrupt (necessarily a PPI) which is always delivered to
> whichever VCPU is currently resident on a given PCPU and use this to
> arrange to deliver vtimer interrupts direct to the current VCPU, taking
> care of context switching the status (is it pending? active? etc) of the
> PPI while context switching the vtimer (something which the h/w supports
> quite easily and which then avoids races wrt guests unmasking the timer
> while it is still active in the interrupt virtual controller but not the
> real one).
> 
> My first cut ([0], from January this year) keyed this behaviour on an ARM
> specific struct irq_guest field ->d (domain) being NULL, which I wasn't
> terribly comfortable with. In my WIP v2 I have current added a new boolean
> field to struct irq_guest, which seems rather wasteful due to rounding up
> of the size etc and so I'm not really happy with that either (I also
> considered and rejected other tricks like stealing a bit from an existing
> field, e.g. the unsigned virq).
> 
> So, I was considering using a new bit in the common desc->status bit. Upon
> checking xen/irq.h I found there was already IRQ_PER_CPU which appears to
> be completely unused in the current code base (it looks to have been used
> by IA64 and PPC, but never by x86 and not currently used AFAICT by ARM, x86
> or common code.)
> 
> It seems like IRQ_GUEST|IRQ_PER_CPU is a pretty good fit for the semantics
> I want, but I think it would be confusing to define it as meaning such only
> on ARM and having either common code or other architectures ascribe a
> different meaning (other than "meaningless on this arch") to it.
> 
> I think the other possible use case of this scenario (routing a PPI to a
> specific guest) seems rather unlikely (since it would imply pinning in
> various odd ways). I think that use case can also be dealt with by the
> context switching code which can reroute the interrupt as needed (i.e.
> unmask only when the relevant vcpu is running).
> 
> Any thoughts?
> 
> Arguably all PPIs on ARM (even "normal" ones) ought to have IRQ_PER_CPU set
> too but that is something of an aside.

It seems reasonable.