newer Linux: handle_level_irq() vs. handle_fasteoi_irq()

newer Linux: handle_level_irq() vs. handle_fasteoi_irq()

[Jan Beulich]

In order to be able to properly disable level triggered interrupts that,
due to malfunctioning hardware/firmware, never get de-asserted,
the use of handle_level_irq() in both pv-ops Dom0 and forward
ported kernels (starting with 2.6.19) seems problematic: Other than
with 2.6.18's use of __do_IRQ(), irq_chip->end() doesn't get called
anymore, and there also is no replacement call made from that
function (->umask() is only being called on non-disabled IRQs), and
hence the respective logic contained in pirq_end() doesn't really get
used anymore. handle_fasteoi_irq(), otoh, has an unconditional
->eoi() callout at its end, which would suit these needs.

The main difference of handle_fasteoi_irq() compared with
handle_level_irq() is that the IRQ doesn't get masked upfront.
I'm not really that much into the necessary details here, but it
would seem that using handle_fasteoi_irq() should be possible
if a hypothetical pirq_eoi() called clear_evtchn() along with
what end_pirq() currently does.

Any insight on potential problems with this would be
appreciated.

Jan

Re: newer Linux: handle_level_irq() vs. handle_fasteoi_irq()

[Konrad Rzeszutek Wilk]

On Thu, Apr 22, 2010 at 01:09:08PM +0100, Jan Beulich wrote:
> In order to be able to properly disable level triggered interrupts that,
> due to malfunctioning hardware/firmware, never get de-asserted,
> the use of handle_level_irq() in both pv-ops Dom0 and forward
> ported kernels (starting with 2.6.19) seems problematic: Other than
> with 2.6.18's use of __do_IRQ(), irq_chip->end() doesn't get called
> anymore, and there also is no replacement call made from that
> function (->umask() is only being called on non-disabled IRQs), and
> hence the respective logic contained in pirq_end() doesn't really get
> used anymore. handle_fasteoi_irq(), otoh, has an unconditional
> ->eoi() callout at its end, which would suit these needs.

In the PV-OPS kernel, there are no calls to manipulate the LAPIC or
IO/APIC anymore. The ACK is replaced with a PHYSDEVOP_eoi hypercall.

The unmasking/masking is done on the event channel. All of the logic
related to handling either the edge or level (or fasteoi level) is done
in the hypervisor.
> 
> The main difference of handle_fasteoi_irq() compared with
> handle_level_irq() is that the IRQ doesn't get masked upfront.
> I'm not really that much into the necessary details here, but it
> would seem that using handle_fasteoi_irq() should be possible
> if a hypothetical pirq_eoi() called clear_evtchn() along with
> what end_pirq() currently does.
> 
> Any insight on potential problems with this would be
> appreciated.

<scratches his head>  I don't know about the XenLinux forward ported
patches but I think in the PV-OPS realm we don't have to worry about it.

Re: newer Linux: handle_level_irq() vs. handle_fasteoi_irq()

[Jan Beulich]

>>> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> 22.04.10 21:38 >>>
>On Thu, Apr 22, 2010 at 01:09:08PM +0100, Jan Beulich wrote:
>> In order to be able to properly disable level triggered interrupts that,
>> due to malfunctioning hardware/firmware, never get de-asserted,
>> the use of handle_level_irq() in both pv-ops Dom0 and forward
>> ported kernels (starting with 2.6.19) seems problematic: Other than
>> with 2.6.18's use of __do_IRQ(), irq_chip->end() doesn't get called
>> anymore, and there also is no replacement call made from that
>> function (->umask() is only being called on non-disabled IRQs), and
>> hence the respective logic contained in pirq_end() doesn't really get
>> used anymore. handle_fasteoi_irq(), otoh, has an unconditional
>> ->eoi() callout at its end, which would suit these needs.
>
>In the PV-OPS kernel, there are no calls to manipulate the LAPIC or
>IO/APIC anymore. The ACK is replaced with a PHYSDEVOP_eoi hypercall.
>
>The unmasking/masking is done on the event channel. All of the logic
>related to handling either the edge or level (or fasteoi level) is done
>in the hypervisor.

Sure, that has always been that way (even in 2.6.18).

One thing I missed though when writing the original mail is that
there is a ->disable() method that could be leveraged. It's empty
in the pv-ops case, but should probably become an alias of
shutdown_irq() (and ->enable already calling startup_pirq()).

>> The main difference of handle_fasteoi_irq() compared with
>> handle_level_irq() is that the IRQ doesn't get masked upfront.
>> I'm not really that much into the necessary details here, but it
>> would seem that using handle_fasteoi_irq() should be possible
>> if a hypothetical pirq_eoi() called clear_evtchn() along with
>> what end_pirq() currently does.
>> 
>> Any insight on potential problems with this would be
>> appreciated.
>
><scratches his head>  I don't know about the XenLinux forward ported
>patches but I think in the PV-OPS realm we don't have to worry about it.

We have to: Xen has no way to know whether to disable an IRQ
without the handling domain(s) telling it.

And given that the fasteoi variant uses one less indirect call, it
would seem still worthwhile to convert to it from the level variant
currently used. While in the pv-ops case dynirq-s mask their event
channel via disable_dynirq(), on the forward ported tree (as well
as 2.6.18) ack_dynirq() and ack_pirq() are identical, hence this
operation could be done directly in evtchn_do_upcall(), and
->eoi() would then replace ->end().

One other inefficiency I noted in the pv-ops tree is that neither
xen_dynamic_chip nor xen_pirq_chip have a mask_ack method,
yet handle_level_irq() would prefer to use that over calling
->mask() and ->ack() separately (and with ->mask() doing
nothing both .ack and .mask_ack could point to the same
function).

Jan