trigger an interrupt in HVM

trigger an interrupt in HVM

[James Harper]

I need to trigger an interrupt on the HVM Xen PCI device. Is this
possible?

Could I just set vcpu_info->evtchn_upcall_pending = 1 and then do a
notify hypercall?

Thanks

James

RE: trigger an interrupt in HVM

[Cui, Dexuan]

What's the reason to try this? :-)
The interrupt injection into HVM guest doesn't use evtchn_upcall_pending -- that's for PV guest.
Maybe you can refer to tools/ioemu/hw/rtl8139.c to see how the rtl8139 device model injects interrupt into hvm guest (pls see pci_set_irq()).

-- Dexuan


-----Original Message-----
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of James Harper
Sent: 2008年8月21日 8:22
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] trigger an interrupt in HVM

I need to trigger an interrupt on the HVM Xen PCI device. Is this
possible?

Could I just set vcpu_info->evtchn_upcall_pending = 1 and then do a
notify hypercall?

Thanks

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: trigger an interrupt in HVM

[James Harper]

> What's the reason to try this? :-)
> The interrupt injection into HVM guest doesn't use
evtchn_upcall_pending -
> - that's for PV guest.
> Maybe you can refer to tools/ioemu/hw/rtl8139.c to see how the rtl8139
> device model injects interrupt into hvm guest (pls see pci_set_irq()).

I should have been more clear. This is from inside the DomU with the
GPLPV drivers.

The various subsystems (xennet, xenvbd) hook onto the same IRQ as the
PCI device. When the PCI device gets an IRQ, signalling that an event
channel has been set, and the event channel is for one of the subsystem
devices, it tells windows that the IRQ wasn't handled, so windows then
tries the other devices.

Windows makes it very tricky to get 'into' the scsiport context, and an
interrupt is the easiest way to do this, so if the PCI device driver
wants to tell the vbd driver to prepare for a suspend, setting a flag in
some shared data and triggering an interrupt is a good way to do this.
The alternative is a scsiport timer that polls the shared data. The
latter works but is a bit ugly.

Previously, I was attaching the subsystem drivers to fake IRQ's (eg in
the range 31-47) and using the asm 'int x' instruction to call them.
This is really bad for performance and caused a few other problems too.

Thanks

James

RE: trigger an interrupt in HVM

[Cui, Dexuan]

Oh, sorry, I didn't realize you meant the case of PV driver.
I think PV drivers for Linux/Windows should be alike here?
(I haven't read any codes of the Windows PV drivers :)

Thanks,
-- Dexuan


-----Original Message-----
From: James Harper [mailto:james.harper@bendigoit.com.au] 
Sent: 2008年8月21日 10:32
To: Cui, Dexuan; xen-devel@lists.xensource.com
Subject: RE: [Xen-devel] trigger an interrupt in HVM

> What's the reason to try this? :-)
> The interrupt injection into HVM guest doesn't use
evtchn_upcall_pending -
> - that's for PV guest.
> Maybe you can refer to tools/ioemu/hw/rtl8139.c to see how the rtl8139
> device model injects interrupt into hvm guest (pls see pci_set_irq()).

I should have been more clear. This is from inside the DomU with the
GPLPV drivers.

The various subsystems (xennet, xenvbd) hook onto the same IRQ as the
PCI device. When the PCI device gets an IRQ, signalling that an event
channel has been set, and the event channel is for one of the subsystem
devices, it tells windows that the IRQ wasn't handled, so windows then
tries the other devices.

Windows makes it very tricky to get 'into' the scsiport context, and an
interrupt is the easiest way to do this, so if the PCI device driver
wants to tell the vbd driver to prepare for a suspend, setting a flag in
some shared data and triggering an interrupt is a good way to do this.
The alternative is a scsiport timer that polls the shared data. The
latter works but is a bit ugly.

Previously, I was attaching the subsystem drivers to fake IRQ's (eg in
the range 31-47) and using the asm 'int x' instruction to call them.
This is really bad for performance and caused a few other problems too.

Thanks

James

RE: trigger an interrupt in HVM

[Tian, Kevin]

>From: James Harper
>Sent: 2008年8月21日 10:32
>
>The various subsystems (xennet, xenvbd) hook onto the same IRQ as the
>PCI device. When the PCI device gets an IRQ, signalling that an event
>channel has been set, and the event channel is for one of the subsystem
>devices, it tells windows that the IRQ wasn't handled, so windows then
>tries the other devices.

Same IRQ as which PCI device? For Linux HVM, a dummy PCI device
is exposed and a platform-pci driver is installed within Linux HVM. When
it accepts interrupt being event pending, it then invokes into each FE
driver directly which is bound to xenbus.

It seems that in your case xenbus is not present at all, and thus each
FE driver is still installed into existing sub-system of same class, and
then you're trying to find a communication channel between dummy
PCI driver and each FE which can't be done by simple function call
and is done by irq sharing by far, is that true?

Maybe to install FE driver onto xen bus type is easier, as the natural
layered function call is then available like in Linux side. I guess I may
still miss something here... :-(

Back to your question. It's viable but simply setting evtchn_upcall_pending
is not enough, as it's not checked at each VM resume path. By far
event pending caused interrupt injection is only triggered from a real
event send operation. One option you may try is to use inter-domain
event channel between PCI driver and each BE driver. I guess inter-
domain binding should allow for remote domain equal to local one,
and then you can simply use notify_remote_via_evtchn to trigger an
injection.

Thanks
Kevin

Re: trigger an interrupt in HVM

[Keir Fraser]

On 21/8/08 01:22, "James Harper" <james.harper@bendigoit.com.au> wrote:

> I need to trigger an interrupt on the HVM Xen PCI device. Is this
> possible?
> 
> Could I just set vcpu_info->evtchn_upcall_pending = 1 and then do a
> notify hypercall?

There is no such hypercall. In PV guests we can do any hypercall and the
evtchn_upcall_pending will be picked up. This isn't the case for an HVM
guest -- the IRQ is only asserted if an event is set pending by the
hypervisor itself.

If you must be able to trigger event delivery via the HVM IRQ mechanism from
within your HVM guest then I suggest you allocate an IPI event channel and
do a notify hypercall on that. That will have the side effect of triggering
an interrupt and 'flushing through' all other pending events.

 -- Keir

RE: trigger an interrupt in HVM

[James Harper]

> There is no such hypercall. In PV guests we can do any hypercall and
the
> evtchn_upcall_pending will be picked up. This isn't the case for an
HVM
> guest -- the IRQ is only asserted if an event is set pending by the
> hypervisor itself.
> 
> If you must be able to trigger event delivery via the HVM IRQ
mechanism
> from
> within your HVM guest then I suggest you allocate an IPI event channel
and
> do a notify hypercall on that. That will have the side effect of
> triggering
> an interrupt and 'flushing through' all other pending events.
> 

As an alternative, could I do an alloc_unbound (DOMID_SELF, DOMID_SELF)
and then bind_interdomain to it?

As another alternative, I have previously been using the asm 'int x'
instructions to call isr's... that could work too although of course
without the source code to windows it's hard to tell exactly what the
implications of that might be :)

James

Re: trigger an interrupt in HVM

[Keir Fraser]

On 22/8/08 02:24, "James Harper" <james.harper@bendigoit.com.au> wrote:

> As an alternative, could I do an alloc_unbound (DOMID_SELF, DOMID_SELF)
> and then bind_interdomain to it?

Yes, it's just that that burns two event-channel ports, whereas an IPI port
burns just one.

> As another alternative, I have previously been using the asm 'int x'
> instructions to call isr's... that could work too although of course
> without the source code to windows it's hard to tell exactly what the
> implications of that might be :)

I wouldn't advise it -- the ISR will EOI the LAPIC even though the LAPIC did
not deliver the interrupt. A safer way to go might be to self-IPI via the
LAPIC.

 -- Keir

RE: trigger an interrupt in HVM

[James Harper]

> If you must be able to trigger event delivery via the HVM IRQ
mechanism
> from
> within your HVM guest then I suggest you allocate an IPI event channel
and
> do a notify hypercall on that. That will have the side effect of
> triggering
> an interrupt and 'flushing through' all other pending events.
> 

That doesn't seem to work... I allocate the IPI event channel which in
my case gives me event channel 7, but a notify on port 7 doesn't appear
to end up calling the IRQ... or if it does, it doesn't set the bits for
that port.

James

RE: trigger an interrupt in HVM

[James Harper]

> 
> > If you must be able to trigger event delivery via the HVM IRQ
> mechanism
> > from
> > within your HVM guest then I suggest you allocate an IPI event
channel
> and
> > do a notify hypercall on that. That will have the side effect of
> > triggering
> > an interrupt and 'flushing through' all other pending events.
> >
> 
> That doesn't seem to work... I allocate the IPI event channel which in
> my case gives me event channel 7, but a notify on port 7 doesn't
appear
> to end up calling the IRQ... or if it does, it doesn't set the bits
for
> that port.
> 

Disregard that. I wasn't unmasking the event channel.

James

Re: trigger an interrupt in HVM

[Keir Fraser]

On 26/8/08 04:19, "James Harper" <james.harper@bendigoit.com.au> wrote:

>> If you must be able to trigger event delivery via the HVM IRQ
> mechanism
>> from
>> within your HVM guest then I suggest you allocate an IPI event channel
> and
>> do a notify hypercall on that. That will have the side effect of
>> triggering
>> an interrupt and 'flushing through' all other pending events.
>> 
> 
> That doesn't seem to work... I allocate the IPI event channel which in
> my case gives me event channel 7, but a notify on port 7 doesn't appear
> to end up calling the IRQ... or if it does, it doesn't set the bits for
> that port.

You did request to bind the IPI port to vcpu0? Otherwise it won't work.

Also you will need to handle that port in your IRQ handler somehow (at least
the event selector bit and the event pending bit). Otherwise the IPI will
obviously only fire once.

 -- Keir