Dynamic-irq's in HVM domains

Dynamic-irq's in HVM domains

[James Harper]

'cat /proc/interrupts' in a Linux PV domain shows interrupts starting at
256 and labelled as 'Dynamic-irq'. Are these available in a HVM domain,
eg under Windows?

Thanks

James

RE: Dynamic-irq's in HVM domains

[James Harper]

I'm really desperate for an answer to this question...

Failing that, can I use physical IRQ's in a HVM domain to bind to? Eg
nothing is using IRQ 26, can I use that with bind_pirq? The only docs I
can find says 'Furthermore the calling domain must be sufficiently
privileged.', but I'm not sure how privileged 'sufficiently' means in
this context.

Thanks

James

> 
> 'cat /proc/interrupts' in a Linux PV domain shows interrupts starting
at
> 256 and labelled as 'Dynamic-irq'. Are these available in a HVM
domain,
> eg under Windows?
> 

RE: Dynamic-irq's in HVM domains

[Ian Campbell]

[Inverting your top-post]

> > 'cat /proc/interrupts' in a Linux PV domain shows interrupts starting
> > at 256 and labelled as 'Dynamic-irq'. Are these available in a HVM
> > domain eg under Windows?

On Fri, 2007-11-30 at 13:29 +1100, James Harper wrote:
> I'm really desperate for an answer to this question...

Dynamic IRQs are just a convenient Linux interface used to represent
event channels in order that they can be used with the regular IRQ
subsystem. Although I think they were added by Xen folk I don't think
there's anything Xen specific or special about them.

Event channels are (of course) exposed to HVM guests. The Linux
PV-on-HVM drivers export them itself via their own parallel
interrupt-like subsystem because not all the native versions supported
by the PV-on-HVM drivers have dynamic IRQs. See evtchn_interrupt() and
the other code in unmodified_drivers/linux-2.6/platform-pci/evtchn.c

Perhaps Windows has a similar concept to dynamic IRQs in its IRQ
subsystem, I'm afraid not being a Windows type I have no idea about
that. You can still use event channels outside of the regular windows
IRQ subsystem though, as demonstrated above.

> Failing that, can I use physical IRQ's in a HVM domain to bind to? Eg
> nothing is using IRQ 26, can I use that with bind_pirq? The only docs I
> can find says 'Furthermore the calling domain must be sufficiently
> privileged.', but I'm not sure how privileged 'sufficiently' means in
> this context.

I doubt that will work, I expect privileged means either domain 0 or a
domain with a PCI passthrough device attached...

Ian.

RE: Dynamic-irq's in HVM domains

[James Harper]

> > > 'cat /proc/interrupts' in a Linux PV domain shows interrupts
starting
> > > at 256 and labelled as 'Dynamic-irq'. Are these available in a HVM
> > > domain eg under Windows?
> 
> On Fri, 2007-11-30 at 13:29 +1100, James Harper wrote:
> > I'm really desperate for an answer to this question...
> 
> Dynamic IRQs are just a convenient Linux interface used to represent
> event channels in order that they can be used with the regular IRQ
> subsystem. Although I think they were added by Xen folk I don't think
> there's anything Xen specific or special about them.
> 
> Event channels are (of course) exposed to HVM guests. The Linux
> PV-on-HVM drivers export them itself via their own parallel
> interrupt-like subsystem because not all the native versions supported
> by the PV-on-HVM drivers have dynamic IRQs. See evtchn_interrupt() and
> the other code in unmodified_drivers/linux-2.6/platform-pci/evtchn.c
> 
> Perhaps Windows has a similar concept to dynamic IRQs in its IRQ
> subsystem, I'm afraid not being a Windows type I have no idea about
> that. You can still use event channels outside of the regular windows
> IRQ subsystem though, as demonstrated above.

I'm doing that... see below as to why it doesn't quite meet my needs...

> 
> > Failing that, can I use physical IRQ's in a HVM domain to bind to?
Eg
> > nothing is using IRQ 26, can I use that with bind_pirq? The only
docs I
> > can find says 'Furthermore the calling domain must be sufficiently
> > privileged.', but I'm not sure how privileged 'sufficiently' means
in
> > this context.
> 
> I doubt that will work, I expect privileged means either domain 0 or a
> domain with a PCI passthrough device attached...
> 

Hmmm... the problem I am facing is that I found 'class drivers' too
limiting in Windows, so I am trying to go down the scsi 'miniport'
route. There are 2 problems with that... the first is that due to a bug
in the Windows miniport implementation, you have to assign a hardware
i/o or memory resource to the driver. I have overcome that without too
much pain though.

The second problem is that the 'event complete' notification for
scsiport has to occur within the context of the scsi miniport driver
itself. I just can't use the event channel notification and call
'ScsiPortNotification' from there to say that the operation is complete
(it just gets ignored). And of course being Windows, I can't go poking
around in the source to see if there is another way to do it.

The only way I can think of is to assign an IRQ to the miniport driver,
and somehow get that to fire. If I can do that, everything will work
just fine.

I made the assumption that 'Dynamic-irq' is the same as virq... if not,
what is a virq? It appears to be something that the hypervisor is aware
of as there is a hypercall to bind to one.

Thanks for the response.

James

Re: Dynamic-irq's in HVM domains

[Pasi Kärkkäinen]

On Fri, Nov 30, 2007 at 08:02:08PM +1100, James Harper wrote:
> > 
> > > Failing that, can I use physical IRQ's in a HVM domain to bind to?
> Eg
> > > nothing is using IRQ 26, can I use that with bind_pirq? The only
> docs I
> > > can find says 'Furthermore the calling domain must be sufficiently
> > > privileged.', but I'm not sure how privileged 'sufficiently' means
> in
> > > this context.
> > 
> > I doubt that will work, I expect privileged means either domain 0 or a
> > domain with a PCI passthrough device attached...
> > 
> 
> Hmmm... the problem I am facing is that I found 'class drivers' too
> limiting in Windows, so I am trying to go down the scsi 'miniport'
> route. There are 2 problems with that... the first is that due to a bug
> in the Windows miniport implementation, you have to assign a hardware
> i/o or memory resource to the driver. I have overcome that without too
> much pain though.
> 

Might be a stupid question, but isn't "storport" the recommended driver
nowadays.. instead of scsi miniport? 

-- Pasi

RE: Dynamic-irq's in HVM domains

[James Harper]

> 
> Might be a stupid question, but isn't "storport" the recommended driver
> nowadays.. instead of scsi miniport?
> 

It is for Windows 2003 and above, but isn't available for XP or 2000. Porting a scsiport to a storport driver is very straightforward so I thought I'd do the former and then the latter.

James

Re: Dynamic-irq's in HVM domains

[Pasi Kärkkäinen]

On Fri, Nov 30, 2007 at 08:13:01PM +1100, James Harper wrote:
> > 
> > Might be a stupid question, but isn't "storport" the recommended driver
> > nowadays.. instead of scsi miniport?
> > 
> 
> It is for Windows 2003 and above, but isn't available for XP or 2000.
> Porting a scsiport to a storport driver is very straightforward so 
> I thought I'd do the former and then the latter.
> 

OK. Well, that makes sense.

-- Pasi

RE: Dynamic-irq's in HVM domains

[Ian Campbell]

I can't help with the Windows side I'm afraid, perhaps there is a
windows dev newsgroup or something that might be useful?

On Fri, 2007-11-30 at 20:02 +1100, James Harper wrote:
> I made the assumption that 'Dynamic-irq' is the same as virq... if
> not, what is a virq? It appears to be something that the hypervisor is
> aware of as there is a hypercall to bind to one.

They are IRQ sources from the hypervisor (as opposed to from a virtual
or physical device or another domain). From xen/include/public/xen.h:
        #define VIRQ_TIMER      0  /* V. Timebase update, and/or requested timeout.  */
        #define VIRQ_DEBUG      1  /* V. Request guest to dump debug info.           */
        #define VIRQ_CONSOLE    2  /* G. (DOM0) Bytes received on emergency console. */
        #define VIRQ_DOM_EXC    3  /* G. (DOM0) Exceptional event for some domain.   */
        #define VIRQ_TBUF       4  /* G. (DOM0) Trace buffer has records available.  */
        #define VIRQ_DEBUGGER   6  /* G. (DOM0) A domain has paused for debugging.   */
        #define VIRQ_XENOPROF   7  /* V. XenOprofile interrupt: new sample available */
        #define VIRQ_CON_RING   8  /* G. (DOM0) Bytes received on console            */
plus a handful of VIRQ_ARCH<n>

They are used by binding them to an event channel using
EVTCHNOP_bind_virq, as you've probably noticed.

Ian.

RE: Dynamic-irq's in HVM domains

[James Harper]

> 
> I can't help with the Windows side I'm afraid, perhaps there is a
> windows dev newsgroup or something that might be useful?

There might be. Curiously, windows communities are a bit different...
people like to keep their cards closer to their chest :)

> On Fri, 2007-11-30 at 20:02 +1100, James Harper wrote:
> > I made the assumption that 'Dynamic-irq' is the same as virq... if
> > not, what is a virq? It appears to be something that the hypervisor
is
> > aware of as there is a hypercall to bind to one.
> 
> They are IRQ sources from the hypervisor (as opposed to from a virtual
> or physical device or another domain). From xen/include/public/xen.h:

<snip>

> They are used by binding them to an event channel using
> EVTCHNOP_bind_virq, as you've probably noticed.

Ah. I had them backwards. I was assuming you could say 'when an event
channel fires, call this virq instead' and the virq would jump somewhere
into your idt.

I'll keep searching for another way...

Thanks

James

Re: Dynamic-irq's in HVM domains

[Mark Williamson]

> > I can't help with the Windows side I'm afraid, perhaps there is a
> > windows dev newsgroup or something that might be useful?
>
> There might be. Curiously, windows communities are a bit different...
> people like to keep their cards closer to their chest :)
>
> > On Fri, 2007-11-30 at 20:02 +1100, James Harper wrote:
> > > I made the assumption that 'Dynamic-irq' is the same as virq... if
> > > not, what is a virq? It appears to be something that the hypervisor
>
> is
>
> > > aware of as there is a hypercall to bind to one.
> >
> > They are IRQ sources from the hypervisor (as opposed to from a virtual
> > or physical device or another domain). From xen/include/public/xen.h:
>
> <snip>
>
> > They are used by binding them to an event channel using
> > EVTCHNOP_bind_virq, as you've probably noticed.
>
> Ah. I had them backwards. I was assuming you could say 'when an event
> channel fires, call this virq instead' and the virq would jump somewhere
> into your idt.
>
> I'll keep searching for another way...

Doesn't windows have some kind of deferred work processing like Linux softirqs 
or workqueues that you could used to schedule work to happen soon in driver 
context?  That'd enable you to do everything within existing windows APIs.

Afraid I've not done any Windows development, either ;-)

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

RE: Dynamic-irq's in HVM domains

[James Harper]

> Doesn't windows have some kind of deferred work processing like Linux
> softirqs
> or workqueues that you could used to schedule work to happen soon in
> driver
> context?  That'd enable you to do everything within existing windows
APIs.

Yes, but all scsiport calls have to be done from within the context of
the scsiport code. Aside from the request processing path where you get
a new srb (scsi request block) to process, you have two choices - the
isr, and a timer callback. Performance with the timer callback is
horrible, for obvious reasons. An isr has to be hooked to an actual irq.
Therein lies the problem...

The more I read about this the more I find other people looking for the
same sort of solution... I may have to abandon the whole scsiport idea,
which would be a shame.

Thanks

James