Re: [PATCH v3.1 01/15] xen/x86: remove XENFEAT_hvm_pirqs for PVHv2 guests

[Roger Pau Monne <roger.pau@citrix.com>]

On Thu, Nov 03, 2016 at 10:22:37AM -0400, Konrad Rzeszutek Wilk wrote:
> On Thu, Nov 03, 2016 at 01:35:37PM +0100, Roger Pau Monne wrote:
> > On Mon, Oct 31, 2016 at 10:32:47AM -0600, Jan Beulich wrote:
> > > >>> On 29.10.16 at 10:59, <roger.pau@citrix.com> wrote:
> > > > PVHv2 guests, unlike HVM guests, won't have the option to route interrupts
> > > > from physical or emulated devices over event channels using PIRQs. This
> > > > applies to both DomU and Dom0 PVHv2 guests.
> > > > 
> > > > Introduce a new XEN_X86_EMU_USE_PIRQ to notify Xen whether a HVM guest can
> > > > route physical interrupts (even from emulated devices) over event channels,
> > > > and is thus allowed to use some of the PHYSDEV ops.
> > > > 
> > > > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > > 
> > > The patch looks fine now for its purpose, but I'm hesitant to ack it
> > > without us having settled on whether we indeed mean to hide all
> > > those physdev ops from Dom0. In particular I don't recall this (and
> > > the reasoning behind it) having got written down somewhere.
> > 
> > I'm planning to add the following doc update together with this commit:
> > 
> > diff --git a/docs/misc/hvmlite.markdown b/docs/misc/hvmlite.markdown
> > index 946908e..4fc757f 100644
> > --- a/docs/misc/hvmlite.markdown
> > +++ b/docs/misc/hvmlite.markdown
> > @@ -75,3 +75,38 @@ info structure that's passed at boot time (field rsdp_paddr).
> >  
> >  Description of paravirtualized devices will come from XenStore, just as it's
> >  done for HVM guests.
> > +
> > +## Interrupts ##
> > +
> > +### Interrupts from physical devices ###
> > +
> > +Interrupts from physical devices are delivered using native methods, this is
> > +done in order to take advantage of new hardware assisted virtualization
> > +functions, like posted interrupts. This implies that PVHv2 guests with physical
> > +devices will also have the necessary interrupt controllers in order to manage
> > +the delivery of interrupts from those devices, using the same interfaces that
> > +are available on native hardware.
> > +
> > +### Interrupts from paravirtualized devices ###
> > +
> > +Interrupts from paravirtualized devices are delivered using event channels, see
> > +[Event Channel Internals][event_channels] for more detailed information about
> 
> Is this a must? This mechanism was designed before vAPIC was present -
> and has the inherent disadvantage that:
> 
>  1) It can't use vAPIC (it actually has to disable this as it needs to
>     turn on VMX interrupt window to do this).
> 
>  2) It is hackish. It completly bypasses the APIC and it uses the
>     VM_ENTRY_INTR_INFO (suppose to be used for traps).
> 
>  3) It is also racy for events that are more than 64 values apart (with
>     the old 2 level one). That is you can have this callback vector
>     being injected couple of times - as the OS interrupt handler does not
>     mask the events.
> 
>  4) It causes the guest an VMEXIT (to stop it so that we can tweak
>     the VM_ENTRY_INTR_INFO).
> 
> If we really want to use it, could we instead use the per-vector
> that Paul added? HVMOP_set_evtchn_upcall_vector?

PVHv2 should be able to use the same mechanism as HVM guests for event 
channel delivery. I'm not familiar with HVMOP_set_evtchn_upcall_vector, but 
AFAICT it's very similar to HVM_PARAM_CALLBACK_IRQ with the difference that 
each vCPU can specify different vectors, right?

> Or perhaps just add events -> MSI-X mechanism and then we can also
> do this under normal HVM guests?

That would be OK, but I think this is something that's out of the scope 
here. If this is ever implemented for HVM guests PVHv2 should also be able 
to use it, provided they have a local APIC.

> Either option would require changes in Linux/FreeBSD to deal with this.
> > +event channels. In order to inject interrupts into the guest an IDT vector is
> > +used. This is the same mechanism used on PVHVM guests, and allows having
> > +per-cpu interrupts that can be also used to deliver timers or IPIs if desired.
> > +
> > +In order to register the callback IDT vector the `HVMOP_set_param` hypercall
> > +is used with the following values:
> > +
> > +    domid = DOMID_SELF
> > +    index = HVM_PARAM_CALLBACK_IRQ
> > +    value = (0x2 << 56) | vector_value
> > +
> > +The OS has to program the IDT for the `vector_value` using the baremetal
> > +mechanism.
> > +
> > +In order to know which event channel has fired, we need to look into the
> > +information provided in the `shared_info` structure. The `evtchn_pending`
> > +array is used as a bitmap in order to find out which event channel has
> > +fired. Event channels can also be masked by setting it's port value in the
> > +`shared_info->evtchn_mask` bitmap.
> 
> .. Well that is for the 2-level, but the FIFO is a bit different.

Right, I've just copy-pasted this from the classic PVH documented, which is 
clearly outdated now. We should have a document that describes how event 
channels should be used, both l2 and fifo implementations.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel