RE: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding

Wu, Feng
2015-06-18T09:16:35Z

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 5794 bytes --]

> -----Original Message-----
> From: Alex Williamson [mailto:alex.williamson@redhat.com]
> Sent: Tuesday, June 16, 2015 12:45 AM
> To: Eric Auger
> Cc: Avi Kivity; Wu, Feng; kvm@vger.kernel.org; linux-kernel@vger.kernel.org;
> pbonzini@redhat.com; mtosatti@redhat.com
> Subject: Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding
>
> On Mon, 2015-06-15 at 18:17 +0200, Eric Auger wrote:
> > Hi Alex, all,
> > On 06/12/2015 09:03 PM, Alex Williamson wrote:
> > > On Fri, 2015-06-12 at 21:48 +0300, Avi Kivity wrote:
> > >> On 06/12/2015 06:41 PM, Alex Williamson wrote:
> > >>> On Fri, 2015-06-12 at 00:23 +0000, Wu, Feng wrote:
> > >>>>> -----Original Message-----
> > >>>>> From: Avi Kivity [mailto:avi.kivity@gmail.com]
> > >>>>> Sent: Friday, June 12, 2015 3:59 AM
> > >>>>> To: Wu, Feng; kvm@vger.kernel.org; linux-kernel@vger.kernel.org
> > >>>>> Cc: pbonzini@redhat.com; mtosatti@redhat.com;
> > >>>>> alex.williamson@redhat.com; eric.auger@linaro.org
> > >>>>> Subject: Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding
> > >>>>>
> > >>>>> On 06/11/2015 01:51 PM, Feng Wu wrote:
> > >>>>>> From: Eric Auger <eric.auger@linaro.org>
> > >>>>>>
> > >>>>>> This patch adds and documents a new KVM_DEV_VFIO_DEVICE
> group
> > >>>>>> and 2 device attributes: KVM_DEV_VFIO_DEVICE_FORWARD_IRQ,
> > >>>>>> KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. The purpose is to be
> able
> > >>>>>> to set a VFIO device IRQ as forwarded or not forwarded.
> > >>>>>> the command takes as argument a handle to a new struct named
> > >>>>>> kvm_vfio_dev_irq.
> > >>>>> Is there no way to do this automatically? After all, vfio knows that a
> > >>>>> device interrupt is forwarded to some eventfd, and kvm knows that
> some
> > >>>>> eventfd is forwarded to a guest interrupt. If they compare notes
> > >>>>> through a central registry, they can figure out that the interrupt needs
> > >>>>> to be forwarded.
> > >>>> Oh, just like Eric mentioned in his reply, this description is out of context
> of
> > >>>> this series, I will remove them in the next version.
> > >>>
> > >>> I suspect Avi's question was more general. While forward/unforward is
> > >>> out of context for this series, it's very similar in nature to
> > >>> enabling/disabling posted interrupts. So I think the question remains
> > >>> whether we really need userspace to participate in creating this
> > >>> shortcut or if kvm and vfio can some how orchestrate figuring it out
> > >>> automatically.
> > >>>
> > >>> Personally I don't know how we could do it automatically. We've always
> > >>> relied on userspace to independently setup vfio and kvm such that
> > >>> neither have any idea that the other is there and update each side
> > >>> independently when anything changes. So it seems consistent to
> continue
> > >>> that here. It doesn't seem like there's much to gain performance-wise
> > >>> either, updates should be a relatively rare event I'd expect.
> > >>>
> > >>> There's really no metadata associated with an eventfd, so "comparing
> > >>> notes" automatically might imply some central registration entity. That
> > >>> immediately sounds like a much more complex solution, but maybe Avi
> has
> > >>> some ideas to manage it. Thanks,
> > >>>
> > >>
> > >> The idea is to have a central registry maintained by a posted interrupts
> > >> manager. Both vfio and kvm pass the filp (along with extra information)
> > >> to the posted interrupts manager, which, when it detects a filp match,
> > >> tells each of them what to do.
> > >>
> > >> The advantages are:
> > >> - old userspace gains the optimization without change
> > >> - a userspace API is more expensive to maintain than internal kernel
> > >> interfaces (CVEs, documentation, maintaining backwards compatibility)
> > >> - if you can do it without a new interface, this indicates that all the
> > >> information in the new interface is redundant. That means you have to
> > >> check it for consistency with the existing information, so it's extra
> > >> work (likely, it's exactly what the posted interrupt manager would be
> > >> doing anyway).
> > >
> > > Yep, those all sound like good things and I believe that's similar in
> > > design to the way we had originally discussed this interaction at
> > > LPC/KVM Forum several years ago. I'd be in favor of that approach.
> >
> > I guess this discussion also is relevant wrt "[RFC v6 00/16] KVM-VFIO
> > IRQ forward control" series? Or is that "central registry maintained by
> > a posted interrupts manager" something more specific to x86?
>
> I'd think we'd want it for any sort of offload and supporting both
> posted-interrupts and irq-forwarding would be a good validation. I
> imagine there would be registration/de-registration callbacks separate
> for interrupt producers vs interrupt consumers. Each registration
> function would likely provide a struct of callbacks, probably similar to
> the get_symbol callbacks proposed for the kvm-vfio device on the IRQ
> producer side. The eventfd would be the token that the manager would
> use to match producers and consumers. The hard part is probably
> figuring out what information to retrieve from the producer and provide
> to the consumer in a generic way between pci and platform, but as an
> internal interface, it's not a big deal if we screw it up a few times to
> start. Thanks,

On posted-interrupts side, the main purpose of the new APIs is to update
the IRTE when guest changes vMSI/vMSIx configuration. Alex, do you have
any detailed ideas for the new solution to achieve this purpose? It should
be helpful if you can share some!

Thanks,
Feng

>
> Alex

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éÝ¶\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dÊ‡Ú™ë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥