From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752554AbbJFOn4 (ORCPT ); Tue, 6 Oct 2015 10:43:56 -0400 Received: from mail-wi0-f173.google.com ([209.85.212.173]:34707 "EHLO mail-wi0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751979AbbJFOny (ORCPT ); Tue, 6 Oct 2015 10:43:54 -0400 Subject: Re: [PATCH v3 2/3] uio_pci_generic: add MSI/MSI-X support To: "Michael S. Tsirkin" , Avi Kivity References: <1443991398-23761-1-git-send-email-vladz@cloudius-systems.com> <1443991398-23761-3-git-send-email-vladz@cloudius-systems.com> <20151005031159.GB27303@kroah.com> <56123493.9000602@scylladb.com> <20151005094932.GA5236@kroah.com> <56124EDB.3070701@scylladb.com> <20151006143821.GA11541@redhat.com> Cc: Greg KH , linux-kernel@vger.kernel.org, hjk@hansjkoch.de, corbet@lwn.net, bruce.richardson@intel.com, avi@cloudius-systems.com, gleb@cloudius-systems.com, stephen@networkplumber.org, alexander.duyck@gmail.com From: Vlad Zolotarov Message-ID: <5613DE26.1090202@cloudius-systems.com> Date: Tue, 6 Oct 2015 17:43:50 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <20151006143821.GA11541@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/06/15 17:38, Michael S. Tsirkin wrote: > On Mon, Oct 05, 2015 at 01:20:11PM +0300, Avi Kivity wrote: >> On 10/05/2015 12:49 PM, Greg KH wrote: >>> On Mon, Oct 05, 2015 at 11:28:03AM +0300, Avi Kivity wrote: >>>> Of course it has to be documented, but this just follows vfio. >>>> >>>> Eventfd is a natural enough representation of an interrupt; both kvm and >>>> vfio use it, and are also able to share the eventfd, allowing a vfio >>>> interrupt to generate a kvm interrupt, without userspace intervention, and >>>> one day without even kernel intervention. >>> That's nice and wonderful, but it's not how UIO works today, so this is >>> now going to be a mix and match type interface, with no justification so >>> far as to why to create this new api and exactly how this is all going >>> to be used from userspace. >> The intended user is dpdk (http://dpdk.org), which is a family of userspace >> networking drivers for high performance networking applications. >> >> The natural device driver for dpdk is vfio, which both provides memory >> protection and exposes msi/msix interrupts. However, in many cases vfio >> cannot be used, either due to the lack of an iommu (for example, in >> virtualized environments) or out of a desire to avoid the iommus performance >> impact. >> >> The challenge in exposing msix interrupts to user space is that there are >> many of them, so you can't simply poll the device fd. If you do, how do you >> know which interrupt was triggered? The solution that vfio adopted was to >> associate each interrupt with an eventfd, allowing it to be individually >> polled. Since you can pass an eventfd with SCM_RIGHTS, and since kvm can >> trigger guest interrupts using an eventfd, the solution is very flexible. >> >>> Example code would be even better... >>> >>> >> >> This is the vfio dpdk interface code: >> >> http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c >> >> basically, the equivalent uio msix code would be very similar if uio adopts >> a similar interface: >> >> http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_pci_uio.c >> >> (current code lacks msi/msix support, of course). > So you really want a driver that behaves exactly like vfio. > Which immediately begs a question: why not extend vfio > to cover your usecase. The only "like VFIO" behavior we implement here is binding the MSI-X interrupt notification to eventfd descriptor. This doesn't justifies the hassle of implementing IOMMU-less VFIO mode. >