From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stephen Hemminger Subject: Re: VFIO no-iommu Date: Thu, 17 Dec 2015 15:29:09 -0800 Message-ID: <20151217152909.4c7cbaff@xeon-e3> References: <60420822.AbcfvjLZCk@xps13> <17700135.Qc9aIsHGGP@xps13> <2580440.9rYeIQLTBH@xps13> <20151217203816.1bc6bd2c@jvn> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org To: Jan Viktorin Return-path: Received: from mail-pf0-f180.google.com (mail-pf0-f180.google.com [209.85.192.180]) by dpdk.org (Postfix) with ESMTP id 787F48D3A for ; Fri, 18 Dec 2015 00:29:02 +0100 (CET) Received: by mail-pf0-f180.google.com with SMTP id v86so38376176pfa.2 for ; Thu, 17 Dec 2015 15:29:02 -0800 (PST) In-Reply-To: <20151217203816.1bc6bd2c@jvn> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, 17 Dec 2015 20:38:16 +0100 Jan Viktorin wrote: > On Thu, 17 Dec 2015 11:09:23 +0100 > Thomas Monjalon wrote: > > > Hi, > > > > 2015-12-17 09:52, Burakov, Anatoly: > > > > > > > On Tue, Dec 15, 2015 at 09:53:18AM -0700, Alex Williamson wrote: > > > > > > > So it works. Is it acceptable? Useful? Sufficiently complete? > > > > > > > Does it imply deprecating the uio interface? I believe the > > > > > > > feature that started this discussion was support for MSI/X > > > > > > > interrupts so that VFs can support some kind of interrupt (uio > > > > > > > only supports INTx since it doesn't allow DMA). Implementing that > > > > > > > would be the ultimate test of whether this provides dpdk with not > > > > > > > only a more consistent interface, but the feature dpdk wants > > > > > > > that's missing in uio. Thanks, > > > > > > > > > > Ferruh has done a great job so far testing Alex's patch, very few changes > > > > from DPDK side seem to be required as far as existing functionality goes (not > > > > sure about VF interrupts mentioned by Alex). However, one thing that > > > > concerns me is usability. While it is true that no-IOMMU mode in VFIO would > > > > mean uio interfaces could be deprecated in time, the no-iommu mode is way > > > > more hassle than using igb_uio/uio_pci_generic because it will require a > > > > kernel recompile as opposed to simply compiling and insmod'ding an out-of- > > > > tree driver. So, in essence, if you don't want an IOMMU, it's becoming that > > > > much harder to use DPDK. Would that be something DPDK is willing to live > > > > with in the absence of uio interfaces? > > > > > > > > Excuse me if I missed something obvious. > > > > Why a kernel compilation is needed? > > > > > > Well, not really full kernel compilation, but in the default configuration, VFIO driver would not support NOIOMMU mode. I.e. it's not compiled by default. Support for no-iommu should be enabled in kernel config and compiled in. So, whoever is going to use DPDK with VFIO-no-iommu will have to download kernel tree and recompile the VFIO module and install it. That's obviously way more hassle than simply compiling an out-of-tree driver that's already included and works with an out-of-the-box kernel. > > > > The "out-of-the-box kernel" is configured by your distribution. > > So we don't know yet what will be their choice. > > If the distribution supports DPDK, it should be enabled. > > I have a question as I am not involved in all possible DPDK > configurations, platforms, etc. and not yet very involved in vfio. What > are the devices which do not have IOMMU? If I have, say, DPDK 2.3 with > vfio-noiommu, which platforms (or computer systems) I am targeting? > > Would it be an Intel-based system? Would it be PPC8, ARM? > > If it is ARMv7... I would say that the fact I have to explicitly enable > the no-IOMMU feature and rebuild the kernel (or whatever) is just OK. As > for such systems, it is common to have a quite customized OS. Well, > the big distributions are able to run on those devices, that's true... > However, in such case, the users are usually skilled enough to take > care of having their own special Linux kernel. > > So, is the fact the distributions would not support the no-IOMMU setup > in their default configuration really an issue? Will some very common > Intel/DPDK-based box need this? > > Regards > Jan So far: * broken hardware (many systems including those from Dell) do not provide working IOMMU because of Bios bugs etc. * Linux guest in VMware/KVM/Hyper-V. There is no IOMMU emulation in most of these systems. * Older smaller systems (ie Atom) may not have IOMMU