From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753562AbdFVQ7k (ORCPT ); Thu, 22 Jun 2017 12:59:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56228 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751283AbdFVQ7j (ORCPT ); Thu, 22 Jun 2017 12:59:39 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D13BA796EE Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=alex.williamson@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D13BA796EE Date: Thu, 22 Jun 2017 10:59:35 -0600 From: Alex Williamson To: Nitin Saxena Cc: linux-kernel@vger.kernel.org, qemu-devel , Peter Xu Subject: Re: Query on VFIO in Virtual machine Message-ID: <20170622105935.2dbd574d@w520.home> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 22 Jun 2017 16:59:39 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [cc +qemu-devel, +peterx] On Thu, 22 Jun 2017 22:18:06 +0530 Nitin Saxena wrote: > Hi, > > I have a PCI device connected as an endpoint to Intel host machine. > The requirement is to run dpdk like user space data path application > in VM using PCI PF passthrough (SRIOV disabled). This application > works fine on host kernel and uses VFIO to get MSIX interrupts from > PCI device. We are trying to run this existing application in VM using > PCI passthrough. This application has capability to use > VFIO_IOMMU_TYPE1 as wells as VFIO_NOIOMMU. > > On Intel host machine VT-d has been enabled and using virt-manager PCI > device PF is assigned to the VM. This makes virt-manager to implicitly > binds PCI device PF to vfio with vfio_iommu_type1. The VM LINUX kernel > was booted with intel_iommu=on as boot parameter. > > My question: Is it possible that vfio can coexist in host (by > virt-manager) as well as VM (by application)? If yes, does application > running inside VM needs to configure VFIO with iommu_type=IOMMU or > iommu_type=no-iommu. > > In VM I tried inserting vfio_iommu_type1.ko kernel module which failed > with "No such device error". Thats why I am confused whether my > requirement is legitimate or not. What could be the best solution? This is really more of a QEMU question. In order to use vfio_iommu_type1 in the guest, you need an iommu in the guest. The most recent release of QEMU supports this with an emulated VT-d device. Therefore if you create a VM with emulated VT-d and a device assigned through vfio-pci, you can expose it to userspace in the VM with physical iommu protection. Without an iommu in the VM, you'd be limited to no-iommu support for VM userspace, the physical iommu would only protect the device to the extent of VM memory, no to specific userspace mappings within the VM. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56367) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dO5SM-000416-GT for qemu-devel@nongnu.org; Thu, 22 Jun 2017 12:59:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dO5SH-0007ws-HE for qemu-devel@nongnu.org; Thu, 22 Jun 2017 12:59:46 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34000) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dO5SH-0007va-7g for qemu-devel@nongnu.org; Thu, 22 Jun 2017 12:59:41 -0400 Date: Thu, 22 Jun 2017 10:59:35 -0600 From: Alex Williamson Message-ID: <20170622105935.2dbd574d@w520.home> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Query on VFIO in Virtual machine List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Nitin Saxena Cc: linux-kernel@vger.kernel.org, qemu-devel , Peter Xu [cc +qemu-devel, +peterx] On Thu, 22 Jun 2017 22:18:06 +0530 Nitin Saxena wrote: > Hi, > > I have a PCI device connected as an endpoint to Intel host machine. > The requirement is to run dpdk like user space data path application > in VM using PCI PF passthrough (SRIOV disabled). This application > works fine on host kernel and uses VFIO to get MSIX interrupts from > PCI device. We are trying to run this existing application in VM using > PCI passthrough. This application has capability to use > VFIO_IOMMU_TYPE1 as wells as VFIO_NOIOMMU. > > On Intel host machine VT-d has been enabled and using virt-manager PCI > device PF is assigned to the VM. This makes virt-manager to implicitly > binds PCI device PF to vfio with vfio_iommu_type1. The VM LINUX kernel > was booted with intel_iommu=on as boot parameter. > > My question: Is it possible that vfio can coexist in host (by > virt-manager) as well as VM (by application)? If yes, does application > running inside VM needs to configure VFIO with iommu_type=IOMMU or > iommu_type=no-iommu. > > In VM I tried inserting vfio_iommu_type1.ko kernel module which failed > with "No such device error". Thats why I am confused whether my > requirement is legitimate or not. What could be the best solution? This is really more of a QEMU question. In order to use vfio_iommu_type1 in the guest, you need an iommu in the guest. The most recent release of QEMU supports this with an emulated VT-d device. Therefore if you create a VM with emulated VT-d and a device assigned through vfio-pci, you can expose it to userspace in the VM with physical iommu protection. Without an iommu in the VM, you'd be limited to no-iommu support for VM userspace, the physical iommu would only protect the device to the extent of VM memory, no to specific userspace mappings within the VM. Thanks, Alex