From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yang Zhang Subject: Re: Enable more than 255 VCPU support without irq remapping function in the guest Date: Wed, 4 May 2016 09:46:21 +0800 Message-ID: <5729546D.8010006@gmail.com> References: <571F93CA.40200@intel.com> <571F9487.5090009@siemens.com> <20160426164939.GA18900@potion> <57203B9D.6020402@gmail.com> <57204D28.4070706@siemens.com> <572088D0.7040805@gmail.com> <57208A54.40502@siemens.com> <57216341.80006@gmail.com> <5721B394.9050008@siemens.com> <20160428153251.GA17368@potion> <5722C247.6040004@gmail.com> <5722EA47.80205@siemens.com> <57280475.2080506@gmail.com> <6449BFFE-245B-4641-AEB4-9F666A7CDBE8@gmail.com> <57282F41.8030802@siemens.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , "Lan, Tianyu" , Paolo Bonzini , kvm@vger.kernel.org, Thomas Gleixner , mst@redhat.com, x86@kernel.org, Peter Xu , Igor Mammedov To: Jan Kiszka , Nadav Amit Return-path: Received: from mail-oi0-f47.google.com ([209.85.218.47]:36242 "EHLO mail-oi0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756480AbcEDBqo (ORCPT ); Tue, 3 May 2016 21:46:44 -0400 Received: by mail-oi0-f47.google.com with SMTP id x201so48861354oif.3 for ; Tue, 03 May 2016 18:46:43 -0700 (PDT) In-Reply-To: <57282F41.8030802@siemens.com> Sender: kvm-owner@vger.kernel.org List-ID: On 2016/5/3 12:55, Jan Kiszka wrote: > On 2016-05-03 04:03, Nadav Amit wrote: >> Yang Zhang wrote: >> >>> I think it is not only interrupt. There must have the DMAR emulation and >>> the cost for DMA is heavy in VM(DMA operations are very frequently). I >>> cannot remember whether there are strong dependency in hardware between >>> DMAR and IR(I know IR is relying on QI). Even hardware dependency is ok, >>> is it ok for OS running in hardware with IR but without DMAR? >> >> Do you know a way for the IOMMU to report that DMAR is disabled, while IR >> is enabled? > > The hardware cannot decide about disabling this, but the guest can, of > course. In fact, you can even configure Linux to have DMAR off by > default until you pass "intel_iommu=on" on the command line (I think > distros still do this - at least they used to). No idea about other > OSes, though. If we can disable DMAR in guest, it should be enough. > >> >> Anyhow, the VM can use IOMMU passthrough mode to avoid most IOMMU overhead. >> Regardless, a recent patch-set should improve DMAR performance >> considerably [1]. > > The bottleneck with emulated DMAR is rather in QEMU. But DMAR can be > almost as cheap as IR once we get it running for VFIO and vhost: both > need proper caching because they do not work with QEMU in the loop for > each and every DMA transfer. Still no need to deviate from physical > hardware. Sorry, i don't know detail about how VFIO and vhost work with IR. But it seems hard to do proper caching since DMA allocations are very frequently in Linux unless we move the whole iommu emulation to kernel. Another idea is using two iommus: one for Qemu and one for device in kernel like vfio and vhost. I did the similar thing in Xen in several years ago which uses two iommus solution and it works well in my experiment environment. Besides, this solution is easy for nested device pass-through. The Page 32 of [1] has more detail. [1] http://docplayer.net/10559370-Nested-virtualization.html -- best regards yang