From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39842) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bA985-0004iP-Av for qemu-devel@nongnu.org; Tue, 07 Jun 2016 01:00:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bA97z-0007CY-CB for qemu-devel@nongnu.org; Tue, 07 Jun 2016 01:00:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:2389) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bA97z-0007CT-3u for qemu-devel@nongnu.org; Tue, 07 Jun 2016 01:00:35 -0400 Date: Tue, 7 Jun 2016 13:00:28 +0800 From: Peter Xu Message-ID: <20160607050028.GA1039@pxdev.xzpeter.org> References: <57408FDB.1010000@web.de> <20160602084439.GB3477@pxdev.xzpeter.org> <20160602070046.761be49c@ul30vt.home> <5750313C.4000709@web.de> <20160606050407.GB21254@pxdev.xzpeter.org> <20160606071141.31d2008e@ul30vt.home> <20160606134317.GJ21254@pxdev.xzpeter.org> <20160606110211.2c9bc8ef@ul30vt.home> <20160607032032.GC3800@pxdev.xzpeter.org> <20160606215809.6ba9e66e@ul30vt.home> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20160606215809.6ba9e66e@ul30vt.home> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH v3 1/3] IOMMU: add VTD_CAP_CM to vIOMMU capability exposed to guest List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Williamson Cc: Jan Kiszka , "Aviv B.D" , qemu-devel@nongnu.org, "Michael S. Tsirkin" On Mon, Jun 06, 2016 at 09:58:09PM -0600, Alex Williamson wrote: > On Tue, 7 Jun 2016 11:20:32 +0800 > Peter Xu wrote: [...] > > Only if cap_caching_mode() is set (which is bit 7, the CM bit), we > > will send these invalidations. What I meant is that, we should allow > > user to specify the CM bit, so that when we are not using VFIO > > devices, we can skip the above flush_content() and flush_iotlb() > > etc... So, besides the truth that we have some guests do not support > > CM bit (like Jailhouse), performance might be another consideration > > point that we should allow user to specify the CM bit themselfs. >=20 > I'm dubious of this, IOMMU drivers are already aware that hardware > flushes are expensive and do batching to optimize it. The queued > invalidation mechanism itself is meant to allow asynchronous > invalidations. QEMU invalidating a virtual IOMMU might very well be > faster than hardware. Agree. However it seems that current Linux is still not taking this advantage... check qi_flush_context() and qi_flush_iotlb(). qi_submit_sync() is used for both, which sends one invalidation with a explicit wait to make sure it's sync. >=20 > > >=20 > > > C: Page-walk Coherency > > > This field indicates if hardware access to the root, context, > > > extended-context and interrupt-remap tables, and second-level pag= ing > > > structures for requests-without PASID, are coherent (snooped) or = not. > > > =E2=80=A2 0: Indicates hardware accesses to remapping structure= s are non-coherent. > > > =E2=80=A2 1: Indicates hardware accesses to remapping structure= s are coherent. > > >=20 > > > Without both CM=3D0 and C=3D0, our only virtualization mechanism fo= r > > > maintaining a hardware cache coherent with the guest view of the io= mmu > > > would be to shadow all of the VT-d structures. For purely emulated > > > devices, maybe we can get away with that, but I doubt the current > > > ghashes used for the iotlb are prepared for it. =20 > >=20 > > Actually I haven't noticed this bit yet. I see that this will decide > > whether guest kernel need to send specific clflush() when modifying > > IOMMU PTEs, but shouldn't we flush the memory cache always so that we > > can sure IOMMU can see the same memory data as CPU does? >=20 > I think it would be a question of how much the g_hash code really buys > us in the VT-d code, it might be faster to do a lookup each time if it > means fewer flushes. Those hashes are useless overhead for assigned > devices, so maybe we can avoid them when we only have assigned > devices ;) Thanks, Errr, I just noticed that VFIO devices do not need emulated cache. There are indeed lots of pending works TBD on vIOMMU side... Thanks! -- peterx