From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: guest assigned device MMIO maps with WC: does this work correctly? Date: Thu, 21 Nov 2013 10:18:34 +0200 Message-ID: <20131121081834.GA20041@redhat.com> References: <20131120195815.GA11239@redhat.com> <20131121071855.GD2108@redhat.com> <20131121080207.GA19812@redhat.com> <20131121080126.GG2108@redhat.com> <20131121081627.GA19951@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm@vger.kernel.org To: Gleb Natapov Return-path: Received: from mx1.redhat.com ([209.132.183.28]:28530 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750902Ab3KUIPU (ORCPT ); Thu, 21 Nov 2013 03:15:20 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id rAL8FK7Z024833 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 21 Nov 2013 03:15:20 -0500 Content-Disposition: inline In-Reply-To: <20131121081627.GA19951@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, Nov 21, 2013 at 10:16:27AM +0200, Michael S. Tsirkin wrote: > On Thu, Nov 21, 2013 at 10:01:27AM +0200, Gleb Natapov wrote: > > On Thu, Nov 21, 2013 at 10:02:07AM +0200, Michael S. Tsirkin wrote: > > > On Thu, Nov 21, 2013 at 09:18:55AM +0200, Gleb Natapov wrote: > > > > On Wed, Nov 20, 2013 at 09:58:15PM +0200, Michael S. Tsirkin wrote: > > > > > I see this in kvm: > > > > > > > > > > static u64 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool > > > > > is_mmio) > > > > > { > > > > > u64 ret; > > > > > > > > > > /* For VT-d and EPT combination > > > > > * 1. MMIO: always map as UC > > > > > * 2. EPT with VT-d: > > > > > * a. VT-d without snooping control feature: can't guarantee > > > > > * the > > > > > * result, try to trust guest. > > > > > * b. VT-d with snooping control feature: snooping control > > > > > * feature of > > > > > * VT-d engine can guarantee the cache correctness. Just > > > > > * set it > > > > > * to WB to keep consistent with host. So the same as item > > > > > * 3. > > > > > * 3. EPT without VT-d: always map as WB and set IPAT=1 to keep > > > > > * consistent with host MTRR > > > > > */ > > > > > if (is_mmio) > > > > > ret = MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT; > > > > > > > > > > ... > > > > > } > > > > > > > > > > > > > > > does this mean that even if guest maps BAR for an assigned device > > > > > as write combined (or configures such using an MTRR), > > > > > host will override this and use uncacheable in practice? > > > > > > > > > No, it does not mean that. I already answered this once (my previous > > > > answer included below): effective memory type is a combination of MTRR > > > > (EPT MT bits in case of a guest) and PAT bits. See section 11.5.2.2 > > > > in SDM > > > > > > > > > Can you quote chapter name please? > > > My SDM has > > > 11.5.2.2 Denormal-Operand Exception (#D) > > > > > Either your or mine is out of date: > > Selecting Memory Types for Pentium III and More Recent Processor Families > > OK this one I'm familiar with, it describes how PAT > interacts with MTRR. But how does this interact with EPT? > do you remember where's that described? Found it 28.2.4 EPT and Memory Typing sorry about the noise. > > > > on how effective memory type is calculated. > > > > > > > > Since MTRR UC + PAT WC = WC, if guest maps MMIO as WC in a page table > > > > (that what ioremap_wc does), everything works as it should. If guest maps > > > > MMIO as WB (ioremap_cache) and MTRR says MMIO is UC (like any MMIO will > > > > be by default) combined memory type will be UC, so also fine. If guest > > > > maps MMIO range as WB and fixes mtrr for this region to be WB then memory > > > > type will be incorrect in a guest, > > > > > > Meaning MTRR in guest is ignored in this case? > > Yes. > > > > > > > > > but I found only one place that does > > > > it in Linux: drivers/video/vesafb.c. All other uses of ioremap_cache > > > > either remap RAM or used to get whatever memory type configured in MTRR. > > > > > > > > -- > > > > Gleb. > > > > -- > > Gleb.