Re: guest assigned device MMIO maps with WC: does this work correctly?

["Michael S. Tsirkin" <mst@redhat.com>]

On Thu, Nov 21, 2013 at 10:16:27AM +0200, Michael S. Tsirkin wrote:
> On Thu, Nov 21, 2013 at 10:01:27AM +0200, Gleb Natapov wrote:
> > On Thu, Nov 21, 2013 at 10:02:07AM +0200, Michael S. Tsirkin wrote:
> > > On Thu, Nov 21, 2013 at 09:18:55AM +0200, Gleb Natapov wrote:
> > > > On Wed, Nov 20, 2013 at 09:58:15PM +0200, Michael S. Tsirkin wrote:
> > > > > I see this in kvm:
> > > > > 
> > > > > static u64 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool
> > > > > is_mmio)
> > > > > {        
> > > > >         u64 ret;
> > > > >          
> > > > >         /* For VT-d and EPT combination
> > > > >          * 1. MMIO: always map as UC
> > > > >          * 2. EPT with VT-d:
> > > > >          *   a. VT-d without snooping control feature: can't guarantee
> > > > >          *   the
> > > > >          *      result, try to trust guest.
> > > > >          *   b. VT-d with snooping control feature: snooping control
> > > > >          *   feature of
> > > > >          *      VT-d engine can guarantee the cache correctness. Just
> > > > >          *      set it
> > > > >          *      to WB to keep consistent with host. So the same as item
> > > > >          *      3.
> > > > >          * 3. EPT without VT-d: always map as WB and set IPAT=1 to keep
> > > > >          *    consistent with host MTRR 
> > > > >          */
> > > > >         if (is_mmio)
> > > > >                 ret = MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT;
> > > > > 
> > > > > ...
> > > > > }
> > > > > 
> > > > > 
> > > > > does this mean that even if guest maps BAR for an assigned device
> > > > > as write combined (or configures such using an MTRR),
> > > > > host will override this and use uncacheable in practice?
> > > > > 
> > > > No, it does not mean that. I already answered this once (my previous
> > > > answer included below): effective memory type is a combination of MTRR
> > > > (EPT MT bits in case of a guest) and PAT bits. See section 11.5.2.2
> > > > in SDM
> > > 
> > > 
> > > Can you quote chapter name please?
> > > My SDM has
> > > 11.5.2.2 Denormal-Operand Exception (#D)
> > > 
> > Either your or mine is out of date:
> > Selecting Memory Types for Pentium III and More Recent Processor Families
> 
> OK this one I'm familiar with, it describes how PAT
> interacts with MTRR. But how does this interact with EPT?
> do you remember where's that described?

Found it 28.2.4 EPT and Memory Typing
sorry about the noise.


> > > > on how effective memory type is calculated.
> > > > 
> > > >  Since MTRR UC + PAT WC = WC, if guest maps MMIO as WC in a page table
> > > >  (that what ioremap_wc does), everything works as it should. If guest maps
> > > >  MMIO as WB (ioremap_cache) and MTRR says MMIO is UC (like any MMIO will
> > > >  be by default) combined memory type will be UC, so also fine. If guest
> > > >  maps MMIO range as WB and fixes mtrr for this region to be WB then memory
> > > >  type will be incorrect in a guest,
> > > 
> > > Meaning  MTRR in guest is ignored in this case?
> > Yes.
> > 
> > > 
> > > > but I found only one place that does
> > > >  it in Linux: drivers/video/vesafb.c. All other uses of ioremap_cache
> > > >  either remap RAM or used to get whatever memory type configured in MTRR.
> > > > 
> > > > --
> > > > 			Gleb.
> > 
> > --
> > 			Gleb.