Re: guest assigned device MMIO maps with WC: does this work correctly?

["Michael S. Tsirkin" <mst@redhat.com>]

On Thu, Nov 21, 2013 at 10:01:27AM +0200, Gleb Natapov wrote:
> On Thu, Nov 21, 2013 at 10:02:07AM +0200, Michael S. Tsirkin wrote:
> > On Thu, Nov 21, 2013 at 09:18:55AM +0200, Gleb Natapov wrote:
> > > On Wed, Nov 20, 2013 at 09:58:15PM +0200, Michael S. Tsirkin wrote:
> > > > I see this in kvm:
> > > > 
> > > > static u64 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool
> > > > is_mmio)
> > > > {        
> > > >         u64 ret;
> > > >          
> > > >         /* For VT-d and EPT combination
> > > >          * 1. MMIO: always map as UC
> > > >          * 2. EPT with VT-d:
> > > >          *   a. VT-d without snooping control feature: can't guarantee
> > > >          *   the
> > > >          *      result, try to trust guest.
> > > >          *   b. VT-d with snooping control feature: snooping control
> > > >          *   feature of
> > > >          *      VT-d engine can guarantee the cache correctness. Just
> > > >          *      set it
> > > >          *      to WB to keep consistent with host. So the same as item
> > > >          *      3.
> > > >          * 3. EPT without VT-d: always map as WB and set IPAT=1 to keep
> > > >          *    consistent with host MTRR 
> > > >          */
> > > >         if (is_mmio)
> > > >                 ret = MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT;
> > > > 
> > > > ...
> > > > }
> > > > 
> > > > 
> > > > does this mean that even if guest maps BAR for an assigned device
> > > > as write combined (or configures such using an MTRR),
> > > > host will override this and use uncacheable in practice?
> > > > 
> > > No, it does not mean that. I already answered this once (my previous
> > > answer included below): effective memory type is a combination of MTRR
> > > (EPT MT bits in case of a guest) and PAT bits. See section 11.5.2.2
> > > in SDM
> > 
> > 
> > Can you quote chapter name please?
> > My SDM has
> > 11.5.2.2 Denormal-Operand Exception (#D)
> > 
> Either your or mine is out of date:
> Selecting Memory Types for Pentium III and More Recent Processor Families

OK this one I'm familiar with, it describes how PAT
interacts with MTRR. But how does this interact with EPT?
do you remember where's that described?

> > > on how effective memory type is calculated.
> > > 
> > >  Since MTRR UC + PAT WC = WC, if guest maps MMIO as WC in a page table
> > >  (that what ioremap_wc does), everything works as it should. If guest maps
> > >  MMIO as WB (ioremap_cache) and MTRR says MMIO is UC (like any MMIO will
> > >  be by default) combined memory type will be UC, so also fine. If guest
> > >  maps MMIO range as WB and fixes mtrr for this region to be WB then memory
> > >  type will be incorrect in a guest,
> > 
> > Meaning  MTRR in guest is ignored in this case?
> Yes.
> 
> > 
> > > but I found only one place that does
> > >  it in Linux: drivers/video/vesafb.c. All other uses of ioremap_cache
> > >  either remap RAM or used to get whatever memory type configured in MTRR.
> > > 
> > > --
> > > 			Gleb.
> 
> --
> 			Gleb.