Re: cr3 OOS optimisation breaks 32-bit GNU/kFreeBSD guest

[Avi Kivity <avi@redhat.com>]

Marcelo Tosatti wrote:
> On Mon, Feb 23, 2009 at 04:59:37PM +0200, Avi Kivity wrote:
>   
>> Marcelo Tosatti wrote:
>>     
>>>> Thanks for your fast answer and for your help for debugging.
>>>>     
>>>>         
>>> If you confirm that FreeBSD is indeed relying on cr3 to sync global
>>> pages, it might be better to disable the optimization. Lets hope that is
>>> not the case.
>>>   
>>>       
>> cr3 writes explicitly do not flush global pages; otherwise what would be  
>> the point of global pages at all?
>>     
>
> From the Intel TLB doc:
>
> The processor is always free to invalidate additional entries in the TLBs
> and paging-structure caches. The following are some examples:
>
> • MOV to CR3 may invalidate TLB entries for global pages.
>
> The reasoning was if an optimization breaks an important guest which
> contains a bug that happens to not trigger on real HW due to positioning
> of the stars, it is reasonable to disable that optimization.
>   

This means the OS may not rely on the TLB retaining its contents.  For 
example, you can't do

  1. set pte to global+present
  2. access through pte to load tlb entry
  3. clear pte
  4. switch cr3
  5. access through same pte again, relying on tlb entry to service the 
access

So the processor may choose to ignore the global bit on some or all tlb 
entries, but software cannot assume that it does.  Typically it will 
honor the global bit since otherwise it's useless.

I don't think this is what is happening with FreeBSD.  It may be that 
spte population on invlpg is confusing the guest (though that is allowed 
as a speculative read?).  For example, the sequence:

  1. invlpg
  2. set pte to A (present+accessed)
  3. set pte to B (present+accessed)

kvm behaves as if a speculative read always happens between 2 and 3, 
which would be very rare on real hardware.

-- 
error compiling committee.c: too many arguments to function