Re: [Xen-devel] Re: [PATCH 00 of 36] x86/paravirt: groundwork for 64-bit Xen support

[Ingo Molnar <mingo@elte.hu>]

* Jeremy Fitzhardinge <jeremy@goop.org> wrote:

> Ingo Molnar wrote:
>> -tip auto-testing found pagetable corruption (CPA self-test failure):
>>
>> [   32.956015] CPA self-test:
>> [   32.958822]  4k 2048 large 508 gb 0 x 2556[ffff880000000000-ffff88003fe00000] miss 0
>> [   32.964000] CPA ffff88001d54e000: bad pte 1d4000e3
>> [   32.968000] CPA ffff88001d54e000: unexpected level 2
>> [   32.972000] CPA ffff880022c5d000: bad pte 22c000e3
>> [   32.976000] CPA ffff880022c5d000: unexpected level 2
>> [   32.980000] CPA ffff8800200ce000: bad pte 200000e3
>> [   32.984000] CPA ffff8800200ce000: unexpected level 2
>> [   32.988000] CPA ffff8800210f0000: bad pte 210000e3
>>
>> config and full log can be found at:
>>
>>  http://redhat.com/~mingo/misc/config-Mon_Jun_30_11_11_51_CEST_2008.bad
>>  http://redhat.com/~mingo/misc/log-Mon_Jun_30_11_11_51_CEST_2008.bad
>>
>> i've pushed that tree out into tip/tmp.xen-64bit.Mon_Jun_30_11_11. The  
>> only new item in that tree over a well-tested base is x86/xen-64bit, so 
>> i've taken it out again.
>>   
>
> Phew.  OK, I've worked this out.  Short version is that's it's a false  
> alarm, and there was no real failure here.  Long version:
>
>    * I changed the code to create the physical mapping pagetables to
>      reuse any existing mapping rather than replace it.   Specifically,
>      reusing an pud pointed to by the pgd caused this symptom to appear.
>    * The specific PUD being reused is the one created statically in
>      head_64.S, which creates an initial 1GB mapping.
>    * That mapping doesn't have _PAGE_GLOBAL set on it, due to the
>      inconsistency between __PAGE_* and PAGE_*.
>    * The CPA test attempts to clear _PAGE_GLOBAL, and then checks to
>      see that the resulting range is 1) shattered into 4k pages, and 2)
>      has no _PAGE_GLOBAL.
>    * However, since it didn't have _PAGE_GLOBAL on that range to start
>      with, change_page_attr_clear() had nothing to do, and didn't
>      bother shattering the range,
>    * resulting in the reported messages
>
> The simple fix is to set _PAGE_GLOBAL in level2_ident_pgt.
>
> An additional fix to make CPA testing more robust by using some other  
> pagetable bit (one of the unused available-to-software ones).  This  
> would solve spurious CPA test warnings under Xen which uses _PAGE_GLOBAL  
> for its own purposes (ie, not under guest control).
>
> Also, we should revisit the use of _PAGE_GLOBAL in asm-x86/pgtable.h, 
> and use it consistently, and drop MAKE_GLOBAL.  The first time I 
> proposed it it caused breakages in the very early CPA code; with luck 
> that's all fixed now.
>
> Anyway, the simple fix below.  [...]

great - i've applied your fix and re-integrated x86/xen-64bit, it's 
under testing now. (no problems so far)

> [...] I'll put together RFC patches for the other suggestions.  I also 
> split the originating patch into tiny, tiny bisectable pieces.

cool! :)

	Ingo