Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes

[Tomasz Wroblewski <tomasz.wroblewski@gmail.com>]

On 05/15/2014 04:34 PM, Tomasz Wroblewski wrote:
>
> On 05/15/2014 03:39 PM, Tomasz Wroblewski wrote:
>>
>> On 05/15/2014 03:23 PM, Jan Beulich wrote:
>>>>>> On 15.05.14 at 14:10, <tomasz.wroblewski@gmail.com> wrote:
>>>> Not really sure why it only affects 64bit vms but I've just noticed 
>>>> the
>>>> pci BARs for the card are being relocated by hvmloader as per some 
>>>> logs:
>>>>
>>>> (XEN) HVM3: Relocating guest memory for lowmem MMIO space enabled
>>>> (XEN) HVM3: Relocating 0xffff pages from 0e0001000 to 14dc00000 for
>>>> lowmem MMIO hole
>>>> (XEN) HVM3: Relocating 0x1 pages from 0e0000000 to 15dbff000 for 
>>>> lowmem
>>>> MMIO hole
>>>>
>>>> So it might be also related to that.
>>> Indeed it might - what are the (guest) MTRR types for those regions?
>> It's writeback for both the 32bit and 64bit above ranges.
> ... however, after a bit more debugging its uncached at the time 
> hvmloader does the relocation so that's why it ends up like that in 
> EPT tables. It does go to writeback only soon after. Haven't 
> pinpointed the exact time point for that yet nor why it's being 
> updated to writeback, but it seems to be before the guest starts 
> booting (i.e. still on bios screens).
... and after even more I see that the type is uncached at the time the 
relocation is happening because mtrr is disabled at that time and 
get_mtrr_type() function exits with uncached value in the first few 
lines of it. Later when guests enabled MTRR, ept is not updated. So 
maybe the EPTs should be updated in some way at that time, but I guess 
the obvious way of doing it would just reintroduce the DoS covered by 
XSA-60 (just triggerable by fiddling with mtrr state instead of CR0 cd 
bit), so I don't really know..