Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes

[Tomasz Wroblewski <tomasz.wroblewski@gmail.com>]

On 05/15/2014 06:07 PM, Jan Beulich wrote:
>>>> On 15.05.14 at 16:56, <tomasz.wroblewski@gmail.com> wrote:
>> On 05/15/2014 04:34 PM, Tomasz Wroblewski wrote:
>>> On 05/15/2014 03:39 PM, Tomasz Wroblewski wrote:
>>>> On 05/15/2014 03:23 PM, Jan Beulich wrote:
>>>>>>>> On 15.05.14 at 14:10, <tomasz.wroblewski@gmail.com> wrote:
>>>>>> Not really sure why it only affects 64bit vms but I've just noticed
>>>>>> the
>>>>>> pci BARs for the card are being relocated by hvmloader as per some
>>>>>> logs:
>>>>>>
>>>>>> (XEN) HVM3: Relocating guest memory for lowmem MMIO space enabled
>>>>>> (XEN) HVM3: Relocating 0xffff pages from 0e0001000 to 14dc00000 for
>>>>>> lowmem MMIO hole
>>>>>> (XEN) HVM3: Relocating 0x1 pages from 0e0000000 to 15dbff000 for
>>>>>> lowmem
>>>>>> MMIO hole
>>>>>>
>>>>>> So it might be also related to that.
>>>>> Indeed it might - what are the (guest) MTRR types for those regions?
>>>> It's writeback for both the 32bit and 64bit above ranges.
>>> ... however, after a bit more debugging its uncached at the time
>>> hvmloader does the relocation so that's why it ends up like that in
>>> EPT tables. It does go to writeback only soon after. Haven't
>>> pinpointed the exact time point for that yet nor why it's being
>>> updated to writeback, but it seems to be before the guest starts
>>> booting (i.e. still on bios screens).
>> ... and after even more I see that the type is uncached at the time the
>> relocation is happening because mtrr is disabled at that time and
>> get_mtrr_type() function exits with uncached value in the first few
>> lines of it. Later when guests enabled MTRR, ept is not updated. So
>> maybe the EPTs should be updated in some way at that time,
> Which is what -unstable is now doing.
Right.
> But the question remains why this region doesn't get marked UC or
> WC, but WB.
The region doesn't seem to be marked in any way in mtrr so it just goes 
off the default type for that mtrr ((struct mtrr_state*)->def_type) 
which seems to be WB.

> Jan
>