Re: GPU passthrough performance regression in >4GB vms due to XSA-60 changes

[Tomasz Wroblewski <tomasz.wroblewski@gmail.com>]

On 05/19/2014 01:07 PM, Jan Beulich wrote:
>>>> On 19.05.14 at 12:47, <tomasz.wroblewski@gmail.com> wrote:
>> On 05/19/2014 12:38 PM, Jan Beulich wrote:
>>> So perhaps time for sending complete logs, plus suitable information
>>> from inside the guest of how things (RAM, MMIO, MTRRs) end up being
>>> set up?
>> Could be, though please read the explanation I came up in the other post
>> whether its enough, I think it makes sense... 64bit guest BARs are
>> indeed not in use (confirmed from guest). MTRR is setup such that only
>> the low region is UC, which is correct.
> Yes, that's a very sensible theory, which - as just said in the other
> reply - can be easily verified.
>
>> But the RAM relocation code causes the caching on relocated region to be
>> UC instead of WB due to the timing (very early, MTRR disabled) at which
>> it runs, which is incorrect. I am thinking enabling MTRR during that
>> relocation would probably fix it on 4.3
> Except that this is a chicken and egg problem then: In order to
> populate the variable range MTRRs, the BAR assignment (and hence
> the prerequisite RAM relocation) need to be done already.
I am not sure; looking at hvmloader code, wouldn't it be possible to 
calculate the BAR locations first, then update the MTRR var ranges and 
enable it, and only then actually write the BAR registers (from 
precalculated info)? Presumably it's only the write part which needs to 
be done after relocation as it causes qemu to setup mmio etc.

> What we
> might be able to do (after careful evaluation whether backporting
> what we have on -unstable wouldn't be the better option, which
> first of all implies you'd need to test things there) is enable MTRRs
> with WB default, but without any variable ranges set up _before_
> doing the RAM relocation/BAR assignment. The main problem to solve
> there, however, would still be to make sure the EPT entries get
> re-created for the regions that we don't want to be WB (after the
> variable ranges got set up). That, I'm afraid, would still lead us to
> considerations on backporting at least some of the changes from
> -unstable.
Yeah I gave about a day of effort to port us onto unstable and test 
there but it sadly looks to be a bigger job, so leaving that as a last 
resort (though planning to spend couple more days on it soon).


> Jan
>