Re: [PATCH] Don't track all memory when enabling log dirty to track vram

[George Dunlap <george.dunlap@eu.citrix.com>]

On 02/17/2014 10:18 AM, Jan Beulich wrote:
>>>> On 13.02.14 at 17:20, Tim Deegan <tim@xen.org> wrote:
>> At 15:55 +0000 on 13 Feb (1392303343), Jan Beulich wrote:
>>>>>> On 13.02.14 at 16:46, George Dunlap <george.dunlap@eu.citrix.com> wrote:
>>>> On 02/12/2014 12:53 AM, Zhang, Yang Z wrote:
>>>>> George Dunlap wrote on 2014-02-11:
>>>>>> I think I got a bit distracted with the "A isn't really so bad" thing.
>>>>>> Actually, if the overhead of not sharing tables isn't very high, then
>>>>>> B isn't such a bad option.  In fact, B is what I expected Yang to
>>>>>> submit when he originally described the problem.
>>>>> Actually, the first solution came to my mind is B. Then I realized that
>> even
>>>> chose B, we still cannot track the memory updating from DMA(even with A/D
>>>> bit, it still a problem). Also, considering the current usage case of log
>>>> dirty in Xen(only vram tracking has problem), I though A is better.:
>>>> Hypervisor only need to track the vram change. If a malicious guest try to
>>>> DMA to vram range, it only crashed himself (This should be reasonable).
>>>>>> I was going to say, from a release perspective, B is probably the
>>>>>> safest option for now.  But on the other hand, if we've been testing
>>>>>> sharing all this time, maybe switching back over to non-sharing whole-hog has
>>>> the higher risk?
>>>>> Another problem with B is that current VT-d large paging supporting relies
>> on
>>>> the sharing EPT and VT-d page table. This means if we choose B, then we need
>>>> to re-enable VT-d large page. This would be a huge performance impaction for
>>>> Xen 4.4 on using VT-d solution.
>>>>
>>>> OK -- if that's the case, then it definitely tips the balance back to
>>>> A.  Unless Tim or Jan disagrees, can one of you two check it in?
>>>>
>>>> Don't rush your judgement; but it would be nice to have this in before
>>>> RC4, which would mean checking it in today preferrably, or early
>>>> tomorrow at the latest.
>>> That would be Tim then, as he would have to approve of it anyway.
>> Done.
> Actually I'm afraid there are two problems with this patch:
>
> For one, is enabling "global" log dirty mode still going to work
> after VRAM-only mode already got enabled? I ask because the
> paging_mode_log_dirty() check which paging_log_dirty_enable()
> does first thing suggests otherwise to me (i.e. the now
> conditional setting of all p2m entries to p2m_ram_logdirty would
> seem to never get executed). IOW I would think that we're now
> lacking a control operation allowing the transition from dirty VRAM
> tracking mode to full log dirty mode.

Hmm, yes, doing a code inspection, that would appear to be the case.  
This probably wouldn't be caught by osstest, because (as I understand 
it) we never attach to the display, so dirty vram tracking is probably 
never enabled.

> And second, I have been fighting with finding both conditions
> and (eventually) the root cause of a severe performance
> regression (compared to 4.3.x) I'm observing on an EPT+IOMMU
> system. This became _much_ worse after adding in the patch here
> (while in fact I had hoped it might help with the originally observed
> degradation): X startup fails due to timing out, and booting the
> guest now takes about 20 minutes). I didn't find the root cause of
> this yet, but meanwhile I know that
> - the same isn't observable on SVM
> - there's no problem when forcing the domain to use shadow
>    mode
> - there's no need for any device to actually be assigned to the
>    guest
> - the regression is very likely purely graphics related (based on
>    the observation that when running something that regularly but
>    not heavily updates the screen with X up, the guest consumes a
>    full CPU's worth of processing power, yet when that updating
>    doesn't happen, CPU consumption goes down, and it goes further
>    down when shutting down X altogether - at least as log as the
>    patch here doesn't get involved).
> This I'm observing on a Westmere box (and I didn't notice it earlier
> because that's one of those where due to a chipset erratum the
> IOMMU gets turned off by default), so it's possible that this can't
> be seen on more modern hardware. I'll hopefully find time today to
> check this on the one newer (Sandy Bridge) box I have.

So you're saying that the slowdown happens if you have EPT+IOMMU, but 
*not* if you have EPT alone (IOMMU disabled), or shadow + IOMMU?

I have an issue I haven't had time to look into where windows installs 
are sometimes terribly slow on my Nehalem box; but it seems to be only 
with qemu-xen, not qemu-traditional.  I haven't tried with shadow.

  -George