From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kai Huang Subject: Re: PML (Page Modification Logging) design for Xen Date: Thu, 12 Feb 2015 15:34:30 +0800 Message-ID: <54DC5786.6070909@linux.intel.com> References: <54DB129D.3060102@linux.intel.com> <54DB4294.1080406@citrix.com> <54DB6392020000780005F08B@mail.emea.novell.com> <54DC1171.1030000@linux.intel.com> <54DC4C1A.6010404@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Tian, Kevin" , Jan Beulich , Andrew Cooper Cc: "tim@xen.org" , "keir@xen.org" , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On 02/12/2015 03:08 PM, Tian, Kevin wrote: >> From: Kai Huang [mailto:kai.huang@linux.intel.com] >> Sent: Thursday, February 12, 2015 2:46 PM >> >> On 02/12/2015 02:25 PM, Tian, Kevin wrote: >>>> From: Kai Huang [mailto:kai.huang@linux.intel.com] >>>> Sent: Thursday, February 12, 2015 10:35 AM >>>> >>>> On 02/11/2015 09:13 PM, Jan Beulich wrote: >>>>>>>> On 11.02.15 at 12:52, wrote: >>>>>> On 11/02/15 08:28, Kai Huang wrote: >>>>>>> With PML, we don't have to use write protection but just clear D-bit >>>>>>> of EPT entry of guest memory to do dirty logging, with an additional >>>>>>> PML buffer full VMEXIT for 512 dirty GPAs. Theoretically, this can >>>>>>> reduce hypervisor overhead when guest is in dirty logging mode, and >>>>>>> therefore more CPU cycles can be allocated to guest, so it's expected >>>>>>> benchmarks in guest will have better performance comparing to >>>> non-PML. >>>>>> One issue with basic EPT A/D tracking was the scan of the EPT tables. >>>>>> Here, hardware will give us a list of affected gfns, but how is Xen >>>>>> supposed to efficiently clear the dirty bits again? Using EPT >>>>>> misconfiguration is no better than the existing fault path. >>>>> Why not? The misconfiguration exit ought to clear the D bit for all >>>>> 511 entries in the L1 table (and set it for the one entry that is >>>>> currently serving the access). All further D bit handling will then >>>>> be PML based. >>>> Indeed, we clear D-bit in EPT misconfiguration. In my understanding, the >>>> sequences are as follows: >>>> >>>> 1) PML enabled for the domain. >>>> 2) ept_invalidate_emt (or ept_invalidate_emt_range) is called. >>>> 3) Guest accesses specific GPA (which has been invalidated by step 2), >>>> and EPT misconfig is triggered. >>>> 4) Then resolve_misconfig is called, which fixes up GFN (above GPA >> >>>> 12) to p2m_ram_logdirty, and calls ept_p2m_type_to_flags, in which we >>>> clear D-bit of EPT entry (instead of clear W-bit) if p2m type is >>>> p2m_ram_logdirty. Then dirty logging of this GFN will be handled by PML. >>>> >>>> The above 2) ~ 4) will be repeated when log-dirty radix tree is cleared. >>> is ept_invalidate_emt required by existing logdirty mode or by PML enable? >> It's in existing logdirty code. >>> can we clear D bits directly when log-dirty radix tree is cleared to reduce >>> EPT misconfig exits for repeatedly dirtied pages? >> Theoretically we can, and looks logdirty for video ram is done in this >> way (logdirty for the page is re-enabled while it is reported to >> dirty_bitmap). One thing is looks video ram logdirty only exists for HAP >> mode. >> But in current log dirty implementation for global logdirty, at common >> paging layer, the log-dirty radix tree is cleaned in single step after >> reporting all dirty pages to userspace. And it just calls >> ept_invalidate_emt essentially. Therefore we need to modify logdirty >> common code at paging layer to achieve this, which is more like logdirty >> enhancement but not related to PML enabling directly. And any change of >> interface in paging layer requires modification in shadow mode >> accordingly, so currently I just choose not to do it. >> > for general log dirty, ept_invalidate_emt is required because there is > access permission change (dirtied page becomes rw after 1st fault, > so need to change them back to ro again for the new dirty tracking > round). But for PML, there's no permission change at all (always rw), > so such behavior should be noted by general logdirty layer for better > optimization. I'm OK not doing so for initial enabling patch, but it's > something you can think about later. :-) Yes thanks for the point :) Thanks, -Kai > > Thanks > Kevin > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel