From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kai Huang Subject: Re: PML (Page Modification Logging) design for Xen Date: Thu, 12 Feb 2015 15:04:47 +0800 Message-ID: <54DC508F.9080807@linux.intel.com> References: <54DB129D.3060102@linux.intel.com> <54DB61DB020000780005F069@mail.emea.novell.com> <54DC14CB.40305@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Tian, Kevin" , Jan Beulich Cc: "andrew.cooper3@citrix.com" , "xen-devel@lists.xen.org" , "keir@xen.org" , "tim@xen.org" List-Id: xen-devel@lists.xenproject.org On 02/12/2015 03:02 PM, Tian, Kevin wrote: >> From: Kai Huang [mailto:kai.huang@linux.intel.com] >> Sent: Thursday, February 12, 2015 10:50 AM >> >>>> - PML buffer flush >>>> >>>> There are two places we need to flush PML buffer. The first place is PML >>>> buffer full VMEXIT handler (apparently), and the second place is in >>>> paging_log_dirty_op (either peek or clean), as vcpus are running >>>> asynchronously along with paging_log_dirty_op is called from userspace >> via >>>> hypercall, and it's possible there are dirty GPAs logged in vcpus' PML >>>> buffers but not full. Therefore we'd better to flush all vcpus' PML buffers >>>> before reporting dirty GPAs to userspace. >>>> >>>> We handle above two cases by flushing PML buffer at the beginning of all >>>> VMEXITs. This solves the first case above, and it also solves the second >>>> case, as prior to paging_log_dirty_op, domain_pause is called, which kicks >>>> vcpus (that are in guest mode) out of guest mode via sending IPI, which >> cause >>>> VMEXIT, to them. >>>> >>>> This also makes log-dirty radix tree more updated as PML buffer is flushed >>>> on basis of all VMEXITs but not only PML buffer full VMEXIT. >>> Is that really efficient? Flushing the buffer only as needed doesn't >>> seem to be a major problem (apart from the usual preemption issue >>> when dealing with guests with very many vCPU-s, but you certainly >>> recall that at this point HVM is still limited to 128). >>> >>> Apart from these two remarks, the design looks okay to me. >> While keeping log-dirty radix tree more updated is probably irrelevant, >> I do think we'd better to flush PML buffers in paging_log_dirty_op (both >> peek and clear) before reporting dirty pages to userspace, in which case >> I think flushing PML buffer at beginning of VMEXIT is a good idea, as >> domain_pause does the job automatically. I am not sure how much cycles >> will flushing PML buffer contribute but I think it should be relatively >> small comparing to VMEXIT itself, therefore it can be ignored. > it's not intuitive to add overhead (one extra vmread) to every vmexit > just for utilizing the side-effect of one specific exit due to domain_pause. What's the cost of one vmread? It's reasonable to avoid it if it's heavy. > >> An optimized way probably is we only flush PML buffer for external >> interrupt VMEXIT, which domain_pause really triggers, but not at >> beginning of all VMEXITs. But as log as the overhead of flush PML buffer >> is negligible, this optimization is also unnecessary. >> > this optimization is not real optimization as you still stick to implementation > detail of other operations. Would you give me some possible hints? To me above is the most optimized way I can figure :) > If you really want to take use of domain_pause, > piggyback PML flush explicitly in that path make things clearer. domain_pause is called in many code path, looks it's not as optimized as my above one. Thanks, -Kai > > Thanks > Keivn > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel