From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?windows-1252?Q?J=FCrgen_Gro=DF?= Subject: Re: Buggy interaction of live migration and p2m updates Date: Fri, 21 Nov 2014 13:20:52 +0100 Message-ID: <546F2E24.2000800@suse.com> References: <546E32BB.8090909@citrix.com> <1416562990.26869.10.camel@citrix.com> <546F12F2.3020809@citrix.com> <1416566814.26869.14.camel@citrix.com> <546F1CF8.8010900@citrix.com> <1416568502.26869.24.camel@citrix.com> <546F2000.9020802@suse.com> <1416569047.26869.25.camel@citrix.com> <546F3B060200007800049B5D@smtp.nue.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <546F3B060200007800049B5D@smtp.nue.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich , Ian Campbell Cc: Wei Liu , Ian Jackson , Tim Deegan , Xen-devel List , David Vrabel , Andrew Cooper , ShriramRajagopalan , Hongyang Yang List-Id: xen-devel@lists.xenproject.org On 11/21/2014 01:15 PM, Jan Beulich wrote: >>>> On 21.11.14 at 12:24, wrote: >> On Fri, 2014-11-21 at 12:20 +0100, Juergen Gross wrote: >>> On 11/21/2014 12:15 PM, Ian Campbell wrote: >>>> On Fri, 2014-11-21 at 11:07 +0000, Andrew Cooper wrote: >>>>> On 21/11/14 10:46, Ian Campbell wrote: >>>>>> On Fri, 2014-11-21 at 10:24 +0000, Andrew Cooper wrote: >>>>>>> On 21/11/14 09:43, Ian Campbell wrote: >>>>>>>> I don't see any (explicit) mention of the pfn_to_mfn_frame_list_list >>>>>>>> here, where does that fit in? >>>>>>>> >>>>>>> It is referenced several times, although not by its exact name. >>>>>> Hence no explicit mention. >>>>>> >>>>>> It's ambiguous when you refer to "higher level frames" (which I presume >>>>>> are the reference you are referring to) because some kernels (perhaps >>>>>> only historic ones, I've not been keeping up) keep both an N-level tree >>>>>> of their own internally and the toolstack visible frame_list_list >>>>>> (sometimes partially overlapping at some level). Is every reference to >>>>>> "higher level frames" actually intended to be a reference to >>>>>> pfn_to_mfn_frame_list_list or not? >>>>> >>>>> "higher level frames" would be the toolstack-abi-defined first and >>>>> second level lists. The logdirty infrastructure can be used to detect >>>>> writes to these frames, and therefore detect structural changes to the p2m. >>>>> >>>>> I would like to hope that every kernel out there keeps this information >>>>> correctly up-to-date and updates it in an appropriate order... >>>>> >>>>>> >>>>>> It seems like sometimes you are talking at times about tracking the >>>>>> kernel's internal structure and not just pfn_to_mfn_frame_list_list and >>>>>> I'm not sure why that would be. >>>>> >>>>> I apologise for giving this impression. It was not intended. >>>> >>>> Great, I just wanted to be sure we were all on the same page, since >>>> scrobbling around in the kernel's internal data structures would clearly >>>> be mad... >>>> >>>>> >>>>>> I'm also not sure why >>>>>> pfn_to_mfn_frame_list_list is apparently discounted in the linear case, >>>>>> AFAIK the guest is still obliged to keep that up to date regardless of >>>>>> the scheme it uses internally for accessing the p2m. >>>>> >>>>> There are two reasons for the virtual linear p2m, the primary one being >>>>> to break the hard 512GB limit given the old 3-level table. >>>>> >>>>> A 64bit PV guest cannot possibly use the pfn_to_mfn_frame_list_list if >>>>> it needs to actually exceed 512GB of RAM. Therefore, to signal the use >>>>> the virtual linear method, a PV guest explicitly sets >>>>> pfn_to_mfn_frame_list_list to INVALID_MFN, and fills in the brand new >>>>> adjacent information. >>>> >>>> Oh, I hadn't realised this linear p2m stuff involved a guest ABI change. >>>> Have I somehow completely missed the xen.git side of these patches? I >>>> thought I'd only seen linux.git ones (and hence wasn't looking very >>>> closely). >>> >>> V1 of the patches suggesting such a change have been posted a week ago: >>> >>> http://lists.xen.org/archives/html/xen-devel/2014-11/msg01276.html >> >> Ah, thanks. I had indeed ignored that as "just another iteration of the >> linux patches", oops! > > So did I, not the least because it was sent to David and Konrad > rather than Cc-ing the hypervisor side maintainers (other than > me). Oops, sorry. Will do better for V2. Juergen