From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Chen, Tiejun" Subject: Re: [v7][RFC][PATCH 06/13] hvmloader/ram: check if guest memory is out of reserved device memory maps Date: Mon, 03 Nov 2014 19:32:20 +0800 Message-ID: <545767C4.7070806@intel.com> References: <1414136077-18599-1-git-send-email-tiejun.chen@intel.com> <1414136077-18599-7-git-send-email-tiejun.chen@intel.com> <544A84B10200007800042016@mail.emea.novell.com> <544DFDB2.2010508@intel.com> <544E29C70200007800042595@mail.emea.novell.com> <544F49F9.3070208@intel.com> <544F78B80200007800042B95@mail.emea.novell.com> <54509A8A.9060606@intel.com> <5450BE27020000780004304A@mail.emea.novell.com> <5451AC56.7010303@intel.com> <54521100020000780004363A@mail.emea.novell.com> <545320F2.5030103@intel.com> <545354500200007800043D94@mail.emea.novell.com> <5457174C.8020400@intel.com> <5457515102000078000443B0@mail.emea.novell.com> <54574D8F.8060407@intel.com> <54575E2D0200007800044443@mail.emea.novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <54575E2D0200007800044443@mail.emea.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jan Beulich Cc: Yang Z Zhang , Kevin Tian , "tim@xen.org" , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On 2014/11/3 17:51, Jan Beulich wrote: >>>> On 03.11.14 at 10:40, wrote: >> On 2014/11/3 16:56, Jan Beulich wrote: >>>>>> On 03.11.14 at 06:49, wrote: >>>> On 2014/10/31 16:20, Jan Beulich wrote: >>>>>>>> On 31.10.14 at 07:21, wrote: >>>>>>> From: Chen, Tiejun >>>>>>> Sent: Friday, October 31, 2014 1:41 PM >>>>>>> On 2014/10/30 17:20, Jan Beulich wrote: >>>>>>>> Thinking about this some more, this odd universal hole punching in >>>>>>>> the E820 is very likely to end up causing problems. Hence I think >>>>>>>> this really should be optional behavior, with pass through of devices >>>>>>>> associated with RMRRs failing if not used. (This ought to include >>>>>>>> punching holes for _just_ the devices passed through to a guest >>>>>>>> upon creation when the option is not enabled.) >>>>>>> >>>>>>> Yeah, we had a similar discussion internal to add a parameter to force >>>>>>> reserving RMRR. In this case we can't create a VM if these ranges >>>>>>> conflict with anything. So what about this idea? >>>>>>> >>>>>> >>>>>> Adding a new parameter (e.g. 'check-passthrough') looks the right >>>>>> approach. When the parameter is on, RMRR check/hole punch is >>>>>> activated at VM creation. Otherwise we just keep existing behavior. >>>>>> >>>>>> If user configures device pass-through at creation time, this parameter >>>>>> will be set by default. If user wants the VM capable of device hot-plug, >>>>>> an explicit parameter can be added in the config file to enforce RMRR >>>>>> check at creation time. >>>>> >>>>> Not exactly, I specifically described it slightly differently above. When >>>>> devices get passed through and the option is absent, holes should be >>>>> punched only for the RMRRs associated with those devices (i.e. >>>>> ideally none). Of course this means we'll need a way to associate >>>>> RMRRs with devices in the tool stack and hvmloader, i.e. the current >>>>> XENMEM_reserved_device_memory_map alone won't suffice. >>>> >>>> Yeah, current hypercall just provide RMRR entries without that >>>> associated BDF. And especially, in some cases one range may be shared by >>>> multiple devices... >>> >>> Before we decide who's going to do an eventual change we need to >>> determine what behavior we want, and whether this hypercall is >>> really the right one. Quite possibly we'd need a per-domain view >>> along with the global view, and hence rather than modifying this one >>> we may need to introduce e.g. a new domctl. >>> >> >> If we really need to work with a hypercall, maybe we can introduce a >> little bit to construct that to callback with multiple entries like >> this, for instance, >> >> RMRR entry0 have three devices, and entry1 have two devices, >> >> [start0, nr_pages0, bdf0], >> [start0, nr_pages0, bdf1], >> [start0, nr_pages0, bdf2], >> [start1, nr_pages1, bdf3], >> [start1, nr_pages1, bdf4], >> >> Although its cost more buffers, actually as you know this actual case is >> really rare. So maybe this way can be feasible. Then we don't need >> additional hypercall or xenstore. > > Conceptually, as a MEMOP, it has no business reporting BDFs. And > then rather than returning the same address range more than once, > having the caller supply a handle to an array and storing all of the > SBDFs (or perhaps a single segment would suffice along with all the > BDFs) there would seem to be an approach more consistent with > what we do elsewhere. Here I'm wondering if we really need to expose BDFs to tools. Actually tools just want to know those range no matter who owns these entries. I mean we can do this in Xen. When we try to assign device as passthrough, Xen can get that bdf so Xen can pre-check everything inside that hypercall, and Xen can return something like this, #1 If this device has RMRR, we return that rmrr buffer. This is similar with our current implementation. #2 If not, we return 'nr_entries' as '0' to notify hvmloader it has nothing to do. Thanks Tiejun