From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Chen, Tiejun" Subject: Re: [RFC][PATCH 4/5] tools:firmware:hvmloader: reserve RMRR mappings in e820 Date: Fri, 15 Aug 2014 16:21:18 +0800 Message-ID: <53EDC2FE.9060806@intel.com> References: <1407409371-31728-1-git-send-email-tiejun.chen@intel.com> <1407409371-31728-5-git-send-email-tiejun.chen@intel.com> <53E88482020000780002AFB9@mail.emea.novell.com> <53E9F387.60605@intel.com> <53EA23C7020000780002B9A0@mail.emea.novell.com> <53EAB7E0.2020403@intel.com> <53EC26E6.7080707@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Tian, Kevin" , Jan Beulich Cc: "Zhang, Yang Z" , "xen-devel@lists.xen.org" , "ian.jackson@eu.citrix.com" , "ian.campbell@citrix.com" , "stefano.stabellini@eu.citrix.com" List-Id: xen-devel@lists.xenproject.org On 2014/8/15 7:11, Tian, Kevin wrote: >> From: Chen, Tiejun >> Sent: Wednesday, August 13, 2014 8:03 PM >> >> On 2014/8/14 3:10, Tian, Kevin wrote: >>>> From: Chen, Tiejun >>>> Sent: Tuesday, August 12, 2014 5:57 PM >>>> >>>> On 2014/8/12 20:25, Jan Beulich wrote: >>>>>>>> On 12.08.14 at 12:59, wrote: >>>>>> On 2014/8/12 0:00, Tian, Kevin wrote: >>>>>>>> From: Jan Beulich [mailto:JBeulich@suse.com] >>>>>>>> Sent: Sunday, August 10, 2014 11:53 PM >>>>>>>>>>> On 08.08.14 at 23:47, wrote: >>>>>>>>> strictly speaking besides reserving in e820, you should also poke later >>>>>>>>> MMIO BAR allocations to avoid confliction too. Currently it's relative >>>>>>>>> to low_mem_pgend, which is likely to be different from host layout >>>>>>>>> so it's still possible to see a virtual MMIO bar base conflicting to the >>>>>>>>> RMRR ranges which are supposed to be sparse. >>>>>>>> >>>>>>>> Correct. And what's worse: Possible collisions between RMRRs and >>>>>>>> the BIOS we place into the VM need to be taken care of, which may >>>>>>>> turn out rather tricky. >>>>>>>> >>>>>>> >>>>>>> right that becomes tricky. We can provide another hypercall to allow a >>>>>>> VM tell Xen which RMRR can't be assigned due to confliction with gust >>>>>>> BIOS or other hvmloader allocation (if confliction can't be resolved). >>>>>>> >>>>>>> If Xen detects a device owning RMRR is already assigned to the VM, >>>>>>> then fail the hypercall and hvmloader just panic with information to >>>>>>> indicate confliction. >>>>>>> >>>>>>> Otherwise Xen records the information and future dynamic device >>>>>>> assignment like hotplug will be failed if associated RMRR will be in >>>>>>> the confliction list. >>>>>> >>>>>> From my point of view its becoming over complicated. >>>>>> >>>>>> In HVM case, theoretically any devices involving RMRR may be assigned >> to >>>>>> any given VM. So it may not be necessary to introduce such complex >>>>>> mechanism. Therefore, I think we can reserve all RMRR maps simply in >>>>>> e820, and check if MMIO is overlapping with RMRR for every VM. It >> should >>>>>> be acceptable. >>>>> >>>>> Then you didn't understand what Kevin and I said above. Just >>>> >>>> I have to admit I'm poor in this coverage. >>>> >>>>> keep in mind that the RMRRs can conflict not just with MMIO >>>>> ranges inside the guest, but also RAM ranges (which include, as >>>>> mentioned above, the range where the BIOS for the guest gets >>>>> put). >>>>> >>>>> Jan >>>>> >>>> >>>> So just to clarify, as a summary there are four ranges we should be >>>> addressed: >>>> >>>> #1 MMIO in guest >>>> >>>> In my patch [RFC][v2][PATCH 5/6] tools:libxc: check if mmio BAR is out >>>> of RMRR mappings, >>>> >>>> I will check if this is overlapping. >>> >>> hvmloader controls actual mmio BAR allocation, so it's important to have >> >> I guess you're saying pci_setup(). >> >> After setup_guest(), in pci_setup() we will reallocate mmio and ram if >> necessary and possible. Then all final info is reflected to fill into GS >> e820. >> >>> check there. And your patch treats the whole mmio as one big region >>> to check overlapping with RMRR which is too coarse-grained. Better to >> check >> >> But its easy to feasible. >> >>> overlapping every time when an allocation, either of memory ranges, or >>> MMIO ranges, actually happen. >> >> What is your policy to handle a conflict? >> >> I mean those RMRR mapping entries are undermined and often they are not >> continuous. For example, IGD needs two entries in my current BDW, >> >> #1 ab805000 ~ ab819000 >> #2 ad000000 ~ af800000 >> >> So if just one of them conflicts something, how to handle such a case? >> Push mmio out of RMRR? Or allow many mmio hole? As you know IGD can't >> work as long as one of two entries is overlapping. >> >> So I think it may not be necessary to handle this as complicated mechanism. >> >> From my point of view its enough to double check RMRR in GS e820 since >> just do check rather than check-to-fix. If any overlap occurs we will >> post WARNING/ERROR to notify the user, then let user decide what we >> should do next. If they know don't need any PCI passthrough its fine. >> And especially, actually RMRR should be rare. >> > > I'm OK to do check-only w/o check-and-fix, at least it's a step forward > to fail-safe. Thanks a lot. BTW, looks Xen have many known places, even bug, we need to clean or improve, right? So why don't we have an explicit plan to push this step by step? I mean at least we can document these somewhere like this, #1 Notice some known problems #2 Any useful discussion #2 Workaround if possible #3 Next step or plan Its convenient to track them. Once one guy meet this again, he can find enough information to know how to deal with this. Thanks Tiejun > > Thanks > Kevin >