From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Chen, Tiejun" Subject: Re: RMRR Fix Design for Xen Date: Tue, 06 Jan 2015 09:19:32 +0800 Message-ID: <54AB3824.10406@intel.com> References: <1418952066-21064-1-git-send-email-tiejun.chen@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: George Dunlap Cc: "Tian, Kevin" , Wei Liu , Ian Campbell , Stefano Stabellini , Ian Jackson , Tim Deegan , "xen-devel@lists.xen.org" , Jan Beulich , "Zhang, Yang Z" List-Id: xen-devel@lists.xenproject.org On 2015/1/6 1:01, George Dunlap wrote: > On Fri, Dec 19, 2014 at 1:21 AM, Tiejun Chen wrote: >> RMRR Fix Design for Xen >> >> This design is a goal to fix RMRR for Xen. It includes four sectors as >> follows: >> >> * Background >> * What is RMRR >> * Current RMRR Issues >> * Design Overview >> >> We hope this can help us to understand current problem then figure out a >> clean and better solution everyone can agree now to go forward. >> >> Background >> ========== >> >> We first identified this RMRR defect when trying to pass-through IGD device, >> which can be simply fixed by adding an identity mapping in case of shared >> EPT table. However along with the community discussion, it boiled down to >> a more general RMRR problem, i.e. the identity mapping is brute-added >> in hypervisor, w/o considering whether conflicting with an existing guest >> PFN ranges. As a general solution we need invent a new mechanism so >> reserved ranges allocated by hypervisor can be exported to the user space >> toolstack and hvmloader, so conflict can be detected when constructing >> guest PFN layout, with best-effort avoidance policies to further help. >> >> What is RMRR >> ============ >> >> RMRR is a acronym for Reserved Memory Region Reporting. >> >> BIOS may report each such reserved memory region through the RMRR structures, >> along with the devices that requires access to the specified reserved memory >> region. Reserved memory ranges that are either not DMA targets, or memory >> ranges that may be target of BIOS initiated DMA only during pre-boot phase >> (such as from a boot disk drive) must not be included in the reserved memory >> region reporting. The base address of each RMRR region must be 4KB aligned and >> the size must be an integer multiple of 4KB. BIOS must report the RMRR reported >> memory addresses as reserved in the system memory map returned through methods >> suchas INT15, EFI GetMemoryMap etc. The reserved memory region reporting >> structures are optional. If there are no RMRR structures, the system software >> concludes that the platform does not have any reserved memory ranges that are >> DMA targets. >> >> The RMRR regions are expected to be used for legacy usages (such as USB, UMA >> Graphics, etc.) requiring reserved memory. Platform designers shouldavoid or >> limit use of reserved memory regions since these require system software to >> create holes in the DMA virtual address range available to system software and >> its drivers. >> >> The following is grabbed from my BDW: >> >> (XEN) [VT-D]dmar.c:834: found ACPI_DMAR_RMRR: >> (XEN) [VT-D]dmar.c:679: RMRR region: base_addr ab80a000 end_address ab81dfff >> (XEN) [VT-D]dmar.c:834: found ACPI_DMAR_RMRR: >> (XEN) [VT-D]dmar.c:679: RMRR region: base_addr ad000000 end_address af7fffff >> >> Here USB occupies 0xab80a000:0xab81dfff, IGD owns 0xad000000:0xaf7fffff. >> >> Note there are zero or more Reserved Memory Region Reporting (RMRR) in one given >> platform. And multiple devices may share one RMRR range. Additionally RMRR can >> go anyplace. > > Tiejun, > > Thanks for this document -- such a document is really helpful in > figuring out the best way to architect the solution to a problem. > > I hope you don't mind me asking a few additional questions here. Everything is fine to me :) > You've said that: > * RMRR is a range used by devices (typically legacy devices such as > USB, but apparently also newer devices like IGD) > * RMRR ranges are reported by BIOSes > * RMRR ranges should be avoided by the guest. > > I'm still missing a few things, however. > > * In the case of passing through a virtual device, how does the > "range" apply wrt gpfn space and mfn space? I assume in example > above, the range [ab80a000,ab81dfff] corresponds to an mfn range. > When passing through this device to the guest, do pfns > [ab80a000,ab81dfff] need to be mapped to the same mfn range (i.e., 1-1 > mapping), or can they be mapped from somewhere else in pfn space? Always 1:1 mapping here. > > * You've described the range, but later on you talk about Xen > "creating" RMRR mappings. What does this mean? Are there registers > that need to be written? Do the ept / IOMMU tables need some kind of > special flags? We don't use or introduce any special flags. As you know Xen supports two scenarios. In case of non-shared ept, we just create VT-D table to cover those 1:1 mappings but if case of shared ept we always create and share such 1:1 mappings. BTW, this v1 document is not good as a design review so Kevin sent out v2, "(v2) Design proposal for RMRR fix", again. I hope that can provide more as you expect. Thanks Tiejun