From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: [PATCHv4 0/8] x86/xen: fixes for mapping high MMIO regions (and remove _PAGE_IOMAP) Date: Mon, 3 Feb 2014 12:17:06 -0500 Message-ID: <20140203171706.GA6043@phenom.dumpdata.com> References: <1391446897-21998-1-git-send-email-david.vrabel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <1391446897-21998-1-git-send-email-david.vrabel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: David Vrabel Cc: x86@kernel.org, xen-devel@lists.xen.org, Ingo Molnar , "H. Peter Anvin" , Boris Ostrovsky , Thomas Gleixner List-Id: xen-devel@lists.xenproject.org On Mon, Feb 03, 2014 at 05:01:29PM +0000, David Vrabel wrote: > [ x86 maintainers, this is predominately a Xen series but the end > result is the _PAGE_IOMAP PTE flag is removed. See patch #8. ] > > This a fix for the problems with mapping high MMIO regions in certain > cases (e.g., the RDMA drivers) as not all mappers were specifing > _PAGE_IOMAP which meant no valid MFN could be found and the resulting > PTEs would be set as not present, causing subsequent faults. > > It assumes that anything that isn't RAM (whether ballooned out or not) > is an I/O region and thus should be 1:1 in the p2m. Specifically, the > region after the end of the E820 map and the region beyond the end of > the p2m. Ballooned frames are still marked as missing in the p2m as > before. > > As a follow on, pte_mfn_to_pfn() and pte_pfn_to_mfn() are modified to > not use the _PAGE_IOMAP PTE flag and MFN to PFN and PFN to MFN > translations will now do the right thing for all I/O regions. This > means the Xen-specific _PAGE_IOMAP can be removed, > > This series has been tested (in dom0) on all unique machines we have > in out test lab (~100 machines), some of which have PCI devices with > BARs above the end of RAM. This looks good to me and thank you for taking a stab at it. In the coming weeks I can run some more tests with this patchset where: - it is used in a PV PCI domU. - it is used in a PV PCI domU with backend devices (so say a NIC is exported to it and said NIC is backend for other PV devices). - it is used on a regular PV and we do ballooning, migration ,etc. - Get cycles on the box that has InfiniBand MMIOs in the 60TB range. > > Note this does not fix a 32-bit dom0 trying to access BARs above 16 TB > as this is a caused by MFNs/PFNs being limited to 32-bits (unsigned > long). > > You may find it useful to apply patch #3 to more easily review the > updated p2m diagram. > > Changes in v4: > - fix p2m_mid_identity initialization. > > Changes in v3 (not posted): > - use correct end of e820 > - fix xen_remap_domain_mfn_range() > > Changes in v2: > - fix to actually set end-of-RAM to 512 GiB region as 1:1. > - introduce p2m_mid_identity to efficiently store large 1:1 regions. > - Split the _PAGE_IOMAP patch into Xen and generic x86 halves. > > David >