From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60542) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTxY0-00065V-0F for qemu-devel@nongnu.org; Wed, 09 Oct 2013 13:27:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VTxXv-0003gf-1h for qemu-devel@nongnu.org; Wed, 09 Oct 2013 13:27:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:11460) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTxXu-0003gX-Pu for qemu-devel@nongnu.org; Wed, 09 Oct 2013 13:27:38 -0400 Date: Wed, 9 Oct 2013 19:27:34 +0200 From: Igor Mammedov Message-ID: <20131009192734.67770b73@thinkpad> In-Reply-To: <1381324328.12583.102.camel@nilsson.home.kraxel.org> References: <1381321384-17270-1-git-send-email-imammedo@redhat.com> <1381324328.12583.102.camel@nilsson.home.kraxel.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] map 64-bit PCI devices after all possible RAM List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Gerd Hoffmann Cc: pbonzini@redhat.com, kevin@koconnor.net, seabios@seabios.org, qemu-devel@nongnu.org, mst@redhat.com On Wed, 09 Oct 2013 15:12:08 +0200 Gerd Hoffmann wrote: > On Mi, 2013-10-09 at 14:23 +0200, Igor Mammedov wrote: > > I'm posting it to get an oppinion on one of possible approaches > > on where to map a hotplug memory. > > > > This patch assumes that a space for hotplug memory is located right > > after RamSizeOver4G region and QEMU will provide romfile to specify > > where it ends so that BIOS could know from what base to start > > 64-bit PCI devices mapping. > > We should think about both pci hotplug and memory hotplug while being at > it. > > Today the 64bit pci window is mapped right above high memory and is > sized (in acpi tables) according to what is needed to map the devices > present at boot. > > Effect is that there is no extra address space for 64bit bars of > hotplugged pci devices. And the window is also in the way when it comes > to memory hotplug. > > Given that some windows versions don't like the large 64bit windows we > should make the window size configurable. So far from QEMU side it's partially (only memory region mapping and not ACPI window) configurable via {i440FX-pcihost|q35-pcihost}.pci-hole64-size property > > The window location can either be made configurable too, or we simply > place it at the top of the address space, with "address space" being > what the cpu can address according to cpuinfo. An earlier attempt by Michael to push complete PCI window placement info via "etc/pci-info" romfile to Seabios was rejected in favor of letting Seabios to program windows at hardcoded(32-bit/behind high mem) locations with a 64-bit window size (in ACPI) that covers all present devices but doesn't account for future PCI hotplug either. That behavior maintained in his "ACPI in QEMU" series, see: http://patchwork.ozlabs.org/patch/281032/ acpi_get_pci_info()->i440fx_pcihost_get_pci_hole64_end()->pci_bus_get_w64_range() which is then embedded in ACPI table. So end result stays the same as before (no usable 64-bit PCI window for hotlug). But 64-bit PCI window size, which is capped by QEMU to insane legacy 62 bits (memory region size), is a bit of orthogonal to freeing space for memory hotplug before it. > > Current qemu reports this by default: > > $ cat /proc/cpuinfo > model name : QEMU Virtual CPU version 1.5.3 > address sizes : 40 bits physical, 48 bits virtual > > 40 address lines allow 1TB, so we would place the window just below 1TB. > > Comments? More to the point if OS supports/enforces 1Tb physical address space,the RAM and 64-bit PCI hole are going to contend for it, QEMU could abort on startup if they both do not fit in CPU supported address space but I don't see what else it could do. Proposed patch favors RAM vs 64-bit PCI hole and moves the hole behind the possible RAM, which in present state of QEMU potentially leaves the rest of address space up to 62 bits for hole. It has drawback that one can't get a working VM if QEMU is started in memory hotlug mode with old BIOS + PCI devices that require 64-bit bars, otherwise it's backward compatible. PS: As for remedying BSODs because of huge CRS sizes of particular RAM device/PCI window, it might be solved by splitting one big chunk in several smaller, at least it works for RAM device. > > Gerd > > > -- Regards, Igor