From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56656) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ3jP-0005Xn-Ri for qemu-devel@nongnu.org; Fri, 01 Jul 2016 15:04:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bJ3jM-0002S8-KW for qemu-devel@nongnu.org; Fri, 01 Jul 2016 15:04:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43484) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ3jM-0002S2-CI for qemu-devel@nongnu.org; Fri, 01 Jul 2016 15:04:00 -0400 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BA46D7F0A0 for ; Fri, 1 Jul 2016 19:03:58 +0000 (UTC) Date: Fri, 1 Jul 2016 20:03:54 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20160701190354.GB29394@work-vm> References: <9b76415a-23e6-3ded-4dbc-42838cc164b0@redhat.com> <20160622142414.GI30202@redhat.com> <20160623014216-mutt-send-email-mst@redhat.com> <20160622232308.GQ30202@redhat.com> <20160623024400-mutt-send-email-mst@redhat.com> <1466671203.26189.35.camel@redhat.com> <20160629164252.GD10488@work-vm> <1467267046.15123.94.camel@redhat.com> <20160630105908.GA2683@work-vm> <1467303289.15123.102.camel@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1467303289.15123.102.camel@redhat.com> Subject: Re: [Qemu-devel] Default for phys-addr-bits? (was Re: [PATCH 4/5] x86: Allow physical address bits to be set) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Gerd Hoffmann Cc: "Michael S. Tsirkin" , Andrea Arcangeli , Marcel Apfelbaum , Paolo Bonzini , qemu-devel@nongnu.org, Eduardo Habkost * Gerd Hoffmann (kraxel@redhat.com) wrote: > > So that's mapped at an address beyond host phys-bits. > > And it hasn't failed/crashed etc - but I guess maybe nothing is using that 2G space? > > root@fedora ~# dmesg | grep Surface > [ 4.830095] [drm] qxl: 2048M of Surface memory size > > qxl bar 4 (64bit) and qxl bar 1 (32bit) are the same thing. The 64bit > bar can be alot larger obviously. The 32bit bar is just an alias for > the first portion of the 64bit bar. So I guess qxl just falls back to > use bar 1 instead of bar 4 because ioremap() on bar 4 fails. > > > Obviously 128T is a bit silly for maxmem at the moment, however I was worrying what > > happens with 36/39/40bit hosts, and it's not unusual to pick a maxmem that's a few TB > > even if the VMs you're initially creating are only a handful of GB. (oVirt/RHEV seems to use > > a 4TB default for maxmem). > > Oh, ok. Should be fixed I guess. > > > Still, this only hits as a problem if you hit the combination of: > > a) You use large PCI bars > > ovmf will map all 64bit bars high, even without running out of 32bit > address space. And with virtio 1.0 pretty much every virtual machine > will have 64bit bars. OK, yes I got that working, and you're right, it does map it high; (with recent OVMF running virtio 1.0, that needed recent guest/host kernels) and it does fail easily as well if memory doesn't fit, so for example: (all on a xeon with 46 bit physical host) specifying maxmem=1T - upstream build is hanging - but works if I specify phys-bits=46 so yes, it's noticing if the guest phys-bits is too small even if the host can manage it. It's OK if running with small amount of RAM and phys-bits=40 maxmem=64T with any phys-bits hangs. specifying maxmem=64T with phys-bits=46 on xeon and it hangs specifying maxmem=64T with phys-bits=48 on xeon and it hangs specifying maxmem=32T with phys-bits=46-48 on xeon and it works So for example we see: Bus 2, device 4, function 0: SCSI controller: PCI device 1af4:1042 IRQ 10. BAR1: 32 bit memory at 0x98000000 [0x98000fff]. BAR4: 64 bit prefetchable memory at 0x200800000000 [0x2008007fffff]. id "virtio-disk0" and that works nicely. Dave > > cheers, > Gerd > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK