From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gordan Bobic Subject: Re: HVM support for e820_host (Was: Bug: Limitation of <=2GB RAM in domU persists with 4.3.0) Date: Thu, 05 Sep 2013 22:46:34 +0100 Message-ID: <5228FBBA.3070108@bobich.net> References: <1374798084.10269.2.camel@hastur.hellion.org.uk> <20130729180431.GQ5848@phenom.dumpdata.com> <20130903145934.GC1487@konrad-lan.dumpdata.com> <52263CBD.1090402@bobich.net> <52264826.3010402@bobich.net> <20130903210833.GB13777@phenom.dumpdata.com> <20130905020442.GA2459@phenom.dumpdata.com> <5228F3E2.8090905@bobich.net> <5228F7BD.40405@bobich.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <5228F7BD.40405@bobich.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 09/05/2013 10:29 PM, Gordan Bobic wrote: > On 09/05/2013 10:13 PM, Gordan Bobic wrote: >> Right, finally got around to trying this with the latest patch. >> >> With e820_host=0 things work as before: >> >> (XEN) HVM3: BIOS map: >> (XEN) HVM3: f0000-fffff: Main BIOS >> (XEN) HVM3: E820 table: >> (XEN) HVM3: [00]: 00000000:00000000 - 00000000:0009e000: RAM >> (XEN) HVM3: [01]: 00000000:0009e000 - 00000000:000a0000: RESERVED >> (XEN) HVM3: HOLE: 00000000:000a0000 - 00000000:000e0000 >> (XEN) HVM3: [02]: 00000000:000e0000 - 00000000:00100000: RESERVED >> (XEN) HVM3: [03]: 00000000:00100000 - 00000000:e0000000: RAM >> (XEN) HVM3: HOLE: 00000000:e0000000 - 00000000:fc000000 >> (XEN) HVM3: [04]: 00000000:fc000000 - 00000001:00000000: RESERVED >> (XEN) HVM3: [05]: 00000001:00000000 - 00000002:1f800000: RAM >> >> >> I seem to be getting two different E820 table dumps with e820_host=1: >> >> (XEN) HVM1: BIOS map: >> (XEN) HVM1: f0000-fffff: Main BIOS >> (XEN) HVM1: build_e820_table:91 got 8 op.nr_entries >> (XEN) HVM1: E820 table: >> (XEN) HVM1: [00]: 00000000:00000000 - 00000000:3f790000: RAM >> (XEN) HVM1: [01]: 00000000:3f790000 - 00000000:3f79e000: ACPI >> (XEN) HVM1: [02]: 00000000:3f79e000 - 00000000:3f7d0000: NVS >> (XEN) HVM1: [03]: 00000000:3f7d0000 - 00000000:3f7e0000: RESERVED >> (XEN) HVM1: HOLE: 00000000:3f7e0000 - 00000000:3f7e7000 >> (XEN) HVM1: [04]: 00000000:3f7e7000 - 00000000:40000000: RESERVED >> (XEN) HVM1: HOLE: 00000000:40000000 - 00000000:fee00000 >> (XEN) HVM1: [05]: 00000000:fee00000 - 00000000:fee01000: RESERVED >> (XEN) HVM1: HOLE: 00000000:fee01000 - 00000000:ffc00000 >> (XEN) HVM1: [06]: 00000000:ffc00000 - 00000001:00000000: RESERVED >> (XEN) HVM1: [07]: 00000001:00000000 - 00000001:68870000: RAM >> (XEN) HVM1: E820 table: >> (XEN) HVM1: [00]: 00000000:00000000 - 00000000:0009e000: RAM >> (XEN) HVM1: [01]: 00000000:0009e000 - 00000000:000a0000: RESERVED >> (XEN) HVM1: HOLE: 00000000:000a0000 - 00000000:000e0000 >> (XEN) HVM1: [02]: 00000000:000e0000 - 00000000:00100000: RESERVED >> (XEN) HVM1: [03]: 00000000:00100000 - 00000000:a7800000: RAM >> (XEN) HVM1: HOLE: 00000000:a7800000 - 00000000:fc000000 >> (XEN) HVM1: [04]: 00000000:fc000000 - 00000001:00000000: RESERVED >> (XEN) HVM1: Invoking ROMBIOS ... >> >> I cannot quite figure out what is going on here - these tables can't >> both be true. >> >> Looking at the IOMEM on the host, the IOMEM begins at 0xa8000000 and >> goes more or less contiguously up to 0xfec8b000. >> >> Looking at dmesg on domU, the e820 map more or less matches the second >> dump above. >> >> So I guess that should work - the entire IOMEM area of the host is in >> fact not mapped. But since I've passed 8GB of RAM to domU, shouldn't >> there be another usable RAM area after 00000001:00000000 ? > > I should probably also mention that the domU does in fact see 8GB of > RAM, so clearly it is working. > > The PCI IOMEM reservations on the host are: > # lspci -vvv | grep Region | grep Memory | sed -e 's/.*Memory at //' | sort > a8000000 (64-bit, prefetchable) [disabled] [size=128M] > b0000000 (64-bit, prefetchable) [disabled] [size=64M] > b4000000 (64-bit, prefetchable) [size=64M] > b8000000 (64-bit, prefetchable) [size=128M] > c0000000 (64-bit, prefetchable) [size=256M] > d7efc000 (32-bit, non-prefetchable) [size=16K] > d8000000 (64-bit, non-prefetchable) [size=64K] > dc000000 (64-bit, non-prefetchable) [size=16K] > f3df4000 (64-bit, non-prefetchable) [size=16K] > f3df8000 (32-bit, non-prefetchable) [size=1K] > f3dfa000 (32-bit, non-prefetchable) [size=1K] > f3dfc000 (32-bit, non-prefetchable) [size=2K] > f3dfe000 (64-bit, non-prefetchable) [size=256] > f3edc000 (64-bit, non-prefetchable) [size=16K] > f3fdc000 (64-bit, non-prefetchable) [size=16K] > f4000000 (32-bit, non-prefetchable) [disabled] [size=32M] > f7ffc000 (32-bit, non-prefetchable) [disabled] [size=16K] > f8000000 (32-bit, non-prefetchable) [size=32M] > fbcfc000 (32-bit, non-prefetchable) [size=16K] > fbdfe000 (64-bit, non-prefetchable) [disabled] [size=8K] > fbeef000 (32-bit, non-prefetchable) [size=2K] > fbeefc00 (32-bit, non-prefetchable) [size=16] > fec8a000 (32-bit, non-prefetchable) [size=4K] > > > What is a little concerning is that my GPU in dom0 has it's IOMEM mapped at > E0000000-E7FFFFFF > E8000000-EBFFFFFF > EC000000-EDFFFFFF > > Granted, this fits into a convenient hole in the host map > 0xdc004000-0xf3df4000 but I cannot see that hole being listed as such in > the xl dmesg E820 table dump. Is this _really_ working, or is it working > by pure luck? Just doing a bit of testing at the moment. I haven't had a crash yet (it would have happened by now, as things were before). But - I am definitely getting the sort of graphical glitching/corruption in 3D applications that I saw before when assigning > 2688MB of RAM to the domU. That implies that there is still some memory overwriting happening somewhere. Aaand just as I was tying that - I've just had a crash. :'( Back to the drawing board... Gordan