From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stefan Bader Subject: Re: Is: MTRR issue + Xen + memory clipping? Was:Re: Xen not working with stock Debian Wheezy 3.2 kernel on a Core 2 Duo box Date: Tue, 09 Jul 2013 11:46:51 +0200 Message-ID: <51DBDC0B.6010507@canonical.com> References: <20130708103105.GB8027@zion.uk.xensource.com> <51DABE9E02000078000E33A6@nat28.tlf.novell.com> <20130708125012.GC8027@zion.uk.xensource.com> <51DAD7B702000078000E34BA@nat28.tlf.novell.com> <20130708194725.GL4927@phenom.dumpdata.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3819395938349498121==" Return-path: In-Reply-To: <20130708194725.GL4927@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: Wei Liu , Jan Beulich , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============3819395938349498121== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="------------enigAE222BC273723002053FBF07" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigAE222BC273723002053FBF07 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 08.07.2013 21:47, Konrad Rzeszutek Wilk wrote: > On Mon, Jul 08, 2013 at 02:16:07PM +0100, Jan Beulich wrote: >>>>> On 08.07.13 at 14:50, Wei Liu wrote: >>> On Mon, Jul 08, 2013 at 12:29:02PM +0100, Jan Beulich wrote: >>>> One question of course is where this pretty unusual "unusable" >>>> memory block comes from on that system. Is this block visible the >>>> same way when booting a native kernel, or is this being forced to >>>> "unusable" by Xen? >>> >>> Vanilla 3.10 + Xen unstable: >>> [ 0.000000] Xen: [mem 0x0000000000000000-0x000000000008efff] usabl= e >>> [ 0.000000] Xen: [mem 0x000000000008f000-0x00000000000fffff] reser= ved >>> [ 0.000000] Xen: [mem 0x0000000000100000-0x00000000ce699fff] usabl= e >>> [ 0.000000] Xen: [mem 0x00000000ce69a000-0x00000000ce6f0fff] ACPI = NVS >>> [ 0.000000] Xen: [mem 0x00000000ce6f1000-0x00000000cf5fafff] usabl= e >>> [ 0.000000] Xen: [mem 0x00000000cf5fb000-0x00000000cf607fff] reser= ved >>> [ 0.000000] Xen: [mem 0x00000000cf608000-0x00000000cf6a4fff] usabl= e >>> [ 0.000000] Xen: [mem 0x00000000cf6a5000-0x00000000cf6a9fff] ACPI = data >>> [ 0.000000] Xen: [mem 0x00000000cf6aa000-0x00000000cf6aafff] usabl= e >>> [ 0.000000] Xen: [mem 0x00000000cf6ab000-0x00000000cf6f1fff] ACPI = NVS >>> [ 0.000000] Xen: [mem 0x00000000cf6f2000-0x00000000cf6fefff] ACPI = data >>> [ 0.000000] Xen: [mem 0x00000000cf6ff000-0x00000000cf6fffff] usabl= e >>> [ 0.000000] Xen: [mem 0x00000000cf700000-0x00000000cfffffff] reser= ved >>> [ 0.000000] Xen: [mem 0x00000000fee00000-0x00000000fee00fff] reser= ved >>> [ 0.000000] Xen: [mem 0x00000000fff00000-0x00000000ffffffff] reser= ved >>> [ 0.000000] Xen: [mem 0x0000000100000000-0x0000000227ffffff] usabl= e >>> [ 0.000000] Xen: [mem 0x0000000228000000-0x000000022bffffff] unusa= ble >>> [ 0.000000] ERROR: earlyprintk=3D xenboot already used >>> [ 0.000000] NX (Execute Disable) protection: active >>> [ 0.000000] SMBIOS 2.4 present. >>> [ 0.000000] No AGP bridge found >>> [ 0.000000] e820: last_pfn =3D 0x228000 max_arch_pfn =3D 0x4000000= 00 >>> [ 0.000000] e820: last_pfn =3D 0xcf700 max_arch_pfn =3D 0x40000000= 0 >>> [ 0.000000] Scanning 1 areas for low memory corruption >>> [ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff] >>> [ 0.000000] init_memory_mapping: [mem 0x219e00000-0x219ffffff] >>> [ 0.000000] init_memory_mapping: [mem 0x218000000-0x219dfffff] >>> [ 0.000000] init_memory_mapping: [mem 0x200000000-0x217ffffff] >>> [ 0.000000] init_memory_mapping: [mem 0x00100000-0xce699fff] >>> [ 0.000000] init_memory_mapping: [mem 0xce6f1000-0xcf5fafff] >>> [ 0.000000] init_memory_mapping: [mem 0xcf608000-0xcf6a4fff] >>> [ 0.000000] init_memory_mapping: [mem 0xcf6aa000-0xcf6aafff] >>> [ 0.000000] init_memory_mapping: [mem 0xcf6ff000-0xcf6fffff] >>> [ 0.000000] init_memory_mapping: [mem 0x100000000-0x1ffffffff] >>> [ 0.000000] init_memory_mapping: [mem 0x21a000000-0x227ffffff] >>> >>> We also have the similar unusable block, however the kernel doesn't m= ap it. >> >> Right, iirc a fix for this was done not too long ago. Konrad may >> recall further details... >=20 > Stefan hit this I think. With the MTRR clipping blowing things. CC-ing = him > here. Not personally but we had a bug report about this. Unfortunately it was a= bit confusing as in the beginning it was not really obvious whether the probl= em was or was not fixed in later kernels or whether the different installations = used different dom0_mem arguments. Reading the old bug report (which never completed as it seemed the report= ers apparently lost interest) I think the problem was that the hypervisor det= ects the MTRR problem and set the remainder of memory as unusable. Then dom0 kernel (and if I parse that report right, it may have been fixe= d between 3.2 and 3.3 but hard to say when all you get is them saying yes o= r no) would somehow still try to put mappings in there. The link below is that bug report. Not sure of how much help it is. [1] https://bugs.launchpad.net/ubuntu/+source/xen/+bug/1111470 >> >>> 3.2.0-4-amd64 (just found out that there's actually backtrace in dmes= g): >>> [ 0.000000] BIOS-provided physical RAM map: >>> [ 0.000000] BIOS-e820: 0000000000000000 - 000000000008f000 (usabl= e) >>> [ 0.000000] BIOS-e820: 000000000008f000 - 00000000000a0000 (reser= ved) >>> [ 0.000000] BIOS-e820: 00000000000e0000 - 0000000000100000 (reser= ved) >>> [ 0.000000] BIOS-e820: 0000000000100000 - 00000000ce69a000 (usabl= e) >>> [ 0.000000] BIOS-e820: 00000000ce69a000 - 00000000ce6f1000 (ACPI = NVS) >>> [ 0.000000] BIOS-e820: 00000000ce6f1000 - 00000000cf5fb000 (usabl= e) >>> [ 0.000000] BIOS-e820: 00000000cf5fb000 - 00000000cf608000 (reser= ved) >>> [ 0.000000] BIOS-e820: 00000000cf608000 - 00000000cf6a5000 (usabl= e) >>> [ 0.000000] BIOS-e820: 00000000cf6a5000 - 00000000cf6aa000 (ACPI = data) >>> [ 0.000000] BIOS-e820: 00000000cf6aa000 - 00000000cf6ab000 (usabl= e) >>> [ 0.000000] BIOS-e820: 00000000cf6ab000 - 00000000cf6f2000 (ACPI = NVS) >>> [ 0.000000] BIOS-e820: 00000000cf6f2000 - 00000000cf6ff000 (ACPI = data) >>> [ 0.000000] BIOS-e820: 00000000cf6ff000 - 00000000cf700000 (usabl= e) >>> [ 0.000000] BIOS-e820: 00000000cf700000 - 00000000d0000000 (reser= ved) >>> [ 0.000000] BIOS-e820: 00000000fff00000 - 0000000100000000 (reser= ved) >>> [ 0.000000] BIOS-e820: 0000000100000000 - 000000022c000000 (usabl= e) >> >> So no such block right off the BIOS. You're not using Xen TXT code >> by chance? Off the top of my head I don't recall any other place >> where multiple RAM pages might get turned into "unusable". >> >> Jan >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel --------------enigAE222BC273723002053FBF07 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCgAGBQJR29wMAAoJEOhnXe7L7s6jS2gQANJyt/1C2A88FObsUa+bftll XID7GP/vZuidnEzfHYNS7v1cWnREFKElPjn/xxQlkbvSGcwI+1WLRObHCRcQwdTP ZpUBxcrkZ7ebMy+T4ByDEQkDLO9tyMuUav5G5gsvnbh95nZuTVL43KKz5ahSexdW JSQ/uTsY8Zwc0dyHBJaFVzTsxDC0fBpPdBa7Lm+/2/HOyl865TyitgihyNNWfepj gka9yARCx2ZgGQQHNmXZPVobe6EpmC1A9npswNepXPd93keJtSxLb4bGYKFCEw8A /3Zib/wSgH/3NmL5+t35xAXufUIfSNFhuU8QkTX9X/6K/JohIW94P9uD1s+TX67p kPLXP8acZdwibuFxpfG2HvTSws9w0wCsG3v5j4R+GR8iIcEXJFpHHCwtA+t0j95/ ewv0cI2BzgMLS6vcDZRo54f0c6Bd7QhdKww6YQc07diu39yAw0A+2n/lHJk1ZYjB 7Iq4fAdphY9SczhBS5R7O96IfHH9EOL8OdZ6QZ5Jnxeng3OyAXR49ArWZm69Whqd TFKRBDYQdyzBAdR/NnZnVeen4EcToHVybn/0Lio/XN78S6bxKVu+209GIKgyPSeE /JVMtWNjGJcV5GbJJjKwvR0DvznzRPgsnNsqcKI34NjqmN7NtoyAWJUVTGEhAfq/ cd7fpFcDwVUSxgyXGphk =nvR4 -----END PGP SIGNATURE----- --------------enigAE222BC273723002053FBF07-- --===============3819395938349498121== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============3819395938349498121==--