From: Jacob Shin <jacob.shin@amd.com>
To: "H. Peter Anvin" <hpa@zytor.com>
Cc: Borislav Petkov <bp@alien8.de>, Yinghai Lu <yinghai@kernel.org>,
"H. Peter Anvin" <hpa@linux.intel.com>,
"Yu, Fenghua" <fenghua.yu@intel.com>,
"mingo@kernel.org" <mingo@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"linux-tip-commits@vger.kernel.org"
<linux-tip-commits@vger.kernel.org>,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
Stefano Stabellini <Stefano.Stabellini@eu.citrix.com>
Subject: Re: [tip:x86/microcode] x86/microcode_intel_early.c: Early update ucode on Intel's CPU
Date: Wed, 19 Dec 2012 22:16:21 -0600 [thread overview]
Message-ID: <20121220041621.GA23609@jshin-Toonie> (raw)
In-Reply-To: <50D279F9.4000707@zytor.com>
On Wed, Dec 19, 2012 at 06:37:45PM -0800, H. Peter Anvin wrote:
> On 12/19/2012 04:29 PM, Jacob Shin wrote:
> > On Wed, Dec 19, 2012 at 04:24:09PM -0800, H. Peter Anvin wrote:
> >> On 12/19/2012 04:07 PM, Jacob Shin wrote:
> >>>
> >>> From what I remember, accessing memory around the memory hole (not
> >>> just the HT hole, but e038000000 ~ 10000000000 on our mentioned system
> >>> ) generated prefetches because the memory hole was marked as WB in PAT.
> >>>
> >>> I'll take a look at the system again, try the blanket MTRR covering
> >>> 0xe000000000 ~ 1TB, and talk to our BIOS guys.
> >>>
> >>
> >> Yes, but do they all #MC (as opposed to, say, fetching all FFs)?
> >
> > Yes, MCE every time and it was fatal.
> >
>
> OK, one more question... there is something odd with the memory ranges here:
>
> BIOS-e820: [mem 0x0000000100000000-0x000000e037ffffff] usable
> BIOS-e820: [mem 0x000000e038000000-0x000000fcffffffff] reserved
> BIOS-e820: [mem 0x0000010000000000-0x0000011ffeffffff] usable
>
> The first usable range here is 4G to 896G + 896M which is an awfully
> strange number. Similarly, the second range is 1T to 1T + 128G - 16M.
> The little fiddly bits imply that there is either overshoot of some sort
> going on -- possibly reserved memory -- or these are fairly arbitrary
> sizes that don't match any physical bank sizes in which case it should
> be possible to shuffle it differently...
Not exactly sure why the wierd boundaries, I'll have to ask the BIOS
side folks to be sure. But if I were to guess ..
Here is the NUMA spew out, physically there is 128 GB connected to
each memory controller node. The PCI MMIO region starts at 0xc8000000.
4 GB - 0xc8000000 = 0x3800000 (896 MB). So we loose 896 MB due to PCI
MMIO hole, so the first node ends at 128 GB + 896 MB to talk to all of
128 GB off of the first memory controller, and hence the weird 896 MB
offset.
[ 0.000000] SRAT: Node 0 PXM 0 0-a0000
[ 0.000000] SRAT: Node 0 PXM 0 100000-c8000000
[ 0.000000] SRAT: Node 0 PXM 0 100000000-2038000000
[ 0.000000] SRAT: Node 1 PXM 1 2038000000-4038000000
[ 0.000000] SRAT: Node 2 PXM 2 4038000000-6038000000
[ 0.000000] SRAT: Node 3 PXM 3 6038000000-8038000000
[ 0.000000] SRAT: Node 4 PXM 4 8038000000-a038000000
[ 0.000000] SRAT: Node 5 PXM 5 a038000000-c038000000
[ 0.000000] SRAT: Node 6 PXM 6 c038000000-e038000000
[ 0.000000] SRAT: Node 7 PXM 7 10000000000-11fff000000
[ 0.000000] NUMA: Initialized distance table, cnt=8
[ 0.000000] NUMA: Node 0 [0,a0000) + [100000,c8000000) -> [0,c8000000)
[ 0.000000] NUMA: Node 0 [0,c8000000) + [100000000,2038000000) -> [0,2038000000)
[ 0.000000] Initmem setup node 0 0000000000000000-0000002038000000
[ 0.000000] NODE_DATA [0000002037ff5000 - 0000002037ffffff]
[ 0.000000] Initmem setup node 1 0000002038000000-0000004038000000
[ 0.000000] NODE_DATA [0000004037ff5000 - 0000004037ffffff]
[ 0.000000] Initmem setup node 2 0000004038000000-0000006038000000
[ 0.000000] NODE_DATA [0000006037ff5000 - 0000006037ffffff]
[ 0.000000] Initmem setup node 3 0000006038000000-0000008038000000
[ 0.000000] NODE_DATA [0000008037ff5000 - 0000008037ffffff]
[ 0.000000] Initmem setup node 4 0000008038000000-000000a038000000
[ 0.000000] NODE_DATA [000000a037ff5000 - 000000a037ffffff]
[ 0.000000] Initmem setup node 5 000000a038000000-000000c038000000
[ 0.000000] NODE_DATA [000000c037ff5000 - 000000c037ffffff]
[ 0.000000] Initmem setup node 6 000000c038000000-000000e038000000
[ 0.000000] NODE_DATA [000000e037ff2000 - 000000e037ffcfff]
[ 0.000000] Initmem setup node 7 0000010000000000-0000011fff000000
[ 0.000000] NODE_DATA [0000011ffeff1000 - 0000011ffeffbfff]
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal 0x00100000 -> 0x11fff000
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[10] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x00000099
[ 0.000000] 0: 0x00000100 -> 0x000c7ec0
[ 0.000000] 0: 0x00100000 -> 0x02038000
[ 0.000000] 1: 0x02038000 -> 0x04038000
[ 0.000000] 2: 0x04038000 -> 0x06038000
[ 0.000000] 3: 0x06038000 -> 0x08038000
[ 0.000000] 4: 0x08038000 -> 0x0a038000
[ 0.000000] 5: 0x0a038000 -> 0x0c038000
[ 0.000000] 6: 0x0c038000 -> 0x0e038000
[ 0.000000] 7: 0x10000000 -> 0x11fff000
[ 0.000000] On node 0 totalpages: 33553993
[ 0.000000] DMA zone: 56 pages used for memmap
[ 0.000000] DMA zone: 5 pages reserved
[ 0.000000] DMA zone: 3916 pages, LIFO batch:0
[ 0.000000] DMA32 zone: 14280 pages used for memmap
[ 0.000000] DMA32 zone: 800504 pages, LIFO batch:31
[ 0.000000] Normal zone: 447552 pages used for memmap
[ 0.000000] Normal zone: 32287680 pages, LIFO batch:31
[ 0.000000] On node 1 totalpages: 33554432
[ 0.000000] Normal zone: 458752 pages used for memmap
[ 0.000000] Normal zone: 33095680 pages, LIFO batch:31
[ 0.000000] On node 2 totalpages: 33554432
[ 0.000000] Normal zone: 458752 pages used for memmap
[ 0.000000] Normal zone: 33095680 pages, LIFO batch:31
[ 0.000000] On node 3 totalpages: 33554432
[ 0.000000] Normal zone: 458752 pages used for memmap
[ 0.000000] Normal zone: 33095680 pages, LIFO batch:31
[ 0.000000] On node 4 totalpages: 33554432
[ 0.000000] Normal zone: 458752 pages used for memmap
[ 0.000000] Normal zone: 33095680 pages, LIFO batch:31
[ 0.000000] On node 5 totalpages: 33554432
[ 0.000000] Normal zone: 458752 pages used for memmap
[ 0.000000] Normal zone: 33095680 pages, LIFO batch:31
[ 0.000000] On node 6 totalpages: 33554432
[ 0.000000] Normal zone: 458752 pages used for memmap
[ 0.000000] Normal zone: 33095680 pages, LIFO batch:31
[ 0.000000] On node 7 totalpages: 33550336
[ 0.000000] Normal zone: 458696 pages used for memmap
[ 0.000000] Normal zone: 33091640 pages, LIFO batch:31
>
> -hpa
>
>
next prev parent reply other threads:[~2012-12-20 4:16 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-30 1:47 [PATCH v2 00/11] x86/microcode: Early load microcode Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 01/10] Documentation/x86: " Fenghua Yu
2012-11-30 19:46 ` H. Peter Anvin
2012-11-30 20:40 ` Yu, Fenghua
2012-11-30 1:47 ` [PATCH v2 02/10] x86/microcode_intel.h: Define functions and macros for early loading ucode Fenghua Yu
2012-12-01 0:21 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 03/10] x86/microcode_core_early.c: Define interfaces " Fenghua Yu
2012-12-01 0:23 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 04/10] x86/microcode_intel_lib.c: Early update ucode on Intel's CPU Fenghua Yu
2012-12-01 0:24 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 05/10] x86/microcode_intel_early.c: " Fenghua Yu
2012-12-01 0:25 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-12-01 0:55 ` Yinghai Lu
2012-12-04 0:18 ` Yu, Fenghua
2012-12-11 2:39 ` Yinghai Lu
2012-12-11 3:41 ` H. Peter Anvin
2012-12-11 3:55 ` Yinghai Lu
2012-12-11 6:34 ` H. Peter Anvin
2012-12-11 7:07 ` Yinghai Lu
2012-12-11 14:57 ` Borislav Petkov
2012-12-11 16:46 ` Yinghai Lu
2012-12-11 16:48 ` H. Peter Anvin
2012-12-11 17:00 ` Yinghai Lu
2012-12-11 17:06 ` Borislav Petkov
2012-12-11 17:15 ` Yinghai Lu
2012-12-11 17:26 ` Yu, Fenghua
2012-12-11 17:38 ` H. Peter Anvin
2012-12-11 23:53 ` Yinghai Lu
2012-12-11 23:57 ` H. Peter Anvin
2012-12-12 0:27 ` Yinghai Lu
2012-12-12 0:37 ` H. Peter Anvin
2012-12-12 7:14 ` Yinghai Lu
2012-12-12 10:26 ` Yinghai Lu
2012-12-13 1:06 ` Yinghai Lu
2012-12-12 6:57 ` H. Peter Anvin
2012-12-12 13:38 ` Borislav Petkov
2012-12-12 17:43 ` H. Peter Anvin
2012-12-13 5:12 ` H. Peter Anvin
2012-12-13 5:26 ` H. Peter Anvin
2012-12-13 7:01 ` Yinghai Lu
2012-12-13 15:01 ` H. Peter Anvin
2012-12-13 19:13 ` Borislav Petkov
2012-12-13 21:36 ` H. Peter Anvin
2012-12-14 9:11 ` Yinghai Lu
2012-12-14 18:16 ` H. Peter Anvin
2012-12-14 19:46 ` H. Peter Anvin
2012-12-14 20:04 ` Yinghai Lu
2012-12-14 20:08 ` Yinghai Lu
2012-12-14 20:14 ` Yinghai Lu
2012-12-14 20:44 ` H. Peter Anvin
2012-12-15 7:57 ` Yinghai Lu
2012-12-15 19:30 ` H. Peter Anvin
2012-12-15 20:55 ` Yinghai Lu
2012-12-15 21:31 ` H. Peter Anvin
2012-12-15 21:40 ` H. Peter Anvin
2012-12-15 22:13 ` Yinghai Lu
2012-12-15 22:17 ` H. Peter Anvin
2012-12-15 23:15 ` Yinghai Lu
2012-12-15 23:17 ` H. Peter Anvin
2012-12-19 20:37 ` Borislav Petkov
2012-12-19 21:07 ` Jacob Shin
2012-12-19 21:48 ` H. Peter Anvin
2012-12-19 22:05 ` Jacob Shin
2012-12-19 22:25 ` H. Peter Anvin
2012-12-19 22:47 ` Yinghai Lu
2012-12-19 22:59 ` H. Peter Anvin
2012-12-19 22:51 ` Borislav Petkov
2012-12-19 22:59 ` Jacob Shin
2012-12-19 23:03 ` Borislav Petkov
2012-12-19 23:21 ` Jacob Shin
2012-12-19 23:56 ` H. Peter Anvin
2012-12-19 23:22 ` H. Peter Anvin
2012-12-19 23:40 ` Borislav Petkov
2012-12-20 0:02 ` H. Peter Anvin
2012-12-20 0:10 ` Borislav Petkov
2012-12-20 0:15 ` H. Peter Anvin
2012-12-19 23:40 ` Yinghai Lu
2012-12-19 23:43 ` H. Peter Anvin
2012-12-19 23:48 ` Yinghai Lu
2012-12-19 23:40 ` Jacob Shin
2012-12-19 23:45 ` Yinghai Lu
2012-12-19 23:50 ` H. Peter Anvin
2012-12-19 23:55 ` Borislav Petkov
2012-12-19 23:57 ` H. Peter Anvin
2012-12-20 0:07 ` Jacob Shin
2012-12-20 0:24 ` H. Peter Anvin
2012-12-20 0:29 ` Jacob Shin
2012-12-20 0:41 ` H. Peter Anvin
2012-12-20 2:37 ` H. Peter Anvin
2012-12-20 4:16 ` Jacob Shin [this message]
2012-12-20 4:21 ` H. Peter Anvin
2012-12-19 22:55 ` Jacob Shin
2012-12-19 23:00 ` Borislav Petkov
2012-12-19 23:17 ` H. Peter Anvin
2012-12-19 23:30 ` Borislav Petkov
2012-12-19 23:37 ` H. Peter Anvin
2012-12-19 23:23 ` H. Peter Anvin
2012-12-16 2:09 ` Yinghai Lu
2012-12-16 5:17 ` Yinghai Lu
2012-12-16 8:50 ` Yinghai Lu
2012-12-17 22:47 ` Yinghai Lu
2012-12-17 23:11 ` H. Peter Anvin
2012-12-17 23:26 ` Yinghai Lu
2012-12-18 1:11 ` Yinghai Lu
2012-12-18 1:51 ` Yinghai Lu
2012-12-18 2:42 ` Yinghai Lu
2012-12-14 20:10 ` H. Peter Anvin
2012-12-14 20:17 ` Yinghai Lu
2012-12-14 20:52 ` H. Peter Anvin
2012-12-14 21:07 ` Yinghai Lu
2012-12-11 18:02 ` H. Peter Anvin
2012-12-11 18:20 ` H. Peter Anvin
2012-12-11 18:42 ` Yinghai Lu
2012-12-11 18:46 ` H. Peter Anvin
2012-12-11 19:18 ` Yinghai Lu
2012-12-11 19:33 ` H. Peter Anvin
2012-11-30 1:47 ` [PATCH v2 06/10] x86/head_32.S: Early update ucode in 32-bit Fenghua Yu
2012-12-01 0:26 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 07/10] x86/head64.c: Early update ucode in 64-bit Fenghua Yu
2012-12-01 0:27 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 08/10] x86/smpboot.c: Early update ucode on AP Fenghua Yu
2012-12-01 0:28 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 09/10] x86/mm/init.c: Copy ucode from initrd image to memory Fenghua Yu
2012-12-01 0:29 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
2012-11-30 1:47 ` [PATCH v2 10/10] x86/Kconfig: Configurations to enable/disable the feature Fenghua Yu
2012-12-01 0:30 ` [tip:x86/microcode] x86/Kconfig: Configurations to enable/ disable " tip-bot for Fenghua Yu
-- strict thread matches above, loose matches on Subject: below --
2012-12-21 7:44 [PATCH v5 08/12] x86/microcode_intel_early.c: Early update ucode on Intel's CPU Fenghua Yu
2013-01-31 22:33 ` [tip:x86/microcode] " tip-bot for Fenghua Yu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121220041621.GA23609@jshin-Toonie \
--to=jacob.shin@amd.com \
--cc=Stefano.Stabellini@eu.citrix.com \
--cc=bp@alien8.de \
--cc=fenghua.yu@intel.com \
--cc=hpa@linux.intel.com \
--cc=hpa@zytor.com \
--cc=konrad.wilk@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=tglx@linutronix.de \
--cc=yinghai@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).