From: Mike Travis <travis@sgi.com>
To: David Rientjes <rientjes@google.com>
Cc: Ingo Molnar <mingo@elte.hu>, Jack Steiner <steiner@sgi.com>,
Robin Holt <holt@sgi.com>, Len Brown <len.brown@intel.com>,
Thomas Gleixner <tglx@linutronix.de>,
"H. Peter Anvin" <hpa@zytor.com>,
Andrew Morton <akpm@linux-foundation.org>,
Yinghai Lu <yhlu.kernel@gmail.com>,
linux-acpi@vger.kernel.org, x86@kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 3/6] x86: Minimize SRAT messages
Date: Wed, 23 Feb 2011 13:24:11 -0800 [thread overview]
Message-ID: <4D657AFB.4010501@sgi.com> (raw)
In-Reply-To: <alpine.DEB.2.00.1102191725150.27722@chino.kir.corp.google.com>
[I was finally able to get some time on our large UV test system.]
Here are the stats testing on a system with 248 nodes, 606 EFI
mem ranges, 1984 cores
after get_log_buff_early: (17% overflow)
[ 0.000000] early log_buf free: -45723/262183(-17%)
[ 0.000000] first line: : mem339: type=7, attr=0xf, range=[0x00000e6000000000-0x00000e6fff000000) (6552
Here I enabled some cores that were disabled so now the system
has 248 nodes, 606 EFI mem ranges, 2368 cores.
after minimize-time-zero-msgs: (5% overflow)
[0] early log_buf free: -15184/262172(-5%)
[0] first line: [0x000000007226e000-0x0000000072271000) (0MB) <6>[0] EFI: mem67: type=3, attr=0
Condensing the SRAT: PXM APIC messages resulted in 26% space free
in the early log buffer...
Was 2368 lines (for 2368 cores):
779 [0] SRAT: PXM 0 -> APIC 0x0000 -> Node 0
780 [0] SRAT: PXM 0 -> APIC 0x0002 -> Node 0
781 [0] SRAT: PXM 0 -> APIC 0x0004 -> Node 0
...
3145 [0] SRAT: PXM 247 -> APIC 0x3df0 -> Node 247
3146 [0] SRAT: PXM 247 -> APIC 0x3df2 -> Node 247
Now it's 248 lines (for 248 Nodes) (Nodes 0..191 have 10 core cpus.)
777 [0] SRAT: Node 0: PXM:APIC 0:0 :2 :4 :16 :18 :32 :34 :36 :48 :50
778 [0] SRAT: Node 1: PXM:APIC 1:64 :66 :68 :80 :82 :96 :98 :100 :112 :114
779 [0] SRAT: Node 2: PXM:APIC 2:128 :130 :132 :144 :146 :160 :162 :164 :176 :178
...
968 [0] SRAT: Node 190: PXM:APIC 190:12160 :12162 :12164 :12176 :12178 :12192 :12194 :12196 :12208 :12210
969 [0] SRAT: Node 191: PXM:APIC 191:12224 :12226 :12228 :12240 :12242 :12256 :12258 :12260 :12272 :12274
...
1023 [0] SRAT: Node 246: PXM:APIC 246:15744 :15746 :15748 :15760 :15778 :15780 :15792 :15794
1024 [0] SRAT: Node 247: PXM:APIC 247:15808 :15810 :15812 :15826 :15840 :15844 :15856 :15858
[0] early log_buf free: 69649/192523(26%)
[0] first line: <6>[0] Initializing cgroup subsys cpuset <6>[0] Initializing cgroup subsys cpu
My question is is the above decimal satisfactory, or should it be hex as
shown below? (Which will add 8k bytes for the "0x" when there are 4096 cores
but the hex values will be smaller.)
821 [0] SRAT: Node 0: PXM:APIC 0:0x0 :0x2 :0x4 :0x10 :0x12 :0x20 :0x22 :0x24 :0x30 :0x32
822 [0] SRAT: Node 1: PXM:APIC 1:0x40 :0x42 :0x44 :0x50 :0x52 :0x60 :0x62 :0x64 :0x70 :0x72
823 [0] SRAT: Node 2: PXM:APIC 2:0x80 :0x82 :0x84 :0x90 :0x92 :0xa0 :0xa2 :0xa4 :0xb0 :0xb2
...
1011 [0] SRAT: Node 190: PXM:APIC 190:0x2f80 :0x2f82 :0x2f84 :0x2f90 :0x2f92 :0x2fa0 :0x2fa2 :0x2fa4 :0x2fb0 :0x2fb2
1012 [0] SRAT: Node 191: PXM:APIC 191:0x2fc0 :0x2fc2 :0x2fc4 :0x2fd0 :0x2fd2 :0x2fe0 :0x2fe2 :0x2fe4 :0x2ff0 :0x2ff2
...
1067 [0] SRAT: Node 246: PXM:APIC 246:0x3d80 :0x3d82 :0x3d84 :0x3d90 :0x3da2 :0x3da4 :0x3db0 :0x3db2
1068 [0] SRAT: Node 247: PXM:APIC 247:0x3dc0 :0x3dc2 :0x3dc4 :0x3dd2 :0x3de0 :0x3de4 :0x3df0 :0x3df2
I will do some more study to see if affecting only these changes will
be enough to not overflow the early log buffer in a max config system.
(Btw, I have not figured out how to predict ahead of time that this
APIC id is the last one on the Node to insert the '\n'.)
Thanks,
Mike
next prev parent reply other threads:[~2011-02-23 21:24 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-19 2:47 [PATCH 0/6] init: Shrink early messages to prevent overflowing the kernel log buffer Mike Travis
2011-02-19 2:47 ` [PATCH 1/6] ACPI: Minimize X2APIC initial messages Mike Travis
2011-02-20 1:50 ` David Rientjes
2011-02-21 20:42 ` Mike Travis
2011-02-22 0:13 ` David Rientjes
2011-02-19 2:47 ` [PATCH 2/6] x86: Minimize initial e820 messages Mike Travis
2011-02-20 1:51 ` David Rientjes
2011-02-21 20:50 ` Mike Travis
2011-02-22 0:13 ` David Rientjes
2011-02-22 18:43 ` Bjorn Helgaas
2011-02-19 2:47 ` [PATCH 3/6] x86: Minimize SRAT messages Mike Travis
2011-02-20 1:51 ` David Rientjes
2011-02-23 21:24 ` Mike Travis [this message]
2011-02-24 4:12 ` David Rientjes
2011-02-19 2:47 ` [PATCH 4/6] printk: Break out printk_time Mike Travis
2011-02-20 1:51 ` David Rientjes
2011-02-19 2:47 ` [PATCH 5/6] printk: Minimize time zero output Mike Travis
2011-02-20 1:51 ` David Rientjes
2011-02-21 20:56 ` Mike Travis
2011-02-22 0:13 ` David Rientjes
2011-02-19 2:47 ` [PATCH 6/6] printk: Allocate kernel log buffer earlier Mike Travis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D657AFB.4010501@sgi.com \
--to=travis@sgi.com \
--cc=akpm@linux-foundation.org \
--cc=holt@sgi.com \
--cc=hpa@zytor.com \
--cc=len.brown@intel.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=rientjes@google.com \
--cc=steiner@sgi.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=yhlu.kernel@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox