From: Mike Travis <travis@sgi.com>
To: Ingo Molnar <mingo@elte.hu>, "H. Peter Anvin" <hpa@zytor.com>,
Andrew Morton <akpm@linux-foundation.org>,
David Rientjes <rientjes@google.com>, Len Brown <lenb@kernel.org>
Cc: Jack Steiner <steiner@sgi.com>, Lori Gilbertson <loriann@sgi.com>,
"linux-acpi@vger.kernel.org" <linux-acpi@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Early kernel messages are overflowing the static log buffer
Date: Mon, 13 Dec 2010 15:58:15 -0800 [thread overview]
Message-ID: <4D06B317.2090608@sgi.com> (raw)
Hi Ingo,
We have a problem on customer sites in that the early log buffer
messages overflow the static 128M log buffer before the log_buffer_len
parameter can be processed. This causes a major problem when we
are trying to debug a kernel panic from the customer's panic dump.
I've tried processing the log_buffer_len as soon as possible,
(right after the setup of Bootmem on Node 0), but I still lose
quite a bit. I think I've calculated that there are about
192M of messages output on a 2048 processor 128 node 4TB system
before the log buffer can be dynamically reallocated. And this
increases the complexity a bit because this is in arch dependent
code, so a generic reallocation routine is still needed.
And distros are against any attempt to increase the size of the
static log buffer so that's not really an option.
So it seems we need to reduce the number of messages.
The voluminous message sources before buffer reallocation are:
[ 0.000000] BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
...
[ 0.000000] EFI: mem00: type=3, attr=0xf, range=[0x0000000000000000-0x0000000000001000) (0MB) [
...
[ 0.000000] modified physical RAM map:
[ 0.000000] modified: 0000000000000000 - 0000000000001000 (usable)
...
[ 0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0
...
[ 0.000000] Bootmem setup node 0 0000000000000000-0000000800000000
[ 0.000000] NODE_DATA [000000000000e100 - 00000000000420ff]
[ 0.000000] bootmap [0000000000100000 - 00000000001fffff] pages 100
...
[ 0.000000] early_node_map[137] active PFN ranges
[ 0.000000] 0: 0x00000000 -> 0x00000001
...
[ 0.000000] On node 1 totalpages: 8388608
[ 0.000000] Normal zone: 114688 pages used for memmap
[ 0.000000] Normal zone: 8273920 pages, LIFO batch:31
...
[ 0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x00] enabled)
...
[ 0.000000] PM: Registered nosave memory: 0000000000001000 - 0000000000006000
...
[ 0.000000] pcpu-alloc: [000] 0000 0001 0002 0003 0004 0005 0006 0007 1024 1025 1026 1027 1028 1029 1030 1031
What I'm asking is which of these would be most acceptable to
either remove or replace with some sort of message reduction?
Note that a lot of the messages are completely redundant. Like
the Bootmem setup generally has exactly the same information
(spread over 15 lines), for each of the 128 Nodes.
One patch that I still have was David's "reduce srat verbosity in
the kernel log" that was rejected by you as too complex. This
would have resulted in about a 16:1 reduction in SRAT: messages,
without loss of an information. Before I start another doomed patch,
I'd like to find out the guidelines. Remember, no one really looks
at these messages unless the system panics, usually during startup.
So the information here may be key to diagnosing the problem.
Thanks,
Mike
next reply other threads:[~2010-12-13 23:58 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-13 23:58 Mike Travis [this message]
2010-12-22 13:09 ` Early kernel messages are overflowing the static log buffer Ingo Molnar
2010-12-22 13:17 ` Tejun Heo
2010-12-22 13:22 ` [PATCH] percpu: print out alloc information with KERN_DEBUG instead of KERN_INFO Tejun Heo
2010-12-22 18:56 ` Early kernel messages are overflowing the static log buffer Mike Travis
2010-12-22 19:03 ` Mike Travis
2010-12-22 21:21 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D06B317.2090608@sgi.com \
--to=travis@sgi.com \
--cc=akpm@linux-foundation.org \
--cc=hpa@zytor.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=loriann@sgi.com \
--cc=mingo@elte.hu \
--cc=rientjes@google.com \
--cc=steiner@sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox