From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Michael Neuling To: Greg Kroah-Hartman Subject: Re: [PATCH v3] printk: Have printk() never buffer its data In-reply-to: <20120709170956.GB24728@kroah.com> References: <1341492628.834.3.camel@mop> <30233.1341535279@neuling.org> <29711.1341545979@neuling.org> <30343.1341546469@neuling.org> <21892.1341608647@neuling.org> <1341770155.1011.27.camel@mop> <20120709170956.GB24728@kroah.com> Date: Tue, 10 Jul 2012 08:36:52 +1000 Message-ID: <13501.1341873412@neuling.org> Cc: Kay Sievers , LKML , Steven Rostedt , "Paul E. McKenney" , linuxppc-dev@ozlabs.org, Joe Perches , Andrew Morton , Wu Fengguang , Linus Torvalds , Ingo Molnar List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Greg Kroah-Hartman wrote: > On Sun, Jul 08, 2012 at 07:55:55PM +0200, Kay Sievers wrote: > > On Sat, 2012-07-07 at 07:04 +1000, Michael Neuling wrote: > > > Whole kmsg below. > > > > I guess I have an idea now what's going on. > > > > > 4,47,0;WARNING: at /scratch/mikey/src/linux-ozlabs/arch/powerpc/sysdev/xics/xics-common.c:105 > > > 4,51,0;MSR: 9000000000021032 CR: 24000042 XER: 22000000 > > > 4,54,0;TASK = c000000000b2dd80[0] 'swapper/0' THREAD: c000000000c24000 CPU: 0 > > > > This is the warning on CPU#1, all fine, all in one line. > > > > > 6,74,0;console [tty0] enabled > > > 6,75,0;console [hvc0] enabled > > > > Now the boot consoles are registered, which replays the whole buffer > > that was collected up to this point. During the entire time the console > > semaphore needs to be held, and this can be quite a while. > > > > > 4,87,24545;WARNING: at /scratch/mikey/src/linux-ozlabs/arch/powerpc/sysdev/xics/xics-common.c:105 > > > \4,91,24586;MSR: 9000000000021032 > > > 4,92,24590;< > > > 4,93,24594;SF > > > 4,94,24599;,HV > > > 4,95,24604;,ME > > > 4,96,24609;,IR > > > 4,97,24614;,DR > > > 4,98,24619;,RI > > > 4,99,24623;> > > > 4,104,24661; CPU: 1 > > > > At the same time the CPU#2 prints the same warning with a continuation > > line, but the buffer from CPU#1 can not be flushed to the console, nor > > can the continuation line printk()s from CPU#2 be merged at this point. > > The consoles are still locked and busy with replaying the old log > > messages, so the new continuation data is just stored away in the record > > buffer as it is coming in. > > If the console would be registered a bit earlier, or the warning would > > happen a bit later, we would probably not see any of this. > > > > I can fake something like this just by holding the console semaphore > > over a longer time and printing continuation lines with different CPUs > > in a row. > > > > The patch below seems to work for me. It is also here: > > http://git.kernel.org/?p=linux/kernel/git/kay/patches.git;a=blob;f=kmsg-merge-cont.patch;hb=HEAD > > > > It only applies cleanly on top of this patch: > > http://git.kernel.org/?p=linux/kernel/git/kay/patches.git;a=blob;f=kmsg-syslog-1-byte-read.patch;hb=HEAD > > > > Thanks, > > Kay > > > > > > Subject: kmsg: merge continuation records while printing > > > > In (the unlikely) case our continuation merge buffer is busy, we unfortunately > > can not merge further continuation printk()s into a single record and have to > > store them separately, which leads to split-up output of these lines when they > > are printed. > > > > Add some flags about newlines and prefix existence to these records and try to > > reconstruct the full line again, when the separated records are printed. > > --- > > kernel/printk.c | 119 ++++++++++++++++++++++++++++++++++++-------------------- > > 1 file changed, 77 insertions(+), 42 deletions(-) > > Michael, did this solve the issue for you? It didn't but I've been working with Kay offline and what he pushed to you and is now in your driver-core-linus branch is good. ie. 5becfb1 kmsg: merge continuation records while printing ... works for me. Thanks, Mikey