Re: [PATCH v3] printk: Have printk() never buffer its data

Greg Kroah-Hartman
2012-07-09T17:10:04Z

On Sun, Jul 08, 2012 at 07:55:55PM +0200, Kay Sievers wrote:
> On Sat, 2012-07-07 at 07:04 +1000, Michael Neuling wrote:
> > Whole kmsg below.
>
> I guess I have an idea now what's going on.
>
> > 4,47,0;WARNING: at /scratch/mikey/src/linux-ozlabs/arch/powerpc/sysdev/xics/xics-common.c:105
> > 4,51,0;MSR: 9000000000021032 <SF,HV,ME,IR,DR,RI> CR: 24000042 XER: 22000000
> > 4,54,0;TASK = c000000000b2dd80[0] 'swapper/0' THREAD: c000000000c24000 CPU: 0
>
> This is the warning on CPU#1, all fine, all in one line.
>
> > 6,74,0;console [tty0] enabled
> > 6,75,0;console [hvc0] enabled
>
> Now the boot consoles are registered, which replays the whole buffer
> that was collected up to this point. During the entire time the console
> semaphore needs to be held, and this can be quite a while.
>
> > 4,87,24545;WARNING: at /scratch/mikey/src/linux-ozlabs/arch/powerpc/sysdev/xics/xics-common.c:105
> > \4,91,24586;MSR: 9000000000021032
> > 4,92,24590;<
> > 4,93,24594;SF
> > 4,94,24599;,HV
> > 4,95,24604;,ME
> > 4,96,24609;,IR
> > 4,97,24614;,DR
> > 4,98,24619;,RI
> > 4,99,24623;>
> > 4,104,24661; CPU: 1
>
> At the same time the CPU#2 prints the same warning with a continuation
> line, but the buffer from CPU#1 can not be flushed to the console, nor
> can the continuation line printk()s from CPU#2 be merged at this point.
> The consoles are still locked and busy with replaying the old log
> messages, so the new continuation data is just stored away in the record
> buffer as it is coming in.
> If the console would be registered a bit earlier, or the warning would
> happen a bit later, we would probably not see any of this.
>
> I can fake something like this just by holding the console semaphore
> over a longer time and printing continuation lines with different CPUs
> in a row.
>
> The patch below seems to work for me. It is also here:
> http://git.kernel.org/?p=linux/kernel/git/kay/patches.git;a=blob;f=kmsg-merge-cont.patch;hb=HEAD
>
> It only applies cleanly on top of this patch:
> http://git.kernel.org/?p=linux/kernel/git/kay/patches.git;a=blob;f=kmsg-syslog-1-byte-read.patch;hb=HEAD
>
> Thanks,
> Kay
>
>
> Subject: kmsg: merge continuation records while printing
>
> In (the unlikely) case our continuation merge buffer is busy, we unfortunately
> can not merge further continuation printk()s into a single record and have to
> store them separately, which leads to split-up output of these lines when they
> are printed.
>
> Add some flags about newlines and prefix existence to these records and try to
> reconstruct the full line again, when the separated records are printed.
> ---
> kernel/printk.c | 119 ++++++++++++++++++++++++++++++++++++--------------------
> 1 file changed, 77 insertions(+), 42 deletions(-)

Michael, did this solve the issue for you?

It's a bit "big" of a patch so late in the -rc cycle, is it ok if we
just merge this in 3.6-rc1 and backport it to 3.5.1 if it looks ok
there?

thanks,

greg k-h