[PATCH] console logging detour via printk

[PATCH] console logging detour via printk

[Samo Pogacnik]

Hi,

while i was searching for effective logging of complete console output
produced by the kernel and user phase of the boot process, it turned out
that only kernel messages imho get systematically cached and stored into
log files (if needed). All userspace processes are on their own to use
syslog, which is fine, but there are also many console messages
reporting the boot status via init scripts, .... I came across the
bootlogd daemo, which handles the job of redirecting console output into
a log file, but i find it problematic to use especialy, when using
initial ram disk image.

So in short i came up with an idea to transform console writes into
printks at appropriate code place of some console drivers (the patch
includes code for VT console and SERIAL_CORE console drivers). Printks
eventually reach console device avoiding the patched part of the console
drivers.

I find this feature very interesting for embedded devices with limited
console abilities as well as for other for normal desktops using initial
ram disk images and / or some splash image to beautify booting. It is
always nice to have the possibility to review the complete boot output
at any time.

The more detail configuration of the feature is described in the Kconfig
files within the patch below.

The patch is also available here:
http://84.255.254.67/console_detour-2.6.33.3.patch

have fun, Samo

--
diff --git a_linux-2.6.33.3/drivers/char/Kconfig b_linux-2.6.33.3/drivers/char/Kconfig
index e023682..b5d0909 100644
--- a_linux-2.6.33.3/drivers/char/Kconfig
+++ b_linux-2.6.33.3/drivers/char/Kconfig
@@ -66,6 +66,23 @@ config VT_CONSOLE
 
 	  If unsure, say Y.
 
+config VT_CONSOLE_DETOUR
+	bool "Support for VT console detour via printk"
+	depends on VT_CONSOLE
+	default n
+	---help---
+	  If you do say Y here, the support for writing console messages via
+	  printk is included into VT console code.
+
+	  The feature is usefull to catch all console log. In order to use this
+	  feature, you should specify kernel command line option "detour" or write a
+	  positive number into /proc/sys/kernel/console_detour. You can disable
+	  the feature on-line by writing zero into the proc file. By writing a
+	  negative value into the proc file, the feature is disabled permanently
+	  (until next boot).
+
+	  If unsure, say N.
+
 config HW_CONSOLE
 	bool
 	depends on VT && !S390 && !UML
diff --git a_linux-2.6.33.3/drivers/char/vt.c b_linux-2.6.33.3/drivers/char/vt.c
index 50faa1f..ba649db 100644
--- a_linux-2.6.33.3/drivers/char/vt.c
+++ b_linux-2.6.33.3/drivers/char/vt.c
@@ -2696,6 +2696,16 @@ static int con_write(struct tty_struct *tty, const unsigned char *buf, int count
 {
 	int	retval;
 
+#ifdef CONFIG_VT_CONSOLE_DETOUR
+	if (console_detour) {
+		int idx = vt_console_driver.index - 1;
+ 
+		if ((idx >= 0) && (idx == tty->index)) {
+			console_printk_detour(buf, count);
+			return count;
+		}
+	}
+#endif
 	retval = do_con_write(tty, buf, count);
 	con_flush_chars(tty);
 
diff --git a_linux-2.6.33.3/drivers/serial/Kconfig b_linux-2.6.33.3/drivers/serial/Kconfig
index 9ff47db..20acfab 100644
--- a_linux-2.6.33.3/drivers/serial/Kconfig
+++ b_linux-2.6.33.3/drivers/serial/Kconfig
@@ -1031,6 +1031,23 @@ config SERIAL_CORE
 config SERIAL_CORE_CONSOLE
 	bool
 
+config SERIAL_CORE_CONSOLE_DETOUR
+	bool "Support for serial console detour via printk"
+	depends on SERIAL_CORE_CONSOLE
+	default n
+	---help---
+	  If you do say Y here, the support for writing console messages via
+	  printk is included into serial console code. 
+
+	  The feature is usefull to catch all console log. In order to use this
+	  feature, you should specify kernel command line option "detour" or write a
+	  positive number into /proc/sys/kernel/console_detour. You can disable
+	  the feature on-line by writing zero into the proc file. By writing a
+	  negative value into the proc file, the feature is disabled permanently
+	  (until next boot).
+
+	  If unsure, say N.
+
 config CONSOLE_POLL
 	bool
 
diff --git a_linux-2.6.33.3/drivers/serial/serial_core.c b_linux-2.6.33.3/drivers/serial/serial_core.c
index 7f28307..5e7e762 100644
--- a_linux-2.6.33.3/drivers/serial/serial_core.c
+++ b_linux-2.6.33.3/drivers/serial/serial_core.c
@@ -513,6 +513,12 @@ uart_write(struct tty_struct *tty, const unsigned char *buf, int count)
 	}
 
 	port = state->uart_port;
+#ifdef CONFIG_SERIAL_CORE_CONSOLE_DETOUR
+	if (console_detour && uart_console(port) && (port->cons->flags & CON_ENABLED)) {
+		console_printk_detour(buf, count);
+		return count;
+	}
+#endif
 	circ = &state->xmit;
 
 	if (!circ->buf)
diff --git a_linux-2.6.33.3/include/linux/console.h b_linux-2.6.33.3/include/linux/console.h
index dcca533..bc88030 100644
--- a_linux-2.6.33.3/include/linux/console.h
+++ b_linux-2.6.33.3/include/linux/console.h
@@ -108,6 +108,12 @@ struct console {
 	struct	 console *next;
 };
 
+extern int console_detour;
+extern void console_printk_detour(const unsigned char *, int);
+
+struct ctl_table;
+int detour_sysctl_handler(struct ctl_table *, int, void __user *, size_t *, loff_t *);
+
 extern int console_set_on_cmdline;
 
 extern int add_preferred_console(char *name, int idx, char *options);
diff --git a_linux-2.6.33.3/init/main.c b_linux-2.6.33.3/init/main.c
index 512ba15..5b1ba11 100644
--- a_linux-2.6.33.3/init/main.c
+++ b_linux-2.6.33.3/init/main.c
@@ -249,6 +249,15 @@ static int __init loglevel(char *str)
 
 early_param("loglevel", loglevel);
 
+extern int console_detour;
+static int __init detour(char *str)
+{
+	console_detour = 1;
+	return 0;
+}
+
+early_param("detour", detour);
+
 /*
  * Unknown boot options get handed to init, unless they look like
  * unused parameters (modprobe will find them in /proc/cmdline).
diff --git a_linux-2.6.33.3/kernel/printk.c b_linux-2.6.33.3/kernel/printk.c
index 1751c45..5a8056b 100644
--- a_linux-2.6.33.3/kernel/printk.c
+++ b_linux-2.6.33.3/kernel/printk.c
@@ -1368,6 +1368,51 @@ static int __init disable_boot_consoles(void)
 }
 late_initcall(disable_boot_consoles);
 
+/*
+ * This option can be enabled with kernel command line option "detour" or 
+ * through a proc file (/proc/sys/kernel/console_detour). It enables console 
+ * logging through printk, if supported by enabled console.
+ */
+int console_detour = 0;
+EXPORT_SYMBOL(console_detour);
+
+#define DETOUR_STR_SIZE 512
+void console_printk_detour(const unsigned char *buf, int count)
+{
+	char tmp[DETOUR_STR_SIZE + 1];
+
+	do {
+		if (count > DETOUR_STR_SIZE) {
+			memcpy(tmp, buf, DETOUR_STR_SIZE);
+			tmp[DETOUR_STR_SIZE] = '\0';
+		} else {
+			memcpy(tmp, buf, count);
+			tmp[count] = '\0';
+		}
+		count -= DETOUR_STR_SIZE;
+		printk("%s", tmp);
+	} while (count > 0);
+}
+EXPORT_SYMBOL(console_printk_detour);
+
+int detour_sysctl_handler(struct ctl_table *table, int write,
+	void __user *buffer, size_t *length, loff_t *ppos)
+{
+	static int disable_forever = 0;
+
+	proc_dointvec(table, write, buffer, length, ppos);
+	if (write) {
+		if ((console_detour < 0) || (disable_forever != 0)) {
+			disable_forever = 1;
+			console_detour = 0;
+			return 0;
+		}
+		if (console_detour > 1)
+			console_detour = 1;
+	}
+	return 0;
+}
+
 #if defined CONFIG_PRINTK
 
 /*
diff --git a_linux-2.6.33.3/kernel/sysctl.c b_linux-2.6.33.3/kernel/sysctl.c
index 8a68b24..ab644cb 100644
--- a_linux-2.6.33.3/kernel/sysctl.c
+++ b_linux-2.6.33.3/kernel/sysctl.c
@@ -50,6 +50,7 @@
 #include <linux/ftrace.h>
 #include <linux/slow-work.h>
 #include <linux/perf_event.h>
+#include <linux/console.h>
 
 #include <asm/uaccess.h>
 #include <asm/processor.h>
@@ -936,6 +937,13 @@ static struct ctl_table kern_table[] = {
 		.proc_handler	= proc_dointvec,
 	},
 #endif
+	{
+		.procname	= "console_detour",
+		.data		= &console_detour,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= &detour_sysctl_handler,
+	},
 /*
  * NOTE: do not add new entries to this table unless you have read
  * Documentation/sysctl/ctl_unnumbered.txt
 

Re: [PATCH] console logging detour via printk

[Randy Dunlap]

On Sat, 01 May 2010 00:03:00 +0200 Samo Pogacnik wrote:

> Hi,

> diff --git a_linux-2.6.33.3/drivers/char/Kconfig b_linux-2.6.33.3/drivers/char/Kconfig
> index e023682..b5d0909 100644
> --- a_linux-2.6.33.3/drivers/char/Kconfig
> +++ b_linux-2.6.33.3/drivers/char/Kconfig
> @@ -66,6 +66,23 @@ config VT_CONSOLE
>  
>  	  If unsure, say Y.
>  
> +config VT_CONSOLE_DETOUR
> +	bool "Support for VT console detour via printk"
> +	depends on VT_CONSOLE
> +	default n
> +	---help---
> +	  If you do say Y here, the support for writing console messages via

	  If you say Y here,

> +	  printk is included into VT console code.
> +
> +	  The feature is usefull to catch all console log. In order to use this

	                 useful                       log messages.

> +	  feature, you should specify kernel command line option "detour" or write a
> +	  positive number into /proc/sys/kernel/console_detour. You can disable
> +	  the feature on-line by writing zero into the proc file. By writing a
> +	  negative value into the proc file, the feature is disabled permanently
> +	  (until next boot).
> +
> +	  If unsure, say N.
> +
>  config HW_CONSOLE
>  	bool
>  	depends on VT && !S390 && !UML

> diff --git a_linux-2.6.33.3/drivers/serial/Kconfig b_linux-2.6.33.3/drivers/serial/Kconfig
> index 9ff47db..20acfab 100644
> --- a_linux-2.6.33.3/drivers/serial/Kconfig
> +++ b_linux-2.6.33.3/drivers/serial/Kconfig
> @@ -1031,6 +1031,23 @@ config SERIAL_CORE
>  config SERIAL_CORE_CONSOLE
>  	bool
>  
> +config SERIAL_CORE_CONSOLE_DETOUR
> +	bool "Support for serial console detour via printk"
> +	depends on SERIAL_CORE_CONSOLE
> +	default n
> +	---help---
> +	  If you do say Y here, the support for writing console messages via

	  If you say Y here,

> +	  printk is included into serial console code. 
> +
> +	  The feature is usefull to catch all console log. In order to use this

	                 useful                       log messages.

> +	  feature, you should specify kernel command line option "detour" or write a
> +	  positive number into /proc/sys/kernel/console_detour. You can disable
> +	  the feature on-line by writing zero into the proc file. By writing a
> +	  negative value into the proc file, the feature is disabled permanently
> +	  (until next boot).
> +
> +	  If unsure, say N.
> +

The kernel command line option needs to be added to Documentation/kernel-parameters.txt
also, please.

>  config CONSOLE_POLL
>  	bool
>  
> diff --git a_linux-2.6.33.3/include/linux/console.h b_linux-2.6.33.3/include/linux/console.h
> index dcca533..bc88030 100644
> --- a_linux-2.6.33.3/include/linux/console.h
> +++ b_linux-2.6.33.3/include/linux/console.h
> @@ -108,6 +108,12 @@ struct console {
>  	struct	 console *next;
>  };
>  
> +extern int console_detour;
> +extern void console_printk_detour(const unsigned char *, int);

Please include parameter names in function prototype(s).

> +
> +struct ctl_table;
> +int detour_sysctl_handler(struct ctl_table *, int, void __user *, size_t *, loff_t *);

ditto

> +
>  extern int console_set_on_cmdline;
>  
>  extern int add_preferred_console(char *name, int idx, char *options);


Looks interesting/useful to me.  Thanks.

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 30.04.2010 (pet) ob 15:45 -0700 je Randy Dunlap zapisal(a):
> On Sat, 01 May 2010 00:03:00 +0200 Samo Pogacnik wrote:
> 
> > Hi,
> 
> > diff --git a_linux-2.6.33.3/drivers/char/Kconfig b_linux-2.6.33.3/drivers/char/Kconfig
> > index e023682..b5d0909 100644
> > --- a_linux-2.6.33.3/drivers/char/Kconfig
> > +++ b_linux-2.6.33.3/drivers/char/Kconfig
> > @@ -66,6 +66,23 @@ config VT_CONSOLE
> >  
> >  	  If unsure, say Y.
> >  
> > +config VT_CONSOLE_DETOUR
> > +	bool "Support for VT console detour via printk"
> > +	depends on VT_CONSOLE
> > +	default n
> > +	---help---
> > +	  If you do say Y here, the support for writing console messages via
> 
> 	  If you say Y here,
> 
> > +	  printk is included into VT console code.
> > +
> > +	  The feature is usefull to catch all console log. In order to use this
> 
> 	                 useful                       log messages.
> 
> > +	  feature, you should specify kernel command line option "detour" or write a
> > +	  positive number into /proc/sys/kernel/console_detour. You can disable
> > +	  the feature on-line by writing zero into the proc file. By writing a
> > +	  negative value into the proc file, the feature is disabled permanently
> > +	  (until next boot).
> > +
> > +	  If unsure, say N.
> > +
> >  config HW_CONSOLE
> >  	bool
> >  	depends on VT && !S390 && !UML
> 
> > diff --git a_linux-2.6.33.3/drivers/serial/Kconfig b_linux-2.6.33.3/drivers/serial/Kconfig
> > index 9ff47db..20acfab 100644
> > --- a_linux-2.6.33.3/drivers/serial/Kconfig
> > +++ b_linux-2.6.33.3/drivers/serial/Kconfig
> > @@ -1031,6 +1031,23 @@ config SERIAL_CORE
> >  config SERIAL_CORE_CONSOLE
> >  	bool
> >  
> > +config SERIAL_CORE_CONSOLE_DETOUR
> > +	bool "Support for serial console detour via printk"
> > +	depends on SERIAL_CORE_CONSOLE
> > +	default n
> > +	---help---
> > +	  If you do say Y here, the support for writing console messages via
> 
> 	  If you say Y here,
> 
> > +	  printk is included into serial console code. 
> > +
> > +	  The feature is usefull to catch all console log. In order to use this
> 
> 	                 useful                       log messages.
> 
> > +	  feature, you should specify kernel command line option "detour" or write a
> > +	  positive number into /proc/sys/kernel/console_detour. You can disable
> > +	  the feature on-line by writing zero into the proc file. By writing a
> > +	  negative value into the proc file, the feature is disabled permanently
> > +	  (until next boot).
> > +
> > +	  If unsure, say N.
> > +
> 
> The kernel command line option needs to be added to Documentation/kernel-parameters.txt
> also, please.
> 
> >  config CONSOLE_POLL
> >  	bool
> >  
> > diff --git a_linux-2.6.33.3/include/linux/console.h b_linux-2.6.33.3/include/linux/console.h
> > index dcca533..bc88030 100644
> > --- a_linux-2.6.33.3/include/linux/console.h
> > +++ b_linux-2.6.33.3/include/linux/console.h
> > @@ -108,6 +108,12 @@ struct console {
> >  	struct	 console *next;
> >  };
> >  
> > +extern int console_detour;
> > +extern void console_printk_detour(const unsigned char *, int);
> 
> Please include parameter names in function prototype(s).
> 
> > +
> > +struct ctl_table;
> > +int detour_sysctl_handler(struct ctl_table *, int, void __user *, size_t *, loff_t *);
> 
> ditto
> 
> > +
> >  extern int console_set_on_cmdline;
> >  
> >  extern int add_preferred_console(char *name, int idx, char *options);
> 
> 
> Looks interesting/useful to me.  Thanks.
> 
> ---
> ~Randy
> *** Remember to use Documentation/SubmitChecklist when testing your code ***
Good morning,

Thank you for the corrections and hints. i'll catchup with an update as
soon as i can.

regards, Samo

Re: [PATCH] console logging detour via printk

[Geert Uytterhoeven]

On Sat, May 1, 2010 at 00:03, Samo Pogacnik <samo_pogacnik@t-2.net> wrote:
> while i was searching for effective logging of complete console output
> produced by the kernel and user phase of the boot process, it turned out
> that only kernel messages imho get systematically cached and stored into
> log files (if needed). All userspace processes are on their own to use
> syslog, which is fine, but there are also many console messages
> reporting the boot status via init scripts, .... I came across the
> bootlogd daemo, which handles the job of redirecting console output into
> a log file, but i find it problematic to use especialy, when using
> initial ram disk image.
>
> So in short i came up with an idea to transform console writes into
> printks at appropriate code place of some console drivers (the patch
> includes code for VT console and SERIAL_CORE console drivers). Printks
> eventually reach console device avoiding the patched part of the console
> drivers.

What about catching /dev/console instead of VT console,  SERIAL_CORE
console, ...?
Then it works with whatever console= parameter you specify.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: [PATCH] console logging detour via printk

[Alan Cox]

> while i was searching for effective logging of complete console output
> produced by the kernel and user phase of the boot process, it turned out
> that only kernel messages imho get systematically cached and stored into
> log files (if needed). All userspace processes are on their own to use
> syslog, which is fine, but there are also many console messages
> reporting the boot status via init scripts, .... I came across the
> bootlogd daemo, which handles the job of redirecting console output into
> a log file, but i find it problematic to use especialy, when using
> initial ram disk image.

So you want to patch the kernel because you can't work out how to do this
in userspace ? The distributions seem to have no problem doing this in
user space that I can see. It doesn't seem to be a hard user space
problem, and there are a ton of things you want to do with this sort of
stuff (like network logging) that you can't do in kernel space.

> --- a_linux-2.6.33.3/drivers/char/vt.c
> +++ b_linux-2.6.33.3/drivers/char/vt.c
> @@ -2696,6 +2696,16 @@ static int con_write(struct tty_struct *tty, const unsigned char *buf, int count
>  {
>  	int	retval;
>  
> +#ifdef CONFIG_VT_CONSOLE_DETOUR
> +	if (console_detour) {
> +		int idx = vt_console_driver.index - 1;
> + 
> +		if ((idx >= 0) && (idx == tty->index)) {
> +			console_printk_detour(buf, count);
> +			return count;
> +		}
> +	}
> +#endif

This requires you go around hacking up each device which is not a good
idea and becomes rapidly unmaintainable.

I suspect what you actually need for such logging might be to write a
very simple tty driver whose write method is implemented as printk. That
works in the general case and doesn't require hacking up the code
everywhere else.

However given your init stuff can trivially use openpty to set up a logged
console I am not sure I see the point in doing this in kernel in the
first place.

Alan

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 01.05.2010 (sob) ob 11:00 +0200 je Geert Uytterhoeven zapisal(a):
> On Sat, May 1, 2010 at 00:03, Samo Pogacnik <samo_pogacnik@t-2.net> wrote:
> > while i was searching for effective logging of complete console output
> > produced by the kernel and user phase of the boot process, it turned out
> > that only kernel messages imho get systematically cached and stored into
> > log files (if needed). All userspace processes are on their own to use
> > syslog, which is fine, but there are also many console messages
> > reporting the boot status via init scripts, .... I came across the
> > bootlogd daemo, which handles the job of redirecting console output into
> > a log file, but i find it problematic to use especialy, when using
> > initial ram disk image.
> >
> > So in short i came up with an idea to transform console writes into
> > printks at appropriate code place of some console drivers (the patch
> > includes code for VT console and SERIAL_CORE console drivers). Printks
> > eventually reach console device avoiding the patched part of the console
> > drivers.
> 
> What about catching /dev/console instead of VT console,  SERIAL_CORE
> console, ...?
> Then it works with whatever console= parameter you specify.

Could not agree more, but that was as close as i was able to detect the
common code and provide something that actually works. Maybe this is
already enough to cover all boot consoles? 

> 
> Gr{oetje,eeting}s,
> 
> 						Geert
> 
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
> 
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> 							    -- Linus Torvalds

regards, Samo

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 01.05.2010 (sob) ob 12:04 +0100 je Alan Cox zapisal(a):
> > while i was searching for effective logging of complete console output
> > produced by the kernel and user phase of the boot process, it turned out
> > that only kernel messages imho get systematically cached and stored into
> > log files (if needed). All userspace processes are on their own to use
> > syslog, which is fine, but there are also many console messages
> > reporting the boot status via init scripts, .... I came across the
> > bootlogd daemo, which handles the job of redirecting console output into
> > a log file, but i find it problematic to use especialy, when using
> > initial ram disk image.
> 
> So you want to patch the kernel because you can't work out how to do this
> in userspace ? The distributions seem to have no problem doing this in
> user space that I can see. It doesn't seem to be a hard user space
> problem, and there are a ton of things you want to do with this sort of
> stuff (like network logging) that you can't do in kernel space.

The distros have no problem logging complete console output into log
files or over the network, because they simply do not do it at least for
the initrd part of the boot process (i'd be glad, if i'm wrong). 
Anyway the proposed mechanism nicely bridges the gap between the kernel
boot start and the system logging daemon start-up. It also solves the
chicken and egg problem of how to log console if user space console
logging facility fails.

> 
> > --- a_linux-2.6.33.3/drivers/char/vt.c
> > +++ b_linux-2.6.33.3/drivers/char/vt.c
> > @@ -2696,6 +2696,16 @@ static int con_write(struct tty_struct *tty, const unsigned char *buf, int count
> >  {
> >  	int	retval;
> >  
> > +#ifdef CONFIG_VT_CONSOLE_DETOUR
> > +	if (console_detour) {
> > +		int idx = vt_console_driver.index - 1;
> > + 
> > +		if ((idx >= 0) && (idx == tty->index)) {
> > +			console_printk_detour(buf, count);
> > +			return count;
> > +		}
> > +	}
> > +#endif
> 
> This requires you go around hacking up each device which is not a good
> idea and becomes rapidly unmaintainable.

Agree, it should have been done within a well defined code volume.

> 
> I suspect what you actually need for such logging might be to write a
> very simple tty driver whose write method is implemented as printk. That
> works in the general case and doesn't require hacking up the code
> everywhere else.

Looks to me that some kernel code is welcome:)?

> 
> However given your init stuff can trivially use openpty to set up a logged
> console I am not sure I see the point in doing this in kernel in the
> first place.
> 

As said above, how to bridge kernel boot start and logging daemon
start-up without kernel help, especially when initrd is in the way? imho
it would be too complicated.

> Alan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

thanks for the reply, with best regards, Samo

Re: [PATCH] console logging detour via printk

[Alan Cox]

> The distros have no problem logging complete console output into log
> files or over the network, because they simply do not do it at least for
> the initrd part of the boot process (i'd be glad, if i'm wrong). 

I'd have to double check - but its trivial to move the log if so.

> > I suspect what you actually need for such logging might be to write a
> > very simple tty driver whose write method is implemented as printk. That
> > works in the general case and doesn't require hacking up the code
> > everywhere else.
> 
> Looks to me that some kernel code is welcome:)?

I really don't see the point but if you must do it then doing it as its
own driver would at least avoid making a mess in the rest of the kernel,
at which point it becomes less of a problem

> > However given your init stuff can trivially use openpty to set up a logged
> > console I am not sure I see the point in doing this in kernel in the
> > first place.
> > 
> 
> As said above, how to bridge kernel boot start and logging daemon
> start-up without kernel help, especially when initrd is in the way? imho
> it would be too complicated.

Put the logging start up in the initrd, its just a ramdisk its not
special in any way at all.

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 01.05.2010 (sob) ob 18:36 +0200 je Samo Pogacnik zapisal(a):
> Dne 01.05.2010 (sob) ob 11:00 +0200 je Geert Uytterhoeven zapisal(a):
> > On Sat, May 1, 2010 at 00:03, Samo Pogacnik <samo_pogacnik@t-2.net> wrote:
> > > while i was searching for effective logging of complete console output
> > > produced by the kernel and user phase of the boot process, it turned out
> > > that only kernel messages imho get systematically cached and stored into
> > > log files (if needed). All userspace processes are on their own to use
> > > syslog, which is fine, but there are also many console messages
> > > reporting the boot status via init scripts, .... I came across the
> > > bootlogd daemo, which handles the job of redirecting console output into
> > > a log file, but i find it problematic to use especialy, when using
> > > initial ram disk image.
> > >
> > > So in short i came up with an idea to transform console writes into
> > > printks at appropriate code place of some console drivers (the patch
> > > includes code for VT console and SERIAL_CORE console drivers). Printks
> > > eventually reach console device avoiding the patched part of the console
> > > drivers.
> > 
> > What about catching /dev/console instead of VT console,  SERIAL_CORE
> > console, ...?
> > Then it works with whatever console= parameter you specify.
> 
> Could not agree more, but that was as close as i was able to detect the
> common code and provide something that actually works. Maybe this is
> already enough to cover all boot consoles? 

Silly me, i managed to miss the common console write method. I'll
provide the change, so there is not going to be any specifics for
different console types anymore.

> 
> > 
> > Gr{oetje,eeting}s,
> > 
> > 						Geert
> > 
> > --
> > Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
> > 
> > In personal conversations with technical people, I call myself a hacker. But
> > when I'm talking to journalists I just say "programmer" or something like that.
> > 							    -- Linus Torvalds
> 
> regards, Samo
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 30.04.2010 (pet) ob 15:45 -0700 je Randy Dunlap zapisal(a):
> On Sat, 01 May 2010 00:03:00 +0200 Samo Pogacnik wrote:
> 
> > Hi,
> 
> > diff --git a_linux-2.6.33.3/drivers/char/Kconfig b_linux-2.6.33.3/drivers/char/Kconfig
> > index e023682..b5d0909 100644
> > --- a_linux-2.6.33.3/drivers/char/Kconfig
> > +++ b_linux-2.6.33.3/drivers/char/Kconfig
> > @@ -66,6 +66,23 @@ config VT_CONSOLE
> >  
> >  	  If unsure, say Y.
> >  
> > +config VT_CONSOLE_DETOUR
> > +	bool "Support for VT console detour via printk"
> > +	depends on VT_CONSOLE
> > +	default n
> > +	---help---
> > +	  If you do say Y here, the support for writing console messages via
> 
> 	  If you say Y here,
> 
> > +	  printk is included into VT console code.
> > +
> > +	  The feature is usefull to catch all console log. In order to use this
> 
> 	                 useful                       log messages.
> 
> > +	  feature, you should specify kernel command line option "detour" or write a
> > +	  positive number into /proc/sys/kernel/console_detour. You can disable
> > +	  the feature on-line by writing zero into the proc file. By writing a
> > +	  negative value into the proc file, the feature is disabled permanently
> > +	  (until next boot).
> > +
> > +	  If unsure, say N.
> > +
> >  config HW_CONSOLE
> >  	bool
> >  	depends on VT && !S390 && !UML
> 
> > diff --git a_linux-2.6.33.3/drivers/serial/Kconfig b_linux-2.6.33.3/drivers/serial/Kconfig
> > index 9ff47db..20acfab 100644
> > --- a_linux-2.6.33.3/drivers/serial/Kconfig
> > +++ b_linux-2.6.33.3/drivers/serial/Kconfig
> > @@ -1031,6 +1031,23 @@ config SERIAL_CORE
> >  config SERIAL_CORE_CONSOLE
> >  	bool
> >  
> > +config SERIAL_CORE_CONSOLE_DETOUR
> > +	bool "Support for serial console detour via printk"
> > +	depends on SERIAL_CORE_CONSOLE
> > +	default n
> > +	---help---
> > +	  If you do say Y here, the support for writing console messages via
> 
> 	  If you say Y here,
> 
> > +	  printk is included into serial console code. 
> > +
> > +	  The feature is usefull to catch all console log. In order to use this
> 
> 	                 useful                       log messages.
> 
> > +	  feature, you should specify kernel command line option "detour" or write a
> > +	  positive number into /proc/sys/kernel/console_detour. You can disable
> > +	  the feature on-line by writing zero into the proc file. By writing a
> > +	  negative value into the proc file, the feature is disabled permanently
> > +	  (until next boot).
> > +
> > +	  If unsure, say N.
> > +
> 
> The kernel command line option needs to be added to Documentation/kernel-parameters.txt
> also, please.
> 
> >  config CONSOLE_POLL
> >  	bool
> >  
> > diff --git a_linux-2.6.33.3/include/linux/console.h b_linux-2.6.33.3/include/linux/console.h
> > index dcca533..bc88030 100644
> > --- a_linux-2.6.33.3/include/linux/console.h
> > +++ b_linux-2.6.33.3/include/linux/console.h
> > @@ -108,6 +108,12 @@ struct console {
> >  	struct	 console *next;
> >  };
> >  
> > +extern int console_detour;
> > +extern void console_printk_detour(const unsigned char *, int);
> 
> Please include parameter names in function prototype(s).
> 
> > +
> > +struct ctl_table;
> > +int detour_sysctl_handler(struct ctl_table *, int, void __user *, size_t *, loff_t *);
> 
> ditto
> 
> > +
> >  extern int console_set_on_cmdline;
> >  
> >  extern int add_preferred_console(char *name, int idx, char *options);
> 
> 
> Looks interesting/useful to me.  Thanks.
> 
> ---
> ~Randy
> *** Remember to use Documentation/SubmitChecklist when testing your code ***

Hi, 

Here is the updated patch providing your hints and generalized for any
console type.

regards, Samo

---

Signed-off-by: Samo Pogacnik <samo_pogacnik@t-2.net>
diff --git a_linux-2.6.33.3/Documentation/kernel-parameters.txt b_linux-2.6.33.3/Documentation/kernel-parameters.txt
index e2c7487..ab0072c 100644
--- a_linux-2.6.33.3/Documentation/kernel-parameters.txt
+++ b_linux-2.6.33.3/Documentation/kernel-parameters.txt
@@ -628,6 +628,8 @@ and is between 256 and 4096 characters. It is defined in the file
 			Defaults to the default architecture's huge page size
 			if not specified.
 
+	detour		[KNL] Enable console logging detour via printk.
+
 	dhash_entries=	[KNL]
 			Set number of hash buckets for dentry cache.
 
diff --git a_linux-2.6.33.3/drivers/char/Kconfig b_linux-2.6.33.3/drivers/char/Kconfig
index e023682..43c552e 100644
--- a_linux-2.6.33.3/drivers/char/Kconfig
+++ b_linux-2.6.33.3/drivers/char/Kconfig
@@ -88,6 +88,22 @@ config VT_HW_CONSOLE_BINDING
 	 information. For framebuffer console users, please refer to
 	 <file:Documentation/fb/fbcon.txt>.
 
+config CONSOLE_DETOUR
+	bool "Support for console detour via printk"
+	default n
+	---help---
+	  If you say Y here, the support for writing console messages via
+	  printk is included into the console code.
+
+	  The feature is useful to catch all console log messages.
+	  In order to use this feature, you should specify kernel command line
+	  option "detour" or write a positive number into
+	  /proc/sys/kernel/console_detour. You can disable the feature on-line
+	  by writing zero into the proc file. By writing a negative value into
+	  the proc file, the feature is disabled permanently (until next boot).
+
+	  If unsure, say N.
+
 config DEVKMEM
 	bool "/dev/kmem virtual device support"
 	default y
diff --git a_linux-2.6.33.3/drivers/char/tty_io.c b_linux-2.6.33.3/drivers/char/tty_io.c
index 76253cf..f77de34 100644
--- a_linux-2.6.33.3/drivers/char/tty_io.c
+++ b_linux-2.6.33.3/drivers/char/tty_io.c
@@ -1087,6 +1087,13 @@ ssize_t redirected_tty_write(struct file *file, const char __user *buf,
 	}
 	spin_unlock(&redirect_lock);
 
+#ifdef CONFIG_CONSOLE_DETOUR
+	if (console_detour) {
+		console_printk_detour(buf, count);
+		if (!p)
+			return count;
+	}
+#endif
 	if (p) {
 		ssize_t res;
 		res = vfs_write(p, buf, count, &p->f_pos);
diff --git a_linux-2.6.33.3/include/linux/console.h b_linux-2.6.33.3/include/linux/console.h
index dcca533..354a7a8 100644
--- a_linux-2.6.33.3/include/linux/console.h
+++ b_linux-2.6.33.3/include/linux/console.h
@@ -108,6 +108,13 @@ struct console {
 	struct	 console *next;
 };
 
+extern int console_detour;
+extern void console_printk_detour(const unsigned char *buf, int count);
+
+struct ctl_table;
+int detour_sysctl_handler(struct ctl_table *table, int write,
+		void __user *buffer, size_t *length, loff_t *ppos);
+
 extern int console_set_on_cmdline;
 
 extern int add_preferred_console(char *name, int idx, char *options);
diff --git a_linux-2.6.33.3/init/main.c b_linux-2.6.33.3/init/main.c
index 512ba15..add9e95 100644
--- a_linux-2.6.33.3/init/main.c
+++ b_linux-2.6.33.3/init/main.c
@@ -25,6 +25,7 @@
 #include <linux/bootmem.h>
 #include <linux/acpi.h>
 #include <linux/tty.h>
+#include <linux/console.h>
 #include <linux/gfp.h>
 #include <linux/percpu.h>
 #include <linux/kmod.h>
@@ -249,6 +250,14 @@ static int __init loglevel(char *str)
 
 early_param("loglevel", loglevel);
 
+static int __init detour(char *str)
+{
+	console_detour = 1;
+	return 0;
+}
+
+early_param("detour", detour);
+
 /*
  * Unknown boot options get handed to init, unless they look like
  * unused parameters (modprobe will find them in /proc/cmdline).
diff --git a_linux-2.6.33.3/kernel/printk.c b_linux-2.6.33.3/kernel/printk.c
index 1751c45..953c6ea 100644
--- a_linux-2.6.33.3/kernel/printk.c
+++ b_linux-2.6.33.3/kernel/printk.c
@@ -1368,6 +1368,51 @@ static int __init disable_boot_consoles(void)
 }
 late_initcall(disable_boot_consoles);
 
+/*
+ * This option can be enabled with kernel command line option "detour" or
+ * through a proc file (/proc/sys/kernel/console_detour). It enables console
+ * logging through printk, if supported by enabled console.
+ */
+int console_detour;
+EXPORT_SYMBOL(console_detour);
+
+#define DETOUR_STR_SIZE 512
+void console_printk_detour(const unsigned char *buf, int count)
+{
+	char tmp[DETOUR_STR_SIZE + 1];
+
+	do {
+		if (count > DETOUR_STR_SIZE) {
+			memcpy(tmp, buf, DETOUR_STR_SIZE);
+			tmp[DETOUR_STR_SIZE] = '\0';
+		} else {
+			memcpy(tmp, buf, count);
+			tmp[count] = '\0';
+		}
+		count -= DETOUR_STR_SIZE;
+		printk(KERN_INFO "%s", tmp);
+	} while (count > 0);
+}
+EXPORT_SYMBOL(console_printk_detour);
+
+int detour_sysctl_handler(struct ctl_table *table, int write,
+	void __user *buffer, size_t *length, loff_t *ppos)
+{
+	static int disable_forever;
+
+	proc_dointvec(table, write, buffer, length, ppos);
+	if (write) {
+		if ((console_detour < 0) || (disable_forever != 0)) {
+			disable_forever = 1;
+			console_detour = 0;
+			return 0;
+		}
+		if (console_detour > 1)
+			console_detour = 1;
+	}
+	return 0;
+}
+
 #if defined CONFIG_PRINTK
 
 /*
diff --git a_linux-2.6.33.3/kernel/sysctl.c b_linux-2.6.33.3/kernel/sysctl.c
index 8a68b24..ab644cb 100644
--- a_linux-2.6.33.3/kernel/sysctl.c
+++ b_linux-2.6.33.3/kernel/sysctl.c
@@ -50,6 +50,7 @@
 #include <linux/ftrace.h>
 #include <linux/slow-work.h>
 #include <linux/perf_event.h>
+#include <linux/console.h>
 
 #include <asm/uaccess.h>
 #include <asm/processor.h>
@@ -936,6 +937,13 @@ static struct ctl_table kern_table[] = {
 		.proc_handler	= proc_dointvec,
 	},
 #endif
+	{
+		.procname	= "console_detour",
+		.data		= &console_detour,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= &detour_sysctl_handler,
+	},
 /*
  * NOTE: do not add new entries to this table unless you have read
  * Documentation/sysctl/ctl_unnumbered.txt

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 01.05.2010 (sob) ob 20:41 +0100 je Alan Cox zapisal(a):
> > The distros have no problem logging complete console output into log
> > files or over the network, because they simply do not do it at least for
> > the initrd part of the boot process (i'd be glad, if i'm wrong). 
> 
> I'd have to double check - but its trivial to move the log if so.
> 
> > > I suspect what you actually need for such logging might be to write a
> > > very simple tty driver whose write method is implemented as printk. That
> > > works in the general case and doesn't require hacking up the code
> > > everywhere else.
> > 
> > Looks to me that some kernel code is welcome:)?
> 
> I really don't see the point but if you must do it then doing it as its
> own driver would at least avoid making a mess in the rest of the kernel,
> at which point it becomes less of a problem
> 
> > > However given your init stuff can trivially use openpty to set up a logged
> > > console I am not sure I see the point in doing this in kernel in the
> > > first place.
> > > 
> > 
> > As said above, how to bridge kernel boot start and logging daemon
> > start-up without kernel help, especially when initrd is in the way? imho
> > it would be too complicated.
> 
> Put the logging start up in the initrd, its just a ramdisk its not
> special in any way at all.
I understand and can agree with everything you said, however providing
an additional mechanism might not hurt. Funny enough by generalizing the
patch for any console type, the essential code change is made at the
same place where initial console redirection mechanism has been
implemented. And again, console redirection can not log anything that
has been output to console prior to redirection activation.  

regards, Samo

Re: [PATCH] console logging detour via printk

[Marco Stornelli]

01/05/2010 20:48, Samo Pogacnik wrote:
> Dne 01.05.2010 (sob) ob 12:04 +0100 je Alan Cox zapisal(a):
>>> while i was searching for effective logging of complete console output
>>> produced by the kernel and user phase of the boot process, it turned out
>>> that only kernel messages imho get systematically cached and stored into
>>> log files (if needed). All userspace processes are on their own to use
>>> syslog, which is fine, but there are also many console messages
>>> reporting the boot status via init scripts, .... I came across the
>>> bootlogd daemo, which handles the job of redirecting console output into
>>> a log file, but i find it problematic to use especialy, when using
>>> initial ram disk image.
>>
>> So you want to patch the kernel because you can't work out how to do this
>> in userspace ? The distributions seem to have no problem doing this in
>> user space that I can see. It doesn't seem to be a hard user space
>> problem, and there are a ton of things you want to do with this sort of
>> stuff (like network logging) that you can't do in kernel space.
> 
> The distros have no problem logging complete console output into log
> files or over the network, because they simply do not do it at least for
> the initrd part of the boot process (i'd be glad, if i'm wrong). 

Mmm...It's an interesting problem. I see in my distro (openSuse) a
script called boot.klog that it seems to perform that (even initrd
part). In the file boot.msg I can see the initial prints of the kernel
and user space scripts.

Marco

Re: [PATCH] console logging detour via printk

[Samo Pogacnik]

Dne 02.05.2010 (ned) ob 11:58 +0200 je Marco Stornelli zapisal(a):
> 01/05/2010 20:48, Samo Pogacnik wrote:
> > Dne 01.05.2010 (sob) ob 12:04 +0100 je Alan Cox zapisal(a):
> >>> while i was searching for effective logging of complete console output
> >>> produced by the kernel and user phase of the boot process, it turned out
> >>> that only kernel messages imho get systematically cached and stored into
> >>> log files (if needed). All userspace processes are on their own to use
> >>> syslog, which is fine, but there are also many console messages
> >>> reporting the boot status via init scripts, .... I came across the
> >>> bootlogd daemo, which handles the job of redirecting console output into
> >>> a log file, but i find it problematic to use especialy, when using
> >>> initial ram disk image.
> >>
> >> So you want to patch the kernel because you can't work out how to do this
> >> in userspace ? The distributions seem to have no problem doing this in
> >> user space that I can see. It doesn't seem to be a hard user space
> >> problem, and there are a ton of things you want to do with this sort of
> >> stuff (like network logging) that you can't do in kernel space.
> > 
> > The distros have no problem logging complete console output into log
> > files or over the network, because they simply do not do it at least for
> > the initrd part of the boot process (i'd be glad, if i'm wrong). 
> 
> Mmm...It's an interesting problem. I see in my distro (openSuse) a
> script called boot.klog that it seems to perform that (even initrd
> part). In the file boot.msg I can see the initial prints of the kernel
> and user space scripts.
> 
Thanks for the info. 
Is this boot.klog script from the initrd image or from the real rootfs?
As you can see, i am still suspicious about the initrd part user console
messages:)

Samo

Re: [PATCH] console logging detour via printk

[Marco Stornelli]

Il 02/05/2010 15:29, Samo Pogacnik ha scritto:
> Dne 02.05.2010 (ned) ob 11:58 +0200 je Marco Stornelli zapisal(a):
>> 01/05/2010 20:48, Samo Pogacnik wrote:
>>
>> Mmm...It's an interesting problem. I see in my distro (openSuse) a
>> script called boot.klog that it seems to perform that (even initrd
>> part). In the file boot.msg I can see the initial prints of the kernel
>> and user space scripts.
>>
> Thanks for the info. 
> Is this boot.klog script from the initrd image or from the real rootfs?
> As you can see, i am still suspicious about the initrd part user console
> messages:)
> 
> Samo
> 
> 

In the initrd there's the script blogd.sh:

if test -z "$fboot" -a -z "$quiet" -a -z "$REDIRECT" ; then
    REDIRECT=$(showconsole 2>/dev/null)
    if test -n "$REDIRECT" ; then
	if test "$devpts" != "yes" ; then
	    mount -t devpts devpts /dev/pts
	    devpts=yes
	fi
	> /dev/shm/initrd.msg
	ln -sf /dev/shm/initrd.msg /var/log/boot.msg
	mkdir -p /var/run
	/sbin/blogd $REDIRECT
    fi
fi

And in the rootfs the boot.klog script:

# Read all kernel messages generated until now and put them in one file.
    test -s /var/log/boot.msg && mv -f /var/log/boot.msg /var/log/boot.omsg
    echo Creating /var/log/boot.msg
    if test -x /sbin/klogd ; then
        # klogd syncs out the file
	/sbin/klogd -s -o -n -f /var/log/boot.msg
	test -s /var/log/boot.msg
	rc_status -v1 -r
    elif test -x /bin/dmesg ; then
	/bin/dmesg > /var/log/boot.msg
	/bin/sync
	test -s /var/log/boot.msg
	rc_status -v1 -r
    fi
    if test -e /dev/shm/initrd.msg ; then
	cat /dev/shm/initrd.msg >> /var/log/boot.msg
	rm -f /dev/shm/initrd.msg
    fi
[ --- cut here --- ]

Regards,

Marco