[PATCH v10 0/1] Optionally allow ttynull to be selected as a default console

[PATCH v10 0/1] Optionally allow ttynull to be selected as a default console

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

When switching to a CONFIG_VT=n world, at least on x86 systems,
/dev/console becomes /dev/ttyS0. This can cause some undesired effects.
/dev/console's behavior is now tied to the physical /dev/ttyS0, which when
disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
and users who upgrade to a theoretical vt-less kernel from their
distribution who have a device such as a science instrument connected to
their /dev/ttyS0 port will suddenly see it receive kernel log messages.

When the new CONFIG_NULL_TTY_DEFAULT_CONSOLE option is turned on, this will
allow the ttynull device to be leveraged as the default console. Distributions
that had CONFIG_VT turned on before will be able to leverage this option
to where /dev/console is still backed by a pseudo device, avoiding these
issues, without needing to enable the entire VT subsystem.

v2:
    rebase

v3:
    Clarify commit messages.

    Guard the all the register_console()s in ttynull to prevent it from being
    registered twice.

    Only change the link order if CONFIG_NULL_TTY_CONSOLE is enabled, otherwise
    use the existing order for ttynull if only CONFIG_NULL_TTY is enabled.

    Document why the link order changes in the drivers/tty/Makefile file.

    Replace #ifdefs

v4:
    Remember to actually include the changes to v3 in the cover letter.

v5:
    Correct code formatting in Makefile comment.

v6:
    Change to CONFIG_NULL_TTY_DEFAULT_CONSOLE

    Set the index to -1, and don't set the flags

    Use add_preferred_console() instead of register_console() in ttynull's
    console_initcall when CONFIG_NULL_TTY_DEFAULT_CONSOLE is enabled

v7
    Add a commit message to the first commit, and the Suggested-by

    Correct Kconfig tabs/spaces formatting

    Invert the console_set_on_cmdline check

v8
    Update Documentation.

    Remove the "CONFIG_" in the help text when mentioning other configuration
    options.

    Update commit message

v9
    Spell pseudo correctly in the cover letter.

    Use more clear, and suggested help text for NULL_TTY_DEFAULT_CONSOLE.

    Do not set ttynull's index to -1 anymore.

    Do not rely on link order, call add_preferred_console() in console_init().

    Merge the Documentation/admin-guide/serial-console.rst change into one commit.

v10
    Fix commit message to mention the option correctly. Don't mention the older
    commit option from older revisions.

Adam Simonelli (1):
  printk: Add an option to allow ttynull to be a default console device

 Documentation/admin-guide/serial-console.rst |  4 +++-
 drivers/tty/Kconfig                          | 19 ++++++++++++++++++-
 kernel/printk/printk.c                       |  5 +++++
 3 files changed, 26 insertions(+), 2 deletions(-)

-- 
2.45.2

[PATCH v10 1/1] printk: Add an option to allow ttynull to be a default console device

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

The new option is CONFIG_NULL_TTY_DEFAULT_CONSOLE.

if enabled, and CONFIG_VT is disabled, ttynull will become the default
primary console device, based on the link order.

ttynull will be the only console device usually with this option enabled.
Some architectures do call add_preferred_console() which may add another
console though.

Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
if CONFIG_VT is disabled, the default console device falls back to
/dev/ttyS0 instead of /dev/tty.

This could cause issues in user space, and hardware problems:

1. The user space issues include the case where  /dev/ttyS0 is
disconnected, and the TCGETS ioctl, which some user space libraries use
as a probe to determine if a file is a tty, is called on /dev/console and
fails. Programs that call isatty() on /dev/console and get an incorrect
false value may skip expected logging to /dev/console.

2. The hardware issues include the case if a user has a science instrument
or other device connected to the /dev/ttyS0 port, and they were to upgrade
to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
sent to the device connected to /dev/ttyS0 unless they edit their kernel
command line manually.

The new CONFIG_NULL_TTY_DEFAULT_CONSOLE option will give users and
distribution maintainers an option to avoid this. Disabling CONFIG_VT and
enabling CONFIG_NULL_TTY_DEFAULT_CONSOLE will ensure the default kernel
console behavior is not dependant on hardware configuration by default, and
avoid unexpected new behavior on devices connected to the /dev/ttyS0 serial
port.

Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
---
 Documentation/admin-guide/serial-console.rst |  4 +++-
 drivers/tty/Kconfig                          | 19 ++++++++++++++++++-
 kernel/printk/printk.c                       |  5 +++++
 3 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/Documentation/admin-guide/serial-console.rst b/Documentation/admin-guide/serial-console.rst
index a3dfc2c66e01..1609e7479249 100644
--- a/Documentation/admin-guide/serial-console.rst
+++ b/Documentation/admin-guide/serial-console.rst
@@ -78,7 +78,9 @@ If no console device is specified, the first device found capable of
 acting as a system console will be used. At this time, the system
 first looks for a VGA card and then for a serial port. So if you don't
 have a VGA card in your system the first serial port will automatically
-become the console.
+become the console, unless the kernel is configured with the
+CONFIG_NULL_TTY_DEFAULT_CONSOLE option, then it will default to using the
+ttynull device.
 
 You will need to create a new device to use ``/dev/console``. The official
 ``/dev/console`` is now character device 5,1.
diff --git a/drivers/tty/Kconfig b/drivers/tty/Kconfig
index 63a494d36a1f..7fb81bbaee60 100644
--- a/drivers/tty/Kconfig
+++ b/drivers/tty/Kconfig
@@ -383,7 +383,24 @@ config NULL_TTY
 	  available or desired.
 
 	  In order to use this driver, you should redirect the console to this
-	  TTY, or boot the kernel with console=ttynull.
+	  TTY, boot the kernel with console=ttynull, or enable
+	  NULL_TTY_DEFAULT_CONSOLE.
+
+	  If unsure, say N.
+
+config NULL_TTY_DEFAULT_CONSOLE
+	bool "Support for console on ttynull"
+	depends on NULL_TTY=y && !VT_CONSOLE
+	help
+	  Say Y here if you want the NULL TTY to be used as a /dev/console
+	  device by default.
+
+	  For example, it might be useful to prevent a VT-less kernel from
+	  writing the system log to a random device connected to the serial
+	  port.
+
+	  Another console driver still might get preferred via the command
+	  line, SPCR, or the device tree.
 
 	  If unsure, say N.
 
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 07668433644b..9dd807717cd4 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -4277,6 +4277,11 @@ void __init console_init(void)
 	initcall_t call;
 	initcall_entry_t *ce;
 
+#ifdef CONFIG_NULL_TTY_DEFAULT_CONSOLE
+       if (!console_set_on_cmdline)
+               add_preferred_console("ttynull", 0, NULL);
+#endif
+
 	/* Setup the default TTY line discipline. */
 	n_tty_init();
 
-- 
2.45.2

Re: [PATCH v10 1/1] printk: Add an option to allow ttynull to be a default console device

[Petr Mladek]

On Thu 2025-03-13 20:41:04, adamsimonelli@gmail.com wrote:
> From: Adam Simonelli <adamsimonelli@gmail.com>
> 
> The new option is CONFIG_NULL_TTY_DEFAULT_CONSOLE.
> 
> if enabled, and CONFIG_VT is disabled, ttynull will become the default
> primary console device, based on the link order.

The result is not longer based on the link order.

The linking order affected the ordering the console initcalls.
But this patch calls add_preferred_console() directly in console_init()
before processing the init calls...

I would just remove the ", based on the link order".

> ttynull will be the only console device usually with this option enabled.
> Some architectures do call add_preferred_console() which may add another
> console though.

I would add here the following line:

Motivation:

to clearly separate the description of the new behavior from the motivation.

> Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
> if CONFIG_VT is disabled, the default console device falls back to
> /dev/ttyS0 instead of /dev/tty.
> 
> This could cause issues in user space, and hardware problems:
> 
> 1. The user space issues include the case where  /dev/ttyS0 is
> disconnected, and the TCGETS ioctl, which some user space libraries use
> as a probe to determine if a file is a tty, is called on /dev/console and
> fails. Programs that call isatty() on /dev/console and get an incorrect
> false value may skip expected logging to /dev/console.
> 
> 2. The hardware issues include the case if a user has a science instrument
> or other device connected to the /dev/ttyS0 port, and they were to upgrade
> to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
> sent to the device connected to /dev/ttyS0 unless they edit their kernel
> command line manually.
> 
> The new CONFIG_NULL_TTY_DEFAULT_CONSOLE option will give users and
> distribution maintainers an option to avoid this. Disabling CONFIG_VT and
> enabling CONFIG_NULL_TTY_DEFAULT_CONSOLE will ensure the default kernel
> console behavior is not dependant on hardware configuration by default, and

s/dependant/dependent/

> avoid unexpected new behavior on devices connected to the /dev/ttyS0 serial
> port.
> 
> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> index 07668433644b..9dd807717cd4 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -4277,6 +4277,11 @@ void __init console_init(void)
>  	initcall_t call;
>  	initcall_entry_t *ce;
>  
> +#ifdef CONFIG_NULL_TTY_DEFAULT_CONSOLE
> +       if (!console_set_on_cmdline)
> +               add_preferred_console("ttynull", 0, NULL);

checkpatch.pl reports that there are used spaces instead of
tabs in the two lines above. I suggest to use some editor which takes
care of proper indentation, e.g. emacs or vim and run
./scripts/checkpatch.pl before sending pathes ;-)

> +#endif
> +
>  	/* Setup the default TTY line discipline. */
>  	n_tty_init();

With the above changes:

Reviewed-by: Petr Mladek <pmladek@suse.com>
Tested-by: Petr Mladek <pmladek@suse.com>

There is no need to resend this patch unless there are other comments.
I could make the changes when committing the patch.

Best Regards,
Petr

Re: [PATCH v10 1/1] printk: Add an option to allow ttynull to be a default console device

[Adam Simonelli]

On Friday, March 14, 2025 5:44:43 AM EDT Petr Mladek wrote:
> On Thu 2025-03-13 20:41:04, adamsimonelli@gmail.com wrote:
> > From: Adam Simonelli <adamsimonelli@gmail.com>
> > 
> > The new option is CONFIG_NULL_TTY_DEFAULT_CONSOLE.
> > 
> > if enabled, and CONFIG_VT is disabled, ttynull will become the default
> > primary console device, based on the link order.
> 
> The result is not longer based on the link order.
> 
> The linking order affected the ordering the console initcalls.
> But this patch calls add_preferred_console() directly in console_init()
> before processing the init calls...
> 
> I would just remove the ", based on the link order".
> 
> > ttynull will be the only console device usually with this option enabled.
> > Some architectures do call add_preferred_console() which may add another
> > console though.
> 
> I would add here the following line:
> 
> Motivation:
> 
> to clearly separate the description of the new behavior from the motivation.
> 
> > Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
> > if CONFIG_VT is disabled, the default console device falls back to
> > /dev/ttyS0 instead of /dev/tty.
> > 
> > This could cause issues in user space, and hardware problems:
> > 
> > 1. The user space issues include the case where  /dev/ttyS0 is
> > disconnected, and the TCGETS ioctl, which some user space libraries use
> > as a probe to determine if a file is a tty, is called on /dev/console and
> > fails. Programs that call isatty() on /dev/console and get an incorrect
> > false value may skip expected logging to /dev/console.
> > 
> > 2. The hardware issues include the case if a user has a science instrument
> > or other device connected to the /dev/ttyS0 port, and they were to upgrade
> > to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
> > sent to the device connected to /dev/ttyS0 unless they edit their kernel
> > command line manually.
> > 
> > The new CONFIG_NULL_TTY_DEFAULT_CONSOLE option will give users and
> > distribution maintainers an option to avoid this. Disabling CONFIG_VT and
> > enabling CONFIG_NULL_TTY_DEFAULT_CONSOLE will ensure the default kernel
> > console behavior is not dependant on hardware configuration by default, and
> 
> s/dependant/dependent/
> 
> > avoid unexpected new behavior on devices connected to the /dev/ttyS0 serial
> > port.
> > 
> > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> > index 07668433644b..9dd807717cd4 100644
> > --- a/kernel/printk/printk.c
> > +++ b/kernel/printk/printk.c
> > @@ -4277,6 +4277,11 @@ void __init console_init(void)
> >  	initcall_t call;
> >  	initcall_entry_t *ce;
> >  
> > +#ifdef CONFIG_NULL_TTY_DEFAULT_CONSOLE
> > +       if (!console_set_on_cmdline)
> > +               add_preferred_console("ttynull", 0, NULL);
> 
> checkpatch.pl reports that there are used spaces instead of
> tabs in the two lines above. I suggest to use some editor which takes
> care of proper indentation, e.g. emacs or vim and run
> ./scripts/checkpatch.pl before sending pathes ;-)
> 
> > +#endif
> > +
> >  	/* Setup the default TTY line discipline. */
> >  	n_tty_init();
> 
> With the above changes:
> 
> Reviewed-by: Petr Mladek <pmladek@suse.com>
> Tested-by: Petr Mladek <pmladek@suse.com>
> 
> There is no need to resend this patch unless there are other comments.
> I could make the changes when committing the patch.
> 
> Best Regards,
> Petr
> 
Thanks! I think I will resend though, just so I can get it right

Re: [PATCH v10 1/1] printk: Add an option to allow ttynull to be a default console device

[Adam Simonelli]

On Friday, March 14, 2025 5:44:43 AM EDT Petr Mladek wrote:
> On Thu 2025-03-13 20:41:04, adamsimonelli@gmail.com wrote:
> > From: Adam Simonelli <adamsimonelli@gmail.com>
> > 
> > The new option is CONFIG_NULL_TTY_DEFAULT_CONSOLE.
> > 
> > if enabled, and CONFIG_VT is disabled, ttynull will become the default
> > primary console device, based on the link order.
> 
> The result is not longer based on the link order.
> 
> The linking order affected the ordering the console initcalls.
> But this patch calls add_preferred_console() directly in console_init()
> before processing the init calls...
> 
> I would just remove the ", based on the link order".
> 
> > ttynull will be the only console device usually with this option enabled.
> > Some architectures do call add_preferred_console() which may add another
> > console though.
> 
> I would add here the following line:
> 
> Motivation:
> 
> to clearly separate the description of the new behavior from the motivation.
> 
> > Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
> > if CONFIG_VT is disabled, the default console device falls back to
> > /dev/ttyS0 instead of /dev/tty.
> > 
> > This could cause issues in user space, and hardware problems:
> > 
> > 1. The user space issues include the case where  /dev/ttyS0 is
> > disconnected, and the TCGETS ioctl, which some user space libraries use
> > as a probe to determine if a file is a tty, is called on /dev/console and
> > fails. Programs that call isatty() on /dev/console and get an incorrect
> > false value may skip expected logging to /dev/console.
> > 
> > 2. The hardware issues include the case if a user has a science instrument
> > or other device connected to the /dev/ttyS0 port, and they were to upgrade
> > to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
> > sent to the device connected to /dev/ttyS0 unless they edit their kernel
> > command line manually.
> > 
> > The new CONFIG_NULL_TTY_DEFAULT_CONSOLE option will give users and
> > distribution maintainers an option to avoid this. Disabling CONFIG_VT and
> > enabling CONFIG_NULL_TTY_DEFAULT_CONSOLE will ensure the default kernel
> > console behavior is not dependant on hardware configuration by default, and
> 
> s/dependant/dependent/
> 
> > avoid unexpected new behavior on devices connected to the /dev/ttyS0 serial
> > port.
> > 
> > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> > index 07668433644b..9dd807717cd4 100644
> > --- a/kernel/printk/printk.c
> > +++ b/kernel/printk/printk.c
> > @@ -4277,6 +4277,11 @@ void __init console_init(void)
> >  	initcall_t call;
> >  	initcall_entry_t *ce;
> >  
> > +#ifdef CONFIG_NULL_TTY_DEFAULT_CONSOLE
> > +       if (!console_set_on_cmdline)
> > +               add_preferred_console("ttynull", 0, NULL);
> 
> checkpatch.pl reports that there are used spaces instead of
> tabs in the two lines above. I suggest to use some editor which takes
> care of proper indentation, e.g. emacs or vim and run
> ./scripts/checkpatch.pl before sending pathes ;-)
> 
> > +#endif
> > +
> >  	/* Setup the default TTY line discipline. */
> >  	n_tty_init();
> 
> With the above changes:
> 
> Reviewed-by: Petr Mladek <pmladek@suse.com>
> Tested-by: Petr Mladek <pmladek@suse.com>
> 
> There is no need to resend this patch unless there are other comments.
> I could make the changes when committing the patch.
> 
> Best Regards,
> Petr
> 
Ah, I didn't know if I was allowed to include that myself in the commit message.
Thanks