[PATCH v8 0/4] Optionally allow ttynull to be selected as a default console

[PATCH v8 0/4] Optionally allow ttynull to be selected as a default console

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

When switching to a CONFIG_VT=n world, at least on x86 systems,
/dev/console becomes /dev/ttyS0. This can cause some undesired effects.
/dev/console's behavior is now tied to the physical /dev/ttyS0, which when
disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
and users who upgrade to a theoretical vt-less kernel from their
distribution who have a device such as a science instrument connected to
their /dev/ttyS0 port will suddenly see it receive kernel log messages.

When the new CONFIG_NULL_TTY_DEFAULT_CONSOLE option is turned on, this will
allow the ttynull device to be leveraged as the default console. Distributions
that had CONFIG_VT turned on before will be able to leverage this option
to where /dev/console is still backed by a psuedo device, avoiding these
issues, without needing to enable the entire VT subsystem.

v2:
    rebase

v3:
    Clarify commit messages.

    Guard the all the register_console()s in ttynull to prevent it from being
    registered twice.

    Only change the link order if CONFIG_NULL_TTY_CONSOLE is enabled, otherwise
    use the existing order for ttynull if only CONFIG_NULL_TTY is enabled.

    Document why the link order changes in the drivers/tty/Makefile file.

    Replace #ifdefs

v4:
    Remember to actually include the changes to v3 in the cover letter.

v5:
    Correct code formatting in Makefile comment.

v6:
    Change to CONFIG_NULL_TTY_DEFAULT_CONSOLE

    Set the index to -1, and don't set the flags

    Use add_preferred_console() instead of register_console() in ttynull's
    console_initcall when CONFIG_NULL_TTY_DEFAULT_CONSOLE is enabled

v7
    Add a commit message to the first commit, and the Suggested-by

    Correct Kconfig tabs/spaces formatting

    Invert the console_set_on_cmdline check

v8
    Update Documentation.

    Remove the "CONFIG_" in the help text when mentioning other configuration
    options.

    Update commit message

Adam Simonelli (4):
  ttynull: Always initialize console index to -1
  ttynull: Add an option to allow ttynull to be used as a console device
  tty: Change order of ttynull to be linked sooner if enabled as a
    console.
  Documentation/serial-console: Document
    CONFIG_NULL_TTY_DEFAULT_CONSOLE.

 Documentation/admin-guide/serial-console.rst |  4 +++-
 drivers/tty/Kconfig                          | 15 ++++++++++++++-
 drivers/tty/Makefile                         | 12 ++++++++++++
 drivers/tty/ttynull.c                        | 14 ++++++++++++++
 4 files changed, 43 insertions(+), 2 deletions(-)

-- 
2.45.2

[PATCH v8 1/4] ttynull: Always initialize console index to -1

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

This allows ttynull to be considered in console selection.

Suggested-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
---
 drivers/tty/ttynull.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/tty/ttynull.c b/drivers/tty/ttynull.c
index 6b2f7208b564..d8b51edde628 100644
--- a/drivers/tty/ttynull.c
+++ b/drivers/tty/ttynull.c
@@ -57,6 +57,7 @@ static struct tty_driver *ttynull_device(struct console *c, int *index)
 static struct console ttynull_console = {
 	.name = "ttynull",
 	.device = ttynull_device,
+	.index = -1,
 };
 
 static int __init ttynull_init(void)
-- 
2.45.2

[PATCH v8 2/4] ttynull: Add an option to allow ttynull to be used as a console device

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

The new config option, CONFIG_NULL_TTY_DEFAULT_CONSOLE will allow
ttynull to be initialized by console_initcall() and selected as a
possible console device.

Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
---
 drivers/tty/Kconfig   | 15 ++++++++++++++-
 drivers/tty/ttynull.c | 13 +++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/drivers/tty/Kconfig b/drivers/tty/Kconfig
index 63a494d36a1f..edf557c73586 100644
--- a/drivers/tty/Kconfig
+++ b/drivers/tty/Kconfig
@@ -383,7 +383,20 @@ config NULL_TTY
 	  available or desired.
 
 	  In order to use this driver, you should redirect the console to this
-	  TTY, or boot the kernel with console=ttynull.
+	  TTY, boot the kernel with console=ttynull, or enable
+	  NULL_TTY_DEFAULT_CONSOLE.
+
+	  If unsure, say N.
+
+config NULL_TTY_DEFAULT_CONSOLE
+	bool "Support for console on ttynull"
+	depends on NULL_TTY=y && !VT_CONSOLE
+	help
+	  Say Y here if you want the NULL TTY to be used as a /dev/console
+	  device.
+
+	  This is similar to VT_CONSOLE, but without the dependency on VT.
+	  It uses the ttynull driver as the system console.
 
 	  If unsure, say N.
 
diff --git a/drivers/tty/ttynull.c b/drivers/tty/ttynull.c
index d8b51edde628..67aad3e6f538 100644
--- a/drivers/tty/ttynull.c
+++ b/drivers/tty/ttynull.c
@@ -96,6 +96,19 @@ static int __init ttynull_init(void)
 	return 0;
 }
 
+#ifdef CONFIG_NULL_TTY_DEFAULT_CONSOLE
+static int __init ttynull_register(void)
+{
+	if (console_set_on_cmdline)
+		return 0;
+
+	add_preferred_console("ttynull", 0, NULL);
+
+	return 0;
+}
+console_initcall(ttynull_register);
+#endif
+
 static void __exit ttynull_exit(void)
 {
 	unregister_console(&ttynull_console);
-- 
2.45.2

[PATCH v8 3/4] tty: Change order of ttynull to be linked sooner if enabled as a console.

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

If CONFIG_NULL_TTY_DEFAULT_CONSOLE is enabled, and CONFIG_VT is disabled,
ttynull will become the default primary console device, based on the link
order.

ttynull will be the only console device usually with this option enabled.
Some architectures do call add_preferred_console() which may add another
console though.

Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
if CONFIG_VT is disabled, the default console device falls back to
/dev/ttyS0 instead of /dev/tty.

This could cause issues in user space, and hardware problems:

1. The user space issues include the case where  /dev/ttyS0 is
disconnected, and the TCGETS ioctl, which some user space libraries use
as a probe to determine if a file is a tty, is called on /dev/console and
fails. Programs that call isatty() on /dev/console and get an incorrect
false value may skip expected logging to /dev/console.

2. The hardware issues include the case if a user has a science instrument
or other device connected to the /dev/ttyS0 port, and they were to upgrade
to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
sent to the device connected to /dev/ttyS0 unless they edit their kernel
command line manually.

The new CONFIG_NULL_TTY_CONSOLE option will give users and distribution
maintainers an option to avoid this. Disabling CONFIG_VT and enabling
CONFIG_NULL_TTY_CONSOLE will ensure the default kernel console behavior
is not dependant on hardware configuration by default, and avoid
unexpected new behavior on devices connected to the /dev/ttyS0 serial
port.

Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
---
 drivers/tty/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/tty/Makefile b/drivers/tty/Makefile
index 07aca5184a55..7d0414dc31ed 100644
--- a/drivers/tty/Makefile
+++ b/drivers/tty/Makefile
@@ -11,6 +11,12 @@ obj-$(CONFIG_N_HDLC)		+= n_hdlc.o
 obj-$(CONFIG_N_GSM)		+= n_gsm.o
 
 obj-y				+= vt/
+
+# If ttynull is configured to be a console by default, ensure that it is linked
+# earlier before a real one is selected.
+obj-$(CONFIG_NULL_TTY_DEFAULT_CONSOLE) \
+				+= ttynull.o
+
 obj-$(CONFIG_HVC_DRIVER)	+= hvc/
 obj-y				+= serial/
 obj-$(CONFIG_SERIAL_DEV_BUS)	+= serdev/
@@ -20,7 +26,13 @@ obj-$(CONFIG_AMIGA_BUILTIN_SERIAL) += amiserial.o
 obj-$(CONFIG_MOXA_INTELLIO)	+= moxa.o
 obj-$(CONFIG_MOXA_SMARTIO)	+= mxser.o
 obj-$(CONFIG_NOZOMI)		+= nozomi.o
+
+# If ttynull is enabled, but not as a boot console, it is linked and used later
+# after the real ones.
+ifneq ($(CONFIG_NULL_TTY_DEFAULT_CONSOLE),y)
 obj-$(CONFIG_NULL_TTY)	        += ttynull.o
+endif
+
 obj-$(CONFIG_SYNCLINK_GT)	+= synclink_gt.o
 obj-$(CONFIG_PPC_EPAPR_HV_BYTECHAN) += ehv_bytechan.o
 obj-$(CONFIG_GOLDFISH_TTY)	+= goldfish.o
-- 
2.45.2

[PATCH v8 4/4] Documentation/serial-console: Document CONFIG_NULL_TTY_DEFAULT_CONSOLE.

[adamsimonelli]

From: Adam Simonelli <adamsimonelli@gmail.com>

When the kernel is compiled with CONFIG_NULL_TTY_DEFAULT_CONSOLE and
no console= options are selected, it defaults to using the ttynull
device.
---
 Documentation/admin-guide/serial-console.rst | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/serial-console.rst b/Documentation/admin-guide/serial-console.rst
index a3dfc2c66e01..1609e7479249 100644
--- a/Documentation/admin-guide/serial-console.rst
+++ b/Documentation/admin-guide/serial-console.rst
@@ -78,7 +78,9 @@ If no console device is specified, the first device found capable of
 acting as a system console will be used. At this time, the system
 first looks for a VGA card and then for a serial port. So if you don't
 have a VGA card in your system the first serial port will automatically
-become the console.
+become the console, unless the kernel is configured with the
+CONFIG_NULL_TTY_DEFAULT_CONSOLE option, then it will default to using the
+ttynull device.
 
 You will need to create a new device to use ``/dev/console``. The official
 ``/dev/console`` is now character device 5,1.
-- 
2.45.2

Re: [PATCH v8 0/4] Optionally allow ttynull to be selected as a default console

[Andy Shevchenko]

On Tue, Mar 11, 2025 at 5:31 AM <adamsimonelli@gmail.com> wrote:
>
> From: Adam Simonelli <adamsimonelli@gmail.com>
>
> When switching to a CONFIG_VT=n world, at least on x86 systems,
> /dev/console becomes /dev/ttyS0. This can cause some undesired effects.
> /dev/console's behavior is now tied to the physical /dev/ttyS0, which when
> disconnected can cause isatty() to fail when /dev/ttyS0 is disconnected,
> and users who upgrade to a theoretical vt-less kernel from their
> distribution who have a device such as a science instrument connected to
> their /dev/ttyS0 port will suddenly see it receive kernel log messages.
>
> When the new CONFIG_NULL_TTY_DEFAULT_CONSOLE option is turned on, this will
> allow the ttynull device to be leveraged as the default console. Distributions
> that had CONFIG_VT turned on before will be able to leverage this option
> to where /dev/console is still backed by a psuedo device, avoiding these

pseudo

> issues, without needing to enable the entire VT subsystem.

LGTM, so FWIW,
Acked-by: Andy Shevchenko <andy@kernel.org>

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v8 1/4] ttynull: Always initialize console index to -1

[Petr Mladek]

On Mon 2025-03-10 23:31:30, adamsimonelli@gmail.com wrote:
> From: Adam Simonelli <adamsimonelli@gmail.com>
> 
> This allows ttynull to be considered in console selection.

This is not true. It should be possible to register ttynull even
when .index == 0.

The .index is important only for drivers which support more devices,
e.g. the serial port or virtual terminal.

> 
> Suggested-by: Petr Mladek <pmladek@suse.com>
> Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
> ---
>  drivers/tty/ttynull.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/tty/ttynull.c b/drivers/tty/ttynull.c
> index 6b2f7208b564..d8b51edde628 100644
> --- a/drivers/tty/ttynull.c
> +++ b/drivers/tty/ttynull.c
> @@ -57,6 +57,7 @@ static struct tty_driver *ttynull_device(struct console *c, int *index)
>  static struct console ttynull_console = {
>  	.name = "ttynull",
>  	.device = ttynull_device,
> +	.index = -1,

There is only one "/dev/ttynull". And its index is initialized to "0".
At least it seems to be the last parameter in:

static int __init ttynull_init(void)
{
[...]
	tty_port_link_device(&ttynull_port, driver, 0);
[...]
}

So, I believe this it should be perfectly fine to keep the default "0"
here. Note that it is special for ttynull because it is only one...

IMHO, this patch adds more harm than good :-)

Best Regards,
Petr

Re: [PATCH v8 2/4] ttynull: Add an option to allow ttynull to be used as a console device

[Petr Mladek]

On Mon 2025-03-10 23:31:31, adamsimonelli@gmail.com wrote:
> From: Adam Simonelli <adamsimonelli@gmail.com>
> 
> The new config option, CONFIG_NULL_TTY_DEFAULT_CONSOLE will allow
> ttynull to be initialized by console_initcall() and selected as a
> possible console device.

It would be great to mention here the motivation. It is nicely
explained in the 3rd patch.

> --- a/drivers/tty/Kconfig
> +++ b/drivers/tty/Kconfig
> @@ -383,7 +383,20 @@ config NULL_TTY
>  	  available or desired.
>  
>  	  In order to use this driver, you should redirect the console to this
> -	  TTY, or boot the kernel with console=ttynull.
> +	  TTY, boot the kernel with console=ttynull, or enable
> +	  NULL_TTY_DEFAULT_CONSOLE.
> +
> +	  If unsure, say N.
> +
> +config NULL_TTY_DEFAULT_CONSOLE
> +	bool "Support for console on ttynull"
> +	depends on NULL_TTY=y && !VT_CONSOLE
> +	help
> +	  Say Y here if you want the NULL TTY to be used as a /dev/console
> +	  device.

I would provide few more details, something like:

<proposal>
	  Say Y here if you want the NULL TTY to be used as a /dev/console
	  device by default.

	  For example, it might be useful to prevent a VT-less kernel from
	  writing the system log to a random device connected to
	  the serial port.

	  Another console driver still might get preferred via the command line,
	  SPCR, or the device tree.
</proposal>

> +	  This is similar to VT_CONSOLE, but without the dependency on VT.
> +	  It uses the ttynull driver as the system console.

Honestly, I do not find this paragraph much useful. I would omit it,

>  	  If unsure, say N.
>  
> diff --git a/drivers/tty/ttynull.c b/drivers/tty/ttynull.c
> index d8b51edde628..67aad3e6f538 100644
> --- a/drivers/tty/ttynull.c
> +++ b/drivers/tty/ttynull.c
> @@ -96,6 +96,19 @@ static int __init ttynull_init(void)
>  	return 0;
>  }
>  
> +#ifdef CONFIG_NULL_TTY_DEFAULT_CONSOLE
> +static int __init ttynull_register(void)
> +{
> +	if (console_set_on_cmdline)
> +		return 0;
> +
> +	add_preferred_console("ttynull", 0, NULL);
> +
> +	return 0;
> +}
> +console_initcall(ttynull_register);
> +#endif

I have realized that this does not work without shuffling the linking
order (3rd patch).

I would prefer to avoid the linking order hack and rather call this
in console_init() directly, see
https://lore.kernel.org/r/Z9GPVVTnngGbmbuv@pathway.suse.cz
for more details.

Best Regards,
Petr

Re: [PATCH v8 4/4] Documentation/serial-console: Document CONFIG_NULL_TTY_DEFAULT_CONSOLE.

[Petr Mladek]

On Mon 2025-03-10 23:31:33, adamsimonelli@gmail.com wrote:
> From: Adam Simonelli <adamsimonelli@gmail.com>
> 
> When the kernel is compiled with CONFIG_NULL_TTY_DEFAULT_CONSOLE and
> no console= options are selected, it defaults to using the ttynull
> device.

There is a missing Signed-off-by.

The change looks fine to me. Feel free to use:

Reviewed-by: Petr Mladek <pmladek@suse.com>

That said, I would personally squash this into the patch adding
the CONFIG_NULL_TTY_DEFAULT_CONSOLE configure option.

Best Regards,
Petr

Re: [PATCH v8 1/4] ttynull: Always initialize console index to -1

[Adam Simonelli]

On Wednesday, March 12, 2025 10:38:59 AM EDT Petr Mladek wrote:
> On Mon 2025-03-10 23:31:30, adamsimonelli@gmail.com wrote:
> > From: Adam Simonelli <adamsimonelli@gmail.com>
> > 
> > This allows ttynull to be considered in console selection.
> 
> This is not true. It should be possible to register ttynull even
> when .index == 0.
> 
> The .index is important only for drivers which support more devices,
> e.g. the serial port or virtual terminal.
> 
> > 
> > Suggested-by: Petr Mladek <pmladek@suse.com>
> > Signed-off-by: Adam Simonelli <adamsimonelli@gmail.com>
> > ---
> >  drivers/tty/ttynull.c | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/drivers/tty/ttynull.c b/drivers/tty/ttynull.c
> > index 6b2f7208b564..d8b51edde628 100644
> > --- a/drivers/tty/ttynull.c
> > +++ b/drivers/tty/ttynull.c
> > @@ -57,6 +57,7 @@ static struct tty_driver *ttynull_device(struct console *c, int *index)
> >  static struct console ttynull_console = {
> >  	.name = "ttynull",
> >  	.device = ttynull_device,
> > +	.index = -1,
> 
> There is only one "/dev/ttynull". And its index is initialized to "0".
> At least it seems to be the last parameter in:
> 
> static int __init ttynull_init(void)
> {
> [...]
> 	tty_port_link_device(&ttynull_port, driver, 0);
> [...]
> }
> 
> So, I believe this it should be perfectly fine to keep the default "0"
> here. Note that it is special for ttynull because it is only one...
> 
> IMHO, this patch adds more harm than good :-)
> 
> Best Regards,
> Petr
> 
Understood, thanks for the explanation about that.