Re: [PATCH v6 3/3] tty: Change order of ttynull to be linked sooner if enabled as a console.

[Adam Simonelli <adamsimonelli@gmail.com>]

On Tuesday, March 4, 2025 9:06:17 PM EST Adam Simonelli wrote:
> On Tuesday, March 4, 2025 1:51:52 AM EST Andy Shevchenko wrote:
> > On Tue, Mar 4, 2025 at 5:55 AM <adamsimonelli@gmail.com> wrote:
> > >
> > > From: Adam Simonelli <adamsimonelli@gmail.com>
> > >
> > > If CONFIG_NULL_TTY_DEFAULT_CONSOLE is enabled, and CONFIG_VT is disabled,
> > > ttynull will become the default primary console device, based on the link
> > > order.
> > >
> > > Many distributions ship with CONFIG_VT enabled. On tested desktop hardware
> > > if CONFIG_VT is disabled, the default console device falls back to
> > > /dev/ttyS0 instead of /dev/tty.
> > >
> > > This could cause issues in user space, and hardware problems:
> > >
> > > 1. The user space issues include the case where  /dev/ttyS0 is
> > > disconnected, and the TCGETS ioctl, which some user space libraries use
> > > as a probe to determine if a file is a tty, is called on /dev/console and
> > > fails. Programs that call isatty() on /dev/console and get an incorrect
> > > false value may skip expected logging to /dev/console
> > 
> > Missing period at the end.
> > 
> > > 2. The hardware issues include the case if a user has a science instrument
> > > or other device connected to the /dev/ttyS0 port, and they were to upgrade
> > > to a kernel that is disabling the CONFIG_VT option, kernel logs will then be
> > > sent to the device connected to /dev/ttyS0 unless they edit their kernel
> > > command line manually.
> > >
> > > The new CONFIG_NULL_TTY_CONSOLE option will give users and distribution
> > > maintainers an option to avoid this. Disabling CONFIG_VT and enabling
> > > CONFIG_NULL_TTY_CONSOLE will ensure the default kernel console behavior
> > > is not dependant on hardware configuration by default, and avoid
> > > unexpected new behavior on devices connected to the /dev/ttyS0 serial
> > > port.
> > 
> > ...
> > 
> > >  obj-y                          += vt/
> > 
> > + blank line.
> > 
> > > +# If ttynull is configured to be a console by default, ensure that it is linked
> > > +# earlier before a real one is selected.
> > > +obj-$(CONFIG_NULL_TTY_DEFAULT_CONSOLE) \
> > > +                               += ttynull.o
> > 
> > Here is the question: are you sure that all console drivers that exist
> > in the kernel happen to be here? Have you grepped the source tree for
> > checking this?
> > 
> Grepping for console_initcall, the only other places I see outside of
> drivers/tty/ is 
> 
and if I, as a wacky test try 

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 05c5aa951da7..bcd248c44fc8 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -1159,6 +1159,8 @@ void __init setup_arch(char **cmdline_p)
 
        e820__setup_pci_gap();
 
+       add_preferred_console("ttyS", 0, NULL);
+
 #ifdef CONFIG_VT
 #if defined(CONFIG_VGA_CONSOLE)
        if (!efi_enabled(EFI_BOOT) || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY))


just as a test, and do
cat /proc/consoles 
I see
ttynull0             --- (EC    )  242:0
ttyS0                -W- (E  p a)    4:64
where ttynull0 is at the top of the list

> arch/mips/fw/arc/arc_con.c
> arch/mips/sibyte/common/cfe_console.c
> arch/powerpc/kernel/legacy_serial.c
> arch/powerpc/kernel/udbg.c
> arch/powerpc/platforms/powermac/setup.c
> arch/um/drivers/stderr_console.c
> arch/xtensa/platforms/iss/console.c
> drivers/s390/char/con3215.c
> drivers/s390/char/con3270.c
> drivers/s390/char/sclp_con.c
> drivers/s390/char/sclp_vt220.c
> 
> > ...
> > 
> > > +# If ttynull is enabled, but not as a boot console, it is linked and used later
> > > +# after the real ones.
> > > +ifneq ($(CONFIG_NULL_TTY_DEFAULT_CONSOLE),y)
> > 
> > Also can be written as
> > ifeq ($(...),)
> > but it might be less explicit. Up to you.
> > 
> > >  obj-$(CONFIG_NULL_TTY)         += ttynull.o
> > > +endif
> > 
> > 
> 
>