Re: VT-less kernels, and /dev/console on x86

[nerdopolis <bluescreen_avenger@verizon.net>]

On Thursday, August 22, 2024 6:05:19 AM EDT Petr Mladek wrote:
> On Wed 2024-08-21 13:12:03, nerdopolis wrote:
> > On Tuesday, August 20, 2024 9:29:36 AM EDT Petr Mladek wrote:
> > > On Mon 2024-08-19 11:50:39, nerdopolis wrote:
> > > > On Monday, August 19, 2024 11:09:35 AM EDT Steven Rostedt wrote:
> > > > > On Sun, 18 Aug 2024 10:30:22 -0400
> > > > > nerdopolis <bluescreen_avenger@verizon.net> wrote:
> > > > > 
> > > > > > On Sunday, August 18, 2024 8:33:25 AM EDT nerdopolis wrote:
> > > > > > > On Sunday, August 18, 2024 1:12:14 AM EDT Greg KH wrote:  
> > > > > > > > On Sat, Aug 17, 2024 at 08:09:20PM -0400, nerdopolis wrote:  
> > > > > > > > > Hi
> > > > > > > > > 
> > > > > > > > > I originally brought this up on linux-serial, but I think it makes more sense
> > > > > > > > > that it's part of how printk console device selection works. Without VTs, while
> > > > > > > > > most software is able to handle the situation, some userspace programs expect
> > > > > > > > > /dev/console to still be responsive. Namely systemd. It calls isatty() against
> > > > > > > > > /dev/console, and since /dev/console on VT-less systems currently defaults to
> > > > > > > > > /dev/ttyS0, and when /dev/ttyS0 is disconnected, the ioctl's fail, and it
> > > > > > > > > refuses to write log messages to it.
> > > > > > > > > 
> > > > > > > > > There doesn't seem to be a mailing list for printk, so I had to use
> > > > > > > > > get_maintainer.pl. Hopefully this is correct
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > After some grepping and guessing and testing, and playing around Something like
> > > > > > > > > diff --git a/drivers/tty/Kconfig b/drivers/tty/Kconfig
> > > > > > > > > index a45d423ad10f..f94a4632aab0 100644
> > > > > > > > > --- a/drivers/tty/Kconfig
> > > > > > > > > +++ b/drivers/tty/Kconfig
> > > > > > > > > @@ -384,9 +384,12 @@ config NULL_TTY
> > > > > > > > >  
> > > > > > > > >           In order to use this driver, you should redirect the console to this
> > > > > > > > >           TTY, or boot the kernel with console=ttynull.
> > > > > > > > > -
> > > > > > > > >           If unsure, say N.
> > > > > > > > >  
> > > > > > > > > +config NULL_TTY_CONSOLE
> > > > > > > > > +        bool "Supports /dev/ttynull as a console automatically"
> > > > > > > > > +        depends on NULL_TTY && !VT_CONSOLE
> > > > > > > > > +
> > > > > > > > >  config VCC
> > > > > > > > >         tristate "Sun Virtual Console Concentrator"
> > > > > > > > >         depends on SUN_LDOMS
> > > > > > > > > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> > > > > > > > > index dddb15f48d59..c1554a789de8 100644
> > > > > > > > > --- a/kernel/printk/printk.c
> > > > > > > > > +++ b/kernel/printk/printk.c
> > > > > > > > > @@ -3712,6 +3712,11 @@ void __init console_init(void)
> > > > > > > > >         initcall_t call;
> > > > > > > > >         initcall_entry_t *ce;
> > > > > > > > >  
> > > > > > > > > +#ifdef CONFIG_NULL_TTY_CONSOLE
> > > > > > > > > +       if (!strstr(boot_command_line, "console="))
> > > > > > > > > +               add_preferred_console("ttynull", 0, NULL);
> > > > > > > > > +#endif
> > > > > > > > > +
> > > > > > > > >         /* Setup the default TTY line discipline. */
> > > > > > > > >         n_tty_init();
> > > > > > > > >  
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > seems to work, it conflicts with CONFIG_VT_CONSOLE since it is effectively
> > > > > > > > > redundant, it is optional, so that it doesn't cause any changes to
> > > > > > > > > configurations, that historically had CONFIG_VT_CONSOLE turned off in the past,
> > > > > > > > > and for bootloader configs, it won't change any behavior if the kernel command
> > > > > > > > > line has a console device specified  
> > > > > > > > 
> > > > > > > > What is wrong with just setting the kernel command line for this
> > > > > > > > instead?
> > > > > > > >   
> > > > > > > When they eventually start shipping kernels without VTs, they will then have to
> > > > > > > include a script in their upgrade process that runs
> > > > > > > 
> > > > > > > sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT=\"/GRUB_CMDLINE_LINUX_DEFAULT=\"nomodeset /g" /etc/default/grub  
> > > > > > Ugh, I meant
> > > > > > sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT=\"/GRUB_CMDLINE_LINUX_DEFAULT=\"console=ttynull /g" /etc/default/grub
> > > > > > sorry
> > > > > 
> > > > > If you can modify the kernel .config for this, can you just update:
> > > > > 
> > > > >   CONFIG_CMDLINE_BOOL=y
> > > > >   CONFIG_CMDLINE="console=ttynull"
> > > > > 
> > > > > ?
> > > > > 
> > > > That could work, I think. I'll have to see how that works when a different
> > > > console= is specified on the command line from the bootloader though, I am
> > > > thinking that if console=ttyS0 is then manually specified by a user, there will
> > > > be two devices in /proc/consoles (ttyS0 on top of ttynull), but I admit I don't
> > > > know if there are actual ramifications of that, or not...
> > > 
> > > I guess that it would register both consoles in this case.
> > > 
> > > > I am not sure if real distributions would want this to be the answer I guess I
> > > > will have to see if any others are using CONFIG_CMDLINE_BOOL/CONFIG_CMDLINE,
> > > > although this gives me an idea..
> > > > 
> > > > Would something like this below be more acceptable? I didn't test it yet, but
> > > > just the theory. I am thinking that this could have more use to allow a
> > > > preferred to be set...
> > > > 
> > > > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> > > > index dddb15f48d59..c1554a789de8 100644
> > > > --- a/kernel/printk/printk.c
> > > > +++ b/kernel/printk/printk.c
> > > > @@ -3712,6 +3712,11 @@ void __init console_init(void)
> > > >  	initcall_t call;
> > > >  	initcall_entry_t *ce;
> > > >  
> > > > +#ifdef CONFIG_DEFAULT_CONSOLE_HINT_BOOL
> > > > +       if (!strstr(boot_command_line, "console="))
> > > > +               add_preferred_console(CONFIG_DEFAULT_CONSOLE_HINT, 0, NULL);
> > > > +#endif
> > > > +
> > > >  	/* Setup the default TTY line discipline. */
> > > >  	n_tty_init();
> > > 
> > > This is better. But it does not handle some situations. For example,
> > > default console might also by defined by:
> > > 
> > >    + scpr, see acpi_parse_spcr()
> > >    + device tree, see of_console_check()
> > >    + netconsole=, it is hidden in init_netconsole()
> > > 
> > > I tried to handle this another way. The "ttynull" console was
> > > added when /dev/console could not be opened in console_on_rootfs(),
> > > see the commit 757055ae8dedf5333af17b ("init/console: Use ttynull
> > > as a fallback when there is no console").
> > > 
> > > But it did not work well and we had to revert the change, see
> > > the commit a91bd6223ecd46addc71ee6f ("Revert "init/console: Use
> > > ttynull as a fallback when there is no console").
> > > 
> > > Another idea:
> > > 
> > > 1. We could use the same trick as netconsole. I mean to use:
> > > 
> > > 	ttynull_console.flags |= CON_ENABLED;
> > > 
> > >    to force register_console() to register the console even
> > >    when it is not defined in the list of preferred consoles.
> > > 
> > >    It is a kind of hack. But it looks cleaner that adding
> > >    ttynull console into the list of preferred consoles.
> > > 
> > > 
> > > 2. We need to decide whether the fallback to ttynull console
> > >    is needed as late as possible. It should be done after
> > >    all other drivers call register_console().
> > > 
> > >    I would do it in late_initcall_sync().
> > > 
> > > 
> > > 3. We need to detect when the fallback is needed. The check
> > >    of /dev/console does not work, see
> > >    the commit a91bd6223ecd46addc71ee6f ("Revert "init/console: Use
> > >    ttynull as a fallback when there is no console").
> > > 
> > >    A solution might be to check if @console_list is empty.
> > > 
> > > 
> > > Something like (not even compile tested):
> > > 
> > > diff --git a/drivers/tty/ttynull.c b/drivers/tty/ttynull.c
> > > index 6b2f7208b564..7cd7ba2ec33c 100644
> > > --- a/drivers/tty/ttynull.c
> > > +++ b/drivers/tty/ttynull.c
> > > @@ -59,6 +59,16 @@ static struct console ttynull_console = {
> > >  	.device = ttynull_device,
> > >  };
> > >  
> > > +void __init register_ttynull_console_force(void)
> > > +{
> > > +	if (!ttynull_driver)
> > > +		return;
> > > +
> > > +	/* Force registration by setting the CON_ENABLED flag. */
> > > +	ttynull_console.flags |= CON_ENABLED;
> > > +	register_console(&ttynull_console);
> > > +}
> > > +
> > >  static int __init ttynull_init(void)
> > >  {
> > >  	struct tty_driver *driver;
> > > diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> > > index 054c0e7784fd..004612e6fc7f 100644
> > > --- a/kernel/printk/printk.c
> > > +++ b/kernel/printk/printk.c
> > > @@ -3857,6 +3857,25 @@ static int __init printk_late_init(void)
> > >  }
> > >  late_initcall(printk_late_init);
> > >  
> > > +static int __init console_fallback(void)
> > > +{
> > > +	bool need_fallback = false;
> > > +
> > > +	console_list_lock();
> > > +	/*
> > > +	 * Make sure that there is a console which can be associated
> > > +	 * with /dev/console
> > > +	 */
> > > +	if (hlist_empty(&console_list))
> > > +		need_fallback = true;
> > > +
> > > +	console_list_unlock();
> > > +
> > > +	if (need_fallback)
> > > +		register_ttynull_console_force();
> > > +}
> > > +late_initcall_sync(console_fallback);
> > > +
> > >  #if defined CONFIG_PRINTK
> > >  /* If @con is specified, only wait for that console. Otherwise wait for all. */
> > >  static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progress)
> > > 
> > > 
> > I got it to compile with
> > 
> > --- a/include/linux/console.h
> > +++ b/include/linux/console.h
> > @@ -513,9 +513,12 @@ extern int braille_register_console(struct console *, int index,
> >  extern int braille_unregister_console(struct console *);
> >  #ifdef CONFIG_TTY
> >  extern void console_sysfs_notify(void);
> > +extern void register_ttynull_console_force(void);
> >  #else
> >  static inline void console_sysfs_notify(void)
> >  { }
> > +static inline void register_ttynull_console_force(void)
> > +{ }
> >  #endif
> >  extern bool console_suspend_enabled;
> >  
> > and throwing a "return 0;" in console_fallback()
> > 
> > Seems like at least on x86 though, when I tested (CONFIG_VT_CONSOLE disabled),
> > the console is still ttyS0, even if I add "-serial none" to the QEMU VM.
> > 
> > 
> > 
> > I saw 757055ae8dedf5333af17b, but I feel like the intent is different, if I am
> > understanding that correctly, which I very well could not be, to me it looks
> > like it is using ttynull as a _last_ resort when all else fails, but x86 always
> > seems to have /dev/ttyS0's, unless I make changes before the serial consoles
> > are used.
> > 
> > My concerns are more about the distributions that have had /dev/console work
> > no matter what because of CONFIG_VT_CONSOLE. isatty() always returns true 
> > against /dev/console when enabled. When it is disabled, /dev/ttyS0 wins and now
> > software like systemd that logs there is now at the mercy of how /dev/ttyS0 is
> > physically connected or not for it to be able to log successfully.
> 
> I see. I have misunderstood the original problem.
> 
> Hmm, historically, the kernel tries to enable any available console by
> default. It makes some sense. One would expect that when there is a
> device than the user should be able to access it and see the messages there.
> 
> If I get it correctly, you suggest to do not register serial port
> when it is not physically connected. It makes some sense.
> 
Hmm, now that might work, and is a good idea...
> But I think that this should be implemented on the serial console layer.
> It should not call register_console() when the port is not connected.
> 
> Plus, it would need the above patch which would register
> ttynull_console driver as the final fallback. Otherwise, /dev/console
> would not be backed by any device. Or we could update console_device()
> to return the ttynull_driver as a fallback when there is no console
> registered.
> 
> > One of my _other_ ideas was to in setup_arch() in arch/x86/kernel/setup.c add
> > the "add_preferred_console("ttynull", 0, NULL);" when CONFIG_NULL_TTY_CONSOLE
> > is enabled right above where it calls vgacon_register_screen() for this reason.
> > 
> > It worked, but it probably was not correct either...
> 
> IMHO, this is not acceptable for a generic kernel. The kernel should
> try show the messages whenever possible. It should not ignore serial
> consoles when they are available.
> 
Makes sense, on many desktop distros, the logging was not going to physical
hardware because of CONFIG_VT_CONSOLE being enabled, but I understand there are
many different types of embedded devices though.

Which is why my first thought was a new CONFIG_NULL_TTY_CONSOLE option that
would be disabled by default, but when _enabled_ (Trading CONFIG_VT_CONSOLE for
it), the behaviour could be the same on the serial ports,at least on x86, when it adds ttynull as a preferred console...

like:
#if defined(CONFIG_NULL_TTY_CONSOLE)
	if (!efi_enabled(EFI_BOOT) || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY))
		add_preferred_console("ttynull", 0, NULL);
#endif

in arch/x86/kernel/setup.c to mimic I _think_ about where the VT console would
have been registered, and in the same conditions...

(this would only work in x86 though...)
> Also I am sure that the kernel is used on devices which only
> have a serial console. Such a change would cause regression
> there.
> 
> Best Regards,
> Petr
>