[PATCH v4 00/11] printk: Clean up preferred console handling

[Petr Mladek <pmladek@suse.com>]

this patchset does some clean up of the code for handling preferred
consoles in the console registration code.

  + 1st and 2nd patch try to improve some naming.

  + 3rd patch removes some code duplication. It better defines and
    describes the rules for adding and updating preferred consoles.
    It uses a more defensive coding style.

  + 4-8th patch improve handling of Braille consoles.
    They are preferred via the command line but they do not get
    printk() messages and are not associated with /dev/console.

    The new code makes this more obvious. Also it explicitly
    defines the relation against default consoles and other
    non-Braille preferred consoles.

  + 9th patch removes a hidden side effect of
    try_enable_preferred_console()

  + 10th patch prevents adding more side effectes to
    try_enable_preferred_console(). It allows to call it
    only when there are any non-Braille preferred consoles.

  + 11th patch allows to catch an error when enabling a console
    driver by default.

This patchset is a prerequisite for Marcos' clean up of CON_ENABLE
flag handling. It should prevent regressions caused by the
hidden effects of try_enable_preferred_console(), for example,
see https://lore.kernel.org/r/89409a0f48e6998ff6dd2245691b9954f0e1e435.camel@suse.com

Also I am working on a feature which would allow to explicitly
enable/prefer consoles proposed by SPCR, device tree, or
platform-specific code using a generic "console=platform".
This clean up is a prerequisite, see
https://github.com/pmladek/linux/tree/console-platform-poc1-iter9

Changes against [v3]:

  + Move the handling of pre-enabled consoles into register_console()
    as 1st patch. It prevents a possible invalid access. [Petr]

  + Move the code for handling pre-enabled to console to
    try_enable_console() in 7th patch. The resulting code
    is the same as in v3.

    This 2nd move was not needed in v3 because the code
    was moved to this final location later. [Petr]

  + Switch order of the 10th and 11th patch. Aka make
    try_enable_default_console() return an err before
    refactoring try_enable_console() call
    try_enable_preferred_console() only when there is any.
    [Petr]

  + Initialize err value in try_enable_console() in 11th patch.
    It prevents using a non-initialized variable. [AI:gemini-3]

  + Make sure that pre-enabled consoles won't get registered
    when something went wrong in try_enable_console()
    in 11th patch [Petr]


Changes against [v2]:

  + Move static_assert() after declaration (1st patch) [Steven]

  + Rename update_preferred_console() to add_or_update_preferred_console() [John]
  + WARN_ON() when called without "name" and "devname" [John]
  + print error when reached MAX_PREFERRED_CONSOLES
  + Add comment above code adding new entries [John]
    (all in 4th patch)

  + Call console->setup() under console_lock() in braille_register_console()
    (new patch) [Sashiko]

  + Use "No functional changes." phrase in commit messages. [Steven]

  + Call try_enable_preferred_console() only when there is no
    preferred console (reworked and split 9th patch) [Petr]

  + Added taken Reviewed-by and Acked-by tags [John, Macros]


Changes against [v1]:

  + Fix stale comment (1st patch, Chris)

  + Add comment descibing update_preferred_console() behavior (3rd patch, John)

  + Better describe behavior changes caused by update_preferred_console()
    in the commit message (3rd patch, Chris)

  + Use unsigned int for iterating/indexing preferred_consoles[]
    table (3rd patch, John)

  + Add a check preventing an update via "devname" is already assigned
    to a real "name". (3rd patch, Petr)

  + Use only Braille entries when trying to enable Braille console (Chris)

  + Split all try_enable_*console() calls into new try_enable_console()
    and return early when try_enable_braille_console() succeeded.
    Note that we do not longer need to initialize "err" variable
    in this case. (new 5th patch, Chris, Petr)

  + Put back the explicit check of CON_BRL flag in register_console()
    so that it returns prelimitary only when the driver is used in Braille
    mode. (6th patch, Chris, Petr)

  + Add comments explaining that try_enable_braille_console() might succeeded
    also when newcon->match() succeeded. In this case, the Braille mode can't
    be used because the console device was already used as an early console.[*]
    (6th patch, Chris, Petr)

  + Reset preferred_dev_console_prev when used to revert the value.
    (7th patch, John)

  + Add Acked-by and Reviewed-by tags (1st, 2nd, 4th patch, Chris, Marcos)

[*] This is a questional behavior. I would personally prefer when
    the early console is unregistered at this point and the console
    device is converted into Braille console. I consider the Braille
    mode more important. But it might be changed/fixed later
    in another patchset.


[v1] https://lore.kernel.org/r/20260206165002.496724-1-pmladek@suse.com
[v2] https://lore.kernel.org/r/20260423130015.85175-1-pmladek@suse.com
[v3] https://lore.kernel.org/r/20260602085312.228251-1-pmladek@suse.com

Petr Mladek (11):
  printk: Handle pre-enabled consoles in the top-level
    register_console()
  printk: Rename struct console_cmdline to preferred_console
  printk: Rename preferred_console to preferred_dev_console
  printk: Separate code for adding/updating preferred console metadata
  printk: Cleanup _braille_(un)register_console() wrappers
  console/braille: Lock console->setup() call during the registration
  printk: Separate code for enabling console
  printk: Try to register each console as Braille first
  printk: Do not set Braille console as preferred_console
  printk: Modify try_enable_default_console() to return error/success
  printk: Try enable preferred consoles only when there are any

 .../accessibility/braille/braille_console.c   |   9 +-
 kernel/printk/braille.c                       |  20 +-
 kernel/printk/braille.h                       |  23 +-
 .../{console_cmdline.h => console_register.h} |   6 +-
 kernel/printk/printk.c                        | 402 ++++++++++++------
 5 files changed, 315 insertions(+), 145 deletions(-)
 rename kernel/printk/{console_cmdline.h => console_register.h} (83%)

-- 
2.54.0