[PATCH printk v2 00/38] reduce console_lock scope

[PATCH printk v2 00/38] reduce console_lock scope

[John Ogness]

This is v2 of a series to prepare for threaded/atomic
printing. It is a rework of patches 6-12 of the v1 [0]. From
the v1, patches 1-5 are already mainline and a rework of
patches >12 will be posted in a later series.

This series focuses on reducing the scope of the BKL
console_lock. It achieves this by switching to SRCU and a
dedicated mutex for console list iteration and modification,
respectively. The console_lock will no longer offer this
protection and is completely removed from
(un)register_console() and console_stop/start() code.

All users of the console_lock for list iteration have been
modified. For the call sites where the console_lock is still
needed (because of other reasons), I added comments to explain
exactly why the console_lock was needed.

The base commit for this series is from Paul McKenney's RCU tree
and provides an NMI-safe SRCU implementation [1]. Without the
NMI-safe SRCU implementation, this series is not less safe than
mainline. But we will need the NMI-safe SRCU implementation for
atomic consoles anyway, so we might as well get it in
now. Especially since it _does_ increase the reliability for
mainline in the panic path.

Changes since v2:

general:

- introduce console_is_enabled() to document safe data race on
  console->flags

- switch all "console->flags & CON_ENABLED" code sites to
  console_is_enabled()

- add "for_each_console_srcu" to .clang-format

- cleanup/clarify comments relating to console_lock
  coverage/usage

um:

- kmsg_dumper: use srcu instead of console_lock for list
  iteration

kgdb/kdb:

- configure_kgdboc: keep console_lock for console->device()
  synchronization, use srcu for list iteration

- kgdboc_earlycon_pre_exp_handler: use srcu instead of
  documenting unsafety for list iteration

- kgdboc_earlycon_init: use console_list_lock instead of
  console_lock to lock list

- kdb_msg_write: use srcu instead of documenting unsafety for
  list iteration

tty:

- show_cons_active: keep console_lock for console->device()
  synchronization

fbdev:

- xen-fbfront: xenfb_probe: use srcu instead of console_lock
  for list iteration, introduce console_force_preferred() to
  safely implement hack

proc/consoles:

- show_console_dev: keep console_lock for console->device()
  synchronization

- c_next: use hlist_entry_safe() instead of
  hlist_for_each_entry_continue()

printk:

- remove console_lock from console_stop/start() and
  (un)register_console()

- introduce console_srcu_read_(un)lock() to wrap scru read
  (un)lock

- rename cons_first() macro to console_first()

- for_each_console: add lockdep check instead of introducing
  new for_each_registered_console()

- console_list_lock: add warning if in read-side critical
  section

- release srcu read lock on handover

- console_flush_all: use srcu instead of relying on console
  lock for list iteration

- console_unblank: use srcu instead of relying on console_lock
  for list iteration

- console_flush_on_panic: use srcu for list iteration and
  document console->seq race

- device: keep console_lock for console->device()
  synchronization, usr srcu for list iteration

- register_console: split list adding logic into the 3 distinct
  scenarios

- register_console: set initial sequence number before adding
  to list

- unregister_console: fix ENODEV return value if the console is
  not registered

- console_stop: synchronize srcu

- printk_late_init: use _safe variant of iteration

- __pr_flush: use srcu instead of relying on console_lock for
  list iteration

John Ogness

[0] https://lore.kernel.org/r/20220924000454.3319186-1-john.ogness@linutronix.de
[1] https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/log/?h=srcunmisafe.2022.10.18b

John Ogness (37):
  printk: Convert console_drivers list to hlist
  printk: Prepare for SRCU console list protection
  printk: introduce console_is_enabled() wrapper
  printk: use console_is_enabled()
  tty: nfcon: use console_is_enabled()
  um: kmsg_dump: use console_is_enabled()
  efi: earlycon: use console_is_enabled()
  netconsole: use console_is_enabled()
  tty: hvc: use console_is_enabled()
  tty: serial: earlycon: use console_is_enabled()
  tty: serial: kgdboc: use console_is_enabled()
  tty: serial: pic32_uart: use console_is_enabled()
  tty: serial: samsung_tty: use console_is_enabled()
  tty: serial: serial_core: use console_is_enabled()
  tty: serial: xilinx_uartps: use console_is_enabled()
  tty: tty_io: use console_is_enabled()
  usb: early: xhci-dbc: use console_is_enabled()
  kdb: kdb_io: use console_is_enabled()
  um: kmsg_dumper: use srcu console list iterator
  serial: kgdboc: use srcu console list iterator
  serial: kgdboc: document console_lock usage
  tty: tty_io: document console_lock usage
  xen: fbfront: use srcu console list iterator
  proc: consoles: document console_lock usage
  kdb: use srcu console list iterator
  printk: console_flush_all: use srcu console list iterator
  printk: console_unblank: use srcu console list iterator
  printk: console_flush_on_panic: use srcu console list iterator
  printk: console_device: use srcu console list iterator
  printk: register_console: use srcu console list iterator
  printk: __pr_flush: use srcu console list iterator
  printk: introduce console_list_lock
  serial: kgdboc: use console_list_lock instead of console_lock
  tty: tty_io: use console_list_lock for list synchronization
  proc: consoles: use console_list_lock for list iteration
  printk: relieve console_lock of list synchronization duties
  printk, xen: fbfront: create/use safe function for forcing preferred

Thomas Gleixner (1):
  serial: kgdboc: Lock console list in probe function

 .clang-format                      |   1 +
 arch/m68k/emu/nfcon.c              |   4 +-
 arch/um/kernel/kmsg_dump.c         |  15 +-
 drivers/firmware/efi/earlycon.c    |   4 +-
 drivers/net/netconsole.c           |   4 +-
 drivers/tty/hvc/hvc_console.c      |   2 +-
 drivers/tty/serial/earlycon.c      |   4 +-
 drivers/tty/serial/kgdboc.c        |  37 ++-
 drivers/tty/serial/pic32_uart.c    |   2 +-
 drivers/tty/serial/samsung_tty.c   |   2 +-
 drivers/tty/serial/serial_core.c   |   2 +-
 drivers/tty/serial/xilinx_uartps.c |   2 +-
 drivers/tty/tty_io.c               |  18 +-
 drivers/usb/early/xhci-dbc.c       |   2 +-
 drivers/video/fbdev/xen-fbfront.c  |  16 +-
 fs/proc/consoles.c                 |  20 +-
 include/linux/console.h            |  75 +++++-
 include/linux/serial_core.h        |   2 +-
 kernel/debug/kdb/kdb_io.c          |   7 +-
 kernel/printk/printk.c             | 373 +++++++++++++++++++++--------
 20 files changed, 438 insertions(+), 154 deletions(-)


base-commit: c2d158a284abd63d727dad7402a2eed650dd4233
-- 
2.30.2

[PATCH printk v2 02/38] printk: Convert console_drivers list to hlist

[John Ogness]

Replace the open coded single linked list with a hlist so a conversion
to SRCU protected list walks can reuse the existing primitives.

Signed-off-by: John Ogness <john.ogness@linutronix.de>
---
 fs/proc/consoles.c      |  3 +-
 include/linux/console.h |  8 ++--
 kernel/printk/printk.c  | 99 +++++++++++++++++++++++------------------
 3 files changed, 63 insertions(+), 47 deletions(-)

diff --git a/fs/proc/consoles.c b/fs/proc/consoles.c
index dfe6ce3505ce..cf2e0788f9c7 100644
--- a/fs/proc/consoles.c
+++ b/fs/proc/consoles.c
@@ -74,8 +74,9 @@ static void *c_start(struct seq_file *m, loff_t *pos)
 static void *c_next(struct seq_file *m, void *v, loff_t *pos)
 {
 	struct console *con = v;
+
 	++*pos;
-	return con->next;
+	return hlist_entry_safe(con->node.next, struct console, node);
 }
 
 static void c_stop(struct seq_file *m, void *v)
diff --git a/include/linux/console.h b/include/linux/console.h
index 8c1686e2c233..7b5f21f9e469 100644
--- a/include/linux/console.h
+++ b/include/linux/console.h
@@ -15,6 +15,7 @@
 #define _LINUX_CONSOLE_H_ 1
 
 #include <linux/atomic.h>
+#include <linux/list.h>
 #include <linux/types.h>
 
 struct vc_data;
@@ -154,14 +155,16 @@ struct console {
 	u64	seq;
 	unsigned long dropped;
 	void	*data;
-	struct	 console *next;
+	struct hlist_node node;
 };
 
+extern struct hlist_head console_list;
+
 /*
  * for_each_console() allows you to iterate on each console
  */
 #define for_each_console(con) \
-	for (con = console_drivers; con != NULL; con = con->next)
+	hlist_for_each_entry(con, &console_list, node)
 
 extern int console_set_on_cmdline;
 extern struct console *early_console;
@@ -174,7 +177,6 @@ enum con_flush_mode {
 extern int add_preferred_console(char *name, int idx, char *options);
 extern void register_console(struct console *);
 extern int unregister_console(struct console *);
-extern struct console *console_drivers;
 extern void console_lock(void);
 extern int console_trylock(void);
 extern void console_unlock(void);
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index e4f1e7478b52..867becc40021 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -79,13 +79,12 @@ int oops_in_progress;
 EXPORT_SYMBOL(oops_in_progress);
 
 /*
- * console_sem protects the console_drivers list, and also
- * provides serialisation for access to the entire console
- * driver system.
+ * console_sem protects console_list and console->flags updates, and also
+ * provides serialization for access to the entire console driver system.
  */
 static DEFINE_SEMAPHORE(console_sem);
-struct console *console_drivers;
-EXPORT_SYMBOL_GPL(console_drivers);
+HLIST_HEAD(console_list);
+EXPORT_SYMBOL_GPL(console_list);
 
 /*
  * System may need to suppress printk message under certain
@@ -2556,7 +2555,7 @@ static int console_cpu_notify(unsigned int cpu)
  * console_lock - lock the console system for exclusive use.
  *
  * Acquires a lock which guarantees that the caller has
- * exclusive access to the console system and the console_drivers list.
+ * exclusive access to the console system and console_list.
  *
  * Can sleep, returns nothing.
  */
@@ -2576,7 +2575,7 @@ EXPORT_SYMBOL(console_lock);
  * console_trylock - try to lock the console system for exclusive use.
  *
  * Try to acquire a lock which guarantees that the caller has exclusive
- * access to the console system and the console_drivers list.
+ * access to the console system and console_list.
  *
  * returns 1 on success, and 0 on failure to acquire the lock.
  */
@@ -2940,11 +2939,20 @@ void console_flush_on_panic(enum con_flush_mode mode)
 	console_may_schedule = 0;
 
 	if (mode == CONSOLE_REPLAY_ALL) {
+		struct hlist_node *tmp;
 		struct console *c;
 		u64 seq;
 
 		seq = prb_first_valid_seq(prb);
-		for_each_console(c)
+		/*
+		 * This cannot use for_each_console() because it's not established
+		 * that the current context has console locked and neither there is
+		 * a guarantee that there is no concurrency in that case.
+		 *
+		 * Open code it for documentation purposes and pretend that
+		 * it works.
+		 */
+		hlist_for_each_entry_safe(c, tmp, &console_list, node)
 			c->seq = seq;
 	}
 	console_unlock();
@@ -3081,6 +3089,9 @@ static void try_enable_default_console(struct console *newcon)
 	       (con->flags & CON_BOOT) ? "boot" : "",	\
 	       con->name, con->index, ##__VA_ARGS__)
 
+#define console_first()				\
+	hlist_entry(console_list.first, struct console, node)
+
 /*
  * The console driver calls this routine during kernel initialization
  * to register the console printing procedure with printk() and to
@@ -3140,8 +3151,8 @@ void register_console(struct console *newcon)
 	 * flag set and will be first in the list.
 	 */
 	if (preferred_console < 0) {
-		if (!console_drivers || !console_drivers->device ||
-		    console_drivers->flags & CON_BOOT) {
+		if (hlist_empty(&console_list) || !console_first()->device ||
+		    console_first()->flags & CON_BOOT) {
 			try_enable_default_console(newcon);
 		}
 	}
@@ -3169,20 +3180,22 @@ void register_console(struct console *newcon)
 	}
 
 	/*
-	 *	Put this console in the list - keep the
-	 *	preferred driver at the head of the list.
+	 * Put this console in the list - keep the
+	 * preferred driver at the head of the list.
 	 */
 	console_lock();
-	if ((newcon->flags & CON_CONSDEV) || console_drivers == NULL) {
-		newcon->next = console_drivers;
-		console_drivers = newcon;
-		if (newcon->next)
-			newcon->next->flags &= ~CON_CONSDEV;
-		/* Ensure this flag is always set for the head of the list */
+	if (hlist_empty(&console_list)) {
+		/* Ensure CON_CONSDEV is always set for the head. */
 		newcon->flags |= CON_CONSDEV;
+		hlist_add_head(&newcon->node, &console_list);
+
+	} else if (newcon->flags & CON_CONSDEV) {
+		/* Only the new head can have CON_CONSDEV set. */
+		console_first()->flags &= ~CON_CONSDEV;
+		hlist_add_head(&newcon->node, &console_list);
+
 	} else {
-		newcon->next = console_drivers->next;
-		console_drivers->next = newcon;
+		hlist_add_behind(&newcon->node, console_list.first);
 	}
 
 	newcon->dropped = 0;
@@ -3209,16 +3222,18 @@ void register_console(struct console *newcon)
 	if (bootcon_enabled &&
 	    ((newcon->flags & (CON_CONSDEV | CON_BOOT)) == CON_CONSDEV) &&
 	    !keep_bootcon) {
-		for_each_console(con)
+		struct hlist_node *tmp;
+
+		hlist_for_each_entry_safe(con, tmp, &console_list, node) {
 			if (con->flags & CON_BOOT)
 				unregister_console(con);
+		}
 	}
 }
 EXPORT_SYMBOL(register_console);
 
 int unregister_console(struct console *console)
 {
-	struct console *con;
 	int res;
 
 	con_printk(KERN_INFO, console, "disabled\n");
@@ -3229,32 +3244,30 @@ int unregister_console(struct console *console)
 	if (res > 0)
 		return 0;
 
-	res = -ENODEV;
 	console_lock();
-	if (console_drivers == console) {
-		console_drivers=console->next;
-		res = 0;
-	} else {
-		for_each_console(con) {
-			if (con->next == console) {
-				con->next = console->next;
-				res = 0;
-				break;
-			}
-		}
+
+	/* Disable it unconditionally */
+	console->flags &= ~CON_ENABLED;
+
+	if (hlist_unhashed(&console->node)) {
+		res = -ENODEV;
+		goto out_unlock;
 	}
 
-	if (res)
-		goto out_disable_unlock;
+	hlist_del_init(&console->node);
 
 	/*
+	 * <HISTORICAL>
 	 * If this isn't the last console and it has CON_CONSDEV set, we
 	 * need to set it on the next preferred console.
+	 * </HISTORICAL>
+	 *
+	 * The above makes no sense as there is no guarantee that the next
+	 * console has any device attached. Oh well....
 	 */
-	if (console_drivers != NULL && console->flags & CON_CONSDEV)
-		console_drivers->flags |= CON_CONSDEV;
+	if (!hlist_empty(&console_list) && console->flags & CON_CONSDEV)
+		console_first()->flags |= CON_CONSDEV;
 
-	console->flags &= ~CON_ENABLED;
 	console_unlock();
 	console_sysfs_notify();
 
@@ -3263,10 +3276,8 @@ int unregister_console(struct console *console)
 
 	return res;
 
-out_disable_unlock:
-	console->flags &= ~CON_ENABLED;
+out_unlock:
 	console_unlock();
-
 	return res;
 }
 EXPORT_SYMBOL(unregister_console);
@@ -3317,10 +3328,11 @@ void __init console_init(void)
  */
 static int __init printk_late_init(void)
 {
+	struct hlist_node *tmp;
 	struct console *con;
 	int ret;
 
-	for_each_console(con) {
+	hlist_for_each_entry_safe(con, tmp, &console_list, node) {
 		if (!(con->flags & CON_BOOT))
 			continue;
 
@@ -3340,6 +3352,7 @@ static int __init printk_late_init(void)
 			unregister_console(con);
 		}
 	}
+
 	ret = cpuhp_setup_state_nocalls(CPUHP_PRINTK_DEAD, "printk:dead", NULL,
 					console_cpu_notify);
 	WARN_ON(ret < 0);
-- 
2.30.2

[PATCH printk v2 25/38] proc: consoles: document console_lock usage

[John Ogness]

The console_lock is held throughout the start/show/stop procedure
to print out device/driver information about all registered
consoles. Since the console_lock is being used for multiple reasons,
explicitly document these reasons. This will be useful when the
console_lock is split into fine-grained locking.

Signed-off-by: John Ogness <john.ogness@linutronix.de>
---
 fs/proc/consoles.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/proc/consoles.c b/fs/proc/consoles.c
index cf2e0788f9c7..32512b477605 100644
--- a/fs/proc/consoles.c
+++ b/fs/proc/consoles.c
@@ -63,6 +63,14 @@ static void *c_start(struct seq_file *m, loff_t *pos)
 	struct console *con;
 	loff_t off = 0;
 
+	/*
+	 * Stop console printing because the device() callback may
+	 * assume the console is not within its write() callback.
+	 *
+	 * Hold the console_lock to guarantee safe traversal of the
+	 * console list. SRCU cannot be used because there is no
+	 * place to store the SRCU cookie.
+	 */
 	console_lock();
 	for_each_console(con)
 		if (off++ == *pos)
-- 
2.30.2

[PATCH printk v2 36/38] proc: consoles: use console_list_lock for list iteration

[John Ogness]

The console_lock is used in part to guarantee safe list iteration.
The console_list_lock should be used because list synchronization
repsponsibility will be removed from the console_lock in a later
change.

Note, the console_lock is still needed to stop console printing.

Signed-off-by: John Ogness <john.ogness@linutronix.de>
---
 fs/proc/consoles.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/fs/proc/consoles.c b/fs/proc/consoles.c
index 32512b477605..77409b176569 100644
--- a/fs/proc/consoles.c
+++ b/fs/proc/consoles.c
@@ -33,7 +33,15 @@ static int show_console_dev(struct seq_file *m, void *v)
 	if (con->device) {
 		const struct tty_driver *driver;
 		int index;
+
+		/*
+		 * Stop console printing because the device() callback may
+		 * assume the console is not within its write() callback.
+		 */
+		console_lock();
 		driver = con->device(con, &index);
+		console_unlock();
+
 		if (driver) {
 			dev = MKDEV(driver->major, driver->minor_start);
 			dev += index;
@@ -64,14 +72,11 @@ static void *c_start(struct seq_file *m, loff_t *pos)
 	loff_t off = 0;
 
 	/*
-	 * Stop console printing because the device() callback may
-	 * assume the console is not within its write() callback.
-	 *
-	 * Hold the console_lock to guarantee safe traversal of the
+	 * Hold the console_list_lock to guarantee safe traversal of the
 	 * console list. SRCU cannot be used because there is no
 	 * place to store the SRCU cookie.
 	 */
-	console_lock();
+	console_list_lock();
 	for_each_console(con)
 		if (off++ == *pos)
 			break;
@@ -89,7 +94,7 @@ static void *c_next(struct seq_file *m, void *v, loff_t *pos)
 
 static void c_stop(struct seq_file *m, void *v)
 {
-	console_unlock();
+	console_list_unlock();
 }
 
 static const struct seq_operations consoles_op = {
-- 
2.30.2

Re: [PATCH printk v2 02/38] printk: Convert console_drivers list to hlist

[Greg Kroah-Hartman]

On Wed, Oct 19, 2022 at 05:01:24PM +0206, John Ogness wrote:
> Replace the open coded single linked list with a hlist so a conversion
> to SRCU protected list walks can reuse the existing primitives.
> 
> Signed-off-by: John Ogness <john.ogness@linutronix.de>
> ---
>  fs/proc/consoles.c      |  3 +-
>  include/linux/console.h |  8 ++--
>  kernel/printk/printk.c  | 99 +++++++++++++++++++++++------------------
>  3 files changed, 63 insertions(+), 47 deletions(-)
> 
> diff --git a/fs/proc/consoles.c b/fs/proc/consoles.c
> index dfe6ce3505ce..cf2e0788f9c7 100644
> --- a/fs/proc/consoles.c
> +++ b/fs/proc/consoles.c
> @@ -74,8 +74,9 @@ static void *c_start(struct seq_file *m, loff_t *pos)
>  static void *c_next(struct seq_file *m, void *v, loff_t *pos)
>  {
>  	struct console *con = v;
> +
>  	++*pos;
> -	return con->next;
> +	return hlist_entry_safe(con->node.next, struct console, node);
>  }
>  
>  static void c_stop(struct seq_file *m, void *v)
> diff --git a/include/linux/console.h b/include/linux/console.h
> index 8c1686e2c233..7b5f21f9e469 100644
> --- a/include/linux/console.h
> +++ b/include/linux/console.h
> @@ -15,6 +15,7 @@
>  #define _LINUX_CONSOLE_H_ 1
>  
>  #include <linux/atomic.h>
> +#include <linux/list.h>
>  #include <linux/types.h>
>  
>  struct vc_data;
> @@ -154,14 +155,16 @@ struct console {
>  	u64	seq;
>  	unsigned long dropped;
>  	void	*data;
> -	struct	 console *next;
> +	struct hlist_node node;
>  };
>  
> +extern struct hlist_head console_list;
> +
>  /*
>   * for_each_console() allows you to iterate on each console
>   */
>  #define for_each_console(con) \
> -	for (con = console_drivers; con != NULL; con = con->next)
> +	hlist_for_each_entry(con, &console_list, node)
>  
>  extern int console_set_on_cmdline;
>  extern struct console *early_console;
> @@ -174,7 +177,6 @@ enum con_flush_mode {
>  extern int add_preferred_console(char *name, int idx, char *options);
>  extern void register_console(struct console *);
>  extern int unregister_console(struct console *);
> -extern struct console *console_drivers;
>  extern void console_lock(void);
>  extern int console_trylock(void);
>  extern void console_unlock(void);
> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> index e4f1e7478b52..867becc40021 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -79,13 +79,12 @@ int oops_in_progress;
>  EXPORT_SYMBOL(oops_in_progress);
>  
>  /*
> - * console_sem protects the console_drivers list, and also
> - * provides serialisation for access to the entire console
> - * driver system.
> + * console_sem protects console_list and console->flags updates, and also
> + * provides serialization for access to the entire console driver system.
>   */
>  static DEFINE_SEMAPHORE(console_sem);
> -struct console *console_drivers;
> -EXPORT_SYMBOL_GPL(console_drivers);
> +HLIST_HEAD(console_list);
> +EXPORT_SYMBOL_GPL(console_list);
>  
>  /*
>   * System may need to suppress printk message under certain
> @@ -2556,7 +2555,7 @@ static int console_cpu_notify(unsigned int cpu)
>   * console_lock - lock the console system for exclusive use.
>   *
>   * Acquires a lock which guarantees that the caller has
> - * exclusive access to the console system and the console_drivers list.
> + * exclusive access to the console system and console_list.
>   *
>   * Can sleep, returns nothing.
>   */
> @@ -2576,7 +2575,7 @@ EXPORT_SYMBOL(console_lock);
>   * console_trylock - try to lock the console system for exclusive use.
>   *
>   * Try to acquire a lock which guarantees that the caller has exclusive
> - * access to the console system and the console_drivers list.
> + * access to the console system and console_list.
>   *
>   * returns 1 on success, and 0 on failure to acquire the lock.
>   */
> @@ -2940,11 +2939,20 @@ void console_flush_on_panic(enum con_flush_mode mode)
>  	console_may_schedule = 0;
>  
>  	if (mode == CONSOLE_REPLAY_ALL) {
> +		struct hlist_node *tmp;
>  		struct console *c;
>  		u64 seq;
>  
>  		seq = prb_first_valid_seq(prb);
> -		for_each_console(c)
> +		/*
> +		 * This cannot use for_each_console() because it's not established
> +		 * that the current context has console locked and neither there is
> +		 * a guarantee that there is no concurrency in that case.
> +		 *
> +		 * Open code it for documentation purposes and pretend that
> +		 * it works.
> +		 */
> +		hlist_for_each_entry_safe(c, tmp, &console_list, node)
>  			c->seq = seq;
>  	}
>  	console_unlock();
> @@ -3081,6 +3089,9 @@ static void try_enable_default_console(struct console *newcon)
>  	       (con->flags & CON_BOOT) ? "boot" : "",	\
>  	       con->name, con->index, ##__VA_ARGS__)
>  
> +#define console_first()				\
> +	hlist_entry(console_list.first, struct console, node)
> +
>  /*
>   * The console driver calls this routine during kernel initialization
>   * to register the console printing procedure with printk() and to
> @@ -3140,8 +3151,8 @@ void register_console(struct console *newcon)
>  	 * flag set and will be first in the list.
>  	 */
>  	if (preferred_console < 0) {
> -		if (!console_drivers || !console_drivers->device ||
> -		    console_drivers->flags & CON_BOOT) {
> +		if (hlist_empty(&console_list) || !console_first()->device ||
> +		    console_first()->flags & CON_BOOT) {
>  			try_enable_default_console(newcon);
>  		}
>  	}
> @@ -3169,20 +3180,22 @@ void register_console(struct console *newcon)
>  	}
>  
>  	/*
> -	 *	Put this console in the list - keep the
> -	 *	preferred driver at the head of the list.
> +	 * Put this console in the list - keep the
> +	 * preferred driver at the head of the list.
>  	 */
>  	console_lock();
> -	if ((newcon->flags & CON_CONSDEV) || console_drivers == NULL) {
> -		newcon->next = console_drivers;
> -		console_drivers = newcon;
> -		if (newcon->next)
> -			newcon->next->flags &= ~CON_CONSDEV;
> -		/* Ensure this flag is always set for the head of the list */
> +	if (hlist_empty(&console_list)) {
> +		/* Ensure CON_CONSDEV is always set for the head. */
>  		newcon->flags |= CON_CONSDEV;
> +		hlist_add_head(&newcon->node, &console_list);
> +
> +	} else if (newcon->flags & CON_CONSDEV) {
> +		/* Only the new head can have CON_CONSDEV set. */
> +		console_first()->flags &= ~CON_CONSDEV;
> +		hlist_add_head(&newcon->node, &console_list);
> +
>  	} else {
> -		newcon->next = console_drivers->next;
> -		console_drivers->next = newcon;
> +		hlist_add_behind(&newcon->node, console_list.first);
>  	}
>  
>  	newcon->dropped = 0;
> @@ -3209,16 +3222,18 @@ void register_console(struct console *newcon)
>  	if (bootcon_enabled &&
>  	    ((newcon->flags & (CON_CONSDEV | CON_BOOT)) == CON_CONSDEV) &&
>  	    !keep_bootcon) {
> -		for_each_console(con)
> +		struct hlist_node *tmp;
> +
> +		hlist_for_each_entry_safe(con, tmp, &console_list, node) {
>  			if (con->flags & CON_BOOT)
>  				unregister_console(con);
> +		}
>  	}
>  }
>  EXPORT_SYMBOL(register_console);
>  
>  int unregister_console(struct console *console)
>  {
> -	struct console *con;
>  	int res;
>  
>  	con_printk(KERN_INFO, console, "disabled\n");
> @@ -3229,32 +3244,30 @@ int unregister_console(struct console *console)
>  	if (res > 0)
>  		return 0;
>  
> -	res = -ENODEV;
>  	console_lock();
> -	if (console_drivers == console) {
> -		console_drivers=console->next;
> -		res = 0;
> -	} else {
> -		for_each_console(con) {
> -			if (con->next == console) {
> -				con->next = console->next;
> -				res = 0;
> -				break;
> -			}
> -		}
> +
> +	/* Disable it unconditionally */
> +	console->flags &= ~CON_ENABLED;
> +
> +	if (hlist_unhashed(&console->node)) {

How can this ever be hit?  The console lock is held, so it shouldn't
have gone away already.  Or am I missing something else here?

Other than that minor question, looks good to me!

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [PATCH printk v2 02/38] printk: Convert console_drivers list to hlist

[John Ogness]

On 2022-10-19, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
>> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
>> index e4f1e7478b52..867becc40021 100644
>> --- a/kernel/printk/printk.c
>> +++ b/kernel/printk/printk.c
>> @@ -3229,32 +3244,30 @@ int unregister_console(struct console *console)
>>  	if (res > 0)
>>  		return 0;
>>  
>> -	res = -ENODEV;
>>  	console_lock();
>> -	if (console_drivers == console) {
>> -		console_drivers=console->next;
>> -		res = 0;
>> -	} else {
>> -		for_each_console(con) {
>> -			if (con->next == console) {
>> -				con->next = console->next;
>> -				res = 0;
>> -				break;
>> -			}
>> -		}
>> +
>> +	/* Disable it unconditionally */
>> +	console->flags &= ~CON_ENABLED;
>> +
>> +	if (hlist_unhashed(&console->node)) {
>
> How can this ever be hit?  The console lock is held, so it shouldn't
> have gone away already.  Or am I missing something else here?

Mainline also has this check. I expect it is for the case that some code
tries to call unregister_console() for a console that is not
registered.

Since register_console() does not return if it succeeded, I suppose some
code somewhere my try to unregister without knowing that it never
registered in the first place.

> Other than that minor question, looks good to me!
>
> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Thanks!

John

Re: [PATCH printk v2 02/38] printk: Convert console_drivers list to hlist

[Greg Kroah-Hartman]

On Wed, Oct 19, 2022 at 11:52:53PM +0206, John Ogness wrote:
> On 2022-10-19, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> >> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
> >> index e4f1e7478b52..867becc40021 100644
> >> --- a/kernel/printk/printk.c
> >> +++ b/kernel/printk/printk.c
> >> @@ -3229,32 +3244,30 @@ int unregister_console(struct console *console)
> >>  	if (res > 0)
> >>  		return 0;
> >>  
> >> -	res = -ENODEV;
> >>  	console_lock();
> >> -	if (console_drivers == console) {
> >> -		console_drivers=console->next;
> >> -		res = 0;
> >> -	} else {
> >> -		for_each_console(con) {
> >> -			if (con->next == console) {
> >> -				con->next = console->next;
> >> -				res = 0;
> >> -				break;
> >> -			}
> >> -		}
> >> +
> >> +	/* Disable it unconditionally */
> >> +	console->flags &= ~CON_ENABLED;
> >> +
> >> +	if (hlist_unhashed(&console->node)) {
> >
> > How can this ever be hit?  The console lock is held, so it shouldn't
> > have gone away already.  Or am I missing something else here?
> 
> Mainline also has this check. I expect it is for the case that some code
> tries to call unregister_console() for a console that is not
> registered.
> 
> Since register_console() does not return if it succeeded, I suppose some
> code somewhere my try to unregister without knowing that it never
> registered in the first place.

Ick, ok, that's fine for now.

What a mess, thanks for working to unwind it!

greg k-h

Re: [PATCH printk v2 02/38] printk: Convert console_drivers list to hlist

[Petr Mladek]

On Wed 2022-10-19 17:01:24, John Ogness wrote:
> Replace the open coded single linked list with a hlist so a conversion
> to SRCU protected list walks can reuse the existing primitives.
> 
> Signed-off-by: John Ogness <john.ogness@linutronix.de>

Reviewed-by: Petr Mladek <pmladek@suse.com>

Just one nit below.

> @@ -3229,32 +3244,30 @@ int unregister_console(struct console *console)
>  	if (res > 0)
>  		return 0;
>  
> -	res = -ENODEV;
>  	console_lock();
> -	if (console_drivers == console) {
> -		console_drivers=console->next;
> -		res = 0;
> -	} else {
> -		for_each_console(con) {
> -			if (con->next == console) {
> -				con->next = console->next;
> -				res = 0;
> -				break;
> -			}
> -		}
> +
> +	/* Disable it unconditionally */
> +	console->flags &= ~CON_ENABLED;
> +
> +	if (hlist_unhashed(&console->node)) {
> +		res = -ENODEV;
> +		goto out_unlock;

Nit: It might make sense to replace this with:

		console_unlock();
		return -ENODEV;

This is the only code path using the extra goto target.

It is just an idea. I do not resist on this change.

>  	}
>  
> -	if (res)
> -		goto out_disable_unlock;
> +	hlist_del_init(&console->node);
>  
>  	/*
> +	 * <HISTORICAL>
>  	 * If this isn't the last console and it has CON_CONSDEV set, we
>  	 * need to set it on the next preferred console.
> +	 * </HISTORICAL>
> +	 *
> +	 * The above makes no sense as there is no guarantee that the next
> +	 * console has any device attached. Oh well....
>  	 */
> -	if (console_drivers != NULL && console->flags & CON_CONSDEV)
> -		console_drivers->flags |= CON_CONSDEV;
> +	if (!hlist_empty(&console_list) && console->flags & CON_CONSDEV)
> +		console_first()->flags |= CON_CONSDEV;
>  
> -	console->flags &= ~CON_ENABLED;
>  	console_unlock();
>  	console_sysfs_notify();
>  
> @@ -3263,10 +3276,8 @@ int unregister_console(struct console *console)
>  
>  	return res;
>  
> -out_disable_unlock:
> -	console->flags &= ~CON_ENABLED;
> +out_unlock:
>  	console_unlock();
> -
>  	return res;
>  }
>  EXPORT_SYMBOL(unregister_console);

Best Regards,
Petr

Re: [PATCH printk v2 02/38] printk: Convert console_drivers list to hlist

[Sergey Senozhatsky]

On (22/10/19 17:01), John Ogness wrote:
> Replace the open coded single linked list with a hlist so a conversion
> to SRCU protected list walks can reuse the existing primitives.
> 
> Signed-off-by: John Ogness <john.ogness@linutronix.de>

Reviewed-by: Sergey Senozhatsky <senozhatsky@chromium.org>

Re: [PATCH printk v2 25/38] proc: consoles: document console_lock usage

[Petr Mladek]

On Wed 2022-10-19 17:01:47, John Ogness wrote:
> The console_lock is held throughout the start/show/stop procedure
> to print out device/driver information about all registered
> consoles. Since the console_lock is being used for multiple reasons,
> explicitly document these reasons. This will be useful when the
> console_lock is split into fine-grained locking.
> 
> Signed-off-by: John Ogness <john.ogness@linutronix.de>
> ---
>  fs/proc/consoles.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/fs/proc/consoles.c b/fs/proc/consoles.c
> index cf2e0788f9c7..32512b477605 100644
> --- a/fs/proc/consoles.c
> +++ b/fs/proc/consoles.c
> @@ -63,6 +63,14 @@ static void *c_start(struct seq_file *m, loff_t *pos)
>  	struct console *con;
>  	loff_t off = 0;
>  
> +	/*
> +	 * Stop console printing because the device() callback may
> +	 * assume the console is not within its write() callback.

Like in previous patches, I would prefer to add more information
about this dependency. An example or if it is just to stay
on the safe side.

> +	 *
> +	 * Hold the console_lock to guarantee safe traversal of the
> +	 * console list. SRCU cannot be used because there is no
> +	 * place to store the SRCU cookie.

It might be possible to crate a custom struct for passing both
the next struct console and SRCU cookie. But it probably
is not worth it.

> +	 */
>  	console_lock();
>  	for_each_console(con)
>  		if (off++ == *pos)

Best Regards,
Petr

Re: [PATCH printk v2 36/38] proc: consoles: use console_list_lock for list iteration

[Petr Mladek]

On Wed 2022-10-19 17:01:58, John Ogness wrote:
> The console_lock is used in part to guarantee safe list iteration.
> The console_list_lock should be used because list synchronization
> repsponsibility will be removed from the console_lock in a later
> change.
> 
> Note, the console_lock is still needed to stop console printing.
> 
> Signed-off-by: John Ogness <john.ogness@linutronix.de>

Reviewed-by: Petr Mladek <pmladek@suse.com>

Best Regards,
Petr