[PATCH v8 6/7] xen/serial: harden serial_tx_buffer checks

[dmukhin@ford.com]

From: Denis Mukhin <dmukhin@ford.com> 

Ensure the user-defined value never crosses 2GB boundary and always
rounded to the next power of 2 to align logic with console driver
conring buffer management code.

Signed-off-by: Denis Mukhin <dmukhin@ford.com>
---
Changes since v7:
- addressed Jan's feedback:
  https://lore.kernel.org/xen-devel/89029dbd-df1f-45d4-8a02-720cd6a42cab@suse.com/
- kept only check for large buffer in serial_async_transmit()
  and a doc update.
---
 docs/misc/xen-command-line.pandoc | 2 ++
 xen/drivers/char/serial.c         | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 1c711fa98086..2be8772b329a 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2396,6 +2396,8 @@ accidentally leaking secrets by releasing pages without proper sanitization.
 
 Set the serial transmit buffer size.
 
+The value provided will be rounded down to the nearest power of 2.
+
 ### serrors (ARM)
 > `= diverse | panic`
 
diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c
index cf0abf1893e5..ba1647309ab8 100644
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -523,6 +523,8 @@ void __init serial_async_transmit(struct serial_port *port)
         return;
     if ( serial_txbufsz < PAGE_SIZE )
         serial_txbufsz = PAGE_SIZE;
+    if ( serial_txbufsz > GB(2) )
+        serial_txbufsz = CONFIG_SERIAL_TX_BUFSIZE;
     while ( serial_txbufsz & (serial_txbufsz - 1) )
         serial_txbufsz &= serial_txbufsz - 1;
     port->txbuf = xvmalloc_array(char, serial_txbufsz);
-- 
2.54.0