[PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[Nicolas Thibert]

__stop_tx() (8250_port.c) only calls the RS485 rs485_stop_tx() hook
(which de-asserts the direction GPIO/RTS line) once it has observed
both UART_LSR_THRE and UART_LSR_TEMT for the last byte. If TEMT is
never seen and the driver hasn't set UART_CAP_NOTEMT, the function
returns without scheduling any retry -- the direction line is left
asserted (driver enabled) forever, with nothing to un-stick it short
of another kernel-visible LSR event.

of_platform_serial_setup() unconditionally wires up the generic em485
GPIO-RTS RS485 support (rs485_config/rs485_start_tx/rs485_stop_tx) for
every port it registers, but never sets UART_CAP_NOTEMT, so any board
using this driver whose 16550-compatible core doesn't reliably surface
TEMT for its shift register hits the stuck-direction-GPIO case above.

Confirmed live on an ath79 QCA9531 board (SoC-internal ns16550a-
compatible UART, RS485 transceiver DE/RE tied together on a GPIO via
rts-gpios, linux,rs485-enabled-at-boot-time): the direction GPIO
correctly asserts for the duration of a transmit, but never
de-asserts afterwards -- confirmed by sampling the GPIO's debugfs
state through and after a multi-hundred-byte write, on both the first
transmit and repeated back-to-back transmits. Setting
UART_CAP_NOTEMT, which makes __stop_tx() fall back to a frame-time-
based timer instead of waiting indefinitely on TEMT, makes the
direction GPIO reliably return low right after each transmit
completes.

Scope the fix to ports that declare a GPIO-controlled direction line
(rts-gpios), rather than setting it unconditionally for every port
this driver registers: this is the class of hardware actually
affected (RTS state has to be explicitly un-stuck by software, unlike
a UART's native RTS pin), and it avoids adding the extra frame-time
margin to ports relying on the native RTS pin, which has not been
observed to need it.

Signed-off-by: Nicolas Thibert <nithibert@gmail.com>
Assisted-by: LLM (Claude Sonnet 5, Anthropic)
---
 drivers/tty/serial/8250/8250_of.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

--- a/drivers/tty/serial/8250/8250_of.c
+++ b/drivers/tty/serial/8250/8250_of.c
@@ -156,6 +156,22 @@ static int of_platform_serial_setup(struct platform_device *ofdev,
 	up->rs485_start_tx = serial8250_em485_start_tx;
 	up->rs485_stop_tx = serial8250_em485_stop_tx;

+	/*
+	 * This generic driver never enables a dedicated line-status
+	 * interrupt on TEMT, so for ports whose RS485 direction is
+	 * controlled via a GPIO (rts-gpios) rather than the native RTS
+	 * pin, __stop_tx() (8250_port.c) can see THRE without TEMT on the
+	 * last byte and bail out without ever retrying -- leaving the
+	 * direction GPIO stuck asserted after the last byte sent, on
+	 * hardware whose shift register doesn't reliably surface TEMT.
+	 * UART_CAP_NOTEMT makes it fall back to a frame-time-based timer
+	 * instead of waiting on that interrupt. Scoped to rts-gpios users
+	 * only, to avoid changing timing for ports relying on the native
+	 * RTS pin, which this has not been observed to affect.
+	 */
+	if (of_property_present(np, "rts-gpios"))
+		up->capabilities |= UART_CAP_NOTEMT;
+
 	switch (type) {
 	case PORT_RT2880:
 		ret = rt288x_setup(port);

Re: [PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The newly added UART_CAP_NOTEMT flag is either silently overwritten or causes the port to lose all of its default hardware capabilities.
--

commit 1d8dc615b4080d6f9bde37f54eafa34f59649ede
Author: Nicolas Thibert <nithibert@gmail.com>

serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

This patch sets UART_CAP_NOTEMT for RS485 ports using rts-gpios to prevent
the direction GPIO from getting stuck due to missing TEMT interrupts.

> diff --git a/drivers/tty/serial/8250/8250_of.c b/drivers/tty/serial/8250/8250_of.c
> index b1561c499acda..600a77e1e6143 100644
> --- a/drivers/tty/serial/8250/8250_of.c
> +++ b/drivers/tty/serial/8250/8250_of.c
> @@ -212,6 +212,22 @@ static int of_platform_serial_setup(struct platform_device *ofdev,
>  	up->rs485_start_tx = serial8250_em485_start_tx;
>  	up->rs485_stop_tx = serial8250_em485_stop_tx;
>  
> +	/*
> +	 * This generic driver never enables a dedicated line-status
> +	 * interrupt on TEMT, so for ports whose RS485 direction is
> +	 * controlled via a GPIO (rts-gpios) rather than the native RTS
> +	 * pin, __stop_tx() (8250_port.c) can see THRE without TEMT on the
> +	 * last byte and bail out without ever retrying -- leaving the
> +	 * direction GPIO stuck asserted after the last byte sent, on
> +	 * hardware whose shift register doesn't reliably surface TEMT.
> +	 * UART_CAP_NOTEMT makes it fall back to a frame-time-based timer
> +	 * instead of waiting on that interrupt. Scoped to rts-gpios users
> +	 * only, to avoid changing timing for ports relying on the native
> +	 * RTS pin, which this has not been observed to affect.
> +	 */
> +	if (of_property_present(np, "rts-gpios"))
> +		up->capabilities |= UART_CAP_NOTEMT;
> +
>  	switch (type) {

[Severity: High]
If the device explicitly configures a FIFO size, does the caller overwrite
this newly added capability?

When of_platform_serial_probe() calls of_platform_serial_setup(), it does
this right after:

drivers/tty/serial/8250/8250_of.c:of_platform_serial_probe() {
	...
	ret = of_platform_serial_setup(ofdev, port_type, &port8250, info);
	if (ret)
		goto err_free;

	if (port8250.port.fifosize)
		port8250.capabilities = UART_CAP_FIFO;
	...
}

Will this direct assignment completely erase the UART_CAP_NOTEMT flag?

Conversely, if the device does not specify a fifosize, port8250.capabilities
will be non-zero because it holds the UART_CAP_NOTEMT flag. Does this
cause the port to miss all its default hardware capabilities?

Later during registration, serial8250_set_defaults() checks if capabilities
is zero before assigning defaults:

drivers/tty/serial/8250/8250_port.c:serial8250_set_defaults() {
	...
	if (!up->capabilities)
		up->capabilities = uart_config[port->type].flags;
	...
}

Since up->capabilities is non-zero, this assignment is skipped entirely,
stripping the port of default features like UART_CAP_FIFO and UART_CAP_SLEEP.

Could we change the assignment in of_platform_serial_probe() to use |=,
and find a different way to merge UART_CAP_NOTEMT with the default flags?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260907152744.1084359-1-nithibert@gmail.com?part=1

[PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[Nicolas Thibert]

__stop_tx() (8250_port.c) only calls the RS485 rs485_stop_tx() hook
(which de-asserts the direction GPIO/RTS line) once it has observed
both UART_LSR_THRE and UART_LSR_TEMT for the last byte. If TEMT is
never seen and the driver hasn't set UART_CAP_NOTEMT, the function
returns without scheduling any retry -- the direction line is left
asserted (driver enabled) forever, with nothing to un-stick it short
of another kernel-visible LSR event.

of_platform_serial_setup()/of_platform_serial_probe() unconditionally
wire up the generic em485 GPIO-RTS RS485 support
(rs485_config/rs485_start_tx/rs485_stop_tx) for every port they
register, but never set UART_CAP_NOTEMT, so any board using this
driver whose 16550-compatible core doesn't reliably surface TEMT for
its shift register hits the stuck-direction-GPIO case above.

Confirmed live on an ath79 QCA9531 board (SoC-internal ns16550a-
compatible UART, RS485 transceiver DE/RE tied together on a GPIO via
rts-gpios, linux,rs485-enabled-at-boot-time): the direction GPIO
correctly asserts for the duration of a transmit, but never
de-asserts afterwards -- confirmed by sampling the GPIO's debugfs
state through and after a multi-hundred-byte write, on both the first
transmit and repeated back-to-back transmits. Setting
UART_CAP_NOTEMT, which makes __stop_tx() fall back to a frame-time-
based timer instead of waiting indefinitely on TEMT, makes the
direction GPIO reliably return low right after each transmit
completes.

Scope the fix to ports that declare a GPIO-controlled direction line
(rts-gpios), rather than setting it unconditionally for every port
this driver registers: this is the class of hardware actually
affected (RTS state has to be explicitly un-stuck by software, unlike
a UART's native RTS pin), and it avoids adding the extra frame-time
margin to ports relying on the native RTS pin, which this has not
been observed to need.

Set it in of_platform_serial_probe(), after the existing
"if (port8250.port.fifosize) port8250.capabilities = UART_CAP_FIFO;"
assignment, rather than in of_platform_serial_setup(): that later
plain assignment (not "|=") unconditionally overwrites
port8250.capabilities on any port whose "fifo-size" DT property is
set, silently discarding a capability bit set earlier in setup(). Not
observed on the reporter's own board (it has no "fifo-size" property,
so port.fifosize stays 0 and that assignment is skipped), but a real
regression waiting to happen on any other of_platform_serial user
that does set "fifo-size" -- which is a common, documented property
for this binding.

Signed-off-by: Nicolas Thibert <nithibert@gmail.com>
Assisted-by: LLM (Claude Sonnet 5, Anthropic)
---
 drivers/tty/serial/8250/8250_of.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

--- a/drivers/tty/serial/8250/8250_of.c
+++ b/drivers/tty/serial/8250/8250_of.c
@@ -233,6 +233,24 @@ static int of_platform_serial_probe(struct platform_device *ofdev)
 			&port8250.overrun_backoff_time_ms) != 0)
 		port8250.overrun_backoff_time_ms = 0;

+	/*
+	 * This generic driver never enables a dedicated line-status
+	 * interrupt on TEMT, so for ports whose RS485 direction is
+	 * controlled via a GPIO (rts-gpios) rather than the native RTS
+	 * pin, __stop_tx() (8250_port.c) can see THRE without TEMT on the
+	 * last byte and bail out without ever retrying -- leaving the
+	 * direction GPIO stuck asserted after the last byte sent, on
+	 * hardware whose shift register doesn't reliably surface TEMT.
+	 * UART_CAP_NOTEMT makes it fall back to a frame-time-based timer
+	 * instead of waiting on that interrupt. Scoped to rts-gpios users
+	 * only, to avoid changing timing for ports relying on the native
+	 * RTS pin, which this has not been observed to affect. Set here,
+	 * after the fifosize-based capabilities assignment above, so it
+	 * isn't clobbered by it.
+	 */
+	if (of_property_present(ofdev->dev.of_node, "rts-gpios"))
+		port8250.capabilities |= UART_CAP_NOTEMT;
+
 	ret = serial8250_register_8250_port(&port8250);
 	if (ret < 0)
 		goto err_dispose;

Re: [PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[sashiko-bot]

> __stop_tx() (8250_port.c) only calls the RS485 rs485_stop_tx() hook
> (which de-asserts the direction GPIO/RTS line) once it has observed
> both UART_LSR_THRE and UART_LSR_TEMT for the last byte. If TEMT is
> never seen and the driver hasn't set UART_CAP_NOTEMT, the function
> returns without scheduling any retry -- the direction line is left
> asserted (driver enabled) forever, with nothing to un-stick it short
> of another kernel-visible LSR event.
> 
> of_platform_serial_setup()/of_platform_serial_probe() unconditionally
> wire up the generic em485 GPIO-RTS RS485 support
> (rs485_config/rs485_start_tx/rs485_stop_tx) for every port they
> register, but never set UART_CAP_NOTEMT, so any board using this
> driver whose 16550-compatible core doesn't reliably surface TEMT for
> its shift register hits the stuck-direction-GPIO case above.
> 
> [ ... ]
> regression waiting to happen on any other of_platform_serial user
> that does set "fifo-size" -- which is a common, documented property
> for this binding.
> 
> Signed-off-by: Nicolas Thibert <nithibert@gmail.com>

Sashiko has reviewed this patch and found no issues. It looks great!

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260907173000.1254045-1-nithibert@gmail.com?part=1