[PATCH v3] serial: port: Don't suspend if the port is still busy

[PATCH v3] serial: port: Don't suspend if the port is still busy

[Yicong Yang]

From: Yicong Yang <yangyicong@hisilicon.com>

We accidently met the issue that the bash prompt is not shown after the
previous command done and until the next input if there's only one CPU
(In our issue other CPUs are isolated by isolcpus=). Further analysis
shows it's because the port entering runtime suspend even if there's
still pending chars in the buffer and the pending chars will only be
processed in next device resuming. We are using amba-pl011 and the
problematic flow is like below:

Bash                                         kworker
tty_write()
  file_tty_write()
    n_tty_write()
      uart_write()
        __uart_start()
          pm_runtime_get() // wakeup waker
            queue_work()
                                             pm_runtime_work()
                                               rpm_resume()
                                                status = RPM_RESUMING
                                                serial_port_runtime_resume()
                                                  port->ops->start_tx()
                                                    pl011_tx_chars()
                                                      uart_write_wakeup()
        […]
        __uart_start()
          pm_runtime_get() < 0 // because runtime status = RPM_RESUMING
                               // later data are not commit to the port driver
                                                status = RPM_ACTIVE
                                                rpm_idle() -> rpm_suspend()

This patch tries to fix this by checking the port busy before entering
runtime suspending. A runtime_suspend callback is added for the port
driver. When entering runtime suspend the callback is invoked, if there's
still pending chars in the buffer then flush the buffer.

Cc: Tony Lindgren <tony@atomide.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Fixes: 84a9582fd203 ("serial: core: Start managing serial controllers to enable runtime PM")
Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Tony Lindgren <tony@atomide.com>
---
Change since v2:
- Narrow the spinlock region per Andy
- Make __serial_port_busy() return -EBUSY if port has pending chars per Andy
Thanks.
Link: https://lore.kernel.org/all/20240206073322.5560-1-yangyicong@huawei.com/

Change since v1:
- Use port lock wrapper per John
- Flush the pending chars and return -EBUSY per Tony.
Thanks.
Link: https://lore.kernel.org/all/20240204031957.58176-1-yangyicong@huawei.com/

 drivers/tty/serial/serial_port.c | 36 +++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/serial/serial_port.c b/drivers/tty/serial/serial_port.c
index 88975a4df306..8d72f7d95009 100644
--- a/drivers/tty/serial/serial_port.c
+++ b/drivers/tty/serial/serial_port.c
@@ -19,8 +19,13 @@
 /* Only considers pending TX for now. Caller must take care of locking */
 static int __serial_port_busy(struct uart_port *port)
 {
-	return !uart_tx_stopped(port) &&
-		uart_circ_chars_pending(&port->state->xmit);
+	if (uart_tx_stopped(port))
+		return 0;
+
+	if (uart_circ_chars_pending(&port->state->xmit))
+		return -EBUSY;
+
+	return 0;
 }
 
 static int serial_port_runtime_resume(struct device *dev)
@@ -46,8 +51,33 @@ static int serial_port_runtime_resume(struct device *dev)
 	return 0;
 }
 
+static int serial_port_runtime_suspend(struct device *dev)
+{
+	struct serial_port_device *port_dev = to_serial_base_port_device(dev);
+	struct uart_port *port;
+	unsigned long flags;
+	int ret;
+
+	port = port_dev->port;
+
+	if (port->flags & UPF_DEAD)
+		return 0;
+
+	uart_port_lock_irqsave(port, &flags);
+	ret = __serial_port_busy(port);
+	if (ret)
+		port->ops->start_tx(port);
+	uart_port_unlock_irqrestore(port, flags);
+
+	if (ret)
+		pm_runtime_mark_last_busy(dev);
+
+	return ret;
+}
+
 static DEFINE_RUNTIME_DEV_PM_OPS(serial_port_pm,
-				 NULL, serial_port_runtime_resume, NULL);
+				 serial_port_runtime_suspend,
+				 serial_port_runtime_resume, NULL);
 
 static int serial_port_probe(struct device *dev)
 {
-- 
2.24.0

Re: [PATCH v3] serial: port: Don't suspend if the port is still busy

[Tony Lindgren]

* Yicong Yang <yangyicong@huawei.com> [240208 07:56]:
> Reviewed-by: Tony Lindgren <tony@atomide.com>
> ---
> Change since v2:
> - Narrow the spinlock region per Andy
> - Make __serial_port_busy() return -EBUSY if port has pending chars per Andy
> Thanks.
> Link: https://lore.kernel.org/all/20240206073322.5560-1-yangyicong@huawei.com/

Looks good to me thanks.

Regards,

Tony

Re: [PATCH v3] serial: port: Don't suspend if the port is still busy

[Jiri Slaby]

Hi,

On 08. 02. 24, 8:52, Yicong Yang wrote:
> From: Yicong Yang <yangyicong@hisilicon.com>
> 
> We accidently met the issue that the bash prompt is not shown after the
> previous command done and until the next input if there's only one CPU
> (In our issue other CPUs are isolated by isolcpus=). Further analysis
> shows it's because the port entering runtime suspend even if there's
> still pending chars in the buffer and the pending chars will only be
> processed in next device resuming. We are using amba-pl011 and the
> problematic flow is like below:

...
> --- a/drivers/tty/serial/serial_port.c
> +++ b/drivers/tty/serial/serial_port.c
> @@ -19,8 +19,13 @@
>   /* Only considers pending TX for now. Caller must take care of locking */
>   static int __serial_port_busy(struct uart_port *port)
>   {
> -	return !uart_tx_stopped(port) &&
> -		uart_circ_chars_pending(&port->state->xmit);
> +	if (uart_tx_stopped(port))
> +		return 0;
> +
> +	if (uart_circ_chars_pending(&port->state->xmit))
> +		return -EBUSY;

Why do you do this change at all? If anything, __serial_port_busy() 
should be made to return a bool and not to return an error. Look how it 
is named -- returning EBUSY is sort of unexpected in my eyes. And if 
this needed to be done, it should have been in a separate patch anyway.

And then:

> @@ -46,8 +51,33 @@ static int serial_port_runtime_resume(struct device *dev)
>   	return 0;
>   }
>   
> +static int serial_port_runtime_suspend(struct device *dev)
> +{
> +	struct serial_port_device *port_dev = to_serial_base_port_device(dev);
> +	struct uart_port *port;
> +	unsigned long flags;
> +	int ret;

bool busy;

> +
> +	port = port_dev->port;
> +
> +	if (port->flags & UPF_DEAD)
> +		return 0;
> +
> +	uart_port_lock_irqsave(port, &flags);
> +	ret = __serial_port_busy(port);
> +	if (ret)

busy = ...
if (busy)

> +		port->ops->start_tx(port);
> +	uart_port_unlock_irqrestore(port, flags);
> +
> +	if (ret)

if (busy)

> +		pm_runtime_mark_last_busy(dev);
> +
> +	return ret;

return busy ? -EBUSY : 0;

> +}

thanks,
-- 
js
suse labs

Re: [PATCH v3] serial: port: Don't suspend if the port is still busy

[Andy Shevchenko]

On Thu, Feb 08, 2024 at 09:27:57AM +0100, Jiri Slaby wrote:
> On 08. 02. 24, 8:52, Yicong Yang wrote:

...

> >   static int __serial_port_busy(struct uart_port *port)
> >   {
> > -	return !uart_tx_stopped(port) &&
> > -		uart_circ_chars_pending(&port->state->xmit);
> > +	if (uart_tx_stopped(port))
> > +		return 0;
> > +
> > +	if (uart_circ_chars_pending(&port->state->xmit))
> > +		return -EBUSY;
> 
> Why do you do this change at all? If anything, __serial_port_busy() should
> be made to return a bool and not to return an error. Look how it is named --
> returning EBUSY is sort of unexpected in my eyes. And if this needed to be
> done, it should have been in a separate patch anyway.

I proposed that with a renaming, so it won't look as boolean.
And I also implied (sorry if it was unclear) that this has to be
done separately, so we are on the same page about this.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3] serial: port: Don't suspend if the port is still busy

[Yicong Yang]

On 2024/2/9 1:25, Andy Shevchenko wrote:
> On Thu, Feb 08, 2024 at 09:27:57AM +0100, Jiri Slaby wrote:
>> On 08. 02. 24, 8:52, Yicong Yang wrote:
> 
> ...
> 
>>>   static int __serial_port_busy(struct uart_port *port)
>>>   {
>>> -	return !uart_tx_stopped(port) &&
>>> -		uart_circ_chars_pending(&port->state->xmit);
>>> +	if (uart_tx_stopped(port))
>>> +		return 0;
>>> +
>>> +	if (uart_circ_chars_pending(&port->state->xmit))
>>> +		return -EBUSY;
>>
>> Why do you do this change at all? If anything, __serial_port_busy() should
>> be made to return a bool and not to return an error. Look how it is named --
>> returning EBUSY is sort of unexpected in my eyes. And if this needed to be
>> done, it should have been in a separate patch anyway.
> 
> I proposed that with a renaming, so it won't look as boolean.
> And I also implied (sorry if it was unclear) that this has to be
> done separately, so we are on the same page about this.
> 

Seems I misunderstand the comment from Andy. Will drop this change which should be
a separate one besides the fix. Will respin a v4 which only narrow the lock region
based on v2 (per Andy).

Thanks.