Default ECHO on TTYs causes unwanted garbage chars

Default ECHO on TTYs causes unwanted garbage chars

[Joakim Tjernlund]

Hi List

I was advised to come here with this problem(started on the USB list).

We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
the device.

After much debugging I found that this are chars sent early in the boot process which then
are buffered and the TTYs default to ECHO chars is the cause.

When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
before one has a chance to disable ECHO. The device then thinks these chars are regular input.

Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
any chars so this does not happen.

What do you think?
 

Re: Default ECHO on TTYs causes unwanted garbage chars

[Greg KH]

On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> Hi List
> 
> I was advised to come here with this problem(started on the USB list).
> 
> We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> the device.
> 
> After much debugging I found that this are chars sent early in the boot process which then
> are buffered and the TTYs default to ECHO chars is the cause.

So some program in the boot sequence is trying to send data out the
device?  Why not just not do that?

> When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
> before one has a chance to disable ECHO. The device then thinks these chars are regular input.

Wait, you said something in the boot process did write to the device,
which would have caused the tty to be opened then, right?

> Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
> any chars so this does not happen.

Where are the characters coming from that would need to be flushed?
When should characters be flushed exactly?

thanks,

greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[Joakim Tjernlund]

On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> 
> On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > Hi List
> > 
> > I was advised to come here with this problem(started on the USB list).
> > 
> > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > the device.
> > 
> > After much debugging I found that this are chars sent early in the boot process which then
> > are buffered and the TTYs default to ECHO chars is the cause.
> 
> So some program in the boot sequence is trying to send data out the
> device?  Why not just not do that?

This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services

> 
> > When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
> > before one has a chance to disable ECHO. The device then thinks these chars are regular input.
> 
> Wait, you said something in the boot process did write to the device,
> which would have caused the tty to be opened then, right?

well, boot process of the device prints and it is enough for the USB cable to be attached but not opened by any app.
The device just see an UART and prints when UART is initialized.

> 
> > Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
> > any chars so this does not happen.
> 
> Where are the characters coming from that would need to be flushed?

Early output from boot, basically whatever prints just after connecting the USB cable.

> When should characters be flushed exactly?

Whatever is in the buffers before opening the tty.
The terminal app(like cu) tries to flush any input when it starts, just to avoid any old chars in the
queue but it is to late then.

 Jocke

> 
> thanks,
> 
> greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[gregkh]

On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
> On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> > 
> > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > > Hi List
> > > 
> > > I was advised to come here with this problem(started on the USB list).
> > > 
> > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > > the device.
> > > 
> > > After much debugging I found that this are chars sent early in the boot process which then
> > > are buffered and the TTYs default to ECHO chars is the cause.
> > 
> > So some program in the boot sequence is trying to send data out the
> > device?  Why not just not do that?
> 
> This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services

So the same device is used for boot console as well as a ttyACM device
later on?

> > > When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
> > > before one has a chance to disable ECHO. The device then thinks these chars are regular input.
> > 
> > Wait, you said something in the boot process did write to the device,
> > which would have caused the tty to be opened then, right?
> 
> well, boot process of the device prints and it is enough for the USB cable to be attached but not opened by any app.
> The device just see an UART and prints when UART is initialized.

What tool does that?  Why not fix that?

> > > Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
> > > any chars so this does not happen.
> > 
> > Where are the characters coming from that would need to be flushed?
> 
> Early output from boot, basically whatever prints just after connecting the USB cable.

Then don't have boot print to that device :)

> > When should characters be flushed exactly?
> 
> Whatever is in the buffers before opening the tty.

But what is supposed to happen to the data that was sent to it while it
was "closed"?

> The terminal app(like cu) tries to flush any input when it starts, just to avoid any old chars in the
> queue but it is to late then.

I strongly just suggest having userspace not write to the device to
start with, that would solve this, right?

thanks,

greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[Joakim Tjernlund]

On Tue, 2020-06-09 at 16:22 +0200, gregkh@linuxfoundation.org wrote:
> 
> On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
> > On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> > > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > > > Hi List
> > > > 
> > > > I was advised to come here with this problem(started on the USB list).
> > > > 
> > > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > > > the device.
> > > > 
> > > > After much debugging I found that this are chars sent early in the boot process which then
> > > > are buffered and the TTYs default to ECHO chars is the cause.
> > > 
> > > So some program in the boot sequence is trying to send data out the
> > > device?  Why not just not do that?
> > 
> > This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services
> 
> So the same device is used for boot console as well as a ttyACM device
> later on?

Not quite, the USB to RS232 chip is integrated on the device and is connected the CPUs RS232,
there is no other port.
I think you could compare with an external USB to RS232 puck. Senario:
- Connect the puck to both computer and your device with an RS232 port.
- Power on the device with the RS232 port.
- Device "boots" and prints stuff on its RS232 port,

some time passes

- Open ttyACM in PC using minicom/cu
Now early history of the boot prints are echoed back from PC to device with RS232

PS:
    Oliver, please help me make this clear. You sent me here :)

> 
> > > > When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
> > > > before one has a chance to disable ECHO. The device then thinks these chars are regular input.
> > > 
> > > Wait, you said something in the boot process did write to the device,
> > > which would have caused the tty to be opened then, right?
> > 
> > well, boot process of the device prints and it is enough for the USB cable to be attached but not opened by any app.
> > The device just see an UART and prints when UART is initialized.
> 
> What tool does that?  Why not fix that?
> 
> > > > Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
> > > > any chars so this does not happen.
> > > 
> > > Where are the characters coming from that would need to be flushed?
> > 
> > Early output from boot, basically whatever prints just after connecting the USB cable.
> 
> Then don't have boot print to that device :)
> 
> > > When should characters be flushed exactly?
> > 
> > Whatever is in the buffers before opening the tty.
> 
> But what is supposed to happen to the data that was sent to it while it
> was "closed"?
> 
> > The terminal app(like cu) tries to flush any input when it starts, just to avoid any old chars in the
> > queue but it is to late then.
> 
> I strongly just suggest having userspace not write to the device to
> start with, that would solve this, right?

It is not user space, it is the serial driver in kernel writing this back automatically.

 Jocke

> 
> thanks,
> 
> greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[Maarten Brock]

On 2020-06-09 17:01, Joakim Tjernlund wrote:
> On Tue, 2020-06-09 at 16:22 +0200, gregkh@linuxfoundation.org wrote:
>> 
>> On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
>> > On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
>> > > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
>> > > > Hi List
>> > > >
>> > > > I was advised to come here with this problem(started on the USB list).
>> > > >
>> > > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
>> > > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
>> > > > the device.
>> > > >
>> > > > After much debugging I found that this are chars sent early in the boot process which then
>> > > > are buffered and the TTYs default to ECHO chars is the cause.
>> > >
>> > > So some program in the boot sequence is trying to send data out the
>> > > device?  Why not just not do that?
>> >
>> > This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services
>> 
>> So the same device is used for boot console as well as a ttyACM device
>> later on?
> 
> Not quite, the USB to RS232 chip is integrated on the device and is
> connected the CPUs RS232,
> there is no other port.
> I think you could compare with an external USB to RS232 puck. Senario:
> - Connect the puck to both computer and your device with an RS232 port.
> - Power on the device with the RS232 port.
> - Device "boots" and prints stuff on its RS232 port,
> 
> some time passes
> 
> - Open ttyACM in PC using minicom/cu
> Now early history of the boot prints are echoed back from PC to device
> with RS232

It sounds like either the USB-RS232 device or the ttyACM driver are 
holding
the incoming data over RS232 even though there is no open connection 
over
USB. My first suspicion would be the USB-RS232 device.

Also, doesn't it help to open minicom with echo off?

Maarten

> 
> PS:
>     Oliver, please help me make this clear. You sent me here :)
> 
>> 
>> > > > When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
>> > > > before one has a chance to disable ECHO. The device then thinks these chars are regular input.
>> > >
>> > > Wait, you said something in the boot process did write to the device,
>> > > which would have caused the tty to be opened then, right?
>> >
>> > well, boot process of the device prints and it is enough for the USB cable to be attached but not opened by any app.
>> > The device just see an UART and prints when UART is initialized.
>> 
>> What tool does that?  Why not fix that?
>> 
>> > > > Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
>> > > > any chars so this does not happen.
>> > >
>> > > Where are the characters coming from that would need to be flushed?
>> >
>> > Early output from boot, basically whatever prints just after connecting the USB cable.
>> 
>> Then don't have boot print to that device :)
>> 
>> > > When should characters be flushed exactly?
>> >
>> > Whatever is in the buffers before opening the tty.
>> 
>> But what is supposed to happen to the data that was sent to it while 
>> it
>> was "closed"?
>> 
>> > The terminal app(like cu) tries to flush any input when it starts, just to avoid any old chars in the
>> > queue but it is to late then.
>> 
>> I strongly just suggest having userspace not write to the device to
>> start with, that would solve this, right?
> 
> It is not user space, it is the serial driver in kernel writing this
> back automatically.
> 
>  Jocke
> 
>> 
>> thanks,
>> 
>> greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[Joakim Tjernlund]

On Tue, 2020-06-09 at 23:26 +0200, Maarten Brock wrote:
> 
> On 2020-06-09 17:01, Joakim Tjernlund wrote:
> > On Tue, 2020-06-09 at 16:22 +0200, gregkh@linuxfoundation.org wrote:
> > > On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
> > > > On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> > > > > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > > > > > Hi List
> > > > > > 
> > > > > > I was advised to come here with this problem(started on the USB list).
> > > > > > 
> > > > > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > > > > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > > > > > the device.
> > > > > > 
> > > > > > After much debugging I found that this are chars sent early in the boot process which then
> > > > > > are buffered and the TTYs default to ECHO chars is the cause.
> > > > > 
> > > > > So some program in the boot sequence is trying to send data out the
> > > > > device?  Why not just not do that?
> > > > 
> > > > This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services
> > > 
> > > So the same device is used for boot console as well as a ttyACM device
> > > later on?
> > 
> > Not quite, the USB to RS232 chip is integrated on the device and is
> > connected the CPUs RS232,
> > there is no other port.
> > I think you could compare with an external USB to RS232 puck. Senario:
> > - Connect the puck to both computer and your device with an RS232 port.
> > - Power on the device with the RS232 port.
> > - Device "boots" and prints stuff on its RS232 port,
> > 
> > some time passes
> > 
> > - Open ttyACM in PC using minicom/cu
> > Now early history of the boot prints are echoed back from PC to device
> > with RS232
> 
> It sounds like either the USB-RS232 device or the ttyACM driver are
> holding
> the incoming data over RS232 even though there is no open connection
> over
> USB. My first suspicion would be the USB-RS232 device.

In theory it could be USB-R232 chip but I think not, I think USB-RS232 sends
over chars once the cable is connected and it gets stored on the PC TTY or USB
drivers.

If I hardcode ECHO to off on the PC side it works, but that is not a very viable solution. 

It's is not the USB ACM driver, talked with Oliver(the USB cdc_acm maintainer) and he
asked med to go here as it looks like a general TTY issue.

> 
> Also, doesn't it help to open minicom with echo off?

No, because this happens at open() before one has a chance to turn off ECHO.

> 
> Maarten
> 
> > PS:
> >     Oliver, please help me make this clear. You sent me here :)
> > 
> > > > > > When the TTY is opened, any chars in the this buffer is ECHOed back over USB to the device,
> > > > > > before one has a chance to disable ECHO. The device then thinks these chars are regular input.
> > > > > 
> > > > > Wait, you said something in the boot process did write to the device,
> > > > > which would have caused the tty to be opened then, right?
> > > > 
> > > > well, boot process of the device prints and it is enough for the USB cable to be attached but not opened by any app.
> > > > The device just see an UART and prints when UART is initialized.
> > > 
> > > What tool does that?  Why not fix that?
> > > 
> > > > > > Seems to me that this behaviour is unwanted in general and and app. should get a chance to flush/discard
> > > > > > any chars so this does not happen.
> > > > > 
> > > > > Where are the characters coming from that would need to be flushed?
> > > > 
> > > > Early output from boot, basically whatever prints just after connecting the USB cable.
> > > 
> > > Then don't have boot print to that device :)
> > > 
> > > > > When should characters be flushed exactly?
> > > > 
> > > > Whatever is in the buffers before opening the tty.
> > > 
> > > But what is supposed to happen to the data that was sent to it while
> > > it
> > > was "closed"?
> > > 
> > > > The terminal app(like cu) tries to flush any input when it starts, just to avoid any old chars in the
> > > > queue but it is to late then.
> > > 
> > > I strongly just suggest having userspace not write to the device to
> > > start with, that would solve this, right?
> > 
> > It is not user space, it is the serial driver in kernel writing this
> > back automatically.
> > 
> >  Jocke
> > 
> > > thanks,
> > > 
> > > greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[gregkh]

On Tue, Jun 09, 2020 at 03:01:14PM +0000, Joakim Tjernlund wrote:
> On Tue, 2020-06-09 at 16:22 +0200, gregkh@linuxfoundation.org wrote:
> > 
> > On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
> > > On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> > > > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > > > > Hi List
> > > > > 
> > > > > I was advised to come here with this problem(started on the USB list).
> > > > > 
> > > > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > > > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > > > > the device.
> > > > > 
> > > > > After much debugging I found that this are chars sent early in the boot process which then
> > > > > are buffered and the TTYs default to ECHO chars is the cause.
> > > > 
> > > > So some program in the boot sequence is trying to send data out the
> > > > device?  Why not just not do that?
> > > 
> > > This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services
> > 
> > So the same device is used for boot console as well as a ttyACM device
> > later on?
> 
> Not quite, the USB to RS232 chip is integrated on the device and is connected the CPUs RS232,
> there is no other port.
> I think you could compare with an external USB to RS232 puck. Senario:
> - Connect the puck to both computer and your device with an RS232 port.
> - Power on the device with the RS232 port.
> - Device "boots" and prints stuff on its RS232 port,
> 
> some time passes
> 
> - Open ttyACM in PC using minicom/cu
> Now early history of the boot prints are echoed back from PC to device with RS232

Ah, ok, so, are you sure that data isn't just "stuck" in the USB-serial
chip's buffers?  That's often the case with many devices as they are
tiny and dumb and try to do the right thing most of the time (like not
drop data that was sent to it.)

thanks,

greg k-h

Re: Default ECHO on TTYs causes unwanted garbage chars

[Joakim Tjernlund]

On Wed, 2020-06-10 at 09:24 +0200, gregkh@linuxfoundation.org wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> On Tue, Jun 09, 2020 at 03:01:14PM +0000, Joakim Tjernlund wrote:
> > On Tue, 2020-06-09 at 16:22 +0200, gregkh@linuxfoundation.org wrote:
> > > On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
> > > > On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> > > > > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > > > > > Hi List
> > > > > > 
> > > > > > I was advised to come here with this problem(started on the USB list).
> > > > > > 
> > > > > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > > > > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > > > > > the device.
> > > > > > 
> > > > > > After much debugging I found that this are chars sent early in the boot process which then
> > > > > > are buffered and the TTYs default to ECHO chars is the cause.
> > > > > 
> > > > > So some program in the boot sequence is trying to send data out the
> > > > > device?  Why not just not do that?
> > > > 
> > > > This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services
> > > 
> > > So the same device is used for boot console as well as a ttyACM device
> > > later on?
> > 
> > Not quite, the USB to RS232 chip is integrated on the device and is connected the CPUs RS232,
> > there is no other port.
> > I think you could compare with an external USB to RS232 puck. Senario:
> > - Connect the puck to both computer and your device with an RS232 port.
> > - Power on the device with the RS232 port.
> > - Device "boots" and prints stuff on its RS232 port,
> > 
> > some time passes
> > 
> > - Open ttyACM in PC using minicom/cu
> > Now early history of the boot prints are echoed back from PC to device with RS232
> 
> Ah, ok, so, are you sure that data isn't just "stuck" in the USB-serial
> chip's buffers?  That's often the case with many devices as they are
> tiny and dumb and try to do the right thing most of the time (like not
> drop data that was sent to it.)

not 100% no but even if it was, why would the PC echo back chars because the TTY gets opened?
I can see that that the chars are sent back by the PC, not just sure if the PC received them
at that same time or if the PC got them when the cable was connected, stored inside some driver and
written back once the TTY is opened, before one have had a chance to adjust baudrate/echo etc.?

  Jocke

Re: Default ECHO on TTYs causes unwanted garbage chars

[Joakim Tjernlund]

On Wed, 2020-06-10 at 09:42 +0200, Joakim Tjernlund wrote:
> On Wed, 2020-06-10 at 09:24 +0200, gregkh@linuxfoundation.org wrote:
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > 
> > 
> > On Tue, Jun 09, 2020 at 03:01:14PM +0000, Joakim Tjernlund wrote:
> > > On Tue, 2020-06-09 at 16:22 +0200, gregkh@linuxfoundation.org wrote:
> > > > On Tue, Jun 09, 2020 at 01:13:06PM +0000, Joakim Tjernlund wrote:
> > > > > On Tue, 2020-06-09 at 13:57 +0200, Greg KH wrote:
> > > > > > On Tue, Jun 09, 2020 at 11:38:49AM +0000, Joakim Tjernlund wrote:
> > > > > > > Hi List
> > > > > > > 
> > > > > > > I was advised to come here with this problem(started on the USB list).
> > > > > > > 
> > > > > > > We have a USB to RS232 bridge which presents itself as an ttyACM and the first connect after power on,
> > > > > > > we see some garbage chars transmitted back from USB host(PC) to out device which becomes input to
> > > > > > > the device.
> > > > > > > 
> > > > > > > After much debugging I found that this are chars sent early in the boot process which then
> > > > > > > are buffered and the TTYs default to ECHO chars is the cause.
> > > > > > 
> > > > > > So some program in the boot sequence is trying to send data out the
> > > > > > device?  Why not just not do that?
> > > > > 
> > > > > This is the boot console. Both u-boot and Linux prints a lot there, then init prints while starting services
> > > > 
> > > > So the same device is used for boot console as well as a ttyACM device
> > > > later on?
> > > 
> > > Not quite, the USB to RS232 chip is integrated on the device and is connected the CPUs RS232,
> > > there is no other port.
> > > I think you could compare with an external USB to RS232 puck. Senario:
> > > - Connect the puck to both computer and your device with an RS232 port.
> > > - Power on the device with the RS232 port.
> > > - Device "boots" and prints stuff on its RS232 port,
> > > 
> > > some time passes
> > > 
> > > - Open ttyACM in PC using minicom/cu
> > > Now early history of the boot prints are echoed back from PC to device with RS232
> > 
> > Ah, ok, so, are you sure that data isn't just "stuck" in the USB-serial
> > chip's buffers?  That's often the case with many devices as they are
> > tiny and dumb and try to do the right thing most of the time (like not
> > drop data that was sent to it.)
> 
> not 100% no but even if it was, why would the PC echo back chars because the TTY gets opened?
> I can see that that the chars are sent back by the PC, not just sure if the PC received them
> at that same time or if the PC got them when the cable was connected, stored inside some driver and
> written back once the TTY is opened, before one have had a chance to adjust baudrate/echo etc.?
> 
>   Jocke

Gentle reminder ..

It also occurs to to me that current TTY behaviour would also ECHO any chars received between opening
the TTY and turning ECHO off. The window is smaller but it is there.

 Jocke