serial8250 madness

serial8250 madness

[Andrew Walrond]

Is anyone active in the serial code?

I thought I would make some in-house servers send output via ttyS0 from 
grub, kernel boot and console, like our rented remote servers do. I set 
it all up and basically nothing worked properly, so I started 
investigating with a null modem cable, setserial and cu to see what was 
happening.

Machine A

ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0 
as I wanted.
ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc 
but just doesn't work at all.

Machine B

ttyS0 and ttyS1 will both receive, but neither will send. Go figure. 
Tested using a null modem cable, setserial and cu against the known good 
machine above.

Machine C

Grub and kernel outputs fine during boot, right up til the serial8250 
output appears in the kernel log, at which point it then prints 'LSR 
safety check engaged' and reboots.
Tried again without the console=ttyS0,57600 kernel parameter and it 
boots OK but starts printing the 'LSR...' message again as soon as 
agetty tries to attach to ttyS0. Doesn't reboot though...

The Machines are all amd64, various motherboards, identical distros 
using 2.6.28.1 kernel.
Either I have a particularly motley collection of kit, or the serial 
code is in a pretty bad way.

I can provide further info/testing if anyone is active/interested.

Andrew Walrond

Re: serial8250 madness

[David Daney]

Andrew Walrond wrote:
> Is anyone active in the serial code?
> 

Yes.

> [...]
> 
> The Machines are all amd64, various motherboards, identical distros 
> using 2.6.28.1 kernel.
> Either I have a particularly motley collection of kit, or the serial 
> code is in a pretty bad way.
> 

I use serial consoles on a variety of embedded boards every day (albeit 
mostly mips based) ,so I think it works.

You do have to have your kernel configured properly for a serial console 
for it to work right.

I would test it using some other device as the console.  Only once you 
have it working try for the serial console.

David Daney

Re: serial8250 madness

[Andrew Walrond]

David Daney wrote:
> You do have to have your kernel configured properly for a serial 
> console for it to work right.
>
Sorry to be blunt, but I don't think you read my message very carefully! 
I am reporting kernel bug(s) in the serial subsystem, not amateur 
configuration problems. I know what I'm about, but on a random selection 
of 3 servers with identical kernels/software, the serial subsystem works 
fully on none of them. All boot just fine to the normal console.

Andrew Walrond

Re: serial8250 madness

[David Daney]

Andrew Walrond wrote:
> David Daney wrote:
>> You do have to have your kernel configured properly for a serial 
>> console for it to work right.
>>
> Sorry to be blunt, but I don't think you read my message very carefully! 

I did, I read every word of it.  I am sorry I was not specific enough 
for you.  I really meant to ask:

Do you have: CONFIG_SERIAL_8250_CONSOLE=y


David Daney.

Re: serial8250 madness

[Andrew Walrond]

David Daney wrote:
> I did, I read every word of it.  I am sorry I was not specific enough 
> for you.  I really meant to ask:
>
> Do you have: CONFIG_SERIAL_8250_CONSOLE=y
>
"Machine A - ttyS0 works fine. Can get grub output, kernel boot and 
console via ttyS0 as I wanted."
 
So the whole chain works fine on this machine. This couldn't work 
without CONFIG_SERIAL_8250_CONSOLE or if I didn't know _how_ to do it.

But to prove it conclusively
 $ zcat /proc/config.gz | grep CONFIG_SERIAL_8250_CONSOLE 
CONFIG_SERIAL_8250_CONSOLE=y

Sorry for being snappy. Long day ;)

Andrew Walrond

Re: serial8250 madness

[Alan Cox]

> ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0 
> as I wanted.
> ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc 
> but just doesn't work at all.

You don't give much information but if it seems to be non working then I
would look hard at the IRQ/IO reported

> Machine B
> 
> ttyS0 and ttyS1 will both receive, but neither will send. Go figure. 
> Tested using a null modem cable, setserial and cu against the known good 
> machine above.

Boggle - what flow control wiring have you done ?

> Machine C
> 
> Grub and kernel outputs fine during boot, right up til the serial8250 
> output appears in the kernel log, at which point it then prints 'LSR 
> safety check engaged' and reboots.
> Tried again without the console=ttyS0,57600 kernel parameter and it 
> boots OK but starts printing the 'LSR...' message again as soon as 
> agetty tries to attach to ttyS0. Doesn't reboot though...
> 
> The Machines are all amd64, various motherboards, identical distros 
> using 2.6.28.1 kernel.
> Either I have a particularly motley collection of kit, or the serial 
> code is in a pretty bad way.

It is maintained and should be pretty solid. Seria console is one of the
less used bits but when it gets broken people do jump up and down and
shout about it.

The big outstanding bug I know about is the incompatibility between the
PnP 8250 drivers and serial console - that may be your machine C case.
That one is rather hard to fix so on the long rather than short term hit
list.

Alan

Re: serial8250 madness

[Andrew Walrond]

Alan Cox wrote:
>> ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0 
>> as I wanted.
>> ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc 
>> but just doesn't work at all.
>>     
>
> You don't give much information but if it seems to be non working then I
> would look hard at the IRQ/IO reported
>
>   
On Machine A (ttyS0 works fine. ttyS1 dead as dodo):
 $ dmesg | grep ttyS

Command line: console=tty0 console=ttyS0,57600
Kernel command line: console=tty0 console=ttyS0,57600
console [ttyS0] enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A

$ sudo ./setserial -a /dev/ttyS1

/dev/ttyS1, Line 1, UART: 16550A, Port: 0x02f8, IRQ: 3
    Baud_base: 115200, close_delay: 50, divisor: 0
    closing_wait: 3000
    Flags: spd_normal skip_test

user@orac ~ $ cat /proc/interrupts
           CPU0       CPU1      
  0:         35         20   IO-APIC-edge      timer
  1:          0          2   IO-APIC-edge      i8042
  3:          0          1   IO-APIC-edge   
  4:          0        179   IO-APIC-edge      serial
[snip]

>> Machine B
>>
>> ttyS0 and ttyS1 will both receive, but neither will send. Go figure. 
>> Tested using a null modem cable, setserial and cu against the known good 
>> machine above.
>>     
>
> Boggle - what flow control wiring have you done ?
>
>   
:) Same null modem cable as used in all other tests. Setup with 
setserial then connect either ends with the likes of
    cu -s 9600 -l /dev/ttyS?
and this machine can receive and display characters typed on the other 
end, but nothing gets sent the other way.
Have tried with hardware/software flow on/off at both ends. Clueless...

I havent tried yet but will see if grub will push anything out (ie 
independent of the kernel) which will rule out a hardware problem and 
implicate the kernel. But its a relatively new m/b with onboard serial 
so I doubt it's broken.
Will report back.
>
> The big outstanding bug I know about is the incompatibility between the
> PnP 8250 drivers and serial console - that may be your machine C case.
> That one is rather hard to fix so on the long rather than short term hit
> list.
>   
Sounds likely. Any way I can know for sure?
The kernel log looks a bit different on this machine:

Serial: 8250/16550 driver4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
serial 00:09: activated
00:09: ttyS2 at I/O 0x3e8 (irq = 3) is a 16550A
serial 00:0a: activated
00:0a: ttyS3 at I/O 0x2e8 (irq = 5) is a 16550A

compared with Machine A:

Serial: 8250/16550 driver4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A

Thanks for your input

Andrew Walrond
> Alan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>   

Re: serial8250 madness

[Alan Cox]

> Sounds likely. Any way I can know for sure?

Turn off the 8250 PnP support and retest is the best way to test that.

Re: serial8250 madness

[Clark Williams]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=US-ASCII, Size: 1372 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 23 Jan 2009 18:00:33 +0000
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> 
> The big outstanding bug I know about is the incompatibility between the
> PnP 8250 drivers and serial console - that may be your machine C case.
> That one is rather hard to fix so on the long rather than short term hit
> list.
> 
> Alan


I spend a few days trying to figure out a panic in the uart driver that
turned out to be this issue. Would it make sense to add the following?

diff --git a/drivers/serial/Kconfig b/drivers/serial/Kconfig
index 579d63a..35acc6b 100644
- --- a/drivers/serial/Kconfig
+++ b/drivers/serial/Kconfig
@@ -91,7 +91,7 @@ config SERIAL_8250_PCI
 
 config SERIAL_8250_PNP
        tristate "8250/16550 PNP device support" if EMBEDDED
- -       depends on SERIAL_8250 && PNP
+       depends on SERIAL_8250 && PNP && ! SERIAL_8250_CONSOLE
        default SERIAL_8250
        help
          This builds standard PNP serial support. You may be able to

Clark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAkl7L/MACgkQHyuj/+TTEp21NACgpQI89LMX5ZtJbVlMcpa9A/I0
6JkAoIa5Nwt4mvm6UJGzGkoUGKOCzn5V
=YUtB
-----END PGP SIGNATURE-----
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: serial8250 madness

[Bruce Ashfield]

On Fri, Jan 23, 2009 at 12:41 PM, Andrew Walrond <andrew@walrond.org> wrote:
> Is anyone active in the serial code?
>

Definitely. Collectively, I see boards from 4 major archs
boot and use a serial console on a daily basis. Including
a pretty decent set of x86 boards.

> I thought I would make some in-house servers send output via ttyS0 from
> grub, kernel boot and console, like our rented remote servers do. I set it
> all up and basically nothing worked properly, so I started investigating
> with a null modem cable, setserial and cu to see what was happening.
>
> Machine A
>
> ttyS0 works fine. Can get grub output, kernel boot and console via ttyS0 as
> I wanted.
> ttyS1 doesn't work at all. It's detected, enabled in the bios etc etc but
> just doesn't work at all.

Interesting. In the past the BIOS has normally given us
problems when trying to get grub to output via serial,
first contact often appears once the kernel is booting.

>
> Machine B
>
> ttyS0 and ttyS1 will both receive, but neither will send. Go figure. Tested
> using a null modem cable, setserial and cu against the known good machine
> above.
>
> Machine C
>
> Grub and kernel outputs fine during boot, right up til the serial8250 output
> appears in the kernel log, at which point it then prints 'LSR safety check
> engaged' and reboots.
> Tried again without the console=ttyS0,57600 kernel parameter and it boots OK
> but starts printing the 'LSR...' message again as soon as agetty tries to
> attach to ttyS0. Doesn't reboot though...

Haven't witnessed B or C, but have had plenty of flow control
problems with infrastructure that prevented serial from properly
displaying.

I've also seen a few minor issues in the kernel itself, but it's
normally some slightly different chip and/or polling mode changes
interfering with normal operation. None of which seem useful
here.

>
> The Machines are all amd64, various motherboards, identical distros using
> 2.6.28.1 kernel.
> Either I have a particularly motley collection of kit, or the serial code is
> in a pretty bad way.
>
> I can provide further info/testing if anyone is active/interested.

I'm not much help, but I thought some extra data
points *might* be of use.

Bruce

>
> Andrew Walrond
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: serial8250 madness

[Andrew Walrond]

Andrew Walrond wrote:
>
> Machine B
>
> ttyS0 and ttyS1 will both receive, but neither will send. Go figure. 
> Tested using a null modem cable, setserial and cu against the known 
> good machine above.
>
It's not just the linux kernel; grub also refuses to send anything using 
this board's serial devices, so I think the hardware/bios must be 
stuffed somehow.

And since this is an MSI board with 'Microsoft only Live Bios Update 
software (tm)' as the only bios upgrade option, it will be visiting the 
skip in short order.

I haven't seen this before; I hope it's not becoming a new fashion.

Andrew Walrond