What happened to the serial console on BeagleBoard with 2.6.28-rc2?

What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Nathan Monson]

I updated linux-omap git to 2.6.28-rc2, and now I've lost my serial
console.  The kernel seems to be working anyway, but without the
console I can't do much testing.

With DEBUG_LL on, here are the last lines I see during boot:

<6>io scheduler cfq registered (default)
<6>Serial: 8250/16550 driver4 ports, IRQ sharing enabled
<6>serial8250.0: ttyS0 at MMIO 0x4806a000 (irq = 72) is a ST16654
<6>serial8250.0: ttyS1 at MMIO 0x4806c000 (irq = 73) is a ST16654

My console is on ttyS2.  Does something in ttyS2 initialization break
the serial console?

- Nathan

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Rick Bronson]

  It looks like a race condition.  Somehow the 8250 ends up with the
LSB of the baud rate register set to 0xff when it's supposed to be
0x1a.  I'm still trying to figure it out :( The patch below gets it
working a little but there's still a problem after init runs.  Note
that you will need to undef DEBUG_LL in the config.

  Rick

--- linux-omap-2.6/drivers/serial/8250.c.~1~	2008-10-28 10:11:30.000000000 -0700
+++ linux-omap-2.6/drivers/serial/8250.c	2008-10-29 08:33:31.000000000 -0700
@@ -483,6 +483,7 @@ static inline int _serial_dl_read(struct
 /* Uart divisor latch write */
 static inline void _serial_dl_write(struct uart_8250_port *up, int value)
 {
+mdelay (2);
 	serial_outp(up, UART_DLL, value & 0xff);
 	serial_outp(up, UART_DLM, value >> 8 & 0xff);
 }

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Rick Bronson]

  Ignore the last patch I posted.

  I'm not sure this is the right fix for this problem but it seems to
work ;-)  Here is what's happening

  In drivers/serial/8250.c size_fifo() miscalculates the fifo size of
the 3rd UART.  The first 2 UART's calculations are okay.  The 3rd's
miscalculation causes the console port to be messed up.  When I tried
putting debug code in, the problem would sometimes go away (or come
back) depending on what debug code I put in.  So I thought maybe it
was the memory mapping type that was chosen for the UART io memory
range.  That's what led to this patch.  Try it out and tell me if it
works for you.

  I have to wonder if L4_WK_34XX_VIRT needs to be classified as
MT_MEMORY_SO also?

  Rick

--- linux-omap-2.6/arch/arm/mach-omap2/io.c.~1~	2008-10-28 10:10:25.000000000 -0700
+++ linux-omap-2.6/arch/arm/mach-omap2/io.c	2008-11-01 14:58:35.000000000 -0700
@@ -155,7 +155,7 @@ static struct map_desc omap34xx_io_desc[
 		.virtual	= L4_PER_34XX_VIRT,
 		.pfn		= __phys_to_pfn(L4_PER_34XX_PHYS),
 		.length		= L4_PER_34XX_SIZE,
-		.type		= MT_DEVICE
+		.type		= MT_MEMORY_SO  /* debug only */
 	},
 	{
 		.virtual	= L4_EMU_34XX_VIRT,

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[David Brownell]

On Saturday 01 November 2008, Rick Bronson wrote:
> @@ -155,7 +155,7 @@ static struct map_desc omap34xx_io_desc[
>                 .virtual        = L4_PER_34XX_VIRT,
>                 .pfn            = __phys_to_pfn(L4_PER_34XX_PHYS),
>                 .length         = L4_PER_34XX_SIZE,
> -               .type           = MT_DEVICE
> +               .type           = MT_MEMORY_SO  /* debug only */
>         },
>         {
>                 .virtual        = L4_EMU_34XX_VIRT,

Leaving only one map_desc[] entry left using MT_DEVICE not MT_MEMORY_SO...
I didn't try this, since I'm currently not hooking up that serial port,
but dmesg does show(*):

 Serial: 8250/16550 driver3 ports, IRQ sharing disabled
 serial8250.0: ttyS0 at MMIO 0x4806a000 (irq = 72) is a ST16654
 serial8250.0: ttyS1 at MMIO 0x4806c000 (irq = 73) is a ST16654
 serial8250.0: ttyS2 at MMIO 0x49020000 (irq = 74) is a ST16650V2
 console [ttyS2] enabled

Is that what you saw too?

- Dave

(*) Only the third port ("ttyS2") is physically wired of course...
    the second could be, but isn't.
 
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Rick Bronson]

> Leaving only one map_desc[] entry left using MT_DEVICE not MT_MEMORY_SO...
> I didn't try this, since I'm currently not hooking up that serial port,
> but dmesg does show(*):
> 
>  Serial: 8250/16550 driver3 ports, IRQ sharing disabled
>  serial8250.0: ttyS0 at MMIO 0x4806a000 (irq = 72) is a ST16654
>  serial8250.0: ttyS1 at MMIO 0x4806c000 (irq = 73) is a ST16654
>  serial8250.0: ttyS2 at MMIO 0x49020000 (irq = 74) is a ST16650V2
>  console [ttyS2] enabled
> 
> Is that what you saw too?

  Yep, that's what I saw, but only if I had DEBUG_LL=y.  Without that
I had nothing except "Uncompressing Linux........".  ST16650V2 is a
clue that something went awry in auto-detection of the type of 8250
serial port.  With the MT_MEMORY_SO change, I get:

Serial: 8250/16550 driver4 ports, IRQ sharing enabled                           
serial8250.0: ttyS0 at MMIO 0x4806a000 (irq = 72) is a ST16654                  
serial8250.0: ttyS1 at MMIO 0x4806c000 (irq = 73) is a ST16654                  
serial8250.0: ttyS2 at MMIO 0x49020000 (irq = 74) is a ST16654                  
console [ttyS2] enabled                                                         

  I guess we don't see this problem on the other 2 ports since they
are mapped MT_MEMORY_SO and not MT_DEVICE (L4_34XX_PHYS).

  Rick

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[David Brownell]

On Sunday 02 November 2008, Rick Bronson wrote:
> > Is that what you saw too?
> 
>   Yep, that's what I saw, but only if I had DEBUG_LL=y.  Without that
> I had nothing except "Uncompressing Linux........".  

I don't have DEBUG_LL=y ... if you had some kind of network
link configured over USB, could you SSH in?  (I can.)


>   I guess we don't see this problem on the other 2 ports since they
> are mapped MT_MEMORY_SO and not MT_DEVICE (L4_34XX_PHYS).

Hmm, inconsistency is to be avoided.  :)

I wonder what the root cause is ... is the driver missing
various barriers needed to make MT_DEVICE work?  Or is the
MT_DEVICE incorrect in the first place?  "git whatchanged"
didn't show any recent changes that seemed (on a quick
glance) to affect I/O at that level.  Maybe it was one of
the other TTY changes interacting here.

- Dave


--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] serial: Flush posted write on 34xx to make 8250 work (Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?)

[Tony Lindgren]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

* David Brownell <david-b@pacbell.net> [081102 11:26]:
> On Sunday 02 November 2008, Rick Bronson wrote:
> > > Is that what you saw too?
> > 
> >   Yep, that's what I saw, but only if I had DEBUG_LL=y.  Without that
> > I had nothing except "Uncompressing Linux........".  
> 
> I don't have DEBUG_LL=y ... if you had some kind of network
> link configured over USB, could you SSH in?  (I can.)
> 
> 
> >   I guess we don't see this problem on the other 2 ports since they
> > are mapped MT_MEMORY_SO and not MT_DEVICE (L4_34XX_PHYS).
> 
> Hmm, inconsistency is to be avoided.  :)
> 
> I wonder what the root cause is ... is the driver missing
> various barriers needed to make MT_DEVICE work?  Or is the
> MT_DEVICE incorrect in the first place?  "git whatchanged"
> didn't show any recent changes that seemed (on a quick
> glance) to affect I/O at that level.  Maybe it was one of
> the other TTY changes interacting here.

Yeah I don't know what may have changed, or have things just
gotten faster now with cortex since 2.6.27?

Anyways, this patch fixes the issue for me without marking
things strongly ordered. Rick, does this work for you?

Regards,

Tony

[-- Attachment #2: flush-posted-8250.patch --]
[-- Type: text/x-diff, Size: 906 bytes --]

>From 57eea529292418a20a119c5b18d7636cf44992ae Mon Sep 17 00:00:00 2001
From: Tony Lindgren <tony@atomide.com>
Date: Fri, 31 Oct 2008 16:47:42 -0700
Subject: [PATCH] serial: Flush posted write on 34xx to make 8250 work

On omap34xx, we need to flush posted writes for TX.

Signed-off-by: Tony Lindgren <tony@atomide.com>

diff --git a/drivers/serial/8250.c b/drivers/serial/8250.c
index 5f383d8..629a16c 100644
--- a/drivers/serial/8250.c
+++ b/drivers/serial/8250.c
@@ -413,6 +413,13 @@ serial_out(struct uart_8250_port *up, int offset, int value)
 
 	case UPIO_MEM:
 		writeb(value, up->port.membase + offset);
+#ifdef CONFIG_ARCH_OMAP
+		/* Flush posted write for TX */
+		if (cpu_is_omap34xx() && is_omap_port(up) &&
+			(save_offset == UART_TX || save_offset == UART_IER ||
+				save_offset == UART_LCR))
+			value = serial_in(up, UART_LCR); /* Safe to read */
+#endif
 		break;
 
 	case UPIO_RM9000:

Re: [PATCH] serial: Flush posted write on 34xx to make 8250 work (Re:

[Grazvydas Ignotas]

> does this work for you?
Works on Pandora, which had garbled text instead of no output at all.

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Tony Lindgren]

* Rick Bronson <rick@efn.org> [081102 10:07]:
> 
> > Leaving only one map_desc[] entry left using MT_DEVICE not MT_MEMORY_SO...
> > I didn't try this, since I'm currently not hooking up that serial port,
> > but dmesg does show(*):
> > 
> >  Serial: 8250/16550 driver3 ports, IRQ sharing disabled
> >  serial8250.0: ttyS0 at MMIO 0x4806a000 (irq = 72) is a ST16654
> >  serial8250.0: ttyS1 at MMIO 0x4806c000 (irq = 73) is a ST16654
> >  serial8250.0: ttyS2 at MMIO 0x49020000 (irq = 74) is a ST16650V2
> >  console [ttyS2] enabled
> > 
> > Is that what you saw too?
> 
>   Yep, that's what I saw, but only if I had DEBUG_LL=y.  Without that
> I had nothing except "Uncompressing Linux........".  ST16650V2 is a
> clue that something went awry in auto-detection of the type of 8250
> serial port.  With the MT_MEMORY_SO change, I get:
> 
> Serial: 8250/16550 driver4 ports, IRQ sharing enabled                           
> serial8250.0: ttyS0 at MMIO 0x4806a000 (irq = 72) is a ST16654                  
> serial8250.0: ttyS1 at MMIO 0x4806c000 (irq = 73) is a ST16654                  
> serial8250.0: ttyS2 at MMIO 0x49020000 (irq = 74) is a ST16654                  
> console [ttyS2] enabled                                                         
> 
>   I guess we don't see this problem on the other 2 ports since they
> are mapped MT_MEMORY_SO and not MT_DEVICE (L4_34XX_PHYS).

Rick, can you please reply with your Signed-off-by for your patch?

Let's push that one, then continue thinking about the so/device
issues.

Tony

Re: [PATCH] serial: Flush posted write on 34xx to make 8250 work (Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?)

[Ming Lei]

2008/11/3 Tony Lindgren <tony@atomide.com>:
> * David Brownell <david-b@pacbell.net> [081102 11:26]:
>> On Sunday 02 November 2008, Rick Bronson wrote:
>> > > Is that what you saw too?
>> >
>> >   Yep, that's what I saw, but only if I had DEBUG_LL=y.  Without that
>> > I had nothing except "Uncompressing Linux........".
>>
>> I don't have DEBUG_LL=y ... if you had some kind of network
>> link configured over USB, could you SSH in?  (I can.)
>>
>>
>> >   I guess we don't see this problem on the other 2 ports since they
>> > are mapped MT_MEMORY_SO and not MT_DEVICE (L4_34XX_PHYS).
>>
>> Hmm, inconsistency is to be avoided.  :)
>>
>> I wonder what the root cause is ... is the driver missing
>> various barriers needed to make MT_DEVICE work?  Or is the
>> MT_DEVICE incorrect in the first place?  "git whatchanged"
>> didn't show any recent changes that seemed (on a quick
>> glance) to affect I/O at that level.  Maybe it was one of
>> the other TTY changes interacting here.
>
> Yeah I don't know what may have changed, or have things just
> gotten faster now with cortex since 2.6.27?
>
> Anyways, this patch fixes the issue for me without marking
> things strongly ordered. Rick, does this work for you?

This patch still works for me.
Thanks!

>
> Regards,
>
> Tony
>



-- 
Lei Ming

Re: [PATCH] serial: Flush posted write on 34xx to make 8250 work (Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?)

[Tony Lindgren]

* Ming Lei <tom.leiming@gmail.com> [081103 04:59]:
> 2008/11/3 Tony Lindgren <tony@atomide.com>:
> > * David Brownell <david-b@pacbell.net> [081102 11:26]:
> >> On Sunday 02 November 2008, Rick Bronson wrote:
> >> > > Is that what you saw too?
> >> >
> >> >   Yep, that's what I saw, but only if I had DEBUG_LL=y.  Without that
> >> > I had nothing except "Uncompressing Linux........".
> >>
> >> I don't have DEBUG_LL=y ... if you had some kind of network
> >> link configured over USB, could you SSH in?  (I can.)
> >>
> >>
> >> >   I guess we don't see this problem on the other 2 ports since they
> >> > are mapped MT_MEMORY_SO and not MT_DEVICE (L4_34XX_PHYS).
> >>
> >> Hmm, inconsistency is to be avoided.  :)
> >>
> >> I wonder what the root cause is ... is the driver missing
> >> various barriers needed to make MT_DEVICE work?  Or is the
> >> MT_DEVICE incorrect in the first place?  "git whatchanged"
> >> didn't show any recent changes that seemed (on a quick
> >> glance) to affect I/O at that level.  Maybe it was one of
> >> the other TTY changes interacting here.
> >
> > Yeah I don't know what may have changed, or have things just
> > gotten faster now with cortex since 2.6.27?
> >
> > Anyways, this patch fixes the issue for me without marking
> > things strongly ordered. Rick, does this work for you?
> 
> This patch still works for me.
> Thanks!

Well the real problem has been found, there's a bug with the recent
ptebits patches, which causes section mappings to be just MT_MEMORY
instead of MT_DEVICE..

So let's put things on hold until we have a real fix for that on the
linux-arm-kernel mailing list, hopefully today.

Regards,

Tony

RE: [PATCH] serial: Flush posted write on 34xx to make 8250 work (Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?)

[Woodruff, Richard]

> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> owner@vger.kernel.org] On Behalf Of Tony Lindgren
> Well the real problem has been found, there's a bug with the recent
> ptebits patches, which causes section mappings to be just MT_MEMORY
> instead of MT_DEVICE..
>
> So let's put things on hold until we have a real fix for that on the
> linux-arm-kernel mailing list, hopefully today.

That makes a lot more sense.  It was odd to see a flurry of failures.

SO vs. DEVICE were occasional errors usually seen under irq stress. It never took down the serial port.  Mapping things a MEMORY surely would.

Regards,
Richard W.

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Rick Bronson]

Hi Tony,

  I tried your patch to drivers/serial/8250.c and it works fine.  I
was concerned that I would start seeing:

omapfb: omapfb blah blah interrupt 0x4000

  (sorry, can't remember the exact verbage) because I saw them at one
point during the many reboots I did while working on the serial port
problem.  But I didn't see any after about 20 reboots.

  I assume you mean the patch below but why do you want this one if
your 8250.c patch works?

  Rick


Signed-off-by: Rick Bronson <rick@efn.org>

--- linux-omap-2.6/arch/arm/mach-omap2/io.c.~1~	2008-10-28 10:10:25.000000000 -0700
+++ linux-omap-2.6/arch/arm/mach-omap2/io.c	2008-11-01 14:58:35.000000000 -0700
@@ -155,7 +155,7 @@ static struct map_desc omap34xx_io_desc[
 		.virtual	= L4_PER_34XX_VIRT,
 		.pfn		= __phys_to_pfn(L4_PER_34XX_PHYS),
 		.length		= L4_PER_34XX_SIZE,
-		.type		= MT_DEVICE
+		.type		= MT_MEMORY_SO  /* debug only */
 	},
 	{
 		.virtual	= L4_EMU_34XX_VIRT,


> Rick, can you please reply with your Signed-off-by for your patch?
> 
> Let's push that one, then continue thinking about the so/device
> issues.
> 
> Tony
> 

Re: What happened to the serial console on BeagleBoard with 2.6.28-rc2?

[Tony Lindgren]

* Rick Bronson <rick@efn.org> [081104 13:08]:
> Hi Tony,
> 
>   I tried your patch to drivers/serial/8250.c and it works fine.  I
> was concerned that I would start seeing:
> 
> omapfb: omapfb blah blah interrupt 0x4000
> 
>   (sorry, can't remember the exact verbage) because I saw them at one
> point during the many reboots I did while working on the serial port
> problem.  But I didn't see any after about 20 reboots.
> 
>   I assume you mean the patch below but why do you want this one if
> your 8250.c patch works?

Well the patch below, and my i2c-omap and 8250 patches should not be
needed any longer, see commit 6ae3983b779ea5ad87515630e9b9205b76f1e916.

Tony


>   Rick
> 
> 
> Signed-off-by: Rick Bronson <rick@efn.org>
> 
> --- linux-omap-2.6/arch/arm/mach-omap2/io.c.~1~	2008-10-28 10:10:25.000000000 -0700
> +++ linux-omap-2.6/arch/arm/mach-omap2/io.c	2008-11-01 14:58:35.000000000 -0700
> @@ -155,7 +155,7 @@ static struct map_desc omap34xx_io_desc[
>  		.virtual	= L4_PER_34XX_VIRT,
>  		.pfn		= __phys_to_pfn(L4_PER_34XX_PHYS),
>  		.length		= L4_PER_34XX_SIZE,
> -		.type		= MT_DEVICE
> +		.type		= MT_MEMORY_SO  /* debug only */
>  	},
>  	{
>  		.virtual	= L4_EMU_34XX_VIRT,
> 
> 
> > Rick, can you please reply with your Signed-off-by for your patch?
> > 
> > Let's push that one, then continue thinking about the so/device
> > issues.
> > 
> > Tony
> >