Question about arm64 earlycon

Question about arm64 earlycon

[Duc Dang]

Hi Catalin, Marc, Mark, Arnd,

I am testing with 3.12 kernel with earlyprintk enabled and I see some
garbage characters in the console log right before the message
indicating that the real console device is initialized:

<some garbage characters here>01c020000.serial: ttyS0 at MMIO
0x1c020000 (irq = 108, base_baud = 3125000) is a U6_16550A
console [ttyS0] enabled, bootconsole disabled

I looked through early_prink.c file and printk.c file and it looks
like there is case that some early boot code can touch the UART
hardware via ealy console driver while the 'real' console driver is
setting up the same UART port? Please let me know if I missed some
important piece of code that can prevent this.

Regards,
Duc Dang.

Question about arm64 earlycon

[Arnd Bergmann]

On Sunday, October 23, 2016 12:26:59 AM CEST Duc Dang wrote:
> Hi Catalin, Marc, Mark, Arnd,
> 
> I am testing with 3.12 kernel with earlyprintk enabled and I see some
> garbage characters in the console log right before the message
> indicating that the real console device is initialized:
> 
> <some garbage characters here>01c020000.serial: ttyS0 at MMIO
> 0x1c020000 (irq = 108, base_baud = 3125000) is a U6_16550A
> console [ttyS0] enabled, bootconsole disabled
> 
> I looked through early_prink.c file and printk.c file and it looks
> like there is case that some early boot code can touch the UART
> hardware via ealy console driver while the 'real' console driver is
> setting up the same UART port? Please let me know if I missed some
> important piece of code that can prevent this.

I don't think we every supported earlyprintk on arm64, and
earlycon support may have been added later.

If you can't use a modern kernel, try backporting all the
relevant earlycon changes.

	Arnd

Question about arm64 earlycon

[Marc Zyngier]

On 24/10/16 11:06, Arnd Bergmann wrote:
> On Sunday, October 23, 2016 12:26:59 AM CEST Duc Dang wrote:
>> Hi Catalin, Marc, Mark, Arnd,
>>
>> I am testing with 3.12 kernel with earlyprintk enabled and I see some
>> garbage characters in the console log right before the message
>> indicating that the real console device is initialized:
>>
>> <some garbage characters here>01c020000.serial: ttyS0 at MMIO
>> 0x1c020000 (irq = 108, base_baud = 3125000) is a U6_16550A
>> console [ttyS0] enabled, bootconsole disabled

Well, you get two independent, unsynchronized bits of code writing to
the same peripheral. Things are bound to go badly. Unless you're trying
to debug things, do not use earlyprintk in production.

>>
>> I looked through early_prink.c file and printk.c file and it looks
>> like there is case that some early boot code can touch the UART
>> hardware via ealy console driver while the 'real' console driver is
>> setting up the same UART port? Please let me know if I missed some
>> important piece of code that can prevent this.
> 
> I don't think we every supported earlyprintk on arm64, and
> earlycon support may have been added later.

We did support some form of earlyprintk for a while (though not in the
same way as 32bit ARM does), until Rob introduced earlycon.

> If you can't use a modern kernel, try backporting all the
> relevant earlycon changes.

Agreed. That's the most sensible course of action.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Question about arm64 earlycon

[Mark Rutland]

On Mon, Oct 24, 2016 at 11:17:36AM +0100, Marc Zyngier wrote:
> On 24/10/16 11:06, Arnd Bergmann wrote:
> > On Sunday, October 23, 2016 12:26:59 AM CEST Duc Dang wrote:
> >> Hi Catalin, Marc, Mark, Arnd,
> >>
> >> I am testing with 3.12 kernel with earlyprintk enabled and I see some
> >> garbage characters in the console log right before the message
> >> indicating that the real console device is initialized:

What exactly are you passing on the command line?

> >> <some garbage characters here>01c020000.serial: ttyS0 at MMIO
> >> 0x1c020000 (irq = 108, base_baud = 3125000) is a U6_16550A
> >> console [ttyS0] enabled, bootconsole disabled

Was the UART already configured by FW? Had the firmware output anything
at this point?

Did the firmware's UART rate match that of the kernel? If not, the issue
might just be that the rate doesn't match; earlycon/earlyprintk won't
configure that, while the real console will.

Or perhaps we race with some clock configuration...

> >> I looked through early_prink.c file and printk.c file and it looks
> >> like there is case that some early boot code can touch the UART
> >> hardware via ealy console driver while the 'real' console driver is
> >> setting up the same UART port? Please let me know if I missed some
> >> important piece of code that can prevent this.
> > 
> > I don't think we every supported earlyprintk on arm64, and
> > earlycon support may have been added later.
> 
> We did support some form of earlyprintk for a while (though not in the
> same way as 32bit ARM does), until Rob introduced earlycon.

Our earlyprintk up until that point was effectively a less general
earlycon (coming up at a similar time). In fact, in v3.12 we were
already using earlycon structures in arch/arm64/kernel/early_printk.c.

Thanks,
Mark.

Question about arm64 earlycon

[Duc Dang]

On Mon, Oct 24, 2016 at 4:09 AM, Mark Rutland <mark.rutland@arm.com> wrote:
>
> On Mon, Oct 24, 2016 at 11:17:36AM +0100, Marc Zyngier wrote:
> > On 24/10/16 11:06, Arnd Bergmann wrote:
> > > On Sunday, October 23, 2016 12:26:59 AM CEST Duc Dang wrote:
> > >> Hi Catalin, Marc, Mark, Arnd,
> > >>
> > >> I am testing with 3.12 kernel with earlyprintk enabled and I see some
> > >> garbage characters in the console log right before the message
> > >> indicating that the real console device is initialized:
>
> What exactly are you passing on the command line?

This is what I have:
earlyprintk=uart8250-32bit,0x1c020000

I should be more specific: the early console prints characters just
fine (I see all the early boot log). Only at the moment before
switching to the real console, I occasionally see some garbage
characters.
>
> > >> <some garbage characters here>01c020000.serial: ttyS0 at MMIO
> > >> 0x1c020000 (irq = 108, base_baud = 3125000) is a U6_16550A
> > >> console [ttyS0] enabled, bootconsole disabled
>
> Was the UART already configured by FW? Had the firmware output anything
> at this point?

Yes, UART was already configured by FW (U-Boot) and at this point,
U-Boot is already gone.

>
> Did the firmware's UART rate match that of the kernel? If not, the issue
> might just be that the rate doesn't match; earlycon/earlyprintk won't
> configure that, while the real console will.

The baud-rate does not change between U-Boot and kernel.

>
> Or perhaps we race with some clock configuration...
>
> > >> I looked through early_prink.c file and printk.c file and it looks
> > >> like there is case that some early boot code can touch the UART
> > >> hardware via ealy console driver while the 'real' console driver is
> > >> setting up the same UART port? Please let me know if I missed some
> > >> important piece of code that can prevent this.
> > >
> > > I don't think we every supported earlyprintk on arm64, and
> > > earlycon support may have been added later.
> >
> > We did support some form of earlyprintk for a while (though not in the
> > same way as 32bit ARM does), until Rob introduced earlycon.
>
> Our earlyprintk up until that point was effectively a less general
> earlycon (coming up at a similar time). In fact, in v3.12 we were
> already using earlycon structures in arch/arm64/kernel/early_printk.c.

Yes, early_printk.c for arm64 is supported until Rob removed it and
replaced by earlycon at kernel v3.16

>
> Thanks,
> Mark.
Regards,
Duc Dang.

Question about arm64 earlycon

[Duc Dang]

On Mon, Oct 24, 2016 at 3:17 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 24/10/16 11:06, Arnd Bergmann wrote:
>> On Sunday, October 23, 2016 12:26:59 AM CEST Duc Dang wrote:
>>> Hi Catalin, Marc, Mark, Arnd,
>>>
>>> I am testing with 3.12 kernel with earlyprintk enabled and I see some
>>> garbage characters in the console log right before the message
>>> indicating that the real console device is initialized:
>>>
>>> <some garbage characters here>01c020000.serial: ttyS0 at MMIO
>>> 0x1c020000 (irq = 108, base_baud = 3125000) is a U6_16550A
>>> console [ttyS0] enabled, bootconsole disabled
>
> Well, you get two independent, unsynchronized bits of code writing to
> the same peripheral. Things are bound to go badly. Unless you're trying
> to debug things, do not use earlyprintk in production.

This is what I am testing for now: not passing earlyprintk parameter,
and I don't see any weird character.

Is it also true about earlycon? We should not use either earlyprintk
(already gone long time ago) or earlycon for production?

>
>>>
>>> I looked through early_prink.c file and printk.c file and it looks
>>> like there is case that some early boot code can touch the UART
>>> hardware via ealy console driver while the 'real' console driver is
>>> setting up the same UART port? Please let me know if I missed some
>>> important piece of code that can prevent this.
>>
>> I don't think we every supported earlyprintk on arm64, and
>> earlycon support may have been added later.
>
> We did support some form of earlyprintk for a while (though not in the
> same way as 32bit ARM does), until Rob introduced earlycon.
>
>> If you can't use a modern kernel, try backporting all the
>> relevant earlycon changes.
>
> Agreed. That's the most sensible course of action.
>
> Thanks,
>
>         M.
> --
> Jazz is not dead. It just smells funny...
Regards,
Duc Dang.

Question about arm64 earlycon

[Mark Rutland]

On Mon, Oct 24, 2016 at 10:24:19AM -0700, Duc Dang wrote:
> On Mon, Oct 24, 2016 at 4:09 AM, Mark Rutland <mark.rutland@arm.com> wrote:
> >
> > On Mon, Oct 24, 2016 at 11:17:36AM +0100, Marc Zyngier wrote:
> > > On 24/10/16 11:06, Arnd Bergmann wrote:
> > > > On Sunday, October 23, 2016 12:26:59 AM CEST Duc Dang wrote:
> > > >> Hi Catalin, Marc, Mark, Arnd,
> > > >>
> > > >> I am testing with 3.12 kernel with earlyprintk enabled and I see some
> > > >> garbage characters in the console log right before the message
> > > >> indicating that the real console device is initialized:
> >
> > What exactly are you passing on the command line?
> 
> This is what I have:
> earlyprintk=uart8250-32bit,0x1c020000
> 
> I should be more specific: the early console prints characters just
> fine (I see all the early boot log). Only at the moment before
> switching to the real console, I occasionally see some garbage
> characters.

Sorry, I managed to miss that previously. Sorry for the noise.

As Marc said, it sounds like the problem is the lack of synchronisation
between the drivers at hand-over time. Perhaps there's a FIFO still
draining, or the "real" driver doesn't put the UART in a quiescent state
before reinitialising it.

Generally earlycon/earlyprintk is there to debug early boot issues, and
is not there just to get UART output earlier. If you don't need to debug
an issue, turning it off will avoid problems.

If there's a simple change to the "real" driver init sequence to avoid
the temporary glitching, that would be great, but not many people are
going to care. We'd rather keep the earlycon code simple so as to ensure
we can use it to debug early boot failures.

Thanks,
Mark.