Re: ARM A53 and initial MMU mapping for EL0/1/2/3 ?

[Andre Przywara <andre.przywara@arm.com>]

On Wed, 9 Feb 2022 18:07:24 +0000
Joakim Tjernlund <Joakim.Tjernlund@infinera.com> wrote:

Hi,

> On Wed, 2022-02-09 at 13:13 +0000, Andre Przywara wrote:
> > On Wed,  9 Feb 2022 14:05:57 +0100
> > Michael Walle <michael@walle.cc> wrote:
> > 
> > Hi,
> >   
> > > > > The problem I have is that I boot a custom SOC into u-boot and when u-boot tries
> > > > > to boot linux I get an error exception when u-boot calls armv8_switch_to_el2 to enter linux.    
> > > > 
> > > > So that means that U-Boot runs in EL3, is that the first and only firmware
> > > > that you run? I think the EL3 part of U-Boot is not widely used and tested
> > > > beyond the very few platforms that use it.    
> > > 
> > > FWIW, the u-boot on the kontron_sl28 (which is a A72 core) is running
> > > in EL3. But it's a layerscape so it might be worth looking in
> > > arch/arm/cpu/armv8/fsl-layerscale/ if there is anything special.  
> > 
> > Thanks Michael, that's what I meant: the generic EL3 code in
> > arch/arm/cpu/armv8 might depend on more platform specific code to make it
> > work.  
> 
> Thanks guys, will look at these boards. 
> I have learnt that EL2 state in our design may be broken

How so? The exception levels itself are completely capsuled in the CPU
core, so Arm Cortex-A53, in your case. And EL2 surely runs happily there. I
wouldn't know of any peripherals being affected by the EL the core is
running in (ignoring secure state for now).

So what is the problem, exactly? The only EL2 problems I heard of were
firmware related (not setting up EL2 correctly), but we are looking
at the firmware here.

> so I started to look at going directly to EL1
> instead, is that a bad idea?

Please don't do this. For a start, I don't think it's necessary, and also
you are leaving the architectural trails here. In particular you are
getting into trouble with PSCI. Sure, you can somehow hack this up, but it
will stay a hack and it will come back and bite you later. Promised ;-)

> In u-boot I find:
> 
> #ifdef CONFIG_ARMV8_SWITCH_TO_EL1
> static void switch_to_el1(void)
> {
> 	if ((IH_ARCH_DEFAULT == IH_ARCH_ARM64) &&
> 	    (images.os.arch == IH_ARCH_ARM))
> 		armv8_switch_to_el1(0, (u64)gd->bd->bi_arch_number,
> 				    (u64)images.ft_addr, 0,
> 				    (u64)images.ep,
> 				    ES_TO_AARCH32);

Last time I tried this, for an experiment, this part was broken, it sets up
SCTLR wrongly, making the Linux kernel decompressor hang. I am tempted to
ask for removal, since it has architectural problems with PSCI anyway.

> 	else
> 		armv8_switch_to_el1((u64)images.ft_addr, 0, 0, 0,
> 				    images.ep,
> 				    ES_TO_AARCH64);

And this has issues with PSCI as well, since we now skip one EL, which the
spec does not allow.

Cheers,
Andre

> }
> #endif
> ....
> #ifdef CONFIG_ARMV8_SWITCH_TO_EL1
> 		armv8_switch_to_el2((u64)images->ft_addr, 0, 0, 0,
> 				    (u64)switch_to_el1, ES_TO_AARCH64);
> #else
> .....
> 
> which looks promising but when I look closer I see that this seems to do:
> EL3 -> EL1 -> EL2 
> Really? What is the point ? Looks like I need to hack something myself to go EL3->EL1
> or am I missing something ?
> 
>  Jocke 
>