Re: [PATCH] xen/arm64: Use __flush_dcache_area instead of __flush_dcache_all

[Ian Campbell <ian.campbell@citrix.com>]

On Tue, 2014-10-14 at 10:21 +0100, Mark Rutland wrote:
> Hi Roy,
> 
> [...]
> 
> > It seems that for Xen we do need to flush the FDT as well - I get a
> > variety of crashes
> > with a corrupt FDT when cache state is modeled on the FVP model, and
> > Suravee sees similar
> > behavior on Seattle. I was not expecting this, as I looked at the code
> > in Xen and the caches/TLB
> > are enabled quite early on, before the FDT is accessed by Xen.  I then
> > looked at the mappings
> > used by  edk2 and Xen, and found some differences.  Even after
> > modifying edk2 to use the same
> > configuration as Xen, the flushing of the FDT is still required. Xen
> > and edk2 use the same memory
> > attributes  in the MAIR_EL2 register (0xFF), but had different
> > sharing, access perm, and nG settings.
> 
> I don't think the access perm or nG settings should have any effect, but
> the shareability forms part of the memory attributes (along with the
> memory type and cacheability), and there are several rules that apply
> when accessing a memory location with mismatched attributes. See the
> ARMv8 ARM - The AArch64 Application Level Memory Model - Mismatched
> memory attributes.
> 
> In Linux we're likely getting lucky, and the shareability we use varies
> for an SMP or UP kernel. So we need maintenance in at least one of those
> cases. This would also apply to any initrd or other image.
> 
> Do you happen to know the shareability used by EDK2 and Xen?

Xen maps everything inner-shareable. Dunno about EDK2.

Is the real issue here not a lack of specification for some corner cases
of the boot protocol? Can we get that fixed somehow?

Part of me wants to suggest that UEFI (and bootloaders generally) ought
to be cleaning caches for anything they have loaded into RAM before
launching an OS as a matter of good hygiene.

Ian.

> 
> > The flushing of the FDT seems to be required, but I'm not sure why.
> > Does linux access the FDT with the
> > same flat mapping used by edk2?  I think that Xen uses a different
> > virtual mapping, so I suppose this
> > could cause problems with a virtually tagged cache.  (I couldn't find
> > a description of that detail regarding
> > the caches.)  I'd really like to understand why this flush is required
> > for Xen, and to make sure there
> > there isn't other internal edk2 state that would also need flushing.
> 
> The D-caches should behave as if they are PIPT, so the virtual addresses
> used should not be a problem. Linux maps the FDT in the swapper pgdir
> rather than the idmap pgdir.
> 
> Linux might be doing some work that happens to flush the relevant
> portions of the cache, even if accidentally, before accessing the FDT.
> 
> I would also like to understand what's going on here.
> 
> Thanks,
> Mark.