Re: [PATCH] xen/arm64: Use __flush_dcache_area instead of __flush_dcache_all

[Mark Rutland <mark.rutland@arm.com>]

Hi Roy,

[...]

> It seems that for Xen we do need to flush the FDT as well - I get a
> variety of crashes
> with a corrupt FDT when cache state is modeled on the FVP model, and
> Suravee sees similar
> behavior on Seattle. I was not expecting this, as I looked at the code
> in Xen and the caches/TLB
> are enabled quite early on, before the FDT is accessed by Xen.  I then
> looked at the mappings
> used by  edk2 and Xen, and found some differences.  Even after
> modifying edk2 to use the same
> configuration as Xen, the flushing of the FDT is still required. Xen
> and edk2 use the same memory
> attributes  in the MAIR_EL2 register (0xFF), but had different
> sharing, access perm, and nG settings.

I don't think the access perm or nG settings should have any effect, but
the shareability forms part of the memory attributes (along with the
memory type and cacheability), and there are several rules that apply
when accessing a memory location with mismatched attributes. See the
ARMv8 ARM - The AArch64 Application Level Memory Model - Mismatched
memory attributes.

In Linux we're likely getting lucky, and the shareability we use varies
for an SMP or UP kernel. So we need maintenance in at least one of those
cases. This would also apply to any initrd or other image.

Do you happen to know the shareability used by EDK2 and Xen?

> The flushing of the FDT seems to be required, but I'm not sure why.
> Does linux access the FDT with the
> same flat mapping used by edk2?  I think that Xen uses a different
> virtual mapping, so I suppose this
> could cause problems with a virtually tagged cache.  (I couldn't find
> a description of that detail regarding
> the caches.)  I'd really like to understand why this flush is required
> for Xen, and to make sure there
> there isn't other internal edk2 state that would also need flushing.

The D-caches should behave as if they are PIPT, so the virtual addresses
used should not be a problem. Linux maps the FDT in the swapper pgdir
rather than the idmap pgdir.

Linux might be doing some work that happens to flush the relevant
portions of the cache, even if accidentally, before accessing the FDT.

I would also like to understand what's going on here.

Thanks,
Mark.