Re: [Qemu-devel] [PATCH v2] roms: Flush icache when writing roms to guest memory

[Scott Wood <scottwood@freescale.com>]

On Thu, 2013-12-12 at 10:29 +0100, Alexander Graf wrote:
> We use the rom infrastructure to write firmware and/or initial kernel
> blobs into guest address space. So we're basically emulating the cache
> off phase on very early system bootup.
> 
> That phase is usually responsible for clearing the instruction cache for
> anything it writes into cachable memory, to ensure that after reboot we
> don't happen to execute stale bits from the instruction cache.
> 
> So we need to invalidate the icache every time we write a rom into guest
> address space. We do not need to do this for every DMA since the guest
> expects it has to flush the icache manually in that case.

Linux may do this for all DMAs (or more accurately, for all pages that
it makes executable regardless of how it loaded the data), and it's
probably a reasonable assumption for virtio, but on real hardware the
guest may often be able to get away with just invalidating the icache,
without the dcache flush step.

Whether it's worth QEMU flushing in those cases (and finding a way to
exempt virtio), versus documenting it as a known limitation, is
debatable -- but I don't think you can say that the requirements are the
same as for real hardware (on platforms that require such cache
cleaning, and excluding hardware (if any) where DMA is injected directly
into data cache without passing through main memory).  IMHO it would be
safer to start with a policy of always cleaning the cache (on relevant
platforms -- it would be a no-op on x86) when QEMU modifies guest
memory, and then for performance carve out exceptions like virtio.

Breakpoints (and any other memory modifications that might be done by a
debugger) are another situation that requires cache cleaning.

-Scott