Re: 2.6.13-rc3: cache flush missing from somewhere

[Catalin Marinas <catalin.marinas@arm.com>]

"David S. Miller" <davem@davemloft.net> wrote:
> The "lazy dcache flushing" he mentioned only flushes on the
> processor where the store occurred, not on any other cpus.
>
> He took the sparc64 code which, at the time of the flush_dcache_page()
> call, stores the current cpu number in the page->flags and sets a
> bit indicating a flush is needed.  When some condition occurs
> requiring the delayed flush to occur, we look at the cpu number
> in the page and ask that specific cpu to do the flush.

That's a point I missed. The D-cache flushing should take place on the
CPU that wrote the page, not the one that got the page fault (and the
I-cache invalidation on all the CPUs). I don't see why this wouldn't
work.

> I've seen implementations where the I-cache does not snoop local cpu
> stores, but I've never seen one where other cpus do not snoop such
> stores.

On this ARM SMP implementation, only the D-cache snoops the other CPUs
stores, not the I-cache.

> You _HAVE_ to implement handling of I-cache update on L2
> cache line changes to handle updates from devices doing DMA, so why
> in the world special case stores done by other cpus?
>
> It almost sounds impossible to implement this and have the I-cache
> be coherent wrt. DMA transactions.

Shouldn't flush_dcache_page() be called anyway when a page is modified
by the kernel (or by a device via DMA)? With the Harvard cache
architecture in ARM, the I cache should be invalidated even in a
uniprocessor system. For SMP it is just a matter of invalidating it on
all the CPUs (done by issuing an inter-processor interrupt).

> Do you have to flush the whole I-cache every time some device DMAs
> a page into memory, before you can execute instructions out of it?

IMHO, it only needs invalidating the I-cache corresponding to the
DMA'ed page, not the whole I-cache but, as I said, this should be
handled by flush_dcache_page, whether lazily or not.

Thanks,

-- 
Catalin