Re: [PATCHSET] block: fix PIO cache coherency bug, take 2

[Russell King <rmk+lkml@arm.linux.org.uk>]

On Sun, Jun 04, 2006 at 09:44:44PM +0100, Russell King wrote:
> On Sun, Jun 04, 2006 at 12:41:19PM +0900, Tejun Heo wrote:
> > Russell, can you please verify arm's flush_kernel_dcache_page()?
> 
> That should be fine from a theoretical standpoint

I'll add to this statement that the cache flushing on ARM is only
ever required when the page ends up in userspace - if we're reading
a page into the page cache to throw it out via NFS or sendfile then
the cache flush is a complete waste of time.

Why is this?  Well, if the data has been written to the kernel mapping
by the CPU, it may be contained in the cache lines corresponding to
that mapping.  When that memory is passed to a network interface to
send, the network interface either reads data from the kernel mapping
via PIO (in which case it reads the data from the cache), or it
performs DMA.  In the case of DMA, the DMA API handles the cache
coherency issues with respect to dirty data in the kernel mapping.

Moreover, there's the problem of read-ahead.  When data is read from
a block device, more data than that which is required is usually read
in case it's required a short time later.  If the data is not required,
it is thrown away during an eviction cycle.  However, any cache flushing
that you've performed for uses "other than kernel space" is then a waste
of resources - the only time that such handling is needed is when the
data is actually used for these other uses - which in the case of ARM
means userspace.

Given the above, I believe that the method being proposed will be
_far_ too expensive, maybe to the point where (eg) disabling read-
ahead _entirely_ on ARM makes the system overall more efficient.

In this respect, I continue to believe that the way ARM (in principle)
does flush_dcache_page() is what is required here - if the page has
not been mapped into userspace, it merely marks the page as containing
dirty cache lines, and the resulting cache maintainence will only
happen when (and if) the page really does get mapped into userspace.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core