Cache Coherency in PIO Read/Write

Cache Coherency in PIO Read/Write

[Fajun Chen]

Hi,

We use sg/libata for data read/write test in ARM XScale platform (VIVT
cache).  We
experienced data miscompare issue during heavy testing on both DMA and
PIO read/write.  In this particular application, data buffer is
allocated inside sg using alloc_pages() and mmapped to user space.
 So essentially this buffer can be accessed by user space, kernel for
PIO and hardware for DMA.  After changing mmapped vma to noncache, we
haven't seen any data miscompare in DMA read/write, but data
miscompare in PIO read/write still exists sporadically.  I suspect
this is caused by kernel cache alias during PIO read/write.  What I
want to know is how the cache coherency is ensured in PIO read/write.
As an alternative, is there any way to make the kernel access to this
data buffer uncached?

Thanks,
Fajun

RE: Cache Coherency in PIO Read/Write

[Mark Odell]

For PIO mode transfers, you should not have cache coherency issues since
the CPU core is doing the read from/write to hardware.

> -----Original Message-----
> From: linux-ide-owner@vger.kernel.org 
> [mailto:linux-ide-owner@vger.kernel.org] On Behalf Of Fajun Chen
> Sent: Thursday, August 31, 2006 1:03 PM
> To: linux-ide@vger.kernel.org
> Cc: htejun@gmail.com
> Subject: Cache Coherency in PIO Read/Write
> 
> Hi,
> 
> We use sg/libata for data read/write test in ARM XScale 
> platform (VIVT cache).  We experienced data miscompare issue 
> during heavy testing on both DMA and PIO read/write.  In this 
> particular application, data buffer is allocated inside sg 
> using alloc_pages() and mmapped to user space.
>  So essentially this buffer can be accessed by user space, 
> kernel for PIO and hardware for DMA.  After changing mmapped 
> vma to noncache, we haven't seen any data miscompare in DMA 
> read/write, but data miscompare in PIO read/write still 
> exists sporadically.  I suspect this is caused by kernel 
> cache alias during PIO read/write.  What I want to know is 
> how the cache coherency is ensured in PIO read/write.
> As an alternative, is there any way to make the kernel access 
> to this data buffer uncached?
> 
> Thanks,
> Fajun
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-ide" in the body of a message to 
> majordomo@vger.kernel.org More majordomo info at  
> http://vger.kernel.org/majordomo-info.html
> 

Re: Cache Coherency in PIO Read/Write

[Mark Lord]

Mark Odell wrote:
> For PIO mode transfers, you should not have cache coherency issues since
> the CPU core is doing the read from/write to hardware.

One might think so, but cache lines are tagged with virtual addresses,
so using different virtual addresses (kernel vs. userspace) for the 
same data can result in problems like this.

There was a recent thread (this past winter, I believe) devoted to
trying to resolve this issue in 2.6.xx, but I don't remember the final outcome.

Cheers

Re: Cache Coherency in PIO Read/Write

[Matt Reimer]

On 8/31/06, Mark Lord <liml@rtr.ca> wrote:
> Mark Odell wrote:
> > For PIO mode transfers, you should not have cache coherency issues since
> > the CPU core is doing the read from/write to hardware.
>
> One might think so, but cache lines are tagged with virtual addresses,
> so using different virtual addresses (kernel vs. userspace) for the
> same data can result in problems like this.
>
> There was a recent thread (this past winter, I believe) devoted to
> trying to resolve this issue in 2.6.xx, but I don't remember the final outcome.

http://marc.theaimsgroup.com/?l=linux-arm-kernel&m=107384066503818&w=2
http://marc.theaimsgroup.com/?l=linux-arm-kernel&m=107391014528541&w=2
http://lists.arm.linux.org.uk/pipermail/linux-arm-kernel/2004-May/022262.html
http://marc.theaimsgroup.com/?l=linux-ide&m=113517364630085&w=2
http://lkml.org/lkml/2006/1/13/156 and its associated posts

There are probably others. I think Tejun would know best the status of
this issue.

Matt