Re: [PATCH] MIPS: CPU_LOONGSON2EF need software to maintain cache consistency

[Jiaxun Yang <jiaxun.yang@flygoat.com>]

On Tue, 26 May 2020 14:29:50 +0100
Robin Murphy <robin.murphy@arm.com> wrote:

> On 2020-05-26 14:01, Thomas Bogendoerfer wrote:
> > On Tue, May 26, 2020 at 08:40:28PM +0800, Lichao Liu wrote:  
> >> Loongson-2EF need software maintain cache consistency, So when
> >> using streaming DMA, software needs to maintain consistency.
> >>
> >> dma_map_single() is correct, but dma_unmap_single is wrong.
> >>
> >> The function call path:
> >> 'dma_unmap_single->dma_unmap_page_attrs->dma_direct_unmap_page->
> >>   dma_direct_sync_single_for_cpu->arch_sync_dma_for_cpu->
> >>   cpu_needs_post_dma_flush'
> >>
> >> In current version, 'cpu_needs_post_dma_flush' will return false
> >> at Loongon-2EF platform, and dma_unmap_single will not invalidate
> >> cache, driver may access wrong dma data.  
> > 
> > why should it ? CPU must not touch data while it's mapped for DMA.
> >   
> >> I don't know what's the exact meaning of "fill random cachelines
> >> with stale data at any time". I always think
> >> 'cpu_needs_post_dma_flush()' means whether this platform needs
> >> software to maintain cache consistency.  
> > 
> > this will only happen, if cpu speculates creates dirty cache lines
> > by speculation as R10k type of CPUs do.  
> 
> Will it? The usual pattern for this problem is that the CPU 
> speculatively fills a (clean) cache line after a DMA_FROM_DEVICE 
> operation has begun, but before the device has actually written to
> that part of the buffer. Thus a subsequent CPU read after the
> operation is complete can hit in the cache and return the previous
> data rather than the updated data that the device wrote. I don't know
> about MIPS specifically, but that can certainly happen on Arm.

Checked the manual of Loongson-2F again and I must admit the case may
happen on Loongson-2EF processor.

R4k manual didn't show the details of speculative policy but I think
that should be applied to all R4k like processors?

So what we need is a post Invalidate after DMA_FROM_DEVICE?
DMA to device should not affected at all.

The origin R10k errata looks like a different story. Both from and to
device is affected because cache line was incorrectly marked as dirty
and it may writeback old data to memory already modified by uncached
write even CPU doesn't perform any cached write to that line. 

See details at Page 22 of R10k UM[1].
Not sure if BMIPS here is the same case. 

Thanks.

[1]:http://www.ece.mtu.edu/faculty/rmkieckh/cla/4173/REFERENCES/MIPS-R10K-uman1.pdf

- Jiaxun

> 
> >> I found this problem in 4.19.90 kernel's ethernet driver,
> >> and this patch can fix this problem.  
> > 
> > if CPU isn't affected by the R10k speculation problem, it sounds
> > more like wrong usage of DMA api.  
> 
> Sure, explicit accesses to the mapped buffer would be a software
> error, but if legitimate non-overlapping accesses to other data
> nearby can trigger the CPU to speculatively fetch lines from the
> DMA-mapped region, that's generally beyond software's control.
> 
> Robin.