From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marek Vasut Date: Wed, 10 Jul 2013 15:38:30 +0200 Subject: [U-Boot] [RFC][PATCH] arm: Fix flush_dcache_range() on arm926 In-Reply-To: <20130710145812.2a6c5d08@lilith> References: <1373416229-5746-1-git-send-email-marex@denx.de> <20130710145812.2a6c5d08@lilith> Message-ID: <201307101538.30567.marex@denx.de> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de Hi Albert, > Hi Marek, > > On Wed, 10 Jul 2013 02:30:29 +0200, Marek Vasut wrote: > > The flush_dcache_range() on arm926 did not work as expected on i.MX28. > > > > This can be observed during the operation of the FEC ethernet driver > > where the driver did occasionally fail with timeout trying to transmit > > a frame. The FEC ethernet driver uses DMA for transmitting the frame in > > the following fashion: > > > > 0) Set bits in DMA descriptor > > 1) Write DMA descriptor into aligned DRAM address > > 2) Flush D-Cache over the descriptor > > 3) Start DMA > > 4) Invalidate D-Cache over the descriptor > > 5) Test if certain bits in DMA descriptor are unset > > > > Very not so often it happened that the bits in the DMA descriptor were > > still set even after hardware register -- which is not cached -- > > indicated otherwise. > > > > Here I will theoreticise, Albert, can you please correct me if I'm wrong? > > > > This leads me to believe the DMA descriptor was still in the cacheline > > after being flushed in step 2) and during the DMA gets evicted into DRAM > > therefore corrupting the result of readback in 5) . By reading the ARM926 > > datasheet DDI0198E_arm926ejs_r0p5_trm.pdf page 50 table 2-17, it is not > > clear whether cacheline that is Valid+Clean will be invalidated in the > > D-Cache using the "mcr p15, 0, , c7, c14, 1" instruction or whether > > only Valid+Dirty lines are cleaned+invalidated. The other thing that is > > unclear to me is whether a cacheline that is Valid+Clear is written back > > into DRAM when it is evicted from cache. > > The way I see table 2-17, the "if valid and dirty" condition only > applies to the "write" part of the description, not to the "marked not > valid" one; this reading makes the most sense and is the most consistent > with the cache functioning as a whole. So basically all Valid+Dirty get cleaned . Nothing is done on Valid+Clean . Finally, as they are now all Valid+Clean , they all get Invalidated . This is how it'd make the most sense, but I'm afraid this is not what I observe. > As for a valid and clean (rather than "clear", I assume) line being > written if evicted, I think it is not. Eviction is not flushing: the > line is kicked out, plain and brutal. And even if we're talking about > flushing, if the line is clean then it is coherent with the RAM area > it caches, and writing it back there is unneeded. Yes, this _does_ make full sense to me. > > Interestingly enough, running invalidate_dcache_range() after doing the > > flush_dcache_range() over the descriptor solved the problem and I see no > > occasional timeout anymore. This confirms my opinion that the descriptor > > might remain in the cache and be written back during the DMA operation. > > Can you point me to the source code location(s) where the sequence > above is implemented? See: 738 flush_dcache_range(addr, addr + size); in drivers/net/fec_mxc.c . If I put invalidate_dcache_range(addr, addr + size); , all seems to work well. Note it also happens with multiple compilers, Debian gcc 4.8.0, ELDK 5.2 and ELDK 5.3 . Best regards, Marek Vasut