[U-Boot] i.MX51: FEC: Cache coherency problem?

[Albert ARIBAUD <albert.u.boot@aribaud.net>]

Hi David,

Le 20/07/2011 08:29, David Jander a ?crit :
> On Tue, 19 Jul 2011 14:10:48 +0200
> David Jander<david.jander@protonic.nl>  wrote:
>
>> On Tue, 19 Jul 2011 13:20:26 +0200
>> Wolfgang Denk<wd@denx.de>  wrote:
>>
>>> Dear David Jander,
>>>
>>> In message<20110719131744.403a81e6@archvile>  you wrote:
>>>>
>>>> Now I finally know what's wrong and am working on a proposed fix to make
>>>> this one driver cache-aware.
>>>
>>> I would just like to point out that these efforts are highly
>>> appreciated!
>>
>> Thanks.
>> I'll try my best, but I am running out of time, and it is still not as it
>> should. I still have trouble identifying the right place where receive buffer
>> descriptors should be cache-invalidated. At one point, a magic
>> flush_dcache_all() at a certain place in code made it work, but that can't be
>> entirely correct, and if I replace it with only the necessary ranges, it
>> doesn't work correctly anymore, so I guess this flush is also invalidating
>> something I need elsewhere :-(
>> I keep searching.... through the IMO fairly convoluted network code.
>
> Ok, that was not such a problem. I found few nice spots, where
> cache-maintenance code should go IMO:
>
> In fec_rx_task_enable() cleaned RX BD's should be flushed.
> In fec_tx_task_enable() dirty TX BD's should be flushed, as well as data
> buffers.
> In fec_send() Invalidate BD status fields before reading, also in the while
> loop.
> In fec_recv() Invalidate BD and data buffers just before every read.
>
> And of course align all buffers to cache-line size and pad around the end of
> each buffer to cache-line size.
>
> Problem: It doesn't work (tftp transfer stops after a few packets)! :-(
> I am giving up!
>
> Plan B: I'll now try to allocate all buffers in internal RAM (which is not
> cached AFAIK).
>
> Any ideas?

Yes, one: I had issues with the Marvell Ethernet adapter, which has DMA 
as well, not because of cache (it was not active at the time) but 
because of instruction reordering done by the compiler when optimizing, 
which resulted in out-of-order accesses to the descritpors and DMA 
registered, so that the DMA start was written before the descriptors 
were set up. The (proper and clean) solution was to introduce memory 
barriers at strategic points of the driver to ensure that specific DMA 
starts were done only after all writes to the descriptors (and possibly 
buffers).

See drivers/net/mvgbe.c, and llok for 'isb()' instructions.

> Best regards,

Amicalement,
-- 
Albert.