[U-Boot] [linux-sunxi] Uboot error: address not aligned in v7_dcache_inval_range

[Marek Vasut <marex@denx.de>]

On Monday, April 28, 2014 at 09:55:46 PM, Ian Campbell wrote:
> On Sat, 2014-04-26 at 20:27 +0200, Marek Vasut wrote:
> > On Saturday, April 19, 2014 at 03:30:14 PM, Ian Campbell wrote:
> > > On Sun, 2014-04-13 at 23:45 -0400, Shixin Zeng wrote:
> > > > Hi,
> > > > 
> > > > I compiled the current u-boot from
> > > > https://github.com/jwrdegoede/u-boot-sunxi.git for cubieboard2, and
> > > > wrote it to the SD card. I was trying to boot the kernel on my
> > > > computer over network by tftp, however it failed when I ran "dhcp" or
> > > > "tftp" command in uboot with a tons of:
> > > > 
> > > > 
> > > > ERROR: v7_dcache_inval_range - start address is not aligned -
> > > > 0x7fb677e0 ERROR: v7_dcache_inval_range - stop address is not
> > > > aligned - 0x7fb67820
> > > 
> > > I'm seeing this on Cubieboard2 and Cubietruck. It appears to be down to
> > > a change to the upstream designware driver:
> > > 
> > > commit 50b0df814b0f75c08a3d45a017016a75af3edb5d
> > > Author: Alexey Brodkin <Alexey.Brodkin@synopsys.com>
> > > Date:   Wed Jan 22 20:49:09 2014 +0400
> > > 
> > >     net/designware: make driver compatible with data cache
> > >     
> > >     Up until now this driver only worked with data cache disabled.
> > >     To make it work with enabled data cache following changes were
> > > 
> > > required:
> > >      * Flush Tx/Rx buffer descriptors their modification
> > >      * Invalidate Tx/Rx buffer descriptors before reading its values
> > >      * Flush cache for data passed from CPU to GMAC
> > >      * Invalidate cache for data passed from GMAC to CPU
> > > 
> > > http://git.denx.de/?p=u-boot.git;a=commit;h=50b0df814b0f75c08a3d45a0170
> > > 16a7 5af3edb5d
> > > 
> > > I suppose this was only tested on some architecture which allows DMA
> > > flush/invaidation at a fairly fine granularity (at least down to 4 byte
> > > boundaries)
> > 
> > This was a sheer luck this ever worked. Looking at the entire driver, to
> > fix all your issues with DMA and caches, it would be sufficient to
> > re-align "struct dw_eth_dev" properly.
> > 
> > See drivers/net/designware.h:
> > 1) struct dmamacdescr {} is already __aligned(ARCH_DMA_MINALIGN)
> > 
> >    => This structure, if aligned in memory to proper boundary, can be
> >    flushed/
> >    
> >       invalidated without problems.
> > 
> > 2) struct dw_eth_dev {} can be aligned to ANY 4-byte boundary
> > 
> >    But this structure contains two arrays of struct dmamacdescr {} ,
> >    which each have their elements' lenght aligned to ARCH_DMA_MINALIGN
> > 
> > Solution:
> > 
> > Your patch [1/3] and reorder the structure in designware.h so that the
> > struct dmamacdescr tx_mac_descrtable[]
> > struct dmamacdescr rx_mac_descrtable[]
> > are first and anything that does not need to be aligned follows. This
> > way, the DMA descriptors will always be aligned and you need not worry
> > about the flushes. You don't even need to ROUNDUP their length, since
> > they are already fine.
> 
> Unfortunately this isn't sufficient, at least a change in the spirit of
> my second patch (to flush the entire descriptor) is also needed, because
> flushing just a subfield misaligns things again.

Ah, true. Your second patch is needed as well, sorry.

> And my patch 3/3 is still needed because it deals with the data itself
> and not the descriptors.

True.

> So having done all that it doesn't seem that reordering dw_eth_dev is
> necessary.

Reordering dw_eth_dev is necessary. Look:

108 struct dmamacdescr {                    // sizeof() = 0x10
109         u32 txrx_status;                // +0x0
110         u32 dmamac_cntl;                // +0x4
111         void *dmamac_addr;              // +0x8
112         struct dmamacdescr *dmamac_next;// +0xc
113 } __aligned(ARCH_DMA_MINALIGN);         // total size = 0x40

217 struct dw_eth_dev {
218         u32 interface;        // +0x0
219         u32 tx_currdescnum;   // +0x4
220         u32 rx_currdescnum;   // +0x8
221 
222         struct dmamacdescr tx_mac_descrtable[CONFIG_TX_DESCR_NUM];//+0xc

You do memalign() to allocate this. The .$interface ends up at address aligned 
to 64byte boundary (aka. it's cache aligned). Now, the structure is naturally 
aligned so tx_mac_descrtable[0] ends up at +0xc offset from the start of the 
structure , am I right ?

If I am wrong, then the compiler considers struct dmamacdescr {} as a one big 
chunk of data aligned to ARCH_DMA_MINALIGN boundary and thus inserts a big slop 
between rx_currdescnum and tx_mac_descrtable[0] to pad it correctly, which is 
not nice.

Reordering the structure will make sure there is no slop and there is no 
posibility of making tx_mac_descrtable unaligned ever.