[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Matthias Weißer]

Hi

I just activated the dcache on our jadecpu board but it seems to me that 
this breaks some commands.

jade> dcache
Data (writethrough) Cache is OFF
jade> usb start
(Re)start USB...
USB:   scanning bus for devices... 2 USB Device(s) found
        scanning bus for storage devices... 1 Storage Device(s) found
jade> dcache on
Data (writethrough) Cache is ON
jade> usb start
(Re)start USB...
USB:   scanning bus for devices... ERROR: CTL:TIMEOUT
2 USB Device(s) found
        scanning bus for storage devices... 0 Storage Device(s) found
jade> dcache off
Data (writethrough) Cache is OFF
jade> usb start
(Re)start USB...
USB:   scanning bus for devices... 2 USB Device(s) found
        scanning bus for storage devices... 1 Storage Device(s) found
jade>

Same goes for tftpboot on this machine and also on a imx25 based system 
which is currently not in mainline. As there is a timeout involved I had 
the timer implementation under suspicion where some static variables may 
be used before relocation but moving them to gd_t didn't help.

Has anyone an explanation for this behavior? Is anyone out there having 
dcache running on an ARM926 and working usb/tftpboot?

Many thanks,
Matthias

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Wolfgang Denk]

Dear =?ISO-8859-15?Q?Matthias_Wei=DFer?=,

In message <4CF8FE3E.9010105@arcor.de> you wrote:
> 
> I just activated the dcache on our jadecpu board but it seems to me that 
> this breaks some commands.

I am not surprised.

> Has anyone an explanation for this behavior? Is anyone out there having 
> dcache running on an ARM926 and working usb/tftpboot?

Many drivers have not been written to work with enabled caches.

As far as USB is concerned, you might be lucky that your system usies
a EHCI controller, so setting CONFIG_EHCI_DCACHE should help.

As for the network driver, you will probably have to figure out
yourself how to fix that.

Patches welcome.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
When all is said and done, more is said than done.

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Matthias Weißer]

Hello Wolfgang

Am 03.12.2010 15:47, schrieb Wolfgang Denk:
>> Has anyone an explanation for this behavior? Is anyone out there having
>> dcache running on an ARM926 and working usb/tftpboot?
>
> Many drivers have not been written to work with enabled caches.

What is the reason that special handling is needed when dcache is 
enabled? If a driver doesn't use any DMA there should be no need as the 
dcache is only enabled for the RAM and not for any memory mapped IO if I 
understand the code in arch/arm/lib/cache-cp15.c right.

> As far as USB is concerned, you might be lucky that your system usies
> a EHCI controller, so setting CONFIG_EHCI_DCACHE should help.

No, only OHCI.

> As for the network driver, you will probably have to figure out
> yourself how to fix that.

As the memory mapped network controller (SMSC9221) is not cached it 
shouldn't be a problem or do I miss something here?

> Patches welcome.

If someone can give me a little push into the right direction I will try 
but currently I have no idea what to do besides activating the cache 
right before the default "bootm" booting.

Thanks a lot
Matthias

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Wolfgang Denk]

Dear =?ISO-8859-1?Q?Matthias_Wei=DFer?=,

In message <4CF90819.7040904@arcor.de> you wrote:
> 
> >> Has anyone an explanation for this behavior? Is anyone out there having
> >> dcache running on an ARM926 and working usb/tftpboot?
> >
> > Many drivers have not been written to work with enabled caches.
> 
> What is the reason that special handling is needed when dcache is 
> enabled? If a driver doesn't use any DMA there should be no need as the 
> dcache is only enabled for the RAM and not for any memory mapped IO if I 
> understand the code in arch/arm/lib/cache-cp15.c right.

On ARM, device write accesses are typically just store instructions
(in C: assignments to a volatile pointer). With caches on, these
accesses will be - guess what? cached, i. e. they are NOT written to
the device, at least not immediately.  And if you repeatedly read a
register (like when polling for some status bit to change) these
accesses will be cached, too.

You need to make sure that caches are properly flushed / invalidated
at the right points.

> > As far as USB is concerned, you might be lucky that your system usies
> > a EHCI controller, so setting CONFIG_EHCI_DCACHE should help.
> 
> No, only OHCI.

Bad luck.

> As the memory mapped network controller (SMSC9221) is not cached it 
> shouldn't be a problem or do I miss something here?

You said you had enabled the data cache, so why do you think these
accesses are not cached?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Blast medicine anyway!  We've learned to tie into every organ in the
human body but one.  The brain!  The brain is what life is all about.
	-- McCoy, "The Menagerie", stardate 3012.4

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Albert ARIBAUD]

Le 03/12/2010 16:33, Wolfgang Denk a ?crit :

>> What is the reason that special handling is needed when dcache is
>> enabled? If a driver doesn't use any DMA there should be no need as the
>> dcache is only enabled for the RAM and not for any memory mapped IO if I
>> understand the code in arch/arm/lib/cache-cp15.c right.
>
> On ARM, device write accesses are typically just store instructions
> (in C: assignments to a volatile pointer). With caches on, these
> accesses will be - guess what? cached, i. e. they are NOT written to
> the device, at least not immediately.  And if you repeatedly read a
> register (like when polling for some status bit to change) these
> accesses will be cached, too.

In addition to making sure that register reads/write are not bitten by 
caching, note that some controllers have DMA capabilities which require 
proper cache handling for DMA memory buffers -- typically flushing them 
from cache before a DMA to the device, and invalidating their cache 
entries after a DMA from the device.

> Best regards,
>
> Wolfgang Denk

Amicalement,
-- 
Albert.

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Matthias Weißer]

Hello Wolfgang

Am 03.12.2010 16:33, schrieb Wolfgang Denk:
> Dear =?ISO-8859-1?Q?Matthias_Wei=DFer?=,
> 
> In message <4CF90819.7040904@arcor.de> you wrote:
>>
>>>> Has anyone an explanation for this behavior? Is anyone out there having
>>>> dcache running on an ARM926 and working usb/tftpboot?
>>>
>>> Many drivers have not been written to work with enabled caches.
>>
>> What is the reason that special handling is needed when dcache is 
>> enabled? If a driver doesn't use any DMA there should be no need as the 
>> dcache is only enabled for the RAM and not for any memory mapped IO if I 
>> understand the code in arch/arm/lib/cache-cp15.c right.
> 
> On ARM, device write accesses are typically just store instructions
> (in C: assignments to a volatile pointer). With caches on, these
> accesses will be - guess what? cached, i. e. they are NOT written to
>
> the device, at least not immediately.  And if you repeatedly read a
> register (like when polling for some status bit to change) these
> accesses will be cached, too.

I understand this behavior. But it is only true if the memory area in
question is marked as cacheable.

>> As the memory mapped network controller (SMSC9221) is not cached it 
>> shouldn't be a problem or do I miss something here?
> 
> You said you had enabled the data cache, so why do you think these
> accesses are not cached?

Please see arch/arm/lib/cache-cp15.c
The code there creates 4096 page table entries (1MB each) for the whole
4GB address space and initializes each entry in a way that it is not
cacheable (mmu_setup():71). It then changes the page table entries which
are pointing to a RAM area to make these, and only these, cacheable
(dram_bank_mmu_setup():57).

If the whole address space would be cached I would expect that even
writing to NOR flash fails as the write accesses wouldn't reach the
flash chip. But that works perfect on both of my systems.

Thanks
Matthias Wei?er

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Matthias Weißer]

Am 03.12.2010 17:34, schrieb Albert ARIBAUD:
> In addition to making sure that register reads/write are not bitten by 
> caching, note that some controllers have DMA capabilities which require 
> proper cache handling for DMA memory buffers -- typically flushing them 
> from cache before a DMA to the device, and invalidating their cache 
> entries after a DMA from the device.

This is true. DMA and caching can be a lot of fun for a driver developer
:-) But I can guarantee that the network driver doesn't use any DMA
transfer as the hardware doesn't support it.

Thanks
Matthias Wei?er

[U-Boot] arm: Activating dcache breaks 'usb start' and 'tftpboot' on jadecpu

[Wolfgang Denk]

Dear =?ISO-8859-1?Q?Matthias_Wei=DFer?=,

In message <4CF91E29.7000906@arcor.de> you wrote:
> 
> > You said you had enabled the data cache, so why do you think these
> > accesses are not cached?
> 
> Please see arch/arm/lib/cache-cp15.c
> The code there creates 4096 page table entries (1MB each) for the whole
> 4GB address space and initializes each entry in a way that it is not
> cacheable (mmu_setup():71). It then changes the page table entries which
> are pointing to a RAM area to make these, and only these, cacheable
> (dram_bank_mmu_setup():57).

You did not mention this before. You just said: "I enabled dcache"
which for e sounds as if you did this globally.

Well, I'm not an export for AT91 in any way...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The human mind treats a new idea the way the body  treats  a  strange
protein - it rejects it.                                 - P. Medawar