Fwd: Cache mappings and invalidate

Fwd: Cache mappings and invalidate

[Joakim Tjernlund]

Hi again

I did some changes since the first attempt  didn't work very well. I removed
the changes in cfi_cmdset_0001.c and changed the copy_from() function to:

void lumentis_copy_from(struct map_info *map, void *to, unsigned long from, ssize_t len)
{
#ifdef  USE_DCACHE
       dma_cache_inv(map->map_priv_2 + from, len);
       memcpy_fromio(to, (void *)(map->map_priv_2 + from), len);
#else
        memcpy_fromio(to, (void *)(map->map_priv_1 + from), len);
#endif
}

This seems to work(no CRC errors so far) and has no visible impact on performance. When
I get around to enable burst reads form my flash I hope to see a performance gain.

Anybody care to comment on my experiment so far?

          Jocke
 
----------  Forwarded Message  ----------
Subject: Cache mappings and invalidate
Date: Mon, 12 Nov 2001 14:14:14 +0200
From: Joakim Tjernlund <Joakim.Tjernlund@lumentis.se>
To: linux-mtd@lists.infradead.org


Hi all

I am trying to make copy_from use cahced memory by assignig
map_priv_2 = (unsigned long)__ioremap(flash_addr, flash_size, 0);
and then change copy_from routine to:
+#ifndef NO_CACHE
+       memcpy_fromio(to, (void *)(map->map_priv_2 + from), len);
+#else
        memcpy_fromio(to, (void *)(map->map_priv_1 + from), len);
+#endif

There are no cache invalidations in my map file, instead I have
added invalidate_dcache_range() calls to
 drivers/mtd/chips/cfi_cmdset_0001.c(see below) in do_write_oneword(),
 do_write_buffer() and do_erase_oneblock(). Note that this is just a quick
 hack to try out my theory. Does this look sane or should I use another
 invalidate_dcache_range() and/or place the invalidate calls somewhere else?

I am not using burst reads yet, that will come later once i have gotten the
 cached mapping to work.

       Joakim



--- drivers/mtd/chips/cfi_cmdset_0001.c 2001/10/25 12:11:10     1.3
+++ drivers/mtd/chips/cfi_cmdset_0001.c 2001/11/12 12:02:33
@@ -502,6 +502,10 @@
        cfi_udelay(chip->word_write_time);
        spin_lock_bh(chip->mutex);

+#ifndef NO_CACHE
+       invalidate_dcache_range(map->map_priv_2 + adr, map->map_priv_2 + adr
 + 4); /* on a 32 bit bus */ +#endif
+
        timeo = jiffies + (HZ/2);
        z = 0;
        for (;;) {
@@ -691,7 +695,7 @@
        wbufsize = CFIDEV_INTERLEAVE << cfi->cfiq->MaxBufWriteSize;
        adr += chip->start;
        cmd_adr = adr & ~(wbufsize-1);
-
+
        /* Let's determine this according to the interleave only once */
        status_OK = CMD(0x80);

@@ -790,6 +794,10 @@
        cfi_udelay(chip->buffer_write_time);
        spin_lock_bh(chip->mutex);

+#ifndef NO_CACHE
+       invalidate_dcache_range(map->map_priv_2 + adr, map->map_priv_2 + adr
 + len); +#endif
+
        timeo = jiffies + (HZ/2);
        z = 0;
        for (;;) {
@@ -994,6 +1002,10 @@
        spin_unlock_bh(chip->mutex);
        schedule_timeout(HZ);
        spin_lock_bh(chip->mutex);
+
+#ifndef NO_CACHE
+       invalidate_dcache_range(map->map_priv_2 + adr, map->map_priv_2 + adr
 + 0x40000); /* 0x40000 is my erase size */
+#endif

        /* FIXME. Use a timer to check this, and return immediately. */
        /* Once the state machine's known to be working I'll do that */

-------------------------------------------------------

Re: Fwd: Cache mappings and invalidate

[David Woodhouse]

Joakim.Tjernlund@lumentis.se said:
>  Anybody care to comment on my experiment so far? 

Beware of the dma_cache_inv() and invalidate_dcache_range() functions. On 
some architectures they're stupidly misnamed and are in fact no-operations. 

Linus seems to think this is a Good Thing.


--
dwmw2

RE: Fwd: Cache mappings and invalidate

[Joakim Tjernlund]

> 
> Joakim.Tjernlund@lumentis.se said:
> >  Anybody care to comment on my experiment so far? 
> 
> Beware of the dma_cache_inv() and invalidate_dcache_range() functions. On 
> some architectures they're stupidly misnamed and are in fact no-operations.

OK, just checked my arch(MPC860) and they are implemented. Are there
other functions that I should used instead? 
 
> 
> Linus seems to think this is a Good Thing.
hmm, why should this be a Good Thing?


 Jocke
> 
> 
> --
> dwmw2

Re: Fwd: Cache mappings and invalidate

[David Woodhouse]

joakim.tjernlund@lumentis.se said:
>  OK, just checked my arch(MPC860) and they are implemented. Are there
> other functions that I should used instead? 

No. I suggested we implement simon_says_flush_dcache_range() et al, but 
Linus didn't seem to like that :)

> > Linus seems to think this is a Good Thing. 
> hmm, why should this be a Good Thing?

No clue.


--
dwmw2

Re: Fwd: Cache mappings and invalidate

[David Woodhouse]

bjorn.wesen@axis.com said:
>  So what you really want, is to rename those cache flushes in the mm
> code into a name saying what they try to do, let some archs implement
> that as a cache flush if needed, and reserve the real cache flush code
> into a function that you need to implement (and use that in MTD
> perhaps).

> You _dont_ want to implement the flush_cache_* functions that exist
> right now on all architectures, not the way they are used.

Agreed. But Linus didn't want to fix the silly names. He seems to like them.

--
dwmw2

Re: Fwd: Cache mappings and invalidate

[Bjorn Wesen]

On Tue, 13 Nov 2001, David Woodhouse wrote:
> No. I suggested we implement simon_says_flush_dcache_range() et al, but 
> Linus didn't seem to like that :)
> 
> > > Linus seems to think this is a Good Thing. 
> > hmm, why should this be a Good Thing?
> 
> No clue.

I don't know what he is thinking exactly, but one reason is that there is
no way the arch-independent code can know for sure that a cache _needs_ to
be flushed in many cases, because there are so many ways to design a
cache. Erasing a flash sector is obvious, but other things are not and you
certainly don't want to flush the cache if you don't absolutely need to.

All it can do is give a hint which might be useful to implement in some
architectures only.. For example, a cache invalidate is called all the
time during page remappings, but for most architectures you don't need to
flush it at those times and doing it would just waste time. 

So what you really want, is to rename those cache flushes in the mm code
into a name saying what they try to do, let some archs implement that as a
cache flush if needed, and reserve the real cache flush code into a
function that you need to implement (and use that in MTD perhaps).

You _dont_ want to implement the flush_cache_* functions that exist right
now on all architectures, not the way they are used.

/BW

Re: Fwd: Cache mappings and invalidate

[Joakim Tjernlund]

Hi all

Finally I got to enable burst read from flash, but my tests shows no performance gain.

There is no difference when I mont my JFFS2 FS or
if I do 'cp /dev/mtdblock4 /dev/null' or  'cp /dev/mtd4 /dev/null'. Even
if I remove the dma_cache_inv() call there is no performance gain.

It is almost like burst reads are disabled, but if I mess with the burst HW init sequence, 
I get a lot of errors, so I take it that burst reads are enabled.

I have a custom MPC860 board(4k icache, 4k dcache) running at 80 MHz
(bus is 40 Mhz),  128 MB Ram.

 Any Ideas? 

       Jocke

On Tuesday 13 November 2001 16:53, Joakim Tjernlund wrote:
> Hi again
>
> I did some changes since the first attempt  didn't work very well. I
> removed the changes in cfi_cmdset_0001.c and changed the copy_from()
> function to:
>
> void lumentis_copy_from(struct map_info *map, void *to, unsigned long from,
> ssize_t len) {
> #ifdef  USE_DCACHE
>        dma_cache_inv(map->map_priv_2 + from, len);
>        memcpy_fromio(to, (void *)(map->map_priv_2 + from), len);
> #else
>         memcpy_fromio(to, (void *)(map->map_priv_1 + from), len);
> #endif
> }
>
> This seems to work(no CRC errors so far) and has no visible impact on
> performance. When I get around to enable burst reads form my flash I hope
> to see a performance gain.
>
> Anybody care to comment on my experiment so far?
>
>           Jocke
>
> ----------  Forwarded Message  ----------
> Subject: Cache mappings and invalidate
> Date: Mon, 12 Nov 2001 14:14:14 +0200
> From: Joakim Tjernlund <Joakim.Tjernlund@lumentis.se>
> To: linux-mtd@lists.infradead.org
>
>
> Hi all
>
> I am trying to make copy_from use cahced memory by assignig
> map_priv_2 = (unsigned long)__ioremap(flash_addr, flash_size, 0);
> and then change copy_from routine to:
> +#ifndef NO_CACHE
> +       memcpy_fromio(to, (void *)(map->map_priv_2 + from), len);
> +#else
>         memcpy_fromio(to, (void *)(map->map_priv_1 + from), len);
> +#endif
>
> There are no cache invalidations in my map file, instead I have
> added invalidate_dcache_range() calls to
>  drivers/mtd/chips/cfi_cmdset_0001.c(see below) in do_write_oneword(),
>  do_write_buffer() and do_erase_oneblock(). Note that this is just a quick
>  hack to try out my theory. Does this look sane or should I use another
>  invalidate_dcache_range() and/or place the invalidate calls somewhere
> else?
>
> I am not using burst reads yet, that will come later once i have gotten the
>  cached mapping to work.
>
>        Joakim
>
>
>
> --- drivers/mtd/chips/cfi_cmdset_0001.c 2001/10/25 12:11:10     1.3
> +++ drivers/mtd/chips/cfi_cmdset_0001.c 2001/11/12 12:02:33
> @@ -502,6 +502,10 @@
>         cfi_udelay(chip->word_write_time);
>         spin_lock_bh(chip->mutex);
>
> +#ifndef NO_CACHE
> +       invalidate_dcache_range(map->map_priv_2 + adr, map->map_priv_2 +
> adr + 4); /* on a 32 bit bus */ +#endif
> +
>         timeo = jiffies + (HZ/2);
>         z = 0;
>         for (;;) {
> @@ -691,7 +695,7 @@
>         wbufsize = CFIDEV_INTERLEAVE << cfi->cfiq->MaxBufWriteSize;
>         adr += chip->start;
>         cmd_adr = adr & ~(wbufsize-1);
> -
> +
>         /* Let's determine this according to the interleave only once */
>         status_OK = CMD(0x80);
>
> @@ -790,6 +794,10 @@
>         cfi_udelay(chip->buffer_write_time);
>         spin_lock_bh(chip->mutex);
>
> +#ifndef NO_CACHE
> +       invalidate_dcache_range(map->map_priv_2 + adr, map->map_priv_2 +
> adr + len); +#endif
> +
>         timeo = jiffies + (HZ/2);
>         z = 0;
>         for (;;) {
> @@ -994,6 +1002,10 @@
>         spin_unlock_bh(chip->mutex);
>         schedule_timeout(HZ);
>         spin_lock_bh(chip->mutex);
> +
> +#ifndef NO_CACHE
> +       invalidate_dcache_range(map->map_priv_2 + adr, map->map_priv_2 +
> adr + 0x40000); /* 0x40000 is my erase size */
> +#endif
>
>         /* FIXME. Use a timer to check this, and return immediately. */
>         /* Once the state machine's known to be working I'll do that */
>
> -------------------------------------------------------

Re: Fwd: Cache mappings and invalidate

[David Woodhouse]

Joakim.Tjernlund@lumentis.se said:
>  It is almost like burst reads are disabled, but if I mess with the
> burst HW init sequence,  I get a lot of errors, so I take it that
> burst reads are enabled. 

What flash chip are you using, and have you actually put the flash chip in
to synchronous read mode?

--
dwmw2

RE: Fwd: Cache mappings and invalidate

[Joakim Tjernlund]

> Joakim.Tjernlund@lumentis.se said:
> >  It is almost like burst reads are disabled, but if I mess with the
> > burst HW init sequence,  I get a lot of errors, so I take it that
> > burst reads are enabled. 
> 
> What flash chip are you using, and have you actually put the flash chip in
> to synchronous read mode?

I am using Intel Strata flash(E28F128J3A150) and they are in async page-mode(burst) by default.
There is an appnote(AP-694 and AP-715) on www.intel.com that I used as a template. 

My first thougth was that burst was not enabled, but if I changed the timing for the burst
read in the UPM(User Programmable Machine) to something invalid, I get lots of error form JFFS2 so 
I think it is enabled correctly.
 
Can it be the MV kernel?

I am trying to get 2.4.16-pre1 from ppclinux running but I get a lot of oops messages( see below).
Its very easy to generate this opps, just mount JFFS2 and copy a file a few times.
But I also get Oopses if I start a heavy compile on my board on a NFS mounted FS, but it's much harder
to trigger it. I have tried on and off since 2.4.8 to make it run on my custom MPC860 board, but it has
always Oopsed with a sig: 11 sooner or later. Maybe later kernels are broken wrt MPC860?

  Jocke

ksymoops 2.4.1 on i686 2.4.3-12.  Options used
     -v vmlinux (specified)
     -K (default)
     -L (default)
     -O (default)
     -M (specified)
     -e
     -t elf32-powerpc -a powerpc:MPC8XX

Oops: kernel access of bad area, sig: 11
NIP: C00B12E8 XER: 00000000 LR: C00AD078 SP: C3B8BCD0 REGS: c3b8bc20 TRAP: 0300 
MSR: 00009032 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 11
TASK = c3b8a000[82] 'cp' Last syscall: 5 
last math 00000000 last altivec 00000000 
GPR00: 00000000 C3B8BCD0 C3B8A000 C013F90C 00E800E8 02E25280 C3B8BCE4 C0150000 
GPR08: 00000004 C5000000 00000002 02000000 03E40000 10021824 00000000 00000044 
GPR16: 00E252A8 00000000 00000018 02E252A8 00800080 C0150000 C0157880 000019E1 
GPR24: C02EA160 C3A75100 C02EA1C8 00000000 02E25280 02000000 02C652A8 C013F90C 
Call backtrace:                                                                
00800080 C00B1A90 C0082B90 C0082E3C C007F168 C0048048 C002DE7C 
C003C614 C002F078 C002F4D4 C000275C 10019A2C 100028B4 10003DE8 
10004570 10001EA0 100024F0 0FED9DBC 00000000                   
Kernel panic: Aiee, killing interrupt handler!
Warning (Oops_read): Code line not seen, dumping what data is available

>>NIP; c00b12e8 <lumentis_write32+4/10>   <=====
Trace; 00800080 Before first symbol
Trace; c00b1a90 <part_write+ac/c0>
Trace; c0082b90 <mtd_fake_writev+68/cc>
Trace; c0082e3c <jffs2_write_dnode+120/21c>
Trace; c007f168 <jffs2_setattr+400/598>
Trace; c0048048 <notify_change+90/13c>
Trace; c002de7c <do_truncate+70/ac>
Trace; c003c614 <open_namei+504/63c>
Trace; c002f078 <filp_open+58/84>
Trace; c002f4d4 <sys_open+4c/fc>
Trace; c000275c <ret_from_syscall_1+0/b4>
Trace; 10019a2c Before first symbol
Trace; 100028b4 Before first symbol
Trace; 10003de8 Before first symbol
Trace; 10004570 Before first symbol
Trace; 10001ea0 Before first symbol
Trace; 100024f0 Before first symbol
Trace; 0fed9dbc Before first symbol
Trace; 00000000 Before first symbol


1 warning issued.  Results may not be reliable.


> 
> --
> dwmw2
> 
> 

Re: Fwd: Cache mappings and invalidate

[David Woodhouse]

joakim.tjernlund@lumentis.se said:
> Can it be the MV kernel?

I don't see why.

> I am trying to get 2.4.16-pre1 from ppclinux running but I get a lot
> of oops messages( see below). Its very easy to generate this opps,
> just mount JFFS2 and copy a file a few times. 

It's dying in lumentis_write - that's your map driver, isn't it? Check the 
code at 0xc00b12e8, compare the register it's dereferencing with the 
address it's _supposed_ to be using, etc. 

Check bus timings, etc. 

--
dwmw2

RE: Fwd: Cache mappings and invalidate

[Joakim Tjernlund]

> joakim.tjernlund@lumentis.se said:
> > Can it be the MV kernel?
> 
> I don't see why.
I don't think so either, it was just a wild guess.

> 
> > I am trying to get 2.4.16-pre1 from ppclinux running but I get a lot
> > of oops messages( see below). Its very easy to generate this opps,
> > just mount JFFS2 and copy a file a few times. 
> 
> It's dying in lumentis_write - that's your map driver, isn't it? Check the 
> code at 0xc00b12e8, compare the register it's dereferencing with the 
> address it's _supposed_ to be using, etc.

The same code works fine in MV's kernel and there is not much to debug,
lumentis_write32() looks the same as rpx_write32(). Anyhow, I have
received one report that indicates that the latest kernels from
linuxppc_2_4 are broken w.r.t mpc_860 
 
> 
> Check bus timings, etc.
I have, over and over again. I also lowered my bus speed to make sure it's not
a timing issue.

> 
> --
> dwmw2
> 
>