CFI-0002 NOR flash blocking CPU on status register reads

CFI-0002 NOR flash blocking CPU on status register reads

[Jamie Lokier]

Something's been bothering me for a while about a local patch to the
cfi_cmdset_0002.c driver.  What bothers me most is that we're using a
standard NOR chip from Spansion - so why do we need a special patch?

Jörn Engel wrote:
> > I have a particularly crappy one where erase blocks the CPU if the CPU
> > reads from the chip during the erase - so the cfi_cmdset_0002.c file
> > needs a patch to avoid polling the status register for this board.
> > This is topic for another post, though.
> 
> Nothing can prevent bad drivers.  At least they are easy to fix.

It's not a driver problem (as far as I can tell).  The board requires
it - the NOR flash blocks CPU read cycles when reading the status
register.  I didn't find a way to turn it off in CFI-0002 specs.

There's a wire from RY/BY# (ready/busy) on the flash to PB_IORDY on
the CPU.  That's "peripheral bus ready" - it causes I/O wait states
when deasserted during an I/O cycle to the flash.

Let me explain with the aid of a patch, to cfi_cmdset_0002.c:

static inline int do_erase_chip
...
        cfi_send_gen_cmd(0x10, cfi->addr_unlock1, chip->start, map, cfi,
                         CFI_DEVICETYPE_X8, NULL);
        timeo = jiffies + (HZ*20);
        adr = cfi->addr_unlock1;
 
+       /* We need extra delay here since reading bit5/bit6 operation may
+          freeze the chip for some time. */
+       cfi_spin_unlock(chip->mutex);
+       set_current_state(TASK_UNINTERRUPTIBLE);//add by rain
+       schedule_timeout(((1<<cfi->cfiq->ChipEraseTimeoutTyp) * HZ) / 1000);
+       cfi_spin_lock(chip->mutex);

        /* Wait for the end of programing/erasure by using the toggle
         * method.  As long as there is a programming procedure going
         * on, bit 6 of the last written byte is toggling it's state
         * with each consectuve read.  The toggling stops as soon as
         * the procedure is completed.
         *
         * If the process has gone on for too long on the chip bit 5 gets.
         * After bit5 is set you can kill the operation by sending a reset
         * command to the chip.
         */
        dq6 = CMD(1<<6);
        dq5 = CMD(1<<5);
        oldstatus = cfi_read(map, adr);
        status = cfi_read(map, adr);
        while( ((status & dq6) != (oldstatus & dq6)) && 
                ((status & dq5) != dq5) &&
                !time_after(jiffies, timeo)) {
... etc.

If we don't have the explicit time delay (and it's pretty long, about
0.25 to 0.5 seconds), then the cfi_read() below blocks in an I/O wait
state, until the erase is complete.  The Spansion flash outputs a
logic low on RY/BY#, and the CPU blocks when reading.  While the CPU
is blocked, it cannot service interrupts: time gets skewed and other
things that should be serviced don't look pretty.  So, we add the
explicit delay to prevent blocking.

I noticed that mainline sources, even uClinux patches, don't have this
delay.  (The patch is to an old kernel, but I looked for similar in
current).  Does that mean nobody else has this issue?

If true, does that mean this board shouldn't have wired RY/BY# on the
Spansion flash to PB_IORDY on the CPU.  Ok, I can ask for the future
revisions to remove this wire, and only use the delay with boards that
have it.  Then everything else will run smoother while writing flash.

But if I remove that wire, when the CPU is reset (e.g. by watchdog
chip or other means), the flash might be in the middle of a write or
erase operation.  Especially erase, as that's slower than CPU reset.
This is normal, as we often write to JFFS2 on this chip.

Won't the CPU read the initial boot instructions from this flash, get
programming status bytes instead of CPU instructions, and thus get
confused?

What do other designs booting from CFI NOR do about this?

Thanks very much for any insights,
-- Jamie

Re: CFI-0002 NOR flash blocking CPU on status register reads

[Holger Schurig]

> Won't the CPU read the initial boot instructions from this
> flash, get programming status bytes instead of CPU
> instructions, and thus get confused?

The bootloader could/should make sure that every block is in the 
correct state for Linux.

Re: CFI-0002 NOR flash blocking CPU on status register reads

[Jamie Lokier]

Jamie Lokier wrote:
> There's a wire from RY/BY# (ready/busy) on the flash to PB_IORDY on
> the CPU.  That's "peripheral bus ready" - it causes I/O wait states
> when deasserted during an I/O cycle to the flash.

> does that mean this board shouldn't have wired RY/BY# on the
> Spansion flash to PB_IORDY on the CPU.
...
> But if I remove that wire, when the CPU is reset (e.g. by watchdog
> chip or other means), the flash might be in the middle of a write or
> erase operation.
...
> Won't the CPU read the initial boot instructions ... get
> programming status bytes...?

Oh, after reading the documentation more closely, I see the Spansion
chip has a reset signal, which aborts any write/erase operation
immediately, and it's safe to read CPU instructions then.

So I don't need the I/O wait states, and it looks like the board
designers should not have connected RY/BY# to PB_IORDY.

It is copied from a Sigma Designs reference design...  I guess they
did it because their boot monitor's write/erase code does not poll the
status register correctly (checking for toggling DQ6), but simply
waits for the read byte to match what's written.  That would cause
unreliable writing, and adding the wire would fix it.

Is that right?

Assuming so, my remaining problem (other than fixing their boot
monitor's flash code) is a few thousand boards which have the wire,
currently deployed.

The hard delay we patched in works, but assumes "typical erase time"
is enough, which may cease to be true as the chips age.  Is that
expected?  Should I use the max erase/write times instead?

(Oh, the joy of board mis-design workarounds!)

-- Jamie

Re: CFI-0002 NOR flash blocking CPU on status register reads

[Jamie Lokier]

Holger Schurig wrote:
> > Won't the CPU read the initial boot instructions from this
> > flash, get programming status bytes instead of CPU
> > instructions, and thus get confused?
> 
> The bootloader could/should make sure that every block is in the 
> correct state for Linux.

Even the bootloader cannot start, if the flash is not in the correct
programming state - even though bootloader partition is not touched.

(The bootloader instructions are temporarily replaced by garbage while
programming _any_ blocks.)

However, see my other reply.  If the board's reset signal is wired to
the flash, it should force the flash into a suitable state prior to
reading the bootloader.  I have to check if they wired the reset
signal properly: these schematics have too many zero-ohm links which
might be there or not on manufactured boards.

-- Jamie

Re: CFI-0002 NOR flash blocking CPU on status register reads

[Trent Piepho]

On Tue, 10 Jun 2008, Jamie Lokier wrote:
> If we don't have the explicit time delay (and it's pretty long, about
> 0.25 to 0.5 seconds), then the cfi_read() below blocks in an I/O wait
> state, until the erase is complete.  The Spansion flash outputs a
> logic low on RY/BY#, and the CPU blocks when reading.  While the CPU
> is blocked, it cannot service interrupts: time gets skewed and other
> things that should be serviced don't look pretty.  So, we add the
> explicit delay to prevent blocking.
>
> I noticed that mainline sources, even uClinux patches, don't have this
> delay.  (The patch is to an old kernel, but I looked for similar in
> current).  Does that mean nobody else has this issue?
>
> If true, does that mean this board shouldn't have wired RY/BY# on the
> Spansion flash to PB_IORDY on the CPU.  Ok, I can ask for the future
> revisions to remove this wire, and only use the delay with boards that
> have it.  Then everything else will run smoother while writing flash.

We disconnected the RY/BY# line in our design.  It has the flash conected to a
Freescale GPCM local bus, and on that bus de-asserting the LB_READY line is
used to abort a transaction.  So instead of inserting wait states and hogging
the CPU, it aborts the read cycle immediately and frees the CPU, but doesn't
return correct status.  We couldn't think of any good reason to have RY
connected.

> But if I remove that wire, when the CPU is reset (e.g. by watchdog
> chip or other means), the flash might be in the middle of a write or
> erase operation.  Especially erase, as that's slower than CPU reset.
> This is normal, as we often write to JFFS2 on this chip.
>
> Won't the CPU read the initial boot instructions from this flash, get
> programming status bytes instead of CPU instructions, and thus get
> confused?
>
> What do other designs booting from CFI NOR do about this?

The RESET# pin on the flash should take care of that:

     "The RESET# pin provides a hardware method of resetting the device to
     reading array data.  When the RESET# pin is driven low for at least a
     period of tRP, the device immediately terminates any operation in
     progress, Hi-Z all output pins, ...  The RESET# pin may be tied to the
     system reset circuitry.  A system reset would thus also reset the Flash
     memory, enabling the system to read the boot-up firmware from the Flash
     memory."