JFFS2 NAND flash support

JFFS2 NAND flash support

[John Hall]

Hi,

I'm wanting to get JFFS2 to go on some NAND flash (it's actually
Smartmedia, but it's pretty much just NAND flash) with a 2.4.17 kernel.
I've adapted the NAND flash driver in the kernel tree and that now seems
to work for me.

The jffs2 stuff in my kernel sources doesn't seem very up to date so
I've got the latest CVS stuff. Which branch of cvs do I need to use?
Should I update the whole of the mtd subsystem or can I just use the
updated jffs2 stuff?

I also came across some problems with the existing NAND flash driver,
especially with erases. It doesn't seem to mark an erase as done,
  i.e. instr->state = MTD_ERASE_DONE;

Also, when doing an erase it obtains a spin lock that disables all
bottom halves from running. It holds this lock for the duration of the
erase, which is up to 4ms in the code - is this really a good idea? The
Smartmedia specs say that this delay can be up to 400ms! Definitely not
a good idea.

Regards,
John Hall

-- 
OptionExist Limited.
The Irwin Centre, Scotland Road, Dry Drayton, Cambridge. UK.
Tel: +44 (0)1954 211244, Fax: +44 (0)1954 211565
http://www.optionexist.co.uk/

Re: JFFS2 NAND flash support

[Thomas Gleixner]

On Wednesday, 27. March 2002 18:42, John Hall wrote:
> I'm wanting to get JFFS2 to go on some NAND flash (it's actually
> Smartmedia, but it's pretty much just NAND flash) with a 2.4.17 kernel.
It is exactly NAND flash, molded in thin plastic

> I've adapted the NAND flash driver in the kernel tree and that now seems
> to work for me.
maybe you have done duplicated work . See below.

> The jffs2 stuff in my kernel sources doesn't seem very up to date so
> I've got the latest CVS stuff. Which branch of cvs do I need to use?
> Should I update the whole of the mtd subsystem or can I just use the
> updated jffs2 stuff?
The current developer branch. Take care there were a lot of changes in nand.c 
and hardware driver files. See nand.c / autcpu12.c
Adaptions should made in hardware drivers like autcpu12 and not in nand.c
If there's a neccecarity to do this, please contact me or David Woodhouse.

> I also came across some problems with the existing NAND flash driver,
> especially with erases. It doesn't seem to mark an erase as done,
>   i.e. instr->state = MTD_ERASE_DONE;
It's fixed in CVS long ago

> Also, when doing an erase it obtains a spin lock that disables all
> bottom halves from running. It holds this lock for the duration of the
> erase, which is up to 4ms in the code - is this really a good idea? The
> Smartmedia specs say that this delay can be up to 400ms! Definitely not
> a good idea.
There's no better to ensure, that erase read and write can coexist at the 
moment. But you're invited to hack a better solution :)

I this a new card type ? All chips and card types I know, have maximum block 
erase times of 20ms.

-- 
Thomas
_________________________________
Thomas Gleixner <gleixner@autronix.de>
autronix automation http://www.autronix.de

Re: JFFS2 NAND flash support

[David Woodhouse]

gleixner@autronix.de said:
> There's no better to ensure, that erase read and write can coexist at
> the  moment. But you're invited to hack a better solution :)

> I this a new card type ? All chips and card types I know, have maximum
> block  erase times of 20ms.

Disabling bottom halves for 20ms isn't acceptable. The NOR chip drivers get 
this right - or at least attempt to.

--
dwmw2

Re: JFFS2 NAND flash support

[Thomas Gleixner]

On Wednesday, 27. March 2002 19:26, David Woodhouse wrote:
> gleixner@autronix.de said:
> > There's no better to ensure, that erase read and write can coexist at
> > the  moment. But you're invited to hack a better solution :)
> >
> > I this a new card type ? All chips and card types I know, have maximum
> > block  erase times of 20ms.
>
> Disabling bottom halves for 20ms isn't acceptable. The NOR chip drivers get
> this right - or at least attempt to.
You're right, we should reschedule there. I will take care of this in the 
next days.
-- 
Thomas
_________________________________
Thomas Gleixner <gleixner@autronix.de>
autronix automation http://www.autronix.de

RE: JFFS2 NAND flash support

[John Hall]

On 27 March 2002 18:16 Thomas Gleixner <gleixner@autronix.de> wrote:

> > I've adapted the NAND flash driver in the kernel tree and 
> > that now seems
> > to work for me.

> maybe you have done duplicated work . See below.

> > The jffs2 stuff in my kernel sources doesn't seem very up to date so
> > I've got the latest CVS stuff. Which branch of cvs do I need to use?
> > Should I update the whole of the mtd subsystem or can I just use the
> > updated jffs2 stuff?

> The current developer branch. Take care there were a lot of changes in
> nand.c and hardware driver files. See nand.c / autcpu12.c Adaptions
> should made in hardware drivers like autcpu12 and not in nand.c If
> there's a neccecarity to do this, please contact me or David
> Woodhouse.

Which is the current development branch?

Our hardware interface is a btit weird (ALE and CLE are controlled by
address lines...). I also can get interrupts from the busy pin, so I've
changed the code so it doesn't do busy waits as nand.c does.

> > I also came across some problems with the existing NAND flash
> > driver, especially with erases. It doesn't seem to mark an erase as
> > done,
> >   i.e. instr->state = MTD_ERASE_DONE;

> It's fixed in CVS long ago

OK - I hadn't looked at the latest versions very carefully. Now I've
looked, I notice that if the erase fails the user callback is not
called. The documentation on the mtd website seems to imply that you
should set the status to FAILED and still call the callback. Is that
correct?

> > Also, when doing an erase it obtains a spin lock that disables all
> > bottom halves from running. It holds this lock for the duration of
> > the erase, which is up to 4ms in the code - is this really a good
> > idea? The Smartmedia specs say that this delay can be up to 400ms! 

> There's no better to ensure, that erase read and write can coexist at
> the moment. But you're invited to hack a better solution :)

But you're holding a spinlock to ensure that your driver doesn't get
re-entered. Why do bottom halves need to be disabled at all?

> I this a new card type ? All chips and card types I know, have maximum
> block erase times of 20ms.

I'm just going by the Smartmedia spec. It says the maximum erase time is
400ms.

Regards,
John Hall

Re: JFFS2 NAND flash support

[Thomas Gleixner]

On Thursday, 28. March 2002 10:41, John Hall wrote:
> Which is the current development branch?
cvs -d :pserver:anoncvs@cvs.infradead.org:/home/cvs login (password: anoncvs)
cvs -d :pserver:anoncvs@cvs.infradead.org:/home/cvs co mtd
 
> Our hardware interface is a btit weird (ALE and CLE are controlled by
> address lines...). I also can get interrupts from the busy pin, so I've
> changed the code so it doesn't do busy waits as nand.c does.
Designed by hardware engineer, who never wrote a line of code :)
Can you please send me your changes. As I will rework the spinlock issue,
I will try to merge it into the current version. 

> OK - I hadn't looked at the latest versions very carefully. Now I've
> looked, I notice that if the erase fails the user callback is not
> called. The documentation on the mtd website seems to imply that you
> should set the status to FAILED and still call the callback. Is that
> correct?
No, doc is out of date
If you return 0 (.i.e. 'request queued OK') you must call the callback.
If you return nonzero (i.e. request not queued) you don't.

> But you're holding a spinlock to ensure that your driver doesn't get
> re-entered. Why do bottom halves need to be disabled at all?
As I said, I will take care of this in the next days.

> I'm just going by the Smartmedia spec. It says the maximum erase time is
> 400ms.
The Smartmedia spec is a general purpose paper. You must always use the chip 
datasheet at least. If the chip does not meet Smartmedia spec, you can 
complain about that, but it does not help you at all. :)
-- 
Thomas
_________________________________
Thomas Gleixner <gleixner@autronix.de>
autronix automation http://www.autronix.de

Re: JFFS2 NAND flash support

[David Woodhouse]

John.Hall@optionexist.co.uk said:
>  Our hardware interface is a btit weird (ALE and CLE are controlled by
> address lines...). I also can get interrupts from the busy pin, so
> I've changed the code so it doesn't do busy waits as nand.c does.

Good. You can actually implement the async erases that the MTD erase API 
was designed for.

You return zero from the erase() function to indicate that the request was 
started OK, then later you set the instr->state to indicate whether the 
erase succeeded or not, and call the callback.

The only thing to beware of is that users will currently not expect the 
callback to be called in interrupt context; only in bottom half context - 
and I'm thinking of changing that to allow only _process_ context. You can 
use schedule_task() to schedule the callback from the interrupt handler.

--
dwmw2

Re: JFFS2 NAND flash support

[David Woodhouse]

John.Hall@optionexist.co.uk said:
>  I'm guessing that the zlib code has been moved recently so it can be
> shared across several modules in the kernel. I am using 2.4.17; what
> is the best thing for me to do?

ftp.kernel.org:/pub/linux/kernel/people/dwmw2/shared-zlib/

--
dwmw2

RE: JFFS2 NAND flash support

[John Hall]

On 27 March 2002 18:16 Thomas Gleixner <gleixner@autronix.de> wrote:

> > The jffs2 stuff in my kernel sources doesn't seem very up to date so
> > I've got the latest CVS stuff. Which branch of cvs do I need to use?
> > Should I update the whole of the mtd subsystem or can I just use the
> > updated jffs2 stuff?

> The current developer branch. Take care there were a lot of changes in
> nand.c and hardware driver files. See nand.c / autcpu12.c

I've tried compiling the latest stuff on the CVS trunk, but cannot get
JFFS2 to work. I've compiled it as a module, but when I try to load it I
get a whole load of messages about missing zlib_ symbols:

e.g.
/lib/modules/2.4.17-rmk5/kernel/fs/jffs2/jffs2.o: unresolved symbol
zlib_inflate

I'm guessing that the zlib code has been moved recently so it can be
shared across several modules in the kernel. I am using 2.4.17; what is
the best thing for me to do?

Regards,
John Hall

Re: JFFS2 NAND flash support

[Thomas Gleixner]

Hi John,
I put this on the list again.

On Thursday, 28. March 2002 18:35, John Hall wrote:
> On 28 March 2002 15:53 Thomas Gleixner <gleixner@autronix.de> wrote:
> > OK, I had a short glance on your code. It's a pity, that you started a
> > nand.c rewrite instead of discussing that issue before hacking. Don't
> > missunderstand me. I don't have a problem if anybody is hacking around
> > in nand.c.
> >
> > It means: We try to integrate as much as possible in general purpose
> > files like nand.c so it's easier to maintain the hardware specific
> > stuff in small hw-driver files. General purpose files can be
> > maintained for neccecary changes in filesystems, mtd layers etc. so
> > they fit for all underlying hardware drivers without the need to
> > change all of them too. As Smartmedia is exactly the same as NAND it
> > should be one generic driver at all. Reenventing the wheel is not the
> > aim of open software development. You have a chosen a new layout for
> > he oob area. This will not fit into the current JFFS2 layout. So JFFS2
> > will not work that way. We discussed this problem in a long mailthread
> > a few month ago.
>
> Agreed. To be honest, I didn't really expect my stuff to be of interest
> to anyone else. 
It's not a problem if it's of interest for somebody, but it will be a problem 
for you and your customer, if you have a solution, which is hard to 
maintaing, because it is out of the mainstream. And maybe it gives some other 
people a nice idea, how to design their hardware the best way.

> It's bespoke development for our client, and has unusual
> hardware access. Looking back, I probably should have tried to work
> within the existing framework. I originally was hoping to read and write
> standard Smartmedia cards, which have a well defined layout for the oob
> data and use FAT12 as their filesystem. The client has since decided
> that FAT12 isn't robust enough in power loss situations (it's not robust
> in any situation... :). 
Same for me. It's ok for MP3 players :)

> This is the only reason I have put oob data where I have.
We have left the bad block marker where it is has to be on SmartMedia by spec 
and arranged the rest for our convenience :)
The layout scheme is flexible and can be changed easy for other filesystems 
like FAT12, if somebody does the neccecary hack :).

> > If you are interrested in a common solution for your target and the
> > community, I suggest, that we rework the existing nand.c together.
> > It's not a big deal to extend the nand structure with a pointer, which
> > gives you the control for a complete hardware specific command write
> > function. Also it's no problem to modify the wait, that you can choose
> > a timed wait or a interrupt driven one, which then resides in your
> > hardware driver. This would be a solution that ensures improvement of
> > the code, usability for everybody and prevent us from duplicated code.
> > We have already enough in the kernel.

> OK, I will take a look at the feasability of merging the two next week
> after the holidays.
I will check it in the next days too and will prepare the stuff, what's 
neccecary to support your hardware. This are only 3 things to do, IMHO
1. give the possibility to provide a hardware specific command write function
2. give the possibility to provide a hardware specific wait function,which 
can be interrupt driven, if the hardware has this feature.
3. rewrite the erase blocking
Have a look in CVS after your Easter weekend. 

> In your opinion, is the solution that I have chosen (an interrupt
> handler which schedules a tasklet) the best one for handling erases? It
> seems a little little untidy, because reads and writes are handled
> differently since they are synchronous. I haven't done much 2.4
> development, so I'm still feeling my way.
Yep, it's a nice feature at all. But we should keep the synchronous functions 
for read/write as the max. delay is less, than running through an interrupt 
handler.

-- 
Happy Easter
Thomas
_________________________________
Thomas Gleixner <gleixner@autronix.de>
autronix automation http://www.autronix.de

RE: JFFS2 NAND flash support

[John Hall]

On 28 March 2002 18:12 Thomas Gleixner <gleixner@autronix.de> wrote:

> > OK, I will take a look at the feasability of merging the two next
> > week after the holidays.

> I will check it in the next days too and will prepare the stuff,
> what's neccecary to support your hardware. This are only 3 things to
> do, IMHO
> 1. give the possibility to provide a hardware specific command write
> function
> 2. give the possibility to provide a hardware specific wait
> function,which can be interrupt driven, if the hardware has this
> feature.
> 3. rewrite the erase blocking
> Have a look in CVS after your Easter weekend. 

Thomas,

Have you made any progress on this? If not, then I am willing to start
making the changes myself. I've been looking through the code and
started to create a smartmedia hardware-specific file similar to spia.c.
A change that I need is a function in nand.c that fills in most of the
fields of the mtd, as an alternative to nand_scan. I have therefore
added a function nand_fillin_mtd that fills in the fields in the mtd
that are relevant to nand flash apart from those to do with its size.
nand_scan also calls this.

> > In your opinion, is the solution that I have chosen (an interrupt
> > handler which schedules a tasklet) the best one for handling erases?
> > It seems a little little untidy, because reads and writes are
> > handled differently since they are synchronous. I haven't done much
> > 2.4 development, so I'm still feeling my way.

> Yep, it's a nice feature at all. But we should keep the synchronous
> functions for read/write as the max. delay is less, than running
> through an interrupt handler.

You're right - the context switches would give too big an overhead.

Regards,
John Hall