Re: mtd-19990809 including readonly DiskOnChip driver.

Re: mtd-19990809 including readonly DiskOnChip driver.

[David Woodhouse]

jgg@deltatee.com said:
>  I've looked around your new bundle and it looks pretty good, if
> anyone wants to add the CFI probing routing it fits very nicely into
> the mapped.c file, instead of using the flash_jedec routine, a very
> similar routine that does CFI probes needs to be written. 

> I'm not sure the two AMD chips I use support CFI, anyone know off
> hand?

Probably. Try it and find out. The CFI spec is on Intel's web site:
	http://www.pentium.com/design/flcomp/technote/cfi_1_1.htm

> Also, using David's driver as the base driver kinda mucks things up -
> he uses a different erase mechanism with different meanings, and has
> some weird cruft from PCMCIA in there.

Yeah - the idea is that the PCMCIA drivers will be able to use this MTD layer 
when (if) Linus incorporates in into the standard kernel. However, I'd like to 
keep as much of the hardware-specific details _out_ of the MTD layer as 
possible - that's why you'll notice I've moved your mfr and id codes out of 
the struct mtd_info into your own private structure. So if there's PCMCIA 
'cruft' that doesn't need to be there, we can remove it.

Some extra complexity is required for the erases, though, to allow timer-based
operation and retries. I didn't really pay much attention to erases as I pulled
this lot together though - just used what I had last time. So feel free to
point out anything that's not right.


>  Porting the PCMCIA driver and the slram driver to use the mapped
> interface would save alot of code as well, compare:

PCMCIA driver? I haven't started on that yet. If the mapped driver supports 
pageable windows, then the DiskOnChip 1000 driver can use it too. 

> blue{jgg}/tmp/mtd-19990809/kernel#wc -l slram.c octagon-5066.c
>     210 slram.c
>     205 octagon-5066.c

I thought the octagon-5066 driver was yours and already used the mapped 
interface.

> For such a simple device that is quite needlessly long. (mapped
> already implements ram devices)

Yeah - there's definitely scope for removing the redundancy there. The aim of 
this release was mainly to pull all our work together into one codebase, to 
give us a starting point. 

>   I miss devfs support too :<

Sorry, I couldn't test it, and couldn't be sure not to screw it up as I've 
never got round to installing devfs. If you give me a patch which puts it 
back into all the drivers, I'll put it back in.

> Also, do your new NFTL and FTL drivers work over normal block devices?

No. NFTL cannot work over a block device, and can't easily work over most 
memory devices either - you need NAND flash, which has 'out of band' data. 

FTL doesn't either - but there's little need for FTL on block devices. FTL 
_provides_ a block device, not an actual mounted filesystem. So if you want to 
take a duplicate copy of an FTL system, you duplicate the block device not the 
underlying structure.

I'm not sure that running FFS2 from a block device is going to work in the 
long term either - you need access to internal MTD stuff like the erase 
mechanism and erasesize, etc. Your comments in the block device driver and in 
the FFS2 code seem to support that point of view.





> Porting the PCMCIA driver and the slram driver to use the mapped
> interface would save alot of code as well, compare:

> blue{jgg}/tmp/mtd-19990809/kernel#wc -l slram.c octagon-5066.c
>     210 slram.c
>     205 octagon-5066.c

> For such a simple device that is quite needlessly long. (mapped
> already implements ram devices)
>   I miss devfs support too :<

> Also, do your new NFTL and FTL drivers work over normal block devices?
> 



----                                 ----                                 ----
David Woodhouse        David.Woodhouse@mvhi.com       Office: (+44) 1223 810302
 Project Leader,     Process Information Systems      Mobile: (+44) 976 658355
    Axiom (Cambridge) Ltd., Swaffham Bulbeck, Cambridge, CB5 0NA, UK.
             finger dwmw2@ferret.lmh.ox.ac.uk for PGP key.




To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[Jason Gunthorpe]

On Tue, 10 Aug 1999, David Woodhouse wrote:

> > Also, using David's driver as the base driver kinda mucks things up -
> > he uses a different erase mechanism with different meanings, and has
> > some weird cruft from PCMCIA in there.
> 
> Yeah - the idea is that the PCMCIA drivers will be able to use this MTD layer 
> when (if) Linus incorporates in into the standard kernel. However, I'd like to 

I've used the PCMCIA drivers on memory cards, they do not work very well
at all - I -strongly- suspect that many problems would go away simply by
using the fancy detection code that is being put into the mapped layer, so
for me keeping compatibility with the old pcmcia driver isn't such a big
deal. 

> Some extra complexity is required for the erases, though, to allow
> timer-based operation and retries. I didn't really pay much attention to
> erases as I pulled this lot together though - just used what I had last
> time. So feel free to point out anything that's not right. 

That should all go into the highest layer, the device stuff doesn't need
to know it is being called asynchrounously.

> PCMCIA driver? I haven't started on that yet. If the mapped driver supports 
> pageable windows, then the DiskOnChip 1000 driver can use it too. 

The mapped driver is only for pageable windows of 'ordinary'
flash+ram+rom, linear type memories. I don't know how the DoC works so I
can't say if it would be at all usefull. 

If the DoC is just an array of normal flash chips with some paging
mechanism then you could probably do it quite easially.
  
> > Also, do your new NFTL and FTL drivers work over normal block devices?
> 
> No. NFTL cannot work over a block device, and can't easily work over most 
> memory devices either - you need NAND flash, which has 'out of band' data. 

Hum, that's kinda a bummer - Being able to prepare a flash image on my
desktop and then just bang the whole thing onto device is very usefull -
that is our primary means of deploying remote upgrades.
 
> FTL doesn't either - but there's little need for FTL on block devices.
> FTL _provides_ a block device, not an actual mounted filesystem. So if
> you want to take a duplicate copy of an FTL system, you duplicate the
> block device not the underlying structure. 

You still need to format the flash with the FTL information, it is
convient to just have that pre-available in an image.

> I'm not sure that running FFS2 from a block device is going to work in the 
> long term either - you need access to internal MTD stuff like the erase 
> mechanism and erasesize, etc. Your comments in the block device driver and in 
> the FFS2 code seem to support that point of view.

The plan is to make it ask the MTD drive if the device it was mounted
against is a MTD device and get a pointer to the MTD device block. Then it
will not speak block or character, it will just directly manipulate the
device. That solves the problem of having to ensure buffer cache
consistency during erasures and so on..

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[David Hinds]

On Tue, Aug 10, 1999 at 10:33:22AM -0600, Jason Gunthorpe wrote:
> 
> I've used the PCMCIA drivers on memory cards, they do not work very well
> at all - I -strongly- suspect that many problems would go away simply by
> using the fancy detection code that is being put into the mapped layer, so
> for me keeping compatibility with the old pcmcia driver isn't such a big
> deal. 

Hmmm.  What specific problems have you had?  As far as I know, there
are no outstanding bugs with the PCMCIA memory card drivers.  And I
don't really see how changing the card detection method would help:
identifying memory cards is not hard.

(I'm not arguing about the new code one way or the other, since I
haven't looked at it yet.  But as long as I'm supporting the current
code, I'm obviously interested in knowing about bugs)

-- Dave Hinds


To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[Jason Gunthorpe]

On Tue, 10 Aug 1999, David Hinds wrote:

> Hmmm.  What specific problems have you had?  As far as I know, there
> are no outstanding bugs with the PCMCIA memory card drivers.  And I
> don't really see how changing the card detection method would help:
> identifying memory cards is not hard.

Well, the big problem is that it simply cannot detect an empty card (with
no CIS) - I've watched QNX do it so I know it can be done, and I'm pretty
sure they rely on CFI or JEDEC ID flash methods [which is exactly what the
mapped flash driver implements]. There were a few other weird things I had
happen to me while futzing with that, but have forgotten many of the
details.

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[David Hinds]

On Tue, Aug 10, 1999 at 02:49:01PM -0600, Jason Gunthorpe wrote:
> 
> Well, the big problem is that it simply cannot detect an empty card (with
> no CIS) - I've watched QNX do it so I know it can be done, and I'm pretty
> sure they rely on CFI or JEDEC ID flash methods [which is exactly what the
> mapped flash driver implements].

That's certainly true: PCMCIA currently treats cards with no CIS as
static RAM cards.  Mostly because the issue has never really come up
for me.  I think that's a bit different from an unqualified "they do
not work very well at all".

Are linear flash cards with no CIS common?  I'm only familiar with the
Intel cards, which all seem to have a CIS.  I've never had anyone ask
about no-CIS flash card support, as far as I can remember.

-- Dave


To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[Jason Gunthorpe]

On Tue, 10 Aug 1999, David Hinds wrote:

> That's certainly true: PCMCIA currently treats cards with no CIS as
> static RAM cards.  Mostly because the issue has never really come up
> for me.  I think that's a bit different from an unqualified "they do
> not work very well at all".

As far as memory devices go, detecting is a big part of the entire
process. I also had lots of weird trouble getting this memory card up, I
finally did it but it was quite trying.
 
> Are linear flash cards with no CIS common?  I'm only familiar with the
> Intel cards, which all seem to have a CIS.  I've never had anyone ask
> about no-CIS flash card support, as far as I can remember.

On all the memory cards I've seen the CIS is stored on the flash itself,
so if you erase the CIS then it's gone. One of our systems here boots
from a PCMCIA memory card, putting a CIS in there gets in the way. So for
most of my purposes the current driver doesn't work.

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[David Hinds]

On Tue, Aug 10, 1999 at 03:20:36PM -0600, Jason Gunthorpe wrote:
> 
> As far as memory devices go, detecting is a big part of the entire
> process. I also had lots of weird trouble getting this memory card up, I
> finally did it but it was quite trying.

Hmmm, I'd have thought that getting the flash algorithms right was the
bigger issue (that's certainly what took the most time for me).

> On all the memory cards I've seen the CIS is stored on the flash itself,
> so if you erase the CIS then it's gone. One of our systems here boots
> from a PCMCIA memory card, putting a CIS in there gets in the way. So for
> most of my purposes the current driver doesn't work.

What cards, specifically, are you using?

All Intel Series 2 and 2+ cards put the CIS in a separate chip.  The
Series 100 cards have it in the first flash block.

-- Dave


To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[Jason Gunthorpe]

On Tue, 10 Aug 1999, David Hinds wrote:

> > On all the memory cards I've seen the CIS is stored on the flash itself,
> > so if you erase the CIS then it's gone. One of our systems here boots
> > from a PCMCIA memory card, putting a CIS in there gets in the way. So for
> > most of my purposes the current driver doesn't work.
> 
> What cards, specifically, are you using?

These cards are from AMD, memory only cards for use with their UforCE
board. It boots off them (the card stores the bios).

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[David Woodhouse]

dhinds@zen.stanford.edu said:
>  That's certainly true: PCMCIA currently treats cards with no CIS as
> static RAM cards.  Mostly because the issue has never really come up
> for me.  I think that's a bit different from an unqualified "they do
> not work very well at all". 

OK then - would you accept a patch which performs CFI or JEDEC identification 
in the case that there's no CIS, rather than just assuming SRAM? Jason, would 
you be willing to write this and submit it to him?

Also, how would you like us to go about making PCMCIA devices use the MTD 
system? 

>From what I understand of Jason's "mapped" helper, it provides simple way for 
MTD devices with memory windows to SRAM or NOR flash to provide a single 
'change window mapping' routine, and then to be handled automatically. Am I 
right in thinking that most (all?) PCMCIA memory devices work this way? 

If so, I think the best way to get PCMCIA devices to use this system with 
minimal code duplication is to write a new driver for PCMCIA MTDs, which links 
to Jason's helper code. That way, the original drivers aren't littered with 
ifdefs which make them difficult to maintain.

We can provide those drivers separately, as part of our tarball, until such 
time as we get Linus to include the MTD subsystem into his kernel, hopefully 
near the start of the 2.5 series. At that point, they can be imported into 
your PCMCIA tree for conditional compilation when the MTD system is detected 
in the kernel.

How does that sound?



----                                 ----                                 ----
David Woodhouse        David.Woodhouse@mvhi.com       Office: (+44) 1223 810302
 Project Leader,     Process Information Systems      Mobile: (+44) 976 658355
    Axiom (Cambridge) Ltd., Swaffham Bulbeck, Cambridge, CB5 0NA, UK.
             finger dwmw2@ferret.lmh.ox.ac.uk for PGP key.




To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[David Woodhouse]

jgg@deltatee.com said:
> On Tue, 10 Aug 1999, David Woodhouse wrote:
> > Some extra complexity is required for the erases, though, to allow
> > timer-based operation and retries. I didn't really pay much attention to
> > erases as I pulled this lot together though - just used what I had last
> > time. So feel free to point out anything that's not right. 

>  That should all go into the highest layer, the device stuff doesn't
> need to know it is being called asynchrounously. 

That's what I originally thought. However, erases take a long time, and we 
often don't need them to be done synchronously - mostly we'll keep a spare 
erase block already clean, and the newly-erased ones won't be used immediately.

Also, in multi-chip devices, an erase can be in progress on one chip while 
other things are happening to the other chips. Erases can even be interrupted, 
I believe, to allow reads to be performed. That kind of optimisation isn't 
possible unless we allow asynchronous erases.


>  If the DoC is just an array of normal flash chips with some paging
> mechanism then you could probably [use the mapped driver] quite easially.

The DiskOnChip 1000 is. So it should be quite simple, yes. But I have no access 
to DiskOnChip 1000 hardware at the moment - they're out of production. 
M-Systems are trying to see if they can lay their hands on some for me.
If we can't get them, then I'll probably just pull the DoC1000 driver 
completely.

>  Hum, that's kinda a bummer - Being able to prepare a flash image on
> my desktop and then just bang the whole thing onto device is very
> usefull - that is our primary means of deploying remote upgrades.

>  You still need to format the flash with the FTL information, it is
> convient to just have that pre-available in an image. 

Convenience is just a matter of having the tools readily available. If we 
provide a tool which can take a block device image and dump that to flash, 
formatting an NFTL partition as it does so, then it's just as convenient, 
surely? And we can use a standard loopback mount to create and maintain the 
filesystem for the device.

I agree that formatting and then populating the NFTL partition isn't 
quite as convenient (two command lines instead of one), but even that's 
scriptable.

It's fairly easy to make a dump/restore utility dumps the whole of the flash
intact, but that's of limited usefulness. Blindly duplicating the flash will
copy block erase counts and other such information that then becomes invalid on
the new media. For duplicating onto production systems, you'd do just as well
to use the format-as-you-go approach above.


jgg@deltatee.com said (re FFS2 on block devices):
>  The plan is to make it ask the MTD drive if the device it was mounted
> against is a MTD device and get a pointer to the MTD device block.
> Then it will not speak block or character, it will just directly
> manipulate the device. That solves the problem of having to ensure
> buffer cache consistency during erasures and so on.. 

Good - that's exactly what I was thinking of.

----                                 ----                                 ----
David Woodhouse        David.Woodhouse@mvhi.com       Office: (+44) 1223 810302
 Project Leader,     Process Information Systems      Mobile: (+44) 976 658355
    Axiom (Cambridge) Ltd., Swaffham Bulbeck, Cambridge, CB5 0NA, UK.
             finger dwmw2@ferret.lmh.ox.ac.uk for PGP key.




To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

mtd-19990809 including readonly DiskOnChip driver.

[David Woodhouse]

I've updated the MTD package to include the modifications and new drivers made
by Jason Gunthorpe, and also the current version of my DiskOnChip and NFTL
code. 

It's available at:
	ftp://ftp.linux-mtd.infradead.org/pub/mtd/mtd-19990809.tar.gz

It should contain working drivers for:
	slram		- uncached system memory
	doc1000		- The old DiskOnChip 1000 and ISA flash cards.
	doc2000 	- DiskOnChip2000 (not yet Millennium)
	octagon-5066	- Memory-mapped flash found on a Single Board Computer
	vmax301 	- Likewise.

The devices are presented to the user by default as a character device. 
However, three block devices are available:
	
	ftl		- The original FTL filesystem, as used by PCMCIA
	nftl		- The new (NAND) FTL system, for the DiskOnChip
	mtdblock	- Direct block device access to the MTD

For some reason, the mtdblock_request routine seem to never be called. I'm 
obviously doing something stupid, so I'll let one of you tell me why that is :)

Also included is Jason's FFS2 implementation. The bootloader stuff is also 
present, but commented out in the top-level Makefile because I haven't yet 
upgraded my as86. I'd prefer it if we could make this work without 
bleeding-edge utilites. Jason, is that possible?

Soon, I'm intending to merge Alexander Larsson's 'flash.diff' for onboard 
CFI-compliant memories. Volunteers to do that would be appreciated, because it 
would let me concentrate on completing the DiskOnChip and NFTL code.

Happy Hacking.

----                                 ----                                 ----
David Woodhouse        David.Woodhouse@mvhi.com       Office: (+44) 1223 810302
 Project Leader,     Process Information Systems      Mobile: (+44) 976 658355
    Axiom (Cambridge) Ltd., Swaffham Bulbeck, Cambridge, CB5 0NA, UK.
             finger dwmw2@ferret.lmh.ox.ac.uk for PGP key.




To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[Jason Gunthorpe]

On Mon, 9 Aug 1999, David Woodhouse wrote:

> Also included is Jason's FFS2 implementation. The bootloader stuff is also 
> present, but commented out in the top-level Makefile because I haven't yet 
> upgraded my as86. I'd prefer it if we could make this work without 
> bleeding-edge utilites. Jason, is that possible?

The trouble is that your as86 is from RedHat, they are using a bleeding
edge binutils that doesn't entirely work like the real as86 from bcc -
remove the trailing - from the AS invokation and it will work on both.

Put a big warning on the FFS2 stuff too, it needs more testing :>

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk

Re: mtd-19990809 including readonly DiskOnChip driver.

[Jason Gunthorpe]

On Mon, 9 Aug 1999, David Woodhouse wrote:

> Soon, I'm intending to merge Alexander Larsson's 'flash.diff' for onboard 
> CFI-compliant memories. Volunteers to do that would be appreciated, because it 
> would let me concentrate on completing the DiskOnChip and NFTL code.

I've looked around your new bundle and it looks pretty good, if anyone
wants to add the CFI probing routing it fits very nicely into the mapped.c
file, instead of using the flash_jedec routine, a very similar routine
that does CFI probes needs to be written. 

I'm not sure the two AMD chips I use support CFI, anyone know off hand?

Also, using David's driver as the base driver kinda mucks things up - he
uses a different erase mechanism with different meanings, and has some
weird cruft from PCMCIA in there.

Porting the PCMCIA driver and the slram driver to use the mapped interface
would save alot of code as well, compare:

blue{jgg}/tmp/mtd-19990809/kernel#wc -l slram.c octagon-5066.c
    210 slram.c
    205 octagon-5066.c

For such a simple device that is quite needlessly long. (mapped already
implements ram devices)
 
I miss devfs support too :<

Also, do your new NFTL and FTL drivers work over normal block devices?

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@imladris.demon.co.uk