why MTD model ?

why MTD model ?

[Studying MTD]

hi all,

i am curious why we need MTD model ?

Can we use ordinary linux driver model for memory
devices ?

Thank you for your help.


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

Re: why MTD model ?

[David Woodhouse]

studying_mtd@yahoo.com said:
>  i am curious why we need MTD model ?
> Can we use ordinary linux driver model for memory devices ?

To what 'ordinary Linux driver model' are you referring?

--
dwmw2

Re: why MTD model ?

[Studying MTD]

Can somehow we treat(emulate) memory device as a block
device and treat memory device like a hard disk ?

--- David Woodhouse <dwmw2@infradead.org> wrote:
> 
> studying_mtd@yahoo.com said:
> >  i am curious why we need MTD model ?
> > Can we use ordinary linux driver model for memory
> devices ?
> 
> To what 'ordinary Linux driver model' are you
> referring?
> 
> --
> dwmw2
> 
> 
> 
>
______________________________________________________
> Linux MTD discussion mailing list
>
http://lists.infradead.org/mailman/listinfo/linux-mtd/


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

RE: why MTD model ?

[Gregg C Levine]

Hello from Gregg C Levine
As I understand the whole notion behind the MTD collection, the Disk On
Chip devices, corresponds to what you are thinking of. However there is
a module which allows ordinary machine memory to be used for the same
purpose as another MTD array, for testing an application. Please note
that I have not tested any of these theories, nor actually seen a DOC in
action. (A board running, but booting from a DOC, as opposed to other
hardware.)
-------------------
Gregg C Levine hansolofalcon@worldnet.att.net
------------------------------------------------------------
"The Force will be with you...Always." Obi-Wan Kenobi
"Use the Force, Luke."  Obi-Wan Kenobi
(This company dedicates this E-Mail to General Obi-Wan Kenobi )
(This company dedicates this E-Mail to Master Yoda )



> -----Original Message-----
> From: linux-mtd-admin@lists.infradead.org [mailto:linux-mtd-
> admin@lists.infradead.org] On Behalf Of Studying MTD
> Sent: Thursday, June 13, 2002 12:19 PM
> To: David Woodhouse
> Cc: linux-mtd
> Subject: Re: why MTD model ?
> 
> Can somehow we treat(emulate) memory device as a block
> device and treat memory device like a hard disk ?
> 
> --- David Woodhouse <dwmw2@infradead.org> wrote:
> >
> > studying_mtd@yahoo.com said:
> > >  i am curious why we need MTD model ?
> > > Can we use ordinary linux driver model for memory
> > devices ?
> >
> > To what 'ordinary Linux driver model' are you
> > referring?
> >
> > --
> > dwmw2
> >
> >
> >
> >
> ______________________________________________________
> > Linux MTD discussion mailing list
> >
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! - Official partner of 2002 FIFA World Cup
> http://fifaworldcup.yahoo.com
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: why MTD model ?

[David Woodhouse]

studying_mtd@yahoo.com said:
> Can somehow we treat(emulate) memory device as a block device and
> treat memory device like a hard disk ?

Not directly, no. There are drivers which do so -- and present a flash 
device as a block device using translation layers of various complexity, 
but that is not a true representation of the capabilities of the underlying 
device.

--
dwmw2

Re: why MTD model ?

[Studying MTD]

--- David Woodhouse <dwmw2@infradead.org> wrote:
> 
> studying_mtd@yahoo.com said:
> > Can somehow we treat(emulate) memory device as a
> block device and
> > treat memory device like a hard disk ?
> 
> Not directly, no. There are drivers which do so --
> and present a flash 
> device as a block device using translation layers of
> various complexity, 

you mean, it is possible.


> but that is not a true representation of the
> capabilities of the underlying 
> device.
> 

what you mean by "true representation of the
capabilities" ?

what i will miss, if i use memory flash device as
block device and merge memory flash device with other
block devices ?

what is the advantage of MTD model with respect to
block devices , why we use MTD ?

thanks for your help.

> --
> dwmw2
> 
> 
> 
>
______________________________________________________
> Linux MTD discussion mailing list
>
http://lists.infradead.org/mailman/listinfo/linux-mtd/


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

Re: why MTD model ?

[David Woodhouse]

studying_mtd@yahoo.com said:
> you mean, it is possible.
> > but that is not a true representation of the
> > capabilities of the underlying 
> > device.
> 
> what you mean by "true representation of the capabilities" ?
> what i will miss, if i use memory flash device as block device and
> merge memory flash device with other block devices ?

Flash devices have large erase blocks. You cannot just treat them as a block
device with a sector size of 64KiB, etc. A flash device can have sectors
erased independently of write operations, can have write operations
performed independently of erases (e.g. JFFS2 does so just to clear one
extra 'valid' bit in existing nodes', can support writes to arbitrary byte
ranges, etc. The MTD API allows you to make use of those features.

The block device model does not offer a way to handle any of that. It only 
allows you to make atomic updates of fixed-size sectors, which is not 
something that flash devices are naturally capable of. To use flash as a 
block device, you have to have some kind of 'translation layer' hack.

The simplest we have is the 'mtdblock' one, which on receiving a write 
request simply reads the whole of the erase block which it landed in, 
erases that block, then writes it all back out again with the offending 
sector modified. That's obviously very unsafe, but it's OK for setting up 
file systems which are going to be read-only in production.

Others are more complicated and safe w.r.t. power failure, essentially a
complete journalling file system in themselves just to emulate a block
device with small sectors. On top of which people put 'normal' journalling 
file systems.

Having a journalling file system atop a journalling file system sucks. We 
do far better by exposing an API which represents the true functionality of 
the underlying devices and designing a file system to make use of that 
directly. 

--
dwmw2

Re: why MTD model ?

[Studying MTD]

--- David Woodhouse <dwmw2@infradead.org> wrote:
> 
> Flash devices have large erase blocks. You cannot
> just treat them as a block
> device with a sector size of 64KiB, etc. 

We can change the sector size on block devices.

> A flash device can have sectors
> erased independently of write operations, can have
> write operations
> performed independently of erases 

I think, this is not neccesary to keep erase
independent from writing, we can erase before we
write.

> (e.g. JFFS2 does so just to clear one
> extra 'valid' bit in existing nodes', can support
> writes to arbitrary byte
> ranges, etc. 

It is not neccesary that we use JFFS2, we can use any
filesystem on flash but at low level when we are
writing to flash, we can use wear levelling.

> The MTD API allows you to make use of those
features.

Are these are only feature provided by MTD ?

I am not able to understand what is special in MTD
model ?

Please help me.

thanks for your help.


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

Re: why MTD model ?

[David Woodhouse]

studying_mtd@yahoo.com said:
>  We can change the sector size on block devices. 

Even if Linux could cope with sectors larger than PAGE_SIZE, the amount of
space wasted by using 64KiB sectors would be prohibitive. Perhaps reiserfs 
with its tail-packing option could cope, but I doubt it very much.

>  I think, this is not neccesary to keep erase independent from
> writing, we can erase before we write.

Not if you want to be able to write fewer data than an entire eraseblock at 
a time, or if you care about limiting the number of erase cycles.

> It is not neccesary that we use JFFS2, we can use any filesystem on
> flash but at low level when we are writing to flash, we can use wear
> levelling. 

Indeed, this is as I said. You can use a 'normal' journalling file system on
top of another pseudo-filesystem which emulates a block device. Your
'normal' file system will write each block of data to the flash twice --
once to its 'journal' and then again to the actual file system structure, 
while your underlying pseudo-filesystem will faithfully log this activity.

If you are misguided enough to desire this behaviour, it is possible. The 
translation layer is a 'user' of an underlying MTD device driver which 
actually performs the fundamental read/write/erase operations on the flash 
chips. The MTD API represents those fundamental capabilities of the 
hardware.

>  Are these are only feature provided by MTD ? 

By the MTD API, yes. The sole purpose of the MTD API is to represent the
functionality of Memory Technology Devices, and therefore that is all it 
does.

The MTD code base contains drivers for various flash chips, real flash file
systems, and also some of these 'translation layers' which are used to 
emulate a block device. 

>  I am not able to understand what is special in MTD model ? 

There is nothing particularly special about it. It merely represents the
capabilities of the hardware in a way which makes it possible to use it
effectively.

--
dwmw2

Re: why MTD model ?

[Studying MTD]

Dear dwmw2,

Thanks for your help.

If some how i solve these issues :-
1. sector size
2. erase/write
3. wear levelling

Please let me know, which block device is closest to
Flash memory :-
1. Hard Disk
2. Floppy drive
3. PCMCIA memory card
4. Ramdisk
5. or some other block device ?

Thanks for your help.




__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

Re: why MTD model ?

[David Woodhouse]

studying_mtd@yahoo.com said:
>  If some how i solve these issues :- 
> 1. sector size
> 2. erase/write
> 3. wear levelling

The core MTD API isn't there to solve those issues for you, only to
represent the raw hardware -- and we have existing drivers for many types of
raw hardware.

Those issues are solved by whatever _uses_ the MTD devices -- and there are 
a number of solutions already implemented in the MTD code base. The most
sensible way to put a file system on flash is to use a file system
especially designed for that purpose -- such as JFFS2. However, for legacy 
reasons we also have code which uses MTD devices to emulate hard drives, as 
for some strange and inexplicable reason you seem to want.

In the days of DOS, it was difficult to add a real file system, so instead 
people came up with a gross hack to make a flash device appear just like a 
normal BIOS-supported disk drive.

They designed a pseudo-filesystem which emulated a hard drive, and then
provided a BIOS extension which overrode the INT 13h (Disk BIOS) interrupt
handler. Then people would just use a FAT file system on it under DOS as if
it was a normal hard drive. 

When Windows started to gain its own 32-bit device drivers for hard drives, 
the pseudo-filesystems that emulated hard drives were rewritten to run 
under Windows, but people were so used to the idea of a pseudo-fs emulating 
a block device with a 'real' fs on top of that, that the opportunity wasn't 
taken to make the whole thing sane just by having a proper file system 
directly on the flash.

The only half-sane reason for using a translation layer to emulate a block 
device like this is on removable media -- if you need to be able to remove 
the flash device from your Linux box and read it in a DOS or Windows box. 

If your flash isn't going to be removable, then you really don't want to do 
it that way, for reasons which I've discussed at length quite recently.

You should be using a proper file system directly on the flash. This is not 
DOS, and you don't have to repeat DOS-based mistakes.

> Please let me know, which block device is closest to Flash memory :-
> 1. Hard Disk 2. Floppy drive 3. PCMCIA memory card 4. Ramdisk 5. or
> some other block device ? 

Your question is very ambiguous. You might as well ask which tropical fish 
is closest to flash memory. No block device is _similar_ to flash -- they are 
entirely different beasts.

I suppose the hard drive sitting on my desk with a pile of flash chips on 
top of it is 'closest to Flash memory' in one way, but I suspect that's not 
what you were asking.

If you mean which block device is most similar in performance and general 
operation to a system which uses flash to emulate a block device, then the 
answer would be PCMCIA-ATA or CompactFlash cards -- because they _are_ 
using precisely such a translation layer internally. The only difference is 
that they do it internally, in firmware (and generally quite unreliably).

If you mean which block device driver code is most similar to the code 
you'd have to write to implement yet another translation layer, then that 
would be something like the FTL or NFTL code -- they both use the 
underlying MTD drivers for the raw hardware, and perform their own magic 
to present a block device implementation to Linux.

--
dwmw2

RE: why MTD model ?

[Gregg C Levine]

Hello from Gregg C Levine
I agree David with you on all points, except one. Regarding the
reliability of PCMCIA-ATA drives. I have a bunch here right now, and
they are proving to be more reliable then the disk drives I keep buying
from the same shop. The fact that the manufacturer decided to
discontinue them because of size, is irrelevant. The fact that they came
with a FAT-12 file system, and a partition to match is also irrelevant.
-------------------
Gregg C Levine hansolofalcon@worldnet.att.net
------------------------------------------------------------
"The Force will be with you...Always." Obi-Wan Kenobi
"Use the Force, Luke."  Obi-Wan Kenobi
(This company dedicates this E-Mail to General Obi-Wan Kenobi )
(This company dedicates this E-Mail to Master Yoda )



> -----Original Message-----
> From: linux-mtd-admin@lists.infradead.org [mailto:linux-mtd-
> admin@lists.infradead.org] On Behalf Of David Woodhouse
> Sent: Friday, June 14, 2002 8:22 AM
> To: Studying MTD
> Cc: linux-mtd
> Subject: Re: why MTD model ?
> 
> 
> studying_mtd@yahoo.com said:
> >  If some how i solve these issues :-
> > 1. sector size
> > 2. erase/write
> > 3. wear levelling
> 
> The core MTD API isn't there to solve those issues for you, only to
> represent the raw hardware -- and we have existing drivers for many
types of
> raw hardware.
> 
> Those issues are solved by whatever _uses_ the MTD devices -- and
there are
> a number of solutions already implemented in the MTD code base. The
most
> sensible way to put a file system on flash is to use a file system
> especially designed for that purpose -- such as JFFS2. However, for
legacy
> reasons we also have code which uses MTD devices to emulate hard
drives, as
> for some strange and inexplicable reason you seem to want.
> 
> In the days of DOS, it was difficult to add a real file system, so
instead
> people came up with a gross hack to make a flash device appear just
like a
> normal BIOS-supported disk drive.
> 
> They designed a pseudo-filesystem which emulated a hard drive, and
then
> provided a BIOS extension which overrode the INT 13h (Disk BIOS)
interrupt
> handler. Then people would just use a FAT file system on it under DOS
as if
> it was a normal hard drive.
> 
> When Windows started to gain its own 32-bit device drivers for hard
drives,
> the pseudo-filesystems that emulated hard drives were rewritten to run
> under Windows, but people were so used to the idea of a pseudo-fs
emulating
> a block device with a 'real' fs on top of that, that the opportunity
wasn't
> taken to make the whole thing sane just by having a proper file system
> directly on the flash.
> 
> The only half-sane reason for using a translation layer to emulate a
block
> device like this is on removable media -- if you need to be able to
remove
> the flash device from your Linux box and read it in a DOS or Windows
box.
> 
> If your flash isn't going to be removable, then you really don't want
to do
> it that way, for reasons which I've discussed at length quite
recently.
> 
> You should be using a proper file system directly on the flash. This
is not
> DOS, and you don't have to repeat DOS-based mistakes.
> 
> > Please let me know, which block device is closest to Flash memory :-
> > 1. Hard Disk 2. Floppy drive 3. PCMCIA memory card 4. Ramdisk 5. or
> > some other block device ?
> 
> Your question is very ambiguous. You might as well ask which tropical
fish
> is closest to flash memory. No block device is _similar_ to flash --
they are
> entirely different beasts.
> 
> I suppose the hard drive sitting on my desk with a pile of flash chips
on
> top of it is 'closest to Flash memory' in one way, but I suspect
that's not
> what you were asking.
> 
> If you mean which block device is most similar in performance and
general
> operation to a system which uses flash to emulate a block device, then
the
> answer would be PCMCIA-ATA or CompactFlash cards -- because they _are_
> using precisely such a translation layer internally. The only
difference is
> that they do it internally, in firmware (and generally quite
unreliably).
> 
> If you mean which block device driver code is most similar to the code
> you'd have to write to implement yet another translation layer, then
that
> would be something like the FTL or NFTL code -- they both use the
> underlying MTD drivers for the raw hardware, and perform their own
magic
> to present a block device implementation to Linux.
> 
> --
> dwmw2
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

RE: why MTD model ?

[Gregg C Levine]

Hello from Gregg C Levine
If by top posting you mean sending you mail, David, and then the other
name, and finally the list, then fine. This one is only to the list. And
I'm just stating a simple fact. Yes, I agree with you, the FAT partition
isn't necessary. The cards just came with it. And no the UMSDOS method
of creating a Linux distribution does not work in the 2.4.x series. I
don't know why. The folks at Slackware never told me. But I am curious
which patches need to be applied within the patch directory of your MTD
collection to the 2.2.17 kernel to make it work?
-------------------
Gregg C Levine hansolofalcon@worldnet.att.net
------------------------------------------------------------
"The Force will be with you...Always." Obi-Wan Kenobi
"Use the Force, Luke."  Obi-Wan Kenobi
(This company dedicates this E-Mail to General Obi-Wan Kenobi )
(This company dedicates this E-Mail to Master Yoda )



> -----Original Message-----
> From: David Woodhouse [mailto:dwmw2@redhat.com] On Behalf Of David
> Woodhouse
> Sent: Friday, June 14, 2002 8:48 AM
> To: Gregg C Levine
> Cc: 'Studying MTD'
> Subject: Re: why MTD model ?
> 
> 
> No top-posting here please.
> 
> hansolofalcon@worldnet.att.net said:
> > I agree David with you on all points, except one. Regarding the
> > reliability of PCMCIA-ATA drives. I have a bunch here right now, and
> > they are proving to be more reliable then the disk drives I keep
> > buying from the same shop. The fact that the manufacturer decided to
> > discontinue them because of size, is irrelevant. The fact that they
> > came with a FAT-12 file system, and a partition to match is also
> > irrelevant.
> 
> Well, if you're comparing with the reliability of cheap IDE drives,...
:)
> 
> I haven't really dealt with CompactFlash or PCMCIA-ATA cards much, I'm
only
> repeating what others have reported.
> 
> But certainly you don't have to keep the partitioning and the FAT file
> system. You can blow it away and just put an ext2 or ext3 file system
on the
> whole device. Although as discussed, ext2 (just like FAT) isn't safe
w.r.t
> power loss and ext3 causes you to wear the flash out far quicker than
you
> need to.
> 
> I don't think anyone's insane enough to try to run a Linux box with a
FAT
> root file system any more -- does the UMSDOS hack even compile in
recent
> kernels? Without it you won't get device nodes, symlinks, permissions,
etc.
> 
> --
> dwmw2
> 

Re: why MTD model ?

[David Woodhouse]

hansolofalcon@worldnet.att.net said:
> If by top posting you mean sending you mail, David, and then the other
> name, and finally the list, then fine. This one is only to the list.

By top-posting I mean annoying practice of quoting the entire mail and just 
typing your own response at the top. It is normal to quote only those parts 
of the mail to which you're actually responding, and your response to those 
parts immediately below the quoted text.

> And I'm just stating a simple fact. Yes, I agree with you, the FAT
> partition isn't necessary. The cards just came with it. And no the
> UMSDOS method of creating a Linux distribution does not work in the
> 2.4.x series. I don't know why. The folks at Slackware never told me.

UMSDOS was always a bit of an evil hack. When all the core Linux file system
code was overhauled during the 2.3 series, nobody cared enough to update it,
and I believe the new saner core VFS code was more difficult to abuse in the
ways the UMSDOS needed to.

> But I am curious which patches need to be applied within the patch
> directory of your MTD collection to the 2.2.17 kernel to make it work?

Just the inter-module-xxx one, I think.

--
dwmw2

Re: why MTD model ?

[Eric W. Biederman]

Studying MTD <studying_mtd@yahoo.com> writes:

> Please let me know, which block device is closest to
> Flash memory :-
> 1. Hard Disk
> 2. Floppy drive
> 3. PCMCIA memory card
> 4. Ramdisk
> 5. or some other block device ?

The closest are CD-RW disks....

I wonder if it would make sense to port them to the mtd layer?

Eric