Need help understanding cfi_probe and cfi_cmdset_0002

Need help understanding cfi_probe and cfi_cmdset_0002

[jglennon]

I find that qry_present is failing on MVME5100

target board:   Motorola MVME5100, CPU=MPC750
                FLASH = 4x AMD29D323D, is CFI.

kernel:  2.4.2 (Hard Hat Linux 2.0)
MTD:     CVS download of about Tuesday (10/30/01)

I do not see how to "make it" do cfi_cmdset_0002 (amdstd CFI) stuff.
It goes through gen_probe, and since it didn't correctly put chips into CFI
query mode, the qry_present fails.


It appears like the command to enter CFI Query mode not working; subsequent
cfi_reads return "normal" flash data, I think.  So qry_present returns 1.

I say it looks like normal FLASH contents because I see the stuff cfi_read
returns when I use PPC_BUG6> m 0xF4000000 and scroll on down to ...80 which
is 0x10 * osf.

I am likely missing something basic.  Like incorrect xconfig settings!
Help appreciated!

Enclosed is .config.
Thank you!

Joe Glennon
GET Engineering
San Diego CA
---------------------
.config:
(deleted down to MTD stuff...)
#
# Memory Technology Devices (MTD)
#
CONFIG_MTD=y
CONFIG_MTD_DEBUG=y
CONFIG_MTD_DEBUG_VERBOSE=3
# CONFIG_MTD_PARTITIONS is not set

#
# User Modules And Translation Layers
#
CONFIG_MTD_CHAR=y
# CONFIG_MTD_BLOCK is not set
# CONFIG_MTD_BLOCK_RO is not set
# CONFIG_FTL is not set
# CONFIG_NFTL is not set

#
# RAM/ROM/Flash chip drivers
#
CONFIG_MTD_CFI=y
# CONFIG_MTD_JEDECPROBE is not set
CONFIG_MTD_GEN_PROBE=y
CONFIG_MTD_CFI_ADV_OPTIONS=y
CONFIG_MTD_CFI_NOSWAP=y
# CONFIG_MTD_CFI_BE_BYTE_SWAP is not set
# CONFIG_MTD_CFI_LE_BYTE_SWAP is not set
CONFIG_MTD_CFI_GEOMETRY=y
# CONFIG_MTD_CFI_B1 is not set
# CONFIG_MTD_CFI_B2 is not set
CONFIG_MTD_CFI_B4=y
# CONFIG_MTD_CFI_I1 is not set
# CONFIG_MTD_CFI_I2 is not set
CONFIG_MTD_CFI_I4=y
# CONFIG_MTD_CFI_INTELEXT is not set
CONFIG_MTD_CFI_AMDSTD=y
# CONFIG_MTD_RAM is not set
# CONFIG_MTD_ROM is not set
# CONFIG_MTD_ABSENT is not set
# CONFIG_MTD_OBSOLETE_CHIPS is not set

#
# Mapping drivers for chip access
#
CONFIG_MTD_PHYSMAP=y
CONFIG_MTD_PHYSMAP_START=0xF4000000
CONFIG_MTD_PHYSMAP_LEN=0x1000000
CONFIG_MTD_PHYSMAP_BUSWIDTH=4
# CONFIG_MTD_TQM8XXL is not set
# CONFIG_MTD_RPXLITE is not set
# CONFIG_MTD_CFI_FLAGADM is not set
# CONFIG_MTD_PCI is not set

 --(No self-contained configured...)
 --(No DOC configured...)

# CONFIG_MTD_NAND is not set
# Block devices
#
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_NBD is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y

(deletions)


#
# File systems
#
# CONFIG_JFFS_FS is not set
CONFIG_JFFS2_FS=y
CONFIG_JFFS2_FS_DEBUG=3
# CONFIG_CRAMFS is not set
# CONFIG_RAMFS is not set
CONFIG_ISO9660_FS=y
CONFIG_PROC_FS=y
CONFIG_DEVPTS_FS=y
CONFIG_EXT2_FS=y



______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Need help understanding cfi_probe and cfi_cmdset_0002

[David Woodhouse]

jglennon@getntds.com said:
> target board:   Motorola MVME5100, CPU=MPC750
>                 FLASH = 4x AMD29D323D, is CFI.

Are you 100% sure these are arranged as 4 8-bit devices side-by-side on the 
32-bit bus? It's more common in my experience to have 2 devices together in 
16-bit mode, and to have two such pairs consecutively mapped in the address 
space if there are four chips.

Turn off CONFIG_MTD_CFI_ADV_OPTIONS and let it try all the probes.

--
dwmw2

mkfs.jffs2 propsal and a question about burst reads

[Joakim Tjernlund]

Hi it's me again ..

with some more questions/suggestions

1)
   Before I can install a new JFFS2 FS image(produced with mkfs.jffs2) I have
   to erase the whole flash partition, then copy the image(with cp image /dev/mtdlock4)
   the device. Then I boot linux and now JFFS2 will re-erase all unused blocks. 
   If I skip the initial erase, then JFFS2 will be confused by the earlier
   JFFS2 contens which remain after the previous JFFS2 FS that were on the flash.

   I think this is the expected behavior, correct?

   I find it unconvinient and time consuming to erase the WHOLE flash partiton
   before I can install a new FS image. Therefore I wonder if it would
   be possible to add some flag/end marker in JFFS2 that mkfs.jffs2 could
   set, so that JFFS2 knows that the flash space after the maker does not
   contain valid JFFS2 data. Once JFFS2 has recongnized its FS, JFFS2 would
   delete the end marker and resume normal behaviour. 
   
   What do you think?  

2)
   I am using Intel Strata Flash and this flash can do burst reads, so was
   thinking that if I were to enable burst read I would gain performance.

   I rember some discussion about this a while back, but I did not see 
   any positive reports about gained performance. Has anybody done
   this and did you gain any performance?

  Joakim 

Re: mkfs.jffs2 propsal and a question about burst reads

[Jörn Engel]

Moin!

> 1)
>    Before I can install a new JFFS2 FS image(produced with mkfs.jffs2) I have
>    to erase the whole flash partition, then copy the image(with cp image /dev/mtdlock4)
>    the device. Then I boot linux and now JFFS2 will re-erase all unused blocks. 
>    If I skip the initial erase, then JFFS2 will be confused by the earlier
>    JFFS2 contens which remain after the previous JFFS2 FS that were on the flash.
> 
>    I think this is the expected behavior, correct?
> 
>    I find it unconvinient and time consuming to erase the WHOLE flash partiton
>    before I can install a new FS image. Therefore I wonder if it would
>    be possible to add some flag/end marker in JFFS2 that mkfs.jffs2 could
>    set, so that JFFS2 knows that the flash space after the maker does not
>    contain valid JFFS2 data. Once JFFS2 has recongnized its FS, JFFS2 would
>    delete the end marker and resume normal behaviour. 

That would be simple, if jffs2 was strictly log-structured. But from
what I have heard/read about it, the wear levelling makes all delete
blocks equal. There is no last block, that you can mark in some way.

Hmm. BTW, is it just me, or do other peaople agree, that the j in jffs
is a bug? Journaling has little to do with a log-structure.

Jörn

-- 
When your language is nowhere near Turing-complete, 
syntactic sugar can be your friend.

Re: mkfs.jffs2 propsal and a question about burst reads

[Michael Rothwell]

> Hmm. BTW, is it just me, or do other peaople agree,
> that the j in jffs is a bug? Journaling has little
> to do with a log-structure.

Well, you can think of a log-structured filesystem as "all journal," so the
"J" applies.

Re: mkfs.jffs2 propsal and a question about burst reads

[Jörn Engel]

> Well, you can think of a log-structured filesystem as "all journal," so the
> "J" applies.

As far as I recall, the "trick" with journaling filesystems is to log
only the metadata. On normal hard drives, this speeds up the most
common operations, reads, and is still sufficient for most people.
Real log-structured filesystems are considered slow in every-day
useage and to my knowledge only one was ever implemented for some bsd
flavor.
But this is getting quite academic now. :)

Jörn

-- 
In disagreements with loved ones, deal only
with the current situation. Don't bring up the past.

Re: mkfs.jffs2 propsal and a question about burst reads

[Bjorn Wesen]

On Tue, 6 Nov 2001, Jörn Engel wrote:
> > Well, you can think of a log-structured filesystem as "all journal," so the
> > "J" applies.
> 
> As far as I recall, the "trick" with journaling filesystems is to log
> only the metadata. On normal hard drives, this speeds up the most

I think the history was the other way around - the systems which log also
the data as opposed to metadata were introduced later. The literature
about journaling filesystems seems a bit vague on the subject, so I don't
see it as any problem to put all filesystems of this type in a class and
call it "journaling". After all, it's a journal with or without userdata.. 

The name "log structured" is equally confusing since your normal
journaling system is also centred around a log :)

> Real log-structured filesystems are considered slow in every-day
> useage and to my knowledge only one was ever implemented for some bsd
> flavor.

Datalogging as opposed to simply metadata logging is necessary on a flash
because you can't rewrite blocks at random. It's probably not a good
technique on a harddisk except if you really want to guarantee actual file
consistency on a crash, but that would require transactional user-mode
support anyway (something we've discussed on jffs-dev some times). 

/BW

Re: mkfs.jffs2 propsal and a question about burst reads

[Jörn Engel]

Hi!

This discussion is increasingly off-topic. But it remains interesting,
so what the hell!

> > As far as I recall, the "trick" with journaling filesystems is to log
> > only the metadata. On normal hard drives, this speeds up the most
> 
> I think the history was the other way around - the systems which log also
> the data as opposed to metadata were introduced later. The literature
> about journaling filesystems seems a bit vague on the subject, so I don't
> see it as any problem to put all filesystems of this type in a class and
> call it "journaling". After all, it's a journal with or without userdata.. 
> 
> The name "log structured" is equally confusing since your normal
> journaling system is also centred around a log :)

Ok, I can live with that. Still, metadata-only-journaling is a very
clumsy way to name something, but since the need for this rarely
arises - oh well!

> Datalogging as opposed to simply metadata logging is necessary on a flash
> because you can't rewrite blocks at random. It's probably not a good
> technique on a harddisk except if you really want to guarantee actual file
> consistency on a crash, but that would require transactional user-mode
> support anyway (something we've discussed on jffs-dev some times). 

On flash, user data journaling (how I hate this) is the only way to
go. A normal filesystem on top of a read-modify-write block driver is
theoretically possible, but it comes close to insanity to actually use
non-atomic "disk"-operations.
On a regular hard drive, user data journaling maximizes write
performance but will only achieve decent read performance when data is
read successively that was written successively, too. Very unlikely.
On the other hand, it is trivial to write a disk defragmentation tool
for these filesystems. Just read and rewrite everything in the
preferred order and the filesystem logic takes care of the rest.

Jörn

-- 
The cheapest, fastest and most reliable components of a computer system are
those that aren't there  --  Gordon Bell, DEC labratories

re: journaling

[Bjorn Wesen]

On Wed, 7 Nov 2001, Jörn Engel wrote:
> On a regular hard drive, user data journaling maximizes write
> performance but will only achieve decent read performance when data is
> read successively that was written successively, too. Very unlikely.

On the other hand, a "normal" filesystem does not optimize the read speed
in any other way either. I don't see how you can optimize in another way
than putting data which belong together logically together
physically.. the LFS can do this even better than traditional systems
because data is moved naturally during the GC process and can be
reorganized.

The main problem with logstructured filesystems is that it's difficult to
scalably _locate_ the data you want to read. That's why JFFS1,2 have long
mount-times. Checkpointing the in-core representation would help a bit,
but it would still not scale well with filesystem size.

Some sort of hierarchical checkpointing could perhaps work.. I haven't
read very much about that yet but it's a natural issue for JFFS3 :)

The second problem with LFS is the GC process. After some time, you
reach a state where the log _is_ full and every write has to force a part
of the GC. There is a lot of tweaking which can be done in that area. As
you note below it solves fragmentation as a byproduct, but getting good
performance and low latency is difficult.

> On the other hand, it is trivial to write a disk defragmentation tool
> for these filesystems. Just read and rewrite everything in the
> preferred order and the filesystem logic takes care of the rest.

/BW

Re: journaling

[Jörn Engel]

Hi!

> On the other hand, a "normal" filesystem does not optimize the read speed
> in any other way either. I don't see how you can optimize in another way
> than putting data which belong together logically together
> physically.. the LFS can do this even better than traditional systems
> because data is moved naturally during the GC process and can be
> reorganized.

Traditionally, read and write optimization means seek minimization, as
traditional FSs are based on hard drives.
Thus read optimization means putting data that will be read together -
large files or all the files in a directory - in one or few spots.
Without GC-optimizations or a defragmentation tool of some sort, LFSs
are very bad at this. And these optimizations are much more
complicated than the  ones for blocklist-filesystems, like ext and
friends.
Making an FS like ext journaling is done simply to get rid of the
fscks on boot time and should only reduce write and mount performance
to a small degree. 

> The main problem with logstructured filesystems is that it's difficult to
> scalably _locate_ the data you want to read. That's why JFFS1,2 have long
> mount-times. Checkpointing the in-core representation would help a bit,
> but it would still not scale well with filesystem size.

Definitely true and to some degree independent of the background
medium. The only optimizations I can think of eat up either RAM or
disk/flash. Not using optimizations eats up cpu an io time. Tough
choice.

> Some sort of hierarchical checkpointing could perhaps work.. I haven't
> read very much about that yet but it's a natural issue for JFFS3 :)

Sounds interesting. Can you quickly describe this one?

> The second problem with LFS is the GC process. After some time, you
> reach a state where the log _is_ full and every write has to force a part
> of the GC. There is a lot of tweaking which can be done in that area. As
> you note below it solves fragmentation as a byproduct, but getting good
> performance and low latency is difficult.

Getting low latency should be nearly impossible in the jffs case. The
write will always have to wait until a complete delete block is
finished. This can take several seconds, depending on your flash.
Performance is achieved by lazyness, usually. Do the least necessary,
maybe a little more for future optimization, but not much. This links
the issue to the location performance issue, as you have to figure out
what data to collect as garbage and what to keep.

Jörn

-- 
Live a good, honorable life. Then when you
get older and think back, you'll be able to enjoy
it a second time.

Re: journaling

[Bjorn Wesen]

On Thu, 8 Nov 2001, Jörn Engel wrote:
> large files or all the files in a directory - in one or few spots.
> Without GC-optimizations or a defragmentation tool of some sort, LFSs
> are very bad at this. And these optimizations are much more

But you can't have a LFS without a GC, so.. 

> > Some sort of hierarchical checkpointing could perhaps work.. I haven't
> > read very much about that yet but it's a natural issue for JFFS3 :)
> 
> Sounds interesting. Can you quickly describe this one?

You need to find structures to put on flash (or disk), which serve the
same purpose that directories do for finding files on a normal filesystem,
but for the log in the jffs case.

> > The second problem with LFS is the GC process. After some time, you
> > reach a state where the log _is_ full and every write has to force a part
> 
> Getting low latency should be nearly impossible in the jffs case. The
> write will always have to wait until a complete delete block is
> finished. This can take several seconds, depending on your flash.

The way out of this is to never fill the flash filesystem, and allow the
GC to operate before any writes, so there always is a bit of room to do a
low-latency write. Both JFFS's now do this, but it requires some awareness
from the system designer.. like, as long as you dont fill the filesystem
you can always write X bytes without waiting. 

Also, you _can_ queue up writes so the writing process does not have to
wait (JFFS2 has a write-queue (?)). As long as you don't loose
synchronicity this might be a good trade-off.

/BW

Re: mkfs.jffs2 propsal and a question about burst reads

[David Woodhouse]

joern@wohnheim.fh-wedel.de said:
>  That would be simple, if jffs2 was strictly log-structured. But from
> what I have heard/read about it, the wear levelling makes all delete
> blocks equal. There is no last block, that you can mark in some way. 

True during normal operation, but when you first program a JFFS2 filesystem 
image into a flash partition you do get it all at the beginning.


--
dwmw2

Re: mkfs.jffs2 propsal and a question about burst reads

[David Woodhouse]

joakim.tjernlund@lumentis.se said:
>    I find it unconvinient and time consuming to erase the WHOLE flash partiton
>    before I can install a new FS image. Therefore I wonder if it would
>    be possible to add some flag/end marker in JFFS2 that mkfs.jffs2 could
>    set, so that JFFS2 knows that the flash space after the maker does not
>    contain valid JFFS2 data. Once JFFS2 has recongnized its FS, JFFS2 would
>    delete the end marker and resume normal behaviour. 

Then JFFS2 would just have to erase the blocks in question anyway. Better 
to just fix your bootloader or whatever you use to program the image in the 
first place so that it erases any sectors which need it, and puts a clean 
marker in them.

>    I am using Intel Strata Flash and this flash can do burst reads, so was
>    thinking that if I were to enable burst read I would gain performance. 

You need chipset magic to do that. 

--
dwmw2

RE: mkfs.jffs2 propsal and a question about burst reads

[Joakim Tjernlund]

> 
> joakim.tjernlund@lumentis.se said:
> >    I find it unconvinient and time consuming to erase the WHOLE flash partiton
> >    before I can install a new FS image. Therefore I wonder if it would
> >    be possible to add some flag/end marker in JFFS2 that mkfs.jffs2 could
> >    set, so that JFFS2 knows that the flash space after the maker does not
> >    contain valid JFFS2 data. Once JFFS2 has recongnized its FS, JFFS2 would
> >    delete the end marker and resume normal behaviour. 
> 
> Then JFFS2 would just have to erase the blocks in question anyway. Better 
> to just fix your bootloader or whatever you use to program the image in the 
> first place so that it erases any sectors which need it, and puts a clean 
> marker in them.

hmm, I don't follow you here. I don't mind that much that the blocks are erased
again by JFFS2, since it will(should) be a background job, but I would like
like to be able to tell JFFS2 whatever is after the initial image should be
considered non valid JFFS2 data and be put on the "to be erased" queue. 

> 
> >    I am using Intel Strata Flash and this flash can do burst reads, so was
> >    thinking that if I were to enable burst read I would gain performance. 
> 
> You need chipset magic to do that.

I have the chipset magic(I think). I am using a MPC860 that has 
User Programmable Machine's(UPM). I am using one UPM to control my SDRAM
and the other UPM could be used to control the Flash. 

> 
> --
> dwmw2
> 
> 

Re: mkfs.jffs2 propsal and a question about burst reads

[Johan Adolfsson]

----- Original Message ----- 
From: Joakim Tjernlund <joakim.tjernlund@lumentis.se>
To: <linux-mtd@lists.infradead.org>
Sent: Tuesday, November 06, 2001 12:40
Subject: mkfs.jffs2 propsal and a question about burst reads


> Hi it's me again ..
> 
> with some more questions/suggestions
> 
> 1)
>    Before I can install a new JFFS2 FS image(produced with mkfs.jffs2) I have
>    to erase the whole flash partition, then copy the image(with cp image /dev/mtdlock4)
>    the device. Then I boot linux and now JFFS2 will re-erase all unused blocks. 
>    If I skip the initial erase, then JFFS2 will be confused by the earlier
>    JFFS2 contens which remain after the previous JFFS2 FS that were on the flash.
> 
>    I think this is the expected behavior, correct?
> 
>    I find it unconvinient and time consuming to erase the WHOLE flash partiton
>    before I can install a new FS image. Therefore I wonder if it would
>    be possible to add some flag/end marker in JFFS2 that mkfs.jffs2 could
>    set, so that JFFS2 knows that the flash space after the maker does not
>    contain valid JFFS2 data. Once JFFS2 has recongnized its FS, JFFS2 would
>    delete the end marker and resume normal behaviour. 
>    
>    What do you think?  

Perhaps we should give the erase command a "jffs2-format" option
so that it erases and writes the proper erase marker on the sectors to prevent
JFFS2 to erase them again?
The erase ulitity should then also skip erasing sectors that has the erased 
marker already which might speed up the erase a little.
I guess copying an image to the formatted flash should work even if it contains
erase marks?

/Johan

Re: mkfs.jffs2 propsal and a question about burst reads

[Jörn Engel]

Hi!

> Perhaps we should give the erase command a "jffs2-format" option
> so that it erases and writes the proper erase marker on the sectors to prevent
> JFFS2 to erase them again?
> The erase ulitity should then also skip erasing sectors that has the erased 
> marker already which might speed up the erase a little.
> I guess copying an image to the formatted flash should work even if it contains
> erase marks?

1) Beware of featuritis. This is just a hack for the explained
problem, depends on the fs used (jffs2) and does not give any real new
functionality. Might be good for the sales department, though. :)

Yesterday, I scrambled a fs because it was recognized as jffs (1),
deleted and empty afterwards. Solution? Don't use jffs1 any more and
be smarter next time.

2) Flash is a write-once medium. You cannot overwrite the markers, you
have to delete again.

Jörn

-- 
A defeated army first battles and then seeks victory.
	Sun Tzu

Re: mkfs.jffs2 propsal and a question about burst reads

[Johan Adolfsson]

> 2) Flash is a write-once medium. You cannot overwrite the markers, you
> have to delete again.

If you have some bits in the flash zeroed and the data you want to write
at that position also contains zero bits you should be just fine.
(assuming an erased flash is all 0xFF's )

/Johan

Re: mkfs.jffs2 propsal and a question about burst reads

[Jörn Engel]

> > 2) Flash is a write-once medium. You cannot overwrite the markers, you
> > have to delete again.
> 
> If you have some bits in the flash zeroed and the data you want to write
> at that position also contains zero bits you should be just fine.
> (assuming an erased flash is all 0xFF's )

Granted. This might be useful for the rw block device driver to save
some delete operations, but is rare enough to just keep things simple
and ignore it. :)

Jörn

-- 
Live a good, honorable life. Then when you
get older and think back, you'll be able to enjoy
it a second time.