Can you really write a jffs2 image with nandwrite?

Can you really write a jffs2 image with nandwrite?

[Peter Grayson]

I am using a PowerPC 405 based Linux system running 2.6.10 kernel with
CVS head of mtd patched in. The board has a 256MB Samsung NAND flash
part with 2048 byte page size, 64 spare bytes per page, and 64 pages per
eraseblock (128kB eraseblock).

I want to use jffs2 partitions on this flash part. I can mount an empty
partition with jffs2 and that works well, but I have been unable to
weild mkfs.jffs2 and nandwrite to successfully write jffs2 images to
flash.

>From reading the mkfs.jffs2 code, it seems that mkfs.jffs2 has no
concept of out-of-bounds data. Consequently, it appears that the
nandwrite expects to find out-of-bounds data immediately after the in-
bounds data for every page. I have looked at the bits of the jffs2 image
produced by mkfs.jffs2 and I can see that it does not really produce out
of bounds data.

Below is the mkfs.jffs2 command line I am using. I have tried many other
combinations of options, but they do not yield appreciably different
results.

mkfs.jffs2
 	--pagesize=2048 \
	--eraseblock=128 \
	--pad \
	--output=image.jffs2 \
	--compression-mode none \
	--no-cleanmarkers \
	--big-endian \
	--squash \
	--verbose \
	--root=$ROOT_DIR

When I try using this image with "nandwrite -j", the image appears to be
written successfully, but when I try to mount the new filesystem, mount
fails with the following error message:

  mount: Mounting /dev/mtdblock0 on /mnt failed: Invalid argument

If I try to use "nandwrite -j -o", nandwrite complains about the image
not being aligned:

  Input file is not page aligned: Success
  Data did not fit into device, due to bad blocks
  : Success

In this case, nandwrite bails out early and does not write anything to
flash.

Is there something obvious I am missing? I have read the FAQ at:

  http://www.linux-mtd.infradead.org/tech/faq.html

so I know that the working theory is that this should work, but it is
not working for me. Has anyone else had a similar experience?

Thanks in advance for any help you can offer.

- Pete

Re: Can you really write a jffs2 image with nandwrite?

[Thomas Gleixner]

On Mon, 2005-02-28 at 19:14 -0700, Peter Grayson wrote:
> From reading the mkfs.jffs2 code, it seems that mkfs.jffs2 has no
> concept of out-of-bounds data. Consequently, it appears that the
> nandwrite expects to find out-of-bounds data immediately after the in-
> bounds data for every page. I have looked at the bits of the jffs2 image
> produced by mkfs.jffs2 and I can see that it does not really produce out
> of bounds data.

nandwrite does not expect oob data for jffs2 images.

> Below is the mkfs.jffs2 command line I am using. I have tried many other
> combinations of options, but they do not yield appreciably different
> results.
> 
> mkfs.jffs2
>  	--pagesize=2048 \
> 	--eraseblock=128 \
			
This one if definitely wrong. You meant 128kiB or 131072. 
	
> 	--pad \
> 	--output=image.jffs2 \
> 	--compression-mode none \

Why do you switch off compression ?

> 	--no-cleanmarkers \
> 	--big-endian \
> 	--squash \
> 	--verbose \
> 	--root=$ROOT_DIR
> 
> When I try using this image with "nandwrite -j", the image appears to be
> written successfully, but when I try to mount the new filesystem, mount
> fails with the following error message:
> 
>   mount: Mounting /dev/mtdblock0 on /mnt failed: Invalid argument

Did you ?
mount -t jffs2 /dev/mtdblock0 /mnt

> If I try to use "nandwrite -j -o", nandwrite complains about the image
> not being aligned:

JFFS2 has no oob data. nandwrite -j is correct.

> Is there something obvious I am missing? I have read the FAQ at:
> 
>   http://www.linux-mtd.infradead.org/tech/faq.html

I'm happy that somebody actually read it :)

> so I know that the working theory is that this should work, but it is
> not working for me. Has anyone else had a similar experience?

Before using nandwrite please erase the flash with 
flash_eraseall -j /dev/mtd0 .

tglx

Re: Can you really write a jffs2 image with nandwrite?

[Peter Grayson]

On Tue, 2005-03-01 at 03:47 +0100, Thomas Gleixner wrote:
> On Mon, 2005-02-28 at 19:14 -0700, Peter Grayson wrote:
> > From reading the mkfs.jffs2 code, it seems that mkfs.jffs2 has no
> > concept of out-of-bounds data. Consequently, it appears that the
> > nandwrite expects to find out-of-bounds data immediately after the in-
> > bounds data for every page. I have looked at the bits of the jffs2 image
> > produced by mkfs.jffs2 and I can see that it does not really produce out
> > of bounds data.
> 
> nandwrite does not expect oob data for jffs2 images.

But it appears nandwrite can accept oob data. It does have the --oob
option and when this option is set nandwrite reads the oob data from the
image and does an ioctl() to the mtd character device to write it.

> 
> > Below is the mkfs.jffs2 command line I am using. I have tried many other
> > combinations of options, but they do not yield appreciably different
> > results.
> > 
> > mkfs.jffs2
> >  	--pagesize=2048 \
> > 	--eraseblock=128 \
> 			
> This one if definitely wrong. You meant 128kiB or 131072. 

mkfs.jffs2 interprets "128" as "128KiB", but I should be more explicit.

> 	
> > 	--pad \
> > 	--output=image.jffs2 \
> > 	--compression-mode none \
> 
> Why do you switch off compression ?

I have made images both ways -- in this case I was looking at hex dumps
of images and sans compression the dumps were a little more human-
readable. In the end, I do want to use compression.

> 
> > 	--no-cleanmarkers \
> > 	--big-endian \
> > 	--squash \
> > 	--verbose \
> > 	--root=$ROOT_DIR
> > 
> > When I try using this image with "nandwrite -j", the image appears to be
> > written successfully, but when I try to mount the new filesystem, mount
> > fails with the following error message:
> > 
> >   mount: Mounting /dev/mtdblock0 on /mnt failed: Invalid argument
> 
> Did you ?
> mount -t jffs2 /dev/mtdblock0 /mnt

Yes.

> 
> > If I try to use "nandwrite -j -o", nandwrite complains about the image
> > not being aligned:
> 
> JFFS2 has no oob data. nandwrite -j is correct.
> 
> > Is there something obvious I am missing? I have read the FAQ at:
> > 
> >   http://www.linux-mtd.infradead.org/tech/faq.html
> 
> I'm happy that somebody actually read it :)
> 
> > so I know that the working theory is that this should work, but it is
> > not working for me. Has anyone else had a similar experience?
> 
> Before using nandwrite please erase the flash with 
> flash_eraseall -j /dev/mtd0 .
> 

This was definitely useful. I am now able to write images with
nandwrite. Here are the magic steps I am taking:


1) mkfs.jffs2 \
	--eraseblock=128KiB \
	--pad \
	--output=image.jffs2 \
	--compression-mode=priority \
	--no-cleanmarkers \
	--big-endian \
	--squash \
	--verbose \
	--root=$ROOT_DIR
2) flash_eraseall -j /dev/mtd0
3) nandwrite /dev/mtd0 image.jffs2

With mkfs.jffs2, I had some confusion about the pagesize option. I was
thinking that it corresponded to the NAND pagesize, but that is not
correct, it sets the jffs2 filesystem page size.

With nandwrite, the -j option causes things to not work. This was a
great stumbling block for me. It proved to be critical to _not_ use the
-j (or -o) option with nandwrite. This is in direct conflict with
instructions in the FAQ.

Thanks for the help, this definitely got me jumpstarted.

Re: Can you really write a jffs2 image with nandwrite?

[Thomas Gleixner]

On Tue, 2005-03-01 at 11:11 -0700, Peter Grayson wrote:
> > nandwrite does not expect oob data for jffs2 images.
> 
> But it appears nandwrite can accept oob data. It does have the --oob
> option and when this option is set nandwrite reads the oob data from the
> image and does an ioctl() to the mtd character device to write it.

Yes it can, but JFFS2 images contain no oob data

> With mkfs.jffs2, I had some confusion about the pagesize option. I was
> thinking that it corresponded to the NAND pagesize, but that is not
> correct, it sets the jffs2 filesystem page size.

Oops, missed that one.

> With nandwrite, the -j option causes things to not work. This was a
> great stumbling block for me. It proved to be critical to _not_ use the
> -j (or -o) option with nandwrite. This is in direct conflict with
> instructions in the FAQ.

Could you please correct the FAQ and add the steps you took and send a
patch ?

tglx

Re: Can you really write a jffs2 image with nandwrite?

[Peter Grayson]

[-- Attachment #1: Type: text/plain, Size: 93 bytes --]

> Could you please correct the FAQ and add the steps you took and send a
> patch ?

Attached

[-- Attachment #2: faq.patch --]
[-- Type: text/x-patch, Size: 8234 bytes --]

--- faq.html.orig	2005-03-01 13:41:43.962402197 -0700
+++ faq.html	2005-03-01 13:39:40.574764329 -0700
@@ -5,7 +5,7 @@
 <STYLE TYPE="text/css">
 <!--
 body {
-font-family: Arial,Helvetica,Geneva,Swiss,SunSans-Regular; 
+font-family: Arial,Helvetica,Geneva,Swiss,SunSans-Regular;
 }
 -->
 </STYLE>
@@ -38,7 +38,7 @@
 after to release the chip again.</p>
 
 <h2>NAND specific</h2>
-<h4>Can I boot from NAND Flash?</h4>
+<h4>Can I boot from NAND Flash ?</h4>
 <p>Not from a bare NAND chip. You need a glue logic around, which gives you memory access to the
 chip on bootup, like the DiskOnChip devices do. This will be a quite complex CPLD. An alternative
 is to use a small e.g. 1MB NOR Flash, which contains the boot code and maybe a compressed kernel
@@ -46,7 +46,7 @@
 <p>Some newer chips make the first page available for reading after power up. This could be
 helpful for starting a small 256/512/2048 byte bootcode. At the time of this writing there is no
 tested implementation of this.</p>
-<p>Samsungs S3C2410 ARM based SOC-CPU provides a mechanism to boot from NAND flash.</p>
+<p>Samsung's S3C2410 ARM based SOC-CPU provides a mechanism to boot from NAND flash.</p>
 
 <h4>Is there support for 32bit wide NAND Flash ?</h4>
 <p>No. The generic NAND driver supports 8/16 bit wide NAND Flash only. 32 bit NAND Flash can
@@ -59,19 +59,33 @@
 driver. The JFFS2 code, which handles the writebuffer and the out of band (spare) area of NAND doesn't support
 32 bit neither.</p>
 
-<h4>Can I use mtdutils erase / eraseall</h4>
-<p>Yes, the latest nand driver code forces the protection of bad block information. It's safe to erase a 
-NAND&nbsp;flash with erase(all) /dev/mtdX.
-It's recommended to use eraseall with -j or --jffs2 option set. This will write the cleanmarker to the 
-out of band area. So you can mount the filesystem and no further formatting is needed. This formatting
-is also neccecary before you copy a fs-image to the chip.
+<h4>Can I use mtdutils flash_erase and flash_eraseall ?</h4>
+<p>Yes, the latest nand driver code protects bad block information. It's safe to erase a
+NAND flash with flash_erase or flash_eraseall.</p>
+<p>
+It's recommended to use eraseall with -j or --jffs2 option set. This will write the cleanmarker to the
+out of band area so you can mount the filesystem without needing any further formatting. This formatting
+is also necessary before you copy a filesystem image to the chip.
 </p>
 
 <h4>Can I copy a JFFS2 Image to NAND via /dev/mtdX ?</h4>
-<p>Yes, as long as your chip does not contain bad blocks. Make sure, that the erasesize you set to mkfs.jffs2 
-is the same as the erasesize of your chip. Data will only be written with ECC, if your default settings for
-oob-layout selection are set for JFFS2. For bad block aware copying, use nandwrite from mtd-utils.<br>
-nandwrite -j /dev/mtdX image
+<p>Yes, as long as your chip does not contain bad blocks. Make sure, that the erasesize you set to mkfs.jffs2
+is the same as the erasesize of your chip. Data will only be written with ECC if your default settings for
+oob-layout selection are set for JFFS2. For bad block aware copying use nandwrite from mtd-utils.</p>
+<p>The following is an example procedure for creating a jffs2 image, erasing flash, and writing flash:</p>
+<code>
+$ mkfs.jffs2 \<br>
+&nbsp;&nbsp;&nbsp;&nbsp;--eraseblock=128KiB \<br>
+&nbsp;&nbsp;&nbsp;&nbsp;--pad \<br>
+&nbsp;&nbsp;&nbsp;&nbsp;--output=image.jffs2 \<br>
+&nbsp;&nbsp;&nbsp;&nbsp;--compression-mode=priority \<br>
+&nbsp;&nbsp;&nbsp;&nbsp;--no-cleanmarkers \<br>
+&nbsp;&nbsp;&nbsp;&nbsp;--root=$ROOT_DIR<br>
+<br>
+$ flash_eraseall -j /dev/mtdX<br>
+<br>
+$ nandwrite /dev/mtdX image.jffs2<br>
+</code>
 </p>
 
 <h4>Can I use seek/read/write on /dev/mtdX ?</h4>
@@ -81,37 +95,37 @@
 </p>
 <p>Recent changes to the NAND layer now make it possible to read/write across a block boundary.</P>
 
-<h4>Must my bootloader be aware of NAND FLASH&nbsp;?</h4>
-<p>Yes, if you use your bootloader to erase the FLASH chip and copy a filesystem image to it. 
+<h4>Must my bootloader be aware of NAND FLASH ?</h4>
+<p>Yes, if you use your bootloader to erase the FLASH chip and copy a filesystem image to it.
 For erase make sure, that you don't erase factory-marked bad blocks. They are marked in the 6th
- byte (offset 0x5) in the out of band area of the first page of a block. The block is bad, if any bit in 
- this byte is zero. If you erase such a block, the bad block information is erased too and lost. Further 
+ byte (offset 0x5) in the out of band area of the first page of a block. The block is bad, if any bit in
+ this byte is zero. If you erase such a block, the bad block information is erased too and lost. Further
  use of this block will lead to erroneous results</p>
-<p>For JFFS2 filesystems it's recommended to programm the JFFS2 erased marker into the out of band area of the 
-first page in each erased block after erasing the block. Do not program it into the data area of the page !</p>
+<p>For JFFS2 filesystems it's recommended to programm the JFFS2 erased marker into the out of band area of the
+first page in each erased block after erasing the block. Do not program it into the data area of the page!</p>
 
 <p>For 256 byte pagesize devices program the following data into the out of band area:</p>
-		
+
 <table border="1" cellpadding="5" cellspacing="0">
 <tr><td>Offset</td><td>0x06</td><td>0x07</td></tr>
 <tr><td>Data</td><td>0x85</td><td>0x19</td></tr>
 </table>
 
 <p>For 512 byte pagesize devices program the following data into the out of band area:</p>
-		
+
 <table border="1" cellpadding="5" cellspacing="0">
 <tr><td>Offset</td><td>0x08</td><td>0x09</td><td>0x0a</td><td>0x0b</td><td>0x0c</td><td>0x0d</td>
 <td>0x0e</td><td>0x0f</td></tr>
 <tr><td>Data</td><td>0x85</td><td>0x19</td><td>0x03</td><td>0x20</td><td>0x08</td><td>0x00</td>
 <td>0x00</td><td>0x00</td></tr>
 </table>
-		
-<p>If you copy a filesystem image to the chip, it's recommended to write it with ECC. You can use 
-the ECC code in the nand driver to do this.  If you have a bad block on your chip, just skip this 
+
+<p>If you copy a filesystem image to the chip, it's recommended to write it with ECC. You can use
+the ECC code in the nand driver to do this.  If you have a bad block on your chip, just skip this
 block and copy the data to the next block. JFFS2 and other NAND flash aware filesystems handle this gap.</p>
 
-<h4>Can I program NAND via JTAG&nbsp;?</h4>
-<p>Yes, you can. The requirements are the same as for bootloaders</p>
+<h4>Can I program NAND via JTAG ?</h4>
+<p>Yes, you can. The requirements are the same as for bootloaders.</p>
 
 <h2>JFFS2 specific</h2>
 
@@ -141,12 +155,12 @@
 
 <h4>How is ensured, that data is written to flash ?</h4>
 <p>On <b>NOR FLASH</b> each write goes directly into the FLASH</p>
-<p>On <b>NAND FLASH and NOR ECC FLASH</b> we have a writebuffer for writing only full pages to the chips. 
-There could be a loss of data, when the writebuffer is not flushed before power down. There are some 
-mechanisms to ensure, that the writebuffer is flushed. You can force the flush of the writebuffer by 
-using fsync() or sync() in your application. 
-JFFS2 has a timed flush of the write buffer, which forces the flush of the buffer to flash, if there are no 
-writes to the filesystem for more than about 5 seconds. The time depends on the cycletime of kupdated, which 
+<p>On <b>NAND FLASH and NOR ECC FLASH</b> we have a writebuffer for writing only full pages to the chips.
+There could be a loss of data, when the writebuffer is not flushed before power down. There are some
+mechanisms to ensure, that the writebuffer is flushed. You can force the flush of the writebuffer by
+using fsync() or sync() in your application.
+JFFS2 has a timed flush of the write buffer, which forces the flush of the buffer to flash, if there are no
+writes to the filesystem for more than about 5 seconds. The time depends on the cycletime of kupdated, which
 can be adjusted via <b>/proc/sys/vm/dirty_expire_centisecs</b>. <br>
 When you unmount the filesystem the buffer is flushed too.</p>
 <p>If the partition is being used as a root filesystem, which obviously cannot be dismounted, the filesystem

Re: Can you really write a jffs2 image with nandwrite?

[Artem B. Bityuckiy]

Peter Grayson wrote:
 > This was definitely useful. I am now able to write images with
> nandwrite. Here are the magic steps I am taking:
> 
> 
> 1) mkfs.jffs2 \
> 	--eraseblock=128KiB \
> 	--pad \
> 	--output=image.jffs2 \
> 	--compression-mode=priority \
> 	--no-cleanmarkers \
> 	--big-endian \
> 	--squash \
> 	--verbose \
> 	--root=$ROOT_DIR

Peter, if you don't want to have problems in future, please use --pad=2048 
option instead of --pad. The latter is only acceptable in case of NOR 
flash. This is fairly confusing and we definitely have to either notice 
this in the mks.jffs2 help output or just hack mkfs.jffs2 code and assume 
--pad=Page Size by default for NANDs. You're welcome to do either of this 
and to send your patch :-).

Please, refer to 
http://lists.infradead.org/pipermail/linux-mtd/2004-December/011051.html 
for more details. HTH.

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.