[PATCH 1/3] nand: update out-of-date docs/FAQ, mention UBIFS

[PATCH 1/3] nand: update out-of-date docs/FAQ, mention UBIFS

[Matthew L. Creech]

Signed-off-by: Matthew L. Creech <mlcreech@gmail.com>
---
 doc/nand.xml |   34 +++++++++++++++++++---------------
 faq/nand.xml |   11 +++++++----
 2 files changed, 26 insertions(+), 19 deletions(-)

diff --git a/doc/nand.xml b/doc/nand.xml
index 7254af5..1d55685 100644
--- a/doc/nand.xml
+++ b/doc/nand.xml
@@ -124,30 +124,33 @@ want to a page. The consecutive writes to a page, before erasing it again, are
 restricted to 1-3 writes, depending on the manufacturers specifications. This applies 
 similar to the spare area. This makes it necessary for the filesystem to handle a writebuffer,
 which contains data, that is less than a page</p>
-<p>At the moment there are only a few filesystems, which support NAND</p>
+<p>At the moment there are only a few filesystems which support NAND:</p>
 <ul>
 <li>JFFS2 and YAFFS for bare NAND Flash and SmartMediaCards </li>
 <li>NTFL for DiskOnChip devices </li>
 <li>TRUEFFS from M-Systems for DiskOnChip devices</li>
 <li>SmartMedia DOS-FAT as defined by the SSFDC Forum</li>
+<li>UBIFS for bare NAND flash</li>
 </ul>
-<p>JFFS2 and NTFL are Open Source, while TRUEFFS is a proprietary solution. 
+<p>JFFS2, NTFL, and UBIFS are Open Source, while TRUEFFS is a proprietary solution. 
 SmartMedia DOS-Fat is a specification from SSFDC forum. It is somewhat open under a 
 non disclosure agreement with Toshiba, who owns all rights on this specifications. NTFL is 
 designed for the usage of DiskOnChip devices. JFFS2 supports raw NAND chips and 
 SmartMediaCards at the moment. A JFFS2 support for DiskOnChip devices, based on the 
 NAND code, is planned. There are some other Open Source projects for NAND filesystem 
-support, but there's no other working solution than JFFS2 and YAFFS at the moment of this writing. 
+support, but there's no other working solution than JFFS, YAFFS, and UBIFS at the moment of this writing.
 YAFFS is available from <a href="http://www.aleph1.co.uk/armlinux/projects/yaffs">YAFFS-Homepage</a>.
 YAFFS is faster than JFFS2 and consumes less RAM, JFFS2 provides on the fly file compression and
-decompression, which is very helpful for small FLASHs.</p>
+decompression, which is very helpful for small FLASHs.  UBIFS provides compression similar to JFFS2, but
+is usually a better choice for large NAND flash chips (see <a href="ubifs.html">here</a> for more 
+information on UBIFS, including some notes about how it compares to JFFS2).</p>
 <p> There is currently no support for the wide spread SmartMedia DOS-FAT filesystem, 
  mainly because it's not a reliable filesystem for industrial usage. It's ok for 
  multimedia applications. The hardware support layer is designed to support an 
  implementation of SmartMedia DOS-FAT. There are some efforts to implement it, but it's
  in an early stage. There are a couple of SmartMedia Card adaptors for USB, PCMCIA, FireWire 
  ... with Linux drivers available, which support the SmartMedia DOS-FAT. </p>
-<p>JFFS2 and YAFFS include bad block management, wear leveling, error correction and provide
+<p>JFFS2, YAFFS, and UBIFS include bad block management, wear leveling, error correction and provide
 reliable filesystems for industrial use on top of NAND Flash.</p>
 </div>
 
@@ -255,22 +258,22 @@ pages this byte is reserved</td></tr>
 
 <h3>Where can you get the code ?</h3>
 <div>
-<p>The latest changes to JFFS2 and the underlying NAND code are not in the 
-kernel code at the moment. The latest code is available from 
-<a href="../source.html">CVS and daily snapshots</a></p>
+<p>NAND support is now present in the upstream kernel code, including JFFS2
+and UBIFS. The latest code is also available from
+<a href="../source.html">GIT and daily snapshots</a>.</p>
 
-<p>There are four layers of software</p>
+<p>If using JFFS2, there are four layers of software:</p>
 <ol>
 <li>JFFS2: filesystem driver</li>
 <li>MTD: Memory Technology Devices driver</li>
 <li>NAND: generic NAND driver </li>
 <li>Hardware specific driver </li>
 </ol>
-<p>the MTD driver just provides a mount point for JFFS2. The generic NAND 
+<p>The MTD driver just provides a mount point for JFFS2. The generic NAND 
 driver provides all functions, which are necessary to identify, read, write 
 and erase NAND Flash. The hardware dependent functions are provided by 
 the hardware driver. They provide mainly the hardware access information and
-functions for the generic NAND driver. For YAFFS applies the same.</p>
+functions for the generic NAND driver.  The same applies for YAFFS.</p>
 </div>
 
 <h3>API Documentation</h3>
@@ -324,13 +327,13 @@ CONFIG_JFFS2_FS_NAND=y</p>
 <hr size="2" />
 		
 <h2>FAQ</h2>
-<p>Please see the NAND section in <a href="../faq/nand.html">MTD FAQ's</a></p>
+<p>Please see the NAND section in <a href="../faq/nand.html">MTD FAQs</a></p>
 
 <hr size="2" />
 		
 <h2>References:</h2>
 <h3>Open Source</h3>
-<p>JFFS2 and NTFL are located on this <a href="../index.html">website</a>.<br />
+<p>JFFS2, NTFL, and UBIFS are located on this <a href="../index.html">website</a>.<br />
 YAFFS is located at <a href="http://www.aleph1.co.uk/armlinux/projects/yaffs">YAFFS-Homepage</a>.</p>
 <h3>Hardware</h3>
 <p><a href="http://www.toshiba.com/taec">Toshiba</a></p>
@@ -343,8 +346,9 @@ YAFFS is located at <a href="http://www.aleph1.co.uk/armlinux/projects/yaffs">YA
 <h2>Maintainers</h2>
 <p>JFFS2 is maintained by David Woodhouse</p>
 <p>The generic NAND driver is maintained by Thomas Gleixner</p>
-<p>Please don't contact them direct. Ask your questions on the 
-<a href="../mail.html">mtd-mailing-list</a>.</p>
+<p>UBIFS is maintained by Artem Bityutskiy</p>
+<p>Please don't contact them directly. Ask your questions on the
+<a href="../mail.html">linux-mtd mailinglist</a>.</p>
 <p>Any suggestions, improvements, bug-reports and bug-fixes are welcome</p>
 		
 	<INCLUDE file="../inc/footer.tmpl" />
diff --git a/faq/nand.xml b/faq/nand.xml
index 75acc72..9d9cd06 100644
--- a/faq/nand.xml
+++ b/faq/nand.xml
@@ -32,13 +32,16 @@ Not from a bare NAND chip. You need a glue logic around, which gives you memory
 access to the chip on boot-up, like the DiskOnChip devices do. This will be a
 quite complex CPLD. An alternative is to use a small e.g. 1MB NOR flash, which
 contains the boot code and maybe a compressed kernel image. Then you can use
-JFFS2 on NAND as your root file-system.
+JFFS2 or UBIFS on NAND as your root file-system.
 </P>
 
 <P>
-Some newer chips make the first page available for reading after power up. This
-could be helpful for starting a small 256/512/2048 byte boot-code. At the time
-of this writing there is no tested implementation of this.
+Some chips have internal circuitry which is capable of reading and booting from
+the first page of NAND after power up. This allows you to boot a small
+(e.g. 256/512/2048 byte) boot code, which can initialize SDRAM, then read and
+execute a more complete bootloader from subsequent NAND blocks.  An example is
+U-Boot, which can be built as a 2-stage "initial" and "secondary" bootloader
+for use with NAND.
 </P>
 
 <P>
-- 
1.7.4.1

[PATCH 2/3] UBI FAQ: correct link

[Matthew L. Creech]

Signed-off-by: Matthew L. Creech <mlcreech@gmail.com>
---
 faq/ubi.xml |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/faq/ubi.xml b/faq/ubi.xml
index 6ddb23c..294403a 100644
--- a/faq/ubi.xml
+++ b/faq/ubi.xml
@@ -439,7 +439,7 @@ reboots.</p>
 
 <p>Yes, it may, as long as the flash is supported by the MTD layer. UBI does
 not use OOB and it requires data to be written sequentially (see
-<a href="ubi.html#L_mlc">here</a>). UBI guarantees that the difference
+<a href="ubi.html#L_restrict">here</a>). UBI guarantees that the difference
 between maximum and minimum erase-counters is withing certain threshold,
 which is 4096 by default. Since MLC flashes have quite low eraseblock
 life-cycle (about 1000-10000, unlike 100000-1000000 for SLC NAND and NOR
-- 
1.7.4.1

[PATCH 3/3] UBIFS FAQ: clarify NAND "disturb" errors

[Matthew L. Creech]

Signed-off-by: Matthew L. Creech <mlcreech@gmail.com>
---
 faq/ubifs.xml |   78 ++++++++++++++++++++++++++++++++++++--------------------
 1 files changed, 50 insertions(+), 28 deletions(-)

diff --git a/faq/ubifs.xml b/faq/ubifs.xml
index 3348da4..bd91733 100644
--- a/faq/ubifs.xml
+++ b/faq/ubifs.xml
@@ -67,8 +67,10 @@ some specific aspects of MLC NAND flashes:</p>
 
 <ul>
 	<li>MLC NAND flashes are more "faulty" than SLC, so they use stronger
-	ECC codes which occupy whole OOB area; this is not a problem
-	for UBI/UBIFS, because neither UBIFS nor UBI use OOB area;</li>
+	ECC codes; these ECC codes often occupy whole OOB area (as do the
+	ECC codes on some newer SLC flashes, which are more error-prone than
+	previous generations of flash); this is not a problem for UBI/UBIFS,
+	because neither UBIFS nor UBI use OOB area;</li>
 
 	<li>when the data are written to an eraseblock, they have to be written
 	sequentially, from the beginning of the eraseblock to the end of it;
@@ -80,22 +82,54 @@ some specific aspects of MLC NAND flashes:</p>
 	uses deterministic wear-leveling algorithm
 	(see <a href="ubi.html#L_mlc">this</a> section);</li>
 
-	<li>MLC flashes have so called "read-disturb" property, which means
-	that NAND page read operation may introduce a permanent bit change; the
-	ECC code would fix it, but more read operations may introduce more bit
-	changes and soft ECC errors may turn into hard ECC errors; well, even
-	SLC NAND flashes have this property, but the probability of bit changes
-	is much lower in SLC NAND; however, this should not be a problem
-	because UBI is doing scrubbing; in other words, once UBI notices that
-	there is a correctable bit-flip in an eraseblock, it moves the contents
-	of this physical eraseblock to a different physical eraseblocks, and
-	re-maps corresponding logical eraseblocks to the new physical
-	eraseblock; so UBI refreshes the data and gets rid of bit-flips, thus
-	improving data integrity.</li>
+	<li>MLC flashes exhibit bit flips as a result of "program disturb" and
+	"read disturb" errors (see
+	<a href="http://www.klabs.org/richcontent/MemoryContent/nvmt_symp/nvmts_2002/docs/12/12_dan_p.pdf">here</a>).
+	Note that SLC flashes have these same errors, but they are much more
+	common on MLC:
+	<ul>
+		<li>NAND flashes have a so called "read-disturb" property, which
+		means that a NAND page read operation may introduce a permanent
+		bit change; the ECC code would fix it, but more read operations
+		may introduce more bit changes and soft ECC errors may turn
+		into hard ECC errors; however, when these errors occur on the
+		same page that is being read, this should not be a problem
+		because UBI is doing scrubbing; in other words, once UBI notices
+		that there is a correctable bit-flip in an eraseblock, it moves
+		the contents of this physical eraseblock to a different physical
+		eraseblock, and re-maps the corresponding logical eraseblock to
+		the new physical eraseblock; so UBI refreshes the data and gets
+		rid of bit-flips, thus improving data integrity.</li>
+
+		<li>"Read-disturb" errors can also occur on a page <i>other</i>
+		that the one being read, but which is within the same
+		eraseblock. This is not a problem if the read operations are
+		spread around somewhat evenly within the eraseblock, since the
+		bit-flip will soon be detected and corrected through the
+		"scrubbing" process described above.  However if a particular
+		page within a block is rarely read, scrubbing will not have a
+		chance to fix errors, and they may accumulate over time until
+		they are unfixable. This is very similar the next problem:</li>
+
+		<li>NAND flashes also have a "program-disturb" property,
+		which means that if you program a NAND page, you may introduce
+		a bit-flip in a different NAND page.  The bit change can be
+		fixed by ECC, but with time the changes may accumulate
+		and become unfixable.  Current UBI bit-flip handling only
+		partially helps here, because it is passive, which means that
+		UBI notices bit-flips only when performing users' read requests.
+		So if you never read the NAND page which accumulates bit-flips,
+		UBI will never notice this. One solution to these problems is
+		to implement a kind of "flash crawler" which would read all of
+		the NAND pages in the background from time to time, making UBI
+		notice and fix bit-flips.  However, this is not implemented
+		today.
+		</li>
+	</ul></li>
 </ul>
 
-<p>However, there are 2 other aspects which may need closer attention. The
-first one is the "paired pages" problem (e.g., see
+<p>There is another aspect of MLC flashes which may need closer attention: the
+"paired pages" problem (e.g., see
 <a href="http://www.celinux.org/elc08_presentations/ELC2008 Filesystem support on Multi Level Cell flash in open source.ppt">this</a>
 Power Point presentation). Namely, MLC NAND pages are coupled in a sense
 that if you cut power while writing to a page, you corrupt not only this page,
@@ -106,18 +140,6 @@ distances). So if you write data to, say, page 3 and cut the power, you may
 end up with corrupted data in page 0. UBIFS is not ready to handle this
 problem at the moment and this needs some work.</p>
 
-<p>The second aspect is the "program-disturb" MLC NAND property (see
-<a href="http://www.klabs.org/richcontent/MemoryContent/nvmt_symp/nvmts_2002/docs/12/12_dan_p.pdf">here</a>),
-which means that if you program an MLC NAND page, you may introduce a bit-change
-in a different NAND page. Well, the bit change will be fixed by ECC, but with time
-the changes may accumulate and become unfixable. Current UBI bit-flip handling
-only partially helps here, because it is passive, which means that UBI notices
-bit-flips only when performing users read requests, so if you never read the
-MLC NAND area which accumulates bit-flips, UBI will never notice this. However,
-it is not difficult to implement a kind of "flash crawler" which would read the
-flash in background from time to time and make UBI notice and fix
-bit-flips.</p>
-
 <p>Nevertheless, UBIFS authors never worked with real raw MLC NAND flash, so we
 might have missed or misinterpreted some MLC NAND aspects. Any feed-back is
 appreciated.</p>
-- 
1.7.4.1

Re: [PATCH 1/3] nand: update out-of-date docs/FAQ, mention UBIFS

[Artem Bityutskiy]

On Wed, 2011-07-27 at 00:32 -0400, Matthew L. Creech wrote:
> Signed-off-by: Matthew L. Creech <mlcreech@gmail.com>
> ---
>  doc/nand.xml |   34 +++++++++++++++++++---------------
>  faq/nand.xml |   11 +++++++----
>  2 files changed, 26 insertions(+), 19 deletions(-)

Sorry for long delay, a have had vacation. Pushed to mtd-www, thanks!

-- 
Best Regards,
Artem Bityutskiy