[U-Boot] [PATCH 1/6] mtd: add writebufsize field to mtd_info struct

[U-Boot] [PATCH 1/6] mtd: add writebufsize field to mtd_info struct

[Holger Brunck]

This field will be used to indicate the write buffer size
of the MTD device. UBI will set it's minimal I/O unit size
(min_io_size) to the indicated write buffer size. By this
change we intend to fix failed recovery of UBIFS partitions
we currently observe on NOR flash when mounting the partition
after unclean unmount.

Currently the min_io_size is set to mtd->writesize (which is 1
byte for NOR flash). But flash programming is often done from
prepared write buffer containing multiple bytes and is performed
in one programming operation which could be interrupted by a power
cut or a system reset causing corrupted (partially written) areas
in a flash sector. Knowing the size of potentially corrupted areas
UBIFS scanning and recovery algorithms are able to perform
successful recovery.

Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
---
 include/linux/mtd/mtd.h |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/include/linux/mtd/mtd.h b/include/linux/mtd/mtd.h
index 3b18d7d..14d6f70 100644
--- a/include/linux/mtd/mtd.h
+++ b/include/linux/mtd/mtd.h
@@ -129,6 +129,17 @@ struct mtd_info {
 	 */
 	u_int32_t writesize;
 
+	/*
+	 * Size of the write buffer used by the MTD. MTD devices having a write
+	 * buffer can write multiple writesize chunks at a time. E.g. while
+	 * writing 4 * writesize bytes to a device with 2 * writesize bytes
+	 * buffer the MTD driver can (but doesn't have to) do 2 writesize
+	 * operations, but not 4. Currently, all NANDs have writebufsize
+	 * equivalent to writesize (NAND page size). Some NOR flashes do have
+	 * writebufsize greater than writesize.
+	 */
+	u_int32_t writebufsize;
+
 	u_int32_t oobsize;   /* Amount of OOB data per block (e.g. 16) */
 	u_int32_t oobavail;  /* Available OOB bytes per block */
 
-- 
1.7.0.5

[U-Boot] [PATCH 2/6] cfi_mtd: add writebufsize initialization

[Holger Brunck]

Initialize mtd->writebufsize to the value obtained by
by the CFI informations.

Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
---
 drivers/mtd/cfi_mtd.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/cfi_mtd.c b/drivers/mtd/cfi_mtd.c
index cbcc165..29e32b2 100644
--- a/drivers/mtd/cfi_mtd.c
+++ b/drivers/mtd/cfi_mtd.c
@@ -241,6 +241,7 @@ int cfi_mtd_init(void)
 		mtd->flags		= MTD_CAP_NORFLASH;
 		mtd->size		= fi->size;
 		mtd->writesize		= 1;
+		mtd->writebufsize	= fi->buffer_size;
 
 		mtd->erase		= cfi_mtd_erase;
 		mtd->read		= cfi_mtd_read;
-- 
1.7.0.5

[U-Boot] [PATCH 3/6] mtd, nand: add mtd->writebufsize initialization

[Holger Brunck]

Initialize mtd->writebufsize to be equal to mtd->writesize.

Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
---
 drivers/mtd/nand/nand_base.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index 3b96b0e..26ae720 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -2843,6 +2843,7 @@ int nand_scan_tail(struct mtd_info *mtd)
 	mtd->unlock = NULL;
 	mtd->block_isbad = nand_block_isbad;
 	mtd->block_markbad = nand_block_markbad;
+	mtd->writebufsize = mtd->writesize;
 
 	/* propagate ecc.layout to mtd_info */
 	mtd->ecclayout = chip->ecc.layout;
-- 
1.7.0.5

[U-Boot] [PATCH 4/6] mtd, onenand: add mtd writebufsize initialization

[Holger Brunck]

Initialize mtd->writebufsize to be equal to mtd->writesize.

Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
---
 drivers/mtd/onenand/onenand_base.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/onenand/onenand_base.c b/drivers/mtd/onenand/onenand_base.c
index 24e02c2..8881672 100644
--- a/drivers/mtd/onenand/onenand_base.c
+++ b/drivers/mtd/onenand/onenand_base.c
@@ -2623,6 +2623,7 @@ static int onenand_probe(struct mtd_info *mtd)
 	mtd->sync = onenand_sync;
 	mtd->block_isbad = onenand_block_isbad;
 	mtd->block_markbad = onenand_block_markbad;
+	mtd->writebufsize = mtd->writesize;
 
 	return 0;
 }
-- 
1.7.0.5

[U-Boot] [PATCH 5/6] mtd: initialize writebufsize in the MTD object of a partition

[Holger Brunck]

Propagate the writebufsize to the partition's MTD object so
that UBI can set correct value for it's minimal I/O size
using the writebufsize field of MTD object of the partition.

Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
---
 drivers/mtd/mtdconcat.c |    1 +
 drivers/mtd/mtdpart.c   |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/drivers/mtd/mtdconcat.c b/drivers/mtd/mtdconcat.c
index fc22701..a33dcfc 100644
--- a/drivers/mtd/mtdconcat.c
+++ b/drivers/mtd/mtdconcat.c
@@ -606,6 +606,7 @@ struct mtd_info *mtd_concat_create(struct mtd_info *subdev[],	/* subdevices to c
 	concat->mtd.size = subdev[0]->size;
 	concat->mtd.erasesize = subdev[0]->erasesize;
 	concat->mtd.writesize = subdev[0]->writesize;
+	concat->mtd.writebufsize = subdev[0]->writebufsize;
 	concat->mtd.subpage_sft = subdev[0]->subpage_sft;
 	concat->mtd.oobsize = subdev[0]->oobsize;
 	concat->mtd.oobavail = subdev[0]->oobavail;
diff --git a/drivers/mtd/mtdpart.c b/drivers/mtd/mtdpart.c
index f647e43..150f2ad 100644
--- a/drivers/mtd/mtdpart.c
+++ b/drivers/mtd/mtdpart.c
@@ -296,6 +296,7 @@ static struct mtd_part *add_one_partition(struct mtd_info *master,
 	slave->mtd.flags = master->flags & ~part->mask_flags;
 	slave->mtd.size = part->size;
 	slave->mtd.writesize = master->writesize;
+	slave->mtd.writebufsize = master->writebufsize;
 	slave->mtd.oobsize = master->oobsize;
 	slave->mtd.oobavail = master->oobavail;
 	slave->mtd.subpage_sft = master->subpage_sft;
-- 
1.7.0.5

[U-Boot] [PATCH 6/6] UBI: use mtd->writebufsize to set minimal I/O unit size

[Holger Brunck]

Previously we used mtd->writesize field to set UBI's minimal
I/O unit size. This sometimes caused UBIFS recovery issues
when mounting an uncleanly unmounted UBIFS partition on NOR
flash since mtd->writesize is 1 byte for NOR flash. The
MTD CFI driver however often performs writing multiple
bytes in one programming operation using the chip's write
buffer. We have to use the size of this write buffer as
a minimal I/O unit size for UBI on NOR flash to fix the
observed UBIFS recovery issues.

Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
---
 drivers/mtd/ubi/build.c |   28 +++++++++++++++++++++++++++-
 1 files changed, 27 insertions(+), 1 deletions(-)

diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index 3ea0e6c..7c2e1dc 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -542,7 +542,33 @@ static int io_init(struct ubi_device *ubi)
 	if (ubi->mtd->block_isbad && ubi->mtd->block_markbad)
 		ubi->bad_allowed = 1;
 
-	ubi->min_io_size = ubi->mtd->writesize;
+	/*
+	 * Set UBI min. I/O size (@ubi->min_io_size). We use @mtd->writebufsize
+	 * for these purposes, not @mtd->writesize. At the moment this does not
+	 * matter for NAND, because currently @mtd->writebufsize is equivalent to
+	 * @mtd->writesize for all NANDs. However, some CFI NOR flashes may
+	 * have @mtd->writebufsize which is multiple of @mtd->writesize.
+	 *
+	 * The reason we use @mtd->writebufsize for @ubi->min_io_size is that
+	 * UBI and UBIFS recovery algorithms rely on the fact that if there was
+	 * an unclean power cut, then we can find offset of the last corrupted
+	 * node, align the offset to @ubi->min_io_size, read the rest of the
+	 * eraseblock starting from this offset, and check whether there are
+	 * only 0xFF bytes. If yes, then we are probably dealing with a
+	 * corruption caused by a power cut, if not, then this is probably some
+	 * severe corruption.
+	 *
+	 * Thus, we have to use the maximum write unit size of the flash, which
+	 * is @mtd->writebufsize, because @mtd->writesize is the minimum write
+	 * size, not the maximum.
+	 */
+	if (ubi->mtd->type == MTD_NANDFLASH)
+		ubi_assert(ubi->mtd->writebufsize == ubi->mtd->writesize);
+	else if (ubi->mtd->type == MTD_NORFLASH)
+		ubi_assert(ubi->mtd->writebufsize % ubi->mtd->writesize == 0);
+
+	ubi->min_io_size = ubi->mtd->writebufsize;
+
 	ubi->hdrs_min_io_size = ubi->mtd->writesize >> ubi->mtd->subpage_sft;
 
 	/*
-- 
1.7.0.5

[U-Boot] [PATCH 1/6] mtd: add writebufsize field to mtd_info struct

[Detlev Zundel]

Hi Stefan,

do you have any comment on this series?  I delegated the patches to you
in patchwork already :)  No seriously, the patches look sane and are in
accordance to what is done in Linux....

> This field will be used to indicate the write buffer size
> of the MTD device. UBI will set it's minimal I/O unit size
> (min_io_size) to the indicated write buffer size. By this
> change we intend to fix failed recovery of UBIFS partitions
> we currently observe on NOR flash when mounting the partition
> after unclean unmount.
>
> Currently the min_io_size is set to mtd->writesize (which is 1
> byte for NOR flash). But flash programming is often done from
> prepared write buffer containing multiple bytes and is performed
> in one programming operation which could be interrupted by a power
> cut or a system reset causing corrupted (partially written) areas
> in a flash sector. Knowing the size of potentially corrupted areas
> UBIFS scanning and recovery algorithms are able to perform
> successful recovery.
>
> Signed-off-by: Holger Brunck <holger.brunck@keymile.com>

Thanks
  Detlev

-- 
vi vi vi - the roman numeral of the beast.
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de

[U-Boot] [PATCH 1/6] mtd: add writebufsize field to mtd_info struct

[Holger Brunck]

Hi Detlev,

Detlev Zundel wrote:
> Hi Stefan,
> 
> do you have any comment on this series?  I delegated the patches to you
> in patchwork already :)  No seriously, the patches look sane and are in
> accordance to what is done in Linux....
> 
>> This field will be used to indicate the write buffer size
>> of the MTD device. UBI will set it's minimal I/O unit size
>> (min_io_size) to the indicated write buffer size. By this
>> change we intend to fix failed recovery of UBIFS partitions
>> we currently observe on NOR flash when mounting the partition
>> after unclean unmount.
>>
>> Currently the min_io_size is set to mtd->writesize (which is 1
>> byte for NOR flash). But flash programming is often done from
>> prepared write buffer containing multiple bytes and is performed
>> in one programming operation which could be interrupted by a power
>> cut or a system reset causing corrupted (partially written) areas
>> in a flash sector. Knowing the size of potentially corrupted areas
>> UBIFS scanning and recovery algorithms are able to perform
>> successful recovery.
>>
>> Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
> 

as I told in another mail, the min I/O size adaption patch leads to
incompatibilites for the UBIFS and therefore the similar patch in linux kernel
was reverted. But anyway the first five patches in the patch serie are already
part of the mtd layer of the linux kernel.

So the patches 1-5 of this series can also be committed to u-boot. Whats your
opinion?

Regards
Holger Brunck

[U-Boot] [PATCH 1/6] mtd: add writebufsize field to mtd_info struct

[Stefan Roese]

Hi Holger & Detlev,

On Tuesday 01 February 2011 09:26:56 Holger Brunck wrote:
> > do you have any comment on this series?  I delegated the patches to you
> > in patchwork already :)  No seriously, the patches look sane and are in
> > accordance to what is done in Linux....

I'm back from vacation now and slowly picking up...
 
> >> This field will be used to indicate the write buffer size
> >> of the MTD device. UBI will set it's minimal I/O unit size
> >> (min_io_size) to the indicated write buffer size. By this
> >> change we intend to fix failed recovery of UBIFS partitions
> >> we currently observe on NOR flash when mounting the partition
> >> after unclean unmount.
> >> 
> >> Currently the min_io_size is set to mtd->writesize (which is 1
> >> byte for NOR flash). But flash programming is often done from
> >> prepared write buffer containing multiple bytes and is performed
> >> in one programming operation which could be interrupted by a power
> >> cut or a system reset causing corrupted (partially written) areas
> >> in a flash sector. Knowing the size of potentially corrupted areas
> >> UBIFS scanning and recovery algorithms are able to perform
> >> successful recovery.
> >> 
> >> Signed-off-by: Holger Brunck <holger.brunck@keymile.com>
> 
> as I told in another mail, the min I/O size adaption patch leads to
> incompatibilites for the UBIFS and therefore the similar patch in linux
> kernel was reverted. But anyway the first five patches in the patch serie
> are already part of the mtd layer of the linux kernel.
> 
> So the patches 1-5 of this series can also be committed to u-boot. Whats
> your opinion?

I have no problems with applying patches 1-5. But they have been submitted 
after the merge window was closed, so they will not make it into the next 
release.

I'm volunteering to push those patches into "next" (once its available) via 
one of my git repositories (UBI/UBIFS or CFI) if nobody else objects.

Cheers,
Stefan

--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-0 Fax: (+49)-8142-66989-80 Email: office at denx.de

[U-Boot] [PATCH 1/6] mtd: add writebufsize field to mtd_info struct

[Holger Brunck]

Hi Stefan,

Stefan Roese wrote:
> 
>> as I told in another mail, the min I/O size adaption patch leads to
>> incompatibilites for the UBIFS and therefore the similar patch in linux
>> kernel was reverted. But anyway the first five patches in the patch serie
>> are already part of the mtd layer of the linux kernel.
>>
>> So the patches 1-5 of this series can also be committed to u-boot. Whats
>> your opinion?
> 
> I have no problems with applying patches 1-5. But they have been
> submitted after the merge window was closed, so they will not make it
> into the next release.
> 
> I'm volunteering to push those patches into "next" (once its available)
> via one of my git repositories (UBI/UBIFS or CFI) if nobody else objects.
> 

Ok, thanks.

Regards
Holger