[PATCH 0/5] MTD: modify mtd api to return bitflip info on read operations

[PATCH 0/5] MTD: modify mtd api to return bitflip info on read operations

[Mike Dunn]

Hi,

These patches propose a change to the mtd API for the purpose of returning to
the caller information on the number of bit errors corrected by the ecc
facilities of the device during read operations.  The affected functions are
read() and read_oob().

Currently, the -EUCLEAN value returned by read() and read_oob() is the only
information available to the caller regarding bit error corrections.  This
return value indicates simply that one or more bit errors were corrected.  To
make matters worse, this applies to the entire read operation, which can
potentially span the entire device.  Some NAND flash chips are error prone, and
compensate for that by using strong ecc algorithms capable of correcting
multiple errors in a single page.  In order for higher level code (e.g. UBI) to
effectively detect degradation of the integrity of erase blocks on these
devices, more detailed information is needed.

For the read() method, an unsigned int * argument is added, which the driver
uses to return to the caller the maximum number of bitflips that were corrected
on any single page.  For read_oob(), an element is added to the mtd_oob_ops
structure for the same purpose.  Devices without ecc capabilities (NOR flash,
etc) would simply set the value to 0.

The first patch shows the propsed api change, and the remainder implement it
throughout the mtd subsystem.  It has been tested on mtdram, nandsim,
onenandsim, and the diskonchip G4 flash (nand driver currently out-of-tree), on
partitioned and unpartitioned devices.  Drivers for other devices have been
compile-tested only, but the changes are trivial in most cases.  Comments,
criticisms, objections, gratefully received.

Thanks,
Mike


Mike Dunn (5):
  mtd api changed to return bitflips on read operations
  backport mtd api change to mtd infrastructure
  backport mtd api change to nand, onenand infrastructure
  backport mtd api change to ubi
  backport mtd api change to everything else: devices, chips,
    partitioning, translation layers, etc

 drivers/mtd/afs.c                   |    8 +++++-
 drivers/mtd/ar7part.c               |   10 +++++---
 drivers/mtd/chips/cfi_cmdset_0001.c |    9 +++++--
 drivers/mtd/chips/cfi_cmdset_0002.c |    9 +++++--
 drivers/mtd/chips/cfi_cmdset_0020.c |    9 +++++--
 drivers/mtd/chips/map_absent.c      |    9 +++++--
 drivers/mtd/chips/map_ram.c         |    7 ++++-
 drivers/mtd/chips/map_rom.c         |    7 ++++-
 drivers/mtd/devices/block2mtd.c     |    5 +++-
 drivers/mtd/devices/doc2000.c       |    8 ++++--
 drivers/mtd/devices/doc2001.c       |    8 +++++-
 drivers/mtd/devices/doc2001plus.c   |    7 ++++-
 drivers/mtd/devices/docg3.c         |    5 +++-
 drivers/mtd/devices/lart.c          |    4 ++-
 drivers/mtd/devices/m25p80.c        |    4 ++-
 drivers/mtd/devices/ms02-nv.c       |    5 ++-
 drivers/mtd/devices/mtd_dataflash.c |    5 ++-
 drivers/mtd/devices/mtdram.c        |    3 +-
 drivers/mtd/devices/phram.c         |    4 ++-
 drivers/mtd/devices/pmc551.c        |    5 +++-
 drivers/mtd/devices/slram.c         |    7 ++++-
 drivers/mtd/devices/sst25l.c        |    5 +++-
 drivers/mtd/ftl.c                   |   34 ++++++++++++++++++++---------
 drivers/mtd/inftlcore.c             |    9 +++++--
 drivers/mtd/inftlmount.c            |    9 +++++--
 drivers/mtd/lpddr/lpddr_cmds.c      |    4 +-
 drivers/mtd/maps/bcm963xx-flash.c   |    7 ++++-
 drivers/mtd/mtdblock.c              |   11 +++++++--
 drivers/mtd/mtdblock_ro.c           |    4 ++-
 drivers/mtd/mtdchar.c               |   16 +++++++++----
 drivers/mtd/mtdconcat.c             |   12 +++++++---
 drivers/mtd/mtdoops.c               |    4 ++-
 drivers/mtd/mtdpart.c               |    4 +-
 drivers/mtd/mtdswap.c               |    7 ++++-
 drivers/mtd/nand/diskonchip.c       |    6 +++-
 drivers/mtd/nand/nand_base.c        |   19 ++++++++++++++--
 drivers/mtd/nand/nand_bbt.c         |   11 +++++++--
 drivers/mtd/nftlcore.c              |    9 +++++--
 drivers/mtd/nftlmount.c             |    9 +++++--
 drivers/mtd/onenand/onenand_base.c  |    5 +++-
 drivers/mtd/redboot.c               |    4 ++-
 drivers/mtd/rfd_ftl.c               |   18 +++++++++++----
 drivers/mtd/ssfdc.c                 |    7 ++++-
 drivers/mtd/tests/mtd_pagetest.c    |   40 ++++++++++++++++++++++------------
 drivers/mtd/tests/mtd_readtest.c    |    4 ++-
 drivers/mtd/tests/mtd_speedtest.c   |   12 +++++++---
 drivers/mtd/tests/mtd_stresstest.c  |    3 +-
 drivers/mtd/tests/mtd_subpagetest.c |   13 +++++++---
 drivers/mtd/tests/mtd_torturetest.c |    3 +-
 drivers/mtd/ubi/io.c                |    9 +++++--
 include/linux/mtd/mtd.h             |   11 ++++++++-
 include/linux/mtd/pmc551.h          |    3 +-
 52 files changed, 315 insertions(+), 135 deletions(-)

-- 
1.7.3.4

Re: [PATCH 0/5] MTD: modify mtd api to return bitflip info on read operations

[Thomas Petazzoni]

Hello,

Le Wed, 30 Nov 2011 09:00:48 -0800,
Mike Dunn <mikedunn@newsguy.com> a écrit :

> Thanks Thomas.  For some reason this post never made it to my inbox.  Pasted from the list archive...

Huh, strange. Let's see if this one reaches your Inbox.

> > Making this change in patch 1 will break the build if someone bisects
> > kernel changes between patch 1 and your other patches.
> 
> Then I guess it should be a single patch.  As a general rule, should individual patches 
> not be interdependent?

My understanding is that the kernel should be buildable at any given
point, so that things such as "git bisect" can easily be used. So, it's
not so much a "interdependence" question but rather a "buildability"
question.

Basically, you have to ensure that the kernel still builds when patch 1
is enabled, when patches 1+2 are enabled, when patches 1+2+3 are
enabled, etc.

> Well, then it wouldn't be a generic mtd read operation, and higher layers would have to be 
> aware of the type of device and call the appropriate read function.
> Fewer drivers would need to be patched, but patches to infrastructure code would be complicated 
> and make the resulting code ugly, I think.

True that having some drivers implementing the standard ->read()
method and some others implementing the extended ->readext() method
would be ugly.

> I'm not a flash expert, but I think that the greatest number of bitflips on any one page 
> is a sufficient diagnostic of the integrity of an eraseblock.  One bad page will make the 
> entire eraseblock unusable, so I'm not sure it's useful to report bitflips for specific pages.  

Ok, understood, that makes sense to me (but I'm not a flash expert
either).

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [PATCH 0/5] MTD: modify mtd api to return bitflip info on read operations

[Mike Dunn]

Thanks Thomas.  For some reason this post never made it to my inbox.  Pasted from the list archive...

Le Mon, 28 Nov 2011 17:01:17 -0800,
Mike Dunn <mikedunn at newsguy.com <http://lists.infradead.org/mailman/listinfo/linux-mtd>> a écrit :

>/ +	/*
/>/ +	 * max_bitflips returns to caller the greatest number of bit errors
/>/ +	 * corrected on any one minimum i/o unit (e.g., nand page)
/>/ +	 */
/>/ +	int (*read) (struct mtd_info *mtd, loff_t from, size_t len,
/>/ +		     size_t *retlen, u_char *buf, unsigned int *max_bitflips);
/>/  
/>/ -	int (*read) (struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf);
/
> Making this change in patch 1 will break the build if someone bisects
> kernel changes between patch 1 and your other patches.


Then I guess it should be a single patch.  As a general rule, should individual patches 
not be interdependent?


> Also, seeing the large number of users that don't use/need the new
> max_bitflips argument, wouldn't it be better to add a new, separate
> ->readext() operation (or another better name) ? This would probably
> reduce the patch size quite a bit.


Well, then it wouldn't be a generic mtd read operation, and higher layers would have to be 
aware of the type of device and call the appropriate read function.
Fewer drivers would need to be patched, but patches to infrastructure code would be complicated 
and make the resulting code ugly, I think.


> Also, another option is to allow max_bitflips to be NULL, which would
> simplify things such as :

> +	unsigned int max_bitflips;
 
> -	ret = mtd->read(mtd, ptr, sizeof(fs), &sz, (u_char *) &fs);
> +	ret = mtd->read(mtd, ptr, sizeof(fs), &sz, (u_char *) &fs,
> +			&max_bitflips);


> to

> -	ret = mtd->read(mtd, ptr, sizeof(fs), &sz, (u_char *) &fs);
> +	ret = mtd->read(mtd, ptr, sizeof(fs), &sz, (u_char *) &fs,
> +			NULL);

> and would therefore avoid the need for defining an useless variable.


Yes, you're right.  I will do this in the next patch.


> Another question: is the max_bitflips information sufficient (i.e on a
> large read with multiple pages, you will only get the value for the
> worst page) ? Don't you need the bitflip count on a per-page basis ?


I'm not a flash expert, but I think that the greatest number of bitflips on any one page 
is a sufficient diagnostic of the integrity of an eraseblock.  One bad page will make the 
entire eraseblock unusable, so I'm not sure it's useful to report bitflips for specific pages.  
Maybe some other statistics would be useful, such as the total number of bitflips during the 
read operation, but I didn't want to make it unnecesarily complicated, and it's a vast 
improvement over the current state of affairs.

Thanks again Thomas.  I hope some others will offer opinions.  Resolving this is the first step toward 
getting a robust ubifs on my nand flash!

Mike