[PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario J. Rugiero]

Some chips used under a custom vendor driver can get their blocks
incorrectly detected as bad blocks, out of incompatibilities
between such drivers and MTD drivers.
When there are too many misdetected bad blocks, the device becomes
unusable because a bad block table can't be allocated, aside from
all the legitimately good blocks which become unusable under these
conditions.
This adds a build option to workaround the issue by enabling the
user to free up space regardless of what the driver thinks about
the blocks.

Example usage: recovering NAND chips on sunxi devices, as explained
here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues

Signed-off-by: Mario J. Rugiero <mrugiero@gmail.com>
---
 drivers/mtd/nand/Kconfig     | 9 +++++++++
 drivers/mtd/nand/nand_base.c | 2 ++
 2 files changed, 11 insertions(+)

diff --git a/drivers/mtd/nand/Kconfig b/drivers/mtd/nand/Kconfig
index c3029528063b..e5b29fab4a60 100644
--- a/drivers/mtd/nand/Kconfig
+++ b/drivers/mtd/nand/Kconfig
@@ -36,6 +36,15 @@ config MTD_NAND_ECC_BCH
 	  ECC codes. They are used with NAND devices requiring more than 1 bit
 	  of error correction.
 
+config MTD_NAND_ERASE_BADBLOCKS
+	bool "Enable erasing of bad blocks (DANGEROUS)]"
+	default n
+	help
+	  This enables support for attempting to erase bad blocks.
+	  It is needed to workaround too many badblocks issue on chips used
+	  under custom, incompatible vendor drivers.
+	  Say N if unsure.
+
 config MTD_SM_COMMON
 	tristate
 	default n
diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index d474378ed810..0216dfc67976 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -3206,6 +3206,7 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
 	instr->state = MTD_ERASING;
 
 	while (len) {
+		#ifndef CONFIG_MTD_NAND_ERASE_BADBLOCKS
 		/* Check if we have a bad block, we do not erase bad blocks! */
 		if (nand_block_checkbad(mtd, ((loff_t) page) <<
 					chip->page_shift, allowbbt)) {
@@ -3214,6 +3215,7 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
 			instr->state = MTD_ERASE_FAILED;
 			goto erase_exit;
 		}
+		#endif
 
 		/*
 		 * Invalidate the page cache, if we erase the block which
-- 
2.13.0

[PATCH v2] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario J. Rugiero]

Some chips used under a custom vendor driver can get their blocks
incorrectly detected as bad blocks, out of incompatibilities
between such drivers and MTD drivers.
When there are too many misdetected bad blocks, the device becomes
unusable because a bad block table can't be allocated, aside from
all the legitimately good blocks which become unusable under these
conditions.
This adds a build option to workaround the issue by enabling the
user to free up space regardless of what the driver thinks about
the blocks.

Example usage: recovering NAND chips on sunxi devices, as explained
here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues

Signed-off-by: Mario J. Rugiero <mrugiero@gmail.com>

v2: Fix a typo in the build option description.

---
 drivers/mtd/nand/Kconfig     | 9 +++++++++
 drivers/mtd/nand/nand_base.c | 2 ++
 2 files changed, 11 insertions(+)

diff --git a/drivers/mtd/nand/Kconfig b/drivers/mtd/nand/Kconfig
index c3029528063b..e0a32a34b6bf 100644
--- a/drivers/mtd/nand/Kconfig
+++ b/drivers/mtd/nand/Kconfig
@@ -36,6 +36,15 @@ config MTD_NAND_ECC_BCH
 	  ECC codes. They are used with NAND devices requiring more than 1 bit
 	  of error correction.
 
+config MTD_NAND_ERASE_BADBLOCKS
+	bool "Enable erasing of bad blocks (DANGEROUS)"
+	default n
+	help
+	  This enables support for attempting to erase bad blocks.
+	  It is needed to workaround too many badblocks issue on chips used
+	  under custom, incompatible vendor drivers.
+	  Say N if unsure.
+
 config MTD_SM_COMMON
 	tristate
 	default n
diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index d474378ed810..0216dfc67976 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -3206,6 +3206,7 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
 	instr->state = MTD_ERASING;
 
 	while (len) {
+		#ifndef CONFIG_MTD_NAND_ERASE_BADBLOCKS
 		/* Check if we have a bad block, we do not erase bad blocks! */
 		if (nand_block_checkbad(mtd, ((loff_t) page) <<
 					chip->page_shift, allowbbt)) {
@@ -3214,6 +3215,7 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
 			instr->state = MTD_ERASE_FAILED;
 			goto erase_exit;
 		}
+		#endif
 
 		/*
 		 * Invalidate the page cache, if we erase the block which
-- 
2.13.0

[PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario J. Rugiero]

Some chips used under a custom vendor driver can get their blocks
incorrectly detected as bad blocks, out of incompatibilities
between such drivers and MTD drivers.
When there are too many misdetected bad blocks, the device becomes
unusable because a bad block table can't be allocated, aside from
all the legitimately good blocks which become unusable under these
conditions.
This adds a build option to workaround the issue by enabling the
user to free up space regardless of what the driver thinks about
the blocks. It still warns about it, since that's potentially
dangerous.

Example usage: recovering NAND chips on sunxi devices, as explained
here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues

v3: Warn when erasing a bad block, as suggested by Andrea Scian.

v2: Fix a typo in the config description.

Signed-off-by: Mario J. Rugiero <mrugiero@gmail.com>
---
 drivers/mtd/nand/Kconfig     | 9 +++++++++
 drivers/mtd/nand/nand_base.c | 2 ++
 2 files changed, 11 insertions(+)

diff --git a/drivers/mtd/nand/Kconfig b/drivers/mtd/nand/Kconfig
index c3029528063b..e0a32a34b6bf 100644
--- a/drivers/mtd/nand/Kconfig
+++ b/drivers/mtd/nand/Kconfig
@@ -36,6 +36,15 @@ config MTD_NAND_ECC_BCH
 	  ECC codes. They are used with NAND devices requiring more than 1 bit
 	  of error correction.
 
+config MTD_NAND_ERASE_BADBLOCKS
+	bool "Enable erasing of bad blocks (DANGEROUS)"
+	default n
+	help
+	  This enables support for attempting to erase bad blocks.
+	  It is needed to workaround too many badblocks issue on chips used
+	  under custom, incompatible vendor drivers.
+	  Say N if unsure.
+
 config MTD_SM_COMMON
 	tristate
 	default n
diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index d474378ed810..282410813a9c 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -3211,8 +3211,10 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
 					chip->page_shift, allowbbt)) {
 			pr_warn("%s: attempt to erase a bad block at page 0x%08x\n",
 				    __func__, page);
+			#ifndef CONFIG_MTD_NAND_ERASE_BADBLOCKS
 			instr->state = MTD_ERASE_FAILED;
 			goto erase_exit;
+			#endif
 		}
 
 		/*
-- 
2.13.0

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

+MTD maintainers

Hi Mario,

Can you please Cc NAND/MTD maintainers next time.

On Fri, 12 May 2017 04:39:46 -0300
"Mario J. Rugiero" <mrugiero@gmail.com> wrote:

> Some chips used under a custom vendor driver can get their blocks
> incorrectly detected as bad blocks, out of incompatibilities
> between such drivers and MTD drivers.
> When there are too many misdetected bad blocks, the device becomes
> unusable because a bad block table can't be allocated, aside from
> all the legitimately good blocks which become unusable under these
> conditions.
> This adds a build option to workaround the issue by enabling the
> user to free up space regardless of what the driver thinks about
> the blocks. It still warns about it, since that's potentially
> dangerous.
> 
> Example usage: recovering NAND chips on sunxi devices, as explained
> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues
> 
> v3: Warn when erasing a bad block, as suggested by Andrea Scian.
> 
> v2: Fix a typo in the config description.

For your future submissions, please put the changelog after the --- so
that it's dropped when applying the patch.

> 
> Signed-off-by: Mario J. Rugiero <mrugiero@gmail.com>
> ---

Here:

v3: ...

v2: ...

>  drivers/mtd/nand/Kconfig     | 9 +++++++++
>  drivers/mtd/nand/nand_base.c | 2 ++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/drivers/mtd/nand/Kconfig b/drivers/mtd/nand/Kconfig
> index c3029528063b..e0a32a34b6bf 100644
> --- a/drivers/mtd/nand/Kconfig
> +++ b/drivers/mtd/nand/Kconfig
> @@ -36,6 +36,15 @@ config MTD_NAND_ECC_BCH
>  	  ECC codes. They are used with NAND devices requiring more than 1 bit
>  	  of error correction.
>  
> +config MTD_NAND_ERASE_BADBLOCKS
> +	bool "Enable erasing of bad blocks (DANGEROUS)"
> +	default n
> +	help
> +	  This enables support for attempting to erase bad blocks.
> +	  It is needed to workaround too many badblocks issue on chips used
> +	  under custom, incompatible vendor drivers.
> +	  Say N if unsure.
> +
>  config MTD_SM_COMMON
>  	tristate
>  	default n
> diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
> index d474378ed810..282410813a9c 100644
> --- a/drivers/mtd/nand/nand_base.c
> +++ b/drivers/mtd/nand/nand_base.c
> @@ -3211,8 +3211,10 @@ int nand_erase_nand(struct mtd_info *mtd, struct erase_info *instr,
>  					chip->page_shift, allowbbt)) {
>  			pr_warn("%s: attempt to erase a bad block at page 0x%08x\n",
>  				    __func__, page);
> +			#ifndef CONFIG_MTD_NAND_ERASE_BADBLOCKS

#ifndef statements should be at the beginning of the line. But anyway,
I think we all agree that always forcing bad block erasure is a bad
idea. If we accept to support this feature, this should be done through
a per-NAND-chip debugfs entry, with fine grained selection of the block
that we're allowing to be forcibly erased.

Here is a suggestion:

echo all > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
echo none > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
echo X,Y-Z,... > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks

where X is an eraseblock number, and X-Y is a range of eraseblocks.

BTW, maybe we should create and expose a top-level mtd debugfs directory
to avoid exposing MTD related things at the root of the debugfs FS.

Something like:

/sys/kernel/debug/mtd/<dev-type>/...

so for the NAND related bits it would be

/sys/kernel/debug/mtd/nand/...

MTD maintainers, any opinion on this?

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Boris,

Am 15.05.2017 um 10:21 schrieb Boris Brezillon:
 > #ifndef statements should be at the beginning of the line. But anyway,
> I think we all agree that always forcing bad block erasure is a bad
> idea. If we accept to support this feature, this should be done through
> a per-NAND-chip debugfs entry, with fine grained selection of the block
> that we're allowing to be forcibly erased.
> 
> Here is a suggestion:
> 
> echo all > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
> echo none > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
> echo X,Y-Z,... > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
> 
> where X is an eraseblock number, and X-Y is a range of eraseblocks.

Will a write to that file trigger the erase or just allow it?

IMHO we can keept it simple such as:
echo y > /sys/kernel/debug/nand/<nand-chip-name>/allow-bad-block-erase

Then a user can erase whatever he wants...

> BTW, maybe we should create and expose a top-level mtd debugfs directory
> to avoid exposing MTD related things at the root of the debugfs FS.
> 
> Something like:
> 
> /sys/kernel/debug/mtd/<dev-type>/...
> 
> so for the NAND related bits it would be
> 
> /sys/kernel/debug/mtd/nand/...

Yes, this makes sense to me.

And since this is debugfs we can do what we want.

Thanks,
//richard

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Mon, 15 May 2017 11:23:55 +0200
Richard Weinberger <richard@nod.at> wrote:

> Boris,
> 
> Am 15.05.2017 um 10:21 schrieb Boris Brezillon:
>  > #ifndef statements should be at the beginning of the line. But anyway,
> > I think we all agree that always forcing bad block erasure is a bad
> > idea. If we accept to support this feature, this should be done through
> > a per-NAND-chip debugfs entry, with fine grained selection of the block
> > that we're allowing to be forcibly erased.
> > 
> > Here is a suggestion:
> > 
> > echo all > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
> > echo none > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
> > echo X,Y-Z,... > /sys/kernel/debug/nand/<nand-chip-name>/erase-bad-blocks
> > 
> > where X is an eraseblock number, and X-Y is a range of eraseblocks.  
> 
> Will a write to that file trigger the erase or just allow it?

Just allows it.

> 
> IMHO we can keept it simple such as:
> echo y > /sys/kernel/debug/nand/<nand-chip-name>/allow-bad-block-erase
> 
> Then a user can erase whatever he wants...

Well, I was proposing to do that because several users can use the same
NAND chip in parallel, and allowing to forcibly erase a bad block at
the NAND chip level can be dangerous in this case.
Here is a real example: 2 users are accessing 2 different partitions,
one wants to force bad block erasure on partition 1, while the other
just wants to normally erase partition 0. With the global
"allow/disallow bad block erasure" approach, you're just likely to
erase bad blocks in partition 0 as well.

> 
> > BTW, maybe we should create and expose a top-level mtd debugfs directory
> > to avoid exposing MTD related things at the root of the debugfs FS.
> > 
> > Something like:
> > 
> > /sys/kernel/debug/mtd/<dev-type>/...
> > 
> > so for the NAND related bits it would be
> > 
> > /sys/kernel/debug/mtd/nand/...  
> 
> Yes, this makes sense to me.
> 
> And since this is debugfs we can do what we want.

Yep.

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Boris,

Am 15.05.2017 um 11:41 schrieb Boris Brezillon:
>> IMHO we can keept it simple such as:
>> echo y > /sys/kernel/debug/nand/<nand-chip-name>/allow-bad-block-erase
>>
>> Then a user can erase whatever he wants...
> 
> Well, I was proposing to do that because several users can use the same
> NAND chip in parallel, and allowing to forcibly erase a bad block at
> the NAND chip level can be dangerous in this case.
> Here is a real example: 2 users are accessing 2 different partitions,
> one wants to force bad block erasure on partition 1, while the other
> just wants to normally erase partition 0. With the global
> "allow/disallow bad block erasure" approach, you're just likely to
> erase bad blocks in partition 0 as well.

Hmm, I'm not sure whether it makes sense to be smart in this case.
If somebody needs to fixup his bad blocks, like Mario described, there
are no other users and such an action should only be done when you
really know what you are doing.
This is a debug knob and not a knob that should be used in production
or while other parts of the chip are in use.

Thanks,
//richard

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Mon, 15 May 2017 12:10:58 +0200
Richard Weinberger <richard@nod.at> wrote:

> Boris,
> 
> Am 15.05.2017 um 11:41 schrieb Boris Brezillon:
> >> IMHO we can keept it simple such as:
> >> echo y > /sys/kernel/debug/nand/<nand-chip-name>/allow-bad-block-erase
> >>
> >> Then a user can erase whatever he wants...  
> > 
> > Well, I was proposing to do that because several users can use the same
> > NAND chip in parallel, and allowing to forcibly erase a bad block at
> > the NAND chip level can be dangerous in this case.
> > Here is a real example: 2 users are accessing 2 different partitions,
> > one wants to force bad block erasure on partition 1, while the other
> > just wants to normally erase partition 0. With the global
> > "allow/disallow bad block erasure" approach, you're just likely to
> > erase bad blocks in partition 0 as well.  
> 
> Hmm, I'm not sure whether it makes sense to be smart in this case.
> If somebody needs to fixup his bad blocks, like Mario described, there
> are no other users and such an action should only be done when you
> really know what you are doing.
> This is a debug knob and not a knob that should be used in production
> or while other parts of the chip are in use.

Fair enough. Let's keep it simple then.

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

Hi all,

I took note of all of your feedback. I'll be waiting until wednesday
for more feedback, and then provide two new patchsets.
Patchset 1 will focus on creating the debugfs entries for mtd.
Patchset 2 will add the ability to toggle force erasing to the debugfs.
Does that sound right to you?

Regards,
Mario.

Re: [PATCH v3] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Mon, 15 May 2017 10:16:51 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> Hi all,
> 
> I took note of all of your feedback. I'll be waiting until wednesday
> for more feedback, and then provide two new patchsets.
> Patchset 1 will focus on creating the debugfs entries for mtd.
> Patchset 2 will add the ability to toggle force erasing to the debugfs.

Keep in mind that this should be done on a per device basis (one
debugfs dir per NAND device created under /sys/kernel/debug/mtd/nand/).

> Does that sound right to you?

Yep.

Thanks,

Boris

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Mario,

On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote:
> Some chips used under a custom vendor driver can get their blocks
> incorrectly detected as bad blocks, out of incompatibilities
> between such drivers and MTD drivers.
> When there are too many misdetected bad blocks, the device becomes
> unusable because a bad block table can't be allocated, aside from
> all the legitimately good blocks which become unusable under these
> conditions.
> This adds a build option to workaround the issue by enabling the
> user to free up space regardless of what the driver thinks about
> the blocks.

Hmm, this sounds like a gross hack.

> Example usage: recovering NAND chips on sunxi devices, as explained
> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues

What this wiki suggests is not wise.
How can you know which blocks are really bad and which not?

-- 
Thanks,
//richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>:
> Mario,
>
> On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote:
>> Some chips used under a custom vendor driver can get their blocks
>> incorrectly detected as bad blocks, out of incompatibilities
>> between such drivers and MTD drivers.
>> When there are too many misdetected bad blocks, the device becomes
>> unusable because a bad block table can't be allocated, aside from
>> all the legitimately good blocks which become unusable under these
>> conditions.
>> This adds a build option to workaround the issue by enabling the
>> user to free up space regardless of what the driver thinks about
>> the blocks.
>
> Hmm, this sounds like a gross hack.
It is, but I see no other solution. The NAND chips were used in an
incompatible way by a hack-n-slash driver made by allwinner, and
trying to load them with a proper MTD driver fails miserably if this
is not done.
If anyone can propose a better solution I'll more than happily implement it.
I'm open to suggestions, and of course I'm open to rejection of my
patches if needed.
>
>> Example usage: recovering NAND chips on sunxi devices, as explained
>> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues
>
> What this wiki suggests is not wise.
> How can you know which blocks are really bad and which not?
You don't, at least not without an even grosser hack implementing read
support for their incompatible format.
Would that be better? I might attempt it if desired.
>
> --
> Thanks,
> //richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 05:16:08 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>:
> > Mario,
> >
> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote:  
> >> Some chips used under a custom vendor driver can get their blocks
> >> incorrectly detected as bad blocks, out of incompatibilities
> >> between such drivers and MTD drivers.
> >> When there are too many misdetected bad blocks, the device becomes
> >> unusable because a bad block table can't be allocated, aside from
> >> all the legitimately good blocks which become unusable under these
> >> conditions.
> >> This adds a build option to workaround the issue by enabling the
> >> user to free up space regardless of what the driver thinks about
> >> the blocks.  
> >
> > Hmm, this sounds like a gross hack.  
> It is, but I see no other solution. The NAND chips were used in an
> incompatible way by a hack-n-slash driver made by allwinner, and
> trying to load them with a proper MTD driver fails miserably if this
> is not done.
> If anyone can propose a better solution I'll more than happily implement it.
> I'm open to suggestions, and of course I'm open to rejection of my
> patches if needed.

u-boot provides the nand.scrub command, which does exactly what you're
looking for. And no, I don't think it's a good idea to allow erasing
bad blocks, at least not by default.

If we really want to support this feature in linux, this should be
explicitly enabled through debugfs.

> >  
> >> Example usage: recovering NAND chips on sunxi devices, as explained
> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues  
> >
> > What this wiki suggests is not wise.
> > How can you know which blocks are really bad and which not?  
> You don't, at least not without an even grosser hack implementing read
> support for their incompatible format.
> Would that be better? I might attempt it if desired.

No, please don't do that, at least not in the kernel. If you really
want to parse the old format, you should develop a tool that reads NAND
pages in raw mode, stores the list of bad blocks somewhere and then
re-use this list to select which blocks should be forcibly erased.

Not sure it's worth the pain :-).

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Boris, Mario,

Am 12.05.2017 um 10:24 schrieb Boris Brezillon:
>>> Hmm, this sounds like a gross hack.  
>> It is, but I see no other solution. The NAND chips were used in an
>> incompatible way by a hack-n-slash driver made by allwinner, and
>> trying to load them with a proper MTD driver fails miserably if this
>> is not done.
>> If anyone can propose a better solution I'll more than happily implement it.
>> I'm open to suggestions, and of course I'm open to rejection of my
>> patches if needed.
> 
> u-boot provides the nand.scrub command, which does exactly what you're
> looking for. And no, I don't think it's a good idea to allow erasing
> bad blocks, at least not by default.

To make this very clear for all MTD users out there, scrubbing the NAND and
losing the information which blocks are bad is awful.
Bad blocks can work somehow and fail much later in funny ways.
UBI/FS problems ahead...

Do this only if you *really* know what you are doing.

Thanks,
//richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 10:33:27 +0200
Richard Weinberger <richard@nod.at> wrote:

> Boris, Mario,
> 
> Am 12.05.2017 um 10:24 schrieb Boris Brezillon:
> >>> Hmm, this sounds like a gross hack.    
> >> It is, but I see no other solution. The NAND chips were used in an
> >> incompatible way by a hack-n-slash driver made by allwinner, and
> >> trying to load them with a proper MTD driver fails miserably if this
> >> is not done.
> >> If anyone can propose a better solution I'll more than happily implement it.
> >> I'm open to suggestions, and of course I'm open to rejection of my
> >> patches if needed.  
> > 
> > u-boot provides the nand.scrub command, which does exactly what you're
> > looking for. And no, I don't think it's a good idea to allow erasing
> > bad blocks, at least not by default.  
> 
> To make this very clear for all MTD users out there, scrubbing the NAND and
> losing the information which blocks are bad is awful.
> Bad blocks can work somehow and fail much later in funny ways.
> UBI/FS problems ahead...
> 
> Do this only if you *really* know what you are doing.

I'm clearly not encouraging people to use nand.scrub, it's just that
sometime you don't have a choice, and this is the case here:
Allwinner is putting non-FF data in the BBM region, and when we switch
from an Allwinner kernel to a mainline kernel, the mainline kernel
considers all programmed blocks as bad.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Boris,

Am 12.05.2017 um 10:44 schrieb Boris Brezillon:
>> Do this only if you *really* know what you are doing.
> 
> I'm clearly not encouraging people to use nand.scrub, it's just that
> sometime you don't have a choice, and this is the case here:
> Allwinner is putting non-FF data in the BBM region, and when we switch
> from an Allwinner kernel to a mainline kernel, the mainline kernel
> considers all programmed blocks as bad.

This is the "...only if you *really* know what you are doing" part. :-)

Thanks,
//richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
> On Fri, 12 May 2017 05:16:08 -0300
> Mario Rugiero <mrugiero@gmail.com> wrote:
>
>> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>:
>> > Mario,
>> >
>> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote:
>> >> Some chips used under a custom vendor driver can get their blocks
>> >> incorrectly detected as bad blocks, out of incompatibilities
>> >> between such drivers and MTD drivers.
>> >> When there are too many misdetected bad blocks, the device becomes
>> >> unusable because a bad block table can't be allocated, aside from
>> >> all the legitimately good blocks which become unusable under these
>> >> conditions.
>> >> This adds a build option to workaround the issue by enabling the
>> >> user to free up space regardless of what the driver thinks about
>> >> the blocks.
>> >
>> > Hmm, this sounds like a gross hack.
>> It is, but I see no other solution. The NAND chips were used in an
>> incompatible way by a hack-n-slash driver made by allwinner, and
>> trying to load them with a proper MTD driver fails miserably if this
>> is not done.
>> If anyone can propose a better solution I'll more than happily implement it.
>> I'm open to suggestions, and of course I'm open to rejection of my
>> patches if needed.
>
> u-boot provides the nand.scrub command, which does exactly what you're
> looking for. And no, I don't think it's a good idea to allow erasing
> bad blocks, at least not by default.
>
> If we really want to support this feature in linux, this should be
> explicitly enabled through debugfs.
If I do this, does it stand a chance at getting upstream?
If so, I'll have it done soon.
Note however that the build option is disabled by default. I get that
there should also be one runtime option, disabled by default, exposed
through debugfs. Does that sound right?
>
>> >
>> >> Example usage: recovering NAND chips on sunxi devices, as explained
>> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues
>> >
>> > What this wiki suggests is not wise.
>> > How can you know which blocks are really bad and which not?
>> You don't, at least not without an even grosser hack implementing read
>> support for their incompatible format.
>> Would that be better? I might attempt it if desired.
>
> No, please don't do that, at least not in the kernel. If you really
> want to parse the old format, you should develop a tool that reads NAND
> pages in raw mode, stores the list of bad blocks somewhere and then
> re-use this list to select which blocks should be forcibly erased.
>
> Not sure it's worth the pain :-).
It's worth the pain to me. I'm dealing with a bit rotten 3.4 based
pile of cr*p on production because of this. Whatever I have to do to
get those machines running the mainline kernel is worth it.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 05:34:10 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
> > On Fri, 12 May 2017 05:16:08 -0300
> > Mario Rugiero <mrugiero@gmail.com> wrote:
> >  
> >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <richard.weinberger@gmail.com>:  
> >> > Mario,
> >> >
> >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com> wrote:  
> >> >> Some chips used under a custom vendor driver can get their blocks
> >> >> incorrectly detected as bad blocks, out of incompatibilities
> >> >> between such drivers and MTD drivers.
> >> >> When there are too many misdetected bad blocks, the device becomes
> >> >> unusable because a bad block table can't be allocated, aside from
> >> >> all the legitimately good blocks which become unusable under these
> >> >> conditions.
> >> >> This adds a build option to workaround the issue by enabling the
> >> >> user to free up space regardless of what the driver thinks about
> >> >> the blocks.  
> >> >
> >> > Hmm, this sounds like a gross hack.  
> >> It is, but I see no other solution. The NAND chips were used in an
> >> incompatible way by a hack-n-slash driver made by allwinner, and
> >> trying to load them with a proper MTD driver fails miserably if this
> >> is not done.
> >> If anyone can propose a better solution I'll more than happily implement it.
> >> I'm open to suggestions, and of course I'm open to rejection of my
> >> patches if needed.  
> >
> > u-boot provides the nand.scrub command, which does exactly what you're
> > looking for. And no, I don't think it's a good idea to allow erasing
> > bad blocks, at least not by default.
> >
> > If we really want to support this feature in linux, this should be
> > explicitly enabled through debugfs.  
> If I do this, does it stand a chance at getting upstream?
> If so, I'll have it done soon.
> Note however that the build option is disabled by default. I get that
> there should also be one runtime option, disabled by default, exposed
> through debugfs. Does that sound right?
> >  
> >> >  
> >> >> Example usage: recovering NAND chips on sunxi devices, as explained
> >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues  
> >> >
> >> > What this wiki suggests is not wise.
> >> > How can you know which blocks are really bad and which not?  
> >> You don't, at least not without an even grosser hack implementing read
> >> support for their incompatible format.
> >> Would that be better? I might attempt it if desired.  
> >
> > No, please don't do that, at least not in the kernel. If you really
> > want to parse the old format, you should develop a tool that reads NAND
> > pages in raw mode, stores the list of bad blocks somewhere and then
> > re-use this list to select which blocks should be forcibly erased.
> >
> > Not sure it's worth the pain :-).  
> It's worth the pain to me. I'm dealing with a bit rotten 3.4 based
> pile of cr*p on production because of this. Whatever I have to do to
> get those machines running the mainline kernel is worth it.

No, I meant, doing that vs scrubbing the NAND. Note that MLC support is
not reliable in mainline, so I'd strongly discourage to use a mainline
kernel right now, unless you have an SLC NAND.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 05:56:40 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> El may. 12, 2017 5:46, "Boris Brezillon" <boris.brezillon@free-electrons.com>
> escribió:
> 
> On Fri, 12 May 2017 05:34:10 -0300
> Mario Rugiero <mrugiero@gmail.com> wrote:
> 
> > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-
> electrons.com>:  
> > > On Fri, 12 May 2017 05:16:08 -0300
> > > Mario Rugiero <mrugiero@gmail.com> wrote:
> > >  
> > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <  
> richard.weinberger@gmail.com>:  
> > >> > Mario,
> > >> >
> > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com>  
> wrote:
> > >> >> Some chips used under a custom vendor driver can get their blocks
> > >> >> incorrectly detected as bad blocks, out of incompatibilities
> > >> >> between such drivers and MTD drivers.
> > >> >> When there are too many misdetected bad blocks, the device becomes
> > >> >> unusable because a bad block table can't be allocated, aside from
> > >> >> all the legitimately good blocks which become unusable under these
> > >> >> conditions.
> > >> >> This adds a build option to workaround the issue by enabling the
> > >> >> user to free up space regardless of what the driver thinks about
> > >> >> the blocks.  
> > >> >
> > >> > Hmm, this sounds like a gross hack.  
> > >> It is, but I see no other solution. The NAND chips were used in an
> > >> incompatible way by a hack-n-slash driver made by allwinner, and
> > >> trying to load them with a proper MTD driver fails miserably if this
> > >> is not done.
> > >> If anyone can propose a better solution I'll more than happily  
> implement it.
> > >> I'm open to suggestions, and of course I'm open to rejection of my
> > >> patches if needed.  
> > >
> > > u-boot provides the nand.scrub command, which does exactly what you're
> > > looking for. And no, I don't think it's a good idea to allow erasing
> > > bad blocks, at least not by default.
> > >
> > > If we really want to support this feature in linux, this should be
> > > explicitly enabled through debugfs.  
> > If I do this, does it stand a chance at getting upstream?
> > If so, I'll have it done soon.
> > Note however that the build option is disabled by default. I get that
> > there should also be one runtime option, disabled by default, exposed
> > through debugfs. Does that sound right?  
> > >  
> > >> >  
> > >> >> Example usage: recovering NAND chips on sunxi devices, as explained
> > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues  
> > >> >
> > >> > What this wiki suggests is not wise.
> > >> > How can you know which blocks are really bad and which not?  
> > >> You don't, at least not without an even grosser hack implementing read
> > >> support for their incompatible format.
> > >> Would that be better? I might attempt it if desired.  
> > >
> > > No, please don't do that, at least not in the kernel. If you really
> > > want to parse the old format, you should develop a tool that reads NAND
> > > pages in raw mode, stores the list of bad blocks somewhere and then
> > > re-use this list to select which blocks should be forcibly erased.
> > >
> > > Not sure it's worth the pain :-).  
> > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based
> > pile of cr*p on production because of this. Whatever I have to do to
> > get those machines running the mainline kernel is worth it.  
> 
> No, I meant, doing that vs scrubbing the NAND. Note that MLC support is
> not reliable in mainline, so I'd strongly discourage to use a mainline
> kernel right now, unless you have an SLC NAND.
> 
> I know. Sunxi's driver doesn't seem stable either, though, and I've read
> using an MLC chip as SLC by half The storage capacity was a viable
> solution.

Well, yes, but it's not supported either (at least not in mainline).

> If it isn't implemented right now, I might implement that
> solution in The meantime to a proper fix. Sadly, I'm not skilled enough for
> that final solution.

I have a branch containing the work we did we Richard to reliably
support MLC NANDs. It's still WIP, but should give a rough idea of the
solution we're heading to [1].

[1]https://github.com/bbrezillon/linux-sunxi/commits/bb/4.7/ubi-mlc

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 6:02 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
> On Fri, 12 May 2017 05:56:40 -0300
> Mario Rugiero <mrugiero@gmail.com> wrote:
>
>> El may. 12, 2017 5:46, "Boris Brezillon" <boris.brezillon@free-electrons.com>
>> escribió:
>>
>> On Fri, 12 May 2017 05:34:10 -0300
>> Mario Rugiero <mrugiero@gmail.com> wrote:
>>
>> > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-
>> electrons.com>:
>> > > On Fri, 12 May 2017 05:16:08 -0300
>> > > Mario Rugiero <mrugiero@gmail.com> wrote:
>> > >
>> > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <
>> richard.weinberger@gmail.com>:
>> > >> > Mario,
>> > >> >
>> > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com>
>> wrote:
>> > >> >> Some chips used under a custom vendor driver can get their blocks
>> > >> >> incorrectly detected as bad blocks, out of incompatibilities
>> > >> >> between such drivers and MTD drivers.
>> > >> >> When there are too many misdetected bad blocks, the device becomes
>> > >> >> unusable because a bad block table can't be allocated, aside from
>> > >> >> all the legitimately good blocks which become unusable under these
>> > >> >> conditions.
>> > >> >> This adds a build option to workaround the issue by enabling the
>> > >> >> user to free up space regardless of what the driver thinks about
>> > >> >> the blocks.
>> > >> >
>> > >> > Hmm, this sounds like a gross hack.
>> > >> It is, but I see no other solution. The NAND chips were used in an
>> > >> incompatible way by a hack-n-slash driver made by allwinner, and
>> > >> trying to load them with a proper MTD driver fails miserably if this
>> > >> is not done.
>> > >> If anyone can propose a better solution I'll more than happily
>> implement it.
>> > >> I'm open to suggestions, and of course I'm open to rejection of my
>> > >> patches if needed.
>> > >
>> > > u-boot provides the nand.scrub command, which does exactly what you're
>> > > looking for. And no, I don't think it's a good idea to allow erasing
>> > > bad blocks, at least not by default.
>> > >
>> > > If we really want to support this feature in linux, this should be
>> > > explicitly enabled through debugfs.
>> > If I do this, does it stand a chance at getting upstream?
>> > If so, I'll have it done soon.
>> > Note however that the build option is disabled by default. I get that
>> > there should also be one runtime option, disabled by default, exposed
>> > through debugfs. Does that sound right?
>> > >
>> > >> >
>> > >> >> Example usage: recovering NAND chips on sunxi devices, as explained
>> > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues
>> > >> >
>> > >> > What this wiki suggests is not wise.
>> > >> > How can you know which blocks are really bad and which not?
>> > >> You don't, at least not without an even grosser hack implementing read
>> > >> support for their incompatible format.
>> > >> Would that be better? I might attempt it if desired.
>> > >
>> > > No, please don't do that, at least not in the kernel. If you really
>> > > want to parse the old format, you should develop a tool that reads NAND
>> > > pages in raw mode, stores the list of bad blocks somewhere and then
>> > > re-use this list to select which blocks should be forcibly erased.
>> > >
>> > > Not sure it's worth the pain :-).
>> > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based
>> > pile of cr*p on production because of this. Whatever I have to do to
>> > get those machines running the mainline kernel is worth it.
>>
>> No, I meant, doing that vs scrubbing the NAND. Note that MLC support is
>> not reliable in mainline, so I'd strongly discourage to use a mainline
>> kernel right now, unless you have an SLC NAND.
>>
>> I know. Sunxi's driver doesn't seem stable either, though, and I've read
>> using an MLC chip as SLC by half The storage capacity was a viable
>> solution.
>
> Well, yes, but it's not supported either (at least not in mainline).
>
>> If it isn't implemented right now, I might implement that
>> solution in The meantime to a proper fix. Sadly, I'm not skilled enough for
>> that final solution.
>
> I have a branch containing the work we did we Richard to reliably
> support MLC NANDs. It's still WIP, but should give a rough idea of the
> solution we're heading to [1].
>
> [1]https://github.com/bbrezillon/linux-sunxi/commits/bb/4.7/ubi-mlc
I'll read it carefully later. Is there any rough time estimate for it
to hit mainline?
I'm not expecting a date, but rather something in the lines of
"several weeks, several months, several years".
I think we can do with several months, and we'd be happy to start
local experiments with that timeframe in mind.
Several years might be more than the devices will live, though.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

Also, if you have any pointers to where to study the issue, I might
convince my boss to let me allocate time to lend a hand, if you need
workers.

2017-05-12 6:15 GMT-03:00 Mario Rugiero <mrugiero@gmail.com>:
> 2017-05-12 6:02 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
>> On Fri, 12 May 2017 05:56:40 -0300
>> Mario Rugiero <mrugiero@gmail.com> wrote:
>>
>>> El may. 12, 2017 5:46, "Boris Brezillon" <boris.brezillon@free-electrons.com>
>>> escribió:
>>>
>>> On Fri, 12 May 2017 05:34:10 -0300
>>> Mario Rugiero <mrugiero@gmail.com> wrote:
>>>
>>> > 2017-05-12 5:24 GMT-03:00 Boris Brezillon <boris.brezillon@free-
>>> electrons.com>:
>>> > > On Fri, 12 May 2017 05:16:08 -0300
>>> > > Mario Rugiero <mrugiero@gmail.com> wrote:
>>> > >
>>> > >> 2017-05-12 5:12 GMT-03:00 Richard Weinberger <
>>> richard.weinberger@gmail.com>:
>>> > >> > Mario,
>>> > >> >
>>> > >> > On Fri, May 12, 2017 at 7:39 AM, Mario J. Rugiero <mrugiero@gmail.com>
>>> wrote:
>>> > >> >> Some chips used under a custom vendor driver can get their blocks
>>> > >> >> incorrectly detected as bad blocks, out of incompatibilities
>>> > >> >> between such drivers and MTD drivers.
>>> > >> >> When there are too many misdetected bad blocks, the device becomes
>>> > >> >> unusable because a bad block table can't be allocated, aside from
>>> > >> >> all the legitimately good blocks which become unusable under these
>>> > >> >> conditions.
>>> > >> >> This adds a build option to workaround the issue by enabling the
>>> > >> >> user to free up space regardless of what the driver thinks about
>>> > >> >> the blocks.
>>> > >> >
>>> > >> > Hmm, this sounds like a gross hack.
>>> > >> It is, but I see no other solution. The NAND chips were used in an
>>> > >> incompatible way by a hack-n-slash driver made by allwinner, and
>>> > >> trying to load them with a proper MTD driver fails miserably if this
>>> > >> is not done.
>>> > >> If anyone can propose a better solution I'll more than happily
>>> implement it.
>>> > >> I'm open to suggestions, and of course I'm open to rejection of my
>>> > >> patches if needed.
>>> > >
>>> > > u-boot provides the nand.scrub command, which does exactly what you're
>>> > > looking for. And no, I don't think it's a good idea to allow erasing
>>> > > bad blocks, at least not by default.
>>> > >
>>> > > If we really want to support this feature in linux, this should be
>>> > > explicitly enabled through debugfs.
>>> > If I do this, does it stand a chance at getting upstream?
>>> > If so, I'll have it done soon.
>>> > Note however that the build option is disabled by default. I get that
>>> > there should also be one runtime option, disabled by default, exposed
>>> > through debugfs. Does that sound right?
>>> > >
>>> > >> >
>>> > >> >> Example usage: recovering NAND chips on sunxi devices, as explained
>>> > >> >> here: http://linux-sunxi.org/Mainline_NAND_Howto#Known_issues
>>> > >> >
>>> > >> > What this wiki suggests is not wise.
>>> > >> > How can you know which blocks are really bad and which not?
>>> > >> You don't, at least not without an even grosser hack implementing read
>>> > >> support for their incompatible format.
>>> > >> Would that be better? I might attempt it if desired.
>>> > >
>>> > > No, please don't do that, at least not in the kernel. If you really
>>> > > want to parse the old format, you should develop a tool that reads NAND
>>> > > pages in raw mode, stores the list of bad blocks somewhere and then
>>> > > re-use this list to select which blocks should be forcibly erased.
>>> > >
>>> > > Not sure it's worth the pain :-).
>>> > It's worth the pain to me. I'm dealing with a bit rotten 3.4 based
>>> > pile of cr*p on production because of this. Whatever I have to do to
>>> > get those machines running the mainline kernel is worth it.
>>>
>>> No, I meant, doing that vs scrubbing the NAND. Note that MLC support is
>>> not reliable in mainline, so I'd strongly discourage to use a mainline
>>> kernel right now, unless you have an SLC NAND.
>>>
>>> I know. Sunxi's driver doesn't seem stable either, though, and I've read
>>> using an MLC chip as SLC by half The storage capacity was a viable
>>> solution.
>>
>> Well, yes, but it's not supported either (at least not in mainline).
>>
>>> If it isn't implemented right now, I might implement that
>>> solution in The meantime to a proper fix. Sadly, I'm not skilled enough for
>>> that final solution.
>>
>> I have a branch containing the work we did we Richard to reliably
>> support MLC NANDs. It's still WIP, but should give a rough idea of the
>> solution we're heading to [1].
>>
>> [1]https://github.com/bbrezillon/linux-sunxi/commits/bb/4.7/ubi-mlc
> I'll read it carefully later. Is there any rough time estimate for it
> to hit mainline?
> I'm not expecting a date, but rather something in the lines of
> "several weeks, several months, several years".
> I think we can do with several months, and we'd be happy to start
> local experiments with that timeframe in mind.
> Several years might be more than the devices will live, though.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 06:16:33 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> Also, if you have any pointers to where to study the issue, I might
> convince my boss to let me allocate time to lend a hand, if you need
> workers.

Here are the slides [1] the talk we gave at ELCE last year.
Should be a good starting point to understand the problem and how we'd
like to solve them.

[1]http://events.linuxfoundation.org/sites/events/files/slides/ubi-mlc.pdf

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Am 12.05.2017 um 11:15 schrieb Mario Rugiero:
> I'll read it carefully later. Is there any rough time estimate for it
> to hit mainline?
> I'm not expecting a date, but rather something in the lines of
> "several weeks, several months, several years".
> I think we can do with several months, and we'd be happy to start
> local experiments with that timeframe in mind.
> Several years might be more than the devices will live, though.

Since there is currently no funding it might take much longer.

Thanks,
//richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>:
> Am 12.05.2017 um 11:15 schrieb Mario Rugiero:
>> I'll read it carefully later. Is there any rough time estimate for it
>> to hit mainline?
>> I'm not expecting a date, but rather something in the lines of
>> "several weeks, several months, several years".
>> I think we can do with several months, and we'd be happy to start
>> local experiments with that timeframe in mind.
>> Several years might be more than the devices will live, though.
>
> Since there is currently no funding it might take much longer.
GSoC sounds like a viable workaround.
It'd hahve to wait for next year, though.
I think I'm elligible, so if I still have access to these boards and
Boris can mentor me, we could get funding to speed up the development.
I'd be happy to donate it to have another developer, as I'm already
kinda being paid for this.
What do you think of that?
>
> Thanks,
> //richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 06:26:14 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>:
> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero:  
> >> I'll read it carefully later. Is there any rough time estimate for it
> >> to hit mainline?
> >> I'm not expecting a date, but rather something in the lines of
> >> "several weeks, several months, several years".
> >> I think we can do with several months, and we'd be happy to start
> >> local experiments with that timeframe in mind.
> >> Several years might be more than the devices will live, though.  
> >
> > Since there is currently no funding it might take much longer.  
> GSoC sounds like a viable workaround.
> It'd hahve to wait for next year, though.
> I think I'm elligible, so if I still have access to these boards and
> Boris can mentor me, we could get funding to speed up the development.
> I'd be happy to donate it to have another developer, as I'm already
> kinda being paid for this.
> What do you think of that?

Yep, that's a possibility.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
> On Fri, 12 May 2017 06:26:14 -0300
> Mario Rugiero <mrugiero@gmail.com> wrote:
>
>> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>:
>> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero:
>> >> I'll read it carefully later. Is there any rough time estimate for it
>> >> to hit mainline?
>> >> I'm not expecting a date, but rather something in the lines of
>> >> "several weeks, several months, several years".
>> >> I think we can do with several months, and we'd be happy to start
>> >> local experiments with that timeframe in mind.
>> >> Several years might be more than the devices will live, though.
>> >
>> > Since there is currently no funding it might take much longer.
>> GSoC sounds like a viable workaround.
>> It'd hahve to wait for next year, though.
>> I think I'm elligible, so if I still have access to these boards and
>> Boris can mentor me, we could get funding to speed up the development.
>> I'd be happy to donate it to have another developer, as I'm already
>> kinda being paid for this.
>> What do you think of that?
>
> Yep, that's a possibility.
Great. Could you act as a mentor without GSoC for now?
I think I might be able to help after finishing with the slides, they
seem clear enough, with proper guidance, and that'd help in my current
work.
I think I've read some slides from a previous talk by you, before the
way to go was as clearly defined.

Regards,
Mario.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Boris Brezillon]

On Fri, 12 May 2017 07:06:45 -0300
Mario Rugiero <mrugiero@gmail.com> wrote:

> 2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
> > On Fri, 12 May 2017 06:26:14 -0300
> > Mario Rugiero <mrugiero@gmail.com> wrote:
> >  
> >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>:  
> >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero:  
> >> >> I'll read it carefully later. Is there any rough time estimate for it
> >> >> to hit mainline?
> >> >> I'm not expecting a date, but rather something in the lines of
> >> >> "several weeks, several months, several years".
> >> >> I think we can do with several months, and we'd be happy to start
> >> >> local experiments with that timeframe in mind.
> >> >> Several years might be more than the devices will live, though.  
> >> >
> >> > Since there is currently no funding it might take much longer.  
> >> GSoC sounds like a viable workaround.
> >> It'd hahve to wait for next year, though.
> >> I think I'm elligible, so if I still have access to these boards and
> >> Boris can mentor me, we could get funding to speed up the development.
> >> I'd be happy to donate it to have another developer, as I'm already
> >> kinda being paid for this.
> >> What do you think of that?  
> >
> > Yep, that's a possibility.  
> Great. Could you act as a mentor without GSoC for now?

Sure.

> I think I might be able to help after finishing with the slides, they
> seem clear enough, with proper guidance, and that'd help in my current
> work.
> I think I've read some slides from a previous talk by you, before the
> way to go was as clearly defined.

Actually, most of the development is done already, but we need to
intensively test the implementation and cleanup the code.

I also started to modify mtd-utils [1] to support the new format, but
didn't have time to finish.

[1]https://github.com/bbrezillon/mtd-utils/tree/ubi/mlc

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 7:19 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
> On Fri, 12 May 2017 07:06:45 -0300
> Mario Rugiero <mrugiero@gmail.com> wrote:
>
>> 2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
>> > On Fri, 12 May 2017 06:26:14 -0300
>> > Mario Rugiero <mrugiero@gmail.com> wrote:
>> >
>> >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>:
>> >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero:
>> >> >> I'll read it carefully later. Is there any rough time estimate for it
>> >> >> to hit mainline?
>> >> >> I'm not expecting a date, but rather something in the lines of
>> >> >> "several weeks, several months, several years".
>> >> >> I think we can do with several months, and we'd be happy to start
>> >> >> local experiments with that timeframe in mind.
>> >> >> Several years might be more than the devices will live, though.
>> >> >
>> >> > Since there is currently no funding it might take much longer.
>> >> GSoC sounds like a viable workaround.
>> >> It'd hahve to wait for next year, though.
>> >> I think I'm elligible, so if I still have access to these boards and
>> >> Boris can mentor me, we could get funding to speed up the development.
>> >> I'd be happy to donate it to have another developer, as I'm already
>> >> kinda being paid for this.
>> >> What do you think of that?
>> >
>> > Yep, that's a possibility.
>> Great. Could you act as a mentor without GSoC for now?
>
> Sure.
>
>> I think I might be able to help after finishing with the slides, they
>> seem clear enough, with proper guidance, and that'd help in my current
>> work.
>> I think I've read some slides from a previous talk by you, before the
>> way to go was as clearly defined.
>
> Actually, most of the development is done already, but we need to
> intensively test the implementation and cleanup the code.
If you can describe the tests to me, I think I can have them running
in at least three devices during next week.
I can help with the cleanup, too.
>
> I also started to modify mtd-utils [1] to support the new format, but
> didn't have time to finish.
>
> [1]https://github.com/bbrezillon/mtd-utils/tree/ubi/mlc
I'll try to help with that.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

2017-05-12 7:23 GMT-03:00 Mario Rugiero <mrugiero@gmail.com>:
> 2017-05-12 7:19 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
>> On Fri, 12 May 2017 07:06:45 -0300
>> Mario Rugiero <mrugiero@gmail.com> wrote:
>>
>>> 2017-05-12 6:34 GMT-03:00 Boris Brezillon <boris.brezillon@free-electrons.com>:
>>> > On Fri, 12 May 2017 06:26:14 -0300
>>> > Mario Rugiero <mrugiero@gmail.com> wrote:
>>> >
>>> >> 2017-05-12 6:19 GMT-03:00 Richard Weinberger <richard@nod.at>:
>>> >> > Am 12.05.2017 um 11:15 schrieb Mario Rugiero:
>>> >> >> I'll read it carefully later. Is there any rough time estimate for it
>>> >> >> to hit mainline?
>>> >> >> I'm not expecting a date, but rather something in the lines of
>>> >> >> "several weeks, several months, several years".
>>> >> >> I think we can do with several months, and we'd be happy to start
>>> >> >> local experiments with that timeframe in mind.
>>> >> >> Several years might be more than the devices will live, though.
>>> >> >
>>> >> > Since there is currently no funding it might take much longer.
>>> >> GSoC sounds like a viable workaround.
>>> >> It'd hahve to wait for next year, though.
>>> >> I think I'm elligible, so if I still have access to these boards and
>>> >> Boris can mentor me, we could get funding to speed up the development.
>>> >> I'd be happy to donate it to have another developer, as I'm already
>>> >> kinda being paid for this.
>>> >> What do you think of that?
>>> >
>>> > Yep, that's a possibility.
>>> Great. Could you act as a mentor without GSoC for now?
>>
>> Sure.
>>
>>> I think I might be able to help after finishing with the slides, they
>>> seem clear enough, with proper guidance, and that'd help in my current
>>> work.
>>> I think I've read some slides from a previous talk by you, before the
>>> way to go was as clearly defined.
>>
>> Actually, most of the development is done already, but we need to
>> intensively test the implementation and cleanup the code.
> If you can describe the tests to me, I think I can have them running
> in at least three devices during next week.
> I can help with the cleanup, too.
>>
>> I also started to modify mtd-utils [1] to support the new format, but
>> didn't have time to finish.
>>
>> [1]https://github.com/bbrezillon/mtd-utils/tree/ubi/mlc
> I'll try to help with that.
Back to the original subject, scrubbing from Linux on demand.
Is it a total no-go, or if I add the debugfs entry to make it opt-in
on runtime too it might get accepted?
Is it viable to add it as a new operation, so it can be more explicit
about the risks?
Where would be the proper place to put the debugfs entry?
nand_base.c doesn't seem to deal with debugfs, and exporting the
variable seems too risky IMO.

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Richard Weinberger]

Mario,

Am 12.05.2017 um 12:23 schrieb Mario Rugiero:
>> Actually, most of the development is done already, but we need to
>> intensively test the implementation and cleanup the code.
> If you can describe the tests to me, I think I can have them running
> in at least three devices during next week.
> I can help with the cleanup, too.

Well, the critical and most important task is understanding the existing
issues, see slides, and find a bulletproof solution that scales.

If you have questions, don't hesitate to ask. You can also find
me and Boris on #mtd.

Thanks,
//richard

Re: [PATCH] mtd: nand: add option to erase NAND blocks even if detected as bad.

[Mario Rugiero]

Sorry for the noise, but I'm resending since my phone is fixated on
sending HTML mails, so it was rejected from the list:
After some thought, I believe the proper solution is the one proposed
by Boris: write a little program which classifies the false positives
for bad blocks, and erase those blocks forcibly.
What I'm currently wondering is if the incompatibilities come with
false negatives too. In such a case, we should be able to provide the
MTD driver with the information we just gathered, or is it viable to
just program the BBT we just built?

Regards,
Mario.

PD (not in the original mail): would mtd-tools the right place to do
this, or rather sunxi-tools? I think converting from vendor-specific
layouts to MTD could be a more common problem, so I thought of trying
and making it generic enough to support several backends.

2017-05-13 6:17 GMT-03:00 Richard Weinberger <richard@nod.at>:
> Mario,
>
> Am 12.05.2017 um 12:23 schrieb Mario Rugiero:
>>> Actually, most of the development is done already, but we need to
>>> intensively test the implementation and cleanup the code.
>> If you can describe the tests to me, I think I can have them running
>> in at least three devices during next week.
>> I can help with the cleanup, too.
>
> Well, the critical and most important task is understanding the existing
> issues, see slides, and find a bulletproof solution that scales.
>
> If you have questions, don't hesitate to ask. You can also find
> me and Boris on #mtd.
>
> Thanks,
> //richard