nanddump badblock options

nanddump badblock options

[Brian Norris]

Hello,

I have some ideas to implement in nanddump regarding the variety of bad block
handling options. I thought I'd at least get some feedback before working up a
full patch, so please comment on my ideas.

(1) The comments in nandwrite say that nandwrite is an "inverse operation" to
nanddump. However, take, for example, the following command:

   nandwrite --length=131072 /dev/mtd1 myfile

Then, if we consider that there may be bad blocks at the beginning of the
device, nandwrite may skip to the second block in order to write this data.
Now, the default behavior of nanddump does not at all fit the "inverse" of this
very simple nandwrite operation. While you might expect the following to be an
inverse:

   nanddump --length=131072 /dev/mtd1 --file=myfile.dump

you in fact will not get the same data that you wrote from the original file.
Instead, you will get all 0xFF since by default nanddump substitutes 0xFF for
all the data of the bad block. I call this (unwanted) behavior `padding'.

Thus, in short, I'm recommending that nanddump default to using --skipbad as
a default option, with a new `padbad' option to cover the original behavior.
Perhaps the "default" nanddump should have a warning over a period of time,
before changing the default operation? See (3), Deprecation schedule.

(2) There are now (with my addition of `skipbad', and the current default
`padbad') four methods used for handling bad blocks we come across when dumping
flash data. I think they'd be cleaner if they were all grouped under a single
option that would work something like:

   --bb=METHOD

where METHOD could be `padbad', `dumpbad', `skipbad', and `omitbad'. Notice the
renaming of --noskipbad to --bb=dumpbad, since --noskipbad seems like an
inverse to --skipbad, which it is not. See (5), Summary table.

I think eventually, we would just drop both the short and long options for the
--omitbad, --noskipbad, and --skipbad options.

(3) Deprecation schedule:

Assuming the above is agreeable to everyone, how soon can we:
   * drop the --noskipbad, --skipbad, --omitbad (pluse -b, -k, -N) flags in
     favor of --bb=METHOD?
   * change the default behavior from `padbad' to `skipbad'?

I was thinkig the old methods (--omitbad, --noskipbad, --skipbad) should remain
for the time being, with a warning to tell of their deprecation/removal in next
release.

Additionally, we could perhaps include a warning when nanddump is called
without an explicit BB handling option, alerting users that the default will be
changing to --bb=skipbad in the next release.

(4) Can Mike provide a good explanation for --bb=omitbad in the table below? I
personally don't understand it's exact use, nor do I know how to describe it
best (to provide contrast against the other options), but I understand that you
would like to keep the option. I would appreciate some help.

(5) Summary table:

-----------------------------------------------------------------------------------------------------------------
 Old option         New option                   Comment
-----------------------------------------------------------------------------------------------------------------
 <default>     =>   --bb=padbad                  dump flash data, substituting 0xFF for any bad blocks
 --noskipbad   =>   --bb=dumpbad                 dump flash data, including any bad blocks
 --skipbad     =>   --bb=skipbad, <default>      dump good data, completely skipping any bad blocks (new default)
 --omitbad     =>   --bb=omitbad                 (dump flash data, substituting nothing for any bad blocks?)

Thanks,
Brian

Re: nanddump badblock options

[Artem Bityutskiy]

Hi,

On Wed, 2011-06-08 at 13:01 -0700, Brian Norris wrote:
> I have some ideas to implement in nanddump regarding the variety of bad block
> handling options. I thought I'd at least get some feedback before working up a
> full patch, so please comment on my ideas.
> 
> (1) The comments in nandwrite say that nandwrite is an "inverse operation" to
> nanddump. However, take, for example, the following command:
> 
>    nandwrite --length=131072 /dev/mtd1 myfile
> 
> Then, if we consider that there may be bad blocks at the beginning of the
> device, nandwrite may skip to the second block in order to write this data.
> Now, the default behavior of nanddump does not at all fit the "inverse" of this
> very simple nandwrite operation. While you might expect the following to be an
> inverse:
> 
>    nanddump --length=131072 /dev/mtd1 --file=myfile.dump
> 
> you in fact will not get the same data that you wrote from the original file.
> Instead, you will get all 0xFF since by default nanddump substitutes 0xFF for
> all the data of the bad block. I call this (unwanted) behavior `padding'.
> 
> Thus, in short, I'm recommending that nanddump default to using --skipbad as
> a default option, with a new `padbad' option to cover the original behavior.
> Perhaps the "default" nanddump should have a warning over a period of time,
> before changing the default operation? See (3), Deprecation schedule.

Sounds good to me.

> (2) There are now (with my addition of `skipbad', and the current default
> `padbad') four methods used for handling bad blocks we come across when dumping
> flash data. I think they'd be cleaner if they were all grouped under a single
> option that would work something like:
> 
>    --bb=METHOD
> 
> where METHOD could be `padbad', `dumpbad', `skipbad', and `omitbad'. Notice the
> renaming of --noskipbad to --bb=dumpbad, since --noskipbad seems like an
> inverse to --skipbad, which it is not. See (5), Summary table.
> 
> I think eventually, we would just drop both the short and long options for the
> --omitbad, --noskipbad, and --skipbad options.
> 
> (3) Deprecation schedule:
> 
> Assuming the above is agreeable to everyone, how soon can we:
>    * drop the --noskipbad, --skipbad, --omitbad (pluse -b, -k, -N) flags in
>      favor of --bb=METHOD?
>    * change the default behavior from `padbad' to `skipbad'?

As soon as you implement this stuff and we push it, then one release
with warnings, next release we can remove that stuff. We already have
many changes, but I can wait for yours, then we release mtd-1.4.5, and
then we can kill the options next day.

> I was thinkig the old methods (--omitbad, --noskipbad, --skipbad) should remain
> for the time being, with a warning to tell of their deprecation/removal in next
> release.
> 
> Additionally, we could perhaps include a warning when nanddump is called
> without an explicit BB handling option, alerting users that the default will be
> changing to --bb=skipbad in the next release.

Yes.

> (4) Can Mike provide a good explanation for --bb=omitbad in the table below? I
> personally don't understand it's exact use, nor do I know how to describe it
> best (to provide contrast against the other options), but I understand that you
> would like to keep the option. I would appreciate some help.
> 
> (5) Summary table:
> 
> -----------------------------------------------------------------------------------------------------------------
>  Old option         New option                   Comment
> -----------------------------------------------------------------------------------------------------------------
>  <default>     =>   --bb=padbad                  dump flash data, substituting 0xFF for any bad blocks
>  --noskipbad   =>   --bb=dumpbad                 dump flash data, including any bad blocks
>  --skipbad     =>   --bb=skipbad, <default>      dump good data, completely skipping any bad blocks (new default)
>  --omitbad     =>   --bb=omitbad                 (dump flash data, substituting nothing for any bad blocks?)

Hmm...

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: nanddump badblock options

[Brian Norris]

One last idea, while I'm trying to bring nanddump and nandwrite a
little closer together:

It makes sense for nanddump and nandwrite to have the same defaults
regarding OOB. Right now, nandwrite doesn't write OOB by default, and
nanddump dumps OOB by default. Is it OK to unify this and make
nanddump default to *not* dumping OOB? Then the nanddump '-o' and
'--oob' options would stand for 'Dump OOB data'. Unfortunately, this
makes the '-o' option completely reverse its meaning...

I would include this with my other change-ups if that's alright
(they're ready to send).

Mike:
any comments on my '--omitbad' question?

Thanks,
Brian

Re: nanddump badblock options

[Mike Frysinger]

On Wed, Jun 8, 2011 at 16:01, Brian Norris wrote:
> I have some ideas to implement in nanddump regarding the variety of bad block
> handling options. I thought I'd at least get some feedback before working up a
> full patch, so please comment on my ideas.

your ideas sound good to me

> (4) Can Mike provide a good explanation for --bb=omitbad in the table below? I
> personally don't understand it's exact use, nor do I know how to describe it
> best (to provide contrast against the other options), but I understand that you
> would like to keep the option. I would appreciate some help.
>
> (5) Summary table:
>
> -----------------------------------------------------------------------------------------------------------------
>  Old option         New option                   Comment
> -----------------------------------------------------------------------------------------------------------------
>  <default>     =>   --bb=padbad                  dump flash data, substituting 0xFF for any bad blocks

maybe add a pad option so people can specify the data rather than just
0xff ?  although that's probably just overkill and no one would
actually use it ...

>  --noskipbad   =>   --bb=dumpbad                 dump flash data, including any bad blocks
>  --skipbad     =>   --bb=skipbad, <default>      dump good data, completely skipping any bad blocks (new default)
>  --omitbad     =>   --bb=omitbad                 (dump flash data, substituting nothing for any bad blocks?)

i think omitbad is covered by the other options.  the one i'm really
concerned in keeping is --bb=dumpbad for cases where the bbt says it's
bad, but it isnt really, so i want the data unmolested.
-mike

Re: nanddump badblock options

[Brian Norris]

On Mon, Jun 20, 2011 at 12:22 PM, Mike Frysinger <vapier.adi@gmail.com> wrote:
>> -----------------------------------------------------------------------------------------------------------------
>>  Old option         New option                   Comment
>> -----------------------------------------------------------------------------------------------------------------
>>  <default>     =>   --bb=padbad                  dump flash data, substituting 0xFF for any bad blocks
>
> maybe add a pad option so people can specify the data rather than just
> 0xff ?  although that's probably just overkill and no one would
> actually use it ...

Hmm, I'm thinking that'd be a little bit of overkill.

>>  --noskipbad   =>   --bb=dumpbad                 dump flash data, including any bad blocks
>>  --skipbad     =>   --bb=skipbad, <default>      dump good data, completely skipping any bad blocks (new default)
>>  --omitbad     =>   --bb=omitbad                 (dump flash data, substituting nothing for any bad blocks?)
>
> i think omitbad is covered by the other options.

omitbad isn't *exactly* covered by the other options, but it is very
close to the same as skipbad (recall the conversation when I
introduced --skipbad a couple of weeks ago). The only difference is
that omitbad ensures you dump data only from a specified "address
range", perhaps at the expense of the correct dump length, while
skipbad ensures the correct dump length, continuing past the
"specified range" if bad blocks are found.

I'm all for killing omitbad though, so I'll just respin my patch set
and send out v2...

> the one i'm really
> concerned in keeping is --bb=dumpbad for cases where the bbt says it's
> bad, but it isnt really, so i want the data unmolested.

Sure, that makes sense.

Also, I haven't heard any comments on my idea of changing the default
OOB options, where nanddump should NOT, by default, dump OOB data
(like nandwrite, which does NOT write OOB by default). I'm thinking we
could change the default here and add an option to force it to include
OOB:

--omitoob : do not dump OOB
--oob : dump OOB

So for the next release, I would implement the new --oob flag and add
appropriate warnings, and then following the release, we could change
the default. This leaves the question of the short option, -o, since
its current meaning (--omitoob) conflicts with the -o usage in
nandwrite (--oob). I'm thinking either:
1) Change -o so that it means "--oob" to match nandwrite (i.e.,
include OOB in dump)
2) Kill the short option -o entirely, since it's too confusing to
change it over to the nandwrite-compatible form.

This is mostly just a question of what is least confusing to the
mtd-utils user community; I can adapt to my own code changes!

Brian

Re: nanddump badblock options

[Mike Frysinger]

On Mon, Jun 20, 2011 at 19:09, Brian Norris wrote:
> Also, I haven't heard any comments on my idea of changing the default
> OOB options, where nanddump should NOT, by default, dump OOB data
> (like nandwrite, which does NOT write OOB by default). I'm thinking we
> could change the default here and add an option to force it to include
> OOB:

i think that makes sense

> This is mostly just a question of what is least confusing to the
> mtd-utils user community; I can adapt to my own code changes!

u-boot changed defaults similar to this and there werent really any
complaints.  i believe nandwrite/nanddump output info by default,
right ?  if we make it explicit what it's doing, people should be
fine.
-mike

Re: nanddump badblock options

[Artem Bityutskiy]

On Mon, 2011-06-13 at 12:17 -0700, Brian Norris wrote:
> One last idea, while I'm trying to bring nanddump and nandwrite a
> little closer together:
> 
> It makes sense for nanddump and nandwrite to have the same defaults
> regarding OOB. Right now, nandwrite doesn't write OOB by default, and
> nanddump dumps OOB by default. Is it OK to unify this and make
> nanddump default to *not* dumping OOB? Then the nanddump '-o' and
> '--oob' options would stand for 'Dump OOB data'. Unfortunately, this
> makes the '-o' option completely reverse its meaning...

Yes, I think so. Just invent a sensible "deprecation period" plan and go
ahead with unifying them.

Thanks!

-- 
Best Regards,
Artem Bityutskiy

Re: nanddump badblock options

[Brian Norris]

On Tue, Jun 21, 2011 at 9:42 PM, Artem Bityutskiy <dedekind1@gmail.com> wrote:
> Yes, I think so. Just invent a sensible "deprecation period" plan and go
> ahead with unifying them.
>
> Thanks!

OK, I sent my v3 patch set just now. I think I documented my plan well
enough in the patch set, so let me know how that looks.

Thanks,
Brian