UBI: badblock management confusion!

UBI: badblock management confusion!

[Nancy]

Hi all,
        I don't quite understand the UBI badblock management unit.
But I think there must be something wrong, or I take a wrong operation.
        Please follow my test below:

# ubiformat /dev/mtd5
ubiformat: mtd5 (NAND), size 536870912 bytes (512.0 MiB), 262144 eraseblocks of
262144 bytes (256.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 2047 -- 100 % complete
ubiformat: 2045 eraseblocks have valid erase counter, mean value is 23
ubiformat: bad eraseblocks: 10, 40, 84
ubiformat: formatting eraseblock 2047 -- 100 % complete

# modprobe ubi mtd=5
UBI: physical eraseblock size:   262144 bytes (256 KiB)
UBI: logical eraseblock size:    258048 bytes
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          2048 (aligned 2048)
UBI: data offset:                4096
UBI: attached mtd5 to ubi0
UBI: MTD device name:            "NAND VFAT partition"
UBI: MTD device size:            512 MiB
UBI: number of good PEBs:        2045
UBI: number of bad PEBs:         3
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    256
UBI: number of internal volumes: 1
UBI: number of user volumes:     0
UBI: available PEBs:             2021
UBI: total number of reserved PEBs: 24
UBI: number of PEBs reserved for bad PEB handling: 20
UBI: max/mean erase counter: 38/24
UBI: background thread "ubi_bgt0d" started, PID 233

***************************************************************************
I'm going to make one big volume which take all available space this ubi
device has.

I think the volume size should be equal to
(total number of PEB this ubi device has - total number of reserved PEBs) \
* LEB size
In this case, it should be : (2048-24) * 258048 bytes

But that is not what UBI count.
UBI think the volume size
= (number of good PEBs - total number of reserved PEBs) * LEB size

Q: Number of bad PEBs at initialization are not included in resered for bad
PEBs handling. That mean there should never has badblocks overflow issue
happen. What a good idea! But is there really a possible way to implement this
idea?
   Suppose there's only one volume take all the ubi device space, there's no
badblocks at the very beginning. The user does never shut down the device but
hibernate for a long time. Then, suppose there is 3 badblocks and the volume
just full of data. No space left. Then reboot the device, beside the 3 badblocks
which used to be count in reserved PEBs, now be seperate from the reserved PEBs,
But there's no space, reserved PEB will fail.

   The following test showes even the volume data is blank, there still comes
problem.
   Is it an UBI bug or my wrong operation?
   Thanks for your notice about it!
****************************************************************************

# ubimkvol /dev/ubi0 -n 0 -N vfat -s 509292KiB
Volume ID 0, size 2021 LEBs (521515008 bytes, 497.4 MiB), LEB size
258048 bytes (252.0 KiB), dynamic, name "vfat", alignment 1
# rmmod ubi
UBI: mtd5 is detached from ubi0

*****************************************************************************
"markbadblock" is a tool I force to mark a block bad manually, just for test
*****************************************************************************
# markbadblock -l 17 /dev/mtd5
Block size 262144, page size 2048, OOB size 64
mark block 17 to be badblock done

# modprobe ubi mtd=5
UBI: physical eraseblock size:   262144 bytes (256 KiB)
UBI: logical eraseblock size:    258048 bytes
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          2048 (aligned 2048)
UBI: data offset:                4096
UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB
handling, reserved 19, need 20
UBI: attached mtd5 to ubi0
UBI: MTD device name:            "NAND VFAT partition"
UBI: MTD device size:            512 MiB
UBI: number of good PEBs:        2044
UBI: number of bad PEBs:         4
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    256
UBI: number of internal volumes: 1
UBI: number of user volumes:     1
UBI: available PEBs:             0
UBI: total number of reserved PEBs: 2044
UBI: number of PEBs reserved for bad PEB handling: 19
UBI: max/mean erase counter: 39/24
UBI: background thread "ubi_bgt0d" started, PID 263
# modprobe ubifs

# mount -t ubifs ubi0:ubifs /mnt/1
UBIFS error (pid 293): ubifs_get_sb: cannot open "ubi0:ubifs", error -19
mount: mounting ubi0:ubifs on /mnt/1 failed: No such device
# rmmod ubifs
# rmmod ubi

# ubiformat /dev/mtd5
ubiformat: mtd5 (NAND), size 536870912 bytes (512.0 MiB), 262144 eraseblocks of
262144 bytes (256.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 2047 -- 100 % complete
ubiformat: 2044 eraseblocks have valid erase counter, mean value is 24
ubiformat: bad eraseblocks: 10, 17, 40, 84
ubiformat: formatting eraseblock 2047 -- 100 % complete

-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Adrian Hunter]

ext Nancy wrote:
> Hi all,
>         I don't quite understand the UBI badblock management unit.
> But I think there must be something wrong, or I take a wrong operation.
>         Please follow my test below:
> 
> # ubiformat /dev/mtd5
> ubiformat: mtd5 (NAND), size 536870912 bytes (512.0 MiB), 262144 eraseblocks of
> 262144 bytes (256.0 KiB), min. I/O size 2048 bytes
> libscan: scanning eraseblock 2047 -- 100 % complete
> ubiformat: 2045 eraseblocks have valid erase counter, mean value is 23
> ubiformat: bad eraseblocks: 10, 40, 84
> ubiformat: formatting eraseblock 2047 -- 100 % complete
> 
> # modprobe ubi mtd=5
> UBI: physical eraseblock size:   262144 bytes (256 KiB)
> UBI: logical eraseblock size:    258048 bytes
> UBI: smallest flash I/O unit:    2048
> UBI: VID header offset:          2048 (aligned 2048)
> UBI: data offset:                4096
> UBI: attached mtd5 to ubi0
> UBI: MTD device name:            "NAND VFAT partition"
> UBI: MTD device size:            512 MiB
> UBI: number of good PEBs:        2045
> UBI: number of bad PEBs:         3
> UBI: max. allowed volumes:       128
> UBI: wear-leveling threshold:    256
> UBI: number of internal volumes: 1
> UBI: number of user volumes:     0
> UBI: available PEBs:             2021
> UBI: total number of reserved PEBs: 24
> UBI: number of PEBs reserved for bad PEB handling: 20
> UBI: max/mean erase counter: 38/24
> UBI: background thread "ubi_bgt0d" started, PID 233
> 
> ***************************************************************************
> I'm going to make one big volume which take all available space this ubi
> device has.
> 
> I think the volume size should be equal to
> (total number of PEB this ubi device has - total number of reserved PEBs) \
> * LEB size
> In this case, it should be : (2048-24) * 258048 bytes
> 
> But that is not what UBI count.
> UBI think the volume size
> = (number of good PEBs - total number of reserved PEBs) * LEB size
> 
> Q: Number of bad PEBs at initialization are not included in resered for bad
> PEBs handling. That mean there should never has badblocks overflow issue
> happen. What a good idea! But is there really a possible way to implement this
> idea?

It does work.

>    Suppose there's only one volume take all the ubi device space, there's no
> badblocks at the very beginning. The user does never shut down the device but
> hibernate for a long time. Then, suppose there is 3 badblocks and the volume
> just full of data. No space left. Then reboot the device, beside the 3 badblocks
> which used to be count in reserved PEBs, now be seperate from the reserved PEBs,
> But there's no space, reserved PEB will fail.

No, there will just be a warning - as shown in your test:

	UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB handling, reserved 19, need 20

There *would* be problem if you marked 21 bad and only 20 were reserved for bad block handling.

> 
>    The following test showes even the volume data is blank, there still comes
> problem.
>    Is it an UBI bug or my wrong operation?

Your wrong operation.

>    Thanks for your notice about it!
> ****************************************************************************
> 
> # ubimkvol /dev/ubi0 -n 0 -N vfat -s 509292KiB
> Volume ID 0, size 2021 LEBs (521515008 bytes, 497.4 MiB), LEB size
> 258048 bytes (252.0 KiB), dynamic, name "vfat", alignment 1
> # rmmod ubi
> UBI: mtd5 is detached from ubi0
> 
> *****************************************************************************
> "markbadblock" is a tool I force to mark a block bad manually, just for test
> *****************************************************************************
> # markbadblock -l 17 /dev/mtd5
> Block size 262144, page size 2048, OOB size 64
> mark block 17 to be badblock done
> 
> # modprobe ubi mtd=5
> UBI: physical eraseblock size:   262144 bytes (256 KiB)
> UBI: logical eraseblock size:    258048 bytes
> UBI: smallest flash I/O unit:    2048
> UBI: VID header offset:          2048 (aligned 2048)
> UBI: data offset:                4096
> UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB
> handling, reserved 19, need 20
> UBI: attached mtd5 to ubi0
> UBI: MTD device name:            "NAND VFAT partition"
> UBI: MTD device size:            512 MiB
> UBI: number of good PEBs:        2044
> UBI: number of bad PEBs:         4
> UBI: max. allowed volumes:       128
> UBI: wear-leveling threshold:    256
> UBI: number of internal volumes: 1
> UBI: number of user volumes:     1
> UBI: available PEBs:             0
> UBI: total number of reserved PEBs: 2044
> UBI: number of PEBs reserved for bad PEB handling: 19
> UBI: max/mean erase counter: 39/24
> UBI: background thread "ubi_bgt0d" started, PID 263
> # modprobe ubifs
> 
> # mount -t ubifs ubi0:ubifs /mnt/1
> UBIFS error (pid 293): ubifs_get_sb: cannot open "ubi0:ubifs", error -19

You named the volume "vfat" (-N option of ubimkvol), but you are trying
to mount a volume named "ubifs" i.e. should be:

	mount -t ubifs ubi0:vfat /mnt/1

> mount: mounting ubi0:ubifs on /mnt/1 failed: No such device
> # rmmod ubifs
> # rmmod ubi
> 
> # ubiformat /dev/mtd5
> ubiformat: mtd5 (NAND), size 536870912 bytes (512.0 MiB), 262144 eraseblocks of
> 262144 bytes (256.0 KiB), min. I/O size 2048 bytes
> libscan: scanning eraseblock 2047 -- 100 % complete
> ubiformat: 2044 eraseblocks have valid erase counter, mean value is 24
> ubiformat: bad eraseblocks: 10, 17, 40, 84
> ubiformat: formatting eraseblock 2047 -- 100 % complete
> 

Re: UBI: badblock management confusion!

[Nancy]

On Fri, Apr 25, 2008 at 6:47 PM, Adrian Hunter
<ext-adrian.hunter@nokia.com> wrote:
>
> ext Nancy wrote:
> > Hi all,
> >        I don't quite understand the UBI badblock management unit.
> > But I think there must be something wrong, or I take a wrong operation.
> >        Please follow my test below:
> >
> > # ubiformat /dev/mtd5
> > ubiformat: mtd5 (NAND), size 536870912 bytes (512.0 MiB), 262144
> eraseblocks of
> > 262144 bytes (256.0 KiB), min. I/O size 2048 bytes
> > libscan: scanning eraseblock 2047 -- 100 % complete
> > ubiformat: 2045 eraseblocks have valid erase counter, mean value is 23
> > ubiformat: bad eraseblocks: 10, 40, 84
> > ubiformat: formatting eraseblock 2047 -- 100 % complete
> >
> > # modprobe ubi mtd=5
> > UBI: physical eraseblock size:   262144 bytes (256 KiB)
> > UBI: logical eraseblock size:    258048 bytes
> > UBI: smallest flash I/O unit:    2048
> > UBI: VID header offset:          2048 (aligned 2048)
> > UBI: data offset:                4096
> > UBI: attached mtd5 to ubi0
> > UBI: MTD device name:            "NAND VFAT partition"
> > UBI: MTD device size:            512 MiB
> > UBI: number of good PEBs:        2045
> > UBI: number of bad PEBs:         3
> > UBI: max. allowed volumes:       128
> > UBI: wear-leveling threshold:    256
> > UBI: number of internal volumes: 1
> > UBI: number of user volumes:     0
> > UBI: available PEBs:             2021
> > UBI: total number of reserved PEBs: 24
> > UBI: number of PEBs reserved for bad PEB handling: 20
> > UBI: max/mean erase counter: 38/24
> > UBI: background thread "ubi_bgt0d" started, PID 233
> >
> >
> ***************************************************************************
> > I'm going to make one big volume which take all available space this ubi
> > device has.
> >
> > I think the volume size should be equal to
> > (total number of PEB this ubi device has - total number of reserved PEBs)
> \
> > * LEB size
> > In this case, it should be : (2048-24) * 258048 bytes
> >
> > But that is not what UBI count.
> > UBI think the volume size
> > = (number of good PEBs - total number of reserved PEBs) * LEB size
> >
> > Q: Number of bad PEBs at initialization are not included in resered for
> bad
> > PEBs handling. That mean there should never has badblocks overflow issue
> > happen. What a good idea! But is there really a possible way to implement
> this
> > idea?
> >
>
> It does work.
>
>
> >   Suppose there's only one volume take all the ubi device space, there's
> no
> > badblocks at the very beginning. The user does never shut down the device
> but
> > hibernate for a long time. Then, suppose there is 3 badblocks and the
> volume
> > just full of data. No space left. Then reboot the device, beside the 3
> badblocks
> > which used to be count in reserved PEBs, now be seperate from the reserved
> PEBs,
> > But there's no space, reserved PEB will fail.
> >
>
> No, there will just be a warning - as shown in your test:
>
>
>        UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad
> PEB handling, reserved 19, need 20
>
> There *would* be problem if you marked 21 bad and only 20 were reserved for
> bad block handling.
>
>
> >
> >   The following test showes even the volume data is blank, there still
> comes
> > problem.
> >   Is it an UBI bug or my wrong operation?
> >
>
> Your wrong operation.
>
>
>
> >   Thanks for your notice about it!
> >
> ****************************************************************************
> >
> > # ubimkvol /dev/ubi0 -n 0 -N vfat -s 509292KiB
> > Volume ID 0, size 2021 LEBs (521515008 bytes, 497.4 MiB), LEB size
> > 258048 bytes (252.0 KiB), dynamic, name "vfat", alignment 1
> > # rmmod ubi
> > UBI: mtd5 is detached from ubi0
> >
> >
> *****************************************************************************
> > "markbadblock" is a tool I force to mark a block bad manually, just for
> test
> >
> *****************************************************************************
> > # markbadblock -l 17 /dev/mtd5
> > Block size 262144, page size 2048, OOB size 64
> > mark block 17 to be badblock done
> >
> > # modprobe ubi mtd=5
> > UBI: physical eraseblock size:   262144 bytes (256 KiB)
> > UBI: logical eraseblock size:    258048 bytes
> > UBI: smallest flash I/O unit:    2048
> > UBI: VID header offset:          2048 (aligned 2048)
> > UBI: data offset:                4096
> > UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB
> > handling, reserved 19, need 20
> > UBI: attached mtd5 to ubi0
> > UBI: MTD device name:            "NAND VFAT partition"
> > UBI: MTD device size:            512 MiB
> > UBI: number of good PEBs:        2044
> > UBI: number of bad PEBs:         4
> > UBI: max. allowed volumes:       128
> > UBI: wear-leveling threshold:    256
> > UBI: number of internal volumes: 1
> > UBI: number of user volumes:     1
> > UBI: available PEBs:             0
> > UBI: total number of reserved PEBs: 2044
> > UBI: number of PEBs reserved for bad PEB handling: 19
> > UBI: max/mean erase counter: 39/24
> > UBI: background thread "ubi_bgt0d" started, PID 263
> > # modprobe ubifs
> >
> > # mount -t ubifs ubi0:ubifs /mnt/1
> > UBIFS error (pid 293): ubifs_get_sb: cannot open "ubi0:ubifs", error -19
> >
>
> You named the volume "vfat" (-N option of ubimkvol), but you are trying
> to mount a volume named "ubifs" i.e. should be:
>
>        mount -t ubifs ubi0:vfat /mnt/1

Oh, thank you for ponting out my mistake!
You mean UBI admit the amount of current badblocks are part of
reserved PEBs, right?
If the answer is yes, then the volume size I'm goting to create should
be equal to

(total number of PEB this ubi device has - total number of reserved
PEBs) * LEB size

not what UBI think:
(number of good PEBs - total number of reserved PEBs) * LEB size

Am I right?


-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Nancy]

 > > # modprobe ubi mtd=5
 > > UBI: physical eraseblock size:   262144 bytes (256 KiB)
 > > UBI: logical eraseblock size:    258048 bytes
 > > UBI: smallest flash I/O unit:    2048
 > > UBI: VID header offset:          2048 (aligned 2048)
 > > UBI: data offset:                4096
 > > UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB
 > > handling, reserved 19, need 20
 > > UBI: attached mtd5 to ubi0
 > > UBI: MTD device name:            "NAND VFAT partition"
 > > UBI: MTD device size:            512 MiB
 > > UBI: number of good PEBs:        2044
 > > UBI: number of bad PEBs:         4
 > > UBI: max. allowed volumes:       128
 > > UBI: wear-leveling threshold:    256
 > > UBI: number of internal volumes: 1
 > > UBI: number of user volumes:     1
 > > UBI: available PEBs:             0
 > > UBI: total number of reserved PEBs: 2044
 > > UBI: number of PEBs reserved for bad PEB handling: 19
 > > UBI: max/mean erase counter: 39/24
 > > UBI: background thread "ubi_bgt0d" started, PID 263

pls:
 > > UBI: available PEBs:             0
 > > UBI: total number of reserved PEBs: 2044
Quite wrong displaying messages.


-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Adrian Hunter]

ext Nancy wrote:
> On Fri, Apr 25, 2008 at 6:47 PM, Adrian Hunter
> <ext-adrian.hunter@nokia.com> wrote:
>> ext Nancy wrote:
>>> Hi all,
>>>        I don't quite understand the UBI badblock management unit.
>>> But I think there must be something wrong, or I take a wrong operation.
>>>        Please follow my test below:
>>>
>>> # ubiformat /dev/mtd5
>>> ubiformat: mtd5 (NAND), size 536870912 bytes (512.0 MiB), 262144
>> eraseblocks of
>>> 262144 bytes (256.0 KiB), min. I/O size 2048 bytes
>>> libscan: scanning eraseblock 2047 -- 100 % complete
>>> ubiformat: 2045 eraseblocks have valid erase counter, mean value is 23
>>> ubiformat: bad eraseblocks: 10, 40, 84
>>> ubiformat: formatting eraseblock 2047 -- 100 % complete
>>>
>>> # modprobe ubi mtd=5
>>> UBI: physical eraseblock size:   262144 bytes (256 KiB)
>>> UBI: logical eraseblock size:    258048 bytes
>>> UBI: smallest flash I/O unit:    2048
>>> UBI: VID header offset:          2048 (aligned 2048)
>>> UBI: data offset:                4096
>>> UBI: attached mtd5 to ubi0
>>> UBI: MTD device name:            "NAND VFAT partition"
>>> UBI: MTD device size:            512 MiB
>>> UBI: number of good PEBs:        2045
>>> UBI: number of bad PEBs:         3
>>> UBI: max. allowed volumes:       128
>>> UBI: wear-leveling threshold:    256
>>> UBI: number of internal volumes: 1
>>> UBI: number of user volumes:     0
>>> UBI: available PEBs:             2021
>>> UBI: total number of reserved PEBs: 24
>>> UBI: number of PEBs reserved for bad PEB handling: 20
>>> UBI: max/mean erase counter: 38/24
>>> UBI: background thread "ubi_bgt0d" started, PID 233
>>>
>>>
>> ***************************************************************************
>>> I'm going to make one big volume which take all available space this ubi
>>> device has.
>>>
>>> I think the volume size should be equal to
>>> (total number of PEB this ubi device has - total number of reserved PEBs)
>> \
>>> * LEB size
>>> In this case, it should be : (2048-24) * 258048 bytes
>>>
>>> But that is not what UBI count.
>>> UBI think the volume size
>>> = (number of good PEBs - total number of reserved PEBs) * LEB size
>>>
>>> Q: Number of bad PEBs at initialization are not included in resered for
>> bad
>>> PEBs handling. That mean there should never has badblocks overflow issue
>>> happen. What a good idea! But is there really a possible way to implement
>> this
>>> idea?
>>>
>> It does work.
>>
>>
>>>   Suppose there's only one volume take all the ubi device space, there's
>> no
>>> badblocks at the very beginning. The user does never shut down the device
>> but
>>> hibernate for a long time. Then, suppose there is 3 badblocks and the
>> volume
>>> just full of data. No space left. Then reboot the device, beside the 3
>> badblocks
>>> which used to be count in reserved PEBs, now be seperate from the reserved
>> PEBs,
>>> But there's no space, reserved PEB will fail.
>>>
>> No, there will just be a warning - as shown in your test:
>>
>>
>>        UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad
>> PEB handling, reserved 19, need 20
>>
>> There *would* be problem if you marked 21 bad and only 20 were reserved for
>> bad block handling.
>>
>>
>>>   The following test showes even the volume data is blank, there still
>> comes
>>> problem.
>>>   Is it an UBI bug or my wrong operation?
>>>
>> Your wrong operation.
>>
>>
>>
>>>   Thanks for your notice about it!
>>>
>> ****************************************************************************
>>> # ubimkvol /dev/ubi0 -n 0 -N vfat -s 509292KiB
>>> Volume ID 0, size 2021 LEBs (521515008 bytes, 497.4 MiB), LEB size
>>> 258048 bytes (252.0 KiB), dynamic, name "vfat", alignment 1
>>> # rmmod ubi
>>> UBI: mtd5 is detached from ubi0
>>>
>>>
>> *****************************************************************************
>>> "markbadblock" is a tool I force to mark a block bad manually, just for
>> test
>> *****************************************************************************
>>> # markbadblock -l 17 /dev/mtd5
>>> Block size 262144, page size 2048, OOB size 64
>>> mark block 17 to be badblock done
>>>
>>> # modprobe ubi mtd=5
>>> UBI: physical eraseblock size:   262144 bytes (256 KiB)
>>> UBI: logical eraseblock size:    258048 bytes
>>> UBI: smallest flash I/O unit:    2048
>>> UBI: VID header offset:          2048 (aligned 2048)
>>> UBI: data offset:                4096
>>> UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB
>>> handling, reserved 19, need 20
>>> UBI: attached mtd5 to ubi0
>>> UBI: MTD device name:            "NAND VFAT partition"
>>> UBI: MTD device size:            512 MiB
>>> UBI: number of good PEBs:        2044
>>> UBI: number of bad PEBs:         4
>>> UBI: max. allowed volumes:       128
>>> UBI: wear-leveling threshold:    256
>>> UBI: number of internal volumes: 1
>>> UBI: number of user volumes:     1
>>> UBI: available PEBs:             0
>>> UBI: total number of reserved PEBs: 2044
>>> UBI: number of PEBs reserved for bad PEB handling: 19
>>> UBI: max/mean erase counter: 39/24
>>> UBI: background thread "ubi_bgt0d" started, PID 263
>>> # modprobe ubifs
>>>
>>> # mount -t ubifs ubi0:ubifs /mnt/1
>>> UBIFS error (pid 293): ubifs_get_sb: cannot open "ubi0:ubifs", error -19
>>>
>> You named the volume "vfat" (-N option of ubimkvol), but you are trying
>> to mount a volume named "ubifs" i.e. should be:
>>
>>        mount -t ubifs ubi0:vfat /mnt/1
> 
> Oh, thank you for ponting out my mistake!
> You mean UBI admit the amount of current badblocks are part of
> reserved PEBs, right?

Not exactly.  It is just that UBI does not mind if there are not enough
PEBs to meet the number desired for bad-block handling.

> If the answer is yes, then the volume size I'm goting to create should
> be equal to
> 
> (total number of PEB this ubi device has - total number of reserved
> PEBs) * LEB size
> 
> not what UBI think:
> (number of good PEBs - total number of reserved PEBs) * LEB size
> 
> Am I right?

I am not sure what you are trying to achieve.  You can always use
the -m option of ubimkvol to set volume size to maximum available size.

Re: UBI: badblock management confusion!

[Adrian Hunter]

ext Nancy wrote:
>  > > # modprobe ubi mtd=5
>  > > UBI: physical eraseblock size:   262144 bytes (256 KiB)
>  > > UBI: logical eraseblock size:    258048 bytes
>  > > UBI: smallest flash I/O unit:    2048
>  > > UBI: VID header offset:          2048 (aligned 2048)
>  > > UBI: data offset:                4096
>  > > UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB
>  > > handling, reserved 19, need 20
>  > > UBI: attached mtd5 to ubi0
>  > > UBI: MTD device name:            "NAND VFAT partition"
>  > > UBI: MTD device size:            512 MiB
>  > > UBI: number of good PEBs:        2044
>  > > UBI: number of bad PEBs:         4
>  > > UBI: max. allowed volumes:       128
>  > > UBI: wear-leveling threshold:    256
>  > > UBI: number of internal volumes: 1
>  > > UBI: number of user volumes:     1
>  > > UBI: available PEBs:             0
>  > > UBI: total number of reserved PEBs: 2044
>  > > UBI: number of PEBs reserved for bad PEB handling: 19
>  > > UBI: max/mean erase counter: 39/24
>  > > UBI: background thread "ubi_bgt0d" started, PID 263
> 
> pls:
>  > > UBI: available PEBs:             0
>  > > UBI: total number of reserved PEBs: 2044
> Quite wrong displaying messages.
> 
> 

There are no available PEBs so no more volumes can be created.

The number of reserved PEBs includes all existing volumes.

Re: UBI: badblock management confusion!

[Nancy]

On 4/28/08, Adrian Hunter <ext-adrian.hunter@nokia.com> wrote:
> ext Nancy wrote:
> >  > > # modprobe ubi mtd=5
> >  > > UBI: physical eraseblock size:   262144 bytes (256 KiB)
> >  > > UBI: logical eraseblock size:    258048 bytes
> >  > > UBI: smallest flash I/O unit:    2048
> >  > > UBI: VID header offset:          2048 (aligned 2048)
> >  > > UBI: data offset:                4096
> >  > > UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad
> PEB
> >  > > handling, reserved 19, need 20
> >  > > UBI: attached mtd5 to ubi0
> >  > > UBI: MTD device name:            "NAND VFAT partition"
> >  > > UBI: MTD device size:            512 MiB
> >  > > UBI: number of good PEBs:        2044
> >  > > UBI: number of bad PEBs:         4
> >  > > UBI: max. allowed volumes:       128
> >  > > UBI: wear-leveling threshold:    256
> >  > > UBI: number of internal volumes: 1
> >  > > UBI: number of user volumes:     1
> >  > > UBI: available PEBs:             0
> >  > > UBI: total number of reserved PEBs: 2044
> >  > > UBI: number of PEBs reserved for bad PEB handling: 19
> >  > > UBI: max/mean erase counter: 39/24
> >  > > UBI: background thread "ubi_bgt0d" started, PID 263
> >
> > pls:
> >  > > UBI: available PEBs:             0
> >  > > UBI: total number of reserved PEBs: 2044
> > Quite wrong displaying messages.
> >
> >
> >
>
> There are no available PEBs so no more volumes can be created.
>
> The number of reserved PEBs includes all existing volumes.

I thought available PEBs is the PEBs which only contain erase counter.
total number of reserved PEBs contains only the ones upper layter can't see.

Thank you for your explaination!

-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Nancy]

> > Oh, thank you for ponting out my mistake!
> > You mean UBI admit the amount of current badblocks are part of
> > reserved PEBs, right?
> >
>
> Not exactly.  It is just that UBI does not mind if there are not enough
> PEBs to meet the number desired for bad-block handling.
>
> > If the answer is yes, then the volume size I'm goting to create should
> > be equal to
> >
> > (total number of PEB this ubi device has - total number of reserved
> > PEBs) * LEB size
> >
> > not what UBI think:
> > (number of good PEBs - total number of reserved PEBs) * LEB size
> >
> > Am I right?
> >
>
> I am not sure what you are trying to achieve.  You can always use
> the -m option of ubimkvol to set volume size to maximum available size.
>
Let me clarify the problem I met:
	there are so many Nand flash for burning. Each one of them may have
different numbers of badblocks, but that number do not exceed the reserved
badblock number. Each one of them should have the same maximum avaliable
volume size. But the practice tell no, the maximum avaliable volume size
effect by number of badblocks the current MTD partition has. That's wrong.
You can try by yourself.

-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Adrian Hunter]

ext Nancy wrote:
>>> Oh, thank you for ponting out my mistake!
>>> You mean UBI admit the amount of current badblocks are part of
>>> reserved PEBs, right?
>>>
>> Not exactly.  It is just that UBI does not mind if there are not enough
>> PEBs to meet the number desired for bad-block handling.
>>
>>> If the answer is yes, then the volume size I'm goting to create should
>>> be equal to
>>>
>>> (total number of PEB this ubi device has - total number of reserved
>>> PEBs) * LEB size
>>>
>>> not what UBI think:
>>> (number of good PEBs - total number of reserved PEBs) * LEB size
>>>
>>> Am I right?
>>>
>> I am not sure what you are trying to achieve.  You can always use
>> the -m option of ubimkvol to set volume size to maximum available size.
>>
> Let me clarify the problem I met:
> 	there are so many Nand flash for burning. Each one of them may have
> different numbers of badblocks, but that number do not exceed the reserved
> badblock number. Each one of them should have the same maximum avaliable
> volume size. But the practice tell no, the maximum avaliable volume size
> effect by number of badblocks the current MTD partition has. That's wrong.
> You can try by yourself.
> 

The following section from UBI documentation may help you:

	http://linux-mtd.infradead.org/doc/ubi.html#L_autoresize

Let us know if it doesn't meet your needs.

Re: UBI: badblock management confusion!

[Nancy]

> > Let me clarify the problem I met:
> >        there are so many Nand flash for burning. Each one of them may have
> > different numbers of badblocks, but that number do not exceed the reserved
> > badblock number. Each one of them should have the same maximum avaliable
> > volume size. But the practice tell no, the maximum avaliable volume size
> > effect by number of badblocks the current MTD partition has. That's wrong.
> > You can try by yourself.
> >
> >
>
> The following section from UBI documentation may help you:
>
>
> http://linux-mtd.infradead.org/doc/ubi.html#L_autoresize
>
> Let us know if it doesn't meet your needs.

Sorry, my bad english, I can't get what autoresize for.
I just want to set every Nand same! Just like I defined my MTD
partitions. Each product has same partitions table.
same reserved badblock number.
same user space useable blocks.
Is that requirement abnormal?



-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Adrian Hunter]

Nancy wrote:
>>> Let me clarify the problem I met:
>>>        there are so many Nand flash for burning. Each one of them may have
>>> different numbers of badblocks, but that number do not exceed the reserved
>>> badblock number. Each one of them should have the same maximum avaliable
>>> volume size. But the practice tell no, the maximum avaliable volume size
>>> effect by number of badblocks the current MTD partition has. That's wrong.
>>> You can try by yourself.
>>>
>>>
>> The following section from UBI documentation may help you:
>>
>>
>> http://linux-mtd.infradead.org/doc/ubi.html#L_autoresize
>>
>> Let us know if it doesn't meet your needs.
> 
> Sorry, my bad english, I can't get what autoresize for.
> I just want to set every Nand same! Just like I defined my MTD
> partitions. Each product has same partitions table.
> same reserved badblock number.
> same user space useable blocks.
> Is that requirement abnormal?

Well, the number of initial bad blocks does not affect how many will become
bad over time.  So it is not part of the calculation.  For example, if you
have 2048 PEBs and 17 are bad initially and you want to reserve 1% for (more)
bad blocks then the number reserved is (2048 - 17) / 100 = 20.  That makes the
default maximum volume size 2048 - 17 - 20 = 2011.  Whereas, if you are lucky
and there are no initial bad blocks, then the number reserved for bad block
handling is 2048 / 100 = 20, and the default maximum volume size is
2048 - 20 = 2028.

Just because 17 are initially bad in the first case does not mean only 3 more
will become bad in the time it takes for 20 to become bad in the second case.

So 20 are reserved for bad block handling in both cases.

And that means that the maximum volume size varies.

What we do here is we don't have the same partition table.  We fix the size of
every partition except the last and let the start of each partition be adjusted
if there are bad blocks.  For the last partition, it ends up being smaller by the
total number of bad blocks.  We then use the "autoresize" of UBI to automatically
fit the volume size of the last volume to the partition size.  Since our last
volume is UBIFS, there is no problem because UBIFS also automatically adjusts
(so long as the volume size is larger that the flashed image created by
mkfs.ubifs).

If you want to have the same volume size, then you can but you must make the
size smaller than the maximum.  You will need to consult your documentation
about what the maximum number of initial bad blocks is.  For example, for
OneNAND with 2048 PEBs, when a OneNAND comes from the factory it may have
as many as 40 PEBs that are bad.  So 2048 - 40 = 2008 good PEBs minimum.
2008 / 100 = 20 PEBs reserved for (more) bad blocks (assuming you use the 1%
setting).  2008 - 20 = 1988 PEBs for the volume size.  The exact volume size is
then 1988 * LEB size = 1988 * 258048 (in your case) = 512999424.

Re: UBI: badblock management confusion!

[Nancy]

> Well, the number of initial bad blocks does not affect how many will become
> bad over time.  So it is not part of the calculation.  For example, if you
> have 2048 PEBs and 17 are bad initially and you want to reserve 1% for
> (more)
> bad blocks then the number reserved is (2048 - 17) / 100 = 20.  That makes
> the
> default maximum volume size 2048 - 17 - 20 = 2011.  Whereas, if you are
> lucky
> and there are no initial bad blocks, then the number reserved for bad block
> handling is 2048 / 100 = 20, and the default maximum volume size is
> 2048 - 20 = 2028.
>
> Just because 17 are initially bad in the first case does not mean only 3
> more
> will become bad in the time it takes for 20 to become bad in the second
> case.
>
> So 20 are reserved for bad block handling in both cases.
No, your implement says the initial bad blocks are part of the
reserved badblocks. As you figure out
     UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for
bad PEB handling, reserved 19, need 20
just Warning, it need reserved 20, but found there's one more badblock
comes out during the long using time, so reserved only 19.
Same code, same operation. Can you tell me how UBI figure out when is
the real initial time? I think it has no way to know, it have to admit
each attache MTD partition time is initial time. You said initial
time's badblocks Nand has are not part of reserved badblocks, but the
practice tell it is part of reserved badblock PEBs( the warning is a
prof). If it is part of reserved badblocks, then why the max volume
size have to - current badblocks (it is already in reserved
badblocks). That is a waist. That is what I confused about!


> And that means that the maximum volume size varies.
>
> What we do here is we don't have the same partition table.  We fix the size
> of
> every partition except the last and let the start of each partition be
> adjusted
> if there are bad blocks.  For the last partition, it ends up being smaller
> by the
> total number of bad blocks.  We then use the "autoresize" of UBI to
> automatically
> fit the volume size of the last volume to the partition size.  Since our
> last
> volume is UBIFS, there is no problem because UBIFS also automatically
> adjusts
> (so long as the volume size is larger that the flashed image created by
> mkfs.ubifs).
Oh, too complicated! I don't quite understand.
en...
fix size of every partition except the last!  if last partitions is
full of data, then it can't give any free PEBs to other volumes right?
or you just keep last volume blank, do not use it, like a space
storehouse for other volume's need, if like that, why not simply
reserved the last volume size.


> If you want to have the same volume size, then you can but you must make the
> size smaller than the maximum.  You will need to consult your documentation
> about what the maximum number of initial bad blocks is.  For example, for
> OneNAND with 2048 PEBs, when a OneNAND comes from the factory it may have
> as many as 40 PEBs that are bad.  So 2048 - 40 = 2008 good PEBs minimum.
> 2008 / 100 = 20 PEBs reserved for (more) bad blocks (assuming you use the 1%
> setting).  2008 - 20 = 1988 PEBs for the volume size.  The exact volume size
> is
> then 1988 * LEB size = 1988 * 258048 (in your case) = 512999424.

Compare with reserved badblocks(including the init and future go bad)
by a percentage UBI user can configure by wish. What good autoresize
has?


-- 
Best wishes,
Nancy

Re: UBI: badblock management confusion!

[Adrian Hunter]

Nancy wrote:
>> Well, the number of initial bad blocks does not affect how many will become
>> bad over time.  So it is not part of the calculation.  For example, if you
>> have 2048 PEBs and 17 are bad initially and you want to reserve 1% for
>> (more)
>> bad blocks then the number reserved is (2048 - 17) / 100 = 20.  That makes
>> the
>> default maximum volume size 2048 - 17 - 20 = 2011.  Whereas, if you are
>> lucky
>> and there are no initial bad blocks, then the number reserved for bad block
>> handling is 2048 / 100 = 20, and the default maximum volume size is
>> 2048 - 20 = 2028.
>>
>> Just because 17 are initially bad in the first case does not mean only 3
>> more
>> will become bad in the time it takes for 20 to become bad in the second
>> case.
>>
>> So 20 are reserved for bad block handling in both cases.
> No, your implement says the initial bad blocks are part of the
> reserved badblocks. As you figure out
>      UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for
> bad PEB handling, reserved 19, need 20
> just Warning, it need reserved 20, but found there's one more badblock
> comes out during the long using time, so reserved only 19.
> Same code, same operation. Can you tell me how UBI figure out when is
> the real initial time?

Initial bad blocks is the number of bad blocks when the flash comes from
the factory.

> I think it has no way to know, it have to admit
> each attache MTD partition time is initial time. You said initial
> time's badblocks Nand has are not part of reserved badblocks, but the
> practice tell it is part of reserved badblock PEBs( the warning is a
> prof). If it is part of reserved badblocks, then why the max volume
> size have to - current badblocks (it is already in reserved
> badblocks). That is a waist. That is what I confused about!

There are two numbers and they both depend on the number of bad blocks.

The two numbers are:

	beb_rsvd_pebs
	beb_rsvd_level

These numbers are not written to the flash, they are calculated as follows:

beb_rsvd_level is the number of good PEBs multiplied by the bad block
handling percentage.

beb_rsvd_pebs is the same as beb_rsvd_level unless there are not enough
PEBs.  That is, if the sum of the volume sizes plus beb_rsvd_pebs is
greater than the number of good PEBs, then beb_rsvd_pebs is reduced.

So the number of PEBs reserved for bad block handling depends on the
volume size(s) and the number of bad blocks.

When a new volume is created there must be enough PEBs for it plus the
minimum bad block handling reserve (which is 2 PEBs minimum).

When a new volume is created with ubimkvol, you can use the -m option and
the size is calculated automatically to give the biggest volume size where
beb_rsvd_pebs = beb_rsvd_level.

When the volume size is specified with ubinize you can flag it as "autoresize"
and it will be increased if possible.

>> And that means that the maximum volume size varies.
>>
>> What we do here is we don't have the same partition table.  We fix the size
>> of
>> every partition except the last and let the start of each partition be
>> adjusted
>> if there are bad blocks.  For the last partition, it ends up being smaller
>> by the
>> total number of bad blocks.  We then use the "autoresize" of UBI to
>> automatically
>> fit the volume size of the last volume to the partition size.  Since our
>> last
>> volume is UBIFS, there is no problem because UBIFS also automatically
>> adjusts
>> (so long as the volume size is larger that the flashed image created by
>> mkfs.ubifs).
> Oh, too complicated! I don't quite understand.

Sorry that wasn't very clear.  Please just ignore it.

> en...
> fix size of every partition except the last!  if last partitions is
> full of data, then it can't give any free PEBs to other volumes right?
> or you just keep last volume blank, do not use it, like a space
> storehouse for other volume's need, if like that, why not simply
> reserved the last volume size.

The image for the last partition or volume must always be small enough to fit.

>> If you want to have the same volume size, then you can but you must make the
>> size smaller than the maximum.  You will need to consult your documentation
>> about what the maximum number of initial bad blocks is.  For example, for
>> OneNAND with 2048 PEBs, when a OneNAND comes from the factory it may have
>> as many as 40 PEBs that are bad.  So 2048 - 40 = 2008 good PEBs minimum.
>> 2008 / 100 = 20 PEBs reserved for (more) bad blocks (assuming you use the 1%
>> setting).  2008 - 20 = 1988 PEBs for the volume size.  The exact volume size
>> is
>> then 1988 * LEB size = 1988 * 258048 (in your case) = 512999424.
> 
> Compare with reserved badblocks(including the init and future go bad)
> by a percentage UBI user can configure by wish. What good autoresize
> has?

autoresize is for when you are creating images with ubinize and you want to
use the maximum volume size.  Because the maximum volume size depends on
the number of bad blocks, it changes from device to device.

autoresize means you can specify the worst-case volume size and tell UBI to
make the volume bigger if there are fewer bad blocks than the worst case.

If you are using ubimkvol, you can just use the -m option to get the
maximum size.

If you are using ubimkvol, you can find UBI values in /sys/class/ubi/ubix
where x is the UBI device number (0 if you only have 1 UBI device).

For example, the following is the same as using the -m option:

	modprobe ubi mtd=5

	ebcnt=`cat /sys/class/ubi/ubi0/avail_eraseblocks`
	ebsize=`cat /sys/class/ubi/ubi0/eraseblock_size`

	volsize=`expr $ebcnt \* $ebsize`

	ubimkvol /dev/ubi0 -Nvol -s$volsize

Re: UBI: badblock management confusion!

[Nancy]

Thank you for your informations. They do help a lot!
3 days holiday delay this reply, sorry about that delay!

On 4/30/08, Adrian Hunter <ext-adrian.hunter@nokia.com> wrote:
> Nancy wrote:
> >
> > > Well, the number of initial bad blocks does not affect how many will
> become
> > > bad over time.  So it is not part of the calculation.  For example, if
> you
> > > have 2048 PEBs and 17 are bad initially and you want to reserve 1% for
> > > (more)
> > > bad blocks then the number reserved is (2048 - 17) / 100 = 20.  That
> makes
> > > the
> > > default maximum volume size 2048 - 17 - 20 = 2011.  Whereas, if you are
> > > lucky
> > > and there are no initial bad blocks, then the number reserved for bad
> block
> > > handling is 2048 / 100 = 20, and the default maximum volume size is
> > > 2048 - 20 = 2028.
> > >
> > > Just because 17 are initially bad in the first case does not mean only 3
> > > more
> > > will become bad in the time it takes for 20 to become bad in the second
> > > case.
> > >
> > > So 20 are reserved for bad block handling in both cases.
> > >
> > No, your implement says the initial bad blocks are part of the
> > reserved badblocks. As you figure out
> >     UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for
> > bad PEB handling, reserved 19, need 20
> > just Warning, it need reserved 20, but found there's one more badblock
> > comes out during the long using time, so reserved only 19.
> > Same code, same operation. Can you tell me how UBI figure out when is
> > the real initial time?
> >
>
> Initial bad blocks is the number of bad blocks when the flash comes from
> the factory.
>
> > I think it has no way to know, it have to admit
> > each attache MTD partition time is initial time. You said initial
> > time's badblocks Nand has are not part of reserved badblocks, but the
> > practice tell it is part of reserved badblock PEBs( the warning is a
> > prof). If it is part of reserved badblocks, then why the max volume
> > size have to - current badblocks (it is already in reserved
> > badblocks). That is a waist. That is what I confused about!
> >
>
> There are two numbers and they both depend on the number of bad blocks.
>
> The two numbers are:
>
>        beb_rsvd_pebs
>        beb_rsvd_level
>
> These numbers are not written to the flash, they are calculated as follows:
>
> beb_rsvd_level is the number of good PEBs multiplied by the bad block
> handling percentage.
>
> beb_rsvd_pebs is the same as beb_rsvd_level unless there are not enough
> PEBs.  That is, if the sum of the volume sizes plus beb_rsvd_pebs is
> greater than the number of good PEBs, then beb_rsvd_pebs is reduced.
>
> So the number of PEBs reserved for bad block handling depends on the
> volume size(s) and the number of bad blocks.
>
> When a new volume is created there must be enough PEBs for it plus the
> minimum bad block handling reserve (which is 2 PEBs minimum).
>
> When a new volume is created with ubimkvol, you can use the -m option and
> the size is calculated automatically to give the biggest volume size where
> beb_rsvd_pebs = beb_rsvd_level.
>
> When the volume size is specified with ubinize you can flag it as
> "autoresize"
> and it will be increased if possible.
>
> >
> > > And that means that the maximum volume size varies.
> > >
> > > What we do here is we don't have the same partition table.  We fix the
> size
> > > of
> > > every partition except the last and let the start of each partition be
> > > adjusted
> > > if there are bad blocks.  For the last partition, it ends up being
> smaller
> > > by the
> > > total number of bad blocks.  We then use the "autoresize" of UBI to
> > > automatically
> > > fit the volume size of the last volume to the partition size.  Since our
> > > last
> > > volume is UBIFS, there is no problem because UBIFS also automatically
> > > adjusts
> > > (so long as the volume size is larger that the flashed image created by
> > > mkfs.ubifs).
> > >
> > Oh, too complicated! I don't quite understand.
> >
>
> Sorry that wasn't very clear.  Please just ignore it.
>
> > en...
> > fix size of every partition except the last!  if last partitions is
> > full of data, then it can't give any free PEBs to other volumes right?
> > or you just keep last volume blank, do not use it, like a space
> > storehouse for other volume's need, if like that, why not simply
> > reserved the last volume size.
> >
>
> The image for the last partition or volume must always be small enough to
> fit.
>
> >
> > > If you want to have the same volume size, then you can but you must make
> the
> > > size smaller than the maximum.  You will need to consult your
> documentation
> > > about what the maximum number of initial bad blocks is.  For example,
> for
> > > OneNAND with 2048 PEBs, when a OneNAND comes from the factory it may
> have
> > > as many as 40 PEBs that are bad.  So 2048 - 40 = 2008 good PEBs minimum.
> > > 2008 / 100 = 20 PEBs reserved for (more) bad blocks (assuming you use
> the 1%
> > > setting).  2008 - 20 = 1988 PEBs for the volume size.  The exact volume
> size
> > > is
> > > then 1988 * LEB size = 1988 * 258048 (in your case) = 512999424.
> > >
> >
> > Compare with reserved badblocks(including the init and future go bad)
> > by a percentage UBI user can configure by wish. What good autoresize
> > has?
> >
>
> autoresize is for when you are creating images with ubinize and you want to
> use the maximum volume size.  Because the maximum volume size depends on
> the number of bad blocks, it changes from device to device.
>
> autoresize means you can specify the worst-case volume size and tell UBI to
> make the volume bigger if there are fewer bad blocks than the worst case.
>
> If you are using ubimkvol, you can just use the -m option to get the
> maximum size.
>
> If you are using ubimkvol, you can find UBI values in /sys/class/ubi/ubix
> where x is the UBI device number (0 if you only have 1 UBI device).
>
> For example, the following is the same as using the -m option:
>
>        modprobe ubi mtd=5
>
>        ebcnt=`cat /sys/class/ubi/ubi0/avail_eraseblocks`
>        ebsize=`cat /sys/class/ubi/ubi0/eraseblock_size`
>
>        volsize=`expr $ebcnt \* $ebsize`
>
>        ubimkvol /dev/ubi0 -Nvol -s$volsize
>
>
>


-- 
Best wishes,
Nancy