[Ubiattach] "vtbl_check: too large reserved_pebs"

[Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

Hello

	I went through the docs/FAQs on the site and elsewhere, and
proceeded to create a Ubifs root filesystem and flash a NAND partition
with it.
However, "ubiattach" fails, so I guess I didn't use the right settings
in mkfs.ubifs and/or ubinize and/or ubiformat for use with that NAND
partition.

Here are the steps I took:
==============
#First, get infos about NAND
appliance:/> mtdinfo /dev/mtd2 -u
mtd2
Name:                           filesystem_nand
Type:                           nand
Eraseblock size:                131072 bytes, 128.0 KiB
Amount of eraseblocks:          1984 (260046848 bytes, 248.0 MiB)
Minimum input/output unit size: 2048 bytes
Sub-page size:                  512 bytes
OOB size:                       64 bytes
Character device major/minor:   90:4
Bad blocks are allowed:         true
Device is writable:             true
Default UBI VID header offset:  512
Default UBI data offset:        2048
Default UBI LEB size:           129024 bytes, 126.0 KiB
Maximum UBI volumes count:      128
==============
#Create root filesystem
ubuntu:/> mkfs.ubifs --squash-uids -m 2048 -e 131072 -c 1984 -d
./romfs -D device_table-min.txt -o ./images/rootfs.ubifs

#ubinize image
ubuntu:/> cat ubi.cfg
[ubifs]
mode=ubi
image=./images/rootfs.ubifs
vol_id=0
vol_size=248MiB
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize

ubuntu:/> ubinize -o rootfs.ubi.img -m 2048 -p 128KiB -s 512 ./ubi.cfg
==============
appliance:/> ubiformat /dev/mtd2 -s 512 -f rootfs.ubi.img

appliance:/> ubiattach /dev/ubi_ctrl -m 2
UBI: attaching mtd2 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: smallest flash I/O unit:    2048
UBI: sub-page size:              512
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI error: vtbl_check: too large reserved_pebs 2016, good PEBs 1984
UBI error: vtbl_check: volume table check failed: record 0, error 9
UBI error: ubi_attach_mtd_dev: failed to attach by scanning, error -22
ubiattach: error!: cannot attach mtd2
           error 22 (Invalid argument)
==============

Any idea what I did wrong?

Thank you.

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Artem Bityutskiy]

On Thu, 2011-06-09 at 12:24 +0200, Gilles wrote:
> Hello
> 
> 	I went through the docs/FAQs on the site and elsewhere, and
> proceeded to create a Ubifs root filesystem and flash a NAND partition
> with it.
> However, "ubiattach" fails, so I guess I didn't use the right settings
> in mkfs.ubifs and/or ubinize and/or ubiformat for use with that NAND
> partition.
> 
> Here are the steps I took:
> ==============
> #First, get infos about NAND
> appliance:/> mtdinfo /dev/mtd2 -u
> mtd2
> Name:                           filesystem_nand
> Type:                           nand
> Eraseblock size:                131072 bytes, 128.0 KiB
> Amount of eraseblocks:          1984 (260046848 bytes, 248.0 MiB)
> Minimum input/output unit size: 2048 bytes
> Sub-page size:                  512 bytes
> OOB size:                       64 bytes
> Character device major/minor:   90:4
> Bad blocks are allowed:         true
> Device is writable:             true
> Default UBI VID header offset:  512
> Default UBI data offset:        2048
> Default UBI LEB size:           129024 bytes, 126.0 KiB
> Maximum UBI volumes count:      128
> ==============
> #Create root filesystem
> ubuntu:/> mkfs.ubifs --squash-uids -m 2048 -e 131072 -c 1984 -d

Use  -e 129024

P.S. Yes, I realize the options are inconsistent, and will happily
accept a clean-up :-)

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Thu, 09 Jun 2011 15:20:37 +0300, Artem Bityutskiy
<dedekind1@gmail.com> wrote:
>> Default UBI LEB size:           129024 bytes, 126.0 KiB
>> Maximum UBI volumes count:      128
>> ==============
>> #Create root filesystem
>> ubuntu:/> mkfs.ubifs --squash-uids -m 2048 -e 131072 -c 1984 -d
>
>Use  -e 129024

Thanks Artem, but I'm still getting the error :-/

Here's what I did:
========================
ubuntu:/> mkfs.ubifs --squash-uids -m 2048 -e 129024 -c 1984 -d
./romfs -D device_table-min.txt -o ./images/rootfs.ubifs

ubuntu:/> cat ubi.cfg
[ubifs]
mode=ubi
image=./images/rootfs.ubifs
vol_id=0
;What size to use?
vol_size=248MiB
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize

ubuntu:/> ubinize -o rootfs.ubi.img -m 2048 -p 128KiB -s 512 ./ubi.cfg
ubuntu:/> mv rootfs.ubi.img /var/www

appliance:> cd /var/tmp ; wget -c http://192.168.0.3/rootfs.ubi.img

appliance:> ubiformat /dev/mtd2 -s 512 -f rootfs.ubi.img

appliance:> ubiattach /dev/ubi_ctrl -m 2
UBI: attaching mtd2 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: smallest flash I/O unit:    2048
UBI: sub-page size:              512
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI error: vtbl_check: too large reserved_pebs 2016, good PEBs 1984
UBI error: vtbl_check: volume table check failed: record 0, error 9
UBI error: ubi_attach_mtd_dev: failed to attach by scanning, error -22
ubiattach: error!: cannot attach mtd2
           error 22 (Invalid argument)
========================

According to some out-of-band information, "In most cases, the formula
for the offset boils down to: LEB_size = PEB_size - (((Subpage_size +
Page_size) * Page_size) / Page_size)".

So, using the infos provided by mtdinfo, I find this: 131.072 - (((
512 + 2.048) * 2.048 ) / 2.048 ) = 128.512

Should I use that number instead of 129024?

Also, I wonder if maybe I should leave some room in the NAND partition
for the Ubifs journal + error log? I used vol_size=248MiB in ubi.cfg,
but maybe this is too big?

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Artem Bityutskiy]

On Thu, 2011-06-09 at 12:24 +0200, Gilles wrote:
> vol_size=248MiB

Put, say, 230MiB here. 248MiB is the total amount of space. UBI adds
some overhead, so your volume size will be smaller, see here:

http://www.linux-mtd.infradead.org/doc/ubi.html#L_overhead

The autoresize flag that you specified will make sure your volume will
be automatically re-sized to the maximum size when you first attach the
MTD device.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[David Wagner]

On 06/09/2011 02:26 PM, Gilles Ganault wrote:
> Hello
> 
> At 13:50 09/06/2011, you wrote:
>> > Eraseblock size:                131072 bytes, 128.0 KiB
>> >  [...]
>> > Default UBI LEB size:           129024 bytes, 126.0 KiB
>> > Maximum UBI volumes count:      128
>> > ==============
>> > #Create root filesystem
>> > ubuntu:/> mkfs.ubifs --squash-uids -m 2048 -e 131072 -c 1984 -d
>> > ./romfs -D device_table-min.txt -o ./images/rootfs.ubifs
>> >
>>
>> The -e option of mkfs.ubifs takes the logical eraseblock size.
>> But you specified 131072, the physical eraseblock size.
>> So here, you should have -e 129024
>> The details for computing the LEB size are at
>> http://git.infradead.org/mtd-utils.git/blob/HEAD:/ubi-utils/src/libubigen.c#l40
>>
>> In most cases, the formula for the offset boils down to:
>> LEB_size = PEB_size - (((Subpage_size + Page_size) * Page_size) /
>> Page_size)
> 
> Thanks for the tip, but after recompiling a new root filesystem, I'm
> still getting the same error:
[...]
> BTW, using the formula above, I find this: 131.072 - ((( 512 + 2.048) *
> 2.048 ) / 2.048 ) = 128.512
> 
> I wonder if maybe I should leave some room in the NAND partition for the
> Ubifs journal + error log? I used vol_size=248MiB in ubi.cfg, but maybe
> this is toot big?

(I forgot to reply to all in my first answer)

Oops, sorry, the formula is
((Subpage_size + Page_size)) / Page_size) * Page_size
That is, subpage+page truncated to page.

Maybe you have to take the reserved LEBs into account: 1% of the PEBs.
I'm not sure if they are already included in what mtdinfo displays, though.

-- 
David Wagner, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Thu, 09 Jun 2011 15:38:25 +0300, Artem Bityutskiy
<dedekind1@gmail.com> wrote:
>On Thu, 2011-06-09 at 12:24 +0200, Gilles wrote:
>> vol_size=248MiB
>
>Put, say, 230MiB here. 248MiB is the total amount of space. UBI adds
>some overhead, so your volume size will be smaller, see here:

After updating ubi.cfg with "vol_size=230MiB" and re-running "ubinize
-o rootfs.ubi.img -m 2048 -p 128KiB -s 512 ./ubi.cfg", I reformateed
the appliance and tried to attach mtd2 to Ubi... and it worked! :-)

===============
appliance:> ubiformat /dev/mtd2 -s 512 -f rootfs.ubi.img

appliance:> ubiattach /dev/ubi_ctrl -m 2
UBI: attaching mtd2 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: smallest flash I/O unit:    2048
UBI: sub-page size:              512
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI: volume 0 ("rootfs") re-sized from 1870 to 1961 LEBs
UBI: attached mtd2 to ubi0
UBI: MTD device name:            "filesystem_nand"
UBI: MTD device size:            248 MiB
UBI: number of good PEBs:        1984
UBI: number of bad PEBs:         0
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    4096
UBI: number of internal volumes: 1
UBI: number of user volumes:     1
UBI: available PEBs:             0
UBI: total number of reserved PEBs: 1984
UBI: number of PEBs reserved for bad PEB handling: 19
UBI: max/mean erase counter: 14/9
UBI: image sequence number: 1879706523
mtd: Giving out device 3 to rootfs
UBI: background thread "ubi_bgt0d" started, PID 136
UBI device number 0, total 1984 LEBs (255983616 bytes, 244.1 MiB),
available 0 LEBs (0 bytes), LEB size 129024 bytes (126.0 KiB)
===============

So apparently, I lost about 4MiB in the process (248MiB -> 244.1).

I'll take advantage of this thread to ask a few questions that I have
after reading the docs + experimenting:

1. After running "ubiattach", I get a new mtd3 partition that is
linked to mtd2:

appliance> cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00800000 00001000 "ROMfs"
mtd1: 00800000 00020000 "kernel_nand"
mtd2: 0f800000 00020000 "filesystem_nand"
mtd3: 0f14b800 0001f800 "rootfs"

To reboot and use the NAND partition as root filesystem, am I correct
that I should forget about mtd2, and use this kernel command line:

"ubi.mtd=3 rootfstype=ubifs root=ubi0:rootfs rw"

?

2. ubiformat takes about 10-15mn to complete: Is there a way to speed
things up, or is it just due to the CPU+NAND that the appliance uses?

3. Finally, when do I have to use ubimkvol?

Thank you.

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Thu, 09 Jun 2011 14:42:08 +0200, David Wagner
<david.wagner@free-electrons.com> wrote:
>>> In most cases, the formula for the offset boils down to:
>>> LEB_size = PEB_size - (((Subpage_size + Page_size) * Page_size) /
>>> Page_size)
>> 
>> Thanks for the tip, but after recompiling a new root filesystem, I'm
>> still getting the same error:
>[...]
>> BTW, using the formula above, I find this: 131.072 - ((( 512 + 2.048) *
>> 2.048 ) / 2.048 ) = 128.512
>> 
>> I wonder if maybe I should leave some room in the NAND partition for the
>> Ubifs journal + error log? I used vol_size=248MiB in ubi.cfg, but maybe
>> this is toot big?
>
>(I forgot to reply to all in my first answer)
>
>Oops, sorry, the formula is
>((Subpage_size + Page_size)) / Page_size) * Page_size
>That is, subpage+page truncated to page.

Thanks, but I must have misunderstood the formula, as I find the same
number:

PEB_size - ((Subpage_size + Page_size)) / Page_size) * Page_size =
LEB_size

131.072 - ((512 + 2048 ) / 2048) * 2048) = 128.512

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[David Wagner]

>> Oops, sorry, the formula is
>> ((Subpage_size + Page_size)) / Page_size) * Page_size
>> That is, subpage+page truncated to page.
> 
> Thanks, but I must have misunderstood the formula, as I find the same
> number:
> 
> PEB_size - ((Subpage_size + Page_size)) / Page_size) * Page_size =
> LEB_size
> 
> 131.072 - ((512 + 2048 ) / 2048) * 2048) = 128.512

I should have mentionned that every sub-result has to be considered an
integer, here, sorry.
512+2048 = 2560
2560 / 2048 = 1 // That's where it is important that the result be
                // an integer.
1 * 2048 = 2048
131072 - 2048 = 129024

Regards,
-- 
David Wagner, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Artem Bityutskiy]

On Thu, 2011-06-09 at 17:50 +0200, David Wagner wrote:
> >> Oops, sorry, the formula is
> >> ((Subpage_size + Page_size)) / Page_size) * Page_size
> >> That is, subpage+page truncated to page.
> > 
> > Thanks, but I must have misunderstood the formula, as I find the same
> > number:
> > 
> > PEB_size - ((Subpage_size + Page_size)) / Page_size) * Page_size =
> > LEB_size
> > 
> > 131.072 - ((512 + 2048 ) / 2048) * 2048) = 128.512
> 
> I should have mentionned that every sub-result has to be considered an
> integer, here, sorry.
> 512+2048 = 2560
> 2560 / 2048 = 1 // That's where it is important that the result be
>                 // an integer.
> 1 * 2048 = 2048
> 131072 - 2048 = 129024

An improvement patch to the mtd web-site is welcome - it is very simple
to change:

http://git.infradead.org/mtd-www.git

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Thu, 09 Jun 2011 17:50:57 +0200, David Wagner
<david.wagner@free-electrons.com> wrote:
>I should have mentionned that every sub-result has to be considered an
>integer, here, sorry.
>512+2048 = 2560
>2560 / 2048 = 1 // That's where it is important that the result be
>                // an integer.
>1 * 2048 = 2048
>131072 - 2048 = 129024

Thanks a lot.

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Thu, 09 Jun 2011 12:24:22 +0200, Gilles <gilles.ganault@free.fr>
wrote:
(snip)

Now that I can successfully format + attach + mount the Ubifs NAND
partition, I tried rebooting with the following kernel command line:

==========
bootargs=ethaddr=00:09:45:56:72:9b console=ttyBF0,115200
rootfstype=ubifs ubi.mtd2 root=ubi0:rootfs
==========

However, uClinux isn't happy:
==========
...
UBIFS error (pid 1): ubifs_get_sb: cannot open "ubi0:rootfs", error
-19
VFS: Cannot open root device "ubi0:rootfs" or unknown-block(0,0)
Please append a correct "root=" boot option; here are the available
partitions:
1f00            8192 mtdblock0 (driver?)
1f01            8192 mtdblock1 (driver?)
1f02          253952 mtdblock2 (driver?)
Kernel panic - not syncing: VFS: Unable to mount root fs on
unknown-block(0,0)
Hardware Trace:
==========

I compiled Ubi(fs) statically in the kernel and included the MTD
utilities. Do I need something else for uClinux to be able to
attach/mount a Ubifs partition?

Thank you.

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Thu, 09 Jun 2011 18:17:24 +0200, Gilles <gilles.ganault@free.fr>
wrote:
>Now that I can successfully format + attach + mount the Ubifs NAND
>partition, I tried rebooting with the following kernel command line:
>
>==========
>bootargs=ethaddr=00:09:45:56:72:9b console=ttyBF0,115200
>rootfstype=ubifs ubi.mtd2 root=ubi0:rootfs
>==========

Even after correcting the mistake above ("ubi.mtd2" should be
"ubi=mtd2"), uClinux still can't access Ubifs on mtd2 and crashes.

I double-checked that the kernel has MTD + UBI + Ubifs compiled
statically.

Incidently, are those settings in "bootargs" enough to have the kernel
run the equivalent of ubiattach with the right commands for Ubi(fs)
before attempting to mount the NAND partition as root filesystem?

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Fri, 10 Jun 2011 01:05:56 +0200, Gilles <gilles.ganault@free.fr>
wrote:
>Even after correcting the mistake above ("ubi.mtd2" should be
>"ubi=mtd2"), uClinux still can't access Ubifs on mtd2 and crashes.

Wrong again (blush)

Here's the correct bootargs:
===========
bootargs=... rootfstype=ubifs ubi.mtd=2 root=ubi0:rootfs rw
===========

Don't forget the trailing "rw", or it will be read-only and cause a
few errors.

I'm now successfully running with the root filesystem on the NAND with
Ubifs.

Thanks everyone for the great help.

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Artem Bityutskiy]

On Thu, 2011-06-09 at 17:18 +0200, Gilles wrote:
> 2. ubiformat takes about 10-15mn to complete: Is there a way to speed
> things up, or is it just due to the CPU+NAND that the appliance uses?

http://www.linux-mtd.infradead.org/faq/ubi.html#L_attach_faster

> 3. Finally, when do I have to use ubimkvol?

When you want to dynamically (run-time) create a new UBI volume. You do
not necessarily have to do this.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: [Ubiattach] "vtbl_check: too large reserved_pebs"

[Gilles]

On Fri, 10 Jun 2011 07:03:40 +0300, Artem Bityutskiy
<dedekind1@gmail.com> wrote:
>When you want to dynamically (run-time) create a new UBI volume. You do
>not necessarily have to do this.

Thanks for the infos.