UBI and OneNand

UBI and OneNand

[John]

Hi,

I have a board with a MIPS core and a OneNAND flash.  I plan to split
the OneNAND into a small partition for the boot loader, and a big
partition for the rest. The big partition will be split using UBI into:
  a partition for the Kernel
  a partition for a readonly root file system (squashfs?)
  a read/write JFFS2 partition for persistant data
  a partition for the root file system for use when the box boots next
time
  etc....
The concepts of UBI seem just right.

I have made the kernel from ubi-2.6.git run on my board. Now I would be
grateful for answers to some questions to help me get UBI going.

  1. Should I expect UBI to run on OneNAND without change, or should I
     expect to have to write or adapt some code?

  2. Which git provides the best mtd-utils? Should I get mtd-tools and
     ubi-tools from different gits?

  3. My attempt to build the ubi tools from the git
     users/haver/mtd-utils.git failed.  The tools use the argp
     functions to parse command line args, but my root file system
     based on uclibc doesn't support argp. I propose to collect argp
     sources following instructions in
     http://savannah.gnu.org/cvs/?group=gnulib and try again. Any
     better suggestions?

  4. The UBI faq
     http://www.linux-mtd.infradead.org/faq/ubi.html#L_run_ubi says
     that when UBI is compiled into the Kernel, you should call
     ubi_attach_mtd_dev() to launch the UBI code.  But
     ubi_attach_mtd_dev() is now declared static, so I guess the
     advice is out of date.  How should UBI be started when compiled
     into the kernel?

  5. What is the way to get jffs2 running on an UBI partition?  Should
     I still be aiming to use gluebi to provide an MTD partition on
     top of a UBI partition (whatever that means)? Cook book
     instructions would be very helpful.

Thanks to all for MTD,

John Smith

Re: UBI and OneNand

[Frank Haverkamp]

Hi John,

On Sat, 2006-11-04 at 08:22 +0000, John wrote:
> I have a board with a MIPS core and a OneNAND flash.  I plan to split
> the OneNAND into a small partition for the boot loader, and a big
> partition for the rest. The big partition will be split using UBI into:
>   a partition for the Kernel
>   a partition for a readonly root file system (squashfs?)
>   a read/write JFFS2 partition for persistant data
>   a partition for the root file system for use when the box boots next
> time
>   etc....
> The concepts of UBI seem just right.
> 
> I have made the kernel from ubi-2.6.git run on my board. Now I would be
> grateful for answers to some questions to help me get UBI going.
> 
>   1. Should I expect UBI to run on OneNAND without change, or should I
>      expect to have to write or adapt some code?

UBI should run on mostly any mtd device. I tried it on NAND as well as
NOR mtds.

> 
>   2. Which git provides the best mtd-utils? Should I get mtd-tools and
>      ubi-tools from different gits?

Unfortunately yes. I need to talk to David or maybe Josh to integrate
the ubi-tools into the main mtd-utils.git. Until than take my version.

> 
>   3. My attempt to build the ubi tools from the git
>      users/haver/mtd-utils.git failed.  The tools use the argp
>      functions to parse command line args, but my root file system
>      based on uclibc doesn't support argp. I propose to collect argp
>      sources following instructions in
>      http://savannah.gnu.org/cvs/?group=gnulib and try again. Any
>      better suggestions?

Mhm, I wonder if we should remove the argp parsing again and replace it
by more portable code. I am not sure about this yet.

> 
>   4. The UBI faq
>      http://www.linux-mtd.infradead.org/faq/ubi.html#L_run_ubi says
>      that when UBI is compiled into the Kernel, you should call
>      ubi_attach_mtd_dev() to launch the UBI code.  But
>      ubi_attach_mtd_dev() is now declared static, so I guess the
>      advice is out of date.  How should UBI be started when compiled
>      into the kernel?

We decided that the connection between UBI and its mtd should be made at
the kernel command line, instead of exporting a function to do this.

The syntax is:

  [ubi.]mtd=content[,<vid-hdr-offs>,<data-offs>] ...
               ^
               |
             name of the MTD

If compiled in the kernel the ubi. prefix must be used, if used as module
it works without it. You can offer multiple of those options after each
other to connect more than one MTD with UBI.

The vid-hdr-offs and data-offs parameters are optional. UBI will choose
hopefully resonable default values. If you are not happy with the
defaults, use the parameters to define different ones.

> 
>   5. What is the way to get jffs2 running on an UBI partition?  Should
>      I still be aiming to use gluebi to provide an MTD partition on
>      top of a UBI partition (whatever that means)? Cook book
>      instructions would be very helpful.

Yes, gluebi is used to create the MTD which can be used by JFFS2. We had
some iterations over the code, and finally came up with the code
available in our ubi-2.6.git. The original from Joern was reworked by
Artem, the currently available solution has only very few changes to
JFFS2.

Frank

Re: UBI and OneNand

[Artem Bityutskiy]

Hello,

On Mon, 2006-11-06 at 10:46 +0100, Frank Haverkamp wrote:
> >   1. Should I expect UBI to run on OneNAND without change, or should I
> >      expect to have to write or adapt some code?
> 
> UBI should run on mostly any mtd device. I tried it on NAND as well as
> NOR mtds.
Yeah, in theory you should be able.

> >   4. The UBI faq
> >      http://www.linux-mtd.infradead.org/faq/ubi.html#L_run_ubi says
> >      that when UBI is compiled into the Kernel, you should call
> >      ubi_attach_mtd_dev() to launch the UBI code.  But
> >      ubi_attach_mtd_dev() is now declared static, so I guess the
> >      advice is out of date.  How should UBI be started when compiled
> >      into the kernel?
> 
> We decided that the connection between UBI and its mtd should be made at
> the kernel command line, instead of exporting a function to do this.
I'll update the FAQ.

> >   5. What is the way to get jffs2 running on an UBI partition?  Should
> >      I still be aiming to use gluebi to provide an MTD partition on
> >      top of a UBI partition (whatever that means)? Cook book
> >      instructions would be very helpful.

Just enable "Emulate MTD devices" option in the configuration and UBI
will create an MTD device for each volume, and those MTD devices may be
used to mount JFFS2.

I'll update the FAQ.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI and OneNand

[John Smith]

Frank Haverkamp <haver <at> vnet.ibm.com> writes:

> 
> Hi John,
> 
> On Sat, 2006-11-04 at 08:22 +0000, John wrote:
> > I have a board with a MIPS core and a OneNAND flash.  ...
> > 
> > I have made the kernel from ubi-2.6.git run on my board. Now I would be
> > grateful for answers to some questions to help me get UBI going.
> > 
> >   1. Should I expect UBI to run on OneNAND without change, or should I
> >      expect to have to write or adapt some code?
> 
> UBI should run on mostly any mtd device. I tried it on NAND as well as
> NOR mtds.

I have now tried it. It does run. 

> 
> > 
> >   3. My attempt to build the ubi tools from the git
> >      users/haver/mtd-utils.git failed.  The tools use the argp
> >      functions to parse command line args, but my root file system
> >      based on uclibc doesn't support argp. I propose to collect argp
> >      sources following instructions in
> >      http://savannah.gnu.org/cvs/?group=gnulib and try again. Any
> >      better suggestions?
> 
> Mhm, I wonder if we should remove the argp parsing again and replace it
> by more portable code. I am not sure about this yet.

I did get the ubi-tools to compile with argp code from gnulib.
There is probably an elegant solution using automake and friends 
and the sources from gnulib. But I just hacked a Makefile and a config.h 
together, and changed the names of two functions from getopt.h 
to avoid clashes with my uclibc implementation.

In the ubi-tools directory there is a script "ubi_test.sh". 
This script appeared to run successfully as well.

> 
> > 
> >   5. What is the way to get jffs2 running on an UBI partition?  Should
> >      I still be aiming to use gluebi to provide an MTD partition on
> >      top of a UBI partition (whatever that means)? Cook book
> >      instructions would be very helpful.
> 
> Yes, gluebi is used to create the MTD which can be used by JFFS2. We had
> some iterations over the code, and finally came up with the code
> available in our ubi-2.6.git. The original from Joern was reworked by
> Artem, the currently available solution has only very few changes to
> JFFS2.
> 

I think the following commands just worked:

  mkdir /mnt/nvm
  flash_eraseall /dev/mtd6
  mount -t jffs2 /dev/mtdblock6 /mnt/nvm 


I wonder whether this is the first attempt to run a kernel using
jffs2 and ubi and mtd and onenand and mips? First or not, it seems to work. 

Thanks,
John Smith

Re: UBI and OneNand

[Josh Boyer]

On Mon, 2006-11-06 at 20:16 +0000, John Smith wrote:
> Frank Haverkamp <haver <at> vnet.ibm.com> writes:
> 
> > 
> > Hi John,
> > 
> > On Sat, 2006-11-04 at 08:22 +0000, John wrote:
> > > I have a board with a MIPS core and a OneNAND flash.  ...
> > > 
> > > I have made the kernel from ubi-2.6.git run on my board. Now I would be
> > > grateful for answers to some questions to help me get UBI going.
> > > 
> > >   1. Should I expect UBI to run on OneNAND without change, or should I
> > >      expect to have to write or adapt some code?
> > 
> > UBI should run on mostly any mtd device. I tried it on NAND as well as
> > NOR mtds.
> 
> I have now tried it. It does run. 

Excellent news!

> > Mhm, I wonder if we should remove the argp parsing again and replace it
> > by more portable code. I am not sure about this yet.
> 
> I did get the ubi-tools to compile with argp code from gnulib.
> There is probably an elegant solution using automake and friends 
> and the sources from gnulib. But I just hacked a Makefile and a config.h 
> together, and changed the names of two functions from getopt.h 
> to avoid clashes with my uclibc implementation.

The tools might get a little rip up and rewrite eventually.  We can look
at removing the argp stuff then.  I personally don't like it much, but
never found it to be too big of an issue.

As for automake and friends... we tend to avoid those like the plague.

> I think the following commands just worked:
> 
>   mkdir /mnt/nvm
>   flash_eraseall /dev/mtd6
>   mount -t jffs2 /dev/mtdblock6 /mnt/nvm 
> 
> 
> I wonder whether this is the first attempt to run a kernel using
> jffs2 and ubi and mtd and onenand and mips? First or not, it seems to work. 

Yes, it is.  For both onenand and MIPS.  That is good news.  Thanks for
testing!

If you find any issues, feel free to email the list.

josh

UBI and OneNAND

[박경민]

Hi,

Sorry for changing subject. My mail program has some problem.

> > I think the following commands just worked:
> > 
> >   mkdir /mnt/nvm
> >   flash_eraseall /dev/mtd6
> >   mount -t jffs2 /dev/mtdblock6 /mnt/nvm 
> > 
> > 
> > I wonder whether this is the first attempt to run a kernel using
> > jffs2 and ubi and mtd and onenand and mips? First or not, it seems to work. 

> Yes, it is.  For both onenand and MIPS.  That is good news.  Thanks for
> testing!

Umm. I think 'John' just mounts jffs2 without UBI.

I also tried but I can't mount jffs2

It is my sequence.

# mkdir -p /mnt
# insmod ubi.ko mtd=3

UBI: attached mtd3 to ubi0
UBI: MTD device name:            "rootfs"
UBI: MTD device size:            16 MB
UBI: physical eraseblock size:   131072 bytes (128 KB)
UBI: logical eraseblock size:    126976 bytes
UBI: number of good PEBs:        127
UBI: number of bad PEBs:         1
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          2048 (aligned 2048)
UBI: data offset:                4096
UBI: max. allowed volumes:       124
UBI: wear-levelling threshold:   4096
UBI: number of internal volumes: 2
UBI: number of user volumes:     0
UBI: available PEBs:             122
UBI: total number of reserved PEBs: 5
UBI: number of PEBs reserved for bad PEB handling: 1

# mount -t jffs2 /dev/mtdblock3 /mnt

/ # mount -t jffs2 /dev/mtdblock3 /tmp
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x4255 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0x0001 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000003c: 0xcedc id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000800: 0x4255 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000804: 0x0101 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000808: 0xff7f id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000083c: 0x25b8 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x000010a8: 0x16f1 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001154: 0x16f1 id
jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001200: 0x16f1 id
...
Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes
empty_blocks 0, bad_blocks 1, c->nr_blocks 128
mount: Mounting /dev/mtdblock3 on /tmp failed: Input/output error

Is it right?

Of course. JFFS2 mount without UBI working well.

Thank you,
Kyungmin Park

Re: UBI and OneNAND

[Frank Haverkamp]

Hi,

On Tue, 2006-11-07 at 02:35 +0000, 박경민 wrote:
> Hi,
> 
> Sorry for changing subject. My mail program has some problem.
> 
> > > I think the following commands just worked:
> > > 
> > >   mkdir /mnt/nvm
> > >   flash_eraseall /dev/mtd6
> > >   mount -t jffs2 /dev/mtdblock6 /mnt/nvm 

I think there is a missunderstanding here. UBI acts as volume management
system. The MTD you connect to UBI cannot be mounted. Instead you create
a UBI volume (with ubimkvol) which has an associated mtd which can be
used to host a jffs2. This can be mounted than.

> Umm. I think 'John' just mounts jffs2 without UBI.

Yes, I agree.
> 
> I also tried but I can't mount jffs2
> 
> It is my sequence.
> 
> # mkdir -p /mnt
> # insmod ubi.ko mtd=3

mtd3 is connected with UBI. This mtd is not to be used as rootfs.
Instead a UBI volume needs to be created to host the rootfs (see also
ubi_jffs2_test.sh).

Example:
  ubimkvol -d0 -n10 -tdynamic -NVOL10 -s 1310720
             ^    ^     ^         ^         ^
             |    |     |         |         |
             |    |     |         |       size here in bytes
             |    |     |       name of mtd & volume
             |    |    JFFS2/UBI volumes need to be dynamic
             |  Unique volume number
     UBI device (you may have multiple, but for me only one made sense)

After this has been executed sucessfully, an mtd named VOL10 should
appear.

Mount this with:
  mount -t jffs2 mtd:VOL10 /mnt

> 
> UBI: attached mtd3 to ubi0
> UBI: MTD device name:            "rootfs"

The structure should probably similar like this:
   ubimtd
     |- rootfs dynamic UBI volume (JFFS2)
     |- extra dynamic UBI volume (JFFS2)
     |- kernel0 static UBI volume
     |- kernel1 static UBI volume

> UBI: MTD device size:            16 MB
> UBI: physical eraseblock size:   131072 bytes (128 KB)
> UBI: logical eraseblock size:    126976 bytes
> UBI: number of good PEBs:        127
> UBI: number of bad PEBs:         1
> UBI: smallest flash I/O unit:    2048
> UBI: VID header offset:          2048 (aligned 2048)
> UBI: data offset:                4096
> UBI: max. allowed volumes:       124
> UBI: wear-levelling threshold:   4096
> UBI: number of internal volumes: 2
> UBI: number of user volumes:     0
> UBI: available PEBs:             122
> UBI: total number of reserved PEBs: 5
> UBI: number of PEBs reserved for bad PEB handling: 1
> 
> # mount -t jffs2 /dev/mtdblock3 /mnt
> 

Please do not try to mount the mtd you defined to be used by UBI.
>From this mtd UBI will take the blocks to run the UBI volumes.
The wearleveling will be accross mtd3 in your case.

> / # mount -t jffs2 /dev/mtdblock3 /tmp
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x4255 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0x0001 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000003c: 0xcedc id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000800: 0x4255 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000804: 0x0101 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000808: 0xff7f id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000083c: 0x25b8 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x000010a8: 0x16f1 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001154: 0x16f1 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001200: 0x16f1 id
> ...
> Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes
> empty_blocks 0, bad_blocks 1, c->nr_blocks 128
> mount: Mounting /dev/mtdblock3 on /tmp failed: Input/output error

Frank

Re: UBI and OneNAND

[John Smith]

Frank Haverkamp <haver <at> vnet.ibm.com> writes:

> 
> Hi,
> 
> On Tue, 2006-11-07 at 02:35 +0000, 박경민 wrote:
> > Hi,
> > 
> > Sorry for changing subject. My mail program has some problem.
> > 
> > > > I think the following commands just worked:
> > > > 
> > > >   mkdir /mnt/nvm
> > > >   flash_eraseall /dev/mtd6
> > > >   mount -t jffs2 /dev/mtdblock6 /mnt/nvm 
> 
> I think there is a missunderstanding here. UBI acts as volume management
> system. The MTD you connect to UBI cannot be mounted. Instead you create
> a UBI volume (with ubimkvol) which has an associated mtd which can be
> used to host a jffs2. This can be mounted than.
> 
> > Umm. I think 'John' just mounts jffs2 without UBI.
> 
> Yes, I agree.

My original explanation was incomplete. Here is a fuller version.

The MTD code is compiled into the kernel, rather than as a kernel module.
The kernel command line includes:
   ubi.mtd=4
At boot time, the following is reported to the console:

Using static partition definition
Creating 2 MTD partitions on "Arrow 1504 Flash":
0x00000000-0x00c00000 : "BootLoader"
0x00c00000-0x01000000 : "OldCFE"
OneNAND 16MB 2.65/3.3V 16-bit (0x05)
Scanning device for bad blocks
Creating 4 MTD partitions on "Arrow-Onenand":
0x00000000-0x00100000 : "CFE"
0x00100000-0x00500000 : "Kernel"
0x00500000-0x00ff0000 : "rootfs"
0x00ff0000-0x01000000 : "CFE-NVM"
UBI: background thread "ubi_bgt0d" started, PID 14
UBI: mean erase counter:         1
UBI: attached mtd4 to ubi0
UBI: MTD device name:            "rootfs"
UBI: MTD device size:            10 MB
UBI: physical eraseblock size:   65536 bytes (64 KB)
UBI: logical eraseblock size:    63488 bytes
UBI: number of good PEBs:        175
UBI: number of bad PEBs:         0
UBI: smallest flash I/O unit:    1024
UBI: VID header offset:          1024 (aligned 1024)
UBI: data offset:                2048
UBI: max. allowed volumes:       128
UBI: wear-levelling threshold:   4096
UBI: number of internal volumes: 2
UBI: number of user volumes:     1
UBI: available PEBs:             49
UBI: total number of reserved PEBs: 126
UBI: number of PEBs reserved for bad PEB handling: 1


##
## Then with the busybox shell I can list the MTD partitions
##

/ # cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00c00000 00020000 "BootLoader"    <<< Nor Flash
mtd1: 00400000 00020000 "OldCFE"        <<< Nor Flash
mtd2: 00100000 00010000 "CFE"           <<< OneNAND Flash
mtd3: 00400000 00010000 "Kernel"        <<< OneNAND Flash
mtd4: 00af0000 00010000 "rootfs"        <<< OneNAND Flash
mtd5: 00010000 00010000 "CFE-NVM"       <<< OneNAND Flash
mtd6: 00753800 0000f800 "Kernel"        <<< UBI Partition on mtd4:

##
## I ceate a new 2M byte UBI Volume
##
/ # ubimkvol -s 2000000 -N NVM -d 0

/ # cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00c00000 00020000 "BootLoader"
mtd1: 00400000 00020000 "OldCFE"
mtd2: 00100000 00010000 "CFE"
mtd3: 00400000 00010000 "Kernel"
mtd4: 00af0000 00010000 "rootfs"
mtd5: 00010000 00010000 "CFE-NVM"
mtd6: 00753800 0000f800 "Kernel"
mtd7: 001f0000 0000f800 "NVM"            <<< New UBI Partition

##
## Look at existing devices
##
/ # ls -l /dev/mtdblock?
brw-r-----    1 0        0         31,   0 Jan  1 00:00 /dev/mtdblock0
brw-r-----    1 0        0         31,   1 Jan  1 00:00 /dev/mtdblock1
brw-r-----    1 0        0         31,   2 Jan  1 00:00 /dev/mtdblock2
brw-r-----    1 0        0         31,   3 Jan  1 00:00 /dev/mtdblock3

##
## Create a new device for my new partition
## (I do not have the udev tools installed)
##
/ # mknod /dev/mtdblock7 b 31 7

##
## Mount my new JFFS2/UBI partition
## (Note the erase size is further confirmation that I have a UBI partition)
##
/ # mkdir /mnt/nvm
/ # mount -t jffs2 /dev/mtdblock7 /mnt/nvm
JFFS2 write-buffering enabled buffer (1024) erasesize (63488)


##
## Copy a file to the JFFS2 Partition
##
/ # cp /my_module.ko /mnt/nvm

##
## Check it
##
/ # md5sum /my_module.ko /mnt/nvm/my_module.ko
bbb66a91c259fe42290bfa2952b2265f  /my_module.ko
bbb66a91c259fe42290bfa2952b2265f  /mnt/nvm/my_module.ko

##
## Unmount the jffs2 partition
##
/ # umount /mnt/nvm

##
## Check the file has gone away
##
/mnt/nfs/ubi-tools # md5sum /my_module.ko /mnt/nvm/my_module.ko
bbb66a91c259fe42290bfa2952b2265f  /my_module.ko



##
## Power cycle, and let the box boot up again
##

Using static partition definition
Creating 2 MTD partitions on "Arrow 1504 Flash":
0x00000000-0x00c00000 : "BootLoader"
0x00c00000-0x01000000 : "OldCFE"
OneNAND 16MB 2.65/3.3V 16-bit (0x05)
Scanning device for bad blocks
Creating 4 MTD partitions on "Arrow-Onenand":
0x00000000-0x00100000 : "CFE"
0x00100000-0x00500000 : "Kernel"
0x00500000-0x00ff0000 : "rootfs"
0x00ff0000-0x01000000 : "CFE-NVM"
UBI: background thread "ubi_bgt0d" started, PID 14
UBI: mean erase counter:         1
UBI: attached mtd4 to ubi0
UBI: MTD device name:            "rootfs"
UBI: MTD device size:            10 MB
UBI: physical eraseblock size:   65536 bytes (64 KB)
UBI: logical eraseblock size:    63488 bytes
UBI: number of good PEBs:        175
UBI: number of bad PEBs:         0
UBI: smallest flash I/O unit:    1024
UBI: VID header offset:          1024 (aligned 1024)
UBI: data offset:                2048
UBI: max. allowed volumes:       128
UBI: wear-levelling threshold:   4096
UBI: number of internal volumes: 2
UBI: number of user volumes:     2  <<< this time there is an extra partition
UBI: available PEBs:             17
UBI: total number of reserved PEBs: 158
UBI: number of PEBs reserved for bad PEB handling: 1


BusyBox v1.00 (2006.08.30-22:06+0000) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

/ # cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00c00000 00020000 "BootLoader"  << Nor
mtd1: 00400000 00020000 "OldCFE"      << Nor
mtd2: 00100000 00010000 "CFE"         << OneNAND
mtd3: 00400000 00010000 "Kernel"      << OneNAND
mtd4: 00af0000 00010000 "rootfs"      << OneNAND
mtd5: 00010000 00010000 "CFE-NVM"     << OneNAND
mtd6: 001f0000 0000f800 "NVM"         << UBI on mtd4:
mtd7: 00753800 0000f800 "Kernel"      << UBI on mtd4:

/ # mknod /dev/mtdblock6 b 31 6
/ # mount -t jffs2 /dev/mtdblock6 /mnt/nvm
JFFS2 write-buffering enabled buffer (1024) erasesize (63488)

##
## Check the file in the jffs2 partition
##
/ # md5sum /mnt/nvm/my_module.ko
bbb66a91c259fe42290bfa2952b2265f  /mnt/nvm/my_module.ko
/ # md5sum /my_module.ko
bbb66a91c259fe42290bfa2952b2265f  /my_module.ko


So I am convinced. 
Of course, the credit belongs to all who wrote the code. 
Thanks,
John Smith

Re: UBI and OneNAND

[Artem Bityutskiy]

Hi, few notes:

On Tue, 2006-11-07 at 10:20 +0000, John Smith wrote:
> UBI: data offset:                2048
> UBI: max. allowed volumes:       128
> UBI: wear-levelling threshold:   4096
> UBI: number of internal volumes: 2
> UBI: number of user volumes:     1
> UBI: available PEBs:             49
> UBI: total number of reserved PEBs: 126
> UBI: number of PEBs reserved for bad PEB handling: 1
Hmm, only one PEB is reserved, may be it makes sense to reserve more -
there is a corresponding option.

> 
> 
> ##
> ## Then with the busybox shell I can list the MTD partitions
> ##
> 
> / # cat /proc/mtd
> dev:    size   erasesize  name
> mtd0: 00c00000 00020000 "BootLoader"    <<< Nor Flash
> mtd1: 00400000 00020000 "OldCFE"        <<< Nor Flash
> mtd2: 00100000 00010000 "CFE"           <<< OneNAND Flash
> mtd3: 00400000 00010000 "Kernel"        <<< OneNAND Flash
> mtd4: 00af0000 00010000 "rootfs"        <<< OneNAND Flash
> mtd5: 00010000 00010000 "CFE-NVM"       <<< OneNAND Flash
> mtd6: 00753800 0000f800 "Kernel"        <<< UBI Partition on mtd4:
I would recommend you to use unique names for MTD partitions. You have
two "kernel" partitions.

> ##
> ## Look at existing devices
> ##
> / # ls -l /dev/mtdblock?
> brw-r-----    1 0        0         31,   0 Jan  1 00:00 /dev/mtdblock0
> brw-r-----    1 0        0         31,   1 Jan  1 00:00 /dev/mtdblock1
> brw-r-----    1 0        0         31,   2 Jan  1 00:00 /dev/mtdblock2
> brw-r-----    1 0        0         31,   3 Jan  1 00:00 /dev/mtdblock3
Yo do not need mtdblock devices to mount JFFS2 at all. JFFS2 anyway
works with MTD devices directly and this is just an old way to mount
JFFS2. /dev/mtdX are character devices and you cannot feed the mount
utility by character devices, so this is why fake block devices were
used. Nowadays there is device-less mount support and you may use:

mount mtd7 /mnt/nvm

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI and OneNAND

[Josh Boyer]

On Tue, 2006-11-07 at 13:39 +0200, Artem Bityutskiy wrote:
> Hi, few notes:
> 
> On Tue, 2006-11-07 at 10:20 +0000, John Smith wrote:
> > UBI: data offset:                2048
> > UBI: max. allowed volumes:       128
> > UBI: wear-levelling threshold:   4096
> > UBI: number of internal volumes: 2
> > UBI: number of user volumes:     1
> > UBI: available PEBs:             49
> > UBI: total number of reserved PEBs: 126
> > UBI: number of PEBs reserved for bad PEB handling: 1
> Hmm, only one PEB is reserved, may be it makes sense to reserve more -
> there is a corresponding option.

It's the default.  I think it's too low of a default.  Others disagreed
with me.

Telling every new UBI user "may be it makes sense to reserve more" will
get old quick.  Perhaps the default should be bumped to 5 or so?

josh

Re: UBI and OneNAND

[Artem Bityutskiy]

On Tue, 2006-11-07 at 05:47 -0600, Josh Boyer wrote:
> It's the default.  I think it's too low of a default.  Others disagreed
> with me.
> 
> Telling every new UBI user "may be it makes sense to reserve more" will
> get old quick.  Perhaps the default should be bumped to 5 or so?

Actually with 128MiB flash the default ends up with something like
32EBs, which looks even too large, so I do not know...

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI and OneNAND

[Josh Boyer]

On Tue, 2006-11-07 at 16:10 +0200, Artem Bityutskiy wrote:
> On Tue, 2006-11-07 at 05:47 -0600, Josh Boyer wrote:
> > It's the default.  I think it's too low of a default.  Others disagreed
> > with me.
> > 
> > Telling every new UBI user "may be it makes sense to reserve more" will
> > get old quick.  Perhaps the default should be bumped to 5 or so?
> 
> Actually with 128MiB flash the default ends up with something like
> 32EBs, which looks even too large, so I do not know...

What are you talking about?

(128MiB/128KiB) * .01 = 10.24

So you'd end up with 10EB.

Perhaps we should change MTD_UBI_BEB_RESERVE to be an actual number of
eraseblocks instead of a percent of the device.  So
MTD_UBI_BEB_RESERVE=1 would mean 1 EB, =10 would mean 10.

josh

Re: UBI and OneNAND

[Artem Bityutskiy]

On Tue, 2006-11-07 at 08:24 -0600, Josh Boyer wrote:
> What are you talking about?
> 
> (128MiB/128KiB) * .01 = 10.24
> 
> So you'd end up with 10EB.

Yeah, but somebody showed me UBI on 128 MiB flashes with 32 or 64 KiB
eraseblocks. Probably EB size should be taken into account.

> Perhaps we should change MTD_UBI_BEB_RESERVE to be an actual number of
> eraseblocks instead of a percent of the device.  So
> MTD_UBI_BEB_RESERVE=1 would mean 1 EB, =10 would mean 10.

My concern is a dumb user who may not realize what to enter there.

But may be you are right... If user does not set the number of reserved
PEBs explicitly, we could use some percentage as a default value.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI and OneNAND

[Artem Bityutskiy]

On Tue, 2006-11-07 at 02:35 +0000, 박경민 wrote:
> Hi,
> 
> Sorry for changing subject. My mail program has some problem.
> 
> > > I think the following commands just worked:
> > > 
> > >   mkdir /mnt/nvm
> > >   flash_eraseall /dev/mtd6
> > >   mount -t jffs2 /dev/mtdblock6 /mnt/nvm 
> > > 
> > > 
> > > I wonder whether this is the first attempt to run a kernel using
> > > jffs2 and ubi and mtd and onenand and mips? First or not, it seems to work. 
> 
> > Yes, it is.  For both onenand and MIPS.  That is good news.  Thanks for
> > testing!
> 
> Umm. I think 'John' just mounts jffs2 without UBI.
> 
> I also tried but I can't mount jffs2
> 
> It is my sequence.
> 
> # mkdir -p /mnt
> # insmod ubi.ko mtd=3
> 
> UBI: attached mtd3 to ubi0
> UBI: MTD device name:            "rootfs"
> UBI: MTD device size:            16 MB
> UBI: physical eraseblock size:   131072 bytes (128 KB)
> UBI: logical eraseblock size:    126976 bytes
> UBI: number of good PEBs:        127
> UBI: number of bad PEBs:         1
> UBI: smallest flash I/O unit:    2048
> UBI: VID header offset:          2048 (aligned 2048)
> UBI: data offset:                4096
> UBI: max. allowed volumes:       124
> UBI: wear-levelling threshold:   4096
> UBI: number of internal volumes: 2
> UBI: number of user volumes:     0
> UBI: available PEBs:             122
> UBI: total number of reserved PEBs: 5
> UBI: number of PEBs reserved for bad PEB handling: 1
> 

May I kindly ask you to do the following:

1. Enable "UBI debugging" option
2. Go to "UBI debugging messages  --->" and enable:
   a "User interface unit messages"
   b "Messages from the gluebi sub-unit"
   c "Volume management unit messages"
   d "Eraseblock association unit messages"

Then try to mount JFFS2.

Then do:
# mount -t debugfs none /mnt/debugfs
# cat /mnt/debugfs/ubi/log > log

and send me the log file.

Also see http://www.linux-mtd.infradead.org/faq/ubi.html#L_how_debug
which is a bit out-of-date but still may be helpful.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI and OneNAND

[Artem Bityutskiy]

On Tue, 2006-11-07 at 02:35 +0000, 박경민 wrote:
> It is my sequence.
> 
> # mkdir -p /mnt
> # insmod ubi.ko mtd=3
> 
> UBI: attached mtd3 to ubi0
> UBI: MTD device name:            "rootfs"
> UBI: MTD device size:            16 MB
> UBI: physical eraseblock size:   131072 bytes (128 KB)
> UBI: logical eraseblock size:    126976 bytes
> UBI: number of good PEBs:        127
> UBI: number of bad PEBs:         1
> UBI: smallest flash I/O unit:    2048
> UBI: VID header offset:          2048 (aligned 2048)
> UBI: data offset:                4096
> UBI: max. allowed volumes:       124
> UBI: wear-levelling threshold:   4096
> UBI: number of internal volumes: 2
> UBI: number of user volumes:     0
> UBI: available PEBs:             122
> UBI: total number of reserved PEBs: 5
> UBI: number of PEBs reserved for bad PEB handling: 1
> 
> # mount -t jffs2 /dev/mtdblock3 /mnt
This is incorrect. MTD3 now belongs to UBI only. Do not try to do
anything with it. You can only create UBI volumes now, and mount the
_volumes_!

> 
> / # mount -t jffs2 /dev/mtdblock3 /tmp
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x4255 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0x0001 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000003c: 0xcedc id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000800: 0x4255 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000804: 0x0101 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000808: 0xff7f id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x0000083c: 0x25b8 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x000010a8: 0x16f1 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001154: 0x16f1 id
> jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00001200: 0x16f1 id
> ...
> Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes
> empty_blocks 0, bad_blocks 1, c->nr_blocks 128
> mount: Mounting /dev/mtdblock3 on /tmp failed: Input/output error
Of course it is expected. UBI writes its headers to MTD3 and you try to
feed it to JFFS2. Do not do this.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

UBI and OneNAND

[Kyungmin Park]

Hi,

> > ##
> > ## Look at existing devices
> > ##
> > / # ls -l /dev/mtdblock?
> > brw-r-----    1 0        0         31,   0 Jan  1 00:00 /dev/mtdblock0
> > brw-r-----    1 0        0         31,   1 Jan  1 00:00 /dev/mtdblock1
> > brw-r-----    1 0        0         31,   2 Jan  1 00:00 /dev/mtdblock2
> > brw-r-----    1 0        0         31,   3 Jan  1 00:00 /dev/mtdblock3
> Yo do not need mtdblock devices to mount JFFS2 at all. JFFS2 anyway
> works with MTD devices directly and this is just an old way to mount
> JFFS2. /dev/mtdX are character devices and you cannot feed the mount
> utility by character devices, so this is why fake block devices were
> used. Nowadays there is device-less mount support and you may use:

> mount mtd7 /mnt/nvm

Yes, I can mount ubi with avobe method. 

I can't mount with name but It's possible with new mtdblock node

mtd = 3 -> mtd > 7
# mount -t jffs2 /dev/mtdblock7 /mnt

I will send more detailed description tomorrow.

Thank you,
Kyungmin Park

Re: UBI and OneNAND

[Artem Bityutskiy]

Hello, 

On Tue, 2006-11-07 at 12:52 +0000, Kyungmin Park wrote:
> > Yo do not need mtdblock devices to mount JFFS2 at all. JFFS2 anyway
> > works with MTD devices directly and this is just an old way to mount
> > JFFS2. /dev/mtdX are character devices and you cannot feed the mount
> > utility by character devices, so this is why fake block devices were
> > used. Nowadays there is device-less mount support and you may use:
> 
> > mount mtd7 /mnt/nvm
> 
> Yes, I can mount ubi with avobe method.

The device-less mount I described is a feature of *JFFS2*. It does not
depend on if there is UBI or not.

You are just trying to do an incorrect thing.

It is similar to LVM - you feed hard drives to LVM, then create LVM
volumes, then mount those LVM volumes, not the hard drives. Similar in
case of UBI. You feed mtd3 to UBI, then you create UBI volumes which are
also accessible as emulated MTD devices, and you mount _those_ MTD
devices.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)