Question about ubimkvol vs. mkfs.ubifs

Question about ubimkvol vs. mkfs.ubifs

[Bruce_Leonard]

Hi all,

This is probably a basic question but I'll admit to being lost.  I'm 
relatively new to kernel coding and up till now I've been focused on the 
very low level stuff, i.e., making hardware work and LEDs go 
blinkey-blinkey.  Currently I'm working on a NAND flash driver that I need 
to tie into UBI/UBIFS.  I've got the driver working, I can 
read/write/erase my flash parts, so the low level stuff that I understand 
is good.  Now I'm working at the higher levels/user space levels that I 
don't understand as well.  Basically my question is, what is the 
difference between ubimkvol and mkfs.ubifs?  I can run through the 
following steps and everything works just fine near as I can tell:

insmod my_flash_driver
insmod ubi mtd=1
insmod ubifs
ubimkvol /dev/ubi0 -m -N ubifs
mount -t ubifs ubi0:ubifs /mnt/nand

Once I do all that I can read/write/copy/delete/etc files just fine on the 
NAND flash.  So, like I said it appears to be working just fine.  But 
everything I see on the mailing list seems to be saying I should be using 
mkfs.ubifs.  But if everything works and I'm mounting the device as a 
UBIFS filesystem, what (if anything) is mkfs.ubifs going to do for me that 
I don't already have done?

Oh, another question.  I've seen reference to an 'fs-test' that's supposed 
to be part of mtd-utils, but I can't seem to find it anywhere.  Can 
someone please point me to the sources?  Is it a good test to run?

Thanks for the help.

See 'ya!

Bruce

Re: Question about ubimkvol vs. mkfs.ubifs

[Hamish Moffatt]

On Sun, Apr 06, 2008 at 04:53:37PM -0700, Bruce_Leonard@selinc.com wrote:
> Once I do all that I can read/write/copy/delete/etc files just fine on the 
> NAND flash.  So, like I said it appears to be working just fine.  But 
> everything I see on the mailing list seems to be saying I should be using 
> mkfs.ubifs.  But if everything works and I'm mounting the device as a 
> UBIFS filesystem, what (if anything) is mkfs.ubifs going to do for me that 
> I don't already have done?

mkfs.ubifs is for creating a ubifs image pre-populated with files eg
your root file system, similar to mkfs.jffs2, genext2fs, etc. You don't
need it to create a blank file system - in that case, just mount a blank
volume as you are doing now.


cheers
Hamish
-- 
Hamish Moffatt VK3SB <hamish@debian.org> <hamish@cloud.net.au>

Re: Question about ubimkvol vs. mkfs.ubifs

[Bruce_Leonard]

> mkfs.ubifs is for creating a ubifs image pre-populated with files eg
> your root file system, similar to mkfs.jffs2, genext2fs, etc. You don't
> need it to create a blank file system - in that case, just mount a blank
> volume as you are doing now.

Okay, that makes sense.  Thanks Hamish.

See 'ya!

Bruce

Re: Question about ubimkvol vs. mkfs.ubifs

[Artem Bityutskiy]

Hi,

On Sun, 2008-04-06 at 16:53 -0700, Bruce_Leonard@selinc.com wrote:
> This is probably a basic question but I'll admit to being lost.  I'm 
> relatively new to kernel coding and up till now I've been focused on the 
> very low level stuff, i.e., making hardware work and LEDs go 
> blinkey-blinkey.  Currently I'm working on a NAND flash driver that I need 
> to tie into UBI/UBIFS.  I've got the driver working, I can 
> read/write/erase my flash parts, so the low level stuff that I understand 
> is good. 

Is it a secret what is your flash and what is its size? If it is NAND
I'd recommend you to test it with Adrian's NAND tests:
git://infradead.org/~ahunter/nand-tests.git

> insmod my_flash_driver
> insmod ubi mtd=1
> insmod ubifs
> ubimkvol /dev/ubi0 -m -N ubifs
> mount -t ubifs ubi0:ubifs /mnt/nand

> Once I do all that I can read/write/copy/delete/etc files just fine on the 
> NAND flash.  So, like I said it appears to be working just fine.  But 
> everything I see on the mailing list seems to be saying I should be using 
> mkfs.ubifs.  But if everything works and I'm mounting the device as a 
> UBIFS filesystem, what (if anything) is mkfs.ubifs going to do for me that 
> I don't already have done?

As Hamish explained, you do not have to use mkfs.ubifs if you do not
need it. UBIFS can mount empty volumes just fine. 

In addition to what he said, mkfs.ubifs allows you to tune the
file-system. When you mount an empty volume, UBIFS formats it with
default settings: default journal size, default B-tree fanout, default
compression, etc. With mkfs.ubifs you may define different parameters,
e.g., use zlib instead of lzo, make larger journal, etc.

> Oh, another question.  I've seen reference to an 'fs-test' that's supposed 
> to be part of mtd-utils, but I can't seem to find it anywhere.  Can 
> someone please point me to the sources?  Is it a good test to run?

These are old tests Adrian wrote for JFFS2 and they sit at
mtd-utils.git/tests/fs-tests/. Some of the stress test scripts are
JFFS2-specific - you will notice. But they are easy to fix - just make
them not to run the "gcd_hupper" program. This is the only
JFFS2-specific thing.

Yes, I think this is a good idea to run them. Specifically, we are
extensively using the integrity test for UBIFS.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Question about ubimkvol vs. mkfs.ubifs

[Artem Bityutskiy]

On Mon, 2008-04-07 at 10:27 +0300, Artem Bityutskiy wrote:
> Hi,
> 
> On Sun, 2008-04-06 at 16:53 -0700, Bruce_Leonard@selinc.com wrote:
> > This is probably a basic question but I'll admit to being lost.  I'm 
> > relatively new to kernel coding and up till now I've been focused on the 
> > very low level stuff, i.e., making hardware work and LEDs go 
> > blinkey-blinkey.  Currently I'm working on a NAND flash driver that I need 
> > to tie into UBI/UBIFS.  I've got the driver working, I can 
> > read/write/erase my flash parts, so the low level stuff that I understand 
> > is good. 
> 
> Is it a secret what is your flash and what is its size? If it is NAND
> I'd recommend you to test it with Adrian's NAND tests:
> git://infradead.org/~ahunter/nand-tests.git

Just a note - be careful with torture test - it may harm. Although we
were unable to wear-out SLC OneNAND after even > 2 000 000 erases.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Question about ubimkvol vs. mkfs.ubifs

[Bruce_Leonard]

> 
> Is it a secret what is your flash and what is its size? If it is NAND
> I'd recommend you to test it with Adrian's NAND tests:
> git://infradead.org/~ahunter/nand-tests.git
> 
Nope, no secret.  We're using Samsung K9WAG08U1A NAND parts in 2GiB, 4GiB, 
and 8GiB configurations.  At least we're hoping to get the 4 and 8GiB 
configurations working.  I've run into a problem in the MTD layer with the 
size field.  It's a 32-bit field which isn't big enough to store 
0x100000000 or 0x200000000 which we need for the two larger 
configurations.  Just changing it to a 64-bit field introduces problems 
with other code, so we're looking at using more than one NAND controller 
in our FPGA and concatenating the NAND chips at the MTD level.

Anyway, thanks for the tip on the NAND tests.

Bruce

Re: Question about ubimkvol vs. mkfs.ubifs

[Artem Bityutskiy]

On Mon, 2008-04-07 at 13:07 -0700, Bruce_Leonard@selinc.com wrote:
> > 
> > Is it a secret what is your flash and what is its size? If it is NAND
> > I'd recommend you to test it with Adrian's NAND tests:
> > git://infradead.org/~ahunter/nand-tests.git
> > 
> Nope, no secret.  We're using Samsung K9WAG08U1A NAND parts in 2GiB, 4GiB, 
> and 8GiB configurations.  At least we're hoping to get the 4 and 8GiB 
> configurations working.
If you are thinking about using UBI, it might be not fast enough if you
have slow I/O which is usually the case if there is not NAND controller.
But anyway, I would be very interested to know UBI attach time for your
flash.

>   I've run into a problem in the MTD layer with the 
> size field.  It's a 32-bit field which isn't big enough to store 
> 0x100000000 or 0x200000000 which we need for the two larger 
> configurations.  Just changing it to a 64-bit field introduces problems 
> with other code, so we're looking at using more than one NAND controller 
> in our FPGA and concatenating the NAND chips at the MTD level.

Well, I guess it should not be too difficult to fix these 64 bit issues.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Question about ubimkvol vs. mkfs.ubifs

[Bruce_Leonard]

> > > 
> > Nope, no secret.  We're using Samsung K9WAG08U1A NAND parts in 2GiB, 
4GiB, 
> > and 8GiB configurations.  At least we're hoping to get the 4 and 8GiB 
> > configurations working.
> If you are thinking about using UBI, it might be not fast enough if you
> have slow I/O which is usually the case if there is not NAND controller.
> But anyway, I would be very interested to know UBI attach time for your
> flash.

We're using a custom built PCI NAND controller.  Currently for the 2GiB 
case the mount time is about (I haven't timed it exactly) 3 seconds so it 
should be on the order of 12 seconds for the 8G case.  Which is acceptable 
for us because in the application we're putting this in boots will be rare 
(i.e., years between boots).

> 
> >   I've run into a problem in the MTD layer with the 
> > size field.  It's a 32-bit field which isn't big enough to store 
> > 0x100000000 or 0x200000000 which we need for the two larger 
> > configurations.  Just changing it to a 64-bit field introduces 
problems 
> > with other code, so we're looking at using more than one NAND 
controller 
> > in our FPGA and concatenating the NAND chips at the MTD level.
> 
> Well, I guess it should not be too difficult to fix these 64 bit issues.
> 

One would think so, but I keep running into division and modulo problems 
(because doing 64-bit division in the kernel is strongly frowned upon, for 
good reasons), and once I work through all those issues I get seg faults, 
etc, etc, etc.  You know...the usual development things ;).

I'll try and remember to get some exact mount times for you when we get 
things a bit more stable.

Thanks for the info.

See 'ya!

Bruce

Re: Question about ubimkvol vs. mkfs.ubifs

[Artem Bityutskiy]

On Tue, 2008-04-08 at 09:45 -0700, Bruce_Leonard@selinc.com wrote:
> One would think so, but I keep running into division and modulo problems 
> (because doing 64-bit division in the kernel is strongly frowned upon, for 
> good reasons), and once I work through all those issues I get seg faults, 
> etc, etc, etc.  You know...the usual development things ;).

Yeah. It will take time. It's shame MTD still has this limitation.

> I'll try and remember to get some exact mount times for you when we get 
> things a bit more stable.

I'd appreciate, thanks.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Question about ubimkvol vs. mkfs.ubifs

[Artem Bityutskiy]

On Tue, 2008-04-08 at 09:45 -0700, Bruce_Leonard@selinc.com wrote:
> One would think so, but I keep running into division and modulo problems 
> (because doing 64-bit division in the kernel is strongly frowned upon, for 
> good reasons), and once I work through all those issues I get seg faults, 
> etc, etc, etc.  You know...the usual development things ;).

Hmm. Do you divide 64-bit offset on eraseblock size? Could generally
make sense to change MTD interfaces to accept eraseblock:offset pair
instead?

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)