ubifs mount size

ubifs mount size

[dfoley]

Does this look right (info below) ?
I create a 55MiB volume, but df only shows 33968KiB
This is linux-2.6.24.4. I've also used linux-2.6.27-rc3.


root@tsi-sa2410:~# ubimkvol /dev/ubi1 -N root -s 55MiB
Volume ID 0, size 3755 LEBs (57676800 bytes, 55.0 MiB), LEB size 15360 
bytes (15.0 KiB), dynamic, name "root", alignment 1
root@tsi-sa2410:~# mount -t ubifs ubi1:root /mnt/linux2-root/
UBIFS: default file-system created
UBIFS: background thread "ubifs_bgt1_0" started, PID 930
UBIFS: mounted UBI device 1, volume 0, name "root"
UBIFS: file system size: 57292800 bytes (55950 KiB, 54 MiB, 3730 LEBs)
UBIFS: journal size: 2872320 bytes (2805 KiB, 2 MiB, 187 LEBs)
UBIFS: default compressor: LZO
UBIFS: media format 4, latest format 4
root@tsi-sa2410:~# df
Filesystem           1k-blocks      Used Available Use% Mounted on
rootfs                    8059      5284      2366  69% /
/dev/root                 8059      5284      2366  69% /
/dev/root                 8059      5284      2366  69% /dev/.static/dev
ubi1:root                33968         0     31204   0% /mnt/linux2-root

Re: ubifs mount size

[Adrian Hunter]

dfoley wrote:
> Does this look right (info below) ?
> I create a 55MiB volume, but df only shows 33968KiB
> This is linux-2.6.24.4. I've also used linux-2.6.27-rc3.
> 
> 
> root@tsi-sa2410:~# ubimkvol /dev/ubi1 -N root -s 55MiB
> Volume ID 0, size 3755 LEBs (57676800 bytes, 55.0 MiB), LEB size 15360 
> bytes (15.0 KiB), dynamic, name "root", alignment 1
> root@tsi-sa2410:~# mount -t ubifs ubi1:root /mnt/linux2-root/
> UBIFS: default file-system created
> UBIFS: background thread "ubifs_bgt1_0" started, PID 930
> UBIFS: mounted UBI device 1, volume 0, name "root"
> UBIFS: file system size: 57292800 bytes (55950 KiB, 54 MiB, 3730 LEBs)
> UBIFS: journal size: 2872320 bytes (2805 KiB, 2 MiB, 187 LEBs)
> UBIFS: default compressor: LZO
> UBIFS: media format 4, latest format 4
> root@tsi-sa2410:~# df
> Filesystem           1k-blocks      Used Available Use% Mounted on
> rootfs                    8059      5284      2366  69% /
> /dev/root                 8059      5284      2366  69% /
> /dev/root                 8059      5284      2366  69% /dev/.static/dev
> ubi1:root                33968         0     31204   0% /mnt/linux2-root

Yes this looks right.  UBIFS doesn't really know how much data can be fitted
in the file system, so it makes a worst case estimate.

Re: ubifs mount size

[Artem Bityutskiy]

Hi,

On Mon, 2008-08-18 at 09:36 +0300, Adrian Hunter wrote:
> dfoley wrote:
> > Does this look right (info below) ?
> > I create a 55MiB volume, but df only shows 33968KiB
> > This is linux-2.6.24.4. I've also used linux-2.6.27-rc3.
> > 
> > 
> > root@tsi-sa2410:~# ubimkvol /dev/ubi1 -N root -s 55MiB
> > Volume ID 0, size 3755 LEBs (57676800 bytes, 55.0 MiB), LEB size 15360 
> > bytes (15.0 KiB), dynamic, name "root", alignment 1
> > root@tsi-sa2410:~# mount -t ubifs ubi1:root /mnt/linux2-root/
> > UBIFS: default file-system created
> > UBIFS: background thread "ubifs_bgt1_0" started, PID 930
> > UBIFS: mounted UBI device 1, volume 0, name "root"
> > UBIFS: file system size: 57292800 bytes (55950 KiB, 54 MiB, 3730 LEBs)
> > UBIFS: journal size: 2872320 bytes (2805 KiB, 2 MiB, 187 LEBs)
> > UBIFS: default compressor: LZO
> > UBIFS: media format 4, latest format 4
> > root@tsi-sa2410:~# df
> > Filesystem           1k-blocks      Used Available Use% Mounted on
> > rootfs                    8059      5284      2366  69% /
> > /dev/root                 8059      5284      2366  69% /
> > /dev/root                 8059      5284      2366  69% /dev/.static/dev
> > ubi1:root                33968         0     31204   0% /mnt/linux2-root
> 
> Yes this looks right.  UBIFS doesn't really know how much data can be fitted
> in the file system, so it makes a worst case estimate.

In practice you will be able to fit more than what df shows. I will
write a corresponding FAQ entry which will explain the issue later.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: ubifs mount size

[Bruce_Leonard]

> > 
> > Yes this looks right.  UBIFS doesn't really know how much data canbe 
fitted
> > in the file system, so it makes a worst case estimate.
> 
> In practice you will be able to fit more than what df shows. I will
> write a corresponding FAQ entry which will explain the issue later.
> 

Oh good.  I finally have my 8GiB problem solved (patch forthcoming in the 
next 24 hours probably), but df -h was only showing 6.8GiB and I couldn't 
account for the descrpancy.  I'm looking forward to being able to do that.

Thanks for all the hard work on UBI/UBIFS Artem.  It's working like a 
champ so far.  I'm planning on setting LTP up to run for a couple of weeks 
to really stress it out and I'll let you know how that goes.

FYI, it takes about 45 seconds for the UBI module to load for an 8GiB 
device when I do it by hand.  I think things were a bit faster when UBI is 
built into the kernel.  Once I get that working again I'll update you.

Bruce

Re: ubifs mount size

[Artem Bityutskiy]

Hello dfoley, Bruce,

On Fri, 2008-08-15 at 21:16 -0700, dfoley wrote:
> Does this look right (info below) ?
> I create a 55MiB volume, but df only shows 33968KiB
> This is linux-2.6.24.4. I've also used linux-2.6.27-rc3.
> 
> 
> root@tsi-sa2410:~# ubimkvol /dev/ubi1 -N root -s 55MiB
> Volume ID 0, size 3755 LEBs (57676800 bytes, 55.0 MiB), LEB size 15360 
> bytes (15.0 KiB), dynamic, name "root", alignment 1
> root@tsi-sa2410:~# mount -t ubifs ubi1:root /mnt/linux2-root/
> UBIFS: default file-system created
> UBIFS: background thread "ubifs_bgt1_0" started, PID 930
> UBIFS: mounted UBI device 1, volume 0, name "root"
> UBIFS: file system size: 57292800 bytes (55950 KiB, 54 MiB, 3730 LEBs)
> UBIFS: journal size: 2872320 bytes (2805 KiB, 2 MiB, 187 LEBs)
> UBIFS: default compressor: LZO
> UBIFS: media format 4, latest format 4
> root@tsi-sa2410:~# df
> Filesystem           1k-blocks      Used Available Use% Mounted on
> rootfs                    8059      5284      2366  69% /
> /dev/root                 8059      5284      2366  69% /
> /dev/root                 8059      5284      2366  69% /dev/.static/dev
> ubi1:root                33968         0     31204   0% /mnt/linux2-root

I've explored this a little. Currently UBIFS indeed reports 15%-20% less
free space than it actually has, and we'll try to fix this.

However, even if we fix this, it'll still report less free space than it
has. I have started writing about this here:
http://www.linux-mtd.infradead.org/doc/ubifs.html

I did not finish my writing yet, and I did not actually write the stuff
which is mostly relevant for your situation. But I wrote other
df-related stuff which UBIFS users should probably know about.

One thing I may say about your setup is that you have small eraseblock
size (16KiB) which leads to large mistakes in calculations because the
maximum amount of wasted space at the end of eraseblock is large in your
case.

On Mon, 2008-08-18 at 16:32 -0700, Bruce_Leonard@selinc.com wrote:
> Oh good. I finally have my 8GiB problem solved (patch forthcoming in
> the 
> next 24 hours probably), but df -h was only showing 6.8GiB and I
> couldn't 
> account for the descrpancy. I'm looking forward to being able to do
> that.

Similarly, we will try to make improve the free space reporting, but
it'll lie anyway, but less.

> Thanks for all the hard work on UBI/UBIFS Artem. It's working like a 
> champ so far. I'm planning on setting LTP up to run for a couple of
> weeks 
> to really stress it out and I'll let you know how that goes.

Thank you. But I was not the only person working on UBIFS. Adrian
implemented bigger part of it than me.

Stress tests are appreciated and posting the results here is welcomed.

> FYI, it takes about 45 seconds for the UBI module to load for an 8GiB 
> device when I do it by hand. I think things were a bit faster when UBI
> is 
> built into the kernel. Once I get that working again I'll update you.

Yeah, bad. One straight way to make it twice as fast is to merge 128KiB
physical eraseblocks to 265KiB ones. Then UBI would read twice as less.
And I wouldn't be surprised if UBIFS would become faster.

I mean, you could hack your NAND driver and teach it to join 2 physical
eraseblocks into one, and make all upper layers think you have 256KiB
eraseblocks. Things would become tricky for bad blocks, and you
effectively would treat good PEBs as bad if their pair is bad. But is
probably is not so bad, comparing to twice as faster UBI attach time.
And you would win the space you wasted because of bad blocks by wasting
less space in both UBI and UBIFS.


-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: ubifs mount size

[Artem Bityutskiy]

On Thu, 2008-08-21 at 20:24 +0300, Artem Bityutskiy wrote:
> However, even if we fix this, it'll still report less free space than it
> has. I have started writing about this here:
> http://www.linux-mtd.infradead.org/doc/ubifs.html

As usually, I apologize for my English and appreciate any spelling/bad
phrasing fixes.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: ubifs mount size

[Artem Bityutskiy]

Hi,

On Thu, 2008-08-21 at 20:24 +0300, Artem Bityutskiy wrote:
> > root@tsi-sa2410:~# ubimkvol /dev/ubi1 -N root -s 55MiB
> > Volume ID 0, size 3755 LEBs (57676800 bytes, 55.0 MiB), LEB size 15360 
> > bytes (15.0 KiB), dynamic, name "root", alignment 1
> > root@tsi-sa2410:~# mount -t ubifs ubi1:root /mnt/linux2-root/
> > UBIFS: default file-system created
> > UBIFS: background thread "ubifs_bgt1_0" started, PID 930
> > UBIFS: mounted UBI device 1, volume 0, name "root"
> > UBIFS: file system size: 57292800 bytes (55950 KiB, 54 MiB, 3730 LEBs)
> > UBIFS: journal size: 2872320 bytes (2805 KiB, 2 MiB, 187 LEBs)
> > UBIFS: default compressor: LZO
> > UBIFS: media format 4, latest format 4
> > root@tsi-sa2410:~# df
> > Filesystem           1k-blocks      Used Available Use% Mounted on
> > rootfs                    8059      5284      2366  69% /
> > /dev/root                 8059      5284      2366  69% /
> > /dev/root                 8059      5284      2366  69% /dev/.static/dev
> > ubi1:root                33968         0     31204   0% /mnt/linux2-root
> 
> I've explored this a little. Currently UBIFS indeed reports 15%-20% less
> free space than it actually has, and we'll try to fix this.

We have improved UBIFS free space reporting and it should be much better
now. Please, pull master branch of the ubifs-2.6.git tree to find the
fixes.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: ubifs mount size

[Dallas Foley]

Artem Bityutskiy wrote:
> Hi,
> 
> On Thu, 2008-08-21 at 20:24 +0300, Artem Bityutskiy wrote:
>>> root@tsi-sa2410:~# ubimkvol /dev/ubi1 -N root -s 55MiB
>>> Volume ID 0, size 3755 LEBs (57676800 bytes, 55.0 MiB), LEB size 15360 
>>> bytes (15.0 KiB), dynamic, name "root", alignment 1
>>> root@tsi-sa2410:~# mount -t ubifs ubi1:root /mnt/linux2-root/
>>> UBIFS: default file-system created
>>> UBIFS: background thread "ubifs_bgt1_0" started, PID 930
>>> UBIFS: mounted UBI device 1, volume 0, name "root"
>>> UBIFS: file system size: 57292800 bytes (55950 KiB, 54 MiB, 3730 LEBs)
>>> UBIFS: journal size: 2872320 bytes (2805 KiB, 2 MiB, 187 LEBs)
>>> UBIFS: default compressor: LZO
>>> UBIFS: media format 4, latest format 4
>>> root@tsi-sa2410:~# df
>>> Filesystem           1k-blocks      Used Available Use% Mounted on
>>> rootfs                    8059      5284      2366  69% /
>>> /dev/root                 8059      5284      2366  69% /
>>> /dev/root                 8059      5284      2366  69% /dev/.static/dev
>>> ubi1:root                33968         0     31204   0% /mnt/linux2-root
>> I've explored this a little. Currently UBIFS indeed reports 15%-20% less
>> free space than it actually has, and we'll try to fix this.
> 
> We have improved UBIFS free space reporting and it should be much better
> now. Please, pull master branch of the ubifs-2.6.git tree to find the
> fixes.
> 
Yes, looks good now.

I'm now using a different size nand device (128MiB, 2KiB page size). 2 
partitions in the driver as (16MiB, and 112MiB), which I've just mapped 
1:1 with ubimkvol (ie. ubimkvol /dev/ubi0 -N boot -m and
ubimkvol /dev/ubi1 -N root -m)

root@Linux tsi-tpcxx 2.6.27-rc5-next-20080905 #1 Wed Sep 10 02:22:05 UTC

2008 armv4tl unknowntsi-tpcxx:~# df
Filesystem           1k-blocks      Used Available Use% Mounted on
rootfs                  102744     13096     84588  13% /
ubi1:root               102744     13096     84588  13% /
ubi0:boot                12916      1692     10520  14% /boot