From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.nokia.com ([192.100.122.233] helo=mgw-mx06.nokia.com) by bombadil.infradead.org with esmtps (Exim 4.69 #1 (Red Hat Linux)) id 1MQetQ-0008Se-2B for linux-mtd@lists.infradead.org; Tue, 14 Jul 2009 10:05:54 +0000 Subject: Re: UBI resizing From: Artem Bityutskiy To: Jelle Martijn Kok In-Reply-To: <4A5B4EC7.9010605@youcom.nl> References: <4A5B4EC7.9010605@youcom.nl> Content-Type: multipart/alternative; boundary="=-WHAxrZtp6yT9nun7RA0j" Date: Tue, 14 Jul 2009 13:05:35 +0300 Message-Id: <1247565935.3828.136.camel@localhost.localdomain> Mime-Version: 1.0 Cc: linux-mtd@lists.infradead.org Reply-To: dedekind@infradead.org List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , --=-WHAxrZtp6yT9nun7RA0j Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Hi, On Mon, 2009-07-13 at 17:12 +0200, Jelle Martijn Kok wrote: > I have made an "ubirsvol" application (actually it are just some very > simple changes to your ubimkvol/ubirmvol). If you would like to have it, > I could sent it. Sure, why not to have it in ubi-utils? > Question 1: Why do UBI volumes have fixed sizes ? Hmm, we just designed it this way. We wanted to guarantee that the total about of LEBs on UBI device is not larger that the amount of available PEB, so that you could write data to all LEBs. > As far as I can see, all unused PEBs only have their EC headers are > written in the MTD. The volume headers are left unwritten. Right. > Only PEBs which are attached to a specific volume have their volume > header set. Yes. Some LEBs are mapped and contain data, so the corresponding PEBs have the VID header. The unmapped LEBs are not associated with any PEBs. So there are PEBs which do not have VID header. But UBI guarantees that all LEBs may be written to at the same time - there are enough PEBs reserved. > Is it correct that all unused PEBS are in a pool for the whole ubi > device, and not in a pool per ubi volume ? Correct. Every PEB may be used for any LEB in any volume. When you write to an un-mapped LEB, the wear-levelling subsystem will pick an appropriate PEB. > I do understand that this might make it much harder for UBIFS to > determine its available space as it could actually mean that two UBIFS > filesystems which are mounted on the same ubi (eg. ubi0_0 and ubi0_1) > might see the same amount of available LEBs, and thus UBIFS shows twice > as much space. However this might be very similar as mounting the same > harddrive on your system twice. UBIFS really needs to know size of the volume it sits on. UBIFS has per-LEB data (lprops, or LEB properties) stored in LPT (lprops tree). The size of the tree depends on volume size. And we reserve certain amount of LEBs for LPT at the beginning of the volume. But what exactly would you like to have? Would you like to have UBI volumes with no size at all - unbounded / unlimited? Or you would like to have UBI volumes with size, but larger than the amount of available PEBs? > Question 2: Can UBIFS handle UBI resizes easier ? Somewhat, but we did not pay much attention to this feature, because we did not need it. In UBIFS we have LPT which is a B-tree. We try to keep it as small as possible, because it is changed a lot. The tree nodes are packed very nicely, so that we use bit-fields as pointers there. UBIFS has to reserve some space for this tree. And the amount of space it reserves limits the LPT limits its maximum size. Here is text which tells about this a bit. http://www.linux-mtd.infradead.org/faq/ubifs.html#L_max_leb_cnt Note, small LPT area also means faster LPT handling, because in this case the keys in the LPT nodes are shorter. > I have not looked at all at the inner working of UBIFS, so I do not dare > to say anything sensible. Well, in short: * it is easy to enlarge UBIFS. Just select proper -c option. * to shrink UBIFS, you need a special user-space utility, or you need to implement a special UBIFS "shrink" ioctl. When shrinking, you need to ask UBIFS to move data from last LEBs closer to the beginning of the volume (or do this with a user-space utility, but this is more difficult to implement). > However making the UBI volume smaller, makes UBIFS complain that the > amount of LEBs is incorrect. Yes, because UBIFS stores the volume size in the super-block. And actually in master node. For sure this redundancy is insane, and I wanted to fix this, but then I realized it is too late to change the on-flash format. Anyway, a small ioctl which asks UBIFS to shrink needs to be done. But I think it must not be too difficult to implement. It is about: 1. Garbage-collecting as much as possible to get rid of dirty space. 2. When garbage collect, do this a little differently than usually, and you should move data to the beginning. Even LEBs with no dirt but which sit at the end should be moved to the beginning with GC. I mean, most of the code to do this is there already. > I encountered (in ubidesign.pdf) "Note, that by default UBI will not > reduce the size of dynamic volumes unless the number range of logical > erase blocks which will be removed from the volume contains only unused > blocks. A specical parameter allows to override this default.". > What is this "special parameter" ? I'm not sure. Probably Adrian knows? > If UBI (and UBIFS) sizes would become non-fixed, it would prevent the > need for resizing at all. Not sure what exactly you mean. UBI volumes are re-sizable. UBIFS may be shrinked with some tricks. > It might ease the usage of ubi, for example: > - When creating ubi images (for production) the ubi size does not > matter. (UBI_VTBL_AUTORESIZE_FLG could ultimately be removed) > - ubimkvol, can be called without specifying any size. > - no more wasted space in static volumes. > - bad blocks may be handled easier, as it can simply access the complete > pool. No block have to be reserved on front (UBI might however best not > hand out those last 10 PEBs) Still not sure how would this all work. What df would show? If we do not guarantee we have enough PEBs for each LEB, so UBI would basically does overcommitments, how do we handle situations when users want to write more but we do not have PEBs left? > We currently have the following (working) setup (similar like > ubidesign.pdf - chapter 8): > - We are using a 1024 blocks 128MB nand flash. > - MTD in 2 partitions > - MTD0 contains the bootloader (1 block) > - MTD1 contains 1023 blocks in ubi0 (actually 1022 blocks as we are > blessed with a bad erase block) Well, on other silicons you may be blessed with more :-) > - ubi0 contains 2 volumes. > - ubi0_0 which is the root file system (UBIFS on a dynamic volume) > - ubi0_1 is the kernel (static volume) > - The bootloader loads the kernel from the mtd (ubi0). Note: it > explicitly uses the volume name ("kernel") and not the volume_id to find > the static volume containg So your boot-loader understands UBI? > However the limitations currently are: > - I cannot place the kernel in the rootfs as I do under no circumstance > want to implement UBIFS in the bootloader. Well, u-boot has R/O UBIFS support. But they basically copied UBIFS from the kernel. Implementing this from scratch is not easy. > - I do not simply want to run ubiupdatevol on the kernel volume (ubi0_1) > as this might leave the kernel unusable > - For this to be safely possible I want to write the new kernel into > ubi0_2 (named: "new"). > - I could then do an (atomic) "ubirename /dev/ubi0 new kernel kernel > old" and the new kernel is active > - I do not wish to have that volume always present (and waste space) > - I do at this moment not yet know the size of future kernels. So you want atomic volume upgrade. You could implement a new ioctl for this and do something like: create a special temporary throw-away volume, write stuff there, then re-name it to an existing one. I think this is doable. But of course, you would need to carefully think about accounting: if the volume becomes larger after upgrade, you need to make sure you have spare PEBs. -- Best regards, Artem Bityutskiy (Битюцкий Артём) --=-WHAxrZtp6yT9nun7RA0j Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit Hi,

On Mon, 2009-07-13 at 17:12 +0200, Jelle Martijn Kok wrote: 
I have made an "ubirsvol" application (actually it are just some very 
simple changes to your ubimkvol/ubirmvol). If you would like to have it, 
I could sent it.

Sure, why not to have it in ubi-utils?

Question 1: Why do UBI volumes have fixed sizes ?

Hmm, we just designed it this way. We wanted to guarantee that the total
about of LEBs on UBI device is not larger that the amount of available
PEB, so that you could write data to all LEBs.

As far as I can see, all unused PEBs only have their EC headers are 
written in the MTD. The volume headers are left unwritten.

Right.

Only PEBs which are attached to a specific volume have their volume 
header set.

Yes. Some LEBs are mapped and contain data, so the corresponding PEBs
have the VID header. The unmapped LEBs are not associated with any
PEBs. So there are PEBs which do not have VID header.

But UBI guarantees that all LEBs may be written to at the same time -
there are enough PEBs reserved.

Is it correct that all unused PEBS are in a pool for the whole ubi 
device, and not in a pool per ubi volume ?

Correct. Every PEB may be used for any LEB in any volume. When you write
to an un-mapped LEB, the wear-levelling subsystem will pick an
appropriate PEB.

I do understand that this might make it much harder for UBIFS to 
determine its available space as it could actually mean that two UBIFS 
filesystems which are mounted on the same ubi (eg. ubi0_0 and ubi0_1) 
might see the same amount of available LEBs, and thus UBIFS shows twice 
as much space. However this might be very similar as mounting the same 
harddrive on your system twice.

UBIFS really needs to know size of the volume it sits on. UBIFS has
per-LEB data (lprops, or LEB properties) stored in LPT (lprops tree).
The size of the tree depends on volume size. And we reserve certain
amount of LEBs for LPT at the beginning of the volume.

But what exactly would you like to have? Would you like to have UBI
volumes with no size at all - unbounded / unlimited? Or you would like
to have UBI volumes with size, but larger than the amount of available
PEBs?

Question 2: Can UBIFS handle UBI resizes easier ?

Somewhat, but we did not pay much attention to this feature, because
we did not need it.

In UBIFS we have LPT which is a B-tree. We try to keep it as small as
possible, because it is changed a lot. The tree nodes are packed very
nicely, so that we use bit-fields as pointers there. UBIFS has to
reserve some space for this tree. And the amount of space it reserves
limits the LPT limits its maximum size. Here is text which tells
about this a bit.

http://www.linux-mtd.infradead.org/faq/ubifs.html#L_max_leb_cnt

Note, small LPT area also means faster LPT handling, because in this
case the keys in the LPT nodes are shorter.

I have not looked at all at the inner working of UBIFS, so I do not dare 
to say anything sensible.

Well, in short:

  * it is easy to enlarge UBIFS. Just select proper -c option.
  * to shrink UBIFS, you need a special user-space utility, or you
    need to implement a special UBIFS "shrink" ioctl. When shrinking,
    you need to ask UBIFS to move data from last LEBs closer to the
    beginning of the volume (or do this with a user-space utility,
    but this is more difficult to implement).

However making the UBI volume smaller, makes UBIFS complain that the 
amount of LEBs is incorrect.

Yes, because UBIFS stores the volume size in the super-block. And actually in master node. For sure this redundancy is insane, and
I wanted to fix this, but then I realized it is too late to change
the on-flash format.

Anyway, a small ioctl which asks UBIFS to shrink needs to be done.
But I think it must not be too difficult to implement. It is about:

1. Garbage-collecting as much as possible to get rid of dirty space.
2. When garbage collect, do this a little differently than usually,
   and you should move data to the beginning. Even LEBs with no dirt
   but which sit at the end should be moved to the beginning with GC.

I mean, most of the code to do this is there already.

I encountered (in ubidesign.pdf) "Note, that by default UBI will not 
reduce the size of dynamic volumes unless the number range of logical 
erase blocks which will be removed from the volume contains only unused 
blocks. A specical parameter allows to override this default.".
What is this "special parameter" ?

I'm not sure. Probably Adrian knows?

If UBI (and UBIFS) sizes would become non-fixed, it would prevent the 
need for resizing at all.

Not sure what exactly you mean. UBI volumes are re-sizable. UBIFS may
be shrinked with some tricks.

It might ease the usage of ubi, for example:
- When creating ubi images (for production) the ubi size does not 
matter. (UBI_VTBL_AUTORESIZE_FLG could ultimately be removed)
- ubimkvol, can be called without specifying any size.
- no more wasted space in static volumes.
- bad blocks may be handled easier, as it can simply access the complete 
pool. No block have to be reserved on front (UBI might however best not 
hand out those last 10 PEBs)

Still not sure how would this all work. What df would show? If we do not
guarantee we have enough PEBs for each LEB, so UBI would basically does
overcommitments, how do we handle situations when users want to write
more but we do not have PEBs left?

We currently have the following (working) setup (similar like 
ubidesign.pdf - chapter 8):
- We are using a 1024 blocks 128MB nand flash.
- MTD in 2 partitions
    - MTD0 contains the bootloader (1 block)
    - MTD1 contains 1023 blocks in ubi0 (actually 1022 blocks as we are 
blessed with a bad erase block)

Well, on other silicons you may be blessed with more :-)

- ubi0 contains 2 volumes.
    - ubi0_0 which is the root file system (UBIFS on a dynamic volume)
    - ubi0_1 is the kernel (static volume)
- The bootloader loads the kernel from the mtd (ubi0). Note: it 
explicitly uses the volume name ("kernel") and not the volume_id to find 
the static volume containg

So your boot-loader understands UBI?

However the limitations currently are:
- I cannot place the kernel in the rootfs as I do under no circumstance 
want to implement UBIFS in the bootloader.

Well, u-boot has R/O UBIFS support. But they basically copied UBIFS
from the kernel. Implementing this from scratch is not easy.

- I do not simply want to run ubiupdatevol on the kernel volume (ubi0_1) 
as this might leave the kernel unusable
- For this to be safely possible I want to write the new kernel into 
ubi0_2 (named: "new").
- I could then do an (atomic) "ubirename /dev/ubi0 new kernel kernel 
old" and the new kernel is active
- I do not wish to have that volume always present (and waste space)
- I do at this moment not yet know the size of future kernels.
So you want atomic volume upgrade. You could implement a new ioctl for
this and do something like: create a special temporary throw-away volume, write stuff there, then re-name it to an existing one. I think
this is doable. But of course, you would need to carefully think about
accounting: if the volume becomes larger after upgrade, you need to make
sure you have spare PEBs.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)
--=-WHAxrZtp6yT9nun7RA0j--