Static UBI volumes and ubiupdatevol ?

Static UBI volumes and ubiupdatevol ?

[Ricard Wanderlof]

Hi,

I've looked through the documentation on 
http://www.linux-mtd.infradead.org but I can't find the answer to the 
following question(s):

Using 'static' UBI volumes seems to be the preferred way of creating read 
only file systems.

While it seems possible to use ubinize to make a complete ubi image 
containing static volumes and then write it to the flash using ubiformat, 
is it possible to update just a single volume using ubiupdatevol?

I've tried using ubimkvol to create a static volume, then using 
ubiupdatevol to write an ubifs image to it. That works, but when I try to 
mount it subsequently I get things like

[ 6656.114000] UBIFS: static UBI volume - read-only mode
[ 6656.129000] UBIFS error (pid 866): mount_ubifs: can't format empty UBI 
volume: read-only UBI volume
mount: mounting ubi1_0 on /volume1/ failed: Read-only file system

which in a way makes sense to me; if the volume is static = read only then 
it shouldn't be possible to write to it.

The only thing I haven't tried is matching the size of the image I'm 
writing to the size of the partition. With dynamic volumes that doesn't 
seem to be an issue; as long as there is space for the data in the target 
volume. The free space just seems to get truncated to the available free 
space in the flash volume. (On the other hand, writing an image that is 
smaller than the flash volume doesn't seem to 'grow' to fit the available 
space according to df; it remains the same no matter how large the target 
volume is). So I've been rather careless with the size of the image vs. 
the size of the target volume.

A related question: With static volumes, I assume you still get the 
benefit of scrubbing, which in turn means that UBI can move the data 
around if it deems necessary?

Which begs a final question: is it possible to verify the integrity of a 
static UBI volume? Even with JFFS2, if mounted readonly the flash in that 
particular partition is never written to, so it is possible to use a 
simple checksum to verify data integrity. With JFFS2 this can be 
important, as missing JFFS2 nodes don't necessarily result in mount time 
or run time failures, it just results in missing files or parts of files.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Static UBI volumes and ubiupdatevol ?

[Ricard Wanderlof]

On Wed, 15 Dec 2010, Ricard Wanderlof wrote:

> I've tried using ubimkvol to create a static volume, then using
> ubiupdatevol to write an ubifs image to it. That works, but when I try to
> mount it subsequently I get things like
>
> [ 6656.114000] UBIFS: static UBI volume - read-only mode
> [ 6656.129000] UBIFS error (pid 866): mount_ubifs: can't format empty UBI
> volume: read-only UBI volume
> mount: mounting ubi1_0 on /volume1/ failed: Read-only file system

I don't know if I screwed up big time when I was trying this out before, 
it seems to work fine now just as well as with dynamic volumes. I don't 
know if I was doing ubiupdatevol to the wrong volume (which would explain 
the above error message) or what.

I'd still like to get some input on the other two questions in my previous 
email though...

> A related question: With static volumes, I assume you still get the
> benefit of scrubbing, which in turn means that UBI can move the data
> around if it deems necessary?
>
> Which begs a final question: is it possible to verify the integrity of a
> static UBI volume? Even with JFFS2, if mounted readonly the flash in that
> particular partition is never written to, so it is possible to use a
> simple checksum to verify data integrity. With JFFS2 this can be
> important, as missing JFFS2 nodes don't necessarily result in mount time
> or run time failures, it just results in missing files or parts of files.

I'm also wondering if anyone has any thoughts on upgrading a 'live' 
system, when the flash is managed by UBI.

With a system based on mtd+jffs2 it is actually possible to reflash the 
partition with the [read only] root file system on it without unmounting, 
as long as no accesses are made to the physical flash during the rewrite 
or afterwards. In other words, immediately after reflashing the system 
must be rebooted.

However, with UBI, given that there are background threads which might be 
doing garbage collection etc I'm suspecting such a solution might not be 
as suitable when the flash is managed by UBI. Or is there a some solution 
that someone has tried with any level of success?

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Static UBI volumes and ubiupdatevol ?

[Artem Bityutskiy]

On Wed, 2010-12-15 at 09:06 +0100, Ricard Wanderlof wrote:
> Hi,
> 
> I've looked through the documentation on 
> http://www.linux-mtd.infradead.org but I can't find the answer to the 
> following question(s):
> 
> Using 'static' UBI volumes seems to be the preferred way of creating read 
> only file systems.

No, I think the _original_ idea behind static volumes was storing some
important r/o blobs there without any FS. So UBI static volumes simplify
this by making the volume size being = the contents size, even if the
contents size is not multiple of LEB size, and static volumes also
protect the data with CRC-32.

But UBIFS has its own CRC32 protection, so does not need UBI static
volume services.

> While it seems possible to use ubinize to make a complete ubi image 
> containing static volumes and then write it to the flash using ubiformat, 
> is it possible to update just a single volume using ubiupdatevol
> 
> I've tried using ubimkvol to create a static volume, then using 
> ubiupdatevol to write an ubifs image to it. That works, but when I try to 
> mount it subsequently I get things like
> 
> [ 6656.114000] UBIFS: static UBI volume - read-only mode
> [ 6656.129000] UBIFS error (pid 866): mount_ubifs: can't format empty UBI 
> volume: read-only UBI volume
> mount: mounting ubi1_0 on /volume1/ failed: Read-only file system
> 
> which in a way makes sense to me; if the volume is static = read only then 
> it shouldn't be possible to write to it.

This should work. You somehow wiped the volume instead of updating it,
may be used the -t option for ubiupdatevol.

> A related question: With static volumes, I assume you still get the 
> benefit of scrubbing, which in turn means that UBI can move the data 
> around if it deems necessary?

Yes, sure.

> Which begs a final question: is it possible to verify the integrity of a 
> static UBI volume?

All eraseblocks are protected with CRC-32. Fist time the static volume
is openend, it is checked. See the 'ubi_check_volume()' function. It is
also checked after it has been updated.

To force checking, you need to detach the whole mtd device, and
re-attach it. Or implement an ioctl for this, which is trivial to do.

>  Even with JFFS2, if mounted readonly the flash in that 
> particular partition is never written to,

The same with UBIFS - we never write when we are mounted R/O. There are
many ubifs_assert()s to check this as well.

>  so it is possible to use a 
> simple checksum to verify data integrity.

You can do this with UBIFS I think, did not try though.

>  With JFFS2 this can be 
> important, as missing JFFS2 nodes don't necessarily result in mount time 
> or run time failures, it just results in missing files or parts of files.

Yeah, I think UBIFS will yell and  switch to R/O mode if one of the
nodes is missing or corrupted.

The difference between JFFS2 and UBIFS is that JFFS2 cannot distinguish
between nodes corrupted because of power cuts, and nodes corrupted
because something bad happened to the flash. UBIFS does better job here
- it recovers only from errors which could possibly be caused by power
cuts, and it switches to R/O mode for other errors.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Static UBI volumes and ubiupdatevol ?

[Artem Bityutskiy]

On Wed, 2010-12-15 at 11:43 +0100, Ricard Wanderlof wrote:
> I'm also wondering if anyone has any thoughts on upgrading a 'live' 
> system, when the flash is managed by UBI.
> 
> With a system based on mtd+jffs2 it is actually possible to reflash the 
> partition with the [read only] root file system on it without unmounting, 
> as long as no accesses are made to the physical flash during the rewrite 
> or afterwards. In other words, immediately after reflashing the system 
> must be rebooted.

Hmm, I presume you can do this with UBIFS, but you need to make sure all
the backgound stuff like moving and eraseing eraseblocs is done before
doing this by doing fsync() on your /dev/ubiX_Y device (UBI volume
character device), see the 'vol_cdev_fsync()' function. Ah, and of
course you need to sync UBIFS before this.

Then if you can guarantee that no one is reading writing to it, you can
change the flash.

Another option is to remount R/O, if you can make sure there are no
processes keeping files open in R/W mode.

> However, with UBI, given that there are background threads which might be 
> doing garbage collection etc I'm suspecting such a solution might not be 
> as suitable when the flash is managed by UBI. Or is there a some solution 
> that someone has tried with any level of success?

Just call sync(), then fsync on the UBI volume, and you won't have
background processes if there is not I/O.

Be beware, even reading from UBIFS can cause scrubbing in UBI.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Static UBI volumes and ubiupdatevol ?

[Ricard Wanderlof]

On Thu, 16 Dec 2010, Artem Bityutskiy wrote:

> > Using 'static' UBI volumes seems to be the preferred way of creating read 
> > only file systems.
> 
> No, I think the _original_ idea behind static volumes was storing some
> important r/o blobs there without any FS. So UBI static volumes simplify
> this by making the volume size being = the contents size, even if the
> contents size is not multiple of LEB size, and static volumes also
> protect the data with CRC-32.

That's good news, because that's what exactly what I'd need.

> But UBIFS has its own CRC32 protection, so does not need UBI static
> volume services.
> [...] 
> All eraseblocks are protected with CRC-32. Fist time the static volume
> is openend, it is checked. See the 'ubi_check_volume()' function. It is
> also checked after it has been updated.

The thing is, I'd like to be able to checksum, or otherwise verify, a 
couple of read-only partitions on the flash _before_ attempting to boot 
Linux. No point in trying to boot Linux if the kernel or the root file 
system is not intact. So although it is nice that UBI verifies static 
volumes during attach, and UBIFS during mount, it is really too late as I 
need to do this in the boot loader.

Of course, extending the bootloader to be able to verify UBI volumes is 
probably doable. I could then use cramfs, squashfs or similar for the root 
file system, with gluebi, and use the static volume UBI CRC32 for 
verification by the bootloader. Makeing the boot loader understand UBIFS I 
think would be too much work.

>> With a system based on mtd+jffs2 it is actually possible to reflash the
>> partition with the [read only] root file system on it without unmounting,
>> as long as no accesses are made to the physical flash during the rewrite
>> or afterwards. In other words, immediately after reflashing the system
>> must be rebooted.
> The same with UBIFS - we never write when we are mounted R/O. There are
> many ubifs_assert()s to check this as well.

... but UBI might still do writes during the course of scrubbing I guess, 
unless ...

> [ ... ] you need to make sure all the backgound stuff like moving and 
> eraseing eraseblocs is done before doing this by doing fsync() on your 
> /dev/ubiX_Y device (UBI volume character device), see the 
> 'vol_cdev_fsync()' function. Ah, and of course you need to sync UBIFS 
> before this.

That would be no problem, it is basically the same stuff we need to do 
with JFFS2 before reflashing.

> >  so it is possible to use a 
> > simple checksum to verify data integrity.
> 
> You can do this with UBIFS I think, did not try though.

What I meant was 'a simple checksum on the raw mtd partition', which can't 
work with UBIFS as the UBI erase counters etc can change over time as a 
result of scrubbing.

I guess I could read from the /dev/ubix_y device to just get the raw UBIFS 
data which shouldn't change even in the event of scrubbing? Or should I 
use gluebi to get both a raw mtd partition corresponding to the raw UBIFS 
data, as well as using UBIFS (on /dev/ubix_y) for the file system?

> The difference between JFFS2 and UBIFS is that JFFS2 cannot distinguish
> between nodes corrupted because of power cuts, and nodes corrupted
> because something bad happened to the flash. UBIFS does better job here
> - it recovers only from errors which could possibly be caused by power
> cuts, and it switches to R/O mode for other errors.
> [...]
> Just call sync(), then fsync on the UBI volume, and you won't have
> background processes if there is not I/O.
>
> Be beware, even reading from UBIFS can cause scrubbing in UBI.

That's very good. My basic problem though is still that I need to verify 
the integrity before booting, although it is very nice that the file 
system can detect errors during mount. With JFFS2 there's always a 
nagging idea that errors will go undetected and we won't notice until a 
file is missing or contains zeros.

I often get complaints from other developers in our company saying "can't 
we suppress these 'jffs2 crc error' messages if they don't actually 
indicate a failure. My answer is always a) We don't know if they indicate 
a failure or not and b) Since mostly they are harmless, come back when you 
have found that a file has disappeared from the file system or other file 
problem. So far no one has come back with any complaints referring to b).

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Static UBI volumes and ubiupdatevol ?

[Ricard Wanderlof]

(Crud, I was trying to concatenate both replies into one message to avoid 
two threads on the same issue but then I thought of something else after 
sending the first one off).

On Thu, 16 Dec 2010, Artem Bityutskiy wrote:

>> With a system based on mtd+jffs2 it is actually possible to reflash the
>> partition with the [read only] root file system on it without unmounting,
>> as long as no accesses are made to the physical flash during the rewrite
>> or afterwards. In other words, immediately after reflashing the system
>> must be rebooted.
>
> Hmm, I presume you can do this with UBIFS, but you need to make sure all
> the backgound stuff like moving and eraseing eraseblocs is done before
> doing this by doing fsync() on your /dev/ubiX_Y device (UBI volume
> character device), see the 'vol_cdev_fsync()' function. Ah, and of
> course you need to sync UBIFS before this.
>
> Then if you can guarantee that no one is reading writing to it, you can
> change the flash.

It would be nice to use ubiupdatevol to do this, but it won't let me do 
that if the volume is mounted (naturally). I don't know if this is 
something that is enforced by ubiupdatevol or ubi. I would assume the 
latter but I haven't checked the code.

Of course I could always write my own "ubiupdatevol" to deal with this but 
it's always nice to use existing and supported tools.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Static UBI volumes and ubiupdatevol ?

[Artem Bityutskiy]

On Fri, 2010-12-17 at 10:33 +0100, Ricard Wanderlof wrote:
> > The same with UBIFS - we never write when we are mounted R/O. There are
> > many ubifs_assert()s to check this as well.
> 
> ... but UBI might still do writes during the course of scrubbing I guess, 
> unless ...

Yes, you are right.

> > >  so it is possible to use a 
> > > simple checksum to verify data integrity.
> > 
> > You can do this with UBIFS I think, did not try though.
> 
> What I meant was 'a simple checksum on the raw mtd partition', which can't 
> work with UBIFS as the UBI erase counters etc can change over time as a 
> result of scrubbing.

Ah, yes, you see, you know the things quite well yourself :-)

> I guess I could read from the /dev/ubix_y device to just get the raw UBIFS 
> data which shouldn't change even in the event of scrubbing? Or should I 
> use gluebi to get both a raw mtd partition corresponding to the raw UBIFS 
> data, as well as using UBIFS (on /dev/ubix_y) for the file system?

No, gluebi is not needed, it would just give you an /dev/mtdZ device
with contents equivalent to /dev/ubiX_Y

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Static UBI volumes and ubiupdatevol ?

[Ricard Wanderlof]

On Fri, 17 Dec 2010, Artem Bityutskiy wrote:

> Ah, yes, you see, you know the things quite well yourself :-)

Well, I try to convince myself that I think I know, but there's always the 
very real and likely possibilty that I'm wrong because I've misunderstood 
something.

>> I guess I could read from the /dev/ubix_y device to just get the raw UBIFS
>> data which shouldn't change even in the event of scrubbing? Or should I
>> use gluebi to get both a raw mtd partition corresponding to the raw UBIFS
>> data, as well as using UBIFS (on /dev/ubix_y) for the file system?
>
> No, gluebi is not needed, it would just give you an /dev/mtdZ device
> with contents equivalent to /dev/ubiX_Y

So what does gluebi in fact add, does it just supply all the necessary 
ioctl's to make /dev/ubiX_Y look like a /dev/mtdZ device, in addition to 
the already existing functionality of being able to read and write to the 
volume using read and write? (Yes, I could dig through the code, but even 
though it might give me the answer on what it looks like right now, it 
might not tell what the intention was...).

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Static UBI volumes and ubiupdatevol ?

[Artem Bityutskiy]

On Fri, 2010-12-17 at 13:31 +0100, Ricard Wanderlof wrote:
> On Fri, 17 Dec 2010, Artem Bityutskiy wrote:
> 
> > Ah, yes, you see, you know the things quite well yourself :-)
> 
> Well, I try to convince myself that I think I know, but there's always the 
> very real and likely possibilty that I'm wrong because I've misunderstood 
> something.
> 
> >> I guess I could read from the /dev/ubix_y device to just get the raw UBIFS
> >> data which shouldn't change even in the event of scrubbing? Or should I
> >> use gluebi to get both a raw mtd partition corresponding to the raw UBIFS
> >> data, as well as using UBIFS (on /dev/ubix_y) for the file system?
> >
> > No, gluebi is not needed, it would just give you an /dev/mtdZ device
> > with contents equivalent to /dev/ubiX_Y
> 
> So what does gluebi in fact add, does it just supply all the necessary 
> ioctl's to make /dev/ubiX_Y look like a /dev/mtdZ device, in addition to 
> the already existing functionality of being able to read and write to the 
> volume using read and write? (Yes, I could dig through the code, but even 
> though it might give me the answer on what it looks like right now, it 
> might not tell what the intention was...).

Original idea was to put JFFS2 on top of UBI volumes. Gluebi is doing
just that - turns UBI volume into an MTD device, then you have mount it
with JFFS2.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Static UBI volumes and ubiupdatevol ?

[Artem Bityutskiy]

On Fri, 2010-12-17 at 10:39 +0100, Ricard Wanderlof wrote:
> (Crud, I was trying to concatenate both replies into one message to avoid 
> two threads on the same issue but then I thought of something else after 
> sending the first one off).
> 
> On Thu, 16 Dec 2010, Artem Bityutskiy wrote:
> 
> >> With a system based on mtd+jffs2 it is actually possible to reflash the
> >> partition with the [read only] root file system on it without unmounting,
> >> as long as no accesses are made to the physical flash during the rewrite
> >> or afterwards. In other words, immediately after reflashing the system
> >> must be rebooted.
> >
> > Hmm, I presume you can do this with UBIFS, but you need to make sure all
> > the backgound stuff like moving and eraseing eraseblocs is done before
> > doing this by doing fsync() on your /dev/ubiX_Y device (UBI volume
> > character device), see the 'vol_cdev_fsync()' function. Ah, and of
> > course you need to sync UBIFS before this.
> >
> > Then if you can guarantee that no one is reading writing to it, you can
> > change the flash.
> 
> It would be nice to use ubiupdatevol to do this, but it won't let me do 
> that if the volume is mounted (naturally). I don't know if this is 
> something that is enforced by ubiupdatevol or ubi. I would assume the 
> latter but I haven't checked the code.
> 
> Of course I could always write my own "ubiupdatevol" to deal with this but 
> it's always nice to use existing and supported tools.

This is UBI - it does not allow updating if someone keeps the volume
open. When UBI is mounted, it keeps the volume open. It closes the
volume when unmounted.

It looks error-prone to allow updating while there are users, from UBI
POW, I think.

And I cannot think of a nice way of doing this.

Why can't you just
1. lazily unmount UBIFS
2. find all users of UBIFS and kill them
3. this will make UBIFS finally unmount
4. you safely update the UBI volume

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Static UBI volumes and ubiupdatevol ?

[Ricard Wanderlof]

On Fri, 17 Dec 2010, Artem Bityutskiy wrote:

>>>> With a system based on mtd+jffs2 it is actually possible to reflash the
>>>> partition with the [read only] root file system on it without unmounting,
>>>> as long as no accesses are made to the physical flash during the rewrite
>>>> or afterwards. In other words, immediately after reflashing the system
>>>> must be rebooted.
>>>
>>> Hmm, I presume you can do this with UBIFS, but you need to make sure all
>>> the backgound stuff like moving and eraseing eraseblocs is done before
>>> doing this by doing fsync() on your /dev/ubiX_Y device (UBI volume
>>> character device), see the 'vol_cdev_fsync()' function. Ah, and of
>>> course you need to sync UBIFS before this.
>>>
>>> Then if you can guarantee that no one is reading writing to it, you can
>>> change the flash.
>>
>> It would be nice to use ubiupdatevol to do this, but it won't let me do
>> that if the volume is mounted (naturally). I don't know if this is
>> something that is enforced by ubiupdatevol or ubi. I would assume the
>> latter but I haven't checked the code.
>>
>> Of course I could always write my own "ubiupdatevol" to deal with this but
>> it's always nice to use existing and supported tools.
>
> This is UBI - it does not allow updating if someone keeps the volume
> open. When UBI is mounted, it keeps the volume open. It closes the
> volume when unmounted.

Yes, it make definitely makes sense to have it work this way to avoid 
problems.

> It looks error-prone to allow updating while there are users, from UBI
> POW, I think.

I agree. It just happens to be the way our firmware update process works 
currently ... :-)

> And I cannot think of a nice way of doing this.
>
> Why can't you just
> 1. lazily unmount UBIFS
> 2. find all users of UBIFS and kill them
> 3. this will make UBIFS finally unmount
> 4. you safely update the UBI volume

The problem is that in this case it is our root file system which is to be 
reflashed, so we can't just unmount it. I have an idea about using 
pivot_root though, in a similar manner to the way the root file system is 
switched from initrd to disk during an ordinary Linux boot with initrd.

Something like
1. Close down everything.
2. Create a tmpfs and mount it.
3. Copy all necessary binaries and libraries to the tmpfs.
4. Use pivot_root to switch root file systems.
5. Unmount the original (UBIFS) root file system.
6. Rewrite the root file system using ubiupdatevol.
7. Reboot.

Right now I'm stumped at how to restart init in order for it to relase its 
stdin,stdout and stderr, which still refer to the (old) /dev/console, but 
there must be a solution somehow as the same situation must occur with an 
ordinary (initrd) kernel booting.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Static UBI volumes and ubiupdatevol ?

[Artem Bityutskiy]

On Fri, 2010-12-17 at 16:02 +0100, Ricard Wanderlof wrote:
> On Fri, 17 Dec 2010, Artem Bityutskiy wrote:
> 
> >>>> With a system based on mtd+jffs2 it is actually possible to reflash the
> >>>> partition with the [read only] root file system on it without unmounting,
> >>>> as long as no accesses are made to the physical flash during the rewrite
> >>>> or afterwards. In other words, immediately after reflashing the system
> >>>> must be rebooted.
> >>>
> >>> Hmm, I presume you can do this with UBIFS, but you need to make sure all
> >>> the backgound stuff like moving and eraseing eraseblocs is done before
> >>> doing this by doing fsync() on your /dev/ubiX_Y device (UBI volume
> >>> character device), see the 'vol_cdev_fsync()' function. Ah, and of
> >>> course you need to sync UBIFS before this.
> >>>
> >>> Then if you can guarantee that no one is reading writing to it, you can
> >>> change the flash.
> >>
> >> It would be nice to use ubiupdatevol to do this, but it won't let me do
> >> that if the volume is mounted (naturally). I don't know if this is
> >> something that is enforced by ubiupdatevol or ubi. I would assume the
> >> latter but I haven't checked the code.
> >>
> >> Of course I could always write my own "ubiupdatevol" to deal with this but
> >> it's always nice to use existing and supported tools.
> >
> > This is UBI - it does not allow updating if someone keeps the volume
> > open. When UBI is mounted, it keeps the volume open. It closes the
> > volume when unmounted.
> 
> Yes, it make definitely makes sense to have it work this way to avoid 
> problems.
> 
> > It looks error-prone to allow updating while there are users, from UBI
> > POW, I think.
> 
> I agree. It just happens to be the way our firmware update process works 
> currently ... :-)
> 
> > And I cannot think of a nice way of doing this.
> >
> > Why can't you just
> > 1. lazily unmount UBIFS
> > 2. find all users of UBIFS and kill them
> > 3. this will make UBIFS finally unmount
> > 4. you safely update the UBI volume
> 
> The problem is that in this case it is our root file system which is to be 
> reflashed, so we can't just unmount it. I have an idea about using 
> pivot_root though, in a similar manner to the way the root file system is 
> switched from initrd to disk during an ordinary Linux boot with initrd.
> 
> Something like
> 1. Close down everything.
> 2. Create a tmpfs and mount it.
> 3. Copy all necessary binaries and libraries to the tmpfs.
> 4. Use pivot_root to switch root file systems.
> 5. Unmount the original (UBIFS) root file system.
> 6. Rewrite the root file system using ubiupdatevol.
> 7. Reboot.
> 
> Right now I'm stumped at how to restart init in order for it to relase its 
> stdin,stdout and stderr, which still refer to the (old) /dev/console, but 
> there must be a solution somehow as the same situation must occur with an 
> ordinary (initrd) kernel booting.

Well, I guess it is possible to teach UBI to do roughly something like
this:

1. block all I/O from upper layers (UBIFS)
2. flush the background works queue, i.e., finish all the pending
srubbings, etc.
3. do the volume update, even though there are users, but they are
blocked.

After this there is no way back, so the users are blocked forever. This
should be interruptable blocking, so users should either interrupt, or
you should reboot.

Should not be too difficult to do, but not sure how upstreamable that
would be, but you can try, why not?

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Static UBI volumes and ubiupdatevol ?

[Russ Dill]

On Fri, Dec 17, 2010 at 8:02 AM, Ricard Wanderlof
<ricard.wanderlof@axis.com> wrote:
>
> On Fri, 17 Dec 2010, Artem Bityutskiy wrote:
>
>>>>> With a system based on mtd+jffs2 it is actually possible to reflash the
>>>>> partition with the [read only] root file system on it without
>>>>> unmounting,
>>>>> as long as no accesses are made to the physical flash during the
>>>>> rewrite
>>>>> or afterwards. In other words, immediately after reflashing the system
>>>>> must be rebooted.
>>>>
>>>> Hmm, I presume you can do this with UBIFS, but you need to make sure all
>>>> the backgound stuff like moving and eraseing eraseblocs is done before
>>>> doing this by doing fsync() on your /dev/ubiX_Y device (UBI volume
>>>> character device), see the 'vol_cdev_fsync()' function. Ah, and of
>>>> course you need to sync UBIFS before this.
>>>>
>>>> Then if you can guarantee that no one is reading writing to it, you can
>>>> change the flash.
>>>
>>> It would be nice to use ubiupdatevol to do this, but it won't let me do
>>> that if the volume is mounted (naturally). I don't know if this is
>>> something that is enforced by ubiupdatevol or ubi. I would assume the
>>> latter but I haven't checked the code.
>>>
>>> Of course I could always write my own "ubiupdatevol" to deal with this
>>> but
>>> it's always nice to use existing and supported tools.
>>
>> This is UBI - it does not allow updating if someone keeps the volume
>> open. When UBI is mounted, it keeps the volume open. It closes the
>> volume when unmounted.
>
> Yes, it make definitely makes sense to have it work this way to avoid
> problems.
>
>> It looks error-prone to allow updating while there are users, from UBI
>> POW, I think.
>
> I agree. It just happens to be the way our firmware update process works
> currently ... :-)
>
>> And I cannot think of a nice way of doing this.
>>
>> Why can't you just
>> 1. lazily unmount UBIFS
>> 2. find all users of UBIFS and kill them
>> 3. this will make UBIFS finally unmount
>> 4. you safely update the UBI volume
>
> The problem is that in this case it is our root file system which is to be
> reflashed, so we can't just unmount it. I have an idea about using
> pivot_root though, in a similar manner to the way the root file system is
> switched from initrd to disk during an ordinary Linux boot with initrd.
>
> Something like
> 1. Close down everything.
> 2. Create a tmpfs and mount it.
> 3. Copy all necessary binaries and libraries to the tmpfs.
> 4. Use pivot_root to switch root file systems.
> 5. Unmount the original (UBIFS) root file system.
> 6. Rewrite the root file system using ubiupdatevol.
> 7. Reboot.
>
> Right now I'm stumped at how to restart init in order for it to relase its
> stdin,stdout and stderr, which still refer to the (old) /dev/console, but
> there must be a solution somehow as the same situation must occur with an
> ordinary (initrd) kernel booting.
>

Busybox init knows how to do this operation. If not, a straightforward
alternative is to kexec your kernel with a small initrd that can do
the work.