Offtopic: ubifs and best practice for supporting browser based firmware upgrades

Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Jaya Kumar]

Hi mtd friends,

Sorry, I have a somewhat offtopic question. I'm using ubifs, and I'm
trying to figure out what would be a good way to handle browser based
firmware upgrades. I've implemented the following:

- NAND is partitioned into bootloader, kernel, ubifs
- system is running from ubifs
- httpd with CGI is running from ubifs
- user uploads firmware containing ubifs.img (150MB) using this CGI
- during the upload, the CGI writes the temporary file (up to 150MB)
to a fat partition on a empty microSD (a non-booting microSD has to be
always present for this to work)
- upon completion of the upload, the CGI unpacks the temporary file
and validates checksums, this of course needs more space since now
both the temporary file and unpacked results are present
- then reboot since we can't ubiformat/reflash from the running ubifs
partition and I can't detach from it for same reason.
- upon reboot, u-boot script checks for the presence of the ubifs.img
in the microSD fat partition, if it is there then it nand erases the
ubifs partition, then does nandwrite of the ubifs.img
- I realize this loses the erase counters since it is simplistic
- system boots and then removes the upgrade files so that it doesn't
repeat for each bootup

This seems clunky and I recognize that if the upgrade is interrupted,
the unit would be bricked. I was curious how other people implement
this and what the best practices are. I was thinking whether I should
have another partition for a small upgrade-only OS, so if u-boot
detects ubifs.img or a user button press, it boots into the
upgrade-only OS and then I can run ubiformat from there. I would
welcome any advice/suggestions.

Thanks,
jayakumar

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 2595 bytes --]

On Mon, 2012-09-03 at 18:45 +0800, Jaya Kumar wrote:
> Hi mtd friends,
> 
> Sorry, I have a somewhat offtopic question. I'm using ubifs, and I'm
> trying to figure out what would be a good way to handle browser based
> firmware upgrades. I've implemented the following:
> 
> - NAND is partitioned into bootloader, kernel, ubifs
> - system is running from ubifs
> - httpd with CGI is running from ubifs
> - user uploads firmware containing ubifs.img (150MB) using this CGI
> - during the upload, the CGI writes the temporary file (up to 150MB)
> to a fat partition on a empty microSD (a non-booting microSD has to be
> always present for this to work)
> - upon completion of the upload, the CGI unpacks the temporary file
> and validates checksums, this of course needs more space since now
> both the temporary file and unpacked results are present
> - then reboot since we can't ubiformat/reflash from the running ubifs
> partition and I can't detach from it for same reason.
> - upon reboot, u-boot script checks for the presence of the ubifs.img
> in the microSD fat partition, if it is there then it nand erases the
> ubifs partition, then does nandwrite of the ubifs.img
> - I realize this loses the erase counters since it is simplistic

If you use ubiformat for erasing, then the erasecounters are preserved.

> - system boots and then removes the upgrade files so that it doesn't
> repeat for each bootup

> This seems clunky and I recognize that if the upgrade is interrupted,
> the unit would be bricked. I was curious how other people implement
> this and what the best practices are. I was thinking whether I should
> have another partition for a small upgrade-only OS, so if u-boot
> detects ubifs.img or a user button press, it boots into the
> upgrade-only OS and then I can run ubiformat from there. I would
> welcome any advice/suggestions.

Well, if you use mSD, you may have space to save the contents of the
partitions which you are trying to update, which would allow to
roll-back to the old system should something go wrong.

Another approach people use is that they have "mirror" UBI volume of the
same size, like "ubifs-reserve". Then the update image is put to
ubifs-reserve, and after this you use the UBI volume rename feature and
rename "ubifs-reserve into" "ubifs". This will kill the old "ubifs"
volume. After this you may resize "ubifs" to something larger, and
create a new "ubifs-reserve" volume.

See "Volume update" here:
http://www.linux-mtd.infradead.org/doc/ubi.html

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Jaya Kumar]

On Tue, Sep 4, 2012 at 5:40 PM, Artem Bityutskiy <dedekind1@gmail.com> wrote:
> Another approach people use is that they have "mirror" UBI volume of the
> same size, like "ubifs-reserve". Then the update image is put to
> ubifs-reserve, and after this you use the UBI volume rename feature and
> rename "ubifs-reserve into" "ubifs". This will kill the old "ubifs"
> volume. After this you may resize "ubifs" to something larger, and
> create a new "ubifs-reserve" volume.

Thanks Artem. I like above method. But one thing which I did not expect is that

I setup my single mtd partition with 2 volumes, rootfs and
rootfs_reserve. My system is running from volume 0 (rootfs), ie:
firmware upgrade is from there.

root@localhost:/# ubinfo -a
UBI version:                    1
Count of UBI devices:           1
UBI control device major/minor: 10:61
Present UBI devices:            ubi0

ubi0
Volumes count:                           2
Logical eraseblock size:                 129024 bytes, 126.0 KiB
Total amount of logical eraseblocks:     4036 (520740864 bytes, 496.6 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       0
Count of reserved physical eraseblocks:  80
Current maximum erase counter value:     2
Minimum input/output unit size:          2048 bytes
Character device major/minor:            252:0
Present volumes:                         0, 1

Volume ID:   0 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        1829 LEBs (235984896 bytes, 225.1 MiB)
State:       OK
Name:        rootfs
Character device major/minor: 252:1
-----------------------------------
Volume ID:   1 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        2123 LEBs (273917952 bytes, 261.2 MiB)
State:       OK
Name:        rootfs_reserve
Character device major/minor: 252:2

I'm then able to use ubiupdatevol to write the firmware (containing
new rootfs) update to the rootfs_reserve volume.

root@localhost:/# ubiupdatevol /dev/ubi0_1 /mnt/microsd/newfw/ubifs.img

But then when I go to rename rootfs_reserve to rootfs and rootfs to
rootfs_reserve, I get:

root@localhost:/# ubirename /dev/ubi0 rootfs rootfs_reserve
rootfs_reserve rootfs
UBI error: ubi_open_volume: cannot open device 0, volume 0, error -16
UBI error: rename_volumes: cannot open volume 0, error -16
ubirename: error!: cannot rename volumes
           error 16 (Device or resource busy)

It looks like I cannot perform the rename of the running volume, rootfs.

root@localhost:/# ubirename /dev/ubi0 rootfs oldrootfs
UBI error: ubi_open_volume: cannot open device 0, volume 0, error -16
UBI error: rename_volumes: cannot open volume 0, error -16
ubirename: error!: cannot rename volumes
           error 16 (Device or resource busy)

I can rename the reserve fine.
root@localhost:/# ubirename /dev/ubi0 rootfs_reserve somenewname

But I can't kill the name of the running volume.
root@localhost:/# ubirename /dev/ubi0 somenewname rootfs
UBI error: ubi_open_volume: cannot open device 0, volume 0, error -16
UBI error: rename_volumes: cannot open volume "rootfs", error -16
ubirename: error!: cannot rename volumes
           error 16 (Device or resource busy)

I think what this means is that after doing the rootfs upgrade using
ubiupdatevol, I must reboot into a 3rd partition running a minimal OS
to perform the ubirename of roofs_reserve and then reboot again. This
step seems expensive to have to boot into a 3rd partition to do the
rename operation.

I see someone else also has encountered this,
http://comments.gmane.org/gmane.linux.drivers.mtd/39415 and solved by
having a initramfs. I guess I could do that. But still seems messy.

Thanks,
jayakumar

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 1576 bytes --]

On Wed, 2012-09-26 at 13:42 +0800, Jaya Kumar wrote:
> I think what this means is that after doing the rootfs upgrade using
> ubiupdatevol, I must reboot into a 3rd partition running a minimal OS
> to perform the ubirename of roofs_reserve and then reboot again. This
> step seems expensive to have to boot into a 3rd partition to do the
> rename operation.
> 
> I see someone else also has encountered this,
> http://comments.gmane.org/gmane.linux.drivers.mtd/39415 and solved by
> having a initramfs. I guess I could do that. But still seems messy.

Yes, the way it is currently implemented is:

1. ubirename /dev/ubi0 rootfs rootfs_reserve
2. UBI figures out that old 'rootfs' should be removed.
3. UBI changes the volume table and wipes old rootfs from there,
   and changes name 'rootfs_reserve' to 'rootfs'.

   Now if you have a power cut, you will no longer see the old rootfs
   vloume, only the new one.
4. Then UBI actually deletes the old volume from all the in-memory
   data structures. Obviously, you cannot use this volume at the same
   time.

But this can be improved, I did not think a lot about this, but I guess
if you just skip step 4, it should work. The rename will succeed, but
you will still see the old layout. And only when you reboot, you'll have
the old rootfs deleted and the new one being in place.

Or not even reboot - detach and attach back should be enough. I guess
you can hack this.

Also, I guess the rename path should not open the volume in the
exclusive mode.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Ricard Wanderlof]

On Wed, 26 Sep 2012, Artem Bityutskiy wrote:

> On Wed, 2012-09-26 at 13:42 +0800, Jaya Kumar wrote:
>> I think what this means is that after doing the rootfs upgrade using
>> ubiupdatevol, I must reboot into a 3rd partition running a minimal OS
>> to perform the ubirename of roofs_reserve and then reboot again. This
>> step seems expensive to have to boot into a 3rd partition to do the
>> rename operation.
>>
>> I see someone else also has encountered this,
>> http://comments.gmane.org/gmane.linux.drivers.mtd/39415 and solved by
>> having a initramfs. I guess I could do that. But still seems messy.
>
> Yes, the way it is currently implemented is:
>
> 1. ubirename /dev/ubi0 rootfs rootfs_reserve
> 2. UBI figures out that old 'rootfs' should be removed.
> 3. UBI changes the volume table and wipes old rootfs from there,
>   and changes name 'rootfs_reserve' to 'rootfs'.
>
>   Now if you have a power cut, you will no longer see the old rootfs
>   vloume, only the new one.
> 4. Then UBI actually deletes the old volume from all the in-memory
>   data structures. Obviously, you cannot use this volume at the same
>   time.
>
> But this can be improved, I did not think a lot about this, but I guess
> if you just skip step 4, it should work. The rename will succeed, but
> you will still see the old layout. And only when you reboot, you'll have
> the old rootfs deleted and the new one being in place.

But the problem is that the first step fails as the system is still 
running, and you can't unmount / while it is up.

> Also, I guess the rename path should not open the volume in the
> exclusive mode.

Yes, that would be a workable solution, if there weren't any other 
disadvantages (e.g. something getting confused by the volume name 
changing, like df).

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 2265 bytes --]

On Wed, 2012-09-26 at 12:53 +0200, Ricard Wanderlof wrote:
> On Wed, 26 Sep 2012, Artem Bityutskiy wrote:
> 
> > On Wed, 2012-09-26 at 13:42 +0800, Jaya Kumar wrote:
> >> I think what this means is that after doing the rootfs upgrade using
> >> ubiupdatevol, I must reboot into a 3rd partition running a minimal OS
> >> to perform the ubirename of roofs_reserve and then reboot again. This
> >> step seems expensive to have to boot into a 3rd partition to do the
> >> rename operation.
> >>
> >> I see someone else also has encountered this,
> >> http://comments.gmane.org/gmane.linux.drivers.mtd/39415 and solved by
> >> having a initramfs. I guess I could do that. But still seems messy.
> >
> > Yes, the way it is currently implemented is:
> >
> > 1. ubirename /dev/ubi0 rootfs rootfs_reserve
> > 2. UBI figures out that old 'rootfs' should be removed.
> > 3. UBI changes the volume table and wipes old rootfs from there,
> >   and changes name 'rootfs_reserve' to 'rootfs'.
> >
> >   Now if you have a power cut, you will no longer see the old rootfs
> >   vloume, only the new one.
> > 4. Then UBI actually deletes the old volume from all the in-memory
> >   data structures. Obviously, you cannot use this volume at the same
> >   time.
> >
> > But this can be improved, I did not think a lot about this, but I guess
> > if you just skip step 4, it should work. The rename will succeed, but
> > you will still see the old layout. And only when you reboot, you'll have
> > the old rootfs deleted and the new one being in place.
> 
> But the problem is that the first step fails as the system is still 
> running, and you can't unmount / while it is up.
> 
> > Also, I guess the rename path should not open the volume in the
> > exclusive mode.
> 
> Yes, that would be a workable solution, if there weren't any other 
> disadvantages (e.g. something getting confused by the volume name 
> changing, like df).

Well, what I say is that we only do the changes in the on-flash volume
table. The in-memory data structures are not changed. So nothing should
be confused in theory, except the user :-) Something like I'll send in
the follow-up e-mail, which Jaya can probably try.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]

From df2a498002498e52025acf51e0b43064d766aa0a Mon Sep 17 00:00:00 2001
From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Date: Wed, 26 Sep 2012 14:12:28 +0300
Subject: [PATCH] UBI: allow for delayed rename

Just an experimental patch.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 drivers/mtd/ubi/vmt.c |   19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/drivers/mtd/ubi/vmt.c b/drivers/mtd/ubi/vmt.c
index 9169e58..6d7bcd9 100644
--- a/drivers/mtd/ubi/vmt.c
+++ b/drivers/mtd/ubi/vmt.c
@@ -597,25 +597,6 @@ int ubi_rename_volumes(struct ubi_device *ubi, struct list_head *rename_list)
 	struct ubi_rename_entry *re;
 
 	err = ubi_vtbl_rename_volumes(ubi, rename_list);
-	if (err)
-		return err;
-
-	list_for_each_entry(re, rename_list, list) {
-		if (re->remove) {
-			err = ubi_remove_volume(re->desc, 1);
-			if (err)
-				break;
-		} else {
-			struct ubi_volume *vol = re->desc->vol;
-
-			spin_lock(&ubi->volumes_lock);
-			vol->name_len = re->new_name_len;
-			memcpy(vol->name, re->new_name, re->new_name_len + 1);
-			spin_unlock(&ubi->volumes_lock);
-			ubi_volume_notify(ubi, vol, UBI_VOLUME_RENAMED);
-		}
-	}
-
 	if (!err)
 		self_check_volumes(ubi);
 	return err;
-- 
1.7.10.4



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

On Wed, 2012-09-26 at 14:13 +0300, Artem Bityutskiy wrote:
> From df2a498002498e52025acf51e0b43064d766aa0a Mon Sep 17 00:00:00 2001
> From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> Date: Wed, 26 Sep 2012 14:12:28 +0300
> Subject: [PATCH] UBI: allow for delayed rename
> 
> Just an experimental patch.
> 
> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

This have a lot of chances to work. If we do this, it would need to be
an extention to the existing ioctl, to preserve the original behaviour.
Something like a "delayed" flag, and then we can implement this in
ubirename as a --delayed option. Surely there may be better names for
this.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Jaya Kumar]

On Wed, Sep 26, 2012 at 7:13 PM, Artem Bityutskiy <dedekind1@gmail.com> wrote:
> From df2a498002498e52025acf51e0b43064d766aa0a Mon Sep 17 00:00:00 2001
> From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> Date: Wed, 26 Sep 2012 14:12:28 +0300
> Subject: [PATCH] UBI: allow for delayed rename
>
> Just an experimental patch.

Thanks Artem. I gave it a quick try.

I applied the patch, flashed new kernel and then attempted to rename
the rootfs-reserve volume to rootfs while running from the rootfs
volume.

root@localhost:/# ubirename /dev/ubi0 rootfs-reserve rootfs
UBI error: ubi_open_volume: cannot open device 0, volume 1, error -16
UBI error: rename_volumes: cannot open volume "rootfs", error -16
ubirename: error!: cannot rename volumes
           error 16 (Device or resource busy)

UBI DBG (pid 1786): ubi_cdev_ioctl: re-name volumes
UBI DBG (pid 1786): rename_volumes: will rename volume 0 from
"rootfs-reserve" to "rootfs"
UBI error: ubi_open_volume: cannot open device 0, volume 1, error -16
UBI error: rename_volumes: cannot open volume "rootfs", error -16

I also tried the reverse, renaming the running rootfs volume to something else.

root@localhost:/# ubirename /dev/ubi0 rootfs rootfsold
UBI error: ubi_open_volume: cannot open device 0, volume 1, error -16
UBI error: rename_volumes: cannot open volume 1, error -16
ubirename: error!: cannot rename volumes
           error 16 (Device or resource busy)
UBI DBG (pid 2600): ubi_cdev_ioctl: re-name volumes
UBI error: ubi_open_volume: cannot open device 0, volume 1, error -16
UBI error: rename_volumes: cannot open volume 1, error -16

I'm guessing I also need to hack the following in rename_volumes:
desc = ubi_open_volume_nm(ubi->ubi_num, re->new_name, UBI_EXCLUSIVE);

I'll try looking at it further tomorrow.

Thanks,
jayakumar

Re: Offtopic: ubifs and best practice for supporting browser based firmware upgrades

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

On Wed, 2012-09-26 at 20:38 +0800, Jaya Kumar wrote:
> I'm guessing I also need to hack the following in rename_volumes:
> desc = ubi_open_volume_nm(ubi->ubi_num, re->new_name, UBI_EXCLUSIVE);

Probably, you can use UBI_READONLY instead of UBI_EXCLUSIVE, I guess,
just to try if the approach works.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]