Flashing UBIFS image to gluebi mtd

Flashing UBIFS image to gluebi mtd

[Konstantin Tokarev]

Hi all,

I'm migrating rootfs of device from SquashFS on top of gluebi to UBIFS. However, old version of
firmware upgrade program writes UBIFS image to gluebi mtd device instead of using recommended
volume update procedure. While it works for SquashFS images, writing UBIFS image leads to
the next errors messages in random moments:

UBI error: ubi_io_read: error -77 while reading 76 bytes from PEB 769:26000, read 0 bytes
UBIFS error (pid 383): ubifs_read_node: cannot read node 2 from LEB 347:21904, error -5
UBIFS warning (pid 383): ubifs_ro_mode: switched to read-only mode, error -5

However, when device is rebooted first time after upgrade, UBIFS mounts successfully.

Do you have any idea what is going on and how to destinguish "broken" UBIFS volume from
correctly flashed via ubi volume device?

-- 
Regards,
Konstantin

Re: Flashing UBIFS image to gluebi mtd

[Ezequiel Garcia]

Hi Konstantin,

On Thu, Oct 31, 2013 at 05:48:34PM +0400, Konstantin Tokarev wrote:
> 
> I'm migrating rootfs of device from SquashFS on top of gluebi to UBIFS.

Sounds like you could use ubiblock for that setup, which is suitable
for squashfs, and completely independent of ubifs (thus less cpu and
memory hungry and easier to setup).

You can find some patches floating around, but the *latest* work
is here:

http://git.free-electrons.com/users/ezequiel-garcia/linux/log/?h=ubiblock-v4-cache-optional

And you'll need a patch for mtd-utils as well:

http://git.free-electrons.com/users/ezequiel-garcia/mtd-utils

Feel free to use it and report any issues. However, be very
careful as the driver is a bit experimental yet.

Others are already using it, so it should be more or less
fine!
-- 
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com

Re: Flashing UBIFS image to gluebi mtd

[Konstantin Tokarev]

31.10.2013, 19:08, "Ezequiel Garcia" <ezequiel.garcia@free-electrons.com>:
> Hi Konstantin,
>
> On Thu, Oct 31, 2013 at 05:48:34PM +0400, Konstantin Tokarev wrote:
>
>>  I'm migrating rootfs of device from SquashFS on top of gluebi to UBIFS.
>
> Sounds like you could use ubiblock for that setup, which is suitable
> for squashfs, and completely independent of ubifs (thus less cpu and
> memory hungry and easier to setup).

Could you elaborate this a bit more? Do you mean that block device emultaion
layer on top of UBI + FS on top of that can work faster than UBIFS on top of
UBI?

Btw, I'm more interested in finding out in what way my UBIFS volume can be
broken in given scenario and how can I detect it (to reflash once again,
using UBI-aware updater this time).

-- 
Regards,
Konstantin

Re: Flashing UBIFS image to gluebi mtd

[Ezequiel Garcia]

On Thu, Oct 31, 2013 at 10:00:30PM +0400, Konstantin Tokarev wrote:
> 
> 
> 31.10.2013, 19:08, "Ezequiel Garcia" <ezequiel.garcia@free-electrons.com>:
> > Hi Konstantin,
> >
> > On Thu, Oct 31, 2013 at 05:48:34PM +0400, Konstantin Tokarev wrote:
> >
> >>  I'm migrating rootfs of device from SquashFS on top of gluebi to UBIFS.
> >
> > Sounds like you could use ubiblock for that setup, which is suitable
> > for squashfs, and completely independent of ubifs (thus less cpu and
> > memory hungry and easier to setup).
> 
> Could you elaborate this a bit more? Do you mean that block device emultaion
> layer on top of UBI + FS on top of that can work faster than UBIFS on top of
> UBI?
> 

Given you provided no details, I'm not sure why you're migrating from squashfs
(which is a read-only, block oriented fs) to ubifs (which supports r/w on MTD).
But anyway, let me do this homework for you :-)

So, you had squashfs on top of gluebi, right? I guess you had this:

  mtd (plus your NAND driver) -> ubi -> gluebi/mtd -> mtdblock -> squashfs

On the other side, using ubiblock to emulate the block device you'd
end up with this:

  mtd (plus your NAND driver) -> ubi -> ubiblock -> squashfs

Which is less memory (cpu?) hungry than the first alternative.
The latest git branch also contained a non-cached option which
has shown some good behavior, given squashfs does its own caching.

But of course, this is _NOT_ a replacement for UBIFS, and using ubiblock
for R/W is strictly discouraged.

And here's the (already ancient) discussion:

http://lists.infradead.org/pipermail/linux-mtd/2012-November/045011.html

And the v2:
http://lists.infradead.org/pipermail/linux-mtd/2012-December/045274.html

I'll post the latest work as soon as I find some time, but you're free
to pursue your own research.

Hope it helps and best of lucks!
-- 
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com

Re: Flashing UBIFS image to gluebi mtd

[Ricard Wanderlof]

On Thu, 31 Oct 2013, Konstantin Tokarev wrote:

>
>
> 31.10.2013, 19:08, "Ezequiel Garcia" <ezequiel.garcia@free-electrons.com>:
>> Hi Konstantin,
>>
>> On Thu, Oct 31, 2013 at 05:48:34PM +0400, Konstantin Tokarev wrote:
>>
>>>  I'm migrating rootfs of device from SquashFS on top of gluebi to UBIFS.
>>
>> Sounds like you could use ubiblock for that setup, which is suitable
>> for squashfs, and completely independent of ubifs (thus less cpu and
>> memory hungry and easier to setup).
>
> Could you elaborate this a bit more? Do you mean that block device emultaion
> layer on top of UBI + FS on top of that can work faster than UBIFS on top of
> UBI?
>
> Btw, I'm more interested in finding out in what way my UBIFS volume can be
> broken in given scenario and how can I detect it (to reflash once again,
> using UBI-aware updater this time).

You didn't mention how you overwrote your UBI partition with new data. dd 
? nandwrite ? I'm getting the impression your problem is based on the fact 
that when updating the firmware, you are using update firmware which is 
not UBI aware, and you don't get UBI-awareness on the system until you've 
done the update. A chicken-and-egg-situation. One solution in that case 
would be to have an intermediate firmware which was identical to what was 
in the device to start with but that had an UBI-aware upgrade application.

Also I don't think you mentioned exactly your old setup. I'm assuming 
squashfs on top of mtd, with no UBI involved, right?

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Flashing UBIFS image to gluebi mtd

[Konstantin Tokarev]

01.11.2013, 11:48, "Ricard Wanderlof" <ricard.wanderlof@axis.com>:
> On Thu, 31 Oct 2013, Konstantin Tokarev wrote:
>
>>  31.10.2013, 19:08, "Ezequiel Garcia" <ezequiel.garcia@free-electrons.com>:
>>>  Hi Konstantin,
>>>
>>>  On Thu, Oct 31, 2013 at 05:48:34PM +0400, Konstantin Tokarev wrote:
>>>>   I'm migrating rootfs of device from SquashFS on top of gluebi to UBIFS.
>>>  Sounds like you could use ubiblock for that setup, which is suitable
>>>  for squashfs, and completely independent of ubifs (thus less cpu and
>>>  memory hungry and easier to setup).
>>  Could you elaborate this a bit more? Do you mean that block device emultaion
>>  layer on top of UBI + FS on top of that can work faster than UBIFS on top of
>>  UBI?
>>
>>  Btw, I'm more interested in finding out in what way my UBIFS volume can be
>>  broken in given scenario and how can I detect it (to reflash once again,
>>  using UBI-aware updater this time).
>
> You didn't mention how you overwrote your UBI partition with new data. dd
> ? nandwrite ?

I write data to mtd device file (gluebi-based in this case) using write()
function, one mtd eraseblock at a time. Effectively it's the same as dd.

> I'm getting the impression your problem is based on the fact
> that when updating the firmware, you are using update firmware which is
> not UBI aware, and you don't get UBI-awareness on the system until you've
> done the update.

Exactly.

> A chicken-and-egg-situation. One solution in that case
> would be to have an intermediate firmware which was identical to what was
> in the device to start with but that had an UBI-aware upgrade application.

Not a solution, because end user of device might skip this intermediate
firmware accidentally.

It would be perfectly OK if rootfs failed to mount after update because
it led to reflash once again with the same firmware. It would be possible
to do any new consistency check, because initramfs software had already
been upgraded.

The problem is that incorrectly flashed UBIFS mounts without error,
all content is readable and has valid md5sums, and file write and
mv operations also succeed.

>
> Also I don't think you mentioned exactly your old setup. I'm assuming
> squashfs on top of mtd, with no UBI involved, right?

No, on top of UBI (to handle bad blocks and wear leveling).

-- 
Regards,
Konstantin

Re: Flashing UBIFS image to gluebi mtd

[Ricard Wanderlof]

On Fri, 1 Nov 2013, Konstantin Tokarev wrote:

>>>  Btw, I'm more interested in finding out in what way my UBIFS volume can be
>>>  broken in given scenario and how can I detect it (to reflash once again,
>>>  using UBI-aware updater this time).
>>
>> You didn't mention how you overwrote your UBI partition with new data. dd
>> ? nandwrite ?
>
> I write data to mtd device file (gluebi-based in this case) using write()
> function, one mtd eraseblock at a time. Effectively it's the same as dd.

How do you handle potential bad blocks in the flash ?

>> Also I don't think you mentioned exactly your old setup. I'm assuming
>> squashfs on top of mtd, with no UBI involved, right?
>
> No, on top of UBI (to handle bad blocks and wear leveling).

Sorry, my bad, reading your post again I see that you explicitly stated 
squashfs on top of gluebi ...

>> I'm getting the impression your problem is based on the fact
>> that when updating the firmware, you are using update firmware which is
>> not UBI aware, and you don't get UBI-awareness on the system until you've
>> done the update.
>
> Exactly.

... which means that you've been running UBI all the time, but not ubifs. 
(?)

>> A chicken-and-egg-situation. One solution in that case
>> would be to have an intermediate firmware which was identical to what was
>> in the device to start with but that had an UBI-aware upgrade application.
>
> Not a solution, because end user of device might skip this intermediate
> firmware accidentally.

I was afraid of that ... :-)

>
> It would be perfectly OK if rootfs failed to mount after update because
> it led to reflash once again with the same firmware. It would be possible
> to do any new consistency check, because initramfs software had already
> been upgraded.
>
> The problem is that incorrectly flashed UBIFS mounts without error,
> all content is readable and has valid md5sums, and file write and
> mv operations also succeed.

But in your original post, the following line was present in the log:

UBIFS warning (pid 383): ubifs_ro_mode: switched to read-only mode, error 
-5

indicating that writing had been switched off due to the errors detected. 
Of course, this might be arbitrary depending on exactly which errors 
happened to occur at that particular boot instance.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Flashing UBIFS image to gluebi mtd

[Konstantin Tokarev]

01.11.2013, 13:43, "Ricard Wanderlof" <ricard.wanderlof@axis.com>:

>  On Fri, 1 Nov 2013, Konstantin Tokarev wrote:
>>>>    Btw, I'm more interested in finding out in what way my UBIFS volume can be
>>>>    broken in given scenario and how can I detect it (to reflash once again,
>>>>    using UBI-aware updater this time).
>>>   You didn't mention how you overwrote your UBI partition with new data. dd
>>>   ? nandwrite ?
>>   I write data to mtd device file (gluebi-based in this case) using write()
>>   function, one mtd eraseblock at a time. Effectively it's the same as dd.
>  How do you handle potential bad blocks in the flash ?

Bad blocks are detected using ioctl on mtd device (MEMGETBADBLOCK) and skipped,
though AFAIU gluebi mtd cannot have bad blocks.

>>>   Also I don't think you mentioned exactly your old setup. I'm assuming
>>>   squashfs on top of mtd, with no UBI involved, right?
>>   No, on top of UBI (to handle bad blocks and wear leveling).
>  Sorry, my bad, reading your post again I see that you explicitly stated
>  squashfs on top of gluebi ...
>>>   I'm getting the impression your problem is based on the fact
>>>   that when updating the firmware, you are using update firmware which is
>>>   not UBI aware, and you don't get UBI-awareness on the system until you've
>>>   done the update.
>>   Exactly.
>  ... which means that you've been running UBI all the time, but not ubifs.
>  (?)

Yes.

I guess that we should have written SquashFS images using UBI volume update
procedure as well, to UBI device instead of gluebi MTD. Code was written to
write filesystems like JFFS2 to raw MTD's and just reused here without second
thought.

>>>   A chicken-and-egg-situation. One solution in that case
>>>   would be to have an intermediate firmware which was identical to what was
>>>   in the device to start with but that had an UBI-aware upgrade application.
>>   Not a solution, because end user of device might skip this intermediate
>>   firmware accidentally.
>  I was afraid of that ... :-)
>>   It would be perfectly OK if rootfs failed to mount after update because
>>   it led to reflash once again with the same firmware. It would be possible
>>   to do any new consistency check, because initramfs software had already
>>   been upgraded.
>>
>>   The problem is that incorrectly flashed UBIFS mounts without error,
>>   all content is readable and has valid md5sums, and file write and
>>   mv operations also succeed.
>  But in your original post, the following line was present in the log:
>
>  UBIFS warning (pid 383): ubifs_ro_mode: switched to read-only mode, error
>  -5
>
>  indicating that writing had been switched off due to the errors detected.
>  Of course, this might be arbitrary depending on exactly which errors
>  happened to occur at that particular boot instance.

I've read code of gluebi_write in gluebi.c, vol_cdev_ioctl in cdev.c, and
ubi_start_update in upd.c - looks like the only difference between correct
update and what is done in my case is that volume is not wiped out before
writing new volume image here.

(I know that raw mtd on NAND should be erased as well, but this is not done
for gluebi MTD because our code erases MTD only if MTD type == MTD_NANDFLASH)

-- 
Regards,
Konstantin