Ubiblock users call (Re: mvneta / openblocks switch)

Ubiblock users call (Re: mvneta / openblocks switch)

[Ezequiel Garcia]

On Tue, Nov 05, 2013 at 02:53:23PM +0100, Willy Tarreau wrote:
> > 
> > Hmm, could you tell me which github branch is good to try?

[...]

> 
> then I have set up a large collection of patches, including Ezequiel's work
> on the NAND flash, his UBI block devices and
[...]

Hey Willy,

Which patches have you taken for UBI block device? Can you describe your usage?

I've been wanting to push this upstream since a long while, and the patches
seem ready. However, given I'm not using these *at all*, I'd like to have some
real users supporting the feature addition.

Just for reference, the latest work is here:

http://git.free-electrons.com/users/ezequiel-garcia/linux/log/?h=ubiblock-v4-two-cache

And the user stuff:

http://git.free-electrons.com/users/ezequiel-garcia/mtd-utils/log/?h=ubiblkvol

-- 
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com

Re: Ubiblock users call (Re: mvneta / openblocks switch)

[Willy Tarreau]

Hi Ezequiel,

On Tue, Nov 05, 2013 at 11:40:04AM -0300, Ezequiel Garcia wrote:
> On Tue, Nov 05, 2013 at 02:53:23PM +0100, Willy Tarreau wrote:
> > > 
> > > Hmm, could you tell me which github branch is good to try?
> 
> [...]
> 
> > 
> > then I have set up a large collection of patches, including Ezequiel's work
> > on the NAND flash, his UBI block devices and
> [...]
> 
> Hey Willy,
> 
> Which patches have you taken for UBI block device?

I found a patch you posted on a list (probably linux-mtd but I'm not certain)
which received very little feedback. I wanted to give it a try and found that
it was easy to integrate and that it worked fine out of the box.

> Can you describe your usage?

I wanted to avoid ubifs for having used too many non-recoverable FS
corruptions with it in the past (most likely due to a faulty NAND in
the iomega Iconnect, or at least problems with the timings). More
importantly, seeing that there is no fsck equivalent for it is not
something to reassure me, because each time I got an issue, it was
not possible to remount the FS nor to fix it and everything was lost.

Still, I like the idea of UBI and wanted to experiment with something
like this to offer a block device interface. And before starting any
work, I googled around, thinking "probably someone will already have
had this idea". I found your patch and gave it a try. I only played
with it for one evening and did not have any more time to power my
mirabox on since. But I see some potential in it. Having the ability
to support partitions, various FS types, etc... makes it easier to
migrate systems designed to work this way to NAND-based devices. This
is probably also one reason for the success of eMMC which tends to
replace raw NAND on some new boards.

For example, one of my distro's boot scripts automatically detects
the available partitions, checks for a mountable FS there, then
locates the current config and loads it. With a block device, I have
nothing to change at all. With other solutions, I have to reconsider
the boot process.

> I've been wanting to push this upstream since a long while, and the patches
> seem ready. However, given I'm not using these *at all*, I'd like to have some
> real users supporting the feature addition.

I definitely am a supporter. It's still too early for me to judge whether the
design is the best one or not, but the feature of presenting a reliable block
device on top of a NAND is clearly appealing to make NAND-based devices work
more similarly to other devices found in the x86 world and reduce the porting
effort. I was a bit sad to see that your work received to little feedback, and
am ashamed for not having sent you any since (nor retried your two latest
pxa3xx patch series).

> Just for reference, the latest work is here:
> 
> http://git.free-electrons.com/users/ezequiel-garcia/linux/log/?h=ubiblock-v4-two-cache
>
> And the user stuff:
> 
> http://git.free-electrons.com/users/ezequiel-garcia/mtd-utils/log/?h=ubiblkvol

Ah excellent, thank you for the links!

One thing I noticed is that my mirabox's u-boot is able to read and write an
UBI volume, but IIRC, it can read what I format from linux but if I write from
u-boot, the linux cannot reread it. That's where I remember it was late in the
night and I had to go back home, then I haven't had an opportunity to try again
since.

I would find it really convenient to be able to ubi-write a block image from
u-boot and read it from Linux using the same format. It would ease the
(re)installation process making abstraction of bad blocks.

Best regards,
Willy

Re: Ubiblock users call (Re: mvneta / openblocks switch)

[Willy Tarreau]

Hi Ezequiel,

On Tue, Nov 05, 2013 at 03:57:48PM +0100, Willy Tarreau wrote:
> > Can you describe your usage?
>
> For example, one of my distro's boot scripts automatically detects
> the available partitions, checks for a mountable FS there, then
> locates the current config and loads it. With a block device, I have
> nothing to change at all. With other solutions, I have to reconsider
> the boot process.

BTW, I forgot to describe the initial use case I had in mind which led to this:
many of my machines (including the test ones) boot from a rootfs which is a
squashfs stored as an initrd image. At the moment, the squashfs is read by
u-boot from the NAND into memory and the kernel boots from this. Loading a 10
MB rootfs from NAND to RAM takes 1-2 seconds depending on the device, and
consumes 10 MB of RAM all the time. Also, you're always at risk that some bad
blocks appear in the middle of this image (I had one device where I had to
change the location of the initrd partition because of
this).

With ubiblock, I can totally solve this issue :
  - u-boot does not read anymore the initrd, thus saving time and RAM
  - kernel simply mounts the squashfs at boot from ubiblock, resulting
    in direct accesses
  - ubi would take care of bad blocks itself

Doing so prevents me from upgrading the image since it's always mounted,
which is why I wanted to try to write it from u-boot. It did not work so
I wanted to use 2 images and upgrade one while running on the other one.
But I did not make any progress on this yet.

I'll definitely give your latest patches a try, I just don't know when :-)

Best regards,
Willy

Re: Ubiblock users call (Re: mvneta / openblocks switch)

[Ezequiel Garcia]

On Wed, Nov 06, 2013 at 07:30:07AM +0100, Willy Tarreau wrote:
> Hi Ezequiel,
> 
> On Tue, Nov 05, 2013 at 03:57:48PM +0100, Willy Tarreau wrote:
> > > Can you describe your usage?
> >
> > For example, one of my distro's boot scripts automatically detects
> > the available partitions, checks for a mountable FS there, then
> > locates the current config and loads it. With a block device, I have
> > nothing to change at all. With other solutions, I have to reconsider
> > the boot process.
> 
> BTW, I forgot to describe the initial use case I had in mind which led to this:
> many of my machines (including the test ones) boot from a rootfs which is a
> squashfs stored as an initrd image. At the moment, the squashfs is read by
> u-boot from the NAND into memory and the kernel boots from this. Loading a 10
> MB rootfs from NAND to RAM takes 1-2 seconds depending on the device, and
> consumes 10 MB of RAM all the time. Also, you're always at risk that some bad
> blocks appear in the middle of this image (I had one device where I had to
> change the location of the initrd partition because of
> this).
> 
> With ubiblock, I can totally solve this issue :
>   - u-boot does not read anymore the initrd, thus saving time and RAM
>   - kernel simply mounts the squashfs at boot from ubiblock, resulting
>     in direct accesses
>   - ubi would take care of bad blocks itself
> 
> Doing so prevents me from upgrading the image since it's always mounted,
> which is why I wanted to try to write it from u-boot. It did not work so
> I wanted to use 2 images and upgrade one while running on the other one.
> But I did not make any progress on this yet.
> 
> I'll definitely give your latest patches a try, I just don't know when :-)
> 

Thanks for all these details! It'll be definitely interesting if you
could jump in the discussion when the patch is finally submitted.
-- 
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com