[U-Boot] uboot and ZFS

[U-Boot] uboot and ZFS

[Jorgen Lundman]

Sorry if I am posting some kind of FAQ here,

I recently picked up a Cubox (arm) which uses uboot. Added ZFS to Linux and 
made it have a ZFS root filesystem. The only drawback is the small boot 
partition of ext2/fat to read uImage and initrd.

So I am curious about adding ZFS support to uboot. Looking at the sources 
http://git.denx.de/cgi-bin/gitweb.cgi?p=u-boot.git;a=tree
in particular, the ext2 work, I see it was once imported from GRUB.

Comparing that with 
https://github.com/pendor/grub-zfs/tree/master/grub-core/fs/zfs which is a 
fork for ZFS support to GRUB.

At first glance, it would suggest that it should be feasible to add ZFS to 
uboot. But since I only have 'usage-level knowledge' of both boot systems, 
I am unaware of any show-stoppers.

Perhaps memory/object-size is an issue? The ext2 code is some ~1100 lines 
in uboot, but ZFS is 2700 lines in GRUB. Or any of a thousand other issues 
I am unaware about.

Anyone already attempted such a project? Any resources available for such 
porting? Will I even be able to run my own version of uboot on the Cubox?

Lund


-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

[U-Boot] uboot and ZFS

[Wolfgang Denk]

Dear Jorgen,

In message <4FA0A7F3.8010803@lundman.net> you wrote:
> 
> Sorry if I am posting some kind of FAQ here,

Don't worry - AFAICT thisis the first time ZFS came up here.

> At first glance, it would suggest that it should be feasible to add ZFS to 
> uboot. But since I only have 'usage-level knowledge' of both boot systems, 
> I am unaware of any show-stoppers.

Technically it should be no unsolvable problem to add ZFS support to
U-Boot.  However, there are the same licensing issues that prevent ZFS
to be included with the Linux kernel; see
http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue

Like Linux, U-Boot is released under GPL (v2+), which is not
compatible with the Sun CDDL.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Status quo. Latin for "the mess we're in."

[U-Boot] uboot and ZFS

[Marek Vasut]

Dear Jorgen Lundman,

> Sorry if I am posting some kind of FAQ here,
> 
> I recently picked up a Cubox (arm) which uses uboot. Added ZFS to Linux and
> made it have a ZFS root filesystem. The only drawback is the small boot
> partition of ext2/fat to read uImage and initrd.
> 
> So I am curious about adding ZFS support to uboot. Looking at the sources
> http://git.denx.de/cgi-bin/gitweb.cgi?p=u-boot.git;a=tree
> in particular, the ext2 work, I see it was once imported from GRUB.
> 
> Comparing that with
> https://github.com/pendor/grub-zfs/tree/master/grub-core/fs/zfs which is a
> fork for ZFS support to GRUB.
> 
> At first glance, it would suggest that it should be feasible to add ZFS to
> uboot. But since I only have 'usage-level knowledge' of both boot systems,
> I am unaware of any show-stoppers.

Oh please study this and submit a patch, I believe at least Pavel (CCed) would 
be glad to see it in :-)

> 
> Perhaps memory/object-size is an issue? The ext2 code is some ~1100 lines
> in uboot, but ZFS is 2700 lines in GRUB. Or any of a thousand other issues
> I am unaware about.

Well the porting might be quite a long run indeed, but basic booting from this 
should not be that hard. Besides the code would be fairy separate, so it should 
be OK.

> Anyone already attempted such a project? Any resources available for such
> porting? Will I even be able to run my own version of uboot on the Cubox?

I don't know, sorry.
> 
> Lund

Best regards,
Marek Vasut

[U-Boot] uboot and ZFS

[Jorgen Lundman]

>> At first glance, it would suggest that it should be feasible to add ZFS to
>> uboot. But since I only have 'usage-level knowledge' of both boot systems,
>> I am unaware of any show-stoppers.
>
> Oh please study this and submit a patch, I believe at least Pavel (CCed) would
> be glad to see it in :-)
>
>
> Well the porting might be quite a long run indeed, but basic booting from this
> should not be that hard. Besides the code would be fairy separate, so it should
> be OK.

I took the existing CuBox u-boot, and compiled. Then found the EXT4 patches 
done by Siemens, and practised merging that in, compiling, and flashing the 
cubox. I can now boot EXT4. So that is encouraging.

Looking at GRUB zfs, it defines similar functions, namely open(), mount(), 
read(), close() and list(). So perhaps I can get something done. Is there 
quicker ways to test uboot than to flash the hardware every time?

What is the general patch acceptance around, I see ext4 wasn't accepted, 
what are the chances my patches would be?

Lund





-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

[U-Boot] uboot and ZFS

[Jorgen Lundman]

> done by Siemens, and practised merging that in, compiling, and flashing the

Samsung! My bad, credits where due!


-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

[U-Boot] uboot and ZFS

[Marek Vasut]

Dear Jorgen Lundman,

> >> At first glance, it would suggest that it should be feasible to add ZFS
> >> to uboot. But since I only have 'usage-level knowledge' of both boot
> >> systems, I am unaware of any show-stoppers.
> > 
> > Oh please study this and submit a patch, I believe at least Pavel (CCed)
> > would be glad to see it in :-)
> > 
> > 
> > Well the porting might be quite a long run indeed, but basic booting from
> > this should not be that hard. Besides the code would be fairy separate,
> > so it should be OK.
> 
> I took the existing CuBox u-boot, and compiled. Then found the EXT4 patches
> done by Siemens, and practised merging that in, compiling, and flashing the
> cubox. I can now boot EXT4. So that is encouraging.
> 
> Looking at GRUB zfs, it defines similar functions, namely open(), mount(),
> read(), close() and list(). So perhaps I can get something done. Is there
> quicker ways to test uboot than to flash the hardware every time?

If you correctly skip the HW init, you can try booting u-boot from u-boot ... 
(now, WD will hate me for bringing this up :-) ). It's possible, but unsupported 
operation. You might need to do some research on this matter :-)

> 
> What is the general patch acceptance around, I see ext4 wasn't accepted,
> what are the chances my patches would be?

Why wasn't ext4 accepted?

> 
> Lund

Best regards,
Marek Vasut

[U-Boot] uboot and ZFS

[Jorgen Lundman]

> If you correctly skip the HW init, you can try booting u-boot from u-boot ...
> (now, WD will hate me for bringing this up :-) ). It's possible, but unsupported
> operation. You might need to do some research on this matter :-)
>

I was thinking more along the lines of;  Since I am testing a filesystem 
driver, to have a small test binary I can run and just feed a patch to (be 
it block device, or raw file) to test it can traverse, and read the contents.

But it would not be so bad to simulate that with some #ifdefs I guess.


>> What is the general patch acceptance around, I see ext4 wasn't accepted,
>> what are the chances my patches would be?
>
> Why wasn't ext4 accepted?

I have no idea. I found the patches back in Dec, and it is not in the 
repository. Although, I have patched it into my tree on github, it might 
help merging it into uboot. It does appear to work, whereas my zfs does not :)

CuBox>> zfsls usb 0:2
** Bad partition - usb 0:2 **
CuBox>> zfsls usb 0:1
** Bad partition - usb 0:1 **
CuBox>> zfsls usb 0:0
data abort
pc : [<0063ea0c>]          lr : [<0060eef0>]
sp : 005ffe40  ip : 00000000     fp : 00000000
r10: 00000000  r9 : 00652a4c     r8 : 005fffcc
r7 : 00000000  r6 : 00000000     r5 : 007104b0  r4 : 006444a7
r3 : 00000000  r2 : 006f7b80     r1 : 00000000  r0 : ea000012
Flags: nZCv  IRQs off  FIQs off  Mode SVC_32
Resetting CPU ...

Dang :)

Lund

-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

[U-Boot] uboot and ZFS

[Matthias Weißer]

Am 07.05.2012 09:13, schrieb Jorgen Lundman:
>>  If you correctly skip the HW init, you can try booting u-boot from u-boot ...
>>  (now, WD will hate me for bringing this up :-) ). It's possible, but unsupported
>>  operation. You might need to do some research on this matter :-)
>>
>
> I was thinking more along the lines of;  Since I am testing a filesystem
> driver, to have a small test binary I can run and just feed a patch to (be
> it block device, or raw file) to test it can traverse, and read the contents.

You could use sandbox arch for this purpose and use an image of a zfs 
filesystem. You can then use your native toolchain to compile and debug 
u-boot.

Reagards
Matthias

[U-Boot] uboot and ZFS

[Wolfgang Denk]

Dear Jorgen,

In message <4FA7761D.90704@lundman.net> you wrote:
> > If you correctly skip the HW init, you can try booting u-boot from u-boot ...
> > (now, WD will hate me for bringing this up :-) ). It's possible, but unsupported
> > operation. You might need to do some research on this matter :-)
> >
> 
> I was thinking more along the lines of;  Since I am testing a filesystem 
> driver, to have a small test binary I can run and just feed a patch to (be 
> it block device, or raw file) to test it can traverse, and read the contents.
> 
> But it would not be so bad to simulate that with some #ifdefs I guess.

Did you miss my message of Wed, 02 May?

See
http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/130638/focus=130671

Unfortunately there is no legal (license-conforming) way to include
ZFS code into U-Boot and distribute it.

That means, there is no chance to get this added to mainline.


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"One lawyer can steal more than a hundred men with guns."
- The Godfather

[U-Boot] uboot and ZFS

[Jorgen Lundman]

> Did you miss my message of Wed, 02 May?
>
> See
> http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/130638/focus=130671
>
> Unfortunately there is no legal (license-conforming) way to include
> ZFS code into U-Boot and distribute it.
>
> That means, there is no chance to get this added to mainline.
>
>

My apologies, I did mean to reply to that.

The reason the GRUB boot loader can only handle single/mirror device root 
pool (ie, no raidz) is because Sun did not want to commit "any more than 
they had to" to GRUB, which would force the source to GPL.

The source that was written by Sun, for GRUB, is indeed GPL license 
(version 2).

I do not see this as a problem, unless suddenly you do not accept GPL 
licenses. (like 95% of your existing sources). (2000 .c files out of 2412).

Please advice if this is this not acceptable to u-boot. It does not change 
that I want it, so I will simply have to maintain a fork.

Lund

-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

[U-Boot] uboot and ZFS

[Graeme Russ]

Hi Jorgen Lundman,

On 05/07/2012 11:03 PM, Jorgen Lundman wrote:
> 
>> Did you miss my message of Wed, 02 May?
>>
>> See
>> http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/130638/focus=130671
>>
>> Unfortunately there is no legal (license-conforming) way to include
>> ZFS code into U-Boot and distribute it.
>>
>> That means, there is no chance to get this added to mainline.
>>
>>
> 
> My apologies, I did mean to reply to that.
> 
> The reason the GRUB boot loader can only handle single/mirror device root
> pool (ie, no raidz) is because Sun did not want to commit "any more than
> they had to" to GRUB, which would force the source to GPL.
> 
> The source that was written by Sun, for GRUB, is indeed GPL license
> (version 2).
> 
> I do not see this as a problem, unless suddenly you do not accept GPL
> licenses. (like 95% of your existing sources). (2000 .c files out of 2412).
> 
> Please advice if this is this not acceptable to u-boot. It does not change
> that I want it, so I will simply have to maintain a fork.

Technically, U-Boot is GPLv2+ so GPLv2 only code _may_ not be accepted That
is for Wolfgang to decide, but you have been warned :)

Regards,

Graeme

[U-Boot] [PATCH] uboot and ZFS

[Jorgen Lundman]

>
> Technically, U-Boot is GPLv2+ so GPLv2 only code _may_ not be accepted That
> is for Wolfgang to decide, but you have been warned :)
>

Naturally, he is king after all. There will be a thousand reasons to reject 
it, but hopefully not due to license.



I based my GITHUB after the CuBox fork of u-boot. I will leave it up to 
those guys to submit their patches separately.


My fork of u-boot is here:

** https://github.com/lundman/u-boot


To see only the change set for adding ZFS:

** https://github.com/lundman/u-boot/compare/zfs


Since I merged in EXT4 patches to get a feel for u-boot, they are here:

** https://github.com/lundman/u-boot/compare/ext4

Only minor strings changes of "ext2" -> "ext4", otherwise the Samsung patch 
V4 posted in here.



Lund



Useless sample output:


CuBox>> zfsls usb 0:2 /ROOT/@/
zfs reading label 0 at sector 0 (byte 0)
ZFS Pool GUID: 8349310002080761692 (73deb535d9a3c75c) Label: GUID: 
1229383784885
2426185 (aa9c7c635319edc9), txg: 1348, SPA v28, ashift: 9

ZPOOL label 'rpool'

zfs fsname = '/ROOT/' snapname='<NULL>' filename = '/'
<DIR>  lib
<DIR>  sbin
<DIR>  var
<DIR>  bin
<DIR>  dev
<DIR>  selinux
<DIR>  usr
<DIR>  root
<DIR>  home
<DIR>  srv
<DIR>  ttt
<DIR>  proc
<DIR>  tmp
<DIR>  mnt
<DIR>  etc
<DIR>  media
<DIR>  boot
<DIR>  opt
<DIR>  sys
<DIR>  .old-root
<DIR>  src
        rootfile.txt
<DIR>  rpool



CuBox>> zfsload usb 0:2 0x00200000 /ROOT/@/boot/uImage
Loading file "/ROOT/@/boot/uImage" from usb device 0:2 usbda2
zfs reading label 0 at sector 0 (byte 0)
...........
ZFS Pool GUID: 8349310002080761692 (73deb535d9a3c75c) Label: GUID: 
12293837848852426185 (aa9c7c635319edc9), txg: 1348, SPA v28, ashift: 9
zfs endian set to little
zfs fsname = '/ROOT/' snapname='<NULL>' filename = '/boot/uImage'
3623192 bytes read

CuBox>> bootm
## Booting kernel from Legacy Image at 00200000 ...
    Image Name:   Linux-2.6.32.9-dove-5.4.2
    Created:      2012-01-07  10:23:53 UTC
    Image Type:   ARM Linux Kernel Image (uncompressed)
    Data Size:    3623128 Bytes =  3.5 MB
    Load Address: 00008000
    Entry Point:  00008000
    Verifying Checksum ... OK
    Loading Kernel Image ... OK
OK

Starting kernel ...

[    0.000000] Linux version 2.6.32.9-dove-5.4.2 (rabeeh at kossay-desktop) 
(gcc version 4.3.2 (sdk3.2rc1-ct-ng-1.4.1) ) #46 PREEMPT Sat Jan 7 12:23:38 
IST 2012

[U-Boot] [PATCH] uboot and ZFS

[Marek Vasut]

Dear Jorgen Lundman,

> > Technically, U-Boot is GPLv2+ so GPLv2 only code _may_ not be accepted
> > That is for Wolfgang to decide, but you have been warned :)
> 
> Naturally, he is king after all. There will be a thousand reasons to reject
> it, but hopefully not due to license.

I wonder ... Germany never was a kingdom, or am I wrong? :-) So maybe emperor* 
might be more fitting (it was empire I think)?

> I based my GITHUB after the CuBox fork of u-boot. I will leave it up to
> those guys to submit their patches separately.
> 
> 
> My fork of u-boot is here:
> 
> ** https://github.com/lundman/u-boot
> 
> 
> To see only the change set for adding ZFS:
> 
> ** https://github.com/lundman/u-boot/compare/zfs
> 
> 
> Since I merged in EXT4 patches to get a feel for u-boot, they are here:
> 
> ** https://github.com/lundman/u-boot/compare/ext4
> 
> Only minor strings changes of "ext2" -> "ext4", otherwise the Samsung patch
> V4 posted in here.

Yay, so the ext4 works for you? Why wasn't ext4 merged? Can you post the 
reasoning (maybe links to gmane or something?)

Thanks!

> 
> Lund
> 
> 
> 
> Useless sample output:
> 
> 
> CuBox>> zfsls usb 0:2 /ROOT/@/
> zfs reading label 0 at sector 0 (byte 0)
> ZFS Pool GUID: 8349310002080761692 (73deb535d9a3c75c) Label: GUID:
> 1229383784885
> 2426185 (aa9c7c635319edc9), txg: 1348, SPA v28, ashift: 9
> 
> ZPOOL label 'rpool'
> 
> zfs fsname = '/ROOT/' snapname='<NULL>' filename = '/'
> <DIR>  lib
> <DIR>  sbin
> <DIR>  var
> <DIR>  bin
> <DIR>  dev
> <DIR>  selinux
> <DIR>  usr
> <DIR>  root
> <DIR>  home
> <DIR>  srv
> <DIR>  ttt
> <DIR>  proc
> <DIR>  tmp
> <DIR>  mnt
> <DIR>  etc
> <DIR>  media
> <DIR>  boot
> <DIR>  opt
> <DIR>  sys
> <DIR>  .old-root
> <DIR>  src
>         rootfile.txt
> <DIR>  rpool
> 
> 
> 
> CuBox>> zfsload usb 0:2 0x00200000 /ROOT/@/boot/uImage
> Loading file "/ROOT/@/boot/uImage" from usb device 0:2 usbda2
> zfs reading label 0 at sector 0 (byte 0)
> ...........
> ZFS Pool GUID: 8349310002080761692 (73deb535d9a3c75c) Label: GUID:
> 12293837848852426185 (aa9c7c635319edc9), txg: 1348, SPA v28, ashift: 9
> zfs endian set to little
> zfs fsname = '/ROOT/' snapname='<NULL>' filename = '/boot/uImage'
> 3623192 bytes read
> 
> CuBox>> bootm
> ## Booting kernel from Legacy Image at 00200000 ...
>     Image Name:   Linux-2.6.32.9-dove-5.4.2
>     Created:      2012-01-07  10:23:53 UTC
>     Image Type:   ARM Linux Kernel Image (uncompressed)
>     Data Size:    3623128 Bytes =  3.5 MB
>     Load Address: 00008000
>     Entry Point:  00008000
>     Verifying Checksum ... OK
>     Loading Kernel Image ... OK
> OK
> 
> Starting kernel ...
> 
> [    0.000000] Linux version 2.6.32.9-dove-5.4.2 (rabeeh at kossay-desktop)
> (gcc version 4.3.2 (sdk3.2rc1-ct-ng-1.4.1) ) #46 PREEMPT Sat Jan 7 12:23:38
> IST 2012

* Not the star-wars kind :-)

Best regards,

[U-Boot] [PATCH] uboot and ZFS

[Jorgen Lundman]

>
> Yay, so the ext4 works for you? Why wasn't ext4 merged? Can you post the
> reasoning (maybe links to gmane or something?)
>

I have no idea. I just needed ext4 support, and saw someone had done the 
hard work back in Dec. I took the patches and applied them.

I have been informed that ext4 can't read volumes over 1TB. I could take a 
look at fixing that too, if there is enough interest.

ZFS pools over 1TB appears to work.

Lund

-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

[U-Boot] [PATCH] uboot and ZFS

[Graeme Russ]

Hi Jorgen Lundman,

On Mon, May 14, 2012 at 11:12 AM, Jorgen Lundman <lundman@lundman.net> wrote:
>>
>> Yay, so the ext4 works for you? Why wasn't ext4 merged? Can you post the
>> reasoning (maybe links to gmane or something?)
>>
>
> I have no idea. I just needed ext4 support, and saw someone had done the
> hard work back in Dec. I took the patches and applied them.

I think there was still some question as to how the ext4 code was
going to replace the ext2 code and be git-bisectable. I'm personally
really looking forward to the ext4 code going mainline

>
> I have been informed that ext4 can't read volumes over 1TB. I could take a
> look at fixing that too, if there is enough interest.

I suppose the question would be how big the code change is versus how
many people will actually benefit. If it result in a 'big' increase in
.text or .data then it may need to be a compile time option
(CONFIG_SYS_HUGE_EXT4 or something)

>
> ZFS pools over 1TB appears to work.

:)

Regards,

Graeme

[U-Boot] [PATCH] uboot and ZFS

[Jorgen Lundman]

>
> I suppose the question would be how big the code change is versus how
> many people will actually benefit. If it result in a 'big' increase in
> .text or .data then it may need to be a compile time option
> (CONFIG_SYS_HUGE_EXT4 or something)
>

I was guess that the issue lies in that the sector offset is a signed 32bit 
int, and that is what goes wrong. I think it should make no size difference 
at all, but would take a while to track down all calls that pass them along.





-- 
Jorgen Lundman       | <lundman@lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)