questions about GRUB and BTRFS

questions about GRUB and BTRFS

[Anthony Roberts]

Hi,

A quick googling turns up posts that GRUB support for BTRFS is planned. My
curiosity is more towards how this will be managed, because the way this is
currently implemented with software RAID/LVM is quite haphazard. I
therefore have some questions about GRUB + BTRFS:

-With GRUB booting, it's easy to think of awkward use cases and limitations
unless it's capable of discovering BTRFS instances, and can boot by
specifying BTRFS UUID + subvolume. That seems quite ambitious, but is this
planned "eventually"?

-Might it be possible to tweak the userspace component of GRUB to install
the bootloader to every member device? This seems necessary for reliable
booting and rebuilding after a dead disk.

-64 kb at the beginning of the device is plenty for MBR + GRUB stage 1 +
1.5. Might this allow bootable BTRFS without paritions being used at all?
The space used for partitioning is negligible, however we're on the cusp of
disks that are too big to partition with MBR, and GPT booting doesn't seem
well supported yet.

There's obviously no point in getting worked up about this before
production ready support is available in the first place. :) However, I am
curious about what sort of implementation is planned.

-Anthony

Re: questions about GRUB and BTRFS

[Dmitri Nikulin]

On Wed, Feb 25, 2009 at 12:32 PM, Anthony Roberts
<btrfs-devel@arbitraryconstant.com> wrote:
> Hi,
>
> A quick googling turns up posts that GRUB support for BTRFS is planned. My
> curiosity is more towards how this will be managed, because the way this is
> currently implemented with software RAID/LVM is quite haphazard. I
> therefore have some questions about GRUB + BTRFS:

IANAGD (I Am Not A GRUB Developer), but I'll post some intuitive respones.

> -With GRUB booting, it's easy to think of awkward use cases and limitations
> unless it's capable of discovering BTRFS instances, and can boot by
> specifying BTRFS UUID + subvolume. That seems quite ambitious, but is this
> planned "eventually"?

I don't know how much filesystem code can be crammed into the
pre-/boot parts of GRUB, but I doubt it's enough to support btrfs'
advanced features like object-level striping.

For comparison with how the two major ZFS operating systems support root on ZFS:

*Solaris (Open, Nexenta, etc.) support booting from ZFS using GRUB,
but ONLY plain or mirrored, not striped or raid-z. Not sure about
linear, if the kernel is installed on anything but the first vdev.

FreeBSD unofficially supports / on ZFS very well, but you still need a
/boot to let the bootloader find the kernel and modules. However the
kernel itself can be given a ZFS pool and path such as
"zfs:pool/freebsd/root" and it will find all of the ZFS metadata it
needs on disk blocks and the small cache in /boot. However in return
for this /boot you get the ability to boot right off RAID-Z or
whatever you like, because it's using the kernel with full driver and
filesystem code instead of very limited bootloader code.

> -Might it be possible to tweak the userspace component of GRUB to install
> the bootloader to every member device? This seems necessary for reliable
> booting and rebuilding after a dead disk.

Even if you couldn't tweak grub, device-mapper already has an easy way
to mirror just the boot blocks per disk. However GRUB would get
confused since the virtual device does not map to a BIOS boot device.
Legacy BIOS booting is a pain that way. You may as well just write a
shell script to automatically invoke grub-install for each device
individually.

> -64 kb at the beginning of the device is plenty for MBR + GRUB stage 1 +
> 1.5. Might this allow bootable BTRFS without paritions being used at all?
> The space used for partitioning is negligible, however we're on the cusp of
> disks that are too big to partition with MBR, and GPT booting doesn't seem
> well supported yet.

As far as I know, we don't even have a way to boot straight off LVM
(because GRUB doesn't support it, and for a kernel and initrd you need
a supported partition), and btrfs would only be more difficult.

> There's obviously no point in getting worked up about this before
> production ready support is available in the first place. :) However, I am
> curious about what sort of implementation is planned.

Well before production ready support is there, people will already
want to test btrfs as their / (which should be automagic like for
FreeBSD ZFS) and /boot (because they're difficult that way). Long
before reiser4 was even proposed for mainline merge, it already had
GRUB support. Enthusiasts will always believe that even /boot should
be fortified with COW, checksums and snapshotting :)

Especially if btrfs is intended to be the "next default Linux
filesystem" as quoted in many places, it will need /boot support in
some form. I'll personally keep an ext3 /boot for a long time just
because recovery is easier that way.

-- 
Dmitri Nikulin

Centre for Synchrotron Science
Monash University
Victoria 3800, Australia

Re: questions about GRUB and BTRFS

[Lee Trager]

I'm not sure when we should start developing BTRFS support for GRUB but
I do agree that it will be very difficult to support all the features of
BTRFS.  As far as I know GRUB does not support LVM and only supports
RAID1. Doing this shouldn't be that hard to do, in fact it should be
easier to do with BTRFS since the filesystem is aware that it has RAID.
The big problem with BTRFS GRUB support is all the advanced features
BTRFS has to offer. I plan to eventually write a way for BTRFS to force
on or off compression/encryption. One of the main reasons for this is so
the GRUB BTRFS implementation doesn't have to, at least initially,
support compression/encryption. All that would be required is that the
data is left raw. Other features like snapshots would be really great to
be able to access from the boot loader but currently would be very
difficult to implement.

It may even be worth skipping GRUB support and just adding support for
BTRFS in GRUB2 so we can have all BTRFS's features. If you would like to
start working on this I would also talk to the GRUB guys first.

Lee

Dmitri Nikulin wrote:
> On Wed, Feb 25, 2009 at 12:32 PM, Anthony Roberts
> <btrfs-devel@arbitraryconstant.com> wrote:
>   
>> Hi,
>>
>> A quick googling turns up posts that GRUB support for BTRFS is planned. My
>> curiosity is more towards how this will be managed, because the way this is
>> currently implemented with software RAID/LVM is quite haphazard. I
>> therefore have some questions about GRUB + BTRFS:
>>     
>
> IANAGD (I Am Not A GRUB Developer), but I'll post some intuitive respones.
>
>   
>> -With GRUB booting, it's easy to think of awkward use cases and limitations
>> unless it's capable of discovering BTRFS instances, and can boot by
>> specifying BTRFS UUID + subvolume. That seems quite ambitious, but is this
>> planned "eventually"?
>>     
>
> I don't know how much filesystem code can be crammed into the
> pre-/boot parts of GRUB, but I doubt it's enough to support btrfs'
> advanced features like object-level striping.
>
> For comparison with how the two major ZFS operating systems support root on ZFS:
>
> *Solaris (Open, Nexenta, etc.) support booting from ZFS using GRUB,
> but ONLY plain or mirrored, not striped or raid-z. Not sure about
> linear, if the kernel is installed on anything but the first vdev.
>
> FreeBSD unofficially supports / on ZFS very well, but you still need a
> /boot to let the bootloader find the kernel and modules. However the
> kernel itself can be given a ZFS pool and path such as
> "zfs:pool/freebsd/root" and it will find all of the ZFS metadata it
> needs on disk blocks and the small cache in /boot. However in return
> for this /boot you get the ability to boot right off RAID-Z or
> whatever you like, because it's using the kernel with full driver and
> filesystem code instead of very limited bootloader code.
>
>   
>> -Might it be possible to tweak the userspace component of GRUB to install
>> the bootloader to every member device? This seems necessary for reliable
>> booting and rebuilding after a dead disk.
>>     
>
> Even if you couldn't tweak grub, device-mapper already has an easy way
> to mirror just the boot blocks per disk. However GRUB would get
> confused since the virtual device does not map to a BIOS boot device.
> Legacy BIOS booting is a pain that way. You may as well just write a
> shell script to automatically invoke grub-install for each device
> individually.
>
>   
>> -64 kb at the beginning of the device is plenty for MBR + GRUB stage 1 +
>> 1.5. Might this allow bootable BTRFS without paritions being used at all?
>> The space used for partitioning is negligible, however we're on the cusp of
>> disks that are too big to partition with MBR, and GPT booting doesn't seem
>> well supported yet.
>>     
>
> As far as I know, we don't even have a way to boot straight off LVM
> (because GRUB doesn't support it, and for a kernel and initrd you need
> a supported partition), and btrfs would only be more difficult.
>
>   
>> There's obviously no point in getting worked up about this before
>> production ready support is available in the first place. :) However, I am
>> curious about what sort of implementation is planned.
>>     
>
> Well before production ready support is there, people will already
> want to test btrfs as their / (which should be automagic like for
> FreeBSD ZFS) and /boot (because they're difficult that way). Long
> before reiser4 was even proposed for mainline merge, it already had
> GRUB support. Enthusiasts will always believe that even /boot should
> be fortified with COW, checksums and snapshotting :)
>
> Especially if btrfs is intended to be the "next default Linux
> filesystem" as quoted in many places, it will need /boot support in
> some form. I'll personally keep an ext3 /boot for a long time just
> because recovery is easier that way.
>
>   

Re: questions about GRUB and BTRFS

[Thomas Kuther]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

On Mi, 25.02.09 01:03 Lee Trager <lt73@cs.drexel.edu> wrote:

> I'm not sure when we should start developing BTRFS support for GRUB
> but I do agree that it will be very difficult to support all the
> features of BTRFS.  As far as I know GRUB does not support LVM and
> only supports RAID1. Doing this shouldn't be that hard to do, in fact

GRUB2 maybe?

Using trunk I see things like

/lib64/grub/i386-pc/lvm.mod
/lib64/grub/i386-pc/ext2.mod
/lib64/grub/i386-pc/reiserfs.mod
and
/lib64/grub/i386-pc/raid.mod


...which a grub-install will copy to /boot/grub

I haven't tested those though.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: questions about GRUB and BTRFS

[Chris Mason]

On Tue, 2009-02-24 at 18:32 -0700, Anthony Roberts wrote:
> Hi,
> 
> A quick googling turns up posts that GRUB support for BTRFS is planned. My
> curiosity is more towards how this will be managed, because the way this is
> currently implemented with software RAID/LVM is quite haphazard. I
> therefore have some questions about GRUB + BTRFS:
> 

So, I haven't looked very hard at the grub code, and expected to start
the btrfs grub support with something very minimal (single device only
configs).

> -With GRUB booting, it's easy to think of awkward use cases and limitations
> unless it's capable of discovering BTRFS instances, and can boot by
> specifying BTRFS UUID + subvolume. That seems quite ambitious, but is this
> planned "eventually"?

Grub today works by giving it a device and a path.  Since all the btrfs
subvolumes can be found from a path, device + path will actually work
with the subvolume support.

> 
> -Might it be possible to tweak the userspace component of GRUB to install
> the bootloader to every member device? This seems necessary for reliable
> booting and rebuilding after a dead disk.

I need to look into the grub code in detail to answer this.

> 
> -64 kb at the beginning of the device is plenty for MBR + GRUB stage 1 +
> 1.5. Might this allow bootable BTRFS without paritions being used at all?
> The space used for partitioning is negligible, however we're on the cusp of
> disks that are too big to partition with MBR, and GPT booting doesn't seem
> well supported yet.
> 

Part of the reason we're 64kb in was to better support bootloaders.

> There's obviously no point in getting worked up about this before
> production ready support is available in the first place. :) However, I am
> curious about what sort of implementation is planned.

In the ideal implementation, the grub.conf has a list of devices it is
allowed to scan, and we put the FS uuid directly in there, let grub scan
them and we'll be able to boot off multiple volumes in that way.

But, that is a significant project, so we'll have to do it in stages.

-chris

Re: questions about GRUB and BTRFS

[Anthony Roberts]

Hi Chris,

Cheers for the informative response. :)

> In the ideal implementation, the grub.conf has a list of devices it is
> allowed to scan, and we put the FS uuid directly in there, let grub scan
> them and we'll be able to boot off multiple volumes in that way.

Hm... perhaps it doesn't even need that much. It we're specifying devices,
might it simply pull the UUID out of the initial device it's given, kinda
like how the current mounting process works? Enumeration of the drives
changes as they come and go, but it doesn't really matter what "hd0" is
today as long as it's a member device.

This suggests another question for me... right now you can specify a root=
command line to the kernel, though there's other stuff like nfsroot= where
more parameters are needed. Is it possible to add a btrfsroot= option with
a UUID+subvolume?

With these features exposed to the booting process, it seems like there are
a *lot* of very powerful use cases that become possible with management of
the OS itself.

> But, that is a significant project, so we'll have to do it in stages.

Indeed. It's not my expectation it be done initially or even soon, just my
hope that it "eventually" become as flexible in booting as it is once the
system goes multi-user. :)

-Anthony

Re: questions about GRUB and BTRFS

[Chris Mason]

On Wed, 2009-02-25 at 13:04 -0700, Anthony Roberts wrote:
> Hi Chris,
> 
> Cheers for the informative response. :)
> 
> > In the ideal implementation, the grub.conf has a list of devices it is
> > allowed to scan, and we put the FS uuid directly in there, let grub scan
> > them and we'll be able to boot off multiple volumes in that way.
> 
> Hm... perhaps it doesn't even need that much. It we're specifying devices,
> might it simply pull the UUID out of the initial device it's given, kinda
> like how the current mounting process works? Enumeration of the drives
> changes as they come and go, but it doesn't really matter what "hd0" is
> today as long as it's a member device.

The problem is that grub would need some way of knowing which devices to
scan.  I'd hate to see it off running through everything in a san.

> 
> This suggests another question for me... right now you can specify a root=
> command line to the kernel, though there's other stuff like nfsroot= where
> more parameters are needed. Is it possible to add a btrfsroot= option with
> a UUID+subvolume?

I'm not a huge fan of initrds, but the distro initrd scripts already
have this kind of goodness hooked in.  I think that's the best place for
it.

-chris

Re: questions about GRUB and BTRFS

[David Woodhouse]

On Wed, 2009-02-25 at 15:22 -0500, Chris Mason wrote:
> > This suggests another question for me... right now you can specify a root=
> > command line to the kernel, though there's other stuff like nfsroot= where
> > more parameters are needed. Is it possible to add a btrfsroot= option with
> > a UUID+subvolume?
> 
> I'm not a huge fan of initrds, but the distro initrd scripts already
> have this kind of goodness hooked in.  I think that's the best place for
> it.

Definitely. We're talking about _removing_ the nfsroot stuff that
Anthony refers to, in fact, on the basis that it can be done within an
initrd and doesn't have to live in the kernel.

Mount by UUID for other file systems is already handled by initrd; the
kernel can't do it for itself.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation