workaround install boot on btrfs with windows partition scheme

workaround install boot on btrfs with windows partition scheme

[Michael Chang]

[-- Attachment #1: Type: text/plain, Size: 1405 bytes --]

Many shipped Windows created it's first partition aligned in 63
(cylinder) and therefore can't offer enough room for core.img. Even
worse the partitions has been created as logical.

 > sudo /sbin/fdisk -l 
Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0001c622

   Device Boot      Start         End      Blocks   Id  System
   /dev/sda1              63     2056319     1028128+   b  W95 FAT32
   /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
   (LBA)
   /dev/sda5         2060288     5302271     1620992   82  Linux swap /
   Solaris
   /dev/sda6         5304320    47247359    20971520   83  Linux
   /dev/sda7        47249408   125804543    39277568   83  Linux

This leaves us currently no option to succeed in installation if boot is
on btrfs, or any other filesystems that block lists can't be used and
core.img must be embedded in order to be reliably addressed.

The attached patch try to workaround this scenario by placing the core.img
in filesystem's (btrfs) bootloader embedding area if available to overcome
the too small MBR gap which gets loaded by boot.img placed in MBR.

Please kindly review the patch or suggests for how to fix this scenario
sanely.

Thanks.
Michael

[-- Attachment #2: grub2-setup-try-fs-embed-if-mbr-gap-too-small.patch --]
[-- Type: text/x-patch, Size: 1962 bytes --]

Index: grub-2.02~beta2/util/setup.c
===================================================================
--- grub-2.02~beta2.orig/util/setup.c
+++ grub-2.02~beta2/util/setup.c
@@ -501,8 +501,44 @@ SETUP (const char *dir,
       err = grub_util_ldm_embed (dest_dev->disk, &nsec, maxsec,
 				 GRUB_EMBED_PCBIOS, &sectors);
     else if (ctx.dest_partmap)
-      err = ctx.dest_partmap->embed (dest_dev->disk, &nsec, maxsec,
-				     GRUB_EMBED_PCBIOS, &sectors);
+      {
+        err = ctx.dest_partmap->embed (dest_dev->disk, &nsec, maxsec,
+				       GRUB_EMBED_PCBIOS, &sectors);
+#ifdef GRUB_SETUP_BIOS
+	if (err == GRUB_ERR_OUT_OF_RANGE
+	    && strcmp (ctx.dest_partmap->name, "msdos") == 0
+	    && dest_dev->disk->id == root_dev->disk->id
+	    && dest_dev->disk->dev->id == root_dev->disk->dev->id)
+	  {
+	    grub_fs_t root_fs;
+
+	    root_fs = grub_fs_probe (root_dev);
+	    if (root_fs && root_fs->embed)
+	      {
+		grub_disk_addr_t *fs_sectors;
+		unsigned int fs_nsec;
+
+		fs_sectors = NULL;
+		fs_nsec = core_sectors;
+		err = root_fs->embed (root_dev, &fs_nsec, maxsec,
+				      GRUB_EMBED_PCBIOS, &fs_sectors);
+		if (!err && fs_nsec >= core_sectors)
+		  {
+		    grub_util_info ("Your msdos embedding area is too small, will use file system embedding area instead");
+		    sectors = fs_sectors;
+		    nsec = fs_nsec;
+		    ctx.container = root_dev->disk->partition;
+		    core_dev = root_dev;
+		  }
+		else
+		  {
+		    if (fs_sectors)
+		      grub_free (fs_sectors);
+		  }
+	      }
+	  }
+#endif
+      }
     else
       err = fs->embed (dest_dev, &nsec, maxsec,
 		       GRUB_EMBED_PCBIOS, &sectors);
@@ -584,7 +620,7 @@ SETUP (const char *dir,
 
     /* Write the core image onto the disk.  */
     for (i = 0; i < nsec; i++)
-      grub_disk_write (dest_dev->disk, sectors[i], 0,
+      grub_disk_write (core_dev->disk, sectors[i], 0,
 		       GRUB_DISK_SECTOR_SIZE,
 		       core_img + i * GRUB_DISK_SECTOR_SIZE);
 

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

On Thu, Oct 30, 2014 at 11:32 AM, Michael Chang <mchang@suse.com> wrote:
> Many shipped Windows created it's first partition aligned in 63
> (cylinder) and therefore can't offer enough room for core.img. Even
> worse the partitions has been created as logical.
>
>  > sudo /sbin/fdisk -l
> Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk label type: dos
> Disk identifier: 0x0001c622
>
>    Device Boot      Start         End      Blocks   Id  System
>    /dev/sda1              63     2056319     1028128+   b  W95 FAT32
>    /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
>    (LBA)
>    /dev/sda5         2060288     5302271     1620992   82  Linux swap /
>    Solaris
>    /dev/sda6         5304320    47247359    20971520   83  Linux
>    /dev/sda7        47249408   125804543    39277568   83  Linux
>
> This leaves us currently no option to succeed in installation if boot is
> on btrfs, or any other filesystems that block lists can't be used and
> core.img must be embedded in order to be reliably addressed.
>
> The attached patch try to workaround this scenario by placing the core.img
> in filesystem's (btrfs) bootloader embedding area if available to overcome
> the too small MBR gap which gets loaded by boot.img placed in MBR.
>
> Please kindly review the patch or suggests for how to fix this scenario
> sanely.
>

Well, I suggested something similar a way back

http://marc.info/?t=139175229300004&r=1&w=2

I still believe this is more flexible; in particular, /boot/grub on
btrfs has problems with unwritable grubenv (quite a few people are hit
by this now, when openSUSE defaults to single btrfs partition) so
having separate /boot as ext2 makes sense.

Your approach looks too special cased for default (open)SUSE configuration.

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Oct 30, 2014, at 2:32 AM, Michael Chang <mchang@suse.com> wrote:

> Many shipped Windows created it's first partition aligned in 63
> (cylinder) and therefore can't offer enough room for core.img. Even
> worse the partitions has been created as logical.
> 
>> sudo /sbin/fdisk -l 
> Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disk label type: dos
> Disk identifier: 0x0001c622
> 
>   Device Boot      Start         End      Blocks   Id  System
>   /dev/sda1              63     2056319     1028128+   b  W95 FAT32
>   /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
>   (LBA)
>   /dev/sda5         2060288     5302271     1620992   82  Linux swap /
>   Solaris
>   /dev/sda6         5304320    47247359    20971520   83  Linux
>   /dev/sda7        47249408   125804543    39277568   83  Linux
> 
> This leaves us currently no option to succeed in installation if boot is
> on btrfs, or any other filesystems that block lists can't be used and
> core.img must be embedded in order to be reliably addressed.
> 
> The attached patch try to workaround this scenario by placing the core.img
> in filesystem's (btrfs) bootloader embedding area if available to overcome
> the too small MBR gap which gets loaded by boot.img placed in MBR.
> 
> Please kindly review the patch or suggests for how to fix this scenario
> sanely.

Why not have a dedicated partition with MBR type code for core.img, equivalent to BIOSBoot currently used on GPT? freedesktop.org has a proposal to use type code 0xEA for this purpose (in part). The boot.img code in the MBR can arbitrarily jump to any LBA, so 0xEA doesn't need to be a primary partition does it?


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Oct 30, 2014, at 6:42 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 
> I still believe this is more flexible; in particular, /boot/grub on
> btrfs has problems with unwritable grubenv (quite a few people are hit
> by this now, when openSUSE defaults to single btrfs partition) so
> having separate /boot as ext2 makes sense.

Hmm, interesting. What's the nature of this problem with grubenv on btrfs? Is the current grubenv code expecting the file to be contiguous, and due to COW on btrfs it ends up not being contiguous? Does setting xattr +C on grubenv fix the problem?

Having separate /boot on ext was fine as a short/medium term work around, but /boot on btrfs has use case benefits so it really needs to work eventually.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

В Sat, 1 Nov 2014 14:35:57 -0600
Chris Murphy <lists@colorremedies.com> пишет:

> 
> On Oct 30, 2014, at 2:32 AM, Michael Chang <mchang@suse.com> wrote:
> 
> > Many shipped Windows created it's first partition aligned in 63
> > (cylinder) and therefore can't offer enough room for core.img. Even
> > worse the partitions has been created as logical.
> > 
> >> sudo /sbin/fdisk -l 
> > Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
> > Units = sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > Disk label type: dos
> > Disk identifier: 0x0001c622
> > 
> >   Device Boot      Start         End      Blocks   Id  System
> >   /dev/sda1              63     2056319     1028128+   b  W95 FAT32
> >   /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
> >   (LBA)
> >   /dev/sda5         2060288     5302271     1620992   82  Linux swap /
> >   Solaris
> >   /dev/sda6         5304320    47247359    20971520   83  Linux
> >   /dev/sda7        47249408   125804543    39277568   83  Linux
> > 
> > This leaves us currently no option to succeed in installation if boot is
> > on btrfs, or any other filesystems that block lists can't be used and
> > core.img must be embedded in order to be reliably addressed.
> > 
> > The attached patch try to workaround this scenario by placing the core.img
> > in filesystem's (btrfs) bootloader embedding area if available to overcome
> > the too small MBR gap which gets loaded by boot.img placed in MBR.
> > 
> > Please kindly review the patch or suggests for how to fix this scenario
> > sanely.
> 
> Why not have a dedicated partition with MBR type code for core.img, equivalent to BIOSBoot currently used on GPT? freedesktop.org has a proposal to use type code 0xEA for this purpose (in part). The boot.img code in the MBR can arbitrarily jump to any LBA, so 0xEA doesn't need to be a primary partition does it?
> 

It is rarely needed in simple cases; in complicated cases (btrfs or
LVM) we already have space dedicated for core.img. It seems more
logical to use this space.

Also you still need to tell grub-setup to use this special partition at
which point why not extend it to support arbitrary location for
core.img? It could be made check partition type and not refuse to
install on raw partition for special 0xEA type then as a bonus.

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

В Sat, 1 Nov 2014 19:34:56 -0600
Chris Murphy <lists@colorremedies.com> пишет:

> 
> On Oct 30, 2014, at 6:42 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> > 
> > I still believe this is more flexible; in particular, /boot/grub on
> > btrfs has problems with unwritable grubenv (quite a few people are hit
> > by this now, when openSUSE defaults to single btrfs partition) so
> > having separate /boot as ext2 makes sense.
> 
> Hmm, interesting. What's the nature of this problem with grubenv on btrfs? Is the current grubenv code expecting the file to be contiguous, and due to COW on btrfs it ends up not being contiguous? Does setting xattr +C on grubenv fix the problem?
> 

btrfs code does not execute hooks so block list collection always gives
empty block list and file size 0. That's unfortunate, because it also
means e.g. progress display is not possible.

btrfs data is checksummed so overwriting it would involve recomputing
checksums and replacing them. It is not possible to do in place because
checksums are checksummed themselves ... you get the idea.

btrfs can be on multiple devices so we cannot even expect grubenv to be
located on single disk; and blocklists simply do not support it.

btrfs can be compressed so you cannot even expect that new data will
fit into existing space.

> Having separate /boot on ext was fine as a short/medium term work around, but /boot on btrfs has use case benefits so it really needs to work eventually.
> 

So far nobody suggested solution for grubenv on unwritable location.
Not to mention implementation ...

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 1, 2014, at 11:27 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> В Sat, 1 Nov 2014 14:35:57 -0600
> Chris Murphy <lists@colorremedies.com> пишет:
>> 
>> Why not have a dedicated partition with MBR type code for core.img, equivalent to BIOSBoot currently used on GPT? freedesktop.org has a proposal to use type code 0xEA for this purpose (in part). The boot.img code in the MBR can arbitrarily jump to any LBA, so 0xEA doesn't need to be a primary partition does it?
>> 
> 
> It is rarely needed in simple cases; in complicated cases (btrfs or
> LVM) we already have space dedicated for core.img. It seems more
> logical to use this space.

Well actually it isn't rare in simple cases. The most common case on Linux is booting from ext which grub won't embed to either without --force. So we have to have core.img embedded somewhere else. UEFI it's a fixed location. BIOS+GPT it's a fixed location. Only on MBR is it in an unreserved location or forcibly embedded - that's really  where the problem is. It seems a lot simpler to constrain the MBR options down to only a reserved partition just like elsewhere where it's now much simpler because it can only properly go in one location.


> Also you still need to tell grub-setup to use this special partition at
> which point why not extend it to support arbitrary location for
> core.img? It could be made check partition type and not refuse to
> install on raw partition for special 0xEA type then as a bonus.

I never tell grub-setup to use BIOSboot partition type. It always just finds it automatically.



Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> В Sat, 1 Nov 2014 19:34:56 -0600
> Chris Murphy <lists@colorremedies.com> пишет:
> 
>> 
>> On Oct 30, 2014, at 6:42 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>>> 
>>> I still believe this is more flexible; in particular, /boot/grub on
>>> btrfs has problems with unwritable grubenv (quite a few people are hit
>>> by this now, when openSUSE defaults to single btrfs partition) so
>>> having separate /boot as ext2 makes sense.
>> 
>> Hmm, interesting. What's the nature of this problem with grubenv on btrfs? Is the current grubenv code expecting the file to be contiguous, and due to COW on btrfs it ends up not being contiguous? Does setting xattr +C on grubenv fix the problem?
>> 
> 
> btrfs code does not execute hooks so block list collection always gives
> empty block list and file size 0. That's unfortunate, because it also
> means e.g. progress display is not possible.
> 
> btrfs data is checksummed so overwriting it would involve recomputing
> checksums and replacing them. It is not possible to do in place because
> checksums are checksummed themselves ... you get the idea.
> 
> btrfs can be on multiple devices so we cannot even expect grubenv to be
> located on single disk; and blocklists simply do not support it.
> 
> btrfs can be compressed so you cannot even expect that new data will
> fit into existing space.

All the more reason for it to go in its own partition, rather than in some tiny area, or multiple tiny areas in filesystems. It's taken years to coordinate these things.


> 
>> Having separate /boot on ext was fine as a short/medium term work around, but /boot on btrfs has use case benefits so it really needs to work eventually.
>> 
> 
> So far nobody suggested solution for grubenv on unwritable location.
> Not to mention implementation …

Put grubenv on 0xEA/BIOSboot as well. It's GRUB's partition it can put anything it wants there, exactly where it expects to always find it. No coordination with filesystem groups required. I don't see any of the fundamental behaviors about Btrfs you're referring to changing anytime soon because it's simply how the fs works, they aren't bugs. So either GRUB gets its own partition with exclusive domain, or it continues to be hobbled by filesystem dependencies like needing to be forced installed on ext, can't be installed at all to XFS, and barely fits in 64KB on the Btrfs pad.

It's not like there's a shortage of partitions. And for future proofing this, enable GRUB to use more than one such partition. So if there's a 1MB 0xEA/BIOSboot and one day more space is needed, just add another one.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Michael Chang]

On Thu, Oct 30, 2014 at 03:42:29PM +0300, Andrei Borzenkov wrote:
> On Thu, Oct 30, 2014 at 11:32 AM, Michael Chang <mchang@suse.com> wrote:
> > Many shipped Windows created it's first partition aligned in 63
> > (cylinder) and therefore can't offer enough room for core.img. Even
> > worse the partitions has been created as logical.
> >
> >  > sudo /sbin/fdisk -l
> > Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
> > Units = sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > Disk label type: dos
> > Disk identifier: 0x0001c622
> >
> >    Device Boot      Start         End      Blocks   Id  System
> >    /dev/sda1              63     2056319     1028128+   b  W95 FAT32
> >    /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
> >    (LBA)
> >    /dev/sda5         2060288     5302271     1620992   82  Linux swap /
> >    Solaris
> >    /dev/sda6         5304320    47247359    20971520   83  Linux
> >    /dev/sda7        47249408   125804543    39277568   83  Linux
> >
> > This leaves us currently no option to succeed in installation if boot is
> > on btrfs, or any other filesystems that block lists can't be used and
> > core.img must be embedded in order to be reliably addressed.
> >
> > The attached patch try to workaround this scenario by placing the core.img
> > in filesystem's (btrfs) bootloader embedding area if available to overcome
> > the too small MBR gap which gets loaded by boot.img placed in MBR.
> >
> > Please kindly review the patch or suggests for how to fix this scenario
> > sanely.
> >
> 
> Well, I suggested something similar a way back
> 
> http://marc.info/?t=139175229300004&r=1&w=2

Thanks.

> 
> I still believe this is more flexible; in particular, /boot/grub on
> btrfs has problems with unwritable grubenv (quite a few people are hit
> by this now, when openSUSE defaults to single btrfs partition) so
> having separate /boot as ext2 makes sense.

But we can't constrain people from creating this setup if it makes sense
to them. For example they want to manage important kernel updates via
btrfs snapshots, etc.

> 
> Your approach looks too special cased for default (open)SUSE configuration.

The idea is basically treating the filesystem bootloader location as a
fallback install to the (preferred) mbr gaps, just like the blocklist
install will be used when embedding is not possible and core.img is
placed on filesystem.

The users of grub-install/grub-bios-setup may get used to take only stage1
install location into account and let the tools to figure out most feasible
setups for them, they don't have to care about where core.img will be placed
as this is what grub tools would make the best decision for them.

But I'm also second to your approach as it offers more flexibilty to
default behavior and can allow custom cases. With your patch (Will it
eventually be megerd into mainline by the way ?) this patch may not be
necessary, although it helps in setting up the bootloader by
grub-install witj less hassle, as it always did imho.

regards,
Michael

> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

В Sun, 2 Nov 2014 15:11:29 -0700
Chris Murphy <lists@colorremedies.com> пишет:

> 
> On Nov 1, 2014, at 11:27 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 
> > В Sat, 1 Nov 2014 14:35:57 -0600
> > Chris Murphy <lists@colorremedies.com> пишет:
> >> 
> >> Why not have a dedicated partition with MBR type code for core.img, equivalent to BIOSBoot currently used on GPT? freedesktop.org has a proposal to use type code 0xEA for this purpose (in part). The boot.img code in the MBR can arbitrarily jump to any LBA, so 0xEA doesn't need to be a primary partition does it?
> >> 
> > 
> > It is rarely needed in simple cases; in complicated cases (btrfs or
> > LVM) we already have space dedicated for core.img. It seems more
> > logical to use this space.
> 
> Well actually it isn't rare in simple cases. The most common case on Linux is booting from ext which grub won't embed to either without --force.xz_dec_lzma2

We are discussing installation in MBR here, not in partition. 

> So we have to have core.img embedded somewhere else. UEFI it's a fixed
 location. BIOS+GPT it's a fixed location. Only on MBR is it in an
 unreserved location or forcibly embedded - that's really  where the
 problem is. It seems a lot simpler to constrain the MBR options down
 to only a reserved partition just like elsewhere where it's now much
 simpler because it can only properly go in one location.
> 
> 
> > Also you still need to tell grub-setup to use this special partition at
> > which point why not extend it to support arbitrary location for
> > core.img? It could be made check partition type and not refuse to
> > install on raw partition for special 0xEA type then as a bonus.
> 
> I never tell grub-setup to use BIOSboot partition type. It always just finds it automatically.
> 

I do not really trust anyone respect partition types on MBR to be
honest. So I would rather avoid blindly overwriting anything without
explicit user's consent.

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

В Sun, 2 Nov 2014 15:19:43 -0700
Chris Murphy <lists@colorremedies.com> пишет:

> 
> On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 
> > В Sat, 1 Nov 2014 19:34:56 -0600
> > Chris Murphy <lists@colorremedies.com> пишет:
> > 
> >> 
> >> On Oct 30, 2014, at 6:42 AM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> >>> 
> >>> I still believe this is more flexible; in particular, /boot/grub on
> >>> btrfs has problems with unwritable grubenv (quite a few people are hit
> >>> by this now, when openSUSE defaults to single btrfs partition) so
> >>> having separate /boot as ext2 makes sense.
> >> 
> >> Hmm, interesting. What's the nature of this problem with grubenv on btrfs? Is the current grubenv code expecting the file to be contiguous, and due to COW on btrfs it ends up not being contiguous? Does setting xattr +C on grubenv fix the problem?
> >> 
> > 
> > btrfs code does not execute hooks so block list collection always gives
> > empty block list and file size 0. That's unfortunate, because it also
> > means e.g. progress display is not possible.
> > 
> > btrfs data is checksummed so overwriting it would involve recomputing
> > checksums and replacing them. It is not possible to do in place because
> > checksums are checksummed themselves ... you get the idea.
> > 
> > btrfs can be on multiple devices so we cannot even expect grubenv to be
> > located on single disk; and blocklists simply do not support it.
> > 
> > btrfs can be compressed so you cannot even expect that new data will
> > fit into existing space.
> 
> All the more reason for it to go in its own partition, rather than in some tiny area, or multiple tiny areas in filesystems. It's taken years to coordinate these things.
> 
> 
> > 
> >> Having separate /boot on ext was fine as a short/medium term work around, but /boot on btrfs has use case benefits so it really needs to work eventually.
> >> 
> > 
> > So far nobody suggested solution for grubenv on unwritable location.
> > Not to mention implementation …
> 
> Put grubenv on 0xEA/BIOSboot as well. 

Please provide scheme how grub can reliably identify which of multiple
grub partitions on multiple disks is *the* partition where grubenv is
located. 

And it still should work when embedding bootloader in partition. Both
with and without blocklists.

>It's GRUB's partition it can put anything it wants there, exactly where
it expects to always find it. No coordination with filesystem groups
required. I don't see any of the fundamental behaviors about Btrfs
you're referring to changing anytime soon because it's simply how the
fs works, they aren't bugs. So either GRUB gets its own partition with
exclusive domain, or it continues to be hobbled by filesystem
dependencies like needing to be forced installed on ext, can't be
installed at all to XFS, and barely fits in 64KB on the Btrfs pad.
> 
> It's not like there's a shortage of partitions.

There is in real life.

> And for future proofing this, enable GRUB to use more than one such
 partition. So if there's a 1MB 0xEA/BIOSboot and one day more space is
 needed, just add another one.
> 
> 
> Chris Murphy
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 2, 2014, at 10:31 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> В Sun, 2 Nov 2014 15:11:29 -0700
> Chris Murphy <lists@colorremedies.com> пишет:
> 
>> 
>> On Nov 1, 2014, at 11:27 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>> 
>>> В Sat, 1 Nov 2014 14:35:57 -0600
>>> Chris Murphy <lists@colorremedies.com> пишет:
>>>> 
>>>> Why not have a dedicated partition with MBR type code for core.img, equivalent to BIOSBoot currently used on GPT? freedesktop.org has a proposal to use type code 0xEA for this purpose (in part). The boot.img code in the MBR can arbitrarily jump to any LBA, so 0xEA doesn't need to be a primary partition does it?
>>>> 
>>> 
>>> It is rarely needed in simple cases; in complicated cases (btrfs or
>>> LVM) we already have space dedicated for core.img. It seems more
>>> logical to use this space.
>> 
>> Well actually it isn't rare in simple cases. The most common case on Linux is booting from ext which grub won't embed to either without --force.xz_dec_lzma2
> 
> We are discussing installation in MBR here, not in partition. 

The point is that GRUB has difficulty even in the simple case because it either can't fit in the MBR gap, or something else could just as legitimately use the MBR gap. Right now the fallback depends on a complex matrix depending on the file system being used rather than the primary use case rarely depending on fallback because it's a sure thing and consistent regardless of file system used.



> 
>> So we have to have core.img embedded somewhere else. UEFI it's a fixed
> location. BIOS+GPT it's a fixed location. Only on MBR is it in an
> unreserved location or forcibly embedded - that's really  where the
> problem is. It seems a lot simpler to constrain the MBR options down
> to only a reserved partition just like elsewhere where it's now much
> simpler because it can only properly go in one location.
>> 
>> 
>>> Also you still need to tell grub-setup to use this special partition at
>>> which point why not extend it to support arbitrary location for
>>> core.img? It could be made check partition type and not refuse to
>>> install on raw partition for special 0xEA type then as a bonus.
>> 
>> I never tell grub-setup to use BIOSboot partition type. It always just finds it automatically.
>> 
> 
> I do not really trust anyone respect partition types on MBR to be
> honest. So I would rather avoid blindly overwriting anything without
> explicit user's consent.

These two things are orthogonal. Relying on the user to know even basic things about how computers boot, is unreasonable. They don't understand such things, they never will, and expecting them to is simply out of scope.

If you don't trust others to respect partition types, then how do you trust them to respect non-partitioned regions like the MBR gap? The concept works quite well on UEFI and BIOS+GPT, I don't see why it should be any different on MBR, except for the small problem that we don't have as many type codes available. But certainly even that problem is more reliable than nothing at all.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 2, 2014, at 10:36 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> В Sun, 2 Nov 2014 15:19:43 -0700
> Chris Murphy <lists@colorremedies.com> пишет:
> 
>> 
>> On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>>> 
>>> So far nobody suggested solution for grubenv on unwritable location.
>>> Not to mention implementation …
>> 
>> Put grubenv on 0xEA/BIOSboot as well. 
> 
> Please provide scheme how grub can reliably identify which of multiple
> grub partitions on multiple disks is *the* partition where grubenv is
> located. 

Why would the policy be any different than it is now? This isn't a new problem, we already have multiple MBR gaps and hence multiple core.img instances. There is no real change other than explicitly defining a suitably sized partition for GRUB's things to go in, rather than depending on an unreliable location, i.e. the MBR gap which often is either too small or could just as likely be used by something else since it's not reserved space for anyone.

> 
> And it still should work when embedding bootloader in partition. Both
> with and without blocklists.

I'm not asking for any one else's workflow to become broken. I'm saying the primary workflow should be, primary, as in, consistent regardless of the file system used.

> 
>> It's GRUB's partition it can put anything it wants there, exactly where
> it expects to always find it. No coordination with filesystem groups
> required. I don't see any of the fundamental behaviors about Btrfs
> you're referring to changing anytime soon because it's simply how the
> fs works, they aren't bugs. So either GRUB gets its own partition with
> exclusive domain, or it continues to be hobbled by filesystem
> dependencies like needing to be forced installed on ext, can't be
> installed at all to XFS, and barely fits in 64KB on the Btrfs pad.
>> 
>> It's not like there's a shortage of partitions.
> 
> There is in real life.

Not really. There's a shortage of primary partitions on MBR but *if* GRUB boot.img is permitted on the MBR, which is the default use case, then primary partitions aren't needed for core.img+grubenv+grub.cfg, those can go on an extended partition. And then even if Linux /boot is corrupt, I could still get a working GRUB menu that will permit me to boot e.g. Windows on the same disk. Right now this fails.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 2, 2014, at 9:17 PM, Michael Chang <mchang@suse.com> wrote:

> On Thu, Oct 30, 2014 at 03:42:29PM +0300, Andrei Borzenkov wrote:
>> On Thu, Oct 30, 2014 at 11:32 AM, Michael Chang <mchang@suse.com> wrote:
>>> Many shipped Windows created it's first partition aligned in 63
>>> (cylinder) and therefore can't offer enough room for core.img. Even
>>> worse the partitions has been created as logical.
>>> 
>>>> sudo /sbin/fdisk -l
>>> Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
>>> Units = sectors of 1 * 512 = 512 bytes
>>> Sector size (logical/physical): 512 bytes / 512 bytes
>>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>>> Disk label type: dos
>>> Disk identifier: 0x0001c622
>>> 
>>>   Device Boot      Start         End      Blocks   Id  System
>>>   /dev/sda1              63     2056319     1028128+   b  W95 FAT32
>>>   /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
>>>   (LBA)
>>>   /dev/sda5         2060288     5302271     1620992   82  Linux swap /
>>>   Solaris
>>>   /dev/sda6         5304320    47247359    20971520   83  Linux
>>>   /dev/sda7        47249408   125804543    39277568   83  Linux
>>> 
>>> This leaves us currently no option to succeed in installation if boot is
>>> on btrfs, or any other filesystems that block lists can't be used and
>>> core.img must be embedded in order to be reliably addressed.
>>> 
>>> The attached patch try to workaround this scenario by placing the core.img
>>> in filesystem's (btrfs) bootloader embedding area if available to overcome
>>> the too small MBR gap which gets loaded by boot.img placed in MBR.
>>> 
>>> Please kindly review the patch or suggests for how to fix this scenario
>>> sanely.
>>> 
>> 
>> Well, I suggested something similar a way back
>> 
>> http://marc.info/?t=139175229300004&r=1&w=2
> 
> Thanks.
> 
>> 
>> I still believe this is more flexible; in particular, /boot/grub on
>> btrfs has problems with unwritable grubenv (quite a few people are hit
>> by this now, when openSUSE defaults to single btrfs partition) so
>> having separate /boot as ext2 makes sense.
> 
> But we can't constrain people from creating this setup if it makes sense
> to them. For example they want to manage important kernel updates via
> btrfs snapshots, etc.

Well, Btrfs raid1 might make sense for some of them but opensuse will constrain people from multiple device btrfs. So you can just disallow them from /boot on Btrfs by default, and make /boot on ext bigger than 500MB so it can accumulate more kernels.

When it comes to booting multiple tree OS's, I prefer the design and capabilities of OStree, even though it's not particularly Btrfs aware or optimized yet. At least it's very aware of both the tree states, and totally manages the bootloader configuration when changing trees (e.g. rollback) via bootloaderspec drop in scripts.

> 
>> 
>> Your approach looks too special cased for default (open)SUSE configuration.
> 
> The idea is basically treating the filesystem bootloader location as a
> fallback install to the (preferred) mbr gaps, just like the blocklist
> install will be used when embedding is not possible and core.img is
> placed on filesystem.

OK fine, but this just adds to the already complicated matrix of GRUB MBR installations, to have yet another fallback for yet another special case. This proves the primary case is broken that so many fallback methods are even necessary. Already the ESP and BIOSboot workflows are vastly more reliable, and completely consistent regardless of the filesystems being used. It's the MBR gap workflow that's busted. If the primary use case is fixed, most use cases will inherit the benefit with no additional work or testing.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

В Mon, 3 Nov 2014 12:36:24 -0700
Chris Murphy <lists@colorremedies.com> пишет:

> 
> On Nov 2, 2014, at 10:36 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 
> > В Sun, 2 Nov 2014 15:19:43 -0700
> > Chris Murphy <lists@colorremedies.com> пишет:
> > 
> >> 
> >> On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> >>> 
> >>> So far nobody suggested solution for grubenv on unwritable location.
> >>> Not to mention implementation …
> >> 
> >> Put grubenv on 0xEA/BIOSboot as well. 
> > 
> > Please provide scheme how grub can reliably identify which of multiple
> > grub partitions on multiple disks is *the* partition where grubenv is
> > located. 
> 
> Why would the policy be any different than it is now? This isn't a new problem, we already have multiple MBR gaps and hence multiple core.img instances.

Exactly. We have multiple core.img but single location for grub state
and configuration information. It does not matter which core.img is
loaded as long as they all refer to the same /boot/grub. How can
multiple core.img refer to the same 0xEA partition to be sure they read
(and write!) the same grubenv?

> There is no real change other than explicitly defining a suitably
 sized partition for GRUB's things to go in, rather than depending on
 an unreliable location, i.e. the MBR gap which often is either too
 small or could just as likely be used by something else since it's not
 reserved space for anyone.
> 
> > 
> > And it still should work when embedding bootloader in partition. Both
> > with and without blocklists.
> 
> I'm not asking for any one else's workflow to become broken. I'm saying the primary workflow should be, primary, as in, consistent regardless of the file system used.

Please explain how grub should know when to access /boot/grub/grubenv
and when to access 0xEA partition.

> 
> > 
> >> It's GRUB's partition it can put anything it wants there, exactly where
> > it expects to always find it. No coordination with filesystem groups
> > required. I don't see any of the fundamental behaviors about Btrfs
> > you're referring to changing anytime soon because it's simply how the
> > fs works, they aren't bugs. So either GRUB gets its own partition with
> > exclusive domain, or it continues to be hobbled by filesystem
> > dependencies like needing to be forced installed on ext, can't be
> > installed at all to XFS, and barely fits in 64KB on the Btrfs pad.
> >> 
> >> It's not like there's a shortage of partitions.
> > 
> > There is in real life.
> 
> Not really. There's a shortage of primary partitions on MBR but *if* GRUB boot.img is permitted on the MBR, which is the default use case, then primary partitions aren't needed for core.img+grubenv+grub.cfg, those can go on an extended partition. And then even if Linux /boot is corrupt, I could still get a working GRUB menu that will permit me to boot e.g. Windows on the same disk. Right now this fails.
> 
> 
> Chris Murphy
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 3, 2014, at 1:29 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> В Mon, 3 Nov 2014 12:36:24 -0700
> Chris Murphy <lists@colorremedies.com> пишет:
> 
>> 
>> On Nov 2, 2014, at 10:36 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>> 
>>> В Sun, 2 Nov 2014 15:19:43 -0700
>>> Chris Murphy <lists@colorremedies.com> пишет:
>>> 
>>>> 
>>>> On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>>>>> 
>>>>> So far nobody suggested solution for grubenv on unwritable location.
>>>>> Not to mention implementation …
>>>> 
>>>> Put grubenv on 0xEA/BIOSboot as well. 
>>> 
>>> Please provide scheme how grub can reliably identify which of multiple
>>> grub partitions on multiple disks is *the* partition where grubenv is
>>> located. 
>> 
>> Why would the policy be any different than it is now? This isn't a new problem, we already have multiple MBR gaps and hence multiple core.img instances.
> 
> Exactly. We have multiple core.img but single location for grub state
> and configuration information. It does not matter which core.img is
> loaded as long as they all refer to the same /boot/grub.

Which is fragile, BTW, because if Linux /boot becomes corrupt, now I don't even get a GRUB menu and can't boot any other OS on the same system, e.g. Windows or a 2nd Linux instance. GRUB as installed to a device should be completely self contained, it means fewer single points of failure.


> How can
> multiple core.img refer to the same 0xEA partition to be sure they read
> (and write!) the same grubenv?

There's no assurance they do this now. Multiple core.img can have multiple roots on multiple devices, so each may point to different grubenv anyway. We can't be sure they all point to the same grubenv in any case.

HOWEVER, I understand the confusion, it's my fault. 0xEA proposed by Bootloaderspec is FAT32 to be used for Linux /boot and shared. That has all sorts of problems that aren't relevant in this thread, but it's not the equivalent of BIOSboot on GPT, which is an unformatted partition.

I'm suggesting an MBR partition type, equivalent to the BIOSboot partition, for embedding core.img instead of the MBR gap, so that there is always a reliable location for GRUB. If devs want it FAT32 like on UEFI, fine. If you want it unformatted like BIOSboot, fine. I have nothing to say about that. I just want to see the primary use case for installing GRUB on MBR partition drives to actually be the primary use case, rather than seemingly always having to fallback on special cases.

For example on Fedora, since the installer change, they no longer use grub-install --force to install GRUB to ext4 /boot and this has really made a lot of users angry and most of them refuse to learn how to install it manually instead they claim to have moved to different distributions that still use --force by default. 

For example, opensuse's GUI installer still uses --force by default, which by definition is a special case. Their *default* most common case, is now using a workflow explicitly not recommended by GRUB upstream. And the reason why is because the simple case, installing to MBR gap, is unreliable. It has been for a very long time.



> 
>> There is no real change other than explicitly defining a suitably
> sized partition for GRUB's things to go in, rather than depending on
> an unreliable location, i.e. the MBR gap which often is either too
> small or could just as likely be used by something else since it's not
> reserved space for anyone.
>> 
>>> 
>>> And it still should work when embedding bootloader in partition. Both
>>> with and without blocklists.
>> 
>> I'm not asking for any one else's workflow to become broken. I'm saying the primary workflow should be, primary, as in, consistent regardless of the file system used.
> 
> Please explain how grub should know when to access /boot/grub/grubenv
> and when to access 0xEA partition.

Move core.img+grubenv+grub.cfg to the same partition. Do this for UEFI's ESP, and BIOSBoot, and a suitable explicit partition as replacement for MBR gap.

This is also easier to replicate across multiple disks, for raid1/5/6 booting.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Michael Chang]

On Mon, Nov 03, 2014 at 01:04:33PM -0700, Chris Murphy wrote:
> 
> On Nov 2, 2014, at 9:17 PM, Michael Chang <mchang@suse.com> wrote:
> 
> > On Thu, Oct 30, 2014 at 03:42:29PM +0300, Andrei Borzenkov wrote:
> >> On Thu, Oct 30, 2014 at 11:32 AM, Michael Chang <mchang@suse.com> wrote:
> >>> Many shipped Windows created it's first partition aligned in 63
> >>> (cylinder) and therefore can't offer enough room for core.img. Even
> >>> worse the partitions has been created as logical.
> >>> 
> >>>> sudo /sbin/fdisk -l
> >>> Disk /dev/sda: 64.4 GB, 64424509440 bytes, 125829120 sectors
> >>> Units = sectors of 1 * 512 = 512 bytes
> >>> Sector size (logical/physical): 512 bytes / 512 bytes
> >>> I/O size (minimum/optimal): 512 bytes / 512 bytes
> >>> Disk label type: dos
> >>> Disk identifier: 0x0001c622
> >>> 
> >>>   Device Boot      Start         End      Blocks   Id  System
> >>>   /dev/sda1              63     2056319     1028128+   b  W95 FAT32
> >>>   /dev/sda2   *     2058240   125829119    61885440    f  W95 Ext'd
> >>>   (LBA)
> >>>   /dev/sda5         2060288     5302271     1620992   82  Linux swap /
> >>>   Solaris
> >>>   /dev/sda6         5304320    47247359    20971520   83  Linux
> >>>   /dev/sda7        47249408   125804543    39277568   83  Linux
> >>> 
> >>> This leaves us currently no option to succeed in installation if boot is
> >>> on btrfs, or any other filesystems that block lists can't be used and
> >>> core.img must be embedded in order to be reliably addressed.
> >>> 
> >>> The attached patch try to workaround this scenario by placing the core.img
> >>> in filesystem's (btrfs) bootloader embedding area if available to overcome
> >>> the too small MBR gap which gets loaded by boot.img placed in MBR.
> >>> 
> >>> Please kindly review the patch or suggests for how to fix this scenario
> >>> sanely.
> >>> 
> >> 
> >> Well, I suggested something similar a way back
> >> 
> >> http://marc.info/?t=139175229300004&r=1&w=2
> > 
> > Thanks.
> > 
> >> 
> >> I still believe this is more flexible; in particular, /boot/grub on
> >> btrfs has problems with unwritable grubenv (quite a few people are hit
> >> by this now, when openSUSE defaults to single btrfs partition) so
> >> having separate /boot as ext2 makes sense.
> > 
> > But we can't constrain people from creating this setup if it makes sense
> > to them. For example they want to manage important kernel updates via
> > btrfs snapshots, etc.
> 
> Well, Btrfs raid1 might make sense for some of them but opensuse will constrain people from multiple device btrfs. So you can just disallow them from /boot on Btrfs by default, and make /boot on ext bigger than 500MB so it can accumulate more kernels.

On the other hand we do not want to maintain the nightmare of proposing
different disk partitioning for different bootloader/filesystem
combinations. The most significant advantage of grub is it able to
provide unified setups for as many file systems it supports as well as
the cpu-firmware architectures.

The principle is grub can support booting btrfs directly, then separate
/boot will not be considered. We don't want to revive it because of this
particular case.

> 
> When it comes to booting multiple tree OS's, I prefer the design and capabilities of OStree, even though it's not particularly Btrfs aware or optimized yet. At least it's very aware of both the tree states, and totally manages the bootloader configuration when changing trees (e.g. rollback) via bootloaderspec drop in scripts.

It sounds to me that OSTree is a general way to manage your versioned
system binaries exported from build server. Out of my curious why it has
anything to do with btrfs? Anyway many thanks for pointing me to this
cool project. :)

> 
> > 
> >> 
> >> Your approach looks too special cased for default (open)SUSE configuration.
> > 
> > The idea is basically treating the filesystem bootloader location as a
> > fallback install to the (preferred) mbr gaps, just like the blocklist
> > install will be used when embedding is not possible and core.img is
> > placed on filesystem.
> 
> OK fine, but this just adds to the already complicated matrix of GRUB MBR installations, to have yet another fallback for yet another special case. This proves the primary case is broken that so many fallback methods are even necessary. Already the ESP and BIOSboot workflows are vastly more reliable, and completely consistent regardless of the filesystems being used. It's the MBR gap workflow that's busted. If the primary use case is fixed, most use cases will inherit the benefit with no additional work or testing.

We like BIOSboot on GPT, because we know it's value to the bootloader.
But most people just don't realize it and complain grub will require
additioanal partition to work, which is not for leagcy-grub.

The same thing on msdos could be even worse as there's no spec for it.
The biggest challenge is even not any technical stuff but people has
got used to it for years. Not always the good solution will win.

That is I think BIOSboot for msdos makes sense, but better use it as a
(yet another) fallback location for mbr gap not a replacement.

regards,
Michael

Re: workaround install boot on btrfs with windows partition scheme

[Andrei Borzenkov]

В Mon, 3 Nov 2014 14:05:24 -0700
Chris Murphy <lists@colorremedies.com> пишет:

> 
> On Nov 3, 2014, at 1:29 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> 
> > В Mon, 3 Nov 2014 12:36:24 -0700
> > Chris Murphy <lists@colorremedies.com> пишет:
> > 
> >> 
> >> On Nov 2, 2014, at 10:36 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> >> 
> >>> В Sun, 2 Nov 2014 15:19:43 -0700
> >>> Chris Murphy <lists@colorremedies.com> пишет:
> >>> 
> >>>> 
> >>>> On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
> >>>>> 
> >>>>> So far nobody suggested solution for grubenv on unwritable location.
> >>>>> Not to mention implementation …
> >>>> 
> >>>> Put grubenv on 0xEA/BIOSboot as well. 
> >>> 
> >>> Please provide scheme how grub can reliably identify which of multiple
> >>> grub partitions on multiple disks is *the* partition where grubenv is
> >>> located. 
> >> 
> >> Why would the policy be any different than it is now? This isn't a new problem, we already have multiple MBR gaps and hence multiple core.img instances.
> > 
> > Exactly. We have multiple core.img but single location for grub state
> > and configuration information. It does not matter which core.img is
> > loaded as long as they all refer to the same /boot/grub.
> 
> Which is fragile, BTW, because if Linux /boot becomes corrupt, now I don't even get a GRUB menu and can't boot any other OS on the same system, e.g. Windows or a 2nd Linux instance. GRUB as installed to a device should be completely self contained, it means fewer single points of failure.

It is self contained if you treat /boot/grub as bootloader partition.
You are free to create separate filesystem for it and keep it unmounted
by default.

> 
> 
> > How can
> > multiple core.img refer to the same 0xEA partition to be sure they read
> > (and write!) the same grubenv?
> 
> There's no assurance they do this now. Multiple core.img can have multiple roots on multiple devices, so each may point to different grubenv anyway. We can't be sure they all point to the same grubenv in any case.
> 
> HOWEVER, I understand the confusion, it's my fault. 0xEA proposed by Bootloaderspec is FAT32 to be used for Linux /boot and shared. That has all sorts of problems that aren't relevant in this thread, but it's not the equivalent of BIOSboot on GPT, which is an unformatted partition.
> 
> I'm suggesting an MBR partition type, equivalent to the BIOSboot partition, for embedding core.img instead of the MBR gap, so that there is always a reliable location for GRUB. If devs want it FAT32 like on UEFI, fine. If you want it unformatted like BIOSboot, fine. I have nothing to say about that. I just want to see the primary use case for installing GRUB on MBR partition drives to actually be the primary use case, rather than seemingly always having to fallback on special cases.
> 
> For example on Fedora, since the installer change, they no longer use grub-install --force to install GRUB to ext4 /boot and this has really made a lot of users angry and most of them refuse to learn how to install it manually instead they claim to have moved to different distributions that still use --force by default. 
> 
> For example, opensuse's GUI installer still uses --force by default, which by definition is a special case. Their *default* most common case, is now using a workflow explicitly not recommended by GRUB upstream. And the reason why is because the simple case, installing to MBR gap, is unreliable. It has been for a very long time.
> 
> 
> 
> > 
> >> There is no real change other than explicitly defining a suitably
> > sized partition for GRUB's things to go in, rather than depending on
> > an unreliable location, i.e. the MBR gap which often is either too
> > small or could just as likely be used by something else since it's not
> > reserved space for anyone.
> >> 
> >>> 
> >>> And it still should work when embedding bootloader in partition. Both
> >>> with and without blocklists.
> >> 
> >> I'm not asking for any one else's workflow to become broken. I'm saying the primary workflow should be, primary, as in, consistent regardless of the file system used.
> > 
> > Please explain how grub should know when to access /boot/grub/grubenv
> > and when to access 0xEA partition.
> 
> Move core.img+grubenv+grub.cfg to the same partition. Do this for UEFI's ESP, and BIOSBoot, and a suitable explicit partition as replacement for MBR gap.
> 

How do you edit grub.cfg which is now in raw partition?

> This is also easier to replicate across multiple disks, for raid1/5/6 booting.
> 

Which of replica across multiple disks holds *the* authoritative copy
of grubenv and grub.cfg?

> 
> Chris Murphy
> 

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 3, 2014, at 10:50 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:

> В Mon, 3 Nov 2014 14:05:24 -0700
> Chris Murphy <lists@colorremedies.com> пишет:
> 
>> 
>> On Nov 3, 2014, at 1:29 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>> 
>>> В Mon, 3 Nov 2014 12:36:24 -0700
>>> Chris Murphy <lists@colorremedies.com> пишет:
>>> 
>>>> 
>>>> On Nov 2, 2014, at 10:36 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>>>> 
>>>>> В Sun, 2 Nov 2014 15:19:43 -0700
>>>>> Chris Murphy <lists@colorremedies.com> пишет:
>>>>> 
>>>>>> 
>>>>>> On Nov 1, 2014, at 11:37 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
>>>>>>> 
>>>>>>> So far nobody suggested solution for grubenv on unwritable location.
>>>>>>> Not to mention implementation …
>>>>>> 
>>>>>> Put grubenv on 0xEA/BIOSboot as well. 
>>>>> 
>>>>> Please provide scheme how grub can reliably identify which of multiple
>>>>> grub partitions on multiple disks is *the* partition where grubenv is
>>>>> located. 
>>>> 
>>>> Why would the policy be any different than it is now? This isn't a new problem, we already have multiple MBR gaps and hence multiple core.img instances.
>>> 
>>> Exactly. We have multiple core.img but single location for grub state
>>> and configuration information. It does not matter which core.img is
>>> loaded as long as they all refer to the same /boot/grub.
>> 
>> Which is fragile, BTW, because if Linux /boot becomes corrupt, now I don't even get a GRUB menu and can't boot any other OS on the same system, e.g. Windows or a 2nd Linux instance. GRUB as installed to a device should be completely self contained, it means fewer single points of failure.
> 
> It is self contained if you treat /boot/grub as bootloader partition.
> You are free to create separate filesystem for it and keep it unmounted
> by default.

Sure but no distro actually does this, and we're all making an even worse mistake with /boot/efi being persistently mounted for no good reason also.


> 
>> 
>> 
>>> How can
>>> multiple core.img refer to the same 0xEA partition to be sure they read
>>> (and write!) the same grubenv?
>> 
>> There's no assurance they do this now. Multiple core.img can have multiple roots on multiple devices, so each may point to different grubenv anyway. We can't be sure they all point to the same grubenv in any case.
>> 
>> HOWEVER, I understand the confusion, it's my fault. 0xEA proposed by Bootloaderspec is FAT32 to be used for Linux /boot and shared. That has all sorts of problems that aren't relevant in this thread, but it's not the equivalent of BIOSboot on GPT, which is an unformatted partition.
>> 
>> I'm suggesting an MBR partition type, equivalent to the BIOSboot partition, for embedding core.img instead of the MBR gap, so that there is always a reliable location for GRUB. If devs want it FAT32 like on UEFI, fine. If you want it unformatted like BIOSboot, fine. I have nothing to say about that. I just want to see the primary use case for installing GRUB on MBR partition drives to actually be the primary use case, rather than seemingly always having to fallback on special cases.
>> 
>> For example on Fedora, since the installer change, they no longer use grub-install --force to install GRUB to ext4 /boot and this has really made a lot of users angry and most of them refuse to learn how to install it manually instead they claim to have moved to different distributions that still use --force by default. 
>> 
>> For example, opensuse's GUI installer still uses --force by default, which by definition is a special case. Their *default* most common case, is now using a workflow explicitly not recommended by GRUB upstream. And the reason why is because the simple case, installing to MBR gap, is unreliable. It has been for a very long time.
>> 
>> 
>> 
>>> 
>>>> There is no real change other than explicitly defining a suitably
>>> sized partition for GRUB's things to go in, rather than depending on
>>> an unreliable location, i.e. the MBR gap which often is either too
>>> small or could just as likely be used by something else since it's not
>>> reserved space for anyone.
>>>> 
>>>>> 
>>>>> And it still should work when embedding bootloader in partition. Both
>>>>> with and without blocklists.
>>>> 
>>>> I'm not asking for any one else's workflow to become broken. I'm saying the primary workflow should be, primary, as in, consistent regardless of the file system used.
>>> 
>>> Please explain how grub should know when to access /boot/grub/grubenv
>>> and when to access 0xEA partition.
>> 
>> Move core.img+grubenv+grub.cfg to the same partition. Do this for UEFI's ESP, and BIOSBoot, and a suitable explicit partition as replacement for MBR gap.
>> 
> 
> How do you edit grub.cfg which is now in raw partition?

# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates

It's explicitly not user domain. Arguably 3rd party tools shouldn't be editing it either.

Instead there should be a simple drop in script format to inform GRUB how to boot other OS's, or trees, on the system. Then grub.cfg doesn't need to be edited by either user or distribution, let alone shared among multiple distributions (which is really quite an ugly mess). But still permits distributions to modify GRUB menu entries by creating/managing distro specific boot scripts. Very simple ones. This is probably the most useful aspect of Bootloaderspec.


> 
>> This is also easier to replicate across multiple disks, for raid1/5/6 booting.
>> 
> 
> Which of replica across multiple disks holds *the* authoritative copy
> of grubenv and grub.cfg?

Well in a perfect world they'd all be identical, made so at grub-install and mkconfig time, so it wouldn't matter. This envisions agreement to follow a spec, including shared space for bootloader configuration drop in scripts. No one is touching the grub.cfg, or anyone else's scripts. Each distro manages their own scripts.

In reality, each device could have a core.img and grubenv produced by a distro specific GRUB, in which case it's up to the BIOS and whether the grub.cfg it effectively causes to load was created to include awareness of other OS's on the system. It's no different than now except the grubenv is co-located with core.img so that it's ensured to be in a reliable read/write location. Plus grubenv isn't user domain either, we're supposed to use grub-editenv not directly edit the file.


Chris Murphy

Re: workaround install boot on btrfs with windows partition scheme

[Chris Murphy]

On Nov 3, 2014, at 9:50 PM, Michael Chang <mchang@suse.com> wrote:

> On Mon, Nov 03, 2014 at 01:04:33PM -0700, Chris Murphy wrote:
>> 
>> Well, Btrfs raid1 might make sense for some of them but opensuse will constrain people from multiple device btrfs. So you can just disallow them from /boot on Btrfs by default, and make /boot on ext bigger than 500MB so it can accumulate more kernels.
> 
> On the other hand we do not want to maintain the nightmare of proposing
> different disk partitioning for different bootloader/filesystem
> combinations. The most significant advantage of grub is it able to
> provide unified setups for as many file systems it supports as well as
> the cpu-firmware architectures.
> 
> The principle is grub can support booting btrfs directly, then separate
> /boot will not be considered. We don't want to revive it because of this
> particular case.

Yeah I understand and agree with this. It really needs to work for Btrfs. 

> 
>> 
>> When it comes to booting multiple tree OS's, I prefer the design and capabilities of OStree, even though it's not particularly Btrfs aware or optimized yet. At least it's very aware of both the tree states, and totally manages the bootloader configuration when changing trees (e.g. rollback) via bootloaderspec drop in scripts.
> 
> It sounds to me that OSTree is a general way to manage your versioned
> system binaries exported from build server. Out of my curious why it has
> anything to do with btrfs? Anyway many thanks for pointing me to this
> cool project. :)

There are now several ways of having snapshottable and bootable filesystems. Do we want to maintain the nightmare of proposing different ways for booting each of them, or come up with a common scheme?  OSTree could be a model for a common scheme, even if it's not a specific solution in every case.

I'm convinced though, without agreement, the distros will inevitably arrive at totally incompatible schemes far worse than what we've seen up to this point. They're not quite openly hostile to each other, but they're certainly not friendly when it comes to preserving bootability for multiple Linux installations. For example even without considering snapshots, Fedora and openSUSE use completely different ondisk layouts and fs assembly at boot time, and this necessarily affects how each OS is discovered and booted were they to both be on one system at the same time sharing one bootloader as on BIOS/MBR.

Meanwhile OSTree is agnostic to the underlying storage technology used, while desiring future leveraging of those technologies as optimizations (rather than as dependencies). It also now works on BIOS and UEFI systems. It uses Bootloaderspec [1] drop in scripts rather than modifying grub.cfg. And big bonus (although it's the project's raison d'être) is all software updates are always atomic: the tree being modified is not the currently active tree so libraries aren't being yanked out from under any process. So as a rather young project it's getting a lot of things right. Its philosophy is what I think makes it relevant to Btrfs booting even if not the present state of the code.


>> 
>> OK fine, but this just adds to the already complicated matrix of GRUB MBR installations, to have yet another fallback for yet another special case. This proves the primary case is broken that so many fallback methods are even necessary. Already the ESP and BIOSboot workflows are vastly more reliable, and completely consistent regardless of the filesystems being used. It's the MBR gap workflow that's busted. If the primary use case is fixed, most use cases will inherit the benefit with no additional work or testing.
> 
> We like BIOSboot on GPT, because we know it's value to the bootloader.
> But most people just don't realize it and complain grub will require
> additioanal partition to work, which is not for leagcy-grub.
> 
> The same thing on msdos could be even worse as there's no spec for it.
> The biggest challenge is even not any technical stuff but people has
> got used to it for years. Not always the good solution will win.

Sadly true. But I still think that change is completely fair game when it accommodates and streamlines the most common use cases, while permitting the edge cases.

And as for no spec:

a. Let's make one. Bootloaderspec is weak in this area, and is sorta begging for clarity.
b. There's no spec for BIOS+GPT either, yet in effect that combination requires BIOSboot or grub-install fails.

The reason we can get away without a spec for BIOS+GPT is because the constraint essentially requires one workaround where as on MBR it's very loosy goosy, there are 10 ways to solve one problem because of the lack of constraint. So what it takes to solve this on BIOS+MBR is commitment, so I'm perfectly OK with a written spec.


> That is I think BIOSboot for msdos makes sense, but better use it as a
> (yet another) fallback location for mbr gap not a replacement.

It's not a hill I'm willing to die on, but I don't agree. There's essentially no downside except some hypothetical vocal complainers. Default grub-install command should work on MBR disks with the most common layouts and certainly partition 1 beginning at LBA 63 is very common, yet it's treated as an exception requiring fallback. Only because many distributions use --force by default have users not complained a lot more about this. The right way to solve this for Btrfs is not a Btrfs exception, and an XFS exception, and an LVM exception. All three of those (common or increasingly so) cases are solved singularly with BIOSBoot. It's just a matter of committing to a broad solution rather than more fallbacks.



Chris Murphy