[patch 0/2] grub-0.97: btrfs support

[patch 0/2] grub-0.97: btrfs support

[Edward Shishkin]

Hello everyone.

The following patches are for Fedora 10(**).
The distro-independent package will be put to kernel.org a bit later.


           I. Loading kernels from btrfs volumes


Now you can load kernels and initrds from btrfs volumes composed of
many devices.

WARNING!!!
Make sure that all components of your loading btrfs volume(*) are
visible to grub. Otherwise, you'll end with unbootable system.
The list of available grub devices can be obtained, for example,
using tab completion in grub shell.

Number of components of a loading volume is not restricted, however
if it is larger then 128, then the boot process will be slowed down
because of expensive translations (btrfs-device-id -> grub-device-id)
which issue a large number of IOs. We cache only 128 such translations
in grub-0.97 because of high memory pressure.


           II. Installing grub from btrfs volumes


You can install grub from a btrfs image volume(*) composed of many
devices (see above about restrictions). Also you can setup any
component of a btrfs boot(*) volume as grub root device.

NOTE!!! Make sure that all components of image and boot volumes(*)
are visible to grub, otherwise grub installer will return error.

TECHNICAL NOTE (for grub developers):
The unpleasant surprise was that grub installer overwrites
(by default!) the file (stage2), bypassing the file system driver.
I can not understand this: it looks like stepping to the clean water
with dirty shoe. Hope that grub2 won't afford such things.

In order to install grub from a btrfs image volume use special
option (--stage2). This option makes grub installer to rewrite
the file with a help of the OS's file system (i.e, via write (2)).
Any attempts to install without this option will fail with an error
(wrong argument).

The example of possible installation scenario.

Suppose image volume = root volume = loading volume is composed
of devices (hd0,4), (hd0,5), (hd1,5), (hd1,7) and is not an OS's
root. We want to setup (hd0,4) as grub root device and install
grub to the mbr of (hd0).

. build and install grub with btrfs support;
. mount your the "3 in 1" btrfs volume to /mnt;
. create a directory /mnt/grub;
. put the built files stage1, stage2, btrfs_stage1_5, grub.conf,
 etc. to /mnt/grub;
. run grub shell;
. grub> root (hd0,4)
. grub> setup --stage2=/mnt/grub/stage2 (hd0)
. have a fun.

Use info(1) grub for more details.

(*) Glossary:
. loading volume:
 a btrfs volume that contains kernel image and initrd;

. image volume:
 a btrfs volume that contains stage1, stage2, btrfs_stage_1_5,
 and grub.conf files needed for grub installer;

. boot volume:
 a btrfs volume where grub will look for stage2 and grub.conf
 files in boot time.

(**) Link to the Fedora's grub package:
http://ucho.ignum.cz/fedora/linux/releases/10/Fedora/source/SRPMS/grub-0.97-38.fc10.src.rpm 


All comments, bugreports, etc. are welcome as usual.

Thanks,
Edward.

Re: [patch 0/2] grub-0.97: btrfs support

[Johannes Hirte]

Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
> Hello everyone.
... 
> The following patches are for Fedora 10(**).
> The distro-independent package will be put to kernel.org a bit later.
> 
> 
> All comments, bugreports, etc. are welcome as usual.

Ok, I have another comment/bugreport *g*.

I'm testing this patch with gentoo, so the grub sources are not identicaly the 
same. With this patches applied, grub is unable to detect JFS or XFS 
filesystems. XFS is reported as unknown, JFS is reported as btrfs. Reiserfs and 
ext2/3 are detected as expected.

regards,
  Johannes

Re: [patch 0/2] grub-0.97: btrfs support

[Johannes Hirte]

Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
> Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
> > Hello everyone.
> 
> ...
> 
> > The following patches are for Fedora 10(**).
> > The distro-independent package will be put to kernel.org a bit later.
> >
> >
> > All comments, bugreports, etc. are welcome as usual.
> 
> Ok, I have another comment/bugreport *g*.
> 
> I'm testing this patch with gentoo, so the grub sources are not identicaly
>  the same. With this patches applied, grub is unable to detect JFS or XFS
>  filesystems. XFS is reported as unknown, JFS is reported as btrfs.
>  Reiserfs and ext2/3 are detected as expected.

A possible solution is to put FSYS_BTRFS on the end of struct fsys_entry 
fsys_table. I've tested with FSYS_BTFS as the second last entry, the last is 
still FFS.

diff -Nru grub-0.97-r9/stage2/disk_io.c grub-0.97-r10/stage2/disk_io.c
--- grub-0.97-r9/stage2/disk_io.c       2009-12-10 23:41:37.000000000 +0100
+++ grub-0.97-r10/stage2/disk_io.c      2009-12-11 00:50:51.555007247 +0100
@@ -79,6 +79,9 @@
 # ifdef FSYS_ISO9660
   {"iso9660", iso9660_mount, iso9660_read, iso9660_dir, 0, 0},
 # endif
+# ifdef FSYS_BTRFS
+  {"btrfs", btrfs_mount, btrfs_read, btrfs_dir, 0, btrfs_embed},
+# endif
   /* XX FFS should come last as it's superblock is commonly crossing tracks
      on floppies from track 1 to 2, while others only use 1.  */
 # ifdef FSYS_FFS

With this order, XFS and JFS filesystems are identified correct. But I think, 
this is just a workaround.


regards,
  Johannes

Re: [patch 0/2] grub-0.97: btrfs support

[Edward Shishkin]

Johannes Hirte wrote:
> Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
>   
>> Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
>>     
>>> Hello everyone.
>>>       
>> ...
>>
>>     
>>> The following patches are for Fedora 10(**).
>>> The distro-independent package will be put to kernel.org a bit later.
>>>
>>>
>>> All comments, bugreports, etc. are welcome as usual.
>>>       
>> Ok, I have another comment/bugreport *g*.
>>
>> I'm testing this patch with gentoo, so the grub sources are not identicaly
>>  the same. With this patches applied, grub is unable to detect JFS or XFS
>>  filesystems. XFS is reported as unknown, JFS is reported as btrfs.
>>  Reiserfs and ext2/3 are detected as expected.
>>     

Yes, this patch is for Fedora. For other distros
some issues are possible, so please be careful..

Thanks,
Edward.

>
> A possible solution is to put FSYS_BTRFS on the end of struct fsys_entry 
> fsys_table. I've tested with FSYS_BTFS as the second last entry, the last is 
> still FFS.
>
> diff -Nru grub-0.97-r9/stage2/disk_io.c grub-0.97-r10/stage2/disk_io.c
> --- grub-0.97-r9/stage2/disk_io.c       2009-12-10 23:41:37.000000000 +0100
> +++ grub-0.97-r10/stage2/disk_io.c      2009-12-11 00:50:51.555007247 +0100
> @@ -79,6 +79,9 @@
>  # ifdef FSYS_ISO9660
>    {"iso9660", iso9660_mount, iso9660_read, iso9660_dir, 0, 0},
>  # endif
> +# ifdef FSYS_BTRFS
> +  {"btrfs", btrfs_mount, btrfs_read, btrfs_dir, 0, btrfs_embed},
> +# endif
>    /* XX FFS should come last as it's superblock is commonly crossing tracks
>       on floppies from track 1 to 2, while others only use 1.  */
>  # ifdef FSYS_FFS
>
> With this order, XFS and JFS filesystems are identified correct. But I think, 
> this is just a workaround.
>
>
> regards,
>   Johannes
>
>   

Re: [patch 0/2] grub-0.97: btrfs support

[Johannes Hirte]

Am Freitag 11 Dezember 2009 12:17:29 schrieb Edward Shishkin:
> Johannes Hirte wrote:
> > Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
> >   
> >> Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
> >>     
> >>> Hello everyone.
> >>>       
> >> ...
> >>
> >>     
> >>> The following patches are for Fedora 10(**).
> >>> The distro-independent package will be put to kernel.org a bit later.
> >>>
> >>>
> >>> All comments, bugreports, etc. are welcome as usual.
> >>>       
> >> Ok, I have another comment/bugreport *g*.
> >>
> >> I'm testing this patch with gentoo, so the grub sources are not identicaly
> >>  the same. With this patches applied, grub is unable to detect JFS or XFS
> >>  filesystems. XFS is reported as unknown, JFS is reported as btrfs.
> >>  Reiserfs and ext2/3 are detected as expected.
> >>     
> 
> Yes, this patch is for Fedora. For other distros
> some issues are possible, so please be careful..

I've also tested now with the fedora sources. There is the same bug. The btrfs
patch breaks the filesystem detection. All filesystems after btrfs in fsys_table 
aren't detected. Moving btrfs to the end of fsys_table is a workaround but will
interfere with FFS. So this should better be fixed in the btrfs-part of grub,
so that it:

a) doesn't missdetect a JFS filesystem as btrfs
b) doesn't break the detection for remaining filesystems in the array.


regards,
  Johannes

Re: [patch 0/2] grub-0.97: btrfs support

[Edward Shishkin]

[-- Attachment #1: Type: text/plain, Size: 1763 bytes --]

Johannes Hirte wrote:
> Am Freitag 11 Dezember 2009 12:17:29 schrieb Edward Shishkin:
>   
>> Johannes Hirte wrote:
>>     
>>> Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
>>>   
>>>       
>>>> Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
>>>>     
>>>>         
>>>>> Hello everyone.
>>>>>       
>>>>>           
>>>> ...
>>>>
>>>>     
>>>>         
>>>>> The following patches are for Fedora 10(**).
>>>>> The distro-independent package will be put to kernel.org a bit later.
>>>>>
>>>>>
>>>>> All comments, bugreports, etc. are welcome as usual.
>>>>>       
>>>>>           
>>>> Ok, I have another comment/bugreport *g*.
>>>>
>>>> I'm testing this patch with gentoo, so the grub sources are not identicaly
>>>>  the same. With this patches applied, grub is unable to detect JFS or XFS
>>>>  filesystems. XFS is reported as unknown, JFS is reported as btrfs.
>>>>  Reiserfs and ext2/3 are detected as expected.
>>>>     
>>>>         
>> Yes, this patch is for Fedora. For other distros
>> some issues are possible, so please be careful..
>>     
>
> I've also tested now with the fedora sources. There is the same bug. The btrfs
> patch breaks the filesystem detection. All filesystems after btrfs in fsys_table 
> aren't detected. Moving btrfs to the end of fsys_table is a workaround but will
> interfere with FFS. So this should better be fixed in the btrfs-part of grub,
> so that it:
>
> a) doesn't missdetect a JFS filesystem as btrfs
> b) doesn't break the detection for remaining filesystems in the array.
>   

Hello.

Yes, I confirm that xfs, etc. file systems are not detected,
but missdetection jfs as btrfs looks rather fantastic :)

Please, try the attached patch. Report if any problems.

Thanks,
Edward.

[-- Attachment #2: grub-btrfs-mount-fix.patch --]
[-- Type: text/plain, Size: 880 bytes --]

Problem:
XFS, JFS, etc. file systems of the fsys_table are not
detected by grub with grub-0.97-btrfs.patch applied.

BUG:
btrfs_mount() sets ERR_FSYS_MOUNT to the global variable
errnum if no btrfs metadata were found on a partition.
As the result all next calls of devread() (and, hence,
attempts to find metadata of other file systems) failed.

Solution:
Don't set ERR_FSYS_MOUNT, if btrfs metadata were found,
just return 0.

Signed-off-by: Edward Shishkin <edward@redhat.com>
---
 stage2/fsys_btrfs.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- grub-0.97.orig/stage2/fsys_btrfs.c
+++ grub-0.97/stage2/fsys_btrfs.c
@@ -638,7 +638,7 @@ int btrfs_mount(void)
 	if (ret) {
 		btrfs_msg("Drive %lu, partition %lu: no Btrfs metadata\n",
 			  current_drive, part_start);
-		goto error;
+		return 0;
 	}
 	ret = btrfs_uptodate_super_copy(BTRFS_FS_INFO);
 	if (ret)

Re: [patch 0/2] grub-0.97: btrfs support

[Johannes Hirte]

Am Freitag 11 Dezember 2009 16:27:54 schrieb Edward Shishkin:
> Johannes Hirte wrote:
> > Am Freitag 11 Dezember 2009 12:17:29 schrieb Edward Shishkin:
> >> Johannes Hirte wrote:
> >>> Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
> >>>> Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
> >>>>> Hello everyone.
> >>>>
> >>>> ...
> >>>>
> >>>>> The following patches are for Fedora 10(**).
> >>>>> The distro-independent package will be put to kernel.org a bit later.
> >>>>>
> >>>>>
> >>>>> All comments, bugreports, etc. are welcome as usual.
> >>>>
> >>>> Ok, I have another comment/bugreport *g*.
> >>>>
> >>>> I'm testing this patch with gentoo, so the grub sources are not
> >>>> identicaly the same. With this patches applied, grub is unable to
> >>>> detect JFS or XFS filesystems. XFS is reported as unknown, JFS is
> >>>> reported as btrfs. Reiserfs and ext2/3 are detected as expected.
> >>
> >> Yes, this patch is for Fedora. For other distros
> >> some issues are possible, so please be careful..
> >
> > I've also tested now with the fedora sources. There is the same bug. The
> > btrfs patch breaks the filesystem detection. All filesystems after btrfs
> > in fsys_table aren't detected. Moving btrfs to the end of fsys_table is a
> > workaround but will interfere with FFS. So this should better be fixed in
> > the btrfs-part of grub, so that it:
> >
> > a) doesn't missdetect a JFS filesystem as btrfs
> > b) doesn't break the detection for remaining filesystems in the array.
> 
> Hello.
> 
> Yes, I confirm that xfs, etc. file systems are not detected,
> but missdetection jfs as btrfs looks rather fantastic :)
> 
> Please, try the attached patch. Report if any problems.

The patch works, but the problem with misdetected JFS filesystem still 
persists. It happens if the device contained a btrfs filesystem before. I 
assume that the JFS super block starts later on the device as the btrfs one do 
and jfs_mkfs doesn't clean the space ahead of the JFS super block. So if a JFS 
filesystem is created on a device that contained a btrfs before, btrfs_mount 
still detects the beginning of the old btrfs super block and reads crap later 
on.
To avoid this, btrfs detection could be placed after JFS. Are there any 
objections against this?


regards,
  Johannes

Re: [patch 0/2] grub-0.97: btrfs support

[Edward Shishkin]

Johannes Hirte wrote:
> Am Freitag 11 Dezember 2009 16:27:54 schrieb Edward Shishkin:
>   
>> Johannes Hirte wrote:
>>     
>>> Am Freitag 11 Dezember 2009 12:17:29 schrieb Edward Shishkin:
>>>       
>>>> Johannes Hirte wrote:
>>>>         
>>>>> Am Freitag 11 Dezember 2009 00:15:46 schrieb Johannes Hirte:
>>>>>           
>>>>>> Am Freitag 25 September 2009 00:06:23 schrieb Edward Shishkin:
>>>>>>             
>>>>>>> Hello everyone.
>>>>>>>               
>>>>>> ...
>>>>>>
>>>>>>             
>>>>>>> The following patches are for Fedora 10(**).
>>>>>>> The distro-independent package will be put to kernel.org a bit later.
>>>>>>>
>>>>>>>
>>>>>>> All comments, bugreports, etc. are welcome as usual.
>>>>>>>               
>>>>>> Ok, I have another comment/bugreport *g*.
>>>>>>
>>>>>> I'm testing this patch with gentoo, so the grub sources are not
>>>>>> identicaly the same. With this patches applied, grub is unable to
>>>>>> detect JFS or XFS filesystems. XFS is reported as unknown, JFS is
>>>>>> reported as btrfs. Reiserfs and ext2/3 are detected as expected.
>>>>>>             
>>>> Yes, this patch is for Fedora. For other distros
>>>> some issues are possible, so please be careful..
>>>>         
>>> I've also tested now with the fedora sources. There is the same bug. The
>>> btrfs patch breaks the filesystem detection. All filesystems after btrfs
>>> in fsys_table aren't detected. Moving btrfs to the end of fsys_table is a
>>> workaround but will interfere with FFS. So this should better be fixed in
>>> the btrfs-part of grub, so that it:
>>>
>>> a) doesn't missdetect a JFS filesystem as btrfs
>>> b) doesn't break the detection for remaining filesystems in the array.
>>>       
>> Hello.
>>
>> Yes, I confirm that xfs, etc. file systems are not detected,
>> but missdetection jfs as btrfs looks rather fantastic :)
>>
>> Please, try the attached patch. Report if any problems.
>>     
>
> The patch works, but the problem with misdetected JFS filesystem still 
> persists. It happens if the device contained a btrfs filesystem before. I 
> assume that the JFS super block starts later on the device as the btrfs one do 
> and jfs_mkfs doesn't clean the space ahead of the JFS super block. So if a JFS 
> filesystem is created on a device that contained a btrfs before, btrfs_mount 
> still detects the beginning of the old btrfs super block and reads crap later 
> on.
>   

Yup, sticky thing..

> To avoid this, btrfs detection could be placed after JFS. Are there any 
> objections against this?
>   

If so, we might want to put if after XFS,
which is also mistreated in such manner..

Thanks,
Edward.