grub nilfs2 module

grub nilfs2 module

[Jiro SEKIBA]

Hi,

I've written nilfs2 module for grub2 boot loader.

Here is the patch and mail I'v sent to grub-devel ML.
http://lists.gnu.org/archive/html/grub-devel/2010-02/msg00025.html

It may not be mature yet, but I was able to boot kernel
from nilfs2 partition and mount it as rootfs.
So you are able to boot the system that has only nilfs2 partition.

The patch is against bazaar repository top, but actual code is
just a fs/nilfs2.c.  Other patches are against makefiles.
So I think you can able to compile it in stable release with
slight modification.

It would be very nice of you if you try and give me feed backs
to improve the module. 

thank you very much in advance!
-- 
Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Ryusuke Konishi]

Hi,
On Wed, 03 Feb 2010 21:43:46 +0900, Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org> wrote:
> Hi,
> 
> I've written nilfs2 module for grub2 boot loader.
> 
> Here is the patch and mail I'v sent to grub-devel ML.
> http://lists.gnu.org/archive/html/grub-devel/2010-02/msg00025.html

Great!

The patch looks fine to me except some work, for instance,
rollforwarding and checksum based log validation is needed.

But meanwhile, this prototype seems enough to load kernels and
initrds.

I'd like to try it for my laptop.

> It may not be mature yet, but I was able to boot kernel
> from nilfs2 partition and mount it as rootfs.
> So you are able to boot the system that has only nilfs2 partition.
> 
> The patch is against bazaar repository top, but actual code is
> just a fs/nilfs2.c.  Other patches are against makefiles.
> So I think you can able to compile it in stable release with
> slight modification.
> 
> It would be very nice of you if you try and give me feed backs
> to improve the module. 
> 
> thank you very much in advance!
> -- 
> Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

It would be nice if you could tell us belief usage of the patch.

Thank you,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Ryusuke Konishi]

Hi,
On Thu, 04 Feb 2010 10:02:28 +0900 (JST), Ryusuke Konishi <ryusuke-sG5X7nlA6pw@public.gmane.org> wrote:
> Hi,
> On Wed, 03 Feb 2010 21:43:46 +0900, Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org> wrote:
> > Hi,
> > 
> > I've written nilfs2 module for grub2 boot loader.
> > 
> > Here is the patch and mail I'v sent to grub-devel ML.
> > http://lists.gnu.org/archive/html/grub-devel/2010-02/msg00025.html
> 
> Great!
> 
> The patch looks fine to me except some work, for instance,
> rollforwarding and checksum based log validation is needed.
> 
> But meanwhile, this prototype seems enough to load kernels and
> initrds.
> 
> I'd like to try it for my laptop.
> 
> > It may not be mature yet, but I was able to boot kernel
> > from nilfs2 partition and mount it as rootfs.
> > So you are able to boot the system that has only nilfs2 partition.

Let me ask you a question.

One thing which is important in NILFS is that every block in the
partition can be moved by the garbage collection (except two super
blocks).

So, we cannot assume that file blocks stay in a locked position even
for unmodified files.

Doesn't this cause problems for the boot sequence of GRUB2 ?


Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Jiro SEKIBA]

Hi, 

At Thu, 04 Feb 2010 16:54:56 +0900 (JST),
Ryusuke Konishi wrote:
> 
> Hi,
> On Thu, 04 Feb 2010 10:02:28 +0900 (JST), Ryusuke Konishi <ryusuke-sG5X7nlA6pw@public.gmane.org> wrote:
> > Hi,
> > On Wed, 03 Feb 2010 21:43:46 +0900, Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org> wrote:
> > > Hi,
> > > 
> > > I've written nilfs2 module for grub2 boot loader.
> > > 
> > > Here is the patch and mail I'v sent to grub-devel ML.
> > > http://lists.gnu.org/archive/html/grub-devel/2010-02/msg00025.html
> > 
> > Great!
> > 
> > The patch looks fine to me except some work, for instance,
> > rollforwarding and checksum based log validation is needed.
> > 
> > But meanwhile, this prototype seems enough to load kernels and
> > initrds.
> > 
> > I'd like to try it for my laptop.
> > 
> > > It may not be mature yet, but I was able to boot kernel
> > > from nilfs2 partition and mount it as rootfs.
> > > So you are able to boot the system that has only nilfs2 partition.
> 
> Let me ask you a question.
> 
> One thing which is important in NILFS is that every block in the
> partition can be moved by the garbage collection (except two super
> blocks).
> 
> So, we cannot assume that file blocks stay in a locked position even
> for unmodified files.
> 
> Doesn't this cause problems for the boot sequence of GRUB2 ?

That's a good question.

It's basically OK because there usually are room for additional code
besides MBR in PC disk format.  The room size is 32KiB followed by MBR.
The room is not required for linux, but historically for DOS compat.

See the link how modern disk layout is.
http://www.pixelbeat.org/docs/disk/

You can check it with the dd by following
# dd if=/dev/hda of=hda bs=512 count=3 skip=65
# hexdump -C hda > hda.hex
# dd if=/dev/hda1 of=hda1 bs=512 count=3
# hexdump -C hda1 > hda1.hex

NOTE, the differences between two dumps are, I think, mtime in a super block.

So if you install grub2 with root directory in nilfs2, 
grub-install will install nilfs2.mod into that additional 32KiB space
The space is out of nilfs2 patition, so GC won't move the blocks.

thanks

regards

> 
> Regards,
> Ryusuke Konishi
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 


-- 
Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Jiro SEKIBA]

Hi,

> It would be nice if you could tell us belief usage of the patch.

OK, sure.  It's against bazaar repository, so you have to build
it from repository source instead of latest stable release tarball.

To build with stable release(1.97.2), you need to modify fs/nilfs2.c
not to use grub_xasprintf, but use grub_malloc and grub_sprintf.

You may get reject for makefile, but I think it'll be OK.


Here are the instruction to retrieve grub repository and build the softoware.

$ bzr branch http://bzr.savannah.gnu.org/r/grub/trunk/grub
$ cd grub
$ patch -p0 < /path/to/the/patch
$ sh autogen.sh
$ ./configure
$ make

you need "make install" as usual.  It will install under /usr/local
without giving any parameters for ./configure.

The "make install" won't install grub to MBR.
To install grub into MBR, you have to "grub-install" to target disk.

For example, assume that you have nilfs2 filesystem on hdb1,
mounted on /mnt and want to enable hdb as bootable.  You'll do

# grub-install --root-directory=/mnt /dev/hdb

This will install grub into hdb's MBR and other required files into
following 32KiB blocks.  Fuhtermore, it'll install files under
/mnt/boot/grub.  Those files are not required to boot grub to read
the filesystem.

hdb is now self bootable disk.  You'll just connect it as hdb and
grub will boot up.

Note, during grub-install, /boog/grub/grub.cfg will be copied int to
/mnt/boo/grub, which is not the one for hdb.  So you have edit menu
at first time you boot with hdb.  After boot the system, you can reproduce
the menu by grub-mkcofnig > /boot/grub/grub.cfg

Enjoy!

regards,
-- 
Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Ryusuke Konishi]

On Thu, 04 Feb 2010 17:18:04 +0900, Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org> wrote:
> Hi, 
> 
> At Thu, 04 Feb 2010 16:54:56 +0900 (JST),
> Ryusuke Konishi wrote:
> > 
> > Hi,
> > On Thu, 04 Feb 2010 10:02:28 +0900 (JST), Ryusuke Konishi <ryusuke-sG5X7nlA6pw@public.gmane.org> wrote:
> > > Hi,
> > > On Wed, 03 Feb 2010 21:43:46 +0900, Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org> wrote:
> > > > Hi,
> > > > 
> > > > I've written nilfs2 module for grub2 boot loader.
> > > > 
> > > > Here is the patch and mail I'v sent to grub-devel ML.
> > > > http://lists.gnu.org/archive/html/grub-devel/2010-02/msg00025.html
> > > 
> > > Great!
> > > 
> > > The patch looks fine to me except some work, for instance,
> > > rollforwarding and checksum based log validation is needed.
> > > 
> > > But meanwhile, this prototype seems enough to load kernels and
> > > initrds.
> > > 
> > > I'd like to try it for my laptop.
> > > 
> > > > It may not be mature yet, but I was able to boot kernel
> > > > from nilfs2 partition and mount it as rootfs.
> > > > So you are able to boot the system that has only nilfs2 partition.
> > 
> > Let me ask you a question.
> > 
> > One thing which is important in NILFS is that every block in the
> > partition can be moved by the garbage collection (except two super
> > blocks).
> > 
> > So, we cannot assume that file blocks stay in a locked position even
> > for unmodified files.
> > 
> > Doesn't this cause problems for the boot sequence of GRUB2 ?
> 
> That's a good question.
> 
> It's basically OK because there usually are room for additional code
> besides MBR in PC disk format.  The room size is 32KiB followed by MBR.
> The room is not required for linux, but historically for DOS compat.
> 
> See the link how modern disk layout is.
> http://www.pixelbeat.org/docs/disk/
> 
> You can check it with the dd by following
> # dd if=/dev/hda of=hda bs=512 count=3 skip=65
> # hexdump -C hda > hda.hex
> # dd if=/dev/hda1 of=hda1 bs=512 count=3
> # hexdump -C hda1 > hda1.hex
> 
> NOTE, the differences between two dumps are, I think, mtime in a super block.
> 
> So if you install grub2 with root directory in nilfs2, 
> grub-install will install nilfs2.mod into that additional 32KiB space
> The space is out of nilfs2 patition, so GC won't move the blocks.
> 
> thanks

Thanks! That cleared up my doubts on GC.
The reference you cited is informative.

Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Ryusuke Konishi]

Hi,
On Thu, 04 Feb 2010 18:03:35 +0900, Jiro SEKIBA <jir-hfpbi5WX9J54Eiagz67IpQ@public.gmane.org> wrote:
> Hi,
> 
> > It would be nice if you could tell us belief usage of the patch.

s/belief/brief/

;) 

> OK, sure.  It's against bazaar repository, so you have to build
> it from repository source instead of latest stable release tarball.
> 
> To build with stable release(1.97.2), you need to modify fs/nilfs2.c
> not to use grub_xasprintf, but use grub_malloc and grub_sprintf.
> 
> You may get reject for makefile, but I think it'll be OK.
> 
> 
> Here are the instruction to retrieve grub repository and build the softoware.
> 
> $ bzr branch http://bzr.savannah.gnu.org/r/grub/trunk/grub
> $ cd grub
> $ patch -p0 < /path/to/the/patch
> $ sh autogen.sh
> $ ./configure
> $ make
> 
> you need "make install" as usual.  It will install under /usr/local
> without giving any parameters for ./configure.
> 
> The "make install" won't install grub to MBR.
> To install grub into MBR, you have to "grub-install" to target disk.
> 
> For example, assume that you have nilfs2 filesystem on hdb1,
> mounted on /mnt and want to enable hdb as bootable.  You'll do
> 
> # grub-install --root-directory=/mnt /dev/hdb
> 
> This will install grub into hdb's MBR and other required files into
> following 32KiB blocks.  Fuhtermore, it'll install files under
> /mnt/boot/grub.  Those files are not required to boot grub to read
> the filesystem.
> 
> hdb is now self bootable disk.  You'll just connect it as hdb and
> grub will boot up.
> 
> Note, during grub-install, /boog/grub/grub.cfg will be copied int to
> /mnt/boo/grub, which is not the one for hdb.  So you have edit menu
> at first time you boot with hdb.  After boot the system, you can reproduce
> the menu by grub-mkcofnig > /boot/grub/grub.cfg
> 
> Enjoy!

Thank you once again.

I'll catch up the above steps and then comment later.

Ryusuke
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: grub nilfs2 module

[Ryusuke Konishi]

Hi,
On Thu, 04 Feb 2010 18:21:39 +0900 (JST), Ryusuke Konishi wrote:
> > OK, sure.  It's against bazaar repository, so you have to build
> > it from repository source instead of latest stable release tarball.
> > 
> > To build with stable release(1.97.2), you need to modify fs/nilfs2.c
> > not to use grub_xasprintf, but use grub_malloc and grub_sprintf.
> > 
> > You may get reject for makefile, but I think it'll be OK.
> > 
> > 
> > Here are the instruction to retrieve grub repository and build the softoware.
> > 
> > $ bzr branch http://bzr.savannah.gnu.org/r/grub/trunk/grub
> > $ cd grub
> > $ patch -p0 < /path/to/the/patch
> > $ sh autogen.sh
> > $ ./configure
> > $ make
> > 
> > you need "make install" as usual.  It will install under /usr/local
> > without giving any parameters for ./configure.
> > 
> > The "make install" won't install grub to MBR.
> > To install grub into MBR, you have to "grub-install" to target disk.
> > 
> > For example, assume that you have nilfs2 filesystem on hdb1,
> > mounted on /mnt and want to enable hdb as bootable.  You'll do
> > 
> > # grub-install --root-directory=/mnt /dev/hdb
> > 
> > This will install grub into hdb's MBR and other required files into
> > following 32KiB blocks.  Fuhtermore, it'll install files under
> > /mnt/boot/grub.  Those files are not required to boot grub to read
> > the filesystem.
> > 
> > hdb is now self bootable disk.  You'll just connect it as hdb and
> > grub will boot up.
> > 
> > Note, during grub-install, /boog/grub/grub.cfg will be copied int to
> > /mnt/boo/grub, which is not the one for hdb.  So you have edit menu
> > at first time you boot with hdb.  After boot the system, you can reproduce
> > the menu by grub-mkcofnig > /boot/grub/grub.cfg
> > 
> > Enjoy!
> 
> Thank you once again.
> 
> I'll catch up the above steps and then comment later.
> 
> Ryusuke

I have finally succeeded to boot nilfs root without a separate /boot
partition ;)

On my Debian (squeeze) laptop, update-grub was also functional.

I did some trial and error when compiling grub2.  The following
packages were required in my case:

 - bison
 - gettext
 - libfreetype6-dev

And, the module didn't work for the stable version of grub(1.97.2) as
you mentioned above.

Until now, it looks working all right.


If you need any help to go further, please let me know.

Thanks!

Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html