metadata and / and grub-static

metadata and / and grub-static

[Mark Knecht]

Hi,
   I'm trying to do my first RAID install that puts / on RAID1 but I'm
having some trouble. Distro is Gentoo. I'm fairly certain the problem
is in my kernel because grub finds my RAID enabled kernel and starts
booting, but when it gets to the point where it is going to enable md0
and mount / (or I think that's where it is) then it fails with a very
standard message:

VFS - Unable to mount root fs on unknown-block(9,0)

   I know the machine works because I already have a non-RAID install
on sda. The new RAID install is on sdb & sdc and I'm booting to it
from sda. However so far I cannot get past the point where it figures
out what to do with /dev/md0 and moves on.

   I just wanted to check that the messages below, which occurred when
I first built the RAID, aren't a real killer. I am not storing /boot
on this device but I do want to store /. Is there any problem with
recent versions of grub doing this? I chose metadata=1.0 as mdadm
seemed happier with that choice, but I've been unable to mount this
RAID either.

   Again, if this is a kernel issue then I'll work it out. I've built
RAID0 and RAID1 support into the kernel, as well as other things, but
maybe I've missed something else. I just wanted to be doubly sure that
it's not a problem with grub as a boot loader.

Thanks,
Mark

keeper ~ # mdadm --create /dev/md0 --level=1 --raid-devices=2
/dev/sdb3 /dev/sdc3
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/' or '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=1.0
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/' or '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=1.0
Continue creating array? n
mdadm: create aborted.
keeper ~ #  mdadm --create /dev/md0 --level=1 --raid-devices=2
--metadata=1.0 /dev/sdb3 /dev/sdc3
mdadm: array /dev/md0 started.
keeper ~ # cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc3[1] sdb3[0]
      31463228 blocks super 1.0 [2/2] [UU]
      [>....................]  resync =  3.9% (1241664/31463228)
finish=5.2min speed=95512K/sec

unused devices: <none>
keeper ~ #

Re: metadata and / and grub-static

[Mark Knecht]

On Sun, Apr 4, 2010 at 12:34 PM, Mark Knecht <markknecht@gmail.com> wrote:
> Hi,
>   I'm trying to do my first RAID install that puts / on RAID1 but I'm
> having some trouble. Distro is Gentoo. I'm fairly certain the problem
> is in my kernel because grub finds my RAID enabled kernel and starts
> booting, but when it gets to the point where it is going to enable md0
> and mount / (or I think that's where it is) then it fails with a very
> standard message:
>
> VFS - Unable to mount root fs on unknown-block(9,0)
>
>   I know the machine works because I already have a non-RAID install
> on sda. The new RAID install is on sdb & sdc and I'm booting to it
> from sda. However so far I cannot get past the point where it figures
> out what to do with /dev/md0 and moves on.
>
>   I just wanted to check that the messages below, which occurred when
> I first built the RAID, aren't a real killer. I am not storing /boot
> on this device but I do want to store /. Is there any problem with
> recent versions of grub doing this? I chose metadata=1.0 as mdadm
> seemed happier with that choice, but I've been unable to mount this
> RAID either.
>
>   Again, if this is a kernel issue then I'll work it out. I've built
> RAID0 and RAID1 support into the kernel, as well as other things, but
> maybe I've missed something else. I just wanted to be doubly sure that
> it's not a problem with grub as a boot loader.
>
> Thanks,
> Mark
>
> keeper ~ # mdadm --create /dev/md0 --level=1 --raid-devices=2
> /dev/sdb3 /dev/sdc3
> mdadm: Note: this array has metadata at the start and
>    may not be suitable as a boot device.  If you plan to
>    store '/' or '/boot' on this device please ensure that
>    your boot-loader understands md/v1.x metadata, or use
>    --metadata=1.0
> mdadm: Note: this array has metadata at the start and
>    may not be suitable as a boot device.  If you plan to
>    store '/' or '/boot' on this device please ensure that
>    your boot-loader understands md/v1.x metadata, or use
>    --metadata=1.0
> Continue creating array? n
> mdadm: create aborted.
> keeper ~ #  mdadm --create /dev/md0 --level=1 --raid-devices=2
> --metadata=1.0 /dev/sdb3 /dev/sdc3
> mdadm: array /dev/md0 started.
> keeper ~ # cat /proc/mdstat
> Personalities : [raid1]
> md0 : active raid1 sdc3[1] sdb3[0]
>      31463228 blocks super 1.0 [2/2] [UU]
>      [>....................]  resync =  3.9% (1241664/31463228)
> finish=5.2min speed=95512K/sec
>
> unused devices: <none>
> keeper ~ #
>

Answering self - putting / on RAID1 only seems to work using
--metadata=0.90. Tried it with higher versions and had no luck.

Anyway, the system is running well now.

Cheers,
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: metadata and / and grub-static

[Leslie Rhorer]

> > keeper ~ # mdadm --create /dev/md0 --level=1 --raid-devices=2
> > /dev/sdb3 /dev/sdc3
> > mdadm: Note: this array has metadata at the start and
> >    may not be suitable as a boot device.  If you plan to
> >    store '/' or '/boot' on this device please ensure that
> >    your boot-loader understands md/v1.x metadata, or use
> >    --metadata=1.0
> > mdadm: Note: this array has metadata at the start and
> >    may not be suitable as a boot device.  If you plan to
> >    store '/' or '/boot' on this device please ensure that
> >    your boot-loader understands md/v1.x metadata, or use
> >    --metadata=1.0

	As the utility warns you, anything other than 0.9 or 1.0 may have
problems loading as / or /boot, because 1.1 and 1.2 both put the metadata at
the start of the device, in your case a partition.

> > Continue creating array? n
> > mdadm: create aborted.
> > keeper ~ #  mdadm --create /dev/md0 --level=1 --raid-devices=2
> > --metadata=1.0 /dev/sdb3 /dev/sdc3
> > mdadm: array /dev/md0 started.
> > keeper ~ # cat /proc/mdstat
> > Personalities : [raid1]
> > md0 : active raid1 sdc3[1] sdb3[0]
> >      31463228 blocks super 1.0 [2/2] [UU]
> >      [>....................]  resync =  3.9% (1241664/31463228)
> > finish=5.2min speed=95512K/sec
> >
> > unused devices: <none>
> > keeper ~ #
> >
> 
> Answering self - putting / on RAID1 only seems to work using
> --metadata=0.90. Tried it with higher versions and had no luck.

	One may most certainly run a RAID1 / or /boot (or both) using other
than 0.9 metadata.  I'm running two servers with / on a 1.02 superblock and
/boot on a 1.0 superblock, booting with grub 0.97.

	That said, if the restrictions of a 0.9 superblock are not an issue
for your system, then there is nothing particularly wrong with employing a
0.9 superblock.

> Anyway, the system is running well now.

	Unless the restrictions of a 0.9 superblock will cause problems for
you in the future, that's the important thing.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: metadata and / and grub-static

[Mark Knecht]

On Sun, Apr 4, 2010 at 7:26 PM, Leslie Rhorer <lrhorer@satx.rr.com> wrote:
<SNIP>
>>
>> Answering self - putting / on RAID1 only seems to work using
>> --metadata=0.90. Tried it with higher versions and had no luck.
>
>        One may most certainly run a RAID1 / or /boot (or both) using other
> than 0.9 metadata.  I'm running two servers with / on a 1.02 superblock and
> /boot on a 1.0 superblock, booting with grub 0.97.

Are you using an initrd? I could not make it work above 0.90 without
going that way which I didn't want to do.

>
>        That said, if the restrictions of a 0.9 superblock are not an issue
> for your system, then there is nothing particularly wrong with employing a
> 0.9 superblock.
>
>> Anyway, the system is running well now.
>
>        Unless the restrictions of a 0.9 superblock will cause problems for
> you in the future, that's the important thing.
>

I don't see that they will. I'm just building a home desktop/server
machine using 3-drive RAID1. I suspect you are doing more
sophisticated things than I.

Cheers,
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: metadata and / and grub-static

[John Robinson]

On 05/04/2010 15:15, Mark Knecht wrote:
> On Sun, Apr 4, 2010 at 7:26 PM, Leslie Rhorer <lrhorer@satx.rr.com> wrote:
> <SNIP>
>>> Answering self - putting / on RAID1 only seems to work using
>>> --metadata=0.90. Tried it with higher versions and had no luck.
>>        One may most certainly run a RAID1 / or /boot (or both) using other
>> than 0.9 metadata.  I'm running two servers with / on a 1.02 superblock and
>> /boot on a 1.0 superblock, booting with grub 0.97.
> 
> Are you using an initrd? I could not make it work above 0.90 without
> going that way which I didn't want to do.

Yes, the kernel will only auto-assemble arrays with 0.90 metadata, and 
Neil Brown has stated several times that there never will be support for 
auto-assembling anything else in the kernel.

You can still have your /boot with 1.0 metadata using grub 0.97 (or even 
LILO) and your root with any metadata but you will need an initrd 
containing mdadm with which to assemble your arrays before attempting to 
mount the root filesystem.

But if you're determined to avoid an initrd, you're stuck with 0.90 
metadata for your root filesystem. This probably isn't much of a 
limitation since you can still have a 27-drive RAID-6 array of 2TB discs 
with write-intent bitmap if you want. (It might even be 28 discs.) And 
if you had an array as funky as that, you'd probably want to run LVM 
over the top of it, so you'd need an initrd anyway, so you'd be able to 
use 1.x metadata :-)

Since most distros ship highly modular kernels, their installers and 
tools all build initrds, so for most people, needing an initrd is not an 
issue.

Anyway, a quick summary:
* with grub 0.97 or LILO, /boot (more generally, wherever your vmlinuz 
lives) must be RAID-1 on 0.90 or 1.0 metadata, and the members must be 
on drives the BIOS will boot from, usually sda/b/c
* without an initrd, / must be on 0.90 metadata but any RAID level or 
disc location (that the kernel doesn't need extra modules for) is fine
* and combining these, if you're running grub 0.97 (or LILO) and don't 
want a separate /boot array (or keep your vmlinuz in the root) and don't 
want to use an initrd, you're restricted to having your root filesystem 
on RAID-1 on 0.90 metadata on drives the BIOS will boot from

Cheers,

John.

Re: metadata and / and grub-static

[Mark Knecht]

On Mon, Apr 5, 2010 at 7:55 AM, John Robinson
<john.robinson@anonymous.org.uk> wrote:
> On 05/04/2010 15:15, Mark Knecht wrote:
>>
>> On Sun, Apr 4, 2010 at 7:26 PM, Leslie Rhorer <lrhorer@satx.rr.com> wrote:
>> <SNIP>
>>>>
>>>> Answering self - putting / on RAID1 only seems to work using
>>>> --metadata=0.90. Tried it with higher versions and had no luck.
>>>
>>>       One may most certainly run a RAID1 / or /boot (or both) using other
>>> than 0.9 metadata.  I'm running two servers with / on a 1.02 superblock
>>> and
>>> /boot on a 1.0 superblock, booting with grub 0.97.
>>
>> Are you using an initrd? I could not make it work above 0.90 without
>> going that way which I didn't want to do.
>
> Yes, the kernel will only auto-assemble arrays with 0.90 metadata, and Neil
> Brown has stated several times that there never will be support for
> auto-assembling anything else in the kernel.
>
> You can still have your /boot with 1.0 metadata using grub 0.97 (or even
> LILO) and your root with any metadata but you will need an initrd containing
> mdadm with which to assemble your arrays before attempting to mount the root
> filesystem.
>
> But if you're determined to avoid an initrd, you're stuck with 0.90 metadata
> for your root filesystem. This probably isn't much of a limitation since you
> can still have a 27-drive RAID-6 array of 2TB discs with write-intent bitmap
> if you want. (It might even be 28 discs.) And if you had an array as funky
> as that, you'd probably want to run LVM over the top of it, so you'd need an
> initrd anyway, so you'd be able to use 1.x metadata :-)
>
> Since most distros ship highly modular kernels, their installers and tools
> all build initrds, so for most people, needing an initrd is not an issue.
>
> Anyway, a quick summary:
> * with grub 0.97 or LILO, /boot (more generally, wherever your vmlinuz
> lives) must be RAID-1 on 0.90 or 1.0 metadata, and the members must be on
> drives the BIOS will boot from, usually sda/b/c
> * without an initrd, / must be on 0.90 metadata but any RAID level or disc
> location (that the kernel doesn't need extra modules for) is fine
> * and combining these, if you're running grub 0.97 (or LILO) and don't want
> a separate /boot array (or keep your vmlinuz in the root) and don't want to
> use an initrd, you're restricted to having your root filesystem on RAID-1 on
> 0.90 metadata on drives the BIOS will boot from
>
> Cheers,
>
> John.
>
>
John,
   Thanks for the summary. It's succinct and covered the info I needed
this weekend.

   One question comes to mind. I'm about to build a Gentoo machine
that will run a number of copies of Windows in VirtualBox VMs. The
system will be nice, reliable RAID1 for all of Linux, and then 'fast'
RAID0 on different drives for the VM data. As there will be nothing
involved with booting Linux on this RAID - just the VM data. I assume
that there's no problem assembling RAID0 after the boot process has
essentially completed, and that this RAID could use any form of
metadata?

Thanks,
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: metadata and / and grub-static

[John Robinson]

On 05/04/2010 16:52, Mark Knecht wrote:
[...]
> John,
>    Thanks for the summary. It's succinct and covered the info I needed
> this weekend.

In which case, apologies for not answering until Monday afternoon :-)

>    One question comes to mind. I'm about to build a Gentoo machine
> that will run a number of copies of Windows in VirtualBox VMs. The
> system will be nice, reliable RAID1 for all of Linux, and then 'fast'
> RAID0 on different drives for the VM data. As there will be nothing
> involved with booting Linux on this RAID - just the VM data. I assume
> that there's no problem assembling RAID0 after the boot process has
> essentially completed, and that this RAID could use any form of
> metadata?

Yep. You could do this with /home /usr /var too. I expect Gentoo's init 
scripts will run mdadm for you before mounting the other filesystems; 
the other distros generally do.

Cheers,

John.

Re: metadata and / and grub-static

[Mark Knecht]

On Mon, Apr 5, 2010 at 9:31 AM, John Robinson
<john.robinson@anonymous.org.uk> wrote:
> On 05/04/2010 16:52, Mark Knecht wrote:
> [...]
>>
>> John,
>>   Thanks for the summary. It's succinct and covered the info I needed
>> this weekend.
>
> In which case, apologies for not answering until Monday afternoon :-)
>

A little struggle on my side is good for the brain. forces me to read
and learn a I try to do. Yeah, frustrating once in awhile, but a good
thing none the less.

>>   One question comes to mind. I'm about to build a Gentoo machine
>> that will run a number of copies of Windows in VirtualBox VMs. The
>> system will be nice, reliable RAID1 for all of Linux, and then 'fast'
>> RAID0 on different drives for the VM data. As there will be nothing
>> involved with booting Linux on this RAID - just the VM data. I assume
>> that there's no problem assembling RAID0 after the boot process has
>> essentially completed, and that this RAID could use any form of
>> metadata?
>
> Yep. You could do this with /home /usr /var too. I expect Gentoo's init
> scripts will run mdadm for you before mounting the other filesystems; the
> other distros generally do.

Yes, I can turn them on if I choose with rc-update add. I haven't done
that yet as I need to do some reading on that also.

Currently I'm focused on understanding benchmarking a bit better -
iozone, bonnie++ - to see what I currently have with my new RAID1.

Cheers,
Mark

>
> Cheers,
>
> John.
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html