Converting system to raid

Converting system to raid

[Timothy D. Lenz]

I have 2 computers that I am trying to convert to raid. Both where setup with debian netinst cd to hda. The first computer has 2
sata drives. They have been configured into 3 mirrored arrays. md0 is to replace hda and become the new boot. md1 will be the new
swap and md2 will be for storing recordings from vdr.

The second system which has been runing for awhile is basicly the same only it was setup with just 1 sata drive used for storage
which failed. It has been replaced with 3 sata drives setup with 2 mirrors + spare each and a raid5 array for storage. Currently the
first 2 arrays are not being used, the 3rd is mounted as /mnt/md2 and is used for the recordings folder and for some storage. Once I
figure out getting the 2 sata drive system booting with raid, I plan to do the same with the 3 sata drive system.

On the 2 sata drive system:
While booting with hda I have md0 mounted as /mnt/md0. I have marked the first partition bootable on both sda and sdb as bootable
and installed grub by doing the following:

sudo grub --no-floppy
Grub> device (hd0) /dev/sda
Grub> root (hd0,0)
Grub> setup (hd0)
Grub> device (hd0) /dev/sdb
Grub> root (hd0,0)
Grub> setup (hd0)
Grub> quit

I used "sudo cp -axu / /mnt/md0" to copy hda to md0. I changed /mnt/md0/etc/fstab so that "/" os /md0 instead of /hda. I also
changed /mnt/md0/boot/grub/menu.lst first kernel entry to md0:

title  Debian GNU/Linux, kernel 2.6.26.8.20090311.1
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.26.8.20090311.1 root=/dev/md0 ro quiet

Changed kopt in  /mnt/md0/boot/grub/menu.lst
# kopt=root=/dev/md0 ro

I have raid built into the kernel so initrd shouldn't be needed:

CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
# CONFIG_MD_LINEAR is not set
# CONFIG_MD_RAID0 is not set
CONFIG_MD_RAID1=y
# CONFIG_MD_RAID10 is not set
CONFIG_MD_RAID456=y
CONFIG_MD_RAID5_RESHAPE=y
CONFIG_MD_MULTIPATH=y
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
# CONFIG_DM_CRYPT is not set
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
CONFIG_DM_ZERO=y
# CONFIG_DM_MULTIPATH is not set
# CONFIG_DM_DELAY is not set
# CONFIG_DM_UEVENT is not set

I changed the grub boot menu color for md0 menu.lst to show which menu was being used.

When I boot up, I go into cmos amd move the ide to the bottom of the harddrive boot list so that it list sata 1, then sata 2, then
the ide.

When it boots, it should come up with the md0 array. I do get the md0 boot menu, but it hangs at booting kernel.  I some help in irc
freenode/#grub and was told grub seems ok, but there is some problem with the array when trying to boot from it. After booting the
array to the grub menu, he had me switch to grubs command line. Found that sda had become hd0, hda was now hd1 and sdb was still
hd2. Kernel started to boot and saw some md stuff go by:
....
md: considering sdb2...
[13:30] <Vorg> md: adding adb2
[13:30] <Vorg> md: sdb1 has different UUID to sdb2
[13:31] <Vorg> md: adding sda2..
.......

Ended with warning: unable to open an initial console. I was able to scroll back and didn't really see any errors. Don't know if I
missed a change to some config file or something else that should be built into the kernel. I have gone over a lot of "guides" and
don't see what is missing.

Re: Converting system to raid

[KwangErn Liew]

Timothy D. Lenz wrote:
> Ended with warning: unable to open an initial console. I was able to scroll back and didn't really see any errors. Don't know if I
> missed a change to some config file or something else that should be built into the kernel. I have gone over a lot of "guides" and
> don't see what is missing.

FYI, I've never been successful in booting RAID without initrd even 
though all necessary modules are built-in. I guess mdadm.conf requires 
initrd to kick start?


KwangErn

Re: Converting system to raid

[CoolCold]

Yes, he should provide correct /etc/mdadm/mdadm.conf and
update-initramfs -u on md boot, smth like
chroot /mnt/md0
update-initramfs -u

On Fri, Apr 10, 2009 at 1:10 PM, KwangErn Liew <admin@musmo.com> wrote:
>
> Timothy D. Lenz wrote:
>>
>> Ended with warning: unable to open an initial console. I was able to
>> scroll back and didn't really see any errors. Don't know if I
>> missed a change to some config file or something else that should be built
>> into the kernel. I have gone over a lot of "guides" and
>> don't see what is missing.
>
> FYI, I've never been successful in booting RAID without initrd even though
> all necessary modules are built-in. I guess mdadm.conf requires initrd to
> kick start?
>
>
> KwangErn
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
-- 
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

On Thu Apr 09, 2009 at 04:44:25PM -0700, Timothy D. Lenz wrote:

> I used "sudo cp -axu / /mnt/md0" to copy hda to md0. I changed
> /mnt/md0/etc/fstab so that "/" os /md0 instead of /hda.
> 
This won't work properly, for a start.  You _cannot_ copy a running
system - there's bound to be data which hasn't yet been flushed to disk,
as well as locked files.  You need to boot off a CD (or another
partition) and do the copy from there.

HTH,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Timothy D. Lenz]

How can cp not work? every guide I found used ether cp or rsync. Even guides on seting up an auto backup system use cp or rsync.
Seems the only files/folders that shouldn't get copied are the block device ones that are created at boot and are not really on the
drive.

----- Original Message ----- 
From: "Robin Hill" <robin@robinhill.me.uk>
To: <linux-raid@vger.kernel.org>
Sent: Friday, April 10, 2009 6:22 AM
Subject: Re: Converting system to raid

On Thu Apr 09, 2009 at 04:44:25PM -0700, Timothy D. Lenz wrote:

> I used "sudo cp -axu / /mnt/md0" to copy hda to md0. I changed
> /mnt/md0/etc/fstab so that "/" os /md0 instead of /hda.
>
This won't work properly, for a start.  You _cannot_ copy a running
system - there's bound to be data which hasn't yet been flushed to disk,
as well as locked files.  You need to boot off a CD (or another
partition) and do the copy from there.

HTH,
    Robin
-- 
     ___
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

Re: Converting system to raid

[Timothy D. Lenz]

Is that 
  /etc/mdadm/mdadm.conf
or
/mnt/md0/etc/mdadm/mdadm.conf

Both are the same in his case, but of chroot moves the root point to the array, shouldn't it look at that conf?

----- Original Message ----- 
From: "CoolCold" <coolthecold@gmail.com>
To: <admin@musmo.com>
Cc: "Timothy D. Lenz" <tlenz@vorgon.com>; <linux-raid@vger.kernel.org>
Sent: Friday, April 10, 2009 2:55 AM
Subject: Re: Converting system to raid


Yes, he should provide correct /etc/mdadm/mdadm.conf and
update-initramfs -u on md boot, smth like
chroot /mnt/md0
update-initramfs -u

On Fri, Apr 10, 2009 at 1:10 PM, KwangErn Liew <admin@musmo.com> wrote:
>
> Timothy D. Lenz wrote:
>>
>> Ended with warning: unable to open an initial console. I was able to
>> scroll back and didn't really see any errors. Don't know if I
>> missed a change to some config file or something else that should be built
>> into the kernel. I have gone over a lot of "guides" and
>> don't see what is missing.
>
> FYI, I've never been successful in booting RAID without initrd even though
> all necessary modules are built-in. I guess mdadm.conf requires initrd to
> kick start?
>
>
> KwangErn
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>



-- 
-- 
Best regards,
[COOLCOLD-RIPN]

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1067 bytes --]

On Fri Apr 10, 2009 at 12:22:44PM -0700, Timothy D. Lenz wrote:

> How can cp not work? every guide I found used ether cp or rsync. Even
> guides on seting up an auto backup system use cp or rsync.
> Seems the only files/folders that shouldn't get copied are the block
> device ones that are created at boot and are not really on the drive.
> 
They'll all get copied, but you won't necessarily get the correct
contents.  Because the system is in use:
 - files will be being written, so you'll either get a partial copy, or
   an old version.
 - files may be locked, so you won't be able to copy them.

Doing a backup of a running system using cp/rsync will work fine for a
lot of things (especially for config files, documents, etc) but will
almost certainly fail miserably for some (databases for example).

HTH,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]

On Fri Apr 10, 2009 at 11:10:30AM +0200, KwangErn Liew wrote:

>
> Timothy D. Lenz wrote:
>> Ended with warning: unable to open an initial console. I was able to 
>> scroll back and didn't really see any errors. Don't know if I
>> missed a change to some config file or something else that should be built 
>> into the kernel. I have gone over a lot of "guides" and
>> don't see what is missing.
>
> FYI, I've never been successful in booting RAID without initrd even though 
> all necessary modules are built-in. I guess mdadm.conf requires initrd to 
> kick start?
>
There's no problems booting RAID without initrd - you'll need to make
sure that you're using partitions (rather than whole disks) with type
0xFD, and that the array itself is using version 0.9 superblocks
(otherwise the kernel auto-assembly will fail).

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Timothy D. Lenz]

Great, for every bit of nudge forward I end up a step or two back. Not I have to figure out this copy stuff. Is there any easy way
to do some sort of compair after cp/rsync to see if/what was missed? Creating yet another botable disk to get the copy over to the
array would be a bit of a pain. Might be easyer to pull the hda from one computer, stick it in another or usb case and copy to a
temp storage area. on the other computer. Then put the drive back and copy from the temp storage to the array over then network.
Nether computer has any ftp or web servers or a desktop. The older computer with the 3 sata drives does have mysql installed because
I started to mess with setting up mythtv but never finished.


As for the other reply, the sata drives each have 3 primary partitions all set to type fd and formated with ext3. I was going to try
ext4 for the main storage of the newer system but I found there is a problem with kernels .27 and newer on nforce2 and newer boards.
some sort of problem with the bios and apci. How do I find out which superblocks version is used?

----- Original Message ----- 
From: "Robin Hill" <robin@robinhill.me.uk>
To: <linux-raid@vger.kernel.org>
Sent: Friday, April 10, 2009 12:53 PM
Subject: Re: Converting system to raid

On Fri Apr 10, 2009 at 12:22:44PM -0700, Timothy D. Lenz wrote:

> How can cp not work? every guide I found used ether cp or rsync. Even
> guides on seting up an auto backup system use cp or rsync.
> Seems the only files/folders that shouldn't get copied are the block
> device ones that are created at boot and are not really on the drive.
>
They'll all get copied, but you won't necessarily get the correct
contents.  Because the system is in use:
 - files will be being written, so you'll either get a partial copy, or
   an old version.
 - files may be locked, so you won't be able to copy them.

Doing a backup of a running system using cp/rsync will work fine for a
lot of things (especially for config files, documents, etc) but will
almost certainly fail miserably for some (databases for example).

HTH,
    Robin
-- 
     ___
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

On Fri Apr 10, 2009 at 11:10:30AM +0200, KwangErn Liew wrote:

>
> Timothy D. Lenz wrote:
>> Ended with warning: unable to open an initial console. I was able to
>> scroll back and didn't really see any errors. Don't know if I
>> missed a change to some config file or something else that should be built
>> into the kernel. I have gone over a lot of "guides" and
>> don't see what is missing.
>
> FYI, I've never been successful in booting RAID without initrd even though
> all necessary modules are built-in. I guess mdadm.conf requires initrd to
> kick start?
>
There's no problems booting RAID without initrd - you'll need to make
sure that you're using partitions (rather than whole disks) with type
0xFD, and that the array itself is using version 0.9 superblocks
(otherwise the kernel auto-assembly will fail).

Cheers,
    Robin
-- 
     ___
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]

On Fri Apr 10, 2009 at 01:45:00PM -0700, Timothy D. Lenz wrote:

> Great, for every bit of nudge forward I end up a step or two back. Not
> I have to figure out this copy stuff. Is there any easy way to do some
> sort of compair after cp/rsync to see if/what was missed? Creating yet
> another botable disk to get the copy over to the array would be a bit
> of a pain. Might be easyer to pull the hda from one computer, stick it
> in another or usb case and copy to a temp storage area. on the other
> computer. Then put the drive back and copy from the temp storage to
> the array over then network. Nether computer has any ftp or web
> servers or a desktop. The older computer with the 3 sata drives does
> have mysql installed because I started to mess with setting up mythtv
> but never finished.
> 
The easiest option is to use a bootable CD - there's plenty of linux CDs
around (have a look at http://linuxiso.org - my preferred one is GRML)).
Otherwise, yes, putting the drive into another system and doing the copy
there would work fine.

> As for the other reply, the sata drives each have 3 primary partitions
> all set to type fd and formated with ext3. I was going to try ext4 for
> the main storage of the newer system but I found there is a problem
> with kernels .27 and newer on nforce2 and newer boards. some sort of
> problem with the bios and apci. How do I find out which superblocks
> version is used?
> 
Either "mdadm -D" on the array, or "mdadm -E" on one of the partitions
will show you the version.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Timothy D. Lenz]

Strange:
-----------------
sudo mdadm -D /dev/md0
/dev/md0:
        Version : 00.90
  Creation Time : Fri Mar 27 16:49:37 2009
     Raid Level : raid1
     Array Size : 24418688 (23.29 GiB 25.00 GB)
  Used Dev Size : 24418688 (23.29 GiB 25.00 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Fri Apr 10 12:58:59 2009
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 27ce9bd9:4ce6ceb2:670e265f:82e0c670 (local to host LLLx64-32)
         Events : 0.38

    Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1
-----------------

Seems ok, but then:
-----------------
sudo mdadm -E /dev/md0
mdadm: No md superblock detected on /dev/md0.
-----------------
----- Original Message ----- 
From: "Robin Hill" <robin@robinhill.me.uk>
To: <linux-raid@vger.kernel.org>
Sent: Friday, April 10, 2009 1:59 PM
Subject: Re: Converting system to raid

> As for the other reply, the sata drives each have 3 primary partitions
> all set to type fd and formated with ext3. I was going to try ext4 for
> the main storage of the newer system but I found there is a problem
> with kernels .27 and newer on nforce2 and newer boards. some sort of
> problem with the bios and apci. How do I find out which superblocks
> version is used?
> 
Either "mdadm -D" on the array, or "mdadm -E" on one of the partitions
will show you the version.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 2047 bytes --]

On Fri Apr 10, 2009 at 02:26:26PM -0700, Timothy D. Lenz wrote:

> Strange:
> -----------------
> sudo mdadm -D /dev/md0
> /dev/md0:
>         Version : 00.90
>   Creation Time : Fri Mar 27 16:49:37 2009
>      Raid Level : raid1
>      Array Size : 24418688 (23.29 GiB 25.00 GB)
>   Used Dev Size : 24418688 (23.29 GiB 25.00 GB)
>    Raid Devices : 2
>   Total Devices : 2
> Preferred Minor : 0
>     Persistence : Superblock is persistent
> 
>     Update Time : Fri Apr 10 12:58:59 2009
>           State : clean
>  Active Devices : 2
> Working Devices : 2
>  Failed Devices : 0
>   Spare Devices : 0
> 
>            UUID : 27ce9bd9:4ce6ceb2:670e265f:82e0c670 (local to host LLLx64-32)
>          Events : 0.38
> 
>     Number   Major   Minor   RaidDevice State
>        0       8        1        0      active sync   /dev/sda1
>        1       8       17        1      active sync   /dev/sdb1
> -----------------
> 
> Seems ok, but then:
> -----------------
> sudo mdadm -E /dev/md0
> mdadm: No md superblock detected on /dev/md0.
>
Not really - as I said, "mdadm -E" on the partition, so:
    mdadm -E /dev/sda1
or
    mdadm -E /dev/sdb1

But these'll give the same results as you've already got - it is using a
version 0.9 superblock, so ought to get auto-assembled by the kernel.
If it doesn't, you'll get an error about being unable to find/mount the
block device.  From your original email, it looks like this stage is
working okay anyway.  The "Unable to open initial console" error you got
comes after the root FS has been mounted, and would suggest a problem
with the device nodes.  I'd guess that there's some static device nodes
which are needed before udev is mounted - because you did the copy with
a mounted udev, you've missed these.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Jon Lewis]

On Fri, 10 Apr 2009, Robin Hill wrote:

> On Fri Apr 10, 2009 at 12:22:44PM -0700, Timothy D. Lenz wrote:
>
>> How can cp not work? every guide I found used ether cp or rsync. Even
>> guides on seting up an auto backup system use cp or rsync.
>> Seems the only files/folders that shouldn't get copied are the block
>> device ones that are created at boot and are not really on the drive.
>>
> They'll all get copied, but you won't necessarily get the correct
> contents.  Because the system is in use:
> - files will be being written, so you'll either get a partial copy, or
>   an old version.
> - files may be locked, so you won't be able to copy them.
>
> Doing a backup of a running system using cp/rsync will work fine for a
> lot of things (especially for config files, documents, etc) but will
> almost certainly fail miserably for some (databases for example).

Can you cite an example of a file that can't be copied on a "running 
system?"

Sure, there are issues if the system is really up and running multi-user 
with log files that will be copied partially and probably end up 
terminated with extra nulls, database files may have issues (i.e. mysql if 
you don't flush and lock the tables, copied tables will be corrupt but are 
usually trivially repairable), but I've never heard of files not being at 
all readable.

The idea that "it just won't work because critical files are locked" 
sounds like Windows talk.

----------------------------------------------------------------------
  Jon Lewis                   |  I route
  Senior Network Engineer     |  therefore you are
  Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: Converting system to raid

[Timothy D. Lenz]

From: "CoolCold" <coolthecold@gmail.com>

> Yes, he should provide correct /etc/mdadm/mdadm.conf and
> update-initramfs -u on md boot, smth like
> chroot /mnt/md0
> update-initramfs -u

How would this help in a system that doesn't ramdisk built in to the kernel or as a module? Or does it change some other stuff?


I started looking at the stuff that was copied to md0 and /md0/dev is empty. Looking through the guides I found a few things. One
said that using cp had to be from root or not everything would get coppied. I used sudo but I know some things require you to root.
Also this:
===============================
# rsync -avHhx --progress / /mnt/raid-md0

    * If the system wasn't previously in single user mode, move to single user mode and update the data that changed during the
first copy:
(--delete flag tells rsync to delete files from the destination which do not exist on the source):

# rsync -avHhx --progress --delete / /mnt/raid-md0

    * Create needed device nodes:

# cd /mnt/raid-md0/dev/ && MAKEDEV generic
===============================
Using rsync from single user mode still left /mnt/md0/dev empty. I read up in "makedev generic" and it seems to be a shotgun fix
adding way more then is needed. Is there a way to create just what is in /dev?

Re: Converting system to raid

[Jan Ceuleers]

Jon Lewis wrote:
> Can you cite an example of a file that can't be copied on a "running 
> system?"

All files that are open for writing at the time of the copy risk being 
copied incompletely. Use lsof to find them (but that's a snapshot).

> Sure, there are issues if the system is really up and running multi-user 
> with log files that will be copied partially and probably end up 
> terminated with extra nulls, database files may have issues (i.e. mysql 
> if you don't flush and lock the tables, copied tables will be corrupt 
> but are usually trivially repairable), but I've never heard of files not 
> being at all readable.

I submit that these issues, even if you regard them as trivially 
repairable, are best avoided altogether by making the copy while the 
filesystem is not in use. And "trivially repairable" presupposes that 
you know that there is something to be repaired in the first place, and 
then that you know how to do it. Some of these issues may not manifest 
themselves until some time after you've begun using the flawed clone, at 
which time you may have forgotten all about this, may no longer have 
access to the original data etc.

To the OP: if you want to limit downtime resulting from the fact that 
the clone has to be made while the filesystem is not in use, you can use 
rsync to make your copy while the filesystem is still in use (but this 
copy will be incomplete/flawed etc), and then use rsync again after 
you've booted from a CD or whatever. The second rsync run will be much 
faster because it will only copy the files that have changed since the 
first run, including those that were open at the time of the first run.

Cheers, Jan

Re: Converting system to raid

[Goswin von Brederlow]

Jon Lewis <jlewis@lewis.org> writes:

> On Fri, 10 Apr 2009, Robin Hill wrote:
>
>> On Fri Apr 10, 2009 at 12:22:44PM -0700, Timothy D. Lenz wrote:
>>
>>> How can cp not work? every guide I found used ether cp or rsync. Even
>>> guides on seting up an auto backup system use cp or rsync.
>>> Seems the only files/folders that shouldn't get copied are the block
>>> device ones that are created at boot and are not really on the drive.
>>>
>> They'll all get copied, but you won't necessarily get the correct
>> contents.  Because the system is in use:
>> - files will be being written, so you'll either get a partial copy, or
>>   an old version.
>> - files may be locked, so you won't be able to copy them.
>>
>> Doing a backup of a running system using cp/rsync will work fine for a
>> lot of things (especially for config files, documents, etc) but will
>> almost certainly fail miserably for some (databases for example).

What one can easily do is rsync the running system, then init 1 and
rsync again to get all the changed or blocked files. In runlevel 1
there should be nothing left blocking files.

MfG
        Goswin

Re: Converting system to raid

[CoolCold]

On Sat, Apr 11, 2009 at 9:29 AM, Timothy D. Lenz <tlenz@vorgon.com> wrote:
> From: "CoolCold" <coolthecold@gmail.com>
>
>> Yes, he should provide correct /etc/mdadm/mdadm.conf and
>> update-initramfs -u on md boot, smth like
>> chroot /mnt/md0
>> update-initramfs -u
>
> How would this help in a system that doesn't ramdisk built in to the kernel or as a module? Or does it change some other stuff?
Looking into "I have 2 computers that I am trying to convert to raid.
Both where setup with debian netinst cd to hda" - debian has initrd
support within their kernels.
>
>
> I started looking at the stuff that was copied to md0 and /md0/dev is empty. Looking through the guides I found a few things. One
> said that using cp had to be from root or not everything would get coppied. I used sudo but I know some things require you to root.
> Also this:
> ===============================
> # rsync -avHhx --progress / /mnt/raid-md0
>
>    * If the system wasn't previously in single user mode, move to single user mode and update the data that changed during the
> first copy:
> (--delete flag tells rsync to delete files from the destination which do not exist on the source):
>
> # rsync -avHhx --progress --delete / /mnt/raid-md0
>
>    * Create needed device nodes:
>
> # cd /mnt/raid-md0/dev/ && MAKEDEV generic
> ===============================
> Using rsync from single user mode still left /mnt/md0/dev empty. I read up in "makedev generic" and it seems to be a shotgun fix
> adding way more then is needed. Is there a way to create just what is in /dev?
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
-- 
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 2467 bytes --]

On Fri Apr 10, 2009 at 10:29:59PM -0700, Timothy D. Lenz wrote:

> From: "CoolCold" <coolthecold@gmail.com>
> 
> > Yes, he should provide correct /etc/mdadm/mdadm.conf and
> > update-initramfs -u on md boot, smth like
> > chroot /mnt/md0
> > update-initramfs -u
> 
> How would this help in a system that doesn't ramdisk built in to the
> kernel or as a module? Or does it change some other stuff?
> 
This is useless if you don't use an initrd.

> I started looking at the stuff that was copied to md0 and /md0/dev is
> empty. Looking through the guides I found a few things. One said that
> using cp had to be from root or not everything would get coppied. I
> used sudo but I know some things require you to root.
>
This shouldn't matter in this case.  The reason you're getting nothing
copied in /dev is that it's a mounted filesystem, and your copy command
specifically excludes mounted filesystems.

> Also this:
> ===============================
> # rsync -avHhx --progress / /mnt/raid-md0
> 
>     * If the system wasn't previously in single user mode, move to
>     single user mode and update the data that changed during the first
>     copy:
> (--delete flag tells rsync to delete files from the destination which
> do not exist on the source):
> 
> # rsync -avHhx --progress --delete / /mnt/raid-md0
> 
Yes, this is an alternative to running from a bootable CD.  In
single-user mode (init 1), there can be no background applications
running, so there should be no open files to worry about.  I'd still
rather use a bootable CD though.

>     * Create needed device nodes:
> 
> # cd /mnt/raid-md0/dev/ && MAKEDEV generic
> ===============================
> Using rsync from single user mode still left /mnt/md0/dev empty. I
> read up in "makedev generic" and it seems to be a shotgun fix adding
> way more then is needed. Is there a way to create just what is in
> /dev?
> 
Again, the rsync command includes the 'x' option so excludes mounted
filesystems (which is what you want here).  You can copy the necessary
/dev entries manually - I think the only entries you need are
/dev/console & /dev/null, so:
    cp -a /dev/console /dev/null /mnt/raid-md0/dev

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[jim owens]

Robin Hill wrote:
>> From: "CoolCold" <coolthecold@gmail.com>
>>
>> # rsync -avHhx --progress --delete / /mnt/raid-md0
>>
> Yes, this is an alternative to running from a bootable CD.  In
> single-user mode (init 1), there can be no background applications
> running, so there should be no open files to worry about.  I'd still
> rather use a bootable CD though.

Strong ACK on bootable CD as the right method, or doing it all
while init 1... not as a 2-step multi-user mode, single user.

rsync is a very good tool but unless you use:

    -c, --checksum    skip based on checksum, not mod-time & size

the consistency between runs is not 100% safe.  It may be 99.9%,
but the problem is that for file blocks being rewritten you don't
know what is on disk or in memory and mod-time only changes at the
whim of the FS (I'm an FS developer) and at the granularity of
the FS and system clock... nanosecond time stamps are a myth!

And if you are going to use the rsync -c 2-pass method it will
probably take your system off line longer than a CD boot and copy.

At the end of process, is 99.9% good enough?  For much of my
backup needs it is because I only care about not loosing more
than a small amount of work.  But if I'm converting to a new
disk and loosing the old one, I want 100%.

jim

Re: Converting system to raid

[Jeff Garzik]

jim owens wrote:
> nanosecond time stamps are a myth!

Not true, in the age of NO_HZ...

	Jeff

Re: Converting system to raid

[jim owens]

Jeff Garzik wrote:
> jim owens wrote:
>> nanosecond time stamps are a myth!
> 
> Not true, in the age of NO_HZ...

While nanosecond time is real on some hardware platforms,
I don't know of any hard disk filesystem that does metadata
updates at that level of granularity and guarantees distinct
timestamps between writes at extremely short intervals.

Even SSD performance would suck if we did that.

The filesystem mod-time might have nanoseconds in it, but
that does not mean 2 writes 100 nanoseconds apart will have
different mod-times.

jim

Re: Converting system to raid

[Timothy D. Lenz]

Yes, the default setup. But that is because the supplied kernel builds raid as modules. I build a new kernel with stuff needed for
my system ecept for dvb stuf which I get by building v4l tree to get newest drivers. And I build raid into the kernel. Since I
havn't needed ramdisk for anything up till now and you are not suposed to need it when raid is built it, ramdisk is not compiled in
or as a module.

----- Original Message ----- 
From: "CoolCold" <coolthecold@gmail.com>
To: "Timothy D. Lenz" <tlenz@vorgon.com>
Cc: <linux-raid@vger.kernel.org>
Sent: Saturday, April 11, 2009 7:29 AM
Subject: Re: Converting system to raid


On Sat, Apr 11, 2009 at 9:29 AM, Timothy D. Lenz <tlenz@vorgon.com> wrote:
> From: "CoolCold" <coolthecold@gmail.com>
>
>> Yes, he should provide correct /etc/mdadm/mdadm.conf and
>> update-initramfs -u on md boot, smth like
>> chroot /mnt/md0
>> update-initramfs -u
>
> How would this help in a system that doesn't ramdisk built in to the kernel or as a module? Or does it change some other stuff?
Looking into "I have 2 computers that I am trying to convert to raid.
Both where setup with debian netinst cd to hda" - debian has initrd
support within their kernels.
>
>
> I started looking at the stuff that was copied to md0 and /md0/dev is empty. Looking through the guides I found a few things. One
> said that using cp had to be from root or not everything would get coppied. I used sudo but I know some things require you to
root.
> Also this:
> ===============================
> # rsync -avHhx --progress / /mnt/raid-md0
>
> * If the system wasn't previously in single user mode, move to single user mode and update the data that changed during the
> first copy:
> (--delete flag tells rsync to delete files from the destination which do not exist on the source):
>
> # rsync -avHhx --progress --delete / /mnt/raid-md0
>
> * Create needed device nodes:
>
> # cd /mnt/raid-md0/dev/ && MAKEDEV generic
> ===============================
> Using rsync from single user mode still left /mnt/md0/dev empty. I read up in "makedev generic" and it seems to be a shotgun fix
> adding way more then is needed. Is there a way to create just what is in /dev?
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>



-- 
-- 
Best regards,
[COOLCOLD-RIPN]

Re: Converting system to raid

[Timothy D. Lenz]

I used a grml x86-64 boot cd and it seemed to create hda as sdc. I mounted sdc1 and md0 to /mnt and used:

rsync -cavHh --progress --delete /mnt/sdc1 /mnt/md0

After rebooting back to hda1 I changed fstab to put root on /dev/md0 and changed menu.lst on /mnt/md0. Rebooted and moved the ide to
the bottom of the boot list in cmos so the 2 sata drives are listed first. It seems to have booted to md0 ok. Now I need to confirm
everything is set correct.

Before when at the grub boot menu when booting with the ide moved below the sata drives, droping to command line for grub showed
that the ide drive only moved down 1, not 2. It seemed to have:
  hd0 = sda
  hd1 = hda
  hd2 = sdb

Now with the array booted, going to the grub command line and using auto complete for "root (hd0," it seems to be using the device
map at this point which is no longer correct:
(hd0) /dev/hda
(hd1) /dev/sda
(hd2) /dev/sdb

Note that in the other system, because linux was installed with only the ide and 1 sata drive, it doesn't even list the other two in
it's device map. So what is the best/safest way to update/correct this to what ever it should be?
=============================================

Moving swap:
I have used "sudo mkswap /dev/md1" so if I read info correct, I should be able to just change fstab:
  /dev/md1       none            swap    sw              0       0

and reboot to start using md1 array for swap?
=============================================
menu.lst updates:
The guides all call for assorted changes to menu.lst Some of the guides call for adding the line savedefault to kernel entries but
the people in #grub said not to do that with a raid system as it would corrupt the array and I thought I saw something about that
reading about grub but can't find it.

Some guides call for adding "fallback 1" after "default 0". And all call for adding duplicating the first kernel entry but changing
root to (1,0) for the second entry. But some have said it's not needed. Sorry of some of these questions have been answered before,
But I want to make sure everything is updated/corrected as needed before moving on to the other stuff I need to do. Don't want to
get nearly done and have a nasty suprise down the road or the next time I build/install a new kernel because something was missed.

menu.lst as it is now:
# menu.lst - See: grub(8), info grub, update-grub(8)
#            grub-install(8), grub-floppy(8),
#            grub-md5-crypt, /usr/share/doc/grub
#            and /usr/share/doc/grub-legacy-doc/.

## default num
# Set the default entry to the entry number NUM. Numbering starts from 0, and
# the entry number 0 is the default if the command is not used.
#
# You can specify 'saved' instead of a number. In this case, the default entry
# is the entry saved with the command 'savedefault'.
# WARNING: If you are using dmraid do not change this entry to 'saved' or your
# array will desync and will not let you boot your system.
default  0

## timeout sec
# Set a timeout, in SEC seconds, before automatically booting the default entry
# (normally the first entry defined).
timeout  5

# Pretty colours
color cyan/red yellow/black

## password ['--md5'] passwd
# If used in the first section of a menu file, disable all interactive editing
# control (menu entry editor and command-line)  and entries protected by the
# command 'lock'
# e.g. password topsecret
#      password --md5 $1$gLhU0/$aW78kHK1QfV3P2b2znUoe/
# password topsecret

#
# examples
#
# title  Windows 95/98/NT/2000
# root  (hd0,0)
# makeactive
# chainloader +1
#
# title  Linux
# root  (hd0,1)
# kernel /vmlinuz root=/dev/hda2 ro
#

#
# Put static boot stanzas before and/or after AUTOMAGIC KERNEL LIST

### BEGIN AUTOMAGIC KERNELS LIST
## lines between the AUTOMAGIC KERNELS LIST markers will be modified
## by the debian update-grub script except for the default options below

## DO NOT UNCOMMENT THEM, Just edit them to your needs

## ## Start Default Options ##
## default kernel options
## default kernel options for automagic boot options
## If you want special options for specific kernels use kopt_x_y_z
## where x.y.z is kernel version. Minor versions can be omitted.
## e.g. kopt=root=/dev/hda1 ro
##      kopt_2_6_8=root=/dev/hdc1 ro
##      kopt_2_6_8_2_686=root=/dev/hdc2 ro
# kopt=root=/dev/md0 ro

## default grub root device
## e.g. groot=(hd0,0)
# groot=(hd0,0)

## should update-grub create alternative automagic boot options
## e.g. alternative=true
##      alternative=false
# alternative=true

## should update-grub lock alternative automagic boot options
## e.g. lockalternative=true
##      lockalternative=false
# lockalternative=false

## additional options to use with the default boot option, but not with the
## alternatives
## e.g. defoptions=vga=791 resume=/dev/hda5
# defoptions=quiet

## should update-grub lock old automagic boot options
## e.g. lockold=false
##      lockold=true
# lockold=false

## Xen hypervisor options to use with the default Xen boot option
# xenhopt=

## Xen Linux kernel options to use with the default Xen boot option
# xenkopt=console=tty0

## altoption boot targets option
## multiple altoptions lines are allowed
## e.g. altoptions=(extra menu suffix) extra boot options
##      altoptions=(single-user) single
# altoptions=(single-user mode) single

## controls how many kernels should be put into the menu.lst
## only counts the first occurence of a kernel, not the
## alternative kernel options
## e.g. howmany=all
##      howmany=7
# howmany=all

## should update-grub create memtest86 boot option
## e.g. memtest86=true
##      memtest86=false
# memtest86=true

## should update-grub adjust the value of the default booted system
## can be true or false
# updatedefaultentry=false

## should update-grub add savedefault to the default options
## can be true or false
# savedefault=false

## ## End Default Options ##

title  Debian GNU/Linux, kernel 2.6.26.8.20090311.1
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.26.8.20090311.1 root=/dev/md0 ro quiet

title  Debian GNU/Linux, kernel 2.6.26.8.20090311.1 (single-user mode)
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.26.8.20090311.1 root=/dev/md0 ro single

title  Debian GNU/Linux, kernel 2.6.26-1-686
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.26-1-686 root=/dev/hda1 ro quiet
initrd  /boot/initrd.img-2.6.26-1-686

title  Debian GNU/Linux, kernel 2.6.26-1-686 (single-user mode)
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.26-1-686 root=/dev/hda1 ro single
initrd  /boot/initrd.img-2.6.26-1-686

### END DEBIAN AUTOMAGIC KERNELS LIST

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 3988 bytes --]

On Sat Apr 11, 2009 at 05:53:04PM -0700, Timothy D. Lenz wrote:

> I used a grml x86-64 boot cd and it seemed to create hda as sdc. I
> mounted sdc1 and md0 to /mnt and used:
> 
> rsync -cavHh --progress --delete /mnt/sdc1 /mnt/md0
> 
Yes, GRML uses the newer libata drivers for both PATA and SATA, so both
will appear as sd? devices.

> After rebooting back to hda1 I changed fstab to put root on /dev/md0
> and changed menu.lst on /mnt/md0. Rebooted and moved the ide to the
> bottom of the boot list in cmos so the 2 sata drives are listed first.
> It seems to have booted to md0 ok. Now I need to confirm everything is
> set correct.
> 
That's good :)

> Before when at the grub boot menu when booting with the ide moved
> below the sata drives, droping to command line for grub showed that
> the ide drive only moved down 1, not 2. It seemed to have:
>   hd0 = sda
>   hd1 = hda
>   hd2 = sdb
> 
> Now with the array booted, going to the grub command line and using
> auto complete for "root (hd0," it seems to be using the device map at
> this point which is no longer correct:
> (hd0) /dev/hda
> (hd1) /dev/sda
> (hd2) /dev/sdb
> 
> Note that in the other system, because linux was installed with only
> the ide and 1 sata drive, it doesn't even list the other two in it's
> device map. So what is the best/safest way to update/correct this to
> what ever it should be?
>
Just delete/rename the old device map file (/boot/grub/device.map) and
run grub again - it'll rescan the devices and create a new map file.

> Moving swap:
> I have used "sudo mkswap /dev/md1" so if I read info correct, I should
> be able to just change fstab:
>   /dev/md1       none            swap    sw              0       0
> 
> and reboot to start using md1 array for swap?
>
Yup, swap's very simple to change.

> menu.lst updates:
> The guides all call for assorted changes to menu.lst Some of the
> guides call for adding the line savedefault to kernel entries but the
> people in #grub said not to do that with a raid system as it would
> corrupt the array and I thought I saw something about that reading
> about grub but can't find it.
> 
No, best not doing that - it just makes grub always default to the last
chosen option, rather than always using the same default.

> Some guides call for adding "fallback 1" after "default 0". And all
> call for adding duplicating the first kernel entry but changing root
> to (1,0) for the second entry. But some have said it's not needed.
> Sorry of some of these questions have been answered before, but I want
> to make sure everything is updated/corrected as needed before moving
> on to the other stuff I need to do. Don't want to get nearly done and
> have a nasty suprise down the road or the next time I build/install a
> new kernel because something was missed.
> 
Neither of these options are likely to make much difference.  Fallback
just means it will boot the second entry if the first can't be found -
it's more likely that the first will fail to boot, in which case you're
no better off.  As for adding a duplicate entry with a changed root
device - this is unlikely to help at all.  In order for GRUB to boot
that far, it has to have read the stage1 bootloader from the MBR of the
first disk and the stage 1.5, stage 2 and menu files from /boot on the
first disk.  The chances of it managing all of that but then being
unable to read the kernel from the first disk are pretty slim.

I'd just leave it as is - if the first disk fails altogether then it
should boot straight from the second disk, and for other failures you
can just edit the boot commands in grub manually to fix/work around
whatever the issue is.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Jeff Garzik]

jim owens wrote:
> Jeff Garzik wrote:
>> jim owens wrote:
>>> nanosecond time stamps are a myth!
>>
>> Not true, in the age of NO_HZ...
> 
> While nanosecond time is real on some hardware platforms,
> I don't know of any hard disk filesystem that does metadata
> updates at that level of granularity and guarantees distinct
> timestamps between writes at extremely short intervals.
> 
> Even SSD performance would suck if we did that.
> 
> The filesystem mod-time might have nanoseconds in it, but
> that does not mean 2 writes 100 nanoseconds apart will have
> different mod-times.

Modern Linux filesystems absolutely do do metadata updates at that level 
of granularity.

Are you guaranteed district timestamps between writes?  No, but then 
again, metadata updates were never guaranteed between two writes either. 
  You might just be dirtying a mmap'd page, for example.

	Jeff

Re: Converting system to raid

[jim owens]

To continue the pointless (but maybe entertaining) argument :)

Jeff Garzik wrote:
> jim owens wrote:
>> The filesystem mod-time might have nanoseconds in it, but
>> that does not mean 2 writes 100 nanoseconds apart will have
>> different mod-times.
> 
> Modern Linux filesystems absolutely do do metadata updates at that level 
> of granularity.

In filesystems it only counts if it makes it to disk.

10,000,000 write-to-disk metadata updates per second?  I think not.

Even the top-of-the-line intel ssd can only do 3,300 writes per second.

> Are you guaranteed district timestamps between writes?  No, but then 
> again, metadata updates were never guaranteed between two writes either. 
>  You might just be dirtying a mmap'd page, for example.

So this sounds like you at least agree with my original statement
that mod-time is not a 100% guarantee that no write occurred after
you read a particular block of a file.  Regardless of the fact we
disagree about how accurate filesystems record times.

jim

Re: Converting system to raid

[Jeff Garzik]

jim owens wrote:
> To continue the pointless (but maybe entertaining) argument :)
> 
> Jeff Garzik wrote:
>> jim owens wrote:
>>> The filesystem mod-time might have nanoseconds in it, but
>>> that does not mean 2 writes 100 nanoseconds apart will have
>>> different mod-times.
>>
>> Modern Linux filesystems absolutely do do metadata updates at that 
>> level of granularity.
> 
> In filesystems it only counts if it makes it to disk.
> 
> 10,000,000 write-to-disk metadata updates per second?  I think not.

Does every write (mtime update) trigger a write to disk?  I think not :) 
  You can write (== update inode mtime) 10,000,000 times, without ever 
touching disk.  Yet you fstat() still returns a nanosecond-granularity 
mtime.

	Jeff

Re: Converting system to raid

[Timothy D. Lenz]

While booted on the array I renamed device.map and ran:
sudo grub --no-floppy --device-map=/boot/grub/device.map

But the new map is the same:
(hd0) /dev/hda
(hd1) /dev/sda
(hd2) /dev/sdb

----- Original Message ----- 
From: "Robin Hill" <robin@robinhill.me.uk>
To: <linux-raid@vger.kernel.org>
Sent: Sunday, April 12, 2009 4:11 AM
Subject: Re: Converting system to raid

> Before when at the grub boot menu when booting with the ide moved
> below the sata drives, droping to command line for grub showed that
> the ide drive only moved down 1, not 2. It seemed to have:
>   hd0 = sda
>   hd1 = hda
>   hd2 = sdb
> 
> Now with the array booted, going to the grub command line and using
> auto complete for "root (hd0," it seems to be using the device map at
> this point which is no longer correct:
> (hd0) /dev/hda
> (hd1) /dev/sda
> (hd2) /dev/sdb
> 
> Note that in the other system, because linux was installed with only
> the ide and 1 sata drive, it doesn't even list the other two in it's
> device map. So what is the best/safest way to update/correct this to
> what ever it should be?
>
Just delete/rename the old device map file (/boot/grub/device.map) and
run grub again - it'll rescan the devices and create a new map file.

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

On Sun Apr 12, 2009 at 12:55:33PM -0700, Timothy D. Lenz wrote:

> While booted on the array I renamed device.map and ran:
> sudo grub --no-floppy --device-map=/boot/grub/device.map
> 
> But the new map is the same:
> (hd0) /dev/hda
> (hd1) /dev/sda
> (hd2) /dev/sdb
> 
That's a little odd - it should be picking up the BIOS order (though it
may actually depend on what order the kernel picks up the devices).  You
can just edit the file manually anyway.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Converting system to raid

[Timothy D. Lenz]

Welllll, tried to switch the other system over to raid boot and something went wrong ether with the copy or with menu.lst change or
with grub, I think. I changed the boot order moving the ide below the 3 sata drives and before the grub boot menu came up I got an
"error 2"

This system is booting from hda1 and will be booting from md0 like the other system. This system has 3 sata drives, so md0 and md1
where setup has raid1 plus spare.

To set up group I used:
sudo grub --no-floppy

Grub> device (hd0) /dev/sda
Grub> root (hd0,0)
Grub> setup (hd0)

Grub> device (hd0) /dev/sdb
Grub> root (hd0,0)
Grub> setup (hd0)

Grub> device (hd0) /dev/sdc
Grub> root (hd0,0)
Grub> setup (hd0)
Grub> quit

it gave me errors for sdc because it is not formated as formating was done on after the array was made. I assume that madadm will
take care of boot setup if it activates it?

I used grml cd started with "grml swraid" and to copy I used:
rsync -cavHh --progress --delete /mnt/sdd1/ /mnt/md0

While booted from hda1 fdisk -l gives:
sudo fdisk -l

Disk /dev/hda: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x6b381dfe

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1        4660    37431418+  83  Linux
/dev/hda2            4661        4865     1646662+   5  Extended
/dev/hda5            4661        4865     1646631   82  Linux swap / Solaris

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x94140963

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        3040    24418768+  fd  Linux raid autodetect
/dev/sda2            3041        3649     4891792+  fd  Linux raid autodetect
/dev/sda3            3650       60801   459073440   fd  Linux raid autodetect

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xf1814421

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1        3040    24418768+  fd  Linux raid autodetect
/dev/sdb2            3041        3649     4891792+  fd  Linux raid autodetect
/dev/sdb3            3650       60801   459073440   fd  Linux raid autodetect

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x371b6063

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *           1        3040    24418768+  fd  Linux raid autodetect
/dev/sdc2            3041        3649     4891792+  fd  Linux raid autodetect
/dev/sdc3            3650       60801   459073440   fd  Linux raid autodetect

Disk /dev/md0: 25.0 GB, 25004736512 bytes
2 heads, 4 sectors/track, 6104672 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md2: 940.1 GB, 940182208512 bytes
2 heads, 4 sectors/track, 229536672 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md2 doesn't contain a valid partition table

Disk /dev/md1: 5009 MB, 5009113088 bytes
2 heads, 4 sectors/track, 1222928 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table
--------------------------------------------------------------
df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1              36G  2.9G   31G   9% /
tmpfs                1006M     0 1006M   0% /lib/init/rw
udev                   10M  120K  9.9M   2% /dev
tmpfs                1006M     0 1006M   0% /dev/shm
/dev/md0               23G  2.9G   19G  13% /mnt/md0
/dev/md2              862G  227G  592G  28% /mnt/md2
--------------------------------------------------------------
menu.lst for raid

# kopt=root=/dev/md0 ro quiet

# groot=(hd0,0)

title  Debian GNU/Linux, kernel 2.6.25.9.20081002.1
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.25.9.20081002.1 root=/dev/md0 ro quiet

title  Debian GNU/Linux, kernel 2.6.25.9.20081002.1 (single-user mode)
root  (hd0,0)
kernel  /boot/vmlinuz-2.6.25.9.20081002.1 root=/dev/md0 ro quiet single

Re: Converting system to raid

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 2545 bytes --]

On Thu Apr 16, 2009 at 05:47:10PM -0700, Timothy D. Lenz wrote:

> Welllll, tried to switch the other system over to raid boot and
> something went wrong ether with the copy or with menu.lst change or
> with grub, I think. I changed the boot order moving the ide below the
> 3 sata drives and before the grub boot menu came up I got an "error
> 2"
> 
> This system is booting from hda1 and will be booting from md0 like the
> other system. This system has 3 sata drives, so md0 and md1 where
> setup has raid1 plus spare.
> 
Why have a spare?  You'd be better running 3-drive RAID1 arrays.

> To set up group I used:
> sudo grub --no-floppy
> 
> Grub> device (hd0) /dev/sda
> Grub> root (hd0,0)
> Grub> setup (hd0)
> 
> Grub> device (hd0) /dev/sdb
> Grub> root (hd0,0)
> Grub> setup (hd0)
> 
> Grub> device (hd0) /dev/sdc
> Grub> root (hd0,0)
> Grub> setup (hd0)
> Grub> quit
> 
> it gave me errors for sdc because it is not formated as formating was
> done on after the array was made. I assume that madadm will take care
> of boot setup if it activates it?
> 
Not sure what you mean here, but you _must_ have the root filesystem
available before running grub.  What the setup command in grub does is
install the stage1 bootloader (in this case to the MBR), setting a
pointer to the on-disk location of the stage 1.5 bootloader (which
includes the relevant filesystem support, allowing the stage 2
bootloader & menu to be read).  If the stage 1.5 bootloader is moved
on-disk (or doesn't exist in the first place) then booting will fail.

I'd recommend:
    - growing each of the arrays to 3 disks.  This will give improved
      performance; remove the lack of redundancy when a drive fails (as
      you don't have to wait for the array to rebuild); and ensure the
      3rd disk is actually working (having the drive fail during a
      rebuild is not a nice experience).  The downsides will be a
      slightly increased power usage, and increased wear on the drive
      (though I've not heard any concrete evidence that this actually
      has any impact on drive failure).
    - rerun the grub setup for all drives (once the arrays are all
      synched).  This will ensure the boot loader is pointing to the
      correct on-disk location.

HTH,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]