Adaptec 2405 : hardware or software raid?

Adaptec 2405 : hardware or software raid?

[Janek Kozicki]

Hi,

I've bought the cheap Adaptec 2405 SAS controller for the sole
purpose of using 15k rpm HDDs. It turns out that to expose the HDDs
to the system I must first launch the computer from enclosed CD and
create some logical volumes. Those logical volumes can mix HDDs in
various raid configurations (1,0,10,5) and expose a single
device, eg. /dev/sda from multiple HDDs.

This is the first time when I see the "logical volumes" option
mandatory. I was planning to access the HDDs directly and create
software raid on them with mdadm.

It is possible, however to create ONE logical volume per ONE physical
disc, and then I can use mdadm on them.

But now, having seen this Adaptec software/tool to generate the
hardware raid configurations (and no way around it) I started to
wonder what approach is better?

What do you recommend:

- make one logical volume per disc and use only software raid?
- use Adaptec 2405 hardware raid entirely?
- or do you know some way around the hardware and access the SAS
  discs directly? (is that approach safe?)

What has better linux support (which kernel version), other
considerations, etc...

thanks in advance for your help
-- 
Janek Kozicki

Re: Adaptec 2405 : hardware or software raid?

[Andrew Burgess]

On Mon, 2009-05-25 at 17:32 +0200, Janek Kozicki wrote:
> Hi,
> 
> I've bought the cheap Adaptec 2405 SAS controller for the sole
> purpose of using 15k rpm HDDs. It turns out that to expose the HDDs
> to the system I must first launch the computer from enclosed CD and
> create some logical volumes. Those logical volumes can mix HDDs in
> various raid configurations (1,0,10,5) and expose a single
> device, eg. /dev/sda from multiple HDDs.
> 
> This is the first time when I see the "logical volumes" option
> mandatory. I was planning to access the HDDs directly and create
> software raid on them with mdadm.

This datasheet says JBOD is available (Just a Bunch of Disks - exposing
the raw disks to the OS)

http://adaptec.com/en-US/products/Controllers/Hardware/sas/entry/SAS-2405/_details/Series2_datasheet.htm?nc=/en-US/products/Controllers/Hardware/sas/entry/SAS-2405/_details/Series2_datasheet.htm

main page

http://adaptec.com/en-US/products/Controllers/Hardware/sas/entry/SAS-2405/

Upgrading a software RAID

[Maxime Boissonneault]

Hello,
I am using Ubuntu Hardy with 3x500GB drives and the following RAID
configuration :

/boot is on a 100MB RAID1
/ is on a 30GB RAID0
/home is on a 906GB RAID5

I want to replace the 3 drives by 3 1TB drives.

Here is how I planned to do it :
0- Backup my /home on some external disk.
1- backup / with something like :
sudo tar cvpzf /backup.tgz --exclude=/media --exclude=/proc
--exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys
--exclude=/home /
mv /backup.tgz $1
2- Replace 1 disk
3- Boot and let the RAID1 and RAID5 reconstruct
4- Replace 1 other disk
5- Boot and let the RAID1 and RAID5 reconstruct again
6- Replace the last disk
7- Boot and let the RAID1 and RAID5 reconstruct one last time
8- Boot and restore the backup on the RAID0 / partition.
9- Resize the /home partition to 1 TB.

I suspect there will be a problem replacing the primary disk, but I guessed
that I could solve this simply by changing which is the primary disk in the
BIOS.

Is there any other problem that will or could happen ?

For example, I am not sure if the raid manager is on the /boot partition or
on the /. I guess if it is on /, it won't work at all since the raid manager
itself won't be able to run ?
Also, is it possible to boot and access a command line to restore the backup
with a failed / partition ?

I am also unsure about how I should proceed to resize the /home partition.
Is this done through mdadm ?

Please enlight me on any problems that I will have.

Thanks

Re: Upgrading a software RAID

[Bill Davidsen]

Maxime Boissonneault attempted to hijack this thread by posting this 
totally unrelated message:
> Hello,
> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
> configuration :
Please do not respond to obviously off-topic posts.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.

Re: Upgrading a software RAID

[Maxime Boissonneault]

What do you mean ?

I sent this message to the list a couple of days ago and did not get any 
answer yet. How is this message off-topic from a linux-raid list ?

Thanks,

Maxime Boissonneault


Bill Davidsen a écrit :
> Maxime Boissonneault attempted to hijack this thread by posting this 
> totally unrelated message:
>> Hello,
>> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
>> configuration :
> Please do not respond to obviously off-topic posts.
>

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Bill Davidsen]

Maxime Boissonneault wrote:
> What do you mean ?
>
> I sent this message to the list a couple of days ago and did not get 
> any answer yet. How is this message off-topic from a linux-raid list ?
>
> Thanks,
>
> Maxime Boissonneault
>
>
> Bill Davidsen a écrit :
>> Maxime Boissonneault attempted to hijack this thread by posting this 
>> totally unrelated message:
>>> Hello,
>>> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
>>> configuration :
>> Please do not respond to obviously off-topic posts.
>>

You injected this into a discussion of Adaptec 2405 disk controller 
configuration. It has nothing at all to do with the original topic. If 
you want to talk about something else don't force your post into an 
existing thread on an unrelated topic, it screws up threading for anyone 
reading.

It also keeps people who are ignoring the thread from seeing your post.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 3095 bytes --]

On Mon May 25, 2009 at 01:05:15PM -0400, Maxime Boissonneault wrote:

> Hello,
> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
> configuration :
>
> /boot is on a 100MB RAID1
> / is on a 30GB RAID0
> /home is on a 906GB RAID5
>
Ouch - why RAID0 for /?  If you lose a single drive then all the
configuration, etc. is down the drain.  I'd suggest rethinking this
while you're going through the rebuild process anyway.

> I want to replace the 3 drives by 3 1TB drives.
>
> Here is how I planned to do it :
> 0- Backup my /home on some external disk.
> 1- backup / with something like :
> sudo tar cvpzf /backup.tgz --exclude=/media --exclude=/proc
> --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys
> --exclude=/home /
> mv /backup.tgz $1
You'll need to do the tar in single user mode (init 1) to ensure there's
no open files when you're backing up (or use a bootable CD).

> 2- Replace 1 disk
> 3- Boot and let the RAID1 and RAID5 reconstruct
You'll have to boot to a CD then - your root FS is trashed at this
point.

> 4- Replace 1 other disk
> 5- Boot and let the RAID1 and RAID5 reconstruct again
> 6- Replace the last disk
> 7- Boot and let the RAID1 and RAID5 reconstruct one last time
> 8- Boot and restore the backup on the RAID0 / partition.
> 9- Resize the /home partition to 1 TB.
>
> I suspect there will be a problem replacing the primary disk, but I guessed
> that I could solve this simply by changing which is the primary disk in the
> BIOS.
>
> Is there any other problem that will or could happen ?
>
See above - your current plan is pretty much doomed I'm afraid.

> For example, I am not sure if the raid manager is on the /boot partition or
> on the /. I guess if it is on /, it won't work at all since the raid 
> manager
> itself won't be able to run ?
> Also, is it possible to boot and access a command line to restore the 
> backup
> with a failed / partition ?
>
> I am also unsure about how I should proceed to resize the /home partition.
> Is this done through mdadm ?
>
No, you'll have to delete & recreate the partition using fdisk.

> Please enlight me on any problems that I will have.
>

An easier option (assuming you want to keep the arrays as currently
setup) would be to boot from a CD and do a full copy of each of the
500GB disks to the 1TB disks.  You'll then just need to resize the
last partition (assuming /home is set up on the last partition on the
disks anyway) and grow the array and filesystem.

A _better_ option (if possible) would be to install all the drives in
the system concurrently, then you can boot from CD and create arrays on
the new drives and copy the data across.  You'll also need to update
mdadm.conf (and the initrd if you're using one) to indicate the new
array IDs.

HTH,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Upgrading a software RAID

[Maxime Boissonneault]

Robin Hill a écrit :
> On Mon May 25, 2009 at 01:05:15PM -0400, Maxime Boissonneault wrote:
>
>   
>> Hello,
>> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
>> configuration :
>>
>> /boot is on a 100MB RAID1
>> / is on a 30GB RAID0
>> /home is on a 906GB RAID5
>>
>>     
> Ouch - why RAID0 for /?  If you lose a single drive then all the
> configuration, etc. is down the drain.  I'd suggest rethinking this
> while you're going through the rebuild process anyway.
>
>   
That is why I do backups of / on /home regularily. I first did a RAID0 
for performance. I did some testing on performance of RAID0,1,5 and 
RAID1 was pretty lousy (I put the results of my tests here : 
http://cqed.physique.usherbrooke.ca/~mboisson/htpc.php?sec=raid_test )
I guess I should have used a RAID5, but RAID1 seems like a terrible idea 
considering the really bad performances.

>> I want to replace the 3 drives by 3 1TB drives.
>>
>> Here is how I planned to do it :
>> 0- Backup my /home on some external disk.
>> 1- backup / with something like :
>> sudo tar cvpzf /backup.tgz --exclude=/media --exclude=/proc
>> --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys
>> --exclude=/home /
>> mv /backup.tgz $1
>>     
> You'll need to do the tar in single user mode (init 1) to ensure there's
> no open files when you're backing up (or use a bootable CD).
>
>   
I used this backup before to restore the system and it seemed to work 
pretty well.
>> 2- Replace 1 disk
>> 3- Boot and let the RAID1 and RAID5 reconstruct
>>     
> You'll have to boot to a CD then - your root FS is trashed at this
> point.
>
>   
>> 4- Replace 1 other disk
>> 5- Boot and let the RAID1 and RAID5 reconstruct again
>> 6- Replace the last disk
>> 7- Boot and let the RAID1 and RAID5 reconstruct one last time
>> 8- Boot and restore the backup on the RAID0 / partition.
>> 9- Resize the /home partition to 1 TB.
>>
>> I suspect there will be a problem replacing the primary disk, but I guessed
>> that I could solve this simply by changing which is the primary disk in the
>> BIOS.
>>
>> Is there any other problem that will or could happen ?
>>
>>     
> See above - your current plan is pretty much doomed I'm afraid.
>
>   
>> For example, I am not sure if the raid manager is on the /boot partition or
>> on the /. I guess if it is on /, it won't work at all since the raid 
>> manager
>> itself won't be able to run ?
>> Also, is it possible to boot and access a command line to restore the 
>> backup
>> with a failed / partition ?
>>
>> I am also unsure about how I should proceed to resize the /home partition.
>> Is this done through mdadm ?
>>
>>     
> No, you'll have to delete & recreate the partition using fdisk.
>
>   
>> Please enlight me on any problems that I will have.
>>
>>     
>
> An easier option (assuming you want to keep the arrays as currently
> setup) would be to boot from a CD and do a full copy of each of the
> 500GB disks to the 1TB disks.  You'll then just need to resize the
> last partition (assuming /home is set up on the last partition on the
> disks anyway) and grow the array and filesystem.
>
> A _better_ option (if possible) would be to install all the drives in
> the system concurrently, then you can boot from CD and create arrays on
> the new drives and copy the data across.  You'll also need to update
> mdadm.conf (and the initrd if you're using one) to indicate the new
> array IDs.
>
> HTH,
>     Robin
>   

I can not install more drives in the computer. It is a home theater 
computer in a small case. I was expecting to be able to let the raid 
manage the copies itself.

If the / was on a RAID5, would it be able to boot with 2 disks ?
If so, is it possible to convert my RAID0 to a RAID5 ?
For example, I could boot on a CD, backup / onto /home, delete the RAID0 
array and recreate it as RAID5, then restore the backup. Would this work ?


Thanks


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 4041 bytes --]

On Thu May 28, 2009 at 10:32:53AM -0400, Maxime Boissonneault wrote:

>
> Robin Hill a écrit :
>> On Mon May 25, 2009 at 01:05:15PM -0400, Maxime Boissonneault wrote:
>>
>>   
>>> Hello,
>>> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
>>> configuration :
>>>
>>> /boot is on a 100MB RAID1
>>> / is on a 30GB RAID0
>>> /home is on a 906GB RAID5
>>>
>>>     
>> Ouch - why RAID0 for /?  If you lose a single drive then all the
>> configuration, etc. is down the drain.  I'd suggest rethinking this
>> while you're going through the rebuild process anyway.
>>
>>   
> That is why I do backups of / on /home regularily. I first did a RAID0 for 
> performance. I did some testing on performance of RAID0,1,5 and RAID1 was 
> pretty lousy (I put the results of my tests here : 
> http://cqed.physique.usherbrooke.ca/~mboisson/htpc.php?sec=raid_test )
> I guess I should have used a RAID5, but RAID1 seems like a terrible idea 
> considering the really bad performances.
>
There's definitely something up with the RAID1 there - there's no way it
should be that much slower than a single disk (it ought to be about the
same speed).  There have been some recent fixes to RAID1, so a more
recent kernel may help as well.  RAID10 could also be worth considering
as an alternative.

>>> I want to replace the 3 drives by 3 1TB drives.
>>>
>>> Here is how I planned to do it :
>>> 0- Backup my /home on some external disk.
>>> 1- backup / with something like :
>>> sudo tar cvpzf /backup.tgz --exclude=/media --exclude=/proc
>>> --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys
>>> --exclude=/home /
>>> mv /backup.tgz $1
>>>     
>> You'll need to do the tar in single user mode (init 1) to ensure there's
>> no open files when you're backing up (or use a bootable CD).
>>   
> I used this backup before to restore the system and it seemed to work 
> pretty well.
>
Okay - this'll depend on what you're running - databases in particular
can have major issues with backing up while open.

>> An easier option (assuming you want to keep the arrays as currently
>> setup) would be to boot from a CD and do a full copy of each of the
>> 500GB disks to the 1TB disks.  You'll then just need to resize the
>> last partition (assuming /home is set up on the last partition on the
>> disks anyway) and grow the array and filesystem.
>>
>> A _better_ option (if possible) would be to install all the drives in
>> the system concurrently, then you can boot from CD and create arrays on
>> the new drives and copy the data across.  You'll also need to update
>> mdadm.conf (and the initrd if you're using one) to indicate the new
>> array IDs.
>>
> I can not install more drives in the computer. It is a home theater 
> computer in a small case. I was expecting to be able to let the raid manage 
> the copies itself.
>
If you've got the ports (and power), you could just leave the case open
and the disks outside until you've finished the copy.

> If the / was on a RAID5, would it be able to boot with 2 disks ?
> If so, is it possible to convert my RAID0 to a RAID5 ?
> For example, I could boot on a CD, backup / onto /home, delete the RAID0 
> array and recreate it as RAID5, then restore the backup. Would this work ?
>
Yes, a RAID-5 will run fine with just two disks. You could certainly
boot from CD, backup, recreate the RAID, then restore - you'd probably
need to update the mdadm.conf file (and the initrd) to indicate the new
array UUID and details though.

You may also want to read through the GRUB section on
http://en.gentoo-wiki.com/wiki/RAID/Software to see how to install grub
onto all the drives (avoiding the issue of which drive is currently
being booted from).

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Upgrading a software RAID

[Maxime Boissonneault]

>> If the / was on a RAID5, would it be able to boot with 2 disks ?
>> If so, is it possible to convert my RAID0 to a RAID5 ?
>> For example, I could boot on a CD, backup / onto /home, delete the RAID0 
>> array and recreate it as RAID5, then restore the backup. Would this work ?
>>
>>     
> Yes, a RAID-5 will run fine with just two disks. You could certainly
> boot from CD, backup, recreate the RAID, then restore - you'd probably
> need to update the mdadm.conf file (and the initrd) to indicate the new
> array UUID and details though.
>
> You may also want to read through the GRUB section on
> http://en.gentoo-wiki.com/wiki/RAID/Software to see how to install grub
> onto all the drives (avoiding the issue of which drive is currently
> being booted from).
>
>   
Ok. This is the configuration I had :
/dev/md1 was a RAID0 composed of /dev/sda2, /dev/sdb2 and /dev/sdc2

I did this :
mdadm --stop /dev/md1
mdadm --zero-superblock /dev/sda2
mdadm --zero-superblock /dev/sdb2
mdadm --zero-superblock /dev/sdc2

and then this :
mdadm --create --verbose /dev/md1 --level=5 --raid-devices=3 /dev/sda2 
/dev/sdb2 /dev/sdc2

I used the rescue mode from an alternate CD (hardy 8.04.2) to format 
/dev/md1 as ext3. Then, I restored the backup and updated the 
/etc/mdadm/mdadm.conf file with the output of mdadm --detail --scan.

Now, GRUB loads (it was on /dev/md0 which is a RAID1 composed of sd*1), 
and it tries to load Ubuntu, but it just won't load.

You said that I would have to edit initrd... what is that and what do I 
have to change in it ?

Thanks,

Maxime Boissonneault

Re: Upgrading a software RAID

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 2696 bytes --]

On Sat May 30, 2009 at 02:11:52PM -0400, Maxime Boissonneault wrote:

> Ok. This is the configuration I had :
> /dev/md1 was a RAID0 composed of /dev/sda2, /dev/sdb2 and /dev/sdc2
>
> I did this :
> mdadm --stop /dev/md1
> mdadm --zero-superblock /dev/sda2
> mdadm --zero-superblock /dev/sdb2
> mdadm --zero-superblock /dev/sdc2
>
> and then this :
> mdadm --create --verbose /dev/md1 --level=5 --raid-devices=3 /dev/sda2 
> /dev/sdb2 /dev/sdc2
>
> I used the rescue mode from an alternate CD (hardy 8.04.2) to format 
> /dev/md1 as ext3. Then, I restored the backup and updated the 
> /etc/mdadm/mdadm.conf file with the output of mdadm --detail --scan.
>
> Now, GRUB loads (it was on /dev/md0 which is a RAID1 composed of sd*1), and 
> it tries to load Ubuntu, but it just won't load.
>
How far does it get with booting?  It should boot the kernel but fail to
find the root filesystem (if it's an initrd problem anyway).

> You said that I would have to edit initrd... what is that and what do I 
> have to change in it ?
>
The initrd is the ramdisk which the kernel boots to first, containing
the various modules and utility programs (like mdadm) needed to access
the main root filesytem.  A quick search suggests that you need to run
'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
chroot onto the new array first though - something like:

 - Boot off CD
 - Assemble the RAID arrays
 - Create a mount point for the new array (mkdir /mnt/newroot)
 - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
 - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
 - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
 - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
 - Chroot to the new array (chroot /mnt/newroot /bin/bash)
 - Mount the boot filesystem (mount /boot)
 - Update the initramfs (update-initramfs -u)
 - Unmount the boot filesystem (umount /boot)
 - Exit the chroot (exit)
 - Unmount the dev filesystem (umount /mnt/newroot/dev)
 - Unmount the sys filesystem (umount /mnt/newroot/sys)
 - Unmount the proc filesystem (umount /mnt/newroot/proc)
 - Unmount the root filesystem (umount /mnt/newroot)
 - Reboot

That's just from memory though, so watch out for any warnings/errors.
The other issue you _may_ have is the raid5 module missing from the
initrd - you'd best check that raid5 is listed in
/etc/initramfs/modules.

HTH,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Upgrading a software RAID

[Maxime Boissonneault]

Robin Hill a écrit :
> On Sat May 30, 2009 at 02:11:52PM -0400, Maxime Boissonneault wrote:
>
>   
>> Ok. This is the configuration I had :
>> /dev/md1 was a RAID0 composed of /dev/sda2, /dev/sdb2 and /dev/sdc2
>>
>> I did this :
>> mdadm --stop /dev/md1
>> mdadm --zero-superblock /dev/sda2
>> mdadm --zero-superblock /dev/sdb2
>> mdadm --zero-superblock /dev/sdc2
>>
>> and then this :
>> mdadm --create --verbose /dev/md1 --level=5 --raid-devices=3 /dev/sda2 
>> /dev/sdb2 /dev/sdc2
>>
>> I used the rescue mode from an alternate CD (hardy 8.04.2) to format 
>> /dev/md1 as ext3. Then, I restored the backup and updated the 
>> /etc/mdadm/mdadm.conf file with the output of mdadm --detail --scan.
>>
>> Now, GRUB loads (it was on /dev/md0 which is a RAID1 composed of sd*1), and 
>> it tries to load Ubuntu, but it just won't load.
>>
>>     
> How far does it get with booting?  It should boot the kernel but fail to
> find the root filesystem (if it's an initrd problem anyway).
>
>   
It says :
Loading, please wait...
Check root = bootarg cat /proc/cmdline or missing modules, devices : cat 
/proc/modules ls /dev
ALERT! /dev/md1 does not exist. Dropping to a shell!
BusyBox v1.1.3 (Debian 1:1.1.3-5ubuntu12) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

>> You said that I would have to edit initrd... what is that and what do I 
>> have to change in it ?
>>
>>     
> The initrd is the ramdisk which the kernel boots to first, containing
> the various modules and utility programs (like mdadm) needed to access
> the main root filesytem.  A quick search suggests that you need to run
> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
> chroot onto the new array first though - something like:
>
>  - Boot off CD
>  - Assemble the RAID arrays
>  - Create a mount point for the new array (mkdir /mnt/newroot)
>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>  - Mount the boot filesystem (mount /boot)
>  - Update the initramfs (update-initramfs -u)
>  - Unmount the boot filesystem (umount /boot)
>  - Exit the chroot (exit)
>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>  - Unmount the root filesystem (umount /mnt/newroot)
>  - Reboot
>
> That's just from memory though, so watch out for any warnings/errors.
> The other issue you _may_ have is the raid5 module missing from the
> initrd - you'd best check that raid5 is listed in
> /etc/initramfs/modules.
>
> HTH,
>     Robin
>   

RAID5 should not be missing, my /home was a RAID5. The alternate CD 
explicitely allows me to mount with a drive (say /dev/md1). Could I just 
do this and update-initramfs -u, then reboot ?
Why would I have to mount all of these filesystems ? The only thing I 
cared about before was /boot -> /dev/md0, / -> /dev/md1 and /home -> 
/dev/md2.

Thanks

Maxime

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 3603 bytes --]

On Sat May 30, 2009 at 04:03:04PM -0400, Maxime Boissonneault wrote:

>
> Robin Hill a écrit :
>> On Sat May 30, 2009 at 02:11:52PM -0400, Maxime Boissonneault wrote:
>>
>>> Now, GRUB loads (it was on /dev/md0 which is a RAID1 composed of sd*1), 
>>> and it tries to load Ubuntu, but it just won't load.
>>>     
>> How far does it get with booting?  It should boot the kernel but fail to
>> find the root filesystem (if it's an initrd problem anyway).
>>   
> It says :
> Loading, please wait...
> Check root = bootarg cat /proc/cmdline or missing modules, devices : cat 
> /proc/modules ls /dev
> ALERT! /dev/md1 does not exist. Dropping to a shell!
> BusyBox v1.1.3 (Debian 1:1.1.3-5ubuntu12) Built-in shell (ash)
> Enter 'help' for a list of built-in commands.
>
> (initramfs)
>
Okay, that's what I'd expect.

>>> You said that I would have to edit initrd... what is that and what do I 
>>> have to change in it ?
>>>     
>> The initrd is the ramdisk which the kernel boots to first, containing
>> the various modules and utility programs (like mdadm) needed to access
>> the main root filesytem.  A quick search suggests that you need to run
>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
>> chroot onto the new array first though - something like:
>>
>>  - Boot off CD
>>  - Assemble the RAID arrays
>>  - Create a mount point for the new array (mkdir /mnt/newroot)
>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>>  - Mount the boot filesystem (mount /boot)
>>  - Update the initramfs (update-initramfs -u)
>>  - Unmount the boot filesystem (umount /boot)
>>  - Exit the chroot (exit)
>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>>  - Unmount the root filesystem (umount /mnt/newroot)
>>  - Reboot
>>
>> That's just from memory though, so watch out for any warnings/errors.
>> The other issue you _may_ have is the raid5 module missing from the
>> initrd - you'd best check that raid5 is listed in
>> /etc/initramfs/modules.
>
> RAID5 should not be missing, my /home was a RAID5.
Yes, but /home is only mounted after it's accessed the root partition.
It can then read any modules from the root partition, so the raid5
module could still be missing from the initrd.  I suspect it'll have
included it though.

> The alternate CD explicitly allows me to mount with a drive (say
> /dev/md1). Could I just do this and update-initramfs -u, then reboot?
You should be able to, yes.  Worth a try anyway - it's certainly the
easiest option.

> Why would I have to mount all of these filesystems ? The only thing I cared 
> about before was /boot -> /dev/md0, / -> /dev/md1 and /home -> /dev/md2.
>
All these filesystems are standard, so will be taken care of
automatically under normal conditions.  When you're doing a chroot,
you'll need to take care of them manually though (otherwise most
commands will fail to run under the chroot).

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Upgrading a software RAID

[Maxime Boissonneault]

>>> The initrd is the ramdisk which the kernel boots to first, containing
>>> the various modules and utility programs (like mdadm) needed to access
>>> the main root filesytem.  A quick search suggests that you need to run
>>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
>>> chroot onto the new array first though - something like:
>>>
>>>  - Boot off CD
>>>  - Assemble the RAID arrays
>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>>>  - Mount the boot filesystem (mount /boot)
>>>  - Update the initramfs (update-initramfs -u)
>>>  - Unmount the boot filesystem (umount /boot)
>>>  - Exit the chroot (exit)
>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>>>  - Unmount the root filesystem (umount /mnt/newroot)
>>>  - Reboot
>>>
>>> That's just from memory though, so watch out for any warnings/errors.
>>> The other issue you _may_ have is the raid5 module missing from the
>>> initrd - you'd best check that raid5 is listed in
>>> /etc/initramfs/modules.
>>>       
>> RAID5 should not be missing, my /home was a RAID5.
>>     
> Yes, but /home is only mounted after it's accessed the root partition.
> It can then read any modules from the root partition, so the raid5
> module could still be missing from the initrd.  I suspect it'll have
> included it though.
>
>   
I did as you suggested above. I still get the same error message.
I have looked and did not find /etc/initramfs/modules. Instead, I found 
/etc/initramfs-tools/modules. In this file, there were a couple of 
comment lines explaining how to enable modules.
No modules were included.
I uncommented the "raid1" and added "raid5" and ran the procedure again.

Still getting the same problem.

Since the problem seems to be the boot, I will try to do a fresh 
install, then restore my backup while keeping the /boot from the fresh 
install.

Thanks,

Maxime

Re: Upgrading a software RAID

[Maxime Boissonneault]

Maxime Boissonneault a écrit :
>
>>>> The initrd is the ramdisk which the kernel boots to first, containing
>>>> the various modules and utility programs (like mdadm) needed to access
>>>> the main root filesytem.  A quick search suggests that you need to run
>>>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
>>>> chroot onto the new array first though - something like:
>>>>
>>>>  - Boot off CD
>>>>  - Assemble the RAID arrays
>>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
>>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>>>>  - Mount the boot filesystem (mount /boot)
>>>>  - Update the initramfs (update-initramfs -u)
>>>>  - Unmount the boot filesystem (umount /boot)
>>>>  - Exit the chroot (exit)
>>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>>>>  - Unmount the root filesystem (umount /mnt/newroot)
>>>>  - Reboot
>>>>
>>>> That's just from memory though, so watch out for any warnings/errors.
>>>> The other issue you _may_ have is the raid5 module missing from the
>>>> initrd - you'd best check that raid5 is listed in
>>>> /etc/initramfs/modules.
>>>>       
>>> RAID5 should not be missing, my /home was a RAID5.
>>>     
>> Yes, but /home is only mounted after it's accessed the root partition.
>> It can then read any modules from the root partition, so the raid5
>> module could still be missing from the initrd.  I suspect it'll have
>> included it though.
>>
>>   
> I did as you suggested above. I still get the same error message.
> I have looked and did not find /etc/initramfs/modules. Instead, I 
> found /etc/initramfs-tools/modules. In this file, there were a couple 
> of comment lines explaining how to enable modules.
> No modules were included.
> I uncommented the "raid1" and added "raid5" and ran the procedure again.
>
> Still getting the same problem.
>
> Since the problem seems to be the boot, I will try to do a fresh 
> install, then restore my backup while keeping the /boot from the fresh 
> install.
>
> Thanks,
>
> Maxime
>
Ok, I just did that, and I get the same problem. However, now, if I 
press ctrl+alt+del while the system is trying to load, it tells me that 
process rc5 and rc6 have been killed, and it bring me up to the login 
screen of ubuntu.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Bill Davidsen]

Maxime Boissonneault wrote:
>
>>>> The initrd is the ramdisk which the kernel boots to first, containing
>>>> the various modules and utility programs (like mdadm) needed to access
>>>> the main root filesytem.  A quick search suggests that you need to run
>>>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
>>>> chroot onto the new array first though - something like:
>>>>
>>>>  - Boot off CD
>>>>  - Assemble the RAID arrays
>>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
>>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>>>>  - Mount the boot filesystem (mount /boot)
>>>>  - Update the initramfs (update-initramfs -u)
>>>>  - Unmount the boot filesystem (umount /boot)
>>>>  - Exit the chroot (exit)
>>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>>>>  - Unmount the root filesystem (umount /mnt/newroot)
>>>>  - Reboot
>>>>
>>>> That's just from memory though, so watch out for any warnings/errors.
>>>> The other issue you _may_ have is the raid5 module missing from the
>>>> initrd - you'd best check that raid5 is listed in
>>>> /etc/initramfs/modules.
>>>>       
>>> RAID5 should not be missing, my /home was a RAID5.
>>>     
>> Yes, but /home is only mounted after it's accessed the root partition.
>> It can then read any modules from the root partition, so the raid5
>> module could still be missing from the initrd.  I suspect it'll have
>> included it though.
>>
>>   
> I did as you suggested above. I still get the same error message.
> I have looked and did not find /etc/initramfs/modules. Instead, I 
> found /etc/initramfs-tools/modules. In this file, there were a couple 
> of comment lines explaining how to enable modules.
> No modules were included.
> I uncommented the "raid1" and added "raid5" and ran the procedure again.
>
> Still getting the same problem.
>
> Since the problem seems to be the boot, I will try to do a fresh 
> install, then restore my backup while keeping the /boot from the fresh 
> install.

If this were Fedora I would be using mkinitrd to get the right modules 
in the boot image, however Ubuntu requires another method, which I don't 
know well enough to describe. You are probably on the right tract 
though, getting the modules and mdadm.conf available at boot.

-- 
Bill Davidsen <davidsen@tmr.com>
  Even technical things can appear to be magic, if the documentation is
obscure enough. For example, PulseAudio is configured by dancing naked around
a fire at midnight, shaking a rattle with one hand and a LISP manual with the
other, while reciting the GNU manifesto in hexadecimal.

Re: Upgrading a software RAID

[Thomas Fjellstrom]

On Tue June 2 2009, Bill Davidsen wrote:
> Maxime Boissonneault wrote:
> >>>> The initrd is the ramdisk which the kernel boots to first, containing
> >>>> the various modules and utility programs (like mdadm) needed to access
> >>>> the main root filesytem.  A quick search suggests that you need to run
> >>>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
> >>>> chroot onto the new array first though - something like:
> >>>>
> >>>>  - Boot off CD
> >>>>  - Assemble the RAID arrays
> >>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
> >>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
> >>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
> >>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
> >>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
> >>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
> >>>>  - Mount the boot filesystem (mount /boot)
> >>>>  - Update the initramfs (update-initramfs -u)
> >>>>  - Unmount the boot filesystem (umount /boot)
> >>>>  - Exit the chroot (exit)
> >>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
> >>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
> >>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
> >>>>  - Unmount the root filesystem (umount /mnt/newroot)
> >>>>  - Reboot
> >>>>
> >>>> That's just from memory though, so watch out for any warnings/errors.
> >>>> The other issue you _may_ have is the raid5 module missing from the
> >>>> initrd - you'd best check that raid5 is listed in
> >>>> /etc/initramfs/modules.
> >>>
> >>> RAID5 should not be missing, my /home was a RAID5.
> >>
> >> Yes, but /home is only mounted after it's accessed the root partition.
> >> It can then read any modules from the root partition, so the raid5
> >> module could still be missing from the initrd.  I suspect it'll have
> >> included it though.
> >
> > I did as you suggested above. I still get the same error message.
> > I have looked and did not find /etc/initramfs/modules. Instead, I
> > found /etc/initramfs-tools/modules. In this file, there were a couple
> > of comment lines explaining how to enable modules.
> > No modules were included.
> > I uncommented the "raid1" and added "raid5" and ran the procedure again.
> >
> > Still getting the same problem.
> >
> > Since the problem seems to be the boot, I will try to do a fresh
> > install, then restore my backup while keeping the /boot from the fresh
> > install.
>
> If this were Fedora I would be using mkinitrd to get the right modules
> in the boot image, however Ubuntu requires another method, which I don't
> know well enough to describe. You are probably on the right tract
> though, getting the modules and mdadm.conf available at boot.

I assume its more or less like debian, so you just add the modules to 
/etc/initramfs-tools/modules and run: update-initramfs -uk `uname -r`

then the modules you added to the modules file will be added to the initramfs 
file, and loaded into the kernel at that stage.


-- 
Thomas Fjellstrom
tfjellstrom@shaw.ca

Re: Upgrading a software RAID

[Bill Davidsen]

Thomas Fjellstrom wrote:
> On Tue June 2 2009, Bill Davidsen wrote:
>   
>> Maxime Boissonneault wrote:
>>     
>>>>>> The initrd is the ramdisk which the kernel boots to first, containing
>>>>>> the various modules and utility programs (like mdadm) needed to access
>>>>>> the main root filesytem.  A quick search suggests that you need to run
>>>>>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
>>>>>> chroot onto the new array first though - something like:
>>>>>>
>>>>>>  - Boot off CD
>>>>>>  - Assemble the RAID arrays
>>>>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
>>>>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>>>>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>>>>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>>>>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>>>>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>>>>>>  - Mount the boot filesystem (mount /boot)
>>>>>>  - Update the initramfs (update-initramfs -u)
>>>>>>  - Unmount the boot filesystem (umount /boot)
>>>>>>  - Exit the chroot (exit)
>>>>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>>>>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>>>>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>>>>>>  - Unmount the root filesystem (umount /mnt/newroot)
>>>>>>  - Reboot
>>>>>>
>>>>>> That's just from memory though, so watch out for any warnings/errors.
>>>>>> The other issue you _may_ have is the raid5 module missing from the
>>>>>> initrd - you'd best check that raid5 is listed in
>>>>>> /etc/initramfs/modules.
>>>>>>             
>>>>> RAID5 should not be missing, my /home was a RAID5.
>>>>>           
>>>> Yes, but /home is only mounted after it's accessed the root partition.
>>>> It can then read any modules from the root partition, so the raid5
>>>> module could still be missing from the initrd.  I suspect it'll have
>>>> included it though.
>>>>         
>>> I did as you suggested above. I still get the same error message.
>>> I have looked and did not find /etc/initramfs/modules. Instead, I
>>> found /etc/initramfs-tools/modules. In this file, there were a couple
>>> of comment lines explaining how to enable modules.
>>> No modules were included.
>>> I uncommented the "raid1" and added "raid5" and ran the procedure again.
>>>
>>> Still getting the same problem.
>>>
>>> Since the problem seems to be the boot, I will try to do a fresh
>>> install, then restore my backup while keeping the /boot from the fresh
>>> install.
>>>       
>> If this were Fedora I would be using mkinitrd to get the right modules
>> in the boot image, however Ubuntu requires another method, which I don't
>> know well enough to describe. You are probably on the right tract
>> though, getting the modules and mdadm.conf available at boot.
>>     
>
> I assume its more or less like debian, so you just add the modules to 
> /etc/initramfs-tools/modules and run: update-initramfs -uk `uname -r`
>
> then the modules you added to the modules file will be added to the initramfs 
> file, and loaded into the kernel at that stage.
>
>
>   
Is there some explicit action needed to get the mdadm.conf file added to 
the image?

-- 
Bill Davidsen <davidsen@tmr.com>
  Even purely technical things can appear to be magic, if the documentation is
obscure enough. For example, PulseAudio is configured by dancing naked around a
fire at midnight, shaking a rattle with one hand and a LISP manual with the
other, while reciting the GNU manifesto in hexadecimal. The documentation fails
to note that you must circle the fire counter-clockwise in the southern
hemisphere.

Re: Upgrading a software RAID

[Thomas Fjellstrom]

On Tue June 2 2009, Bill Davidsen wrote:
> Thomas Fjellstrom wrote:
> > On Tue June 2 2009, Bill Davidsen wrote:
> >> Maxime Boissonneault wrote:
> >>>>>> The initrd is the ramdisk which the kernel boots to first,
> >>>>>> containing the various modules and utility programs (like mdadm)
> >>>>>> needed to access the main root filesytem.  A quick search suggests
> >>>>>> that you need to run 'update-initramfs -u' on Ubuntu.  You'll need
> >>>>>> to boot off the CD and chroot onto the new array first though -
> >>>>>> something like:
> >>>>>>
> >>>>>>  - Boot off CD
> >>>>>>  - Assemble the RAID arrays
> >>>>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
> >>>>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
> >>>>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
> >>>>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
> >>>>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
> >>>>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
> >>>>>>  - Mount the boot filesystem (mount /boot)
> >>>>>>  - Update the initramfs (update-initramfs -u)
> >>>>>>  - Unmount the boot filesystem (umount /boot)
> >>>>>>  - Exit the chroot (exit)
> >>>>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
> >>>>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
> >>>>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
> >>>>>>  - Unmount the root filesystem (umount /mnt/newroot)
> >>>>>>  - Reboot
> >>>>>>
> >>>>>> That's just from memory though, so watch out for any
> >>>>>> warnings/errors. The other issue you _may_ have is the raid5 module
> >>>>>> missing from the initrd - you'd best check that raid5 is listed in
> >>>>>> /etc/initramfs/modules.
> >>>>>
> >>>>> RAID5 should not be missing, my /home was a RAID5.
> >>>>
> >>>> Yes, but /home is only mounted after it's accessed the root partition.
> >>>> It can then read any modules from the root partition, so the raid5
> >>>> module could still be missing from the initrd.  I suspect it'll have
> >>>> included it though.
> >>>
> >>> I did as you suggested above. I still get the same error message.
> >>> I have looked and did not find /etc/initramfs/modules. Instead, I
> >>> found /etc/initramfs-tools/modules. In this file, there were a couple
> >>> of comment lines explaining how to enable modules.
> >>> No modules were included.
> >>> I uncommented the "raid1" and added "raid5" and ran the procedure
> >>> again.
> >>>
> >>> Still getting the same problem.
> >>>
> >>> Since the problem seems to be the boot, I will try to do a fresh
> >>> install, then restore my backup while keeping the /boot from the fresh
> >>> install.
> >>
> >> If this were Fedora I would be using mkinitrd to get the right modules
> >> in the boot image, however Ubuntu requires another method, which I don't
> >> know well enough to describe. You are probably on the right tract
> >> though, getting the modules and mdadm.conf available at boot.
> >
> > I assume its more or less like debian, so you just add the modules to
> > /etc/initramfs-tools/modules and run: update-initramfs -uk `uname -r`
> >
> > then the modules you added to the modules file will be added to the
> > initramfs file, and loaded into the kernel at that stage.
>
> Is there some explicit action needed to get the mdadm.conf file added to
> the image?

I'm not entirely sure. But automatic arrays will build fine. set the partition 
types to "fd" and make sure the array saved the metadata to the disks, and the 
kernel should bring the arrays up automatically.

-- 
Thomas Fjellstrom
tfjellstrom@shaw.ca

Re: Upgrading a software RAID

[CoolCold]

debian's update-initramfs includes /etc/mdadm/mdadm.conf automatically
on image (re)build

On Tue, Jun 2, 2009 at 11:57 PM, Bill Davidsen <davidsen@tmr.com> wrote:
> Thomas Fjellstrom wrote:
>>
>> On Tue June 2 2009, Bill Davidsen wrote:
>>
>>>
>>> Maxime Boissonneault wrote:
>>>
>>>>>>>
>>>>>>> The initrd is the ramdisk which the kernel boots to first, containing
>>>>>>> the various modules and utility programs (like mdadm) needed to
>>>>>>> access
>>>>>>> the main root filesytem.  A quick search suggests that you need to
>>>>>>> run
>>>>>>> 'update-initramfs -u' on Ubuntu.  You'll need to boot off the CD and
>>>>>>> chroot onto the new array first though - something like:
>>>>>>>
>>>>>>>  - Boot off CD
>>>>>>>  - Assemble the RAID arrays
>>>>>>>  - Create a mount point for the new array (mkdir /mnt/newroot)
>>>>>>>  - Mount the root filesystem (mount /dev/md1 /mnt/newroot)
>>>>>>>  - Mount the proc filesystem (mount -o bind /proc /mnt/newroot/proc)
>>>>>>>  - Mount the sys filesystem (mount -o bind /sys /mnt/newroot/sys)
>>>>>>>  - Mount the dev filesystem (mount -o bind /dev /mnt/newroot/dev)
>>>>>>>  - Chroot to the new array (chroot /mnt/newroot /bin/bash)
>>>>>>>  - Mount the boot filesystem (mount /boot)
>>>>>>>  - Update the initramfs (update-initramfs -u)
>>>>>>>  - Unmount the boot filesystem (umount /boot)
>>>>>>>  - Exit the chroot (exit)
>>>>>>>  - Unmount the dev filesystem (umount /mnt/newroot/dev)
>>>>>>>  - Unmount the sys filesystem (umount /mnt/newroot/sys)
>>>>>>>  - Unmount the proc filesystem (umount /mnt/newroot/proc)
>>>>>>>  - Unmount the root filesystem (umount /mnt/newroot)
>>>>>>>  - Reboot
>>>>>>>
>>>>>>> That's just from memory though, so watch out for any warnings/errors.
>>>>>>> The other issue you _may_ have is the raid5 module missing from the
>>>>>>> initrd - you'd best check that raid5 is listed in
>>>>>>> /etc/initramfs/modules.
>>>>>>>
>>>>>>
>>>>>> RAID5 should not be missing, my /home was a RAID5.
>>>>>>
>>>>>
>>>>> Yes, but /home is only mounted after it's accessed the root partition.
>>>>> It can then read any modules from the root partition, so the raid5
>>>>> module could still be missing from the initrd.  I suspect it'll have
>>>>> included it though.
>>>>>
>>>>
>>>> I did as you suggested above. I still get the same error message.
>>>> I have looked and did not find /etc/initramfs/modules. Instead, I
>>>> found /etc/initramfs-tools/modules. In this file, there were a couple
>>>> of comment lines explaining how to enable modules.
>>>> No modules were included.
>>>> I uncommented the "raid1" and added "raid5" and ran the procedure again.
>>>>
>>>> Still getting the same problem.
>>>>
>>>> Since the problem seems to be the boot, I will try to do a fresh
>>>> install, then restore my backup while keeping the /boot from the fresh
>>>> install.
>>>>
>>>
>>> If this were Fedora I would be using mkinitrd to get the right modules
>>> in the boot image, however Ubuntu requires another method, which I don't
>>> know well enough to describe. You are probably on the right tract
>>> though, getting the modules and mdadm.conf available at boot.
>>>
>>
>> I assume its more or less like debian, so you just add the modules to
>> /etc/initramfs-tools/modules and run: update-initramfs -uk `uname -r`
>>
>> then the modules you added to the modules file will be added to the
>> initramfs file, and loaded into the kernel at that stage.
>>
>>
>>
>
> Is there some explicit action needed to get the mdadm.conf file added to the
> image?
>
> --
> Bill Davidsen <davidsen@tmr.com>
>  Even purely technical things can appear to be magic, if the documentation
> is
> obscure enough. For example, PulseAudio is configured by dancing naked
> around a
> fire at midnight, shaking a rattle with one hand and a LISP manual with the
> other, while reciting the GNU manifesto in hexadecimal. The documentation
> fails
> to note that you must circle the fire counter-clockwise in the southern
> hemisphere.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
-- 
Best regards,
[COOLCOLD-RIPN]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Bill Davidsen]

CoolCold wrote:
> debian's update-initramfs includes /etc/mdadm/mdadm.conf automatically
> on image (re)build
>   

Thanks, wasn't sure if that was the case, thought I'd mention it for 
completeness.

-- 
Bill Davidsen <davidsen@tmr.com>
  Even purely technical things can appear to be magic, if the documentation is
obscure enough. For example, PulseAudio is configured by dancing naked around a
fire at midnight, shaking a rattle with one hand and a LISP manual with the
other, while reciting the GNU manifesto in hexadecimal. The documentation fails
to note that you must circle the fire counter-clockwise in the southern
hemisphere.

Re: Upgrading a software RAID

[Keld Jørn Simonsen]

On Thu, May 28, 2009 at 10:32:53AM -0400, Maxime Boissonneault wrote:
>
> Robin Hill a écrit :
>> On Mon May 25, 2009 at 01:05:15PM -0400, Maxime Boissonneault wrote:
>>
>>   
>>> Hello,
>>> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
>>> configuration :
>>>
>>> /boot is on a 100MB RAID1
>>> / is on a 30GB RAID0
>>> /home is on a 906GB RAID5
>>>
>>>     
>> Ouch - why RAID0 for /?  If you lose a single drive then all the
>> configuration, etc. is down the drain.  I'd suggest rethinking this
>> while you're going through the rebuild process anyway.
>>
>>   
> That is why I do backups of / on /home regularily. I first did a RAID0  
> for performance. I did some testing on performance of RAID0,1,5 and  
> RAID1 was pretty lousy (I put the results of my tests here :  
> http://cqed.physique.usherbrooke.ca/~mboisson/htpc.php?sec=raid_test )
> I guess I should have used a RAID5, but RAID1 seems like a terrible idea  
> considering the really bad performances.

Hmm, why not use raid10,f2 for the / system?
Then you will get almost raid0 read performance, while write performance
is about half of raid0.

A setup is described in
http://linux-raid.osdl.org/index.php/Preventing_against_a_failing_disk

You can do the / system with raid10,f2 with just 2 disks.

Best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Bill Davidsen]

Maxime Boissonneault wrote:
>
> Robin Hill a écrit :
>> On Mon May 25, 2009 at 01:05:15PM -0400, Maxime Boissonneault wrote:
>>
>>  
>>> Hello,
>>> I am using Ubuntu Hardy with 3x500GB drives and the following RAID
>>> configuration :
>>>
>>> /boot is on a 100MB RAID1
>>> / is on a 30GB RAID0
>>> /home is on a 906GB RAID5
>>>
>>>     
>> Ouch - why RAID0 for /?  If you lose a single drive then all the
>> configuration, etc. is down the drain.  I'd suggest rethinking this
>> while you're going through the rebuild process anyway.
>>
>>   
> That is why I do backups of / on /home regularily. I first did a RAID0 
> for performance. I did some testing on performance of RAID0,1,5 and 
> RAID1 was pretty lousy (I put the results of my tests here : 
> http://cqed.physique.usherbrooke.ca/~mboisson/htpc.php?sec=raid_test )
> I guess I should have used a RAID5, but RAID1 seems like a terrible 
> idea considering the really bad performances.
>
>>> I want to replace the 3 drives by 3 1TB drives.
>>>
>>> Here is how I planned to do it :
>>> 0- Backup my /home on some external disk.
>>> 1- backup / with something like :
>>> sudo tar cvpzf /backup.tgz --exclude=/media --exclude=/proc
>>> --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt 
>>> --exclude=/sys
>>> --exclude=/home /
>>> mv /backup.tgz $1
>>>     
>> You'll need to do the tar in single user mode (init 1) to ensure there's
>> no open files when you're backing up (or use a bootable CD).
>>
>>   
> I used this backup before to restore the system and it seemed to work 
> pretty well.
>>> 2- Replace 1 disk
>>> 3- Boot and let the RAID1 and RAID5 reconstruct
>>>     
>> You'll have to boot to a CD then - your root FS is trashed at this
>> point.
>>
>>  
>>> 4- Replace 1 other disk
>>> 5- Boot and let the RAID1 and RAID5 reconstruct again
>>> 6- Replace the last disk
>>> 7- Boot and let the RAID1 and RAID5 reconstruct one last time
>>> 8- Boot and restore the backup on the RAID0 / partition.
>>> 9- Resize the /home partition to 1 TB.
>>>
>>> I suspect there will be a problem replacing the primary disk, but I 
>>> guessed
>>> that I could solve this simply by changing which is the primary disk 
>>> in the
>>> BIOS.
>>>
>>> Is there any other problem that will or could happen ?
>>>
>>>     
>> See above - your current plan is pretty much doomed I'm afraid.
>>
>>  
>>> For example, I am not sure if the raid manager is on the /boot 
>>> partition or
>>> on the /. I guess if it is on /, it won't work at all since the raid 
>>> manager
>>> itself won't be able to run ?
>>> Also, is it possible to boot and access a command line to restore 
>>> the backup
>>> with a failed / partition ?
>>>
>>> I am also unsure about how I should proceed to resize the /home 
>>> partition.
>>> Is this done through mdadm ?
>>>
>>>     
>> No, you'll have to delete & recreate the partition using fdisk.
>>
>>  
>>> Please enlight me on any problems that I will have.
>>>
>>>     
>>
>> An easier option (assuming you want to keep the arrays as currently
>> setup) would be to boot from a CD and do a full copy of each of the
>> 500GB disks to the 1TB disks.  You'll then just need to resize the
>> last partition (assuming /home is set up on the last partition on the
>> disks anyway) and grow the array and filesystem.
>>
>> A _better_ option (if possible) would be to install all the drives in
>> the system concurrently, then you can boot from CD and create arrays on
>> the new drives and copy the data across.  You'll also need to update
>> mdadm.conf (and the initrd if you're using one) to indicate the new
>> array IDs.
>>
>> HTH,
>>     Robin
>>   
>
> I can not install more drives in the computer. It is a home theater 
> computer in a small case. I was expecting to be able to let the raid 
> manage the copies itself.
>
> If the / was on a RAID5, would it be able to boot with 2 disks ?
> If so, is it possible to convert my RAID0 to a RAID5 ?
> For example, I could boot on a CD, backup / onto /home, delete the 
> RAID0 array and recreate it as RAID5, then restore the backup. Would 
> this work ?

Based on my testing (somewhat old now) and regular use, I would say 
raid10 is probably your best bet. It's fast and secure, and with the -f2 
option for "far" copies it's able to give high transfer rates.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Upgrading a software RAID

[Maxime Boissonneault]

>>
>> I can not install more drives in the computer. It is a home theater 
>> computer in a small case. I was expecting to be able to let the raid 
>> manage the copies itself.
>>
>> If the / was on a RAID5, would it be able to boot with 2 disks ?
>> If so, is it possible to convert my RAID0 to a RAID5 ?
>> For example, I could boot on a CD, backup / onto /home, delete the 
>> RAID0 array and recreate it as RAID5, then restore the backup. Would 
>> this work ?
>
> Based on my testing (somewhat old now) and regular use, I would say 
> raid10 is probably your best bet. It's fast and secure, and with the 
> -f2 option for "far" copies it's able to give high transfer rates.

Doesn't RAID10 means RAID 1+0, which requires 4 disks ?

Re: Upgrading a software RAID

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]

On Sat May 30, 2009 at 02:35:24PM -0400, Maxime Boissonneault wrote:

>
>>>
>>> I can not install more drives in the computer. It is a home theater 
>>> computer in a small case. I was expecting to be able to let the raid 
>>> manage the copies itself.
>>>
>>> If the / was on a RAID5, would it be able to boot with 2 disks ?
>>> If so, is it possible to convert my RAID0 to a RAID5 ?
>>> For example, I could boot on a CD, backup / onto /home, delete the RAID0 
>>> array and recreate it as RAID5, then restore the backup. Would this work 
>>> ?
>>
>> Based on my testing (somewhat old now) and regular use, I would say raid10 
>> is probably your best bet. It's fast and secure, and with the -f2 option 
>> for "far" copies it's able to give high transfer rates.
>
> Doesn't RAID10 means RAID 1+0, which requires 4 disks ?
>
Yes and no!  Linux software (md) raid implements RAID10 as a single
layer (rather than layering RAID0 and RAID1), allowing the number of
drives and the number of replicas to be pretty arbitrary.  The basic
principle holds though - you'll still have 2 (or more if specified)
replicas of each data block, and blocks are striped across the drives.

You can read the details on the md manual page.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Upgrading a software RAID

[Bill Davidsen]

Maxime Boissonneault wrote:
>
>>>
>>> I can not install more drives in the computer. It is a home theater 
>>> computer in a small case. I was expecting to be able to let the raid 
>>> manage the copies itself.
>>>
>>> If the / was on a RAID5, would it be able to boot with 2 disks ?
>>> If so, is it possible to convert my RAID0 to a RAID5 ?
>>> For example, I could boot on a CD, backup / onto /home, delete the 
>>> RAID0 array and recreate it as RAID5, then restore the backup. Would 
>>> this work ?
>>
>> Based on my testing (somewhat old now) and regular use, I would say 
>> raid10 is probably your best bet. It's fast and secure, and with the 
>> -f2 option for "far" copies it's able to give high transfer rates.
>
> Doesn't RAID10 means RAID 1+0, which requires 4 disks ?
>
No, they are not the same thing, see the mdadm man page for more 
information. The md module does the whole thing for you, you just select 
the layout, and from my experience and discussions here the "-f2" (far, 
two copies) seems the fastest for read and acceptable fast for write.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.

Re: Upgrading a software RAID

[Keld Jørn Simonsen]

On Sat, May 30, 2009 at 02:35:24PM -0400, Maxime Boissonneault wrote:
>
>>>
>>> I can not install more drives in the computer. It is a home theater  
>>> computer in a small case. I was expecting to be able to let the raid  
>>> manage the copies itself.
>>>
>>> If the / was on a RAID5, would it be able to boot with 2 disks ?
>>> If so, is it possible to convert my RAID0 to a RAID5 ?
>>> For example, I could boot on a CD, backup / onto /home, delete the  
>>> RAID0 array and recreate it as RAID5, then restore the backup. Would  
>>> this work ?
>>
>> Based on my testing (somewhat old now) and regular use, I would say  
>> raid10 is probably your best bet. It's fast and secure, and with the  
>> -f2 option for "far" copies it's able to give high transfer rates.
>
> Doesn't RAID10 means RAID 1+0, which requires 4 disks ?

Linux RAID10 is somewhat different from raid1+0, and can run with just 2
disks, with fine performance for reads in the F2 layout, almost raid0
performance, while write performance is close to raid1, with a file system in operation.

Best regards
keld

Re: Upgrading a software RAID

[Maxime Boissonneault]

I finally succeeded in getting my system back. To remind you, I started 
with this configuration :

/proc/mdstat :

md2 : active raid5 sda3[0] sdc3[2] sdb3[1]
      957040000 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
     
md1 : active raid0 sda2[0] sdc2[2] sdb2[1]
      19534848 blocks level 0 [3/3] [UUU]
     
md0 : active raid1 sda1[0] sdc1[2] sdb1[1]
      96256 blocks [3/3] [UUU]

mount :
/dev/md1 on / type ext3 (rw,relatime,errors=remount-ro)
/dev/md0 on /boot type ext3 (rw,relatime)
/dev/md2 on /home type ext3 (rw,relatime)

And I wanted to change md1 to a RAID5. So, here is what I did and what 
worked.
1- Booted with a LiveCD (Ubuntu 8.04.2 Alternate CD)
2- Backed-up / using tar, put the file on /home
3- Destroyed the array, with mdadm --zero-superblock /dev/sd*2 and mdadm 
--stop /dev/md1
4- Created the new RAID5 array
5- Installed a fresh Ubuntu with the install disc
6- Copied the /boot, /etc/mdadm/mdadm.conf and /etc/fstab from the fresh 
install
7- Restored the backup over the fresh install
8- Restore the files copied at step 6
9- Booted

At this point, it gave me an error saying that the filesystem could not 
be mounted with some number. I found this web page
http://linux.byexamples.com/archives/321/fstab-with-uuid/
which told me how to get the uid for my disks, and I updated the 
/etc/fstab consequently.

Tadam! It works! I can now plan and buy my new 1TB HD and let the RAID 
reconstruct the files, then grow the arrays.

I wrote this email in the hope that it might be usefull to someone someday.


Thanks everyone for your help throughout the day.


Maxime Boissonneault

Re: Upgrading a software RAID

[Bill Davidsen]

Maxime Boissonneault wrote:
> I finally succeeded in getting my system back. To remind you, I 
> started with this configuration :
>
> /proc/mdstat :
>
> md2 : active raid5 sda3[0] sdc3[2] sdb3[1]
>      957040000 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
>     md1 : active raid0 sda2[0] sdc2[2] sdb2[1]
>      19534848 blocks level 0 [3/3] [UUU]
>     md0 : active raid1 sda1[0] sdc1[2] sdb1[1]
>      96256 blocks [3/3] [UUU]
>
> mount :
> /dev/md1 on / type ext3 (rw,relatime,errors=remount-ro)
> /dev/md0 on /boot type ext3 (rw,relatime)
> /dev/md2 on /home type ext3 (rw,relatime)
>
> And I wanted to change md1 to a RAID5. So, here is what I did and what 
> worked.
> 1- Booted with a LiveCD (Ubuntu 8.04.2 Alternate CD)
> 2- Backed-up / using tar, put the file on /home
> 3- Destroyed the array, with mdadm --zero-superblock /dev/sd*2 and 
> mdadm --stop /dev/md1
> 4- Created the new RAID5 array
> 5- Installed a fresh Ubuntu with the install disc
> 6- Copied the /boot, /etc/mdadm/mdadm.conf and /etc/fstab from the 
> fresh install
> 7- Restored the backup over the fresh install
> 8- Restore the files copied at step 6
> 9- Booted
>
> At this point, it gave me an error saying that the filesystem could 
> not be mounted with some number. I found this web page
> http://linux.byexamples.com/archives/321/fstab-with-uuid/
> which told me how to get the uid for my disks, and I updated the 
> /etc/fstab consequently.
>
> Tadam! It works! I can now plan and buy my new 1TB HD and let the RAID 
> reconstruct the files, then grow the arrays.
>
> I wrote this email in the hope that it might be usefull to someone 
> someday.

 It is, and I will belatedly tell you that recent versions of mdadm have 
a "--uuid" option which allows you to set the uuid of the array as you 
create it. However, it sounds as if the problem was not starting the 
array with uuid, but mounting the filesystem, and that uuid can be set 
with tune2fs, using the "-U" option.

I include that for completeness, I agree that changing the things which 
use a new uuid is safer than changing the uuid which could, given 
adequate user error, result in duplicate uuid's on multiple arrays or 
filesystems. This is a "bad thing{tm}" and should be avoided. You did it 
right (IMHO) and congrats on getting it going.

-- 
Bill Davidsen <davidsen@tmr.com>
  Even technical things can appear to be magic, if the documentation is
obscure enough. For example, PulseAudio is configured by dancing naked around
a fire at midnight, shaking a rattle with one hand and a LISP manual with the
other, while reciting the GNU manifesto in hexadecimal.