RAID6 simply does not start as /dev/md8

RAID6 simply does not start as /dev/md8

[Yann Ormanns]

Hello everybody,
I've now been trying for over a week to get my RAID6 working.
I have set up the array by partitioning all six disks (partition type:
Linux (83)) and executing "mdadm -C /dev/md8 --level=6 --raid-devices=6
/dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1".

When the array had finished syncing, I rebooted. /proc/mdstat contained
	md126 : inactive sde1[4](S) sdd1[3](S)
	      3907025072 blocks super 1.2

	md127 : inactive sdb1[1](S) sda1[0](S) sdc1[2](S) sdf1[5](S)
	      7814050144 blocks super 1.2

So I put the following lines into /etc/mdadm.conf (I had forgot this
before):
"DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
ARRAY /dev/md8
devices=/dev/sda1,/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1,/dev/sdf1"
So I rebooted again, now the array was started as follows:

	Personalities : [raid1] [raid6] [raid5] [raid4]
	md8 : inactive sda1[0](S)
	      1953512536 blocks super 1.2

I found out, that the assembly was aborted due to the error, that
/dev/sda1 has no superblock. It seems, as if the kernel would start the
array while booting the system, so mdadm is unable to assemble it
afterwards. If I stop and re-assemble the array, it works fine.

Unfortunately, I have to keep "CONFIG_MD_AUTODETECT" enabled, because /
and the whole system runs on a RAID1. But why does the kernel start the
array, although the used partition type of the disks in my RAID6 is NOT
"linux raid autodetect"?
I simply want to start my RAID6 as /dev/md8.

I use mdadm-3.1.4 and linux-2.6.36-gentoo-r8. The superblocks seem to be
correct - "mdadm -E /dev/sd[a,b,c,d,e,f]1 | grep Name" returns
           Name : Atlas:8  (local to host Atlas)
           Name : Atlas:8  (local to host Atlas)
           Name : Atlas:8  (local to host Atlas)
           Name : Atlas:8  (local to host Atlas)
           Name : Atlas:8  (local to host Atlas)
           Name : Atlas:8  (local to host Atlas)
If I get this right, this array should be started as /dev/md8. However,
it does not. I have tried several combinations of configurations ("linux
raid autodetect" without using mdadm for assembly, "linux" together with
mdadm, and so on), but without any success.
Any help would be really apprectiated.

Best regards,
Yann

Re: RAID6 simply does not start as /dev/md8

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Fri, 08 Apr 2011 10:00:23 +0200
Yann Ormanns <yann-ormanns@web.de> wrote:
 
> So I put the following lines into /etc/mdadm.conf (I had forgot this
> before):
> "DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
> ARRAY /dev/md8
> devices=/dev/sda1,/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1,/dev/sdf1"

After you do various changes to your mdadm.conf, don't forget to update your
initrd. AFAIK mdadm.conf is copied into the compressed initrd image to be used
early at boot, and this copy has to be kept in sync with your version in /etc.

Typically you can update initrd for all installed kernels by running something
like:

  update-initramfs -k all -u

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: RAID6 simply does not start as /dev/md8

[Yann Ormanns]

Subject: Re: RAID6 simply does not start as /dev/md8
From: Roman Mamedov <rm@romanrm.ru>
To: Yann Ormanns <yann-ormanns@web.de>
Date: 2011-04-08 16:15 (+0600)

> After you do various changes to your mdadm.conf, don't forget to update your
> initrd. AFAIK mdadm.conf is copied into the compressed initrd image to be used
> early at boot, and this copy has to be kept in sync with your version in /etc.
> 
> Typically you can update initrd for all installed kernels by running something
> like:
> 
>   update-initramfs -k all -u
> 

Thank you for your quick response. I do not use an initrd - during the
boot process the kernel reconnects and assembles md1,md2,md3,md5,md6 and
md7 (these are all RAID1) without any problems. Only the assembly of the
RAID6 does not work at all.

Best regards,
Yann

Re: RAID6 simply does not start as /dev/md8

[Rudy Zijlstra]

On 04/08/2011 04:19 PM, Yann Ormanns wrote:
> Subject: Re: RAID6 simply does not start as /dev/md8
> From: Roman Mamedov<rm@romanrm.ru>
> To: Yann Ormanns<yann-ormanns@web.de>
> Date: 2011-04-08 16:15 (+0600)
>
>    
>> After you do various changes to your mdadm.conf, don't forget to update your
>> initrd. AFAIK mdadm.conf is copied into the compressed initrd image to be used
>> early at boot, and this copy has to be kept in sync with your version in /etc.
>>
>> Typically you can update initrd for all installed kernels by running something
>> like:
>>
>>    update-initramfs -k all -u
>>
>>      
> Thank you for your quick response. I do not use an initrd - during the
> boot process the kernel reconnects and assembles md1,md2,md3,md5,md6 and
> md7 (these are all RAID1) without any problems. Only the assembly of the
> RAID6 does not work at all.
>
>    
Do you have perhaps old raid metadata on the disks? Are you certain you 
have no raid metadata on the whole disks?
you can check with mdadm --Examine /dev/sdc

Cheers


Rudy

Re: RAID6 simply does not start as /dev/md8

[Yann Ormanns]

Subject: Re: RAID6 simply does not start as /dev/md8
From: Rudy Zijlstra <rudy@grumpydevil.homelinux.org>
To: Yann Ormanns <yann-ormanns@web.de>
Date: 2011-04-08 16:36 (+0200)

> Do you have perhaps old raid metadata on the disks? Are you certain you
> have no raid metadata on the whole disks?
> you can check with mdadm --Examine /dev/sdc
> 
> Cheers
> 
> 
> Rudy
> 

To eliminate this possibility, I "zeroed" all six disks before I created
the array:
dd if=/dev/zero of=/dev/sda bs=1M
...
dd if=/dev/zero of=/dev/sdf bs=1M

Output of mdadm -E /dev/sda1:
	/dev/sda1:
        	  Magic : a92b4efc
	        Version : 1.2
	    Feature Map : 0x0
	     Array UUID : 6286b922:2da22a6c:f8fba333:fc88f457
        	   Name : Atlas:8  (local to host Atlas)
	  Creation Time : Fri Apr  8 00:58:12 2011
	     Raid Level : raid6
	   Raid Devices : 6

	 Avail Dev Size : 3907025072 (1863.01 GiB 2000.40 GB)
	     Array Size : 15628095488 (7452.06 GiB 8001.58 GB)
	  Used Dev Size : 3907023872 (1863.01 GiB 2000.40 GB)
	    Data Offset : 2048 sectors
	   Super Offset : 8 sectors
        	  State : clean
	    Device UUID : 1a016923:a408d8ee:d8f6625f:bcf55b66

	    Update Time : Fri Apr  8 09:53:13 2011
	       Checksum : 69969b9b - correct
        	 Events : 19

	         Layout : left-symmetric
	     Chunk Size : 512K

	   Device Role : Active device 0
	   Array State : AAAAAA ('A' == active, '.' == missing)

The complete output of mdadm -E /dev/sd[a,b,c,d,e,f]1 can be found here:
http://nopaste.info/658b72791c_nl.html

Best regards,
Yann

Re: RAID6 simply does not start as /dev/md8

[Rudy Zijlstra]

On 04/08/2011 04:58 PM, Yann Ormanns wrote:
> Do you have perhaps old raid metadata on the disks? Are you certain you
>> have no raid metadata on the whole disks?
>> you can check with mdadm --Examine /dev/sdc
>>
>> Cheers
>>
>>
>> Rudy
>>
>>      
> To eliminate this possibility, I "zeroed" all six disks before I created
> the array:
> dd if=/dev/zero of=/dev/sda bs=1M
> ...
> dd if=/dev/zero of=/dev/sdf bs=1M
>
> Output of mdadm -E /dev/sda1:
> 	/dev/sda1:
>    
Just to humour me, the output without the partition number? so /dev/sda ?

Cheers,


Rudy

Re: RAID6 simply does not start as /dev/md8

[Yann Ormanns]

Subject: Re: RAID6 simply does not start as /dev/md8
From: Rudy Zijlstra <rudy@grumpydevil.homelinux.org>
To: Yann Ormanns <yann-ormanns@web.de>
Date: 2011-04-08 20:22 (+0200)

> Just to humour me, the output without the partition number? so /dev/sda ?
> 
> Cheers,
> 
> 
> Rudy
> 

Output of "mdadm -E /dev/sd[a,b,c,d,e,f]":
mdadm: No md superblock detected on /dev/sda.
mdadm: No md superblock detected on /dev/sdb.
mdadm: No md superblock detected on /dev/sdc.
mdadm: No md superblock detected on /dev/sdd.
mdadm: No md superblock detected on /dev/sde.
mdadm: No md superblock detected on /dev/sdf.

Best regards,
Yann

RE: RAID6 simply does not start as /dev/md8

[Leslie Rhorer]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Yann Ormanns
> Sent: Friday, April 08, 2011 3:00 AM
> To: linux-raid@vger.kernel.org
> Subject: RAID6 simply does not start as /dev/md8
> 
> Hello everybody,
> I've now been trying for over a week to get my RAID6 working.
> I have set up the array by partitioning all six disks (partition type:

	Why?  From your description, it sounds to me like md8 is built using
the whole disk on all 6 drives.  If so, there's really no need to partition
the drives.

> Linux (83)) and executing "mdadm -C /dev/md8 --level=6 --raid-devices=6
> /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1".
> 
> When the array had finished syncing, I rebooted. /proc/mdstat contained
> 	md126 : inactive sde1[4](S) sdd1[3](S)
> 	      3907025072 blocks super 1.2
> 
> 	md127 : inactive sdb1[1](S) sda1[0](S) sdc1[2](S) sdf1[5](S)
> 	      7814050144 blocks super 1.2
> 
> So I put the following lines into /etc/mdadm.conf (I had forgot this
> before):
> "DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
> ARRAY /dev/md8
> devices=/dev/sda1,/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1,/dev/sdf1"

You should not need the "devices=..." entry, but I would include the name
and the UUID.  Here are the relevant parts of my mdadm.conf:

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

ARRAY /dev/md0 level=raid6 metadata=1.2 num-devices=14
UUID=5ff10d73:a096195f:7a646bba:a68986ca name=RAID-Server:0

ARRAY /dev/md1 level=raid1 metadata=0.90 num-devices=2
UUID=f0a63cae:10406a7a:fa72b0ce:3d8e1057

ARRAY /dev/md2 level=raid1 metadata=1.2 num-devices=2
UUID=4b466602:fb81286c:4ad8dc5c:ad0bd065 name=RAID-Server:2

ARRAY /dev/md3 level=raid1 metadata=1.2 num-devices=2
UUID=5bc11cda:e1b4065f:fbf2fca5:8b12e0ba name=RAID-Server:3

> So I rebooted again, now the array was started as follows:
> 
> 	Personalities : [raid1] [raid6] [raid5] [raid4]
> 	md8 : inactive sda1[0](S)
> 	      1953512536 blocks super 1.2
> 
> I found out, that the assembly was aborted due to the error, that
> /dev/sda1 has no superblock. It seems, as if the kernel would start the
> array while booting the system, so mdadm is unable to assemble it
> afterwards. If I stop and re-assemble the array, it works fine.

	I had similar problems with a name conflict, but that doesn't seem
to be the case here.

> Unfortunately, I have to keep "CONFIG_MD_AUTODETECT" enabled, because /
> and the whole system runs on a RAID1. But why does the kernel start the
> array, although the used partition type of the disks in my RAID6 is NOT
> "linux raid autodetect"?

	Mdadm builds any arrays local to the system at boot.  Why do you
have grief with the kernel starting the array?

> I simply want to start my RAID6 as /dev/md8.
> 
> I use mdadm-3.1.4 and linux-2.6.36-gentoo-r8. The superblocks seem to be
> correct - "mdadm -E /dev/sd[a,b,c,d,e,f]1 | grep Name" returns
>            Name : Atlas:8  (local to host Atlas)
>            Name : Atlas:8  (local to host Atlas)
>            Name : Atlas:8  (local to host Atlas)
>            Name : Atlas:8  (local to host Atlas)
>            Name : Atlas:8  (local to host Atlas)
>            Name : Atlas:8  (local to host Atlas)
> If I get this right, this array should be started as /dev/md8. However,
> it does not. I have tried several combinations of configurations ("linux
> raid autodetect" without using mdadm for assembly, "linux" together with
> mdadm, and so on), but without any success.
> Any help would be really apprectiated.
> 
> Best regards,
> Yann
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RAID6 simply does not start as /dev/md8

[Yann Ormanns]

First of all:
I finally solved the problem by creating an initrd, which assembles all
arrays, so I could deactivate CONFIG_MD_AUTODETECT in the kernel. I also
do not need /etc/mdadm.conf anymore - and it works like a charm.

Thanks to everybody who attended to this problem!
The summary can be found here:
https://forums.gentoo.org/viewtopic-t-868017-highlight-.html

Subject: RE: RAID6 simply does not start as /dev/md8
From: Leslie Rhorer <lrhorer@satx.rr.com>
To: Yann Ormanns <yann-ormanns@web.de>
Date: 2011-04-11 10:06 (-0500)

> From your description, it sounds to me like md8 is built using
> the whole disk on all 6 drives.  If so, there's really no need to partition
> the drives.
> 

That is correct. I simply wanted to eliminate possible mistakes - after
6 or 7 "useless" syncs of the array I tried _everything_, just to get
this thing work.

>> Unfortunately, I have to keep "CONFIG_MD_AUTODETECT" enabled, because /
>> and the whole system runs on a RAID1. But why does the kernel start the
>> array, although the used partition type of the disks in my RAID6 is NOT
>> "linux raid autodetect"?
> 
> 	Mdadm builds any arrays local to the system at boot.  Why do you
> have grief with the kernel starting the array?
> 

The problem was, that - although I did not use the filesystem type
"linux raid autodetect" for my RAID6 - the kernel still seemed to
assemble it. Now, without its autodetection, it works fine.

Best regards,
Yann