degraded raid5 at bootup

degraded raid5 at bootup

[Tupshin Harper]

I have a 5 disk raid5 using autodetect partitions. If I have a clean 
array when I shut down, I always have a degraded array with one disk 
removed when I boot back up. Below I include the relevant dmesg output. 
hdi1 is always the problem. If I do a "mdadm -a /dev/md0 /dev/hdi1", 
then the array rebuilds without a problem, and runs clean until I reboot 
again. Any suggestions?

Kernel is 2.6.8-rc2-mm2, but I've tried it with a few other recent 
kernels (mm and not) and all have the same problem. Distro is Debian Sid.

-Tupshin

md: considering hdl1 ...
md:  adding hdl1 ...
md:  adding hdj1 ...
md: hdi1 has different UUID to hdl1
md:  adding hdh1 ...
md:  adding hde1 ...
md: created md0
md: bind<hde1>
md: bind<hdh1>
md: bind<hdj1>
md: bind<hdl1>
md: running: <hdl1><hdj1><hdh1><hde1>
raid5: device hdl1 operational as raid disk 4
raid5: device hdj1 operational as raid disk 3
raid5: device hdh1 operational as raid disk 1
raid5: device hde1 operational as raid disk 0
raid5: allocated 5242kB for md0
raid5: raid level 5 set md0 active with 4 out of 5 devices, algorithm 2
RAID5 conf printout:
 --- rd:5 wd:4 fd:1
 disk 0, o:1, dev:hde1
 disk 1, o:1, dev:hdh1
 disk 3, o:1, dev:hdj1
 disk 4, o:1, dev:hdl1
md: considering hdi1 ...
md:  adding hdi1 ...
md: md0 already running, cannot run hdi1
md: export_rdev(hdi1)
md: ... autorun DONE.

Re: degraded raid5 at bootup

[John Stoffel]

Tupshin> I have a 5 disk raid5 using autodetect partitions. If I have
Tupshin> a clean array when I shut down, I always have a degraded
Tupshin> array with one disk removed when I boot back up. Below I
Tupshin> include the relevant dmesg output.  hdi1 is always the
Tupshin> problem. If I do a "mdadm -a /dev/md0 /dev/hdi1", then the
Tupshin> array rebuilds without a problem, and runs clean until I
Tupshin> reboot again. Any suggestions?

Are all your disks setup with partition 1 setup as 'Linux Raid'
properly?  Check /dev/hdi using 'cfdisk' and see what partition 1 has
for a type. 

Tupshin> Kernel is 2.6.8-rc2-mm2, but I've tried it with a few other
Tupshin> recent kernels (mm and not) and all have the same
Tupshin> problem. Distro is Debian Sid.

Tupshin> -Tupshin

Tupshin> md: considering hdl1 ...
Tupshin> md:  adding hdl1 ...
Tupshin> md:  adding hdj1 ...
Tupshin> md: hdi1 has different UUID to hdl1

This is a key line here!  You need to update the superblocks on the
raid elements to have the proper stuff.  My personal system is down,
so I can't tell you the exactl command, but make sure you've got the
same setup on all five disks.

I assume they're all the same size, right?  If so, just make sure
they're all the same partition stuff, etc.  

If need be, reboot your system and zero out the first few blocks on
/dev/hdi (say 32 or 64 blocks) and then re-partition the disk, and
then make sure partition 1 is set to Linux Raid.  

Then use the mdadm command to update the superblocks.

John