Re: raid5: degraded after reboot

[Andre Noll <maan@systemlinux.org>]

[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]

On 11:08, Jon Nelson wrote:

> > sda4 was not included because the array has been assembled previously
> > using only sdb4 and sdc4. So the data on sda4 is out of date.
> 
> I don't understand - over months and months it has always been the three
> devices, /dev/sd{a,b,c}4.
> I've added and removed bitmaps and done other things but at the time it
> rebooted the array had been up, "clean" (non-degraded), and comprised of the
> three devices for 4-6 weeks.

You said you had to reboot your box using sysrq. There are chances you
caused the reboot while all pending data was written to sdb4 and sdc4,
but not to sda4. So sda4 appears to be non-fresh after the reboot and,
since mdadm refuses to use non-fresh devices, it kicks sda4.

> > This looks normal. The array is up with two working disks.
> 
> Two of three which, to me, is "abnormal" (ie, the "normal" state is three
> and it's got two).

Sure. I should have said: It's normal if one disk in a raid5 array is
missing (or non-fresh).

> > > Why was /dev/sda4 kicked?
> >
> > Because it was non-fresh ;)
> 
> OK, but what does that MEAN?

To be precise, it means that the event counter for sda4 is less than
the event counter on the other devices in the array. So mdadm must
assume the data on sda4 is out of sync and hence the device can't be
used. If you are not using bitmaps, there is no other way out than
syncing the whole device, i.e. writing good data (computed from sdb4
and sdc4) to sda4.

Hope that helps.
Andre
-- 
The only person who always got his work done by Friday was Robinson Crusoe

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]