split brain mode after reboot

split brain mode after reboot

[Carsten Aulbert]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]

Hi all

a colleague of mine created a raid 1 on a fairly recent machine

Kernel 3.5.0-sabayon

mdadm - v3.2.3 - 23rd December 2011

During operation sda seemed to have been disconnected by the
system/motherboard/whatever but this was not detected before a reboot
was done, after the reboot, sda re-appeared but of course with a much
older version of the mirror:

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4] [multipath] [faulty]
md126 : active raid1 sdb1[1]
      4194240 blocks [2/1] [_U]

md127 : active raid1 sdb3[1]
      235808704 blocks [2/1] [_U]

md0 : active raid1 sda1[0]
      4194240 blocks [2/1] [U_]

md1 : active raid0 sda2[0] sdb2[1]
      8387584 blocks 512k chunks

md2 : active raid1 sda3[0]
      235808704 blocks [2/1] [U_]

unused devices: <none>

As no vital information were on these disks, my question for the list is
just if this is an expected/wanted behavior after such an event and what
one could do to prevent this (besides monitoring via mdadm).

Cheers

Carsten

-- 
Dr. Carsten Aulbert - Max Planck Institute for Gravitational Physics
Callinstrasse 38, 30167 Hannover, Germany
phone/fax: +49 511 762-17185 / -17193
https://wiki.atlas.aei.uni-hannover.de/foswiki/bin/view/ATLAS/WebHome


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2044 bytes --]

Re: split brain mode after reboot

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1602 bytes --]

On Tue, 20 Nov 2012 15:08:45 +0100 Carsten Aulbert
<Carsten.Aulbert@aei.mpg.de> wrote:

> Hi all
> 
> a colleague of mine created a raid 1 on a fairly recent machine
> 
> Kernel 3.5.0-sabayon
> 
> mdadm - v3.2.3 - 23rd December 2011
> 
> During operation sda seemed to have been disconnected by the
> system/motherboard/whatever but this was not detected before a reboot
> was done, after the reboot, sda re-appeared but of course with a much
> older version of the mirror:
> 
> Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
> [raid4] [multipath] [faulty]
> md126 : active raid1 sdb1[1]
>       4194240 blocks [2/1] [_U]
> 
> md127 : active raid1 sdb3[1]
>       235808704 blocks [2/1] [_U]
> 
> md0 : active raid1 sda1[0]
>       4194240 blocks [2/1] [U_]
> 
> md1 : active raid0 sda2[0] sdb2[1]
>       8387584 blocks 512k chunks
> 
> md2 : active raid1 sda3[0]
>       235808704 blocks [2/1] [U_]
> 
> unused devices: <none>
> 
> As no vital information were on these disks, my question for the list is
> just if this is an expected/wanted behavior after such an event and what
> one could do to prevent this (besides monitoring via mdadm).
> 

Expected - probably.
Wanted - no.

I think one half gets assembled by "mdadm --incremental" run from udev, and
the other by a subsequence "mdadm -As" or similar.

mdadm-3.3, which is still under development, has a fix for this so that
--incremental and --assemble don't trip over each other.

http://git.neil.brown.name/?p=mdadm.git;a=commitdiff;h=0431869cec4c673309d9aa

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: split brain mode after reboot

[Carsten Aulbert]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Neil

On 11/20/2012 10:22 PM, NeilBrown wrote:
> 
> Expected - probably. Wanted - no.
> 

good :)

> I think one half gets assembled by "mdadm --incremental" run from
> udev, and the other by a subsequence "mdadm -As" or similar.
> 
> mdadm-3.3, which is still under development, has a fix for this so
> that --incremental and --assemble don't trip over each other.
> 
> http://git.neil.brown.name/?p=mdadm.git;a=commitdiff;h=0431869cec4c673309d9aa

Great,
> 
I think we'll wait for 3.3 and try to backport it to our
current production systems.

Thanks!

Carsten
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlCtxBsACgkQvfAZOV3797y6ZACgk/puEYU+C2AX+SCVJz5ZhY83
QaIAnRCpHW6zK2PYS6CU8+XiCkJwoQZ+
=MnXg
-----END PGP SIGNATURE-----


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2044 bytes --]