Renaming MD devices (metadata=1.1)

Renaming MD devices (metadata=1.1)

[Gilbert Kowarzyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,   - Jan 10, 2012

I am having problems renaming a few md devices and I've been trying
out different things, but none of them seem to work.

The md devices were created with metadata=1.1.

Some sites talk about changing the preferred minor, but I haven't even
found that entry for these devices.

This is all on CentOS 6.2.

Below is the info for one of these devices I am having trouble with. I
would like to name it /dev/md9 but after a reboot it renames itself
/dev/md126.

# mdadm --detail /dev/md127
=================================>
/dev/md127:
        Version : 1.1
  Creation Time : Sun Jan  8 04:58:25 2012
     Raid Level : raid1
     Array Size : 1843198844 (1757.81 GiB 1887.44 GB)
  Used Dev Size : 1843198844 (1757.81 GiB 1887.44 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Mon Jan  9 23:47:52 2012
          State : active
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           Name : xyz.homelinux.com:9  (local to host xyz.homelinux.com)
           UUID : 0fa7f737:7806ca61:fb89edd6:14aa351a
         Events : 20

    Number   Major   Minor   RaidDevice State
       0       8       33        0      active sync   /dev/sdc1
       1       8       49        1      active sync   /dev/sdd1
<=================================


# mdadm --examine /dev/sdc1
=================================>
/dev/sdc1:
          Magic : a92b4efc
        Version : 1.1
    Feature Map : 0x1
     Array UUID : 0fa7f737:7806ca61:fb89edd6:14aa351a
           Name : xyz.homelinux.com:9  (local to host xyz.homelinux.com)
  Creation Time : Sun Jan  8 04:58:25 2012
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 3686397952 (1757.81 GiB 1887.44 GB)
     Array Size : 3686397688 (1757.81 GiB 1887.44 GB)
  Used Dev Size : 3686397688 (1757.81 GiB 1887.44 GB)
    Data Offset : 2048 sectors
   Super Offset : 0 sectors
          State : clean
    Device UUID : 73fb0ea3:e08fd3c6:b3e41f50:c1e3a2ac

Internal Bitmap : 2 sectors from superblock
    Update Time : Tue Jan 10 00:04:29 2012
       Checksum : e6bfee2 - correct
         Events : 20


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing)
<=================================


# mdadm --examine /dev/sdd1
=================================>
/dev/sdd1:
          Magic : a92b4efc
        Version : 1.1
    Feature Map : 0x1
     Array UUID : 0fa7f737:7806ca61:fb89edd6:14aa351a
           Name : xyz.homelinux.com:9  (local to host xyz.homelinux.com)
  Creation Time : Sun Jan  8 04:58:25 2012
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 3686397952 (1757.81 GiB 1887.44 GB)
     Array Size : 3686397688 (1757.81 GiB 1887.44 GB)
  Used Dev Size : 3686397688 (1757.81 GiB 1887.44 GB)
    Data Offset : 2048 sectors
   Super Offset : 0 sectors
          State : clean
    Device UUID : e6cec1cb:40cab8c8:d1cb2a07:af3f2517

Internal Bitmap : 2 sectors from superblock
    Update Time : Tue Jan 10 00:04:29 2012
       Checksum : 5a914fc1 - correct
         Events : 20


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing)
=================================


# mdadm --examine --scan /dev/sdc1
=================================>
ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
name=xyz.homelinux.com:9
<=================================


# mdadm --examine --scan /dev/sdd1
=================================>
ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
name=xyz.homelinux.com:9
<=================================


# mdadm --detail --scan
=================================>
ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
<=================================

# blkid
=================================>
/dev/md127: UUID="c301cf13-cfaf-4f56-887e-c648caf5a931" TYPE="ext4"
<=================================

in /etc/fstab I have:
=================================>
UUID=c301cf13-cfaf-4f56-887e-c648caf5a931 /DATA    ext4  defaults 1 2
<=================================

and in /etc/mdadm.conf I have:
=================================>
ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
<=================================


To me, the line that looks suspicious is the one we get from "mdadm
- --detail --scan". All the devices that "look right" say "ARRAY
/dev/md/X" where X is a number.

I fixed this temporarily by stopping the array and re-assembling it with:

mdadm --assemble /dev/md9 --name=9 --update=name /dev/sdc1 /dev/sdd1


Then the mdadm --detail --scan line says:
ARRAY /dev/md/9 metadata=1.1 name=xyz.homelinux.com:9
UUID=0fa7f737:7806ca61:fb89edd6:14aa351a

... which makes more sense to me (it looks like the other ones.

Yet, when I reboot the computer this "temporarily fixed" array returns
to /dev/md127.

I have since redone these steps saving the changes in the
/etc/mdadm.conf file, but after a reboot the same thing happens
(despite the /etc/mdadm.conf file saying otherwise).

So, I've run out of ideas... help?

Thanks in advance!

Gilbert

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: digital signature
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8L1ugACgkQOYVp1RpLsmUeIwCfa2zFkMzq9UtxrFtl6Qr4U9Qs
kT4AniCJDeJHS2DxOD1G3vYGJ1O+8oNK
=QpQl
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[Michal Soltys]

On 10.01.2012 07:12, Gilbert Kowarzyk wrote:
>
>
> # mdadm --examine --scan /dev/sdc1
> =================================>
> ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> name=xyz.homelinux.com:9
> <=================================
>
>
> # mdadm --examine --scan /dev/sdd1
> =================================>
> ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> name=xyz.homelinux.com:9
> <=================================
>
>
> # mdadm --detail --scan
> =================================>
> ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
> name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> <=================================
>
> # blkid
> =================================>
> /dev/md127: UUID="c301cf13-cfaf-4f56-887e-c648caf5a931" TYPE="ext4"
> <=================================
>
> in /etc/fstab I have:
> =================================>
> UUID=c301cf13-cfaf-4f56-887e-c648caf5a931 /DATA    ext4  defaults 1 2
> <=================================
>
> and in /etc/mdadm.conf I have:
> =================================>
> ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
> name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> <=================================
>
>
> To me, the line that looks suspicious is the one we get from "mdadm
> - --detail --scan". All the devices that "look right" say "ARRAY
> /dev/md/X" where X is a number.
>
> I fixed this temporarily by stopping the array and re-assembling it with:
>
> mdadm --assemble /dev/md9 --name=9 --update=name /dev/sdc1 /dev/sdd1
>

You updated name stored in the metadata, and assembled the array 
explicitly requesting particular standard name. But the names directly 
under /dev are managed/assigned by udev.

As the /dev/md9 was explicitly given in this form, the MD_DEVNAME was 
not present in 'mdadm --detail --export' output, thus /dev/md/9 was not 
created by mdadm's udev rules - but it would have been if you had done 
mdadm --assemble /dev/md/9 ...

>
> Then the mdadm --detail --scan line says:
> ARRAY /dev/md/9 metadata=1.1 name=xyz.homelinux.com:9
> UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
>
> ... which makes more sense to me (it looks like the other ones.
>
> Yet, when I reboot the computer this "temporarily fixed" array returns
> to /dev/md127.

But you should have proper, permanent name (as a symlink) under /dev/md/

Autoassembly (be it -I used with modern mdadm's udev rules) or -A will 
try to honor the device name if the name is in standard format, so for 
mdadm.conf with ARRAY /dev/md/9 ... - you will (usually) get /dev/md9 - 
but this it's not really worth chasing after or relying on (and what if 
the name is already taken ?).

Just write:

ARRAY /dev/md/somename UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
ARRAY somename UUID=0fa7f737:7806ca61:fb89edd6:14aa351a

in which case 'somename' will be created under /dev/md/ regardless of 
the name stored in the metadata, or just

ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a

so the metadata (without hostname prefix) will be used.

So to sum it up - if '9' is what you're after - then the ARRAY line with 
just UUID is all you should be needing - and you should always get 
/dev/md/9 (pointing to something else assigned dynamically).

If udevd is not running (hardly interesting case anymore), then mdadm 
will fall back to the pre-udev behavior and setup the nodes itself, 
though the rules are slightly different then.

Re: Renaming MD devices (metadata=1.1)

[Phil Turmel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/10/2012 01:12 AM, Gilbert Kowarzyk wrote:
[...]
> I have since redone these steps saving the changes in the
> /etc/mdadm.conf file, but after a reboot the same thing happens
> (despite the /etc/mdadm.conf file saying otherwise).
> 
> So, I've run out of ideas... help?

You might just need to update your initramfs.  It needs to have its
own copy of mdadm.conf for use before the root filesystem is
mounted.

I don't use CentOS or its upstream, so I'm not sure what command
will do this for you, but I suspect it is well documented. :-)

HTH,

Phil
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8MM/QACgkQBP+iHzflm3AZYQCcDducIPM4rchAtsWWYLw3DViS
xiAAnR0MYlY47HlkypA98x3xyLErnxoy
=n3cC
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[Gilbert Kowarzyk]

Hello,

Thanks for the quick answer. My replies are below.


>> # mdadm --examine --scan /dev/sdc1
>> =================================>
>> ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
>> name=xyz.homelinux.com:9
>> <=================================
>>
>>
>> # mdadm --examine --scan /dev/sdd1
>> =================================>
>> ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
>> name=xyz.homelinux.com:9
>> <=================================
>>
>>
>> # mdadm --detail --scan
>> =================================>
>> ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
>> name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
>> <=================================
>>
>> # blkid
>> =================================>
>> /dev/md127: UUID="c301cf13-cfaf-4f56-887e-c648caf5a931" TYPE="ext4"
>> <=================================
>>
>> in /etc/fstab I have:
>> =================================>
>> UUID=c301cf13-cfaf-4f56-887e-c648caf5a931 /DATA    ext4  defaults 1 2
>> <=================================
>>
>> and in /etc/mdadm.conf I have:
>> =================================>
>> ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
>> name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
>> <=================================
>>
>>
>> To me, the line that looks suspicious is the one we get from "mdadm
>> - --detail --scan". All the devices that "look right" say "ARRAY
>> /dev/md/X" where X is a number.
>>
>> I fixed this temporarily by stopping the array and re-assembling it with:
>>
>> mdadm --assemble /dev/md9 --name=9 --update=name /dev/sdc1 /dev/sdd1
>>
> 
> You updated name stored in the metadata, and assembled the array
> explicitly requesting particular standard name. But the names directly
> under /dev are managed/assigned by udev.
> 
> As the /dev/md9 was explicitly given in this form, the MD_DEVNAME was
> not present in 'mdadm --detail --export' output, thus /dev/md/9 was not
> created by mdadm's udev rules - but it would have been if you had done
> mdadm --assemble /dev/md/9 ...

Ok, I think I follow this, but I am not sure.

What you are saying is that I am getting /dev/md127 because it's not
being created by udev right?


>> Then the mdadm --detail --scan line says:
>> ARRAY /dev/md/9 metadata=1.1 name=xyz.homelinux.com:9
>> UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
>>
>> ... which makes more sense to me (it looks like the other ones.
>>
>> Yet, when I reboot the computer this "temporarily fixed" array returns
>> to /dev/md127.
> 
> But you should have proper, permanent name (as a symlink) under /dev/md/

I just checked, and indeed I do. The symlinks I have under /dev/md/ are
exactly the ones relating to the md devices I am having problems with
renaming.
There are no symlinks for the other md devices.


> Autoassembly (be it -I used with modern mdadm's udev rules) or -A will
> try to honor the device name if the name is in standard format, so for
> mdadm.conf with ARRAY /dev/md/9 ... - you will (usually) get /dev/md9 -
> but this it's not really worth chasing after or relying on (and what if
> the name is already taken ?).
> 
> Just write:
> 
> ARRAY /dev/md/somename UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> ARRAY somename UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> 
> in which case 'somename' will be created under /dev/md/ regardless of
> the name stored in the metadata, or just
> 
> ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> 
> so the metadata (without hostname prefix) will be used.

so expliciting the metadata=x is not needed? What's its use?


> So to sum it up - if '9' is what you're after - then the ARRAY line with
> just UUID is all you should be needing - and you should always get
> /dev/md/9 (pointing to something else assigned dynamically).
> 
> If udevd is not running (hardly interesting case anymore), then mdadm
> will fall back to the pre-udev behavior and setup the nodes itself,
> though the rules are slightly different then.

I just left ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a in
/etc/mdadm.conf, rebooted, and it's still appearing as /dev/md127.

So I am not sure why... What may I be missing?

Thanks,

Gilbert

Re: Renaming MD devices (metadata=1.1)

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 5405 bytes --]

On Wed, 11 Jan 2012 01:07:38 -0500 Gilbert Kowarzyk <kowarzyk@grm.polymtl.ca>
wrote:

> Hello,
> 
> Thanks for the quick answer. My replies are below.
> 
> 
> >> # mdadm --examine --scan /dev/sdc1
> >> =================================>
> >> ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> >> name=xyz.homelinux.com:9
> >> <=================================
> >>
> >>
> >> # mdadm --examine --scan /dev/sdd1
> >> =================================>
> >> ARRAY /dev/md/9 metadata=1.1 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> >> name=xyz.homelinux.com:9
> >> <=================================
> >>
> >>
> >> # mdadm --detail --scan
> >> =================================>
> >> ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
> >> name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> >> <=================================
> >>
> >> # blkid
> >> =================================>
> >> /dev/md127: UUID="c301cf13-cfaf-4f56-887e-c648caf5a931" TYPE="ext4"
> >> <=================================
> >>
> >> in /etc/fstab I have:
> >> =================================>
> >> UUID=c301cf13-cfaf-4f56-887e-c648caf5a931 /DATA    ext4  defaults 1 2
> >> <=================================
> >>
> >> and in /etc/mdadm.conf I have:
> >> =================================>
> >> ARRAY /dev/md/xyz.homelinux.com:9 metadata=1.1
> >> name=xyz.homelinux.com:9 UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> >> <=================================
> >>
> >>
> >> To me, the line that looks suspicious is the one we get from "mdadm
> >> - --detail --scan". All the devices that "look right" say "ARRAY
> >> /dev/md/X" where X is a number.
> >>
> >> I fixed this temporarily by stopping the array and re-assembling it with:
> >>
> >> mdadm --assemble /dev/md9 --name=9 --update=name /dev/sdc1 /dev/sdd1
> >>
> > 
> > You updated name stored in the metadata, and assembled the array
> > explicitly requesting particular standard name. But the names directly
> > under /dev are managed/assigned by udev.
> > 
> > As the /dev/md9 was explicitly given in this form, the MD_DEVNAME was
> > not present in 'mdadm --detail --export' output, thus /dev/md/9 was not
> > created by mdadm's udev rules - but it would have been if you had done
> > mdadm --assemble /dev/md/9 ...
> 
> Ok, I think I follow this, but I am not sure.
> 
> What you are saying is that I am getting /dev/md127 because it's not
> being created by udev right?

You are getting md127 because you asked for "/dev/md/xyz.home.linux.com:9"
so it gave you that (look in /dev/md) but it needs to give an 'mdXX' name to
the kernel so it chose 127 (because that is where it starts looking for free
numbers) and used that.  Don't worry about  the /dev/mdXX.  You asked for
"/dev/md/xyz.home.linux.com:9' in mdadm.conf,  you got, so just use it and
ignore the rest.



> 
> 
> >> Then the mdadm --detail --scan line says:
> >> ARRAY /dev/md/9 metadata=1.1 name=xyz.homelinux.com:9
> >> UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> >>
> >> ... which makes more sense to me (it looks like the other ones.
> >>
> >> Yet, when I reboot the computer this "temporarily fixed" array returns
> >> to /dev/md127.
> > 
> > But you should have proper, permanent name (as a symlink) under /dev/md/
> 
> I just checked, and indeed I do. The symlinks I have under /dev/md/ are
> exactly the ones relating to the md devices I am having problems with
> renaming.
> There are no symlinks for the other md devices.
> 
> 
> > Autoassembly (be it -I used with modern mdadm's udev rules) or -A will
> > try to honor the device name if the name is in standard format, so for
> > mdadm.conf with ARRAY /dev/md/9 ... - you will (usually) get /dev/md9 -
> > but this it's not really worth chasing after or relying on (and what if
> > the name is already taken ?).
> > 
> > Just write:
> > 
> > ARRAY /dev/md/somename UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> > ARRAY somename UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> > 
> > in which case 'somename' will be created under /dev/md/ regardless of
> > the name stored in the metadata, or just
> > 
> > ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a
> > 
> > so the metadata (without hostname prefix) will be used.
> 
> so expliciting the metadata=x is not needed? What's its use?

Documentation.
It doesn't hurt, and I want "mdadm -Db" to report it, so if I allow it in
mdadm.conf then it is easier to use the output of "mdadm -Db" in mdadm.conf.


> 
> 
> > So to sum it up - if '9' is what you're after - then the ARRAY line with
> > just UUID is all you should be needing - and you should always get
> > /dev/md/9 (pointing to something else assigned dynamically).
> > 
> > If udevd is not running (hardly interesting case anymore), then mdadm
> > will fall back to the pre-udev behavior and setup the nodes itself,
> > though the rules are slightly different then.
> 
> I just left ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a in
> /etc/mdadm.conf, rebooted, and it's still appearing as /dev/md127.

Just as I would expect.  You didn't ask for a specific number, so it just
chose an arbitrary one.

> 
> So I am not sure why... What may I be missing?

Don't know.  Maybe you are expecting something that isn't needed.

What exactly do you want?  And why?

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Renaming MD devices (metadata=1.1) [SOLVED]

[Gilbert Kowarzyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

my reply is below.


>> I have since redone these steps saving the changes in the 
>> /etc/mdadm.conf file, but after a reboot the same thing happens 
>> (despite the /etc/mdadm.conf file saying otherwise).
> 
>> So, I've run out of ideas... help?
> 
> You might just need to update your initramfs.  It needs to have its
> own copy of mdadm.conf for use before the root filesystem is 
> mounted.
> 
> I don't use CentOS or its upstream, so I'm not sure what command 
> will do this for you, but I suspect it is well documented. :-)

So what I did is:

1.- reassemble all drive updating the metadata as I explained before.
2.- re-generating a /etc/mdadm.conf file
3.- update the initramfs for my current kernel.

For CentOS, the command would be (if anyone else ever needs this):

   dracut --force

It seems to have worked!

Thanks again, I don't think I would have guessed this, and I didn't
find it written anywhere I looked (maybe it's there, but I managed to
overlook it!).

Well, I learned a whole bunch of cool things!
(between this and the udev explanation of Michal Soltys).

Have a nice week,

Gilbert
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: digital signature
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8NKvsACgkQOYVp1RpLsmWRoACglzwSiQbt5495OLVaiSQ+82oJ
8yAAn0VoFUu8waP33dDgFpV4/CSr65cJ
=cEbA
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[Gilbert Kowarzyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello!

[...]
>>> As the /dev/md9 was explicitly given in this form, the 
>>> MD_DEVNAME was not present in 'mdadm --detail --export' output,
>>> thus /dev/md/9 was not created by mdadm's udev rules - but it
>>> would have been if you had done mdadm --assemble /dev/md/9 ...
>> 
>> Ok, I think I follow this, but I am not sure.
>> 
>> What you are saying is that I am getting /dev/md127 because it's
>>  not being created by udev right?
> 
> You are getting md127 because you asked for 
> "/dev/md/xyz.home.linux.com:9" so it gave you that (look in 
> /dev/md) but it needs to give an 'mdXX' name to the kernel so it 
> chose 127 (because that is where it starts looking for free 
> numbers) and used that.  Don't worry about  the /dev/mdXX.  You 
> asked for "/dev/md/xyz.home.linux.com:9' in mdadm.conf,  you got, 
> so just use it and ignore the rest.

OK, I think it makes sense now.


[...]
>> so expliciting the metadata=x is not needed? What's its use?
> 
> Documentation. It doesn't hurt, and I want "mdadm -Db" to report 
> it, so if I allow it in mdadm.conf then it is easier to use the 
> output of "mdadm -Db" in mdadm.conf.

I see. Well, that's also good to know.

While we are at it, what is the reasons types 1.0, 1.1 and 1.2 exist?
- From what I understand "either at the end (for 1.0), at the start (for
1.1) or 4K from the start (for 1.2)".

Why do we have these choices, and when should each be used?


>>> So to sum it up - if '9' is what you're after - then the ARRAY
>>>  line with just UUID is all you should be needing - and you 
>>> should always get /dev/md/9 (pointing to something else 
>>> assigned dynamically).
>>> 
>>> If udevd is not running (hardly interesting case anymore), then
>>> mdadm will fall back to the pre-udev behavior and setup the
>>> nodes itself, though the rules are slightly different then.
>> 
>> I just left ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a in 
>> /etc/mdadm.conf, rebooted, and it's still appearing as 
>> /dev/md127.
> 
> Just as I would expect.  You didn't ask for a specific number, so 
> it just chose an arbitrary one.

Ok.


>> So I am not sure why... What may I be missing?
> 
> Don't know.  Maybe you are expecting something that isn't needed.
> 
> What exactly do you want?  And why?

Well, to be honest, what I *really* wanted was to understand why I
wasn't able to rename the arrays. I wanted to understand what the
system was doing, because I don't like not understanding why my setup
doesn't behave the way I expect it to. So, I wanted to learn what I
was missing in my understanding.

How I came about wanting to know this is because I tried renaming the
devices, and it wasn't working, despite me reading different people's
posts on the net. I tried reading the documentation too, but I somehow
missed reading about the updating of initramfs (I thought I only
needed to do that if the device where "/" is changed names).

Indeed, I could have continued keeping these devices named /dev/md127
etc (there is no harm in having those names), but my mind just doesn't
work that way...
I had the need of understanding what was happening and why whatever I
was trying to do wasn't working :)

So I came about to your contributions to the source code, then to your
blog page, then from there to emailing this list as you suggested :)

I figured that if I were to ask these questions, I had to give as much
info as possible to avoid wasting your time (which is why I pasted all
the info I thought could matter, and tried to organize it a bit... as
far as I had come to understand it).

That's the whole story.

So thanks again for taking the time for reading all of this :)
You guys were really responsive!

I wish you all a nice week!

Gilbert
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: digital signature
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8NLrgACgkQOYVp1RpLsmU91gCghIixD+I5czrVAlNifcr7A20Z
QqIAoJaveKMWAzNybI4R1gX0YTsYoYal
=wjtT
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[NeilBrown]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 11 Jan 2012 01:39:59 -0500 Gilbert Kowarzyk <kowarzyk@grm.polymtl.ca>
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hello!
> 
> [...]
> >>> As the /dev/md9 was explicitly given in this form, the 
> >>> MD_DEVNAME was not present in 'mdadm --detail --export' output,
> >>> thus /dev/md/9 was not created by mdadm's udev rules - but it
> >>> would have been if you had done mdadm --assemble /dev/md/9 ...
> >> 
> >> Ok, I think I follow this, but I am not sure.
> >> 
> >> What you are saying is that I am getting /dev/md127 because it's
> >>  not being created by udev right?
> > 
> > You are getting md127 because you asked for 
> > "/dev/md/xyz.home.linux.com:9" so it gave you that (look in 
> > /dev/md) but it needs to give an 'mdXX' name to the kernel so it 
> > chose 127 (because that is where it starts looking for free 
> > numbers) and used that.  Don't worry about  the /dev/mdXX.  You 
> > asked for "/dev/md/xyz.home.linux.com:9' in mdadm.conf,  you got, 
> > so just use it and ignore the rest.
> 
> OK, I think it makes sense now.
> 
> 
> [...]
> >> so expliciting the metadata=x is not needed? What's its use?
> > 
> > Documentation. It doesn't hurt, and I want "mdadm -Db" to report 
> > it, so if I allow it in mdadm.conf then it is easier to use the 
> > output of "mdadm -Db" in mdadm.conf.
> 
> I see. Well, that's also good to know.
> 
> While we are at it, what is the reasons types 1.0, 1.1 and 1.2 exist?
> - From what I understand "either at the end (for 1.0), at the start (for
> 1.1) or 4K from the start (for 1.2)".
> 
> Why do we have these choices, and when should each be used?

1.0 (at the end) is best for RAID1 when booting from the device as the
boot-load just doesn't see the RAID at all.

1.1 (at the start) is generally good because various different things have
metadata at the start, so there is no chance of confusion about what is using
that devices - each possible users will overwrite the metadata of the others.
It is also easier to make the devices larger if you don't need to move the
metadata.

1.2 (4K from the start) has most of the benefits for 1.1 but works for
booting with newer boot loaders - they still want sector 0 for a boot sector
but understand the md superblock.


> 
> 
> >>> So to sum it up - if '9' is what you're after - then the ARRAY
> >>>  line with just UUID is all you should be needing - and you 
> >>> should always get /dev/md/9 (pointing to something else 
> >>> assigned dynamically).
> >>> 
> >>> If udevd is not running (hardly interesting case anymore), then
> >>> mdadm will fall back to the pre-udev behavior and setup the
> >>> nodes itself, though the rules are slightly different then.
> >> 
> >> I just left ARRAY UUID=0fa7f737:7806ca61:fb89edd6:14aa351a in 
> >> /etc/mdadm.conf, rebooted, and it's still appearing as 
> >> /dev/md127.
> > 
> > Just as I would expect.  You didn't ask for a specific number, so 
> > it just chose an arbitrary one.
> 
> Ok.
> 
> 
> >> So I am not sure why... What may I be missing?
> > 
> > Don't know.  Maybe you are expecting something that isn't needed.
> > 
> > What exactly do you want?  And why?
> 
> Well, to be honest, what I *really* wanted was to understand why I
> wasn't able to rename the arrays. I wanted to understand what the
> system was doing, because I don't like not understanding why my setup
> doesn't behave the way I expect it to. So, I wanted to learn what I
> was missing in my understanding.

Commendable!

NeilBrown


> 
> How I came about wanting to know this is because I tried renaming the
> devices, and it wasn't working, despite me reading different people's
> posts on the net. I tried reading the documentation too, but I somehow
> missed reading about the updating of initramfs (I thought I only
> needed to do that if the device where "/" is changed names).
> 
> Indeed, I could have continued keeping these devices named /dev/md127
> etc (there is no harm in having those names), but my mind just doesn't
> work that way...
> I had the need of understanding what was happening and why whatever I
> was trying to do wasn't working :)
> 
> So I came about to your contributions to the source code, then to your
> blog page, then from there to emailing this list as you suggested :)
> 
> I figured that if I were to ask these questions, I had to give as much
> info as possible to avoid wasting your time (which is why I pasted all
> the info I thought could matter, and tried to organize it a bit... as
> far as I had come to understand it).
> 
> That's the whole story.
> 
> So thanks again for taking the time for reading all of this :)
> You guys were really responsive!
> 
> I wish you all a nice week!
> 
> Gilbert
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: digital signature
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAk8NLrgACgkQOYVp1RpLsmU91gCghIixD+I5czrVAlNifcr7A20Z
> QqIAoJaveKMWAzNybI4R1gX0YTsYoYal
> =wjtT
> -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iQIVAwUBTw00Uznsnt1WYoG5AQKjchAAucfvzP2GUy/VMniOvilykPk3wcHI3Xy6
oxj6EIoaBvJ+8XRshY3xxhgzup7Tw55KFVWwb7RTOCtkXfpNFmhv5J0zaH8nXlO9
z8uXtfoFfC6A2CtVxY0JPGaixqCIo+JkJguKzSAea15qh9g6WRdJibXlhuni+a3S
35birxn91FVfvyCAitipKqLFuzETt24Hcm67GIi2DEEXNgtkAO5jkiqteJZXMtmP
Ll2WsmdTF2e4CwRjBd0Vwv6NqMOg10KpRWdOgsI3RW5rSdgrxJCj8YPmkcRYuWm1
WgyNVPF+zX7QqFBPa0kK8bZ9lbTZM20KrI4nqab0ejTr9iyxwpzcJONteAf4A3vw
bRkOOpLcwOwxW4AjX7OywChcDZNVJHrOZfJTlWM9W3urGPhPbuLCjI+3RMdyrxGv
yxZBBTzKHwC5Hm934ZetZze22JTdDx3xpz+TZTX8LqpLrAHIzos+Ap9RSDZ8sn00
qFOzKzY5m8yBQktD2oCeBAekKxmBIBeQ4HQ9TOxWOGPGjMq72MUdkvYps3hH7R8Z
lGRSHt1OynlX/zt/p4+0JfC0HqZhafFDxStREWgvJemUs+7+v0lNrTbe6YIHOLbb
tyCpMm3BaVhdS8KchTLDIjEn/TjZ9xyGJM4g09cWiD5taWLeoqdNUiSX+V5j2ihP
8Gcj+T57omo=
=WExG
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[Gilbert Kowarzyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

>> Why do we have these choices, and when should each be used?
> 
> 1.0 (at the end) is best for RAID1 when booting from the device as
> the boot-load just doesn't see the RAID at all.
> 
> 1.1 (at the start) is generally good because various different
> things have metadata at the start, so there is no chance of
> confusion about what is using that devices - each possible users
> will overwrite the metadata of the others. It is also easier to
> make the devices larger if you don't need to move the metadata.
> 
> 1.2 (4K from the start) has most of the benefits for 1.1 but works
> for booting with newer boot loaders - they still want sector 0 for
> a boot sector but understand the md superblock.

I see. Makes also loads of sense.

I had two more quick questions:

1.- when would one want to turn off bitmaps? I turned them on, but I
haven't found why one may want them off (or for what they should be
turned off).

2.- if I ever have a mismatch_cnt different than zero, how would I go
about finding which drive has the correct info, which was the one that
got corrupted, and how to obtain the correct one?
In the past I've changed the "CHECK" to "REPAIR" in the configuration
script that checks the RAID arrays periodically
(/etc/sysconfig/raid-check), but from what I understood it's a bit of
luck which one the automatic script will choose out of the two drives
(assuming raid1). Should I let it be automatic (with "repair"), and if
not (if I have to check which one is correct manually), where could I
find information to read about how to do this (I am assuming the
answer may be long)?

Thanks!

Gilbert
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: digital signature
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8NslwACgkQOYVp1RpLsmXUYQCfcAv/q33vjS2oqrxdey2wTEtn
yzwAnAhd7KT8Ye4JYO65MWm0dZZaa3d0
=yOmI
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[NeilBrown]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 11 Jan 2012 11:01:33 -0500 Gilbert Kowarzyk <kowarzyk@grm.polymtl.ca>
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hello,
> 
> >> Why do we have these choices, and when should each be used?
> > 
> > 1.0 (at the end) is best for RAID1 when booting from the device as
> > the boot-load just doesn't see the RAID at all.
> > 
> > 1.1 (at the start) is generally good because various different
> > things have metadata at the start, so there is no chance of
> > confusion about what is using that devices - each possible users
> > will overwrite the metadata of the others. It is also easier to
> > make the devices larger if you don't need to move the metadata.
> > 
> > 1.2 (4K from the start) has most of the benefits for 1.1 but works
> > for booting with newer boot loaders - they still want sector 0 for
> > a boot sector but understand the md superblock.
> 
> I see. Makes also loads of sense.
> 
> I had two more quick questions:
> 
> 1.- when would one want to turn off bitmaps? I turned them on, but I
> haven't found why one may want them off (or for what they should be
> turned off).

There can be a performance impact - writes can be a bit slower with bitmaps.
If that is more important to you that resync speed after a crash you might
turn them off.
To change the chunk-size of the bitmap you need to turn them off, then on
again.
If you want to reshape the array you currently need to turn the bitmap off
first (because the bitmap doesn't automatically resize).
You turn the bitmap off with
   mdadm --grow /dev/mdXX --bitmap=none

> 
> 2.- if I ever have a mismatch_cnt different than zero, how would I go
> about finding which drive has the correct info, which was the one that
> got corrupted, and how to obtain the correct one?

"correct" is not a well defined concept here.

> In the past I've changed the "CHECK" to "REPAIR" in the configuration
> script that checks the RAID arrays periodically
> (/etc/sysconfig/raid-check), but from what I understood it's a bit of
> luck which one the automatic script will choose out of the two drives
> (assuming raid1). Should I let it be automatic (with "repair"), and if
> not (if I have to check which one is correct manually), where could I
> find information to read about how to do this (I am assuming the
> answer may be long)?

This:  http://neil.brown.name/blog/20100211050355
touches on the topic ... and is long.


NeilBrown
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iQIVAwUBTw4Vejnsnt1WYoG5AQL41xAArvFsHCrQjeEBSY1Ce9amr4hoHS34nmtE
jhg8z/VbstAJHqKGGemgE96C87yoKkcJOBARw9UhZtwwX99DUzYerRhkyJnNLp9j
bMdCz7/uFk0r6QWVskT8Ufjz+PZALW00/LD6t/TQWsq3F37D0I6AK5JeHqPgkVmB
+wh67N93eLSTZ8ZqqYP6Jit3Fl8XKqIJl5FJUO9JZB0mtmMTSh0xFNRbddwk73v1
BobI226HJK2OHeWkgN2sNqxZ014kq43nqBsMD11NNvZiJswBhERHnmOEIp3PkoBQ
wleGYC3FNpq7ntc6O2h044ax7D7dLDcjJgNpX+zmYd2QkPVOk0XBsxifAQsG97ct
kqBRRffopH1ARCjEfmGAfVYne9VpkO1QKFjG4Q/ha2151V33ZwRCNRaSR9FEF/jp
Jb+qHOzbd9I6Jr2EpuLeoSowdeOEecBSYjPw/SNGxNHmQBKtz/kvMRAOroL8iUgV
fu9tCNMBfGQyjbzK2ZC4OQAp2jRWdHBIP5IKCl92xUqCC+YM4ei1ajVSFyPtWK0c
SqPYzUX6W6qSHFqHWea+mRZuW63tu3sdXQl+LsNPymR6oBQhy7Kx6J40SBktrVMW
iPDfvcmRjHTCdep/KjYTcWM4tNqzL3ny3b8lP99DpW822nrdT93hyNbrK7GGveiU
Aqmaeg0b8eI=
=cpNq
-----END PGP SIGNATURE-----

Re: Renaming MD devices (metadata=1.1)

[Gilbert Kowarzyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Awesome!

Thanks for the info!

Gilbert


On 12-01-11 6:04 PM, NeilBrown wrote:
> On Wed, 11 Jan 2012 11:01:33 -0500 Gilbert Kowarzyk
> <kowarzyk@grm.polymtl.ca> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
> 
>> Hello,
> 
>>>> Why do we have these choices, and when should each be used?
>>> 
>>> 1.0 (at the end) is best for RAID1 when booting from the device
>>> as the boot-load just doesn't see the RAID at all.
>>> 
>>> 1.1 (at the start) is generally good because various different 
>>> things have metadata at the start, so there is no chance of 
>>> confusion about what is using that devices - each possible
>>> users will overwrite the metadata of the others. It is also
>>> easier to make the devices larger if you don't need to move the
>>> metadata.
>>> 
>>> 1.2 (4K from the start) has most of the benefits for 1.1 but
>>> works for booting with newer boot loaders - they still want
>>> sector 0 for a boot sector but understand the md superblock.
> 
>> I see. Makes also loads of sense.
> 
>> I had two more quick questions:
> 
>> 1.- when would one want to turn off bitmaps? I turned them on,
>> but I haven't found why one may want them off (or for what they
>> should be turned off).
> 
> There can be a performance impact - writes can be a bit slower with
> bitmaps. If that is more important to you that resync speed after a
> crash you might turn them off. To change the chunk-size of the
> bitmap you need to turn them off, then on again. If you want to
> reshape the array you currently need to turn the bitmap off first
> (because the bitmap doesn't automatically resize). You turn the
> bitmap off with mdadm --grow /dev/mdXX --bitmap=none
> 
> 
>> 2.- if I ever have a mismatch_cnt different than zero, how would
>> I go about finding which drive has the correct info, which was
>> the one that got corrupted, and how to obtain the correct one?
> 
> "correct" is not a well defined concept here.
> 
>> In the past I've changed the "CHECK" to "REPAIR" in the
>> configuration script that checks the RAID arrays periodically 
>> (/etc/sysconfig/raid-check), but from what I understood it's a
>> bit of luck which one the automatic script will choose out of the
>> two drives (assuming raid1). Should I let it be automatic (with
>> "repair"), and if not (if I have to check which one is correct
>> manually), where could I find information to read about how to do
>> this (I am assuming the answer may be long)?
> 
> This:  http://neil.brown.name/blog/20100211050355 touches on the
> topic ... and is long.
> 
> 
> NeilBrown
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: digital signature
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8OJIsACgkQOYVp1RpLsmUE5QCfduzOuF9EPCJoUbXWsq0kYhZF
OioAnichYkAyv8r1VLdcMoDSU2aiGF0Q
=44rt
-----END PGP SIGNATURE-----