mdadm issue on Ubuntu Januty 9.04 server

mdadm issue on Ubuntu Januty 9.04 server

[fibre raid]

Hi,

I am facing an issue with mdadm on Ubuntu Jaunty 9.04 Server 64-bit edition.

Host:
- Dual quad-core Xeon 5450 3.0 GHz CPUs
- 16 GB RAM
- Ubuntu 9.04 server (2.6.28-11)
- mdadm 2.6.7.1
- LSI 3801E controller with 12 x Seagate Cheetah 15K.5 300GB drives

I am trying to create a RAID 0 md device from two of the 15K.5 drives.
When I boot my Host, it seems like there is some legacy md data in
/proc/mdstat. However, /etc/mdadm.conf is empty.

root@JAUNTY:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md_d1 : inactive sdx[1](S)
      292968640 blocks

md_d0 : inactive sdj[3](S) sdm[0](S) sdf[7](S) sdg[6](S)
      1171874560 blocks

unused devices: <none>


I proceed to stop and remove these md devices and use /proc/mdstat to
verify that they were removed.

root@JAUNTY:~# mdadm -S /dev/md_d1
mdadm: stopped /dev/md_d1
root@JAUNTY:~# mdadm -S /dev/md_d0
mdadm: stopped /dev/md_d0
root@JAUNTY:~# mdadm -r /dev/md_d1
root@JAUNTY:~# mdadm -r /dev/md_d0
root@JAUNTY:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
unused devices: <none>


Now it appears that the md devices are removed.

I then try to create an md device:

root@JAUNTY:~# mdadm -C -f -e 1.0 -R /dev/md0 -l 0 -n 2 -c 64 /dev/sdd /dev/sdg
mdadm: /dev/sdg appears to be part of a raid array:
    level=raid0 devices=8 ctime=Sun Mar 22 23:22:03 2009
mdadm: array /dev/md0 started.
root@JAUNTY:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid0 sdg[1] sdd[0]
      585937408 blocks super 1.0 64k chunks

unused devices: <none>


Why does mdadm report /dev/sdg as part of a raid array? I removed all
the md devices as shown above. Thank you for your help!

Thanks!
-Thomas
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm issue on Ubuntu Januty 9.04 server

[NeilBrown]

On Tue, May 12, 2009 11:32 am, fibre raid wrote:
> Hi,
>
> I am facing an issue with mdadm on Ubuntu Jaunty 9.04 Server 64-bit
> edition.
>
> Host:
> - Dual quad-core Xeon 5450 3.0 GHz CPUs
> - 16 GB RAM
> - Ubuntu 9.04 server (2.6.28-11)
> - mdadm 2.6.7.1
> - LSI 3801E controller with 12 x Seagate Cheetah 15K.5 300GB drives
>
> I am trying to create a RAID 0 md device from two of the 15K.5 drives.
> When I boot my Host, it seems like there is some legacy md data in
> /proc/mdstat. However, /etc/mdadm.conf is empty.
>
> root@JAUNTY:~# cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md_d1 : inactive sdx[1](S)
>       292968640 blocks
>
> md_d0 : inactive sdj[3](S) sdm[0](S) sdf[7](S) sdg[6](S)
>       1171874560 blocks
>
> unused devices: <none>
>

Presumably all these devices have md superblocks on them
that make them look like part of some old array.
Why jaunty partially assembled them isn't clear.  Maybe it
is using --incremental and it is waiting for the rest of the
array to appear.
You can look at each device with --examine.  e.g.
  mdadm --examine /dev/sdx

If it reports an array that you don't want to still have, you can
remove the metadata using
  mdadm --zero-superblock /dev/sdx

>
> I proceed to stop and remove these md devices and use /proc/mdstat to
> verify that they were removed.
>
> root@JAUNTY:~# mdadm -S /dev/md_d1
> mdadm: stopped /dev/md_d1
> root@JAUNTY:~# mdadm -S /dev/md_d0
> mdadm: stopped /dev/md_d0
> root@JAUNTY:~# mdadm -r /dev/md_d1
> root@JAUNTY:~# mdadm -r /dev/md_d0


These last two commands do nothing.  They are asking mdadm to remove
some devices from the arrays /dev/md_d1 and then /dev/md_d0,
 but don't list any devices to remove.  As the arrays are already
stopped, there is nothing to remove anyway.


> root@JAUNTY:~# cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> unused devices: <none>
>
>
> Now it appears that the md devices are removed.

"stopped" is the term we normally use.

>
> I then try to create an md device:
>
> root@JAUNTY:~# mdadm -C -f -e 1.0 -R /dev/md0 -l 0 -n 2 -c 64 /dev/sdd
> /dev/sdg
> mdadm: /dev/sdg appears to be part of a raid array:
>     level=raid0 devices=8 ctime=Sun Mar 22 23:22:03 2009
> mdadm: array /dev/md0 started.
> root@JAUNTY:~# cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
> [raid4] [raid10]
> md0 : active raid0 sdg[1] sdd[0]
>       585937408 blocks super 1.0 64k chunks
>
> unused devices: <none>
>
>
> Why does mdadm report /dev/sdg as part of a raid array? I removed all
> the md devices as shown above. Thank you for your help!

Because it still has metadata on it recording the fact that it
was recently part of an array.  Creating a new array using that
device will have erased any old metadata.
You can remove the metadata without creating an array by using
--zero-superblock as described above.

NeilBrown

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm issue on Ubuntu Januty 9.04 server

[fibre raid]

Thank you for the additional insight Neil.

If I use partitions for my md members (like /dev/sdd1 and /dev/sdg1),
do I run the --zero-superblock on /dev/sdd1 or /dev/sdd for example?

Finally, I am using Superblock v1.0 here, but in reviewing the mdadm
thread archives, it seems that v1.2 is more commonly recommended. Is
there any reason to use one or the other?

Thanks!
-Thomas


On Mon, May 11, 2009 at 7:01 PM, NeilBrown <neilb@suse.de> wrote:
> On Tue, May 12, 2009 11:32 am, fibre raid wrote:
>> Hi,
>>
>> I am facing an issue with mdadm on Ubuntu Jaunty 9.04 Server 64-bit
>> edition.
>>
>> Host:
>> - Dual quad-core Xeon 5450 3.0 GHz CPUs
>> - 16 GB RAM
>> - Ubuntu 9.04 server (2.6.28-11)
>> - mdadm 2.6.7.1
>> - LSI 3801E controller with 12 x Seagate Cheetah 15K.5 300GB drives
>>
>> I am trying to create a RAID 0 md device from two of the 15K.5 drives.
>> When I boot my Host, it seems like there is some legacy md data in
>> /proc/mdstat. However, /etc/mdadm.conf is empty.
>>
>> root@JAUNTY:~# cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> md_d1 : inactive sdx[1](S)
>>       292968640 blocks
>>
>> md_d0 : inactive sdj[3](S) sdm[0](S) sdf[7](S) sdg[6](S)
>>       1171874560 blocks
>>
>> unused devices: <none>
>>
>
> Presumably all these devices have md superblocks on them
> that make them look like part of some old array.
> Why jaunty partially assembled them isn't clear.  Maybe it
> is using --incremental and it is waiting for the rest of the
> array to appear.
> You can look at each device with --examine.  e.g.
>  mdadm --examine /dev/sdx
>
> If it reports an array that you don't want to still have, you can
> remove the metadata using
>  mdadm --zero-superblock /dev/sdx
>
>>
>> I proceed to stop and remove these md devices and use /proc/mdstat to
>> verify that they were removed.
>>
>> root@JAUNTY:~# mdadm -S /dev/md_d1
>> mdadm: stopped /dev/md_d1
>> root@JAUNTY:~# mdadm -S /dev/md_d0
>> mdadm: stopped /dev/md_d0
>> root@JAUNTY:~# mdadm -r /dev/md_d1
>> root@JAUNTY:~# mdadm -r /dev/md_d0
>
>
> These last two commands do nothing.  They are asking mdadm to remove
> some devices from the arrays /dev/md_d1 and then /dev/md_d0,
>  but don't list any devices to remove.  As the arrays are already
> stopped, there is nothing to remove anyway.
>
>
>> root@JAUNTY:~# cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> unused devices: <none>
>>
>>
>> Now it appears that the md devices are removed.
>
> "stopped" is the term we normally use.
>
>>
>> I then try to create an md device:
>>
>> root@JAUNTY:~# mdadm -C -f -e 1.0 -R /dev/md0 -l 0 -n 2 -c 64 /dev/sdd
>> /dev/sdg
>> mdadm: /dev/sdg appears to be part of a raid array:
>>     level=raid0 devices=8 ctime=Sun Mar 22 23:22:03 2009
>> mdadm: array /dev/md0 started.
>> root@JAUNTY:~# cat /proc/mdstat
>> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
>> [raid4] [raid10]
>> md0 : active raid0 sdg[1] sdd[0]
>>       585937408 blocks super 1.0 64k chunks
>>
>> unused devices: <none>
>>
>>
>> Why does mdadm report /dev/sdg as part of a raid array? I removed all
>> the md devices as shown above. Thank you for your help!
>
> Because it still has metadata on it recording the fact that it
> was recently part of an array.  Creating a new array using that
> device will have erased any old metadata.
> You can remove the metadata without creating an array by using
> --zero-superblock as described above.
>
> NeilBrown
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mdadm issue on Ubuntu Januty 9.04 server

[NeilBrown]

On Tue, May 12, 2009 12:07 pm, fibre raid wrote:
> Thank you for the additional insight Neil.
>
> If I use partitions for my md members (like /dev/sdd1 and /dev/sdg1),
> do I run the --zero-superblock on /dev/sdd1 or /dev/sdd for example?

You use --zero-superblock on the same thing which, when used with
--examine, shows details of a raid array that you don't want.

> Finally, I am using Superblock v1.0 here, but in reviewing the mdadm
> thread archives, it seems that v1.2 is more commonly recommended. Is
> there any reason to use one or the other?

1.0 has the metadata at the end of the device
1.1 has it at the very beginning
1.2 has it just after the beginning of the device.

If you are making a raid1 and might in some case want to mount
just one of the devices without having md/raid in the way, then
1.0 is needed.
If the partition is a logical volume that you might want to grow,
then 1.1 or 1.2 is probably best.
There are probably other situations where one is clearly better than
another.  I don't have a full list though.

NeilBrown

Re: mdadm issue on Ubuntu Januty 9.04 server

[John Hughes]

NeilBrown wrote:
> that make them look like part of some old array.
> Why jaunty partially assembled them isn't clear.  Maybe it
> is using --incremental and it is waiting for the rest of the
> array to appear.
>   
In my experience mdadm --incremental just doesn't work on Jaunty.

When you run it on the first disk in the raid it sets the raid up as 
waiting for the rest, creates the /dev/md/xxx device, all looks good.

When you try the second disk (I have a raid-1) it complains that the 
/dev/md/xxx exists and does nothing.

Results - exactly as "fibre raid" complains - you're left with a partly 
set up inactive raid.

Haven't got around to doing a bugreport yet as I don't want to mess with 
my running server.

Re: mdadm issue on Ubuntu Januty 9.04 server

[fibre raid]

Hi all,

I updated my Jaunty installation to mdadm 2.6.9 and am noticing an
issue that I thought was solved in a prior release.

root@NIMBUS2cc0:~# mdadm --assemble /dev/md0
mdadm: metadata format 1.02 unknown, ignored.
mdadm: /dev/md0 has been started with 12 drives.


Why is it reporting the format 1.02 unknown, ignored? I have used the
-e 1.2 superblock command when creating the array, so I am not sure
where 1.02 is coming from.


root@NIMBUS2cc0:~# mdadm -D /dev/md0
mdadm: metadata format 1.02 unknown, ignored.
/dev/md0:
        Version : 1.02
  Creation Time : Wed May 13 18:49:27 2009
     Raid Level : raid0
     Array Size : 5860601856 (5589.11 GiB 6001.26 GB)
   Raid Devices : 12
  Total Devices : 12
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Wed May 13 18:49:27 2009
          State : clean
 Active Devices : 12
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 0

     Chunk Size : 512K

           Name : 0
           UUID : 3d67940f:afb6bf35:1836ba37:ef15a910
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       8       33        1      active sync   /dev/sdc1
       2       8       49        2      active sync   /dev/sdd1
       3       8       65        3      active sync   /dev/sde1
       4       8       81        4      active sync   /dev/sdf1
       5       8       97        5      active sync   /dev/sdg1
       6       8      113        6      active sync   /dev/sdh1
       7       8      129        7      active sync   /dev/sdi1
       8       8      145        8      active sync   /dev/sdj1
       9       8      161        9      active sync   /dev/sdk1
      10       8      177       10      active sync   /dev/sdl1
      11       8      193       11      active sync   /dev/sdm1


thanks!
-Thomas


On Wed, May 13, 2009 at 6:14 AM, John Hughes <john@calva.com> wrote:
> NeilBrown wrote:
>>
>> that make them look like part of some old array.
>> Why jaunty partially assembled them isn't clear.  Maybe it
>> is using --incremental and it is waiting for the rest of the
>> array to appear.
>>
>
> In my experience mdadm --incremental just doesn't work on Jaunty.
>
> When you run it on the first disk in the raid it sets the raid up as waiting
> for the rest, creates the /dev/md/xxx device, all looks good.
>
> When you try the second disk (I have a raid-1) it complains that the
> /dev/md/xxx exists and does nothing.
>
> Results - exactly as "fibre raid" complains - you're left with a partly set
> up inactive raid.
>
> Haven't got around to doing a bugreport yet as I don't want to mess with my
> running server.
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html