I've made a bit of a mess in my raid setup

I've made a bit of a mess in my raid setup

[Federico Foschini]

I'm using ubuntu 12.04.4 Server and this is my current setup:

My hard disks:

    wax@petra:~$ sudo fdisk -l /dev/sda
    Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
    255 testine, 63 settori/tracce, 182401 cilindri, totale 2930277168
settori
    Unita  = settori di 1 * 512 = 512 byte
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Identificativo disco: 0x00000000

    Dispositivo Boot      Start         End      Blocks   Id  System
    /dev/sda1   *          63  2930272064  1465136001   fd  Autorilevamento
raid di Linux

    wax@petra:~$ sudo fdisk -l /dev/sdb
    Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes
    255 testine, 63 settori/tracce, 182401 cilindri, totale 2930277168
settori
    Unita  = settori di 1 * 512 = 512 byte
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Identificativo disco: 0x00000000

    Dispositivo Boot      Start         End      Blocks   Id  System
    /dev/sdb1              63  2930272064  1465136001   fd  Autorilevamento
raid di Linux

    wax@petra:~$ sudo fdisk -l /dev/sdc
    Disk /dev/sdc: 163.9 GB, 163928604672 bytes
    255 testine, 63 settori/tracce, 19929 cilindri, totale 320173056 settori
    Unita  = settori di 1 * 512 = 512 byte
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Identificativo disco: 0x8f8000b1

    Dispositivo Boot      Start         End      Blocks   Id  System
    /dev/sdc1            2048     3905535     1951744   82  Linux swap /
Solaris
    /dev/sdc2         3907582   320172031   158132225    5  Esteso
    /dev/sdc5   *     3907584   320172031   158132224   83  Linux

    wax@petra:~$ sudo fdisk -l /dev/sdd
    Disk /dev/sdd: 3000.6 GB, 3000592982016 bytes
    255 testine, 63 settori/tracce, 364801 cilindri, totale 5860533168
settori
    Unita  = settori di 1 * 512 = 512 byte
    Sector size (logical/physical): 512 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Identificativo disco: 0x00000000
    Il disco /dev/sdd non contiene una tabella delle partizioni valida

This is the mdadm output:

    wax@petra:~$ cat /proc/mdstat
    Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
    md1 : active raid1 sdd[3] md0[2]
          2930135360 blocks super 1.2 [2/2] [UU]
    md0 : active linear sdb[0] sda[1]
          2930277152 blocks super 1.2 0k rounding
    unused devices: <none>

Why sda1 and sdb1 are market as boot? My boot disk is sdc. Also grub is
installed in sdc AND in sda I think that's an error right?

Why sda and sdb have partitions while sdd doesn't?

Is there something wrong with my setup?


-- 
Federico Foschini.

Re: I've made a bit of a mess in my raid setup

[Rudy Zijlstra]

On 08-03-14 11:33, Federico Foschini wrote:
>
> Why sda1 and sdb1 are market as boot? My boot disk is sdc. Also grub is
> installed in sdc AND in sda I think that's an error right?
>
> Why sda and sdb have partitions while sdd doesn't?
I'd see you re-used disks you already had, and did not delete the 
partitions on them when you re-used them in the raid

Is there something wrong with my setup?

Do you have any reason to think so?
>
Cheers

Rudy

Re: I've made a bit of a mess in my raid setup

[Federico Foschini]

The old system had only sda and sdb in raid1. Now I've made a linear
array with sda and sdb and created a new raid1 array with sdd.

>>Is there something wrong with my setup?
>Do you have any reason to think so?

Yes, because yesterday I've update the kernel and grub complained
about "raid array -1 not found".
Also why sda and sdb have both a partition defined as "linux raid" but
sdd doesn't? Where is the correct way to create an array? If I
remember correctly these are the steps I did:

0) The system was /dev/sda1 /dev/sdb1 in raid1 (dev/md0) with a LVM
partition on top of the raid
1) I created a new degradeted raid 1 /dev/md1 array with /dev/sdd
(without defining any partition)
2) I've mounted /dev/md1 and formatted as ext4. Then I copied
everything from /dev/md0
3) I destroyed the old /dev/md0 and than recreated a new /dev/md0 as a
linear array
4) I've added /dev/md0 on /dev/md1 and the drivers start syncying

Than I've update the kernel and grub complained about "raid array -1".
When I restarted the system a Grub error 17 was showed.
So I booted from an usb key and restored grub (I think I installed it
wrongly on /dev/sda instead of the correct drive /dev/sdc) and the
system booted correctly.

-- 
Federico Foschini.

Re: I've made a bit of a mess in my raid setup

[Rudy Zijlstra]

On 08-03-14 12:16, Federico Foschini wrote:
> The old system had only sda and sdb in raid1. Now I've made a linear
> array with sda and sdb and created a new raid1 array with sdd.
>
>>> Is there something wrong with my setup?
>> Do you have any reason to think so?
> Yes, because yesterday I've update the kernel and grub complained
> about "raid array -1 not found".
> Also why sda and sdb have both a partition defined as "linux raid" but
> sdd doesn't? Where is the correct way to create an array? If I
> remember correctly these are the steps I did:
>
> 0) The system was /dev/sda1 /dev/sdb1 in raid1 (dev/md0) with a LVM
> partition on top of the raid
> 1) I created a new degradeted raid 1 /dev/md1 array with /dev/sdd
> (without defining any partition)
> 2) I've mounted /dev/md1 and formatted as ext4. Then I copied
> everything from /dev/md0
> 3) I destroyed the old /dev/md0 and than recreated a new /dev/md0 as a
> linear array
> 4) I've added /dev/md0 on /dev/md1 and the drivers start syncying
>
> Than I've update the kernel and grub complained about "raid array -1".
> When I restarted the system a Grub error 17 was showed.
> So I booted from an usb key and restored grub (I think I installed it
> wrongly on /dev/sda instead of the correct drive /dev/sdc) and the
> system booted correctly.
>
one more question before i give my best guess

Do you use a initrd to load modules during boot, or do does your kernel 
contain all drivers in itself?

Cheers


Rudy

Re: I've made a bit of a mess in my raid setup

[Federico Foschini]

I think ubuntu kernel has mdadm in itself, but I'm not sure. How can I
check? Remember also my boot disc is sdc and it's not part of any raid

Anyway in addition to my configuration I've posted in my first mail
this is my fstab and mdadm.conf:
# /etc/fstab: static file system information.
#
# Use 'blkid -o value -s UUID' to print the universally unique identifier
# for a device; this may be used with UUID= as a more robust way to name
# devices that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    nodev,noexec,nosuid 0       0
# / was on /dev/sdc5 during installation
UUID=a53356cf-05a3-42d8-a2ec-8a7fd4702e30 /               ext4
errors=remount-ro 0       1
# swap was on /dev/sdc1 during installation
UUID=df505cf8-aac8-42d6-805a-908d5909f8d5 none            swap    sw
           0       0
/dev/md1 /media/dischi  ext4 defaults 0 1

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

ARRAY /dev/md0 metadata=1.2 name=petra:0
UUID=952ad074:4039f24a:2ec1808e:d555a88c
ARRAY /dev/md1 metadata=1.2 name=petra:md1
UUID=03c85017:15cecfb6:13d059bc:a8ff8779

Thank you for your help!

2014-03-08 12:42 GMT+01:00 Rudy Zijlstra <rudy@grumpydevil.homelinux.org>:
> On 08-03-14 12:16, Federico Foschini wrote:
>>
>> The old system had only sda and sdb in raid1. Now I've made a linear
>> array with sda and sdb and created a new raid1 array with sdd.
>>
>>>> Is there something wrong with my setup?
>>>
>>> Do you have any reason to think so?
>>
>> Yes, because yesterday I've update the kernel and grub complained
>> about "raid array -1 not found".
>> Also why sda and sdb have both a partition defined as "linux raid" but
>> sdd doesn't? Where is the correct way to create an array? If I
>> remember correctly these are the steps I did:
>>
>> 0) The system was /dev/sda1 /dev/sdb1 in raid1 (dev/md0) with a LVM
>> partition on top of the raid
>> 1) I created a new degradeted raid 1 /dev/md1 array with /dev/sdd
>> (without defining any partition)
>> 2) I've mounted /dev/md1 and formatted as ext4. Then I copied
>> everything from /dev/md0
>> 3) I destroyed the old /dev/md0 and than recreated a new /dev/md0 as a
>> linear array
>> 4) I've added /dev/md0 on /dev/md1 and the drivers start syncying
>>
>> Than I've update the kernel and grub complained about "raid array -1".
>> When I restarted the system a Grub error 17 was showed.
>> So I booted from an usb key and restored grub (I think I installed it
>> wrongly on /dev/sda instead of the correct drive /dev/sdc) and the
>> system booted correctly.
>>
> one more question before i give my best guess
>
> Do you use a initrd to load modules during boot, or do does your kernel
> contain all drivers in itself?
>
> Cheers
>
>
> Rudy



-- 
Federico Foschini.

Re: I've made a bit of a mess in my raid setup

[Rudy Zijlstra]

On 08-03-14 13:08, Federico Foschini wrote:
> I think ubuntu kernel has mdadm in itself, but I'm not sure. How can I
> check? Remember also my boot disc is sdc and it's not part of any raid
urm... mdadm will never be part of the kernel. It is a separate program. 
It makes use of kernel modules though.

Considering the manner you do not answer my question, i am assuming 
default ubuntu kernel, which means an initrd is used.

>
> Anyway in addition to my configuration I've posted in my first mail
> this is my fstab and mdadm.conf:
> # /etc/fstab: static file system information.
> #
> # Use 'blkid -o value -s UUID' to print the universally unique identifier
> # for a device; this may be used with UUID= as a more robust way to name
> # devices that works even if disks are added and removed. See fstab(5).
> #
> # <file system> <mount point>   <type>  <options>       <dump>  <pass>
> proc            /proc           proc    nodev,noexec,nosuid 0       0
> # / was on /dev/sdc5 during installation
> UUID=a53356cf-05a3-42d8-a2ec-8a7fd4702e30 /               ext4
> errors=remount-ro 0       1
> # swap was on /dev/sdc1 during installation
> UUID=df505cf8-aac8-42d6-805a-908d5909f8d5 none            swap    sw
>             0       0
> /dev/md1 /media/dischi  ext4 defaults 0 1
>
> # mdadm.conf
> #
> # Please refer to mdadm.conf(5) for information about this file.
> #
>
> # by default, scan all partitions (/proc/partitions) for MD superblocks.
> # alternatively, specify devices to scan, using wildcards if desired.
> DEVICE partitions
>
> # auto-create devices with Debian standard permissions
> CREATE owner=root group=disk mode=0660 auto=yes
>
> # automatically tag new arrays as belonging to the local system
> HOMEHOST <system>
>
> # instruct the monitoring daemon where to send mail alerts
> MAILADDR root
>
> ARRAY /dev/md0 metadata=1.2 name=petra:0
> UUID=952ad074:4039f24a:2ec1808e:d555a88c
> ARRAY /dev/md1 metadata=1.2 name=petra:md1
> UUID=03c85017:15cecfb6:13d059bc:a8ff8779
>
> Thank you for your help!

In the default condition, this file is not only on your bootdisk, it is 
also in the initrd. I would expect that you updated your HDD copy, but 
not the one contained in the initrd. This leads to the error during boot 
that you see.

As the raid is not needed for the boot, no harm is done and the raid 
simply starts a bit later.

To solve this, you need to update the initrd. Please google for this, i 
do not use ubuntu so am not the best to give instructions on it.
>
> 0) The system was /dev/sda1 /dev/sdb1 in raid1 (dev/md0) with a LVM
> partition on top of the raid
> 1) I created a new degradeted raid 1 /dev/md1 array with /dev/sdd
> (without defining any partition)
> 2) I've mounted /dev/md1 and formatted as ext4. Then I copied
> everything from /dev/md0
> 3) I destroyed the old /dev/md0 and than recreated a new /dev/md0 as a
> linear array
> 4) I've added /dev/md0 on /dev/md1 and the drivers start syncying
>
> Than I've update the kernel and grub complained about "raid array -1".
> When I restarted the system a Grub error 17 was showed.
> So I booted from an usb key and restored grub (I think I installed it
> wrongly on /dev/sda instead of the correct drive /dev/sdc) and the
> system booted correctly.
>
>

Most likely your original raid did use partitions. Then when you 
re-configured you did not delete those partitions, but simply used the 
full disk.

This is the most likely explanation for the state you are now finding. A 
bit confusing, but no harm in itself.

To solve the error during boot, update your initrd

Cheers


Rudy

Re: I've made a bit of a mess in my raid setup

[Can Jeuleers]

On 03/08/2014 01:44 PM, Rudy Zijlstra wrote:
> To solve this, you need to update the initrd. Please google for this, i
> do not use ubuntu so am not the best to give instructions on it.

sudo update-initramfs -u -k all

Re: I've made a bit of a mess in my raid setup

[Federico Foschini]

Thank you for your help. Unfortunatly the server rebooted before I
could run the update-initramfs command and now this is my current
state:

root@petra:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5]
[raid4] [raid10]
md1 : inactive sdd[3](S)
      2930135512 blocks super 1.2

md0 : active linear sdb[0] sda[1]
      2930277152 blocks super 1.2 0k rounding

unused devices: <none>

root@petra:~# mdadm --examine --scan
ARRAY /dev/md0 UUID=e151ea2f:37b942a4:e368bf24:bd0fce41
ARRAY /dev/md/0 metadata=1.2 UUID=952ad074:4039f24a:2ec1808e:d555a88c
name=petra:0
ARRAY /dev/md/md1 metadata=1.2
UUID=03c85017:15cecfb6:13d059bc:a8ff8779 name=petra:md1

root@petra:~# mdadm --examine /dev/md0
/dev/md0:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 03c85017:15cecfb6:13d059bc:a8ff8779
           Name : petra:md1  (local to host petra)
  Creation Time : Sun Feb 16 12:21:56 2014
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 5860292160 (2794.41 GiB 3000.47 GB)
     Array Size : 2930135360 (2794.39 GiB 3000.46 GB)
  Used Dev Size : 5860270720 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : ae9da3ed:c4d422cb:06cf927d:b75bfc36

    Update Time : Sat Mar  8 18:14:00 2014
       Checksum : 51e09d97 - correct
         Events : 74578


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing)

What is going on? Why /dev/md1 is now inactive and /dev/sdd is busy?
And i don't understand why when I examine /dev/md0 the "Name" is
"petra:md1". Also in "--examine --scan" it says I have /dev/md0
/dev/md/0 and /dev/md/1 when on the filesystem only exist /dev/md0 and
/dev/md1



2014-03-08 20:25 GMT+01:00 Can Jeuleers <can.jeuleers@gmail.com>:
> On 03/08/2014 01:44 PM, Rudy Zijlstra wrote:
>> To solve this, you need to update the initrd. Please google for this, i
>> do not use ubuntu so am not the best to give instructions on it.
>
> sudo update-initramfs -u -k all
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Federico Foschini.

Re: I've made a bit of a mess in my raid setup

[Can Jeuleers]

On 03/08/2014 09:23 PM, Federico Foschini wrote:
> What is going on? Why /dev/md1 is now inactive and /dev/sdd is busy?

I don't know Federico, but have you tried updating your initrd and
rebooting? So that the assembly that takes place at boot uses the same
configuration as it would post-boot?

Re: I've made a bit of a mess in my raid setup

[Federico Foschini]

Done and rebooted. I'm in the same condition as before. Also when I
update the initrd a message said "/dev/md1 doesn't appear to be
active"

2014-03-09 8:58 GMT+01:00 Can Jeuleers <can.jeuleers@gmail.com>:
> On 03/08/2014 09:23 PM, Federico Foschini wrote:
>> What is going on? Why /dev/md1 is now inactive and /dev/sdd is busy?
>
> I don't know Federico, but have you tried updating your initrd and
> rebooting? So that the assembly that takes place at boot uses the same
> configuration as it would post-boot?
>



-- 
Federico Foschini.

Re: I've made a bit of a mess in my raid setup

[Federico Foschini]

Hello,
I've consolidated the state of the system, these are the details:

http://pastebin.com/k7wGpw8F

Notice how /dev/md0 has the Name : petra:md1 and /dev/md1 doesn't have
any superblock. Also it appears there is an old mdadm superblock on
/dev/sda1 with my previous array configuration.

So now what I'm trying to achive is this configuration: removing the
hold partitions /dev/sda1 and /dev/sdb1 with the old superblock (by
the way how is it possible i'm using an array with /dev/sda and
/dev/sdb while these devices contains an old partition?) Configuring
correctly the raid 1 array with /dev/sdd and /dev/md0

2014-03-09 9:31 GMT+01:00 Federico Foschini <undicizeri@gmail.com>:
> Done and rebooted. I'm in the same condition as before. Also when I
> update the initrd a message said "/dev/md1 doesn't appear to be
> active"
>
> 2014-03-09 8:58 GMT+01:00 Can Jeuleers <can.jeuleers@gmail.com>:
>> On 03/08/2014 09:23 PM, Federico Foschini wrote:
>>> What is going on? Why /dev/md1 is now inactive and /dev/sdd is busy?
>>
>> I don't know Federico, but have you tried updating your initrd and
>> rebooting? So that the assembly that takes place at boot uses the same
>> configuration as it would post-boot?
>>
>
>
>
> --
> Federico Foschini.



-- 
Federico Foschini.

Re: I've made a bit of a mess in my raid setup

[Rudy Zijlstra]

On 09-03-14 11:34, Federico Foschini wrote:
> Hello,
> I've consolidated the state of the system, these are the details:
>
> http://pastebin.com/k7wGpw8F
>
> Notice how /dev/md0 has the Name : petra:md1 and /dev/md1 doesn't have
> any superblock. Also it appears there is an old mdadm superblock on
> /dev/sda1 with my previous array configuration.
>
> So now what I'm trying to achive is this configuration: removing the
> hold partitions /dev/sda1 and /dev/sdb1 with the old superblock (by
> the way how is it possible i'm using an array with /dev/sda and
> /dev/sdb while these devices contains an old partition?) Configuring
> correctly the raid 1 array with /dev/sdd and /dev/md0
>
Your old raid config used not the whole disk, but partitions on the 
disks. That means the location of the md superblock is different. Thus 
you can create the above situation.

Do you have data on the raid? If not, or if easily re-created, it might 
be best to ensure all old traces are gone before rebuilding.

There are several ways to wipe old superblocks, or a partition table you 
no longer want. For partition table, use you favourite partition program 
and let it create a new table. Write the table out without any partitions.

to wipe old mdadm superblocks,  mdadm /dev/sda1 --zero-superblock, and 
repeat for each partition where you have an old superblock

Cheers

Re: I've made a bit of a mess in my raid setup

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

On Sun, 9 Mar 2014 11:34:56 +0100 Federico Foschini <undicizeri@gmail.com>
wrote:

> Hello,
> I've consolidated the state of the system, these are the details:
> 
> http://pastebin.com/k7wGpw8F

(why do people insist on use pastebin rather than just including all the info
directly in the email !?!?!)

Based on this info the first thing to do it
  mdadm /dev/md1 --add /dev/md0

and wait for it to rebuild.

Then you need to make sure it starts properly on reboot...

What do you  have  in
   /etc/mdadm.conf
(or maybe /etc/mdadm/mdadm.conf on ubuntu) ??

What is in that file in the initrd?

 zcat /boot/initrd | cpio -i --to-stdout etc/mdadm.conf

md0 needs to be listed before md1.  If not you should  change that.

However it could just be that Ubuntu's boot process is broken for multi-level
arrays - it is easy to get wrong and not often tested.


> 
> Notice how /dev/md0 has the Name : petra:md1 and /dev/md1 doesn't have
> any superblock. Also it appears there is an old mdadm superblock on
> /dev/sda1 with my previous array configuration.

md1 does not have a superblock because it is not part of any array.
The components that make up md1 have superblocks.

As suggested  by others,
  mdadm --zero-super /dev/sda1
will remove that old superblock.

You can also remove the partition tables from /dev/sd[ab]1 with
  dd if=/dev/zero count=1 > /dev/sda1
etc.

NeilBrown

> 
> So now what I'm trying to achive is this configuration: removing the
> hold partitions /dev/sda1 and /dev/sdb1 with the old superblock (by
> the way how is it possible i'm using an array with /dev/sda and
> /dev/sdb while these devices contains an old partition?) Configuring
> correctly the raid 1 array with /dev/sdd and /dev/md0
> 
> 2014-03-09 9:31 GMT+01:00 Federico Foschini <undicizeri@gmail.com>:
> > Done and rebooted. I'm in the same condition as before. Also when I
> > update the initrd a message said "/dev/md1 doesn't appear to be
> > active"
> >
> > 2014-03-09 8:58 GMT+01:00 Can Jeuleers <can.jeuleers@gmail.com>:
> >> On 03/08/2014 09:23 PM, Federico Foschini wrote:
> >>> What is going on? Why /dev/md1 is now inactive and /dev/sdd is busy?
> >>
> >> I don't know Federico, but have you tried updating your initrd and
> >> rebooting? So that the assembly that takes place at boot uses the same
> >> configuration as it would post-boot?
> >>
> >
> >
> >
> > --
> > Federico Foschini.
> 
> 
> 


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]