Stopping and starting a RAID1 :Invalid argument

Stopping and starting a RAID1 :Invalid argument

[Ian Brown]

Hello,

I am trying to understand and practice RAID1 functionality: for that
purpose I created two partitions on the same device (yes,
I know that it is recommended that the partitions are
better to be on differrent devices when using RAID1, but this
is for testing purposes only ).
My device is a 64 K USB Disk On Key. (on /dev/sdb)

Than I ran:
dd if=/dev/zero of=/dev/sdb bs=1M count=64

then I created 2 primary partitions for RAID1 with fdisk:
/dev/sdb1 and /dev/sdb2.

fdisk -l /dev/sdb shows :

Disk /dev/sdb: 65 MB, 65470464 bytes
3 heads, 42 sectors/track, 1014 cylinders
Units = cylinders of 126 * 512 = 64512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         256       16107   fd  Linux raid autodetect
/dev/sdb2             257         513       16191   fd  Linux raid autodetect

Both Partitions are of type  Linux raid autodetect ("fd").
The partitions are equal in size (256 cylinders each)
Then I created a RAID1 by running:

mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdb2

I got : mdadm: array /dev/md0 started

cat /proc/mdstat shows:

Personalities : [raid1]
md0 : active raid1 sdb2[1] sdb1[0]
      16000 blocks [2/2] [UU]
      [==>..................]  resync = 12.5% (2112/16000)
finish=0.7min speed=301K/sec
unused devices: <none>

I created ext3 fs on /dev/md0 and /dev/sdb1 and /dev/sdb2.

I found out that I can mount /dev/md0 , but I cannot
mount /dev/sdb1 or /dev/sdb2.

when I try : mount /dev/sdb1 /mnt/sdb1/
I get mount: /dev/sdb1 already mounted or /mnt/sdb1/ busy


I found out that if I stop the RAID (and unmount /dev/md0),
I am able to mount /dev/sdb1 and /dev/sdb2. I also found
out that a file I created (on the path where /mnt/md0
was mounted) was mirrored to /dev/sdb1 and /dev/sdb2, as it should
in RAID1.

The problem is that I cannot run again the RAID
after stopping it.

I stopped it by:
mdadm --stop /dev/md0
cat /proc/mdstat
Personalities : [raid1]
unused devices: <none>

I tried to run it again by:
mdadm --run /dev/md0
mdadm: failed to run array /dev/md0: Invalid argument

Why is it so ?

what is the way to stop and than run again a RAID ?
Am I doing something wrong ? any idea?

Regards,

Ian

Re: Stopping and starting a RAID1 :Invalid argument

[Gordon Henderson]

On Sun, 8 Oct 2006, Ian Brown wrote:


> Then I created a RAID1 by running:
>
> mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdb2
>
> I got : mdadm: array /dev/md0 started
>
> cat /proc/mdstat shows:
>
> Personalities : [raid1]
> md0 : active raid1 sdb2[1] sdb1[0]
>       16000 blocks [2/2] [UU]
>       [==>..................]  resync = 12.5% (2112/16000)
> finish=0.7min speed=301K/sec
> unused devices: <none>

So far so good. Nothing out of the ordinary here for your testing
environment.

> I created ext3 fs on /dev/md0 and /dev/sdb1 and /dev/sdb2.

Eeek!

You have created a filesystem on md0, which uses /dev/sdb1 and /dev/sdb2,
THEN you fiddled with the underlying devices, /dev/sdb1 and /dev/sdb2...

What you have done is effectively corrup the filesystem on /dev/md0, and
possibly even wiped out the superblock on the /dev/md0 device. (which may
be why you can't start it again)

Once you have created a RAID array, you only ever deal with the mdX
device. Leave the underlying devices well alone. They are now owned by the
md device driver.

Start again, and don't fiddle with the underlying /dev/sdbX devices. Do
not mkfs them, and do not mount them.

All you need to do is this:

  mdadm --create /dev/md0 -l1 -n2 /dev/sdb{1,2}
  mkfs -t ext3 /dev/md0
  mount /dev/md0 /mnt

and there you have it.

Use
	df -h /mnt
to let you see the size of your new mounted volume - it'll be 30MB or so.


There is a case under RAID-1 where you can mount the underlying devices,
but thats only in an emergency, with the mdX drivers stopped, and you need
to make absulutely sure you know what you are doing if you do this, and if
you mount then read-write, the you must not re-enable and mount the
overlying mdX device as it won't know which of the 2 mirrors is the
current one and you might get old data. ie. it's a one-way process in an
emergency, mount the underlying device, get the data off and re-create
from scratch, and you can only do this with RAID-1 devices.

Gordon

Re: Stopping and starting a RAID1 :Invalid argument

[Ian Brown]

Hello,

>There is a case under RAID-1 where you can mount the >underlying devices,
>but that's only in an emergency,

Ok , thnks, this point is clear now.

So I made another test. This time without mkfs.

Now , I deleted the partitions and created them anew and did
evertyhing else the same ,only
 this time without mkfs.

Still:

stop succeeds :

mdadm --stop /dev/md0

cat /proc/mdstat

 Personalities : [raid1]
unused devices: <none>

but mdadm --run : give an error:

mdadm --run /dev/md0

mdadm: failed to run array /dev/md0: Invalid argument


Any idea why is it so ? can't I start an array after it was stopped ?


Regards,
Ian

On 10/8/06, Gordon Henderson <gordon@drogon.net> wrote:
> On Sun, 8 Oct 2006, Ian Brown wrote:
>
>
> > Then I created a RAID1 by running:
> >
> > mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdb2
> >
> > I got : mdadm: array /dev/md0 started
> >
> > cat /proc/mdstat shows:
> >
> > Personalities : [raid1]
> > md0 : active raid1 sdb2[1] sdb1[0]
> >       16000 blocks [2/2] [UU]
> >       [==>..................]  resync = 12.5% (2112/16000)
> > finish=0.7min speed=301K/sec
> > unused devices: <none>
>
> So far so good. Nothing out of the ordinary here for your testing
> environment.
>
> > I created ext3 fs on /dev/md0 and /dev/sdb1 and /dev/sdb2.
>
> Eeek!
>
> You have created a filesystem on md0, which uses /dev/sdb1 and /dev/sdb2,
> THEN you fiddled with the underlying devices, /dev/sdb1 and /dev/sdb2...
>
> What you have done is effectively corrup the filesystem on /dev/md0, and
> possibly even wiped out the superblock on the /dev/md0 device. (which may
> be why you can't start it again)
>
> Once you have created a RAID array, you only ever deal with the mdX
> device. Leave the underlying devices well alone. They are now owned by the
> md device driver.
>
> Start again, and don't fiddle with the underlying /dev/sdbX devices. Do
> not mkfs them, and do not mount them.
>
> All you need to do is this:
>
>   mdadm --create /dev/md0 -l1 -n2 /dev/sdb{1,2}
>   mkfs -t ext3 /dev/md0
>   mount /dev/md0 /mnt
>
> and there you have it.
>
> Use
>         df -h /mnt
> to let you see the size of your new mounted volume - it'll be 30MB or so.
>
>
> There is a case under RAID-1 where you can mount the underlying devices,
> but thats only in an emergency, with the mdX drivers stopped, and you need
> to make absulutely sure you know what you are doing if you do this, and if
> you mount then read-write, the you must not re-enable and mount the
> overlying mdX device as it won't know which of the 2 mirrors is the
> current one and you might get old data. ie. it's a one-way process in an
> emergency, mount the underlying device, get the data off and re-create
> from scratch, and you can only do this with RAID-1 devices.
>
> Gordon
>

Re: Stopping and starting a RAID1 :Invalid argument

[Jim Buttafuoco]

try to assemble the array instead, --run is trying to start a partially built array, --stop deactivated the array and 
released all resources, so --run will NOT work, use --assemble (-A) instead.

Good luck 
Jim


---------- Original Message -----------
From: "Ian Brown" <ianbrn@gmail.com>
To: "Gordon Henderson" <gordon@drogon.net>
Cc: linux-raid@vger.kernel.org
Sent: Sun, 8 Oct 2006 15:08:56 +0200
Subject: Re: Stopping and starting a RAID1 :Invalid argument

> Hello,
> 
> >There is a case under RAID-1 where you can mount the >underlying devices,
> >but that's only in an emergency,
> 
> Ok , thnks, this point is clear now.
> 
> So I made another test. This time without mkfs.
> 
> Now , I deleted the partitions and created them anew and did
> evertyhing else the same ,only
>  this time without mkfs.
> 
> Still:
> 
> stop succeeds :
> 
> mdadm --stop /dev/md0
> 
> cat /proc/mdstat
> 
>  Personalities : [raid1]
> unused devices: <none>
> 
> but mdadm --run : give an error:
> 
> mdadm --run /dev/md0
> 
> mdadm: failed to run array /dev/md0: Invalid argument
> 
> Any idea why is it so ? can't I start an array after it was stopped ?
> 
> Regards,
> Ian
> 
> On 10/8/06, Gordon Henderson <gordon@drogon.net> wrote:
> > On Sun, 8 Oct 2006, Ian Brown wrote:
> >
> >
> > > Then I created a RAID1 by running:
> > >
> > > mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdb2
> > >
> > > I got : mdadm: array /dev/md0 started
> > >
> > > cat /proc/mdstat shows:
> > >
> > > Personalities : [raid1]
> > > md0 : active raid1 sdb2[1] sdb1[0]
> > >       16000 blocks [2/2] [UU]
> > >       [==>..................]  resync = 12.5% (2112/16000)
> > > finish=0.7min speed=301K/sec
> > > unused devices: <none>
> >
> > So far so good. Nothing out of the ordinary here for your testing
> > environment.
> >
> > > I created ext3 fs on /dev/md0 and /dev/sdb1 and /dev/sdb2.
> >
> > Eeek!
> >
> > You have created a filesystem on md0, which uses /dev/sdb1 and /dev/sdb2,
> > THEN you fiddled with the underlying devices, /dev/sdb1 and /dev/sdb2...
> >
> > What you have done is effectively corrup the filesystem on /dev/md0, and
> > possibly even wiped out the superblock on the /dev/md0 device. (which may
> > be why you can't start it again)
> >
> > Once you have created a RAID array, you only ever deal with the mdX
> > device. Leave the underlying devices well alone. They are now owned by the
> > md device driver.
> >
> > Start again, and don't fiddle with the underlying /dev/sdbX devices. Do
> > not mkfs them, and do not mount them.
> >
> > All you need to do is this:
> >
> >   mdadm --create /dev/md0 -l1 -n2 /dev/sdb{1,2}
> >   mkfs -t ext3 /dev/md0
> >   mount /dev/md0 /mnt
> >
> > and there you have it.
> >
> > Use
> >         df -h /mnt
> > to let you see the size of your new mounted volume - it'll be 30MB or so.
> >
> >
> > There is a case under RAID-1 where you can mount the underlying devices,
> > but thats only in an emergency, with the mdX drivers stopped, and you need
> > to make absulutely sure you know what you are doing if you do this, and if
> > you mount then read-write, the you must not re-enable and mount the
> > overlying mdX device as it won't know which of the 2 mirrors is the
> > current one and you might get old data. ie. it's a one-way process in an
> > emergency, mount the underlying device, get the data off and re-create
> > from scratch, and you can only do this with RAID-1 devices.
> >
> > Gordon
> >
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
------- End of Original Message -------

Re: Stopping and starting a RAID1 :Invalid argument

[Ian Brown]

Hello,

Well, assemble looks like it does the job

mdadm --stop /dev/md0

mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdb2

mdadm: /dev/md0 has been started with 2 drives.

#cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[0] sdb2[1]
      16000 blocks [2/2] [UU]

unused devices: <none>


In the kernel log I see:
"md0: raid array is not clean -- starting background reconstruction"


What is the meaning of "raid array is not clean" ?

Should it appear in a normal stop and assemble action on a raid 1 ?

I hope that this message does not indicate that something
was not done as it should be.


Regards,
IB



On 10/8/06, Jim Buttafuoco <jim@contactbda.com> wrote:
> try to assemble the array instead, --run is trying to start a partially built array, --stop deactivated the array and
> released all resources, so --run will NOT work, use --assemble (-A) instead.
>
> Good luck
> Jim
>
>
> ---------- Original Message -----------
> From: "Ian Brown" <ianbrn@gmail.com>
> To: "Gordon Henderson" <gordon@drogon.net>
> Cc: linux-raid@vger.kernel.org
> Sent: Sun, 8 Oct 2006 15:08:56 +0200
> Subject: Re: Stopping and starting a RAID1 :Invalid argument
>
> > Hello,
> >
> > >There is a case under RAID-1 where you can mount the >underlying devices,
> > >but that's only in an emergency,
> >
> > Ok , thnks, this point is clear now.
> >
> > So I made another test. This time without mkfs.
> >
> > Now , I deleted the partitions and created them anew and did
> > evertyhing else the same ,only
> >  this time without mkfs.
> >
> > Still:
> >
> > stop succeeds :
> >
> > mdadm --stop /dev/md0
> >
> > cat /proc/mdstat
> >
> >  Personalities : [raid1]
> > unused devices: <none>
> >
> > but mdadm --run : give an error:
> >
> > mdadm --run /dev/md0
> >
> > mdadm: failed to run array /dev/md0: Invalid argument
> >
> > Any idea why is it so ? can't I start an array after it was stopped ?
> >
> > Regards,
> > Ian
> >
> > On 10/8/06, Gordon Henderson <gordon@drogon.net> wrote:
> > > On Sun, 8 Oct 2006, Ian Brown wrote:
> > >
> > >
> > > > Then I created a RAID1 by running:
> > > >
> > > > mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdb2
> > > >
> > > > I got : mdadm: array /dev/md0 started
> > > >
> > > > cat /proc/mdstat shows:
> > > >
> > > > Personalities : [raid1]
> > > > md0 : active raid1 sdb2[1] sdb1[0]
> > > >       16000 blocks [2/2] [UU]
> > > >       [==>..................]  resync = 12.5% (2112/16000)
> > > > finish=0.7min speed=301K/sec
> > > > unused devices: <none>
> > >
> > > So far so good. Nothing out of the ordinary here for your testing
> > > environment.
> > >
> > > > I created ext3 fs on /dev/md0 and /dev/sdb1 and /dev/sdb2.
> > >
> > > Eeek!
> > >
> > > You have created a filesystem on md0, which uses /dev/sdb1 and /dev/sdb2,
> > > THEN you fiddled with the underlying devices, /dev/sdb1 and /dev/sdb2...
> > >
> > > What you have done is effectively corrup the filesystem on /dev/md0, and
> > > possibly even wiped out the superblock on the /dev/md0 device. (which may
> > > be why you can't start it again)
> > >
> > > Once you have created a RAID array, you only ever deal with the mdX
> > > device. Leave the underlying devices well alone. They are now owned by the
> > > md device driver.
> > >
> > > Start again, and don't fiddle with the underlying /dev/sdbX devices. Do
> > > not mkfs them, and do not mount them.
> > >
> > > All you need to do is this:
> > >
> > >   mdadm --create /dev/md0 -l1 -n2 /dev/sdb{1,2}
> > >   mkfs -t ext3 /dev/md0
> > >   mount /dev/md0 /mnt
> > >
> > > and there you have it.
> > >
> > > Use
> > >         df -h /mnt
> > > to let you see the size of your new mounted volume - it'll be 30MB or so.
> > >
> > >
> > > There is a case under RAID-1 where you can mount the underlying devices,
> > > but thats only in an emergency, with the mdX drivers stopped, and you need
> > > to make absulutely sure you know what you are doing if you do this, and if
> > > you mount then read-write, the you must not re-enable and mount the
> > > overlying mdX device as it won't know which of the 2 mirrors is the
> > > current one and you might get old data. ie. it's a one-way process in an
> > > emergency, mount the underlying device, get the data off and re-create
> > > from scratch, and you can only do this with RAID-1 devices.
> > >
> > > Gordon
> > >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> ------- End of Original Message -------
>
>

Re: Stopping and starting a RAID1 :Invalid argument

[Neil Brown]

On Monday October 9, ianbrn@gmail.com wrote:
> 
> In the kernel log I see:
> "md0: raid array is not clean -- starting background reconstruction"
> 
> 
> What is the meaning of "raid array is not clean" ?

It means md believe there could be an inconsistency in the array.
This typically happens due to an unclean shutdown.
Can you post more complete logs ?  100 lines either side should be
plenty.

NeilBrown

Re: Stopping and starting a RAID1 :Invalid argument

[Ian Brown]

Neil,

  First, I just want to emphasize that the message "raid array is not
clean" from the kernel log
appeared after running "mdadm create". (and not "mdadm stop" or "mdadm
assemble")
I don't know if it is normal behavior. (Is it ? this is in fact my question).

one little think: I ran here also mkfs -t ext3... but also
tried without it and it was teh same.

Second, here is exactly what I did and the  kernel log :

My actions:

dd if=/dev/zero of=/dev/sda bs=1M count=64
===
create 2 linux raid partitions equal in size:

afterwards:
fdisk -l /dev/sda shows:

Disk /dev/sda: 65 MB, 65470464 bytes
3 heads, 42 sectors/track, 1014 cylinders
Units = cylinders of 126 * 512 = 64512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         256       16107   fd  Linux raid autodetect
/dev/sda2             257         513       16191   fd  Linux raid autodetect

===
modprobe raid1
mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sda1 /dev/sda2

mdadm: array /dev/md0 started.

cat /proc/mdstat

cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda2[1] sda1[0]
      16000 blocks [2/2] [UU]

unused devices: <none>
===

mkfs -t ext3 /dev/md0
mdadm --stop /dev/md0

cat /proc/mdstat
Personalities : [raid1]
unused devices: <none>
---

mdadm --assemble /dev/md0 /dev/sda1 /dev/sda2
mdadm: /dev/md0 has been started with 2 drives.


The kernel log
============
Oct 11 14:43:53  kernel: md: bind<sda1>
Oct 11 14:43:53  kernel: md0: WARNING: sda2 appears to be on the same
physical disk as sda1. True
Oct 11 14:43:53  kernel:      protection against single-disk failure
might be compromised.
Oct 11 14:43:53  kernel: md: bind<sda2>
Oct 11 14:43:53  kernel: md: md0: raid array is not clean -- starting
background reconstruction
Oct 11 14:43:53  kernel: raid1: raid set md0 active with 2 out of 2 mirrors
Oct 11 14:43:53  kernel: md: syncing RAID array md0
Oct 11 14:43:53 : md: minimum _guaranteed_ reconstruction speed: 1000
KB/sec/disc.
Oct 11 14:43:53 : md: using maximum available idle IO bandwidth (but
not more than 200000 KB/sec) for reconstruction.
Oct 11 14:43:53 : md: using 128k window, over a total of 16000 blocks.
Oct 11 14:44:29 : md: md0: sync done.
Oct 11 14:44:34 : RAID1 conf printout:
Oct 11 14:44:34 :  --- wd:2 rd:2
Oct 11 14:44:34 :  disk 0, wo:0, o:1, dev:sda1
Oct 11 14:44:34 :  disk 1, wo:0, o:1, dev:sda2
===

after mdadm stop:
---
Oct 11 14:46:53 : md: md0 stopped.
Oct 11 14:46:53 : md: unbind<sda2>
Oct 11 14:46:53 : md: export_rdev(sda2)
Oct 11 14:46:53 : md: unbind<sda1>
Oct 11 14:46:53 : md: export_rdev(sda1)
====

after mdadm assemble:
---
Oct 11 14:48:15 : md: md0 stopped.
Oct 11 14:48:15 : md: bind<sda2>
Oct 11 14:48:15 : md0: WARNING: sda1 appears to be on the same
physical disk as sda2. True
Oct 11 14:48:15 :      protection against single-disk failure might be
compromised.
Oct 11 14:48:15 : md: bind<sda1>
Oct 11 14:48:15 : raid1: raid set md0 active with 2 out of 2 mirrors


Regards,
Ian



On 10/10/06, Neil Brown <neilb@suse.de> wrote:
> On Monday October 9, ianbrn@gmail.com wrote:
> >
> > In the kernel log I see:
> > "md0: raid array is not clean -- starting background reconstruction"
> >
> >
> > What is the meaning of "raid array is not clean" ?
>
> It means md believe there could be an inconsistency in the array.
> This typically happens due to an unclean shutdown.
> Can you post more complete logs ?  100 lines either side should be
> plenty.
>
> NeilBrown
>