Moving a RAID5 array

Moving a RAID5 array

[Tim Harvey]

Greetings,

I have an Ateonix NASAS 2040 NAS device with a RAID5 array composed of 4
300GB Maxtor IDE disks.  The NASAS 2040 is a std PC motherboard with a
PCI 'disk on flash' module which contains a linux OS that runs the
system.  Therefore the RAID array is /dev/hda1, /dev/hdb1, /dev/hdc1,
/dev/hdd1.  In my case, the flash based OS crashed due to a power glitch
while the system was recovering from a previous power glitch (single
point of failure was pretty easy to hit unfortunately).

Seeing as I know the NAS uses Linux, I figured perhaps I could use
another linux system to mount the array on, intact.  I have taken 1 of
the drives, installed it as hdc in a Linux FC1 system and examined it
with fdisk:

[root@masterbackend root]# fdisk /dev/hdc

The number of cylinders for this disk is set to 36473.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/hdc: 300.0 GB, 300001443840 bytes
255 heads, 63 sectors/track, 36473 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot    Start       End    Blocks   Id  System
/dev/hdc1             1     36217 290913052   fd  Linux raid autodetect
/dev/hdc2         36218     36281    514080   83  Linux
/dev/hdc3         36282     36473   1542240   83  Linux

Command (m for help):q
[root@masterbackend root]#


Question:  What would the hdc2 and hdc3 partitions be?  I then examined
it using mdadm:

used 'mdadm --examine /dev/hdc1':

[root@masterbackend root]# mdadm --examine /dev/hdc1
/dev/hdc1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 979016d8:1ce73ff7:3251d40a:a184400c
  Creation Time : Sat Sep 13 21:40:22 2003
     Raid Level : raid5
    Device Size : 290912960 (277.44 GiB 297.89 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0

    Update Time : Wed May 26 16:16:36 2004
          State : dirty, no-errors
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0
       Checksum : c2e94cf6 - correct
         Events : 0.24

         Layout : left-symmetric
     Chunk Size : 32K

      Number   Major   Minor   RaidDevice State
this     0       3        1        0      active sync   /dev/hda1
   0     0       3        1        0      active sync   /dev/hda1
   1     1       3       65        1      active sync   /dev/hdb1
   2     2      22        1        2      active sync   /dev/hdc1
   3     3      22       65        3      active sync   /dev/hdd1
[root@masterbackend root]#


Luckily the superblock seems ligit, so I think I can safely mount it.
 
My question is that, not being very familiar with Linux raid or mdadm,
I'm wondering if I can mount the array on my Linux system when the
devices are other than described above (they would likely be /dev/hdc1,
/dev/hdd1, etc as my boot/root drives use hda/hdb).  Additionally, how
should I go about mounting this array in a way that if anything doesn't
work right, it doesn't trash the array.  I'm a little paranoid that I
may do something wrong and the raidutils will start reconstructing or
otherwise altering the information on the array.

Thanks for any help.  I've read over the Linux RAID FAQ and looked
through the man pages for mdadm, but I'm pretty paranoid about loosing
the data and want to consult the experts here.

Thanks for any help,

Tim

Re: Moving a RAID5 array

[maarten van den Berg]

On Friday 28 May 2004 00:04, Tim Harvey wrote:

Disclaimer: I'm not a real expert, but have had my share of raid5 experiences.

> My question is that, not being very familiar with Linux raid or mdadm,
> I'm wondering if I can mount the array on my Linux system when the
> devices are other than described above (they would likely be /dev/hdc1,
> /dev/hdd1, etc as my boot/root drives use hda/hdb).  Additionally, how

I did shuffle my disks at several points in time, and it figures that out by 
itself somehow (there must be something in the superblock identifying which 
drive it is). 

> should I go about mounting this array in a way that if anything doesn't
> work right, it doesn't trash the array.  I'm a little paranoid that I
> may do something wrong and the raidutils will start reconstructing or
> otherwise altering the information on the array.

There are several things to note. You want to disallow the raid to start any 
reconstruction, and you need to skip any fsck at mount, as well as replaying 
of journals if they are journaling filesystems.

Issue #1 Can be assured (AFAIK) by just attaching one disk less. Then the 
array will be in degraded mode and cannot / will not attempt a resync.
In addition, this will enable you to choose another set of disks, should the 
outlook on the first three be less than favourable...

Issue #2 Should be assured by mounting read-only, without prior fsck.

Issue #3 I'm not sure about. Consult the manualpage detailing your specific 
filesystem...  Maybe instead of mounting you could run an fsck with an option 
to do only a test-run / dry-run i.e. allow no writes whatsoever during fsck. 
If that fsck succeeds, chances are very good that all is in order.

Anyway, my guess is, you'll have no trouble at all.  A power-loss is not the 
very worst that can happen to an array. In my humble opinion...

> Thanks for any help.  I've read over the Linux RAID FAQ and looked
> through the man pages for mdadm, but I'm pretty paranoid about loosing
> the data and want to consult the experts here.

<Grin> No backups huh ?  Been there,  done that,  still got my data back. :)

Well, good luck with your endeavors !

Maarten

> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
When I answered where I wanted to go today, they just hung up -- Unknown

RE: Moving a RAID5 array

[Tim Harvey]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of maarten van den Berg
> Sent: Thursday, May 27, 2004 4:20 PM
> To: linux-raid@vger.kernel.org
> Subject: Re: Moving a RAID5 array
> 
> On Friday 28 May 2004 00:04, Tim Harvey wrote:
> 
> Disclaimer: I'm not a real expert, but have had my share of raid5
> experiences.
> 
> > My question is that, not being very familiar with Linux raid or
mdadm,
> > I'm wondering if I can mount the array on my Linux system when the
> > devices are other than described above (they would likely be
/dev/hdc1,
> > /dev/hdd1, etc as my boot/root drives use hda/hdb).  Additionally,
how
> 
> I did shuffle my disks at several points in time, and it figures that
out
> by
> itself somehow (there must be something in the superblock identifying
> which
> drive it is).
> 
> > should I go about mounting this array in a way that if anything
doesn't
> > work right, it doesn't trash the array.  I'm a little paranoid that
I
> > may do something wrong and the raidutils will start reconstructing
or
> > otherwise altering the information on the array.
> 
> There are several things to note. You want to disallow the raid to
start
> any
> reconstruction, and you need to skip any fsck at mount, as well as
> replaying
> of journals if they are journaling filesystems.

It looks like I need to do an 'mdadm --assemble' to assemble the
previously created array into an active array.  But I'm not clear how to
examine the /dev/md0 device to see what partitions are available (fdisk
didn't work in a test... I guess its not compatible with /dev/md0).
 
> 
> Issue #1 Can be assured (AFAIK) by just attaching one disk less. Then
the
> array will be in degraded mode and cannot / will not attempt a resync.
> In addition, this will enable you to choose another set of disks,
should
> the
> outlook on the first three be less than favourable...
> 

Does disallowing the raid to start any reconstruction mean that the
array is not 'started' and that I cannot mount the filesystem?  I guess
I'm not clear if there is a difference between the array being 'started'
and 'allowing reconstruction'.  Is leaving a disk out of the RAID5 array
the only way to force this to occur?  I noticed mdadm has a --readonly
option but I'm not sure if it affects the assemble mode.

> Issue #2 Should be assured by mounting read-only, without prior fsck.
> 

Sounds like a safe bet to mount the partition read-only.  How can I
examine /dev/md0 to see what partitions are available?  I'm fairly
certain XFS is used.  The RAID HOWTO doesn't seem to go into detail
regarding how to examine filesystems on an array.

> Issue #3 I'm not sure about. Consult the manualpage detailing your
> specific
> filesystem...  Maybe instead of mounting you could run an fsck with an
> option
> to do only a test-run / dry-run i.e. allow no writes whatsoever during
> fsck.
> If that fsck succeeds, chances are very good that all is in order.

Is this to ensure no writing occurs in case something didn't work
correctly?  If everything looks ok in a dry-run I'll want to mount and
allow fsck and journaling to complete I assume.  Again, can the
--readonly option be used to keep the device from being written at the
device level for a dry-run?

> 
> Anyway, my guess is, you'll have no trouble at all.  A power-loss is
not
> the
> very worst that can happen to an array. In my humble opinion...
> 
> > Thanks for any help.  I've read over the Linux RAID FAQ and looked
> > through the man pages for mdadm, but I'm pretty paranoid about
loosing
> > the data and want to consult the experts here.
> 
> <Grin> No backups huh ?  Been there,  done that,  still got my data
back.
> :)

I guess I've been educated :)  Ironically the power outages occurred due
to my own blunders while hooking up a 2nd device to use as a backup.

Thanks for your help!

Tim

> 
> Well, good luck with your endeavors !
> 
> Maarten
> 
> > -
> > To unsubscribe from this list: send the line "unsubscribe
linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> --
> When I answered where I wanted to go today, they just hung up --
Unknown
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Moving a RAID5 array

[Neil Brown]

On Thursday May 27, tharvey@alumni.calpoly.edu wrote:
> 
> It looks like I need to do an 'mdadm --assemble' to assemble the
> previously created array into an active array.  But I'm not clear how to
> examine the /dev/md0 device to see what partitions are available (fdisk
> didn't work in a test... I guess its not compatible with /dev/md0).
>  

/dev/md0 will not have any partitions.  It will just have a
filesystem.
You could try 'fsck -n /dev/md0'  to see if the filesystem is recognised.

> > 
> > Issue #1 Can be assured (AFAIK) by just attaching one disk less. Then
> the
> > array will be in degraded mode and cannot / will not attempt a resync.
> > In addition, this will enable you to choose another set of disks,
> should
> > the
> > outlook on the first three be less than favourable...
> > 
> 
> Does disallowing the raid to start any reconstruction mean that the
> array is not 'started' and that I cannot mount the filesystem?  I guess
> I'm not clear if there is a difference between the array being 'started'
> and 'allowing reconstruction'.  Is leaving a disk out of the RAID5 array
> the only way to force this to occur?  I noticed mdadm has a --readonly
> option but I'm not sure if it affects the assemble mode.

"starting" an array is like mounting a filesystem.  It makes it
accessible.
There is no simple way to start an array but not let it
reconstruct. '--readonly' doesn't do this (though it probably
should). 
Leaving one drive out disables reconstruction simply by making it
impossible.

> 
> > Issue #2 Should be assured by mounting read-only, without prior fsck.
> > 
> 
> Sounds like a safe bet to mount the partition read-only.  How can I
> examine /dev/md0 to see what partitions are available?  I'm fairly
> certain XFS is used.  The RAID HOWTO doesn't seem to go into detail
> regarding how to examine filesystems on an array.

There are no partition tables.  Just a filesystem.  Does xfs have a
check-and-repair tool?  Try that.

NeilBrown

RE: Moving a RAID5 array

[Tim Harvey]

I've put 3 out of the 4 disks of the RAID5 array into a linux PC and
have done:

[root@masterbackend root]# mdadm --assemble /dev/md0 /dev/hde1 /dev/hdf1
/dev/hdg1
mdadm: /dev/md0 has been started with 3 drives (out of 4).
[root@masterbackend root]# more /proc/mdstat
Personalities : [raid5]
read_ahead 1024 sectors
md0 : active raid5 hde1[0] hdg1[2] hdf1[1]
      872738880 blocks level 5, 32k chunk, algorithm 2 [4/3] [UUU_]

unused devices: <none>
 [root@masterbackend root]# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.00
  Creation Time : Fri May 28 19:09:21 2004
     Raid Level : raid5
     Array Size : 872738880 (832.31 GiB 893.68 GB)
    Device Size : 290912960 (277.44 GiB 297.89 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Fri May 28 19:37:59 2004
          State : dirty, no-errors
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 32K

    Number   Major   Minor   RaidDevice State
       0      33        1        0      active sync   /dev/hde1
       1      33       65        1      active sync   /dev/hdf1
       2      34        1        2      active sync   /dev/hdg1
       3       0        0        3      faulty removed
           UUID : 3d673dbc:9d78693a:d8ffca9d:d63912cc
         Events : 0.5
[root@masterbackend root]# fsck -N /dev/md0
fsck 1.34 (25-Jul-2003)
[/sbin/fsck.ext2 (1) -- /dev/md0] fsck.ext2 /dev/md0 
[root@masterbackend root]# mkdir /mnt/array
[root@masterbackend root]# mount /dev/md0 /mnt/array
mount: you must specify the filesystem type
[root@masterbackend root]# mount /dev/md0 /mnt/array -t ext2
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       or too many mounted file systems
[root@masterbackend root]# mount /dev/md0 /mnt/array -t jfs
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       or too many mounted file systems
[root@masterbackend root]# mount /dev/md0 /mnt/array -t xfs
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       or too many mounted file systems
[root@masterbackend root]# mount /dev/md0 /mnt/array -t ext3
mount: wrong fs type, bad option, bad superblock on /dev/md0,
       or too many mounted file systems


Am I doing something wrong regarding mounting /dev/md0?  I believe the
filesystem used is supposed to be JFS, but I can't seem to mount it as
jfs,xfs,ext2, or ext3.  Is there some way I can determine what FS is
used?  Could it be a different version of jfs?

Thanks for any help... I believe I'm close!

Tim

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Neil Brown
> Sent: Thursday, May 27, 2004 10:36 PM
> To: Tim Harvey
> Cc: linux-raid@vger.kernel.org
> Subject: RE: Moving a RAID5 array
> 
> On Thursday May 27, tharvey@alumni.calpoly.edu wrote:
> >
> > It looks like I need to do an 'mdadm --assemble' to assemble the
> > previously created array into an active array.  But I'm not clear
how to
> > examine the /dev/md0 device to see what partitions are available
(fdisk
> > didn't work in a test... I guess its not compatible with /dev/md0).
> >
> 
> /dev/md0 will not have any partitions.  It will just have a
> filesystem.
> You could try 'fsck -n /dev/md0'  to see if the filesystem is
recognised.
> 
> > >
> > > Issue #1 Can be assured (AFAIK) by just attaching one disk less.
Then
> > the
> > > array will be in degraded mode and cannot / will not attempt a
resync.
> > > In addition, this will enable you to choose another set of disks,
> > should
> > > the
> > > outlook on the first three be less than favourable...
> > >
> >
> > Does disallowing the raid to start any reconstruction mean that the
> > array is not 'started' and that I cannot mount the filesystem?  I
guess
> > I'm not clear if there is a difference between the array being
'started'
> > and 'allowing reconstruction'.  Is leaving a disk out of the RAID5
array
> > the only way to force this to occur?  I noticed mdadm has a
--readonly
> > option but I'm not sure if it affects the assemble mode.
> 
> "starting" an array is like mounting a filesystem.  It makes it
> accessible.
> There is no simple way to start an array but not let it
> reconstruct. '--readonly' doesn't do this (though it probably
> should).
> Leaving one drive out disables reconstruction simply by making it
> impossible.
> 
> >
> > > Issue #2 Should be assured by mounting read-only, without prior
fsck.
> > >
> >
> > Sounds like a safe bet to mount the partition read-only.  How can I
> > examine /dev/md0 to see what partitions are available?  I'm fairly
> > certain XFS is used.  The RAID HOWTO doesn't seem to go into detail
> > regarding how to examine filesystems on an array.
> 
> There are no partition tables.  Just a filesystem.  Does xfs have a
> check-and-repair tool?  Try that.
> 
> NeilBrown
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid"
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Moving a raid5 array

[Clive Messer]

I currently have a 4 disk raid5 array which I wish to move to another machine. 
In the current machine it is comprised of sda1/sdb1/sdc1/sdd1. When I move it 
to the new machine the disks will all shift up by 2. ie. the partitions will 
now be sdc1/sdd1/sde1/sdf1. If I 'mdadm --assemble' the array on the new 
machine with the new partition names will md figure it all out or am I going 
to trash my data?

For reference my current mdadm.conf ......

DEVICE /dev/sd*[0-9]
ARRAY /dev/md0 level=raid5 num-devices=4 
UUID=f1949e2c:bec621e4:de55ae15:4f336899

Regards

Clive
-- 
Clive Messer <clive@vacuumtube.org.uk>

Re: Moving a raid5 array

[Neil Brown]

On Tuesday September 12, clive@vacuumtube.org.uk wrote:
> 
> I currently have a 4 disk raid5 array which I wish to move to another machine. 
> In the current machine it is comprised of sda1/sdb1/sdc1/sdd1. When I move it 
> to the new machine the disks will all shift up by 2. ie. the partitions will 
> now be sdc1/sdd1/sde1/sdf1. If I 'mdadm --assemble' the array on the new 
> machine with the new partition names will md figure it all out or am I going 
> to trash my data?
> 
> For reference my current mdadm.conf ......
> 
> DEVICE /dev/sd*[0-9]
> ARRAY /dev/md0 level=raid5 num-devices=4 
> UUID=f1949e2c:bec621e4:de55ae15:4f336899

Providing you don't already have a /dev/md0 on the new machine, which
should work perfectly - no confusion.

NeilBrown

Re: Moving a raid5 array

[Clive Messer]

On Tuesday 12 September 2006 17:21, Neil Brown wrote:
> On Tuesday September 12, clive@vacuumtube.org.uk wrote:
> > I currently have a 4 disk raid5 array which I wish to move to another
> > machine. In the current machine it is comprised of sda1/sdb1/sdc1/sdd1.
> > When I move it to the new machine the disks will all shift up by 2. ie.
> > the partitions will now be sdc1/sdd1/sde1/sdf1. If I 'mdadm --assemble'
> > the array on the new machine with the new partition names will md figure
> > it all out or am I going to trash my data?
> >
> > For reference my current mdadm.conf ......
> >
> > DEVICE /dev/sd*[0-9]
> > ARRAY /dev/md0 level=raid5 num-devices=4
> > UUID=f1949e2c:bec621e4:de55ae15:4f336899
>
> Providing you don't already have a /dev/md0 on the new machine, which
> should work perfectly - no confusion.

Thanks Neil. Appreciate the quick response.

Regards

Clive
-- 
Clive Messer <clive@vacuumtube.org.uk>