Can you IMAGE Mirrored OS Drives?

Can you IMAGE Mirrored OS Drives?

[andy liebman]

Okay, I figured out how to take my existing OS installation from a 
single SATA drive and put it on a set of mirrored partitions on two 
different SATA drives. I looked at ALL of the "recipies" I could find on 
the Web -- none of them were quite right for MY situtation.

I will post my recipe here tomorrow for others to see -- maybe it will 
help somebody else. I just want to edit a few things to clarify before I 
post it. Some of the gotchas were were amusing -- like I couldn't run 
LILO on degraded raid1 arrays, and running "mkinitrd" from within 
Mandriva 2006 produces gzipped initrd files, but running "mkinitrd" by 
chrooting in from a LIVE CD produces initrd files that needs cpio to 
decompress and read it.

So, now that I have a running mirrored OS, I have a few more questions.

-- If I were to create disk images of EACH drive (i.e., /dev/sda and 
/dev/sdb), could I restore each of those images to NEW drives -- with 
all of their respective partitions -- and have a working RAIDED OS?  I 
ask because my ultimate goal is to put a RAIDED OS on many systems, and 
once I get ONE working, it would be nice to clone them the way I already 
clone SINGLE OS drives. Can you clone RAIDS? It would mean, of course, 
that the UUIDs would be the same on each system. Is that bad? Is there 
any risk here?

-- I know there was a lot of discussion on the list a while back about 
whether or not to mirror SWAP space. Was there ever any conclusion? It 
seems to me that mirroring will ensure that if one drive fails, you 
don't lose information in virtual memory. My applications rarely if 
every use virtual memory anyway so I'm not worried about speed. I want 
the ultimate in stabilty for the OS drive. Otherwise, I guess I would 
designate swap space on just one drive (but prepare identical partitions 
on each for swap). If the first drive failed (/dev/sda -- with the swap 
space), /dev/sdb would become the new /dev/sda anyway. And so fstab 
would be correct in pointing to /dev/sda.

Sound opinions welcome.

Thanks,
Andy Liebman

RE: Can you IMAGE Mirrored OS Drives?

[Guy]

} 
} -- I know there was a lot of discussion on the list a while back about
} whether or not to mirror SWAP space. Was there ever any conclusion? It
} seems to me that mirroring will ensure that if one drive fails, you
} don't lose information in virtual memory. My applications rarely if
} every use virtual memory anyway so I'm not worried about speed. I want
} the ultimate in stabilty for the OS drive. Otherwise, I guess I would
} designate swap space on just one drive (but prepare identical partitions
} on each for swap). If the first drive failed (/dev/sda -- with the swap
} space), /dev/sdb would become the new /dev/sda anyway. And so fstab
} would be correct in pointing to /dev/sda.

You should mirror swap space!  Otherwise you will have an outage if a disk
fails.  Most places would not accept an outage for a simple disk failure.
IMO.

Guy

} 
} Sound opinions welcome.
} 
} Thanks,
} Andy Liebman
} -
} To unsubscribe from this list: send the line "unsubscribe linux-raid" in
} the body of a message to majordomo@vger.kernel.org
} More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can you IMAGE Mirrored OS Drives?

[Gordon Henderson]

On Tue, 15 Aug 2006, andy liebman wrote:

> -- If I were to create disk images of EACH drive (i.e., /dev/sda and
> /dev/sdb), could I restore each of those images to NEW drives -- with
> all of their respective partitions -- and have a working RAIDED OS?  I
> ask because my ultimate goal is to put a RAIDED OS on many systems, and
> once I get ONE working, it would be nice to clone them the way I already
> clone SINGLE OS drives. Can you clone RAIDS?

Yes.

I did this some time back to create half a dozen identical systems. I did
it with IDE drives, but I don't see why it won't work with SCSI/SATA.

The first thing you need is to make sure the partition tables are
identical on both drives. (you probably don't, but it'll be much easier)

So I started with my good system, and had a copy of the partition table on
hda:

  # sfdisk -d /dev/hda > ~hda.part

Made sure all partitions were insync, then shut it down and remove hdb,
and replace it with a new blank disk of the same size.

It booted just fine, with all partitions degraded.

Put the parittion table on the new blank hdb:

  # sfdisk /dev/hdb < ~/hda.part

then I hot-added each partition. eg:

  # mdadm --add /dev/md1 /dev/hdb1

and so on.

And that's it.

you have in your hand a working disk to put (as hda) into a new system, so
just put in in and repeat the above, then you have 2 working systems. Shut
them down and remove the hdb's and you have 2 degraded systems and 2 new
masters to repeat the process.

It's time consuming, but it will work.

One thing you will need to do is change the hostname on each new system
you bring up, and IP address (if not by DHCP).

> It would mean, of course,
> that the UUIDs would be the same on each system. Is that bad? Is there
> any risk here?

Probably not.

> -- I know there was a lot of discussion on the list a while back about
> whether or not to mirror SWAP space. Was there ever any conclusion?

If you don't mirror (or RAID) swap, and a sector in the swap partition
goes faulty, then it's game over, so if you have swap then it must be
raided. I've put swap on RAID1/5 and 6. It might be sub-optimal on RAID-5
and 6, but hey ...

Gordon

Re: Can you IMAGE Mirrored OS Drives?

[Luca Berra]

On Tue, Aug 15, 2006 at 06:55:09PM -0400, andy liebman wrote:
>I will post my recipe here tomorrow for others to see -- maybe it will 
>help somebody else. I just want to edit a few things to clarify before I 
>post it. Some of the gotchas were were amusing -- like I couldn't run 
>LILO on degraded raid1 arrays, and running "mkinitrd" from within 
damn, could you open a bug on qa.mandriva.com, so we will remember to
fix it?
>Mandriva 2006 produces gzipped initrd files, but running "mkinitrd" by 
>chrooting in from a LIVE CD produces initrd files that needs cpio to 
>decompress and read it.
actually up to kernel 2.6.12-18mdk we defaulted to gzipped cramfs
initrd. now the default is initramfs.

>clone SINGLE OS drives. Can you clone RAIDS? It would mean, of course, 
you can certainly clone raids.

>that the UUIDs would be the same on each system. Is that bad? Is there 
>any risk here?
The only risk is if you ever move one disk from one machine to another.
To work around this you can change the uuid by recreating the array with
mdadm, but you would have to regenerate the initrd and fight again with
lilo :(

>-- I know there was a lot of discussion on the list a while back about 
>whether or not to mirror SWAP space. Was there ever any conclusion? It 
yes, with kernel 2.2 mirroring swap led to deadlocks under memory
pressure. but with modern kernels the only issue that would prevent you
to mirror swap is masochism.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Can you IMAGE Mirrored OS Drives?

[andy liebman]

> On Tue, 15 Aug 2006, andy liebman wrote:
> 
>> -- If I were to create disk images of EACH drive (i.e., /dev/sda and
>> /dev/sdb), could I restore each of those images to NEW drives -- with
>> all of their respective partitions -- and have a working RAIDED OS?  I
>> ask because my ultimate goal is to put a RAIDED OS on many systems, and
>> once I get ONE working, it would be nice to clone them the way I already
>> clone SINGLE OS drives. Can you clone RAIDS?
> 
> Yes.
> 
> I did this some time back to create half a dozen identical systems. I did
> it with IDE drives, but I don't see why it won't work with SCSI/SATA.
> 
> The first thing you need is to make sure the partition tables are
> identical on both drives. (you probably don't, but it'll be much easier)
> 
> So I started with my good system, and had a copy of the partition table on
> hda:
> 
>   # sfdisk -d /dev/hda > ~hda.part
> 
> Made sure all partitions were insync, then shut it down and remove hdb,
> and replace it with a new blank disk of the same size.
> 
> It booted just fine, with all partitions degraded.
> 
> Put the parittion table on the new blank hdb:
> 
>   # sfdisk /dev/hdb < ~/hda.part
> 
> then I hot-added each partition. eg:
> 
>   # mdadm --add /dev/md1 /dev/hdb1
> 
> and so on.
> 

Thanks Gordon,

I may not have been clear what I was asking. I wanted to know if you can 
make DISK IMAGES -- for example, with a program like Norton Ghost or 
Acronis True Image (better) -- of EACH of the two OS drives from a 
mirrored pair. Then restore Image A to one new disk, Image B to another 
disk. And then have a new working mirrored pair.

There is absolutely NO PROBLEM making images of single disks and 
restoring them to new disks (thus, creating clones). And it is very 
fast. For an OS drive with about 4 GBs of data, it only takes about 5 
minutes to make the image and 3 to restore it. So, after making the 
first set of images, it would in theory take under 10 minutes to restore 
a mirrored pair.

I'm just trying to find out if there would be any "gotchas" in restoring 
mirrored drives. I can't think of any, but you never know. I'm not 
worried about hostnames. That's easy to fix.

Andy

Re: Can you IMAGE Mirrored OS Drives?

[Gordon Henderson]

On Wed, 16 Aug 2006, andy liebman wrote:

> Thanks Gordon,
>
> I may not have been clear what I was asking. I wanted to know if you can
> make DISK IMAGES -- for example, with a program like Norton Ghost or
> Acronis True Image (better) -- of EACH of the two OS drives from a
> mirrored pair. Then restore Image A to one new disk, Image B to another
> disk. And then have a new working mirrored pair.

Right (I think) so at the end of the day, you want N servers each with a
pair of mirrored drives in it, or do you want N servers each with one
drive in it? I was assuming N servers with a pair of mirrored drives, all
setup identically...

> There is absolutely NO PROBLEM making images of single disks and
> restoring them to new disks (thus, creating clones). And it is very
> fast. For an OS drive with about 4 GBs of data, it only takes about 5
> minutes to make the image and 3 to restore it. So, after making the
> first set of images, it would in theory take under 10 minutes to restore
> a mirrored pair.

Imaging a partition or whole drive can be done with 'dd', but the
partition/drive needs to be offline or read-only when you do this or
you'll miss the filesystem cache and end up copying incorrect data. What
you can do is fill the partition (if it's a file system) with zeros; dd
if=/dev/zero of=bigfile ; rm bigfile, then feed it through gzip to make
the image smaller if desierd.

> I'm just trying to find out if there would be any "gotchas" in restoring
> mirrored drives. I can't think of any, but you never know. I'm not
> worried about hostnames. That's easy to fix.

Other than losing data you put in the drives after you made the clone, I
don't think so, but I may still be a little confused as to what you are
trying to achieve though ...

Gordon

Re: Can you IMAGE Mirrored OS Drives?

[Gabor Gombas]

On Wed, Aug 16, 2006 at 09:38:54AM +0200, Luca Berra wrote:

> The only risk is if you ever move one disk from one machine to another.
> To work around this you can change the uuid by recreating the array with
> mdadm,

No need to re-create, --update=uuid should be enough according to the
man page.

> but you would have to regenerate the initrd and fight again with lilo :(

Or you can just build a kernel with md built-in and use the kernel-level
RAID autodetection. In situations like this it is _much_ easier and
_much_ more robust than all the initrd-based solutions I have seen.

Also, if you install lilo to the RAID device instead of the MBR, and
install a bog-standard MBR using the install-mbr command on every drive,
your life will be easier.

Gabor

-- 
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------

Re: Can you IMAGE Mirrored OS Drives?

[Gabor Gombas]

On Wed, Aug 16, 2006 at 06:06:24AM -0400, andy liebman wrote:

> There is absolutely NO PROBLEM making images of single disks and 
> restoring them to new disks (thus, creating clones). And it is very 
> fast. For an OS drive with about 4 GBs of data, it only takes about 5 
> minutes to make the image and 3 to restore it. So, after making the 
> first set of images, it would in theory take under 10 minutes to restore 
> a mirrored pair.

Be prepared that these times will be much larger if you try to clone
RAID5, since by looking at just a single disk the image creator program
will not be able to read the file system and identify which parts of the
partition contain useful data and which parts are empty.

Gabor

-- 
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------

Re: Can you IMAGE Mirrored OS Drives?

[Luca Berra]

On Sat, Aug 19, 2006 at 03:13:53AM +0200, Gabor Gombas wrote:
>> but you would have to regenerate the initrd and fight again with lilo :(
>
>Or you can just build a kernel with md built-in and use the kernel-level
>RAID autodetection. In situations like this it is _much_ easier and
>_much_ more robust than all the initrd-based solutions I have seen.

please, can we try not to resurrect again the kernel-level autodetection
flamewar on this list.

L.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Can you IMAGE Mirrored OS Drives?

[Gabor Gombas]

On Sat, Aug 19, 2006 at 09:05:39AM +0200, Luca Berra wrote:

> please, can we try not to resurrect again the kernel-level autodetection
> flamewar on this list.

There is no need for a flame war. In some situations one is better, in
other situations the other is better.

Gabor

-- 
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------

Re: Can you IMAGE Mirrored OS Drives?

[Bill Davidsen]

Alternatives would be to have two such backup devices, and configure 
them, as andy liebman wrote:

>
> I may not have been clear what I was asking. I wanted to know if you 
> can make DISK IMAGES -- for example, with a program like Norton Ghost 
> or Acronis True Image (better) -- of EACH of the two OS drives from a 
> mirrored pair. Then restore Image A to one new disk, Image B to 
> another disk. And then have a new working mirrored pair.

May I say belatedly (I've been flat out since July 1) that if I were 
making a significant number of these clones, I'd write a script so that 
I could clone one drive, drop it in another machine, and let the script 
run on the other machine to finish the job. I have no idea how many of 
these you are doing, but automation is nice to avoid finger checks.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979