[linux-lvm] Mirrored LV

[linux-lvm] Mirrored LV

[Koen Vermeer]

Hi,

I was thinking of using a mirrored LV as a easier to use alternative to
a RAID1 PV (with matching VG and LV). I tried to find the information I
need on how this works and how to set it up, but so far, I failed.

First, am I right in trying to use LVM in this way? I have two disks,
and I just want to mirror some LVs.

Second, I am confused by the apparent need to have this log stored on
another device. Actually, this doesn't seem true in two ways: There's
the option of having the log in memory and then the documentation only
says that the log is 'usually on a separate device'. Does that mean that
using a mirrored LV makes no sense on a system with two disks? Or should
I just keep the log in memory in this case? Or can I use another PV on
one of the disks? Should this be mirrored as well? What happens if I
loose that data?

From what I found when searching for answers, it seems that I'm not the
only one that's confused... But maybe my searching skills are just
lacking. Anyway, I appreciate any help and insights!

Best,
Koen

Re: [linux-lvm] Mirrored LV

[Georges Giralt]

Koen Vermeer a �crit :
> Hi,
> 
> I was thinking of using a mirrored LV as a easier to use alternative to
> a RAID1 PV (with matching VG and LV). I tried to find the information I
> need on how this works and how to set it up, but so far, I failed.
> 
> First, am I right in trying to use LVM in this way? I have two disks,
> and I just want to mirror some LVs.
> 
> Second, I am confused by the apparent need to have this log stored on
> another device. Actually, this doesn't seem true in two ways: There's
> the option of having the log in memory and then the documentation only
> says that the log is 'usually on a separate device'. Does that mean that
> using a mirrored LV makes no sense on a system with two disks? Or should
> I just keep the log in memory in this case? Or can I use another PV on
> one of the disks? Should this be mirrored as well? What happens if I
> loose that data?
> 
>>From what I found when searching for answers, it seems that I'm not the
> only one that's confused... But maybe my searching skills are just
> lacking. Anyway, I appreciate any help and insights!
> 
> Best,
> Koen
Koen,
I confirm that you have to use 3 disks to mirror a VG with the actual 
version of software.
At home, having enough disk, I use 3 disks to get my VG/LV mirrored and 
it works flawlessly since a couple of years.
At work, we have mirrored the PV using 2 devices and the software RAID 
(md) and build the VG/LV on top of that. It runs flawlessly for 4 years 
I think (have to check the exact installation date...) So choice is yours...
HAve a nice day.
P.S. At home, I do not use the whole PV for my LV.
Each PV is split in two dissymitrical parts : a small 100 M partition 
and the rest of the device. The small partition is used for a software 
RAID (md) to hold the /boot. The rest of the disk as a regular PV used 
to build the mirrored VG/LV. This way, if one disk breaks I still can 
boot onto the remaining  one. Of course, it is a manual boot but easier 
compared to searching a useable rescue CD and the correct kernel 
version. Think about it.


-- 
If the only tool you have is a hammer, you tend to see every problem as 
a nail.
                 Abraham Maslow

Re: [linux-lvm] Mirrored LV

[Koen Vermeer]

Op dinsdag 16-09-2008 om 21:56 uur [tijdzone +0200], schreef Georges
Giralt:
> I confirm that you have to use 3 disks to mirror a VG with the actual 
> version of software.

I assume you mean that you have to use 3 disks if you want to put the
log on disk? I mean, you can use 2 disks and have the log in memory,
right?
In any way, if you indeed need 3 disks when putting the log on disk,
then the man page seems to be misleading.

> P.S. At home, I do not use the whole PV for my LV.
> Each PV is split in two dissymitrical parts : a small 100 M partition 
> and the rest of the device. The small partition is used for a software 
> RAID (md) to hold the /boot. The rest of the disk as a regular PV used 
> to build the mirrored VG/LV. This way, if one disk breaks I still can 
> boot onto the remaining  one. Of course, it is a manual boot but easier 
> compared to searching a useable rescue CD and the correct kernel 
> version. Think about it.

I currently have a small /boot partition on both disks, mirrored with
RAID 1. The rest of the disks are LVM partitions. I like the flexibility
of having everything (except /boot) as LVM partitions, rather than
separating them into unmirrored and RAID 1 partitions. But maybe I'm
just out of luck here...

Best,
Koen

Re: [linux-lvm] Mirrored LV

[malahal]

Koen Vermeer [koen@vermeer.tv] wrote:
> Hi,
> 
> I was thinking of using a mirrored LV as a easier to use alternative to
> a RAID1 PV (with matching VG and LV). I tried to find the information I
> need on how this works and how to set it up, but so far, I failed.
> 
> First, am I right in trying to use LVM in this way? I have two disks,
> and I just want to mirror some LVs.

You can by using corelog (aka, log in memory), but read on...

> Second, I am confused by the apparent need to have this log stored on
> another device. Actually, this doesn't seem true in two ways: There's
> the option of having the log in memory and then the documentation only
> says that the log is 'usually on a separate device'. Does that mean that
> using a mirrored LV makes no sense on a system with two disks? Or should
> I just keep the log in memory in this case? Or can I use another PV on
> one of the disks? Should this be mirrored as well? What happens if I
> loose that data?

The log records what regions of your master device are in sync with your
other mirror device. It is used to synchronize the failed regions from
the master to your other device when your 'other device' comes back to
life. This should work whether your log device is in memory or on a PV
by itself as long as you don't de-activate or restart your machine. The
log data on a PV can be useful for reboots.

-Malahal.

Re: [linux-lvm] Mirrored LV

[Koen Vermeer]

Op dinsdag 16-09-2008 om 13:42 uur [tijdzone -0700], schreef
malahal@us.ibm.com:
> The log records what regions of your master device are in sync with your
> other mirror device. It is used to synchronize the failed regions from
> the master to your other device when your 'other device' comes back to
> life. This should work whether your log device is in memory or on a PV
> by itself as long as you don't de-activate or restart your machine. The
> log data on a PV can be useful for reboots.

I had written a lot of text with extra questions, but it seems I just
don't get it - yet.

If I understand you correctly, you're statements are about when the
'other device' is failing. Is this right? If so, then the log should
normally show that everything is synced. In that case, it's not of much
use in normal operation. Now, when one disk fails, I need to replace it,
and rebuild the mirror. So again, I don't need it, because I have to
copy everything anyway. Only when a disk is temporarily gone, it may be
of some use. Does this make sense?

Best,
Koen

Re: [linux-lvm] Mirrored LV

[malahal]

Koen Vermeer [koen@vermeer.tv] wrote:
> If I understand you correctly, you're statements are about when the
> 'other device' is failing. Is this right? If so, then the log should
> normally show that everything is synced. In that case, it's not of much
> use in normal operation. Now, when one disk fails, I need to replace it,
> and rebuild the mirror. So again, I don't need it, because I have to
> copy everything anyway. Only when a disk is temporarily gone, it may be
> of some use. Does this make sense?

Yes, it makes sense for temporarily gone devices. Otherwise use corelog
for your lvcreate command and you should be able to create mirrored LV.

Re: [linux-lvm] Mirrored LV

[Koen Vermeer]

On Tue, 2008-09-16 at 14:57 -0700, malahal@us.ibm.com wrote:
> Yes, it makes sense for temporarily gone devices. Otherwise use corelog
> for your lvcreate command and you should be able to create mirrored LV.

OK, I get that. So, when using corelog, I won't risk loosing data.

But, when using corelog, what would be the expected delay at bootup?

Also, if I use a log on disk, what happens if I put it on the same disk
as either the master or the copy? And, if I do that, should it be on
it's own PV, or on it's own LV?

Best,
Koen

Re: [linux-lvm] Mirrored LV

[Koen Vermeer]

On Tue, 2008-09-16 at 14:57 -0700, malahal@us.ibm.com wrote:
> Koen Vermeer [koen@vermeer.tv] wrote:
> > If I understand you correctly, you're statements are about when the
> > 'other device' is failing. Is this right? If so, then the log should
> > normally show that everything is synced. In that case, it's not of much
> > use in normal operation. Now, when one disk fails, I need to replace it,
> > and rebuild the mirror. So again, I don't need it, because I have to
> > copy everything anyway. Only when a disk is temporarily gone, it may be
> > of some use. Does this make sense?
> Yes, it makes sense for temporarily gone devices. Otherwise use corelog
> for your lvcreate command and you should be able to create mirrored LV.

I tried converting my lv to a mirrored one with --corelog, which worked
perfectly. However, after a reboot, the mirror is being recreated, which
takes quite some time and leaves me with a largely unresponsive system.

Is there some way around this? Would having the log on the master disk
fix this? If so, does it need it's own PV or LV or neither?

Thanks!

Koen

Re: [linux-lvm] Mirrored LV

[malahal]

Koen Vermeer [koen@vermeer.tv] wrote:
> I tried converting my lv to a mirrored one with --corelog, which worked
> perfectly. However, after a reboot, the mirror is being recreated, which
> takes quite some time and leaves me with a largely unresponsive system.
> 
> Is there some way around this? Would having the log on the master disk
> fix this? If so, does it need it's own PV or LV or neither?

It needs it own PV. Create an extra partition on your master disk and
add the PV tou the volume group.  Then run lvcreate again with out
corelog option. By default the command would create a disklog. You can
also specify where you want your log.

--Malahal.

Re: [linux-lvm] Mirrored LV

[Koen Vermeer]

Op vrijdag 19-09-2008 om 10:31 uur [tijdzone -0700], schreef
malahal@us.ibm.com:
> It needs it own PV. Create an extra partition on your master disk and
> add the PV tou the volume group.  Then run lvcreate again with out
> corelog option. By default the command would create a disklog. You can
> also specify where you want your log.

OK, did that! I guess I need one additional partition for each volume
group, then.

Now, what happens if the master disk (which includes the log) dies?
Won't LVM complain that it cannot find the log? I'm not saying I expect
it too, but with this stuff I guess it's better to be safe than sorry!

Best,
Koen

Re: [linux-lvm] Mirrored LV

[Jonathan Brassow]

Just to answer some questions:

You can use just 2 disks to create a mirror and a disk log:
prompt> lvcreate --alloc anywhere -m1 -L 500M -n my_lv my_vg

I wish the allocation policy 'normal' (which is the default) would  
also do this... but instead you must choose a looser allocation policy  
to get what you want.

The reason that you want the disk-based log is to prevent all the  
extra work when you machine comes up.

The reason that LVM wants a separate disk for the log is so the head  
on the storage device doesn't need to go ping-ponging around the  
device.  (The log is touched for almost every write.)  So, its a  
performance thing... and possibly a not-wear-out-your-hardware thing  
too.

It won't matter if the master disk with the log dies.  Your mirror  
will simply become a linear device composed of the last remaining disk  
anyway.

  brassow


On Sep 19, 2008, at 5:01 PM, Koen Vermeer wrote:

> Op vrijdag 19-09-2008 om 10:31 uur [tijdzone -0700], schreef
> malahal@us.ibm.com:
>> It needs it own PV. Create an extra partition on your master disk and
>> add the PV tou the volume group.  Then run lvcreate again with out
>> corelog option. By default the command would create a disklog. You  
>> can
>> also specify where you want your log.
>
> OK, did that! I guess I need one additional partition for each volume
> group, then.
>
> Now, what happens if the master disk (which includes the log) dies?
> Won't LVM complain that it cannot find the log? I'm not saying I  
> expect
> it too, but with this stuff I guess it's better to be safe than sorry!
>
> Best,
> Koen
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Mirrored LV

[malahal]

Jonathan Brassow [jbrassow@redhat.com] wrote:
> Just to answer some questions:
>
> You can use just 2 disks to create a mirror and a disk log:
> prompt> lvcreate --alloc anywhere -m1 -L 500M -n my_lv my_vg
>
> I wish the allocation policy 'normal' (which is the default) would also do 
> this... but instead you must choose a looser allocation policy to get what 
> you want.
>
> The reason that you want the disk-based log is to prevent all the extra 
> work when you machine comes up.

I wish LVM can detect and only re-sync if needed even with corelog!

> The reason that LVM wants a separate disk for the log is so the head on the 
> storage device doesn't need to go ping-ponging around the device.  (The log 
> is touched for almost every write.)  So, its a performance thing... and 
> possibly a not-wear-out-your-hardware thing too.

Nice to know about guaranteed long disk seeks with this approach!

--Malahal.

Re: [linux-lvm] Mirrored LV

[Koen Vermeer]

On Fri, 2008-09-19 at 22:39 -0500, Jonathan Brassow wrote:
> Just to answer some questions:
> 
> You can use just 2 disks to create a mirror and a disk log:
> prompt> lvcreate --alloc anywhere -m1 -L 500M -n my_lv my_vg
> I wish the allocation policy 'normal' (which is the default) would  
> also do this... but instead you must choose a looser allocation policy  
> to get what you want.

With vg_test already containing 2 equally sized PVs (one on /dev/sda,
the other on /dev/sdb), I did this:
vgextend vg_test /dev/sda6
lvconvert -m 1 vg_test/XP
(XP is a small LV with an XP image, /dev/sda6 is a small partition)
That seems to work fine:
lvs -a -o +devices
  XP                     vg_test   mwi-ao  15.00G
XP_mlog 100.00         XP_mimage_0(0),XP_mimage_1(0)                  
  [XP_mimage_0]          vg_test   iwi-ao
15.00G                                           /dev/sda4(2560)                                
  [XP_mimage_1]          vg_test   iwi-ao
15.00G                                           /dev/sdb3(0)                                   
  [XP_mlog]              vg_test   lwi-ao
4.00M                                           /dev/sda6(0) 

> The reason that you want the disk-based log is to prevent all the  
> extra work when you machine comes up.

Yes, I understand that now. It wasn't clear from the documentation that
with corelog, it would rebuild the whole mirror.

> The reason that LVM wants a separate disk for the log is so the head  
> on the storage device doesn't need to go ping-ponging around the  
> device.  (The log is touched for almost every write.)  So, its a  
> performance thing... and possibly a not-wear-out-your-hardware thing  
> too.

I see. So I guess that means that in the general case, an LVM-mirror
approach is not a good substitute for an LVM-on-RAID1 approach? My
original rationale was to have a more flexible system by using
LVM-mirror than I'd get when using LVM-on-RAID1.

What I'd expect is some way to have the log lazily cached in memory. In
that way, there is no ping-ponging, you'd only have to rebuild the
log/mirror when there is a power outage and you wouldn't have the
startup delays, i.e. something like a regular filesystem.

> It won't matter if the master disk with the log dies.  Your mirror  
> will simply become a linear device composed of the last remaining disk  
> anyway.

That makes sense, at least with two drives.

Thanks for your comments!

Koen

Re: [linux-lvm] Mirrored LV

[Larry Dickson]

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

How often does the log get written in normal use, and how big does it get,
how fast? If it's a low-bandwidth sort of thing, could it go to some kind of
solid-state storage?

Larry Dickson
Cutting Edge Networked Storage

On 9/16/08, Koen Vermeer <koen@vermeer.tv> wrote:
>
> Op dinsdag 16-09-2008 om 13:42 uur [tijdzone -0700], schreef
> malahal@us.ibm.com:
> > The log records what regions of your master device are in sync with your
> > other mirror device. It is used to synchronize the failed regions from
> > the master to your other device when your 'other device' comes back to
> > life. This should work whether your log device is in memory or on a PV
> > by itself as long as you don't de-activate or restart your machine. The
> > log data on a PV can be useful for reboots.
>
> I had written a lot of text with extra questions, but it seems I just
> don't get it - yet.
>
> If I understand you correctly, you're statements are about when the
> 'other device' is failing. Is this right? If so, then the log should
> normally show that everything is synced. In that case, it's not of much
> use in normal operation. Now, when one disk fails, I need to replace it,
> and rebuild the mirror. So again, I don't need it, because I have to
> copy everything anyway. Only when a disk is temporarily gone, it may be
> of some use. Does this make sense?
>
> Best,
> Koen
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

[-- Attachment #2: Type: text/html, Size: 2181 bytes --]

Re: [linux-lvm] Mirrored LV

[malahal]

Larry Dickson [ldickson@cuttedge.com] wrote:
>    How often does the log get written in normal use, and how big does it get,
>    how fast? If it's a low-bandwidth sort of thing, could it go to some kind
>    of solid-state storage?

There is a fixed size header and then a bit for each region. So it
really depends on your LV size and the region size. Typically a low
bandwidth sort of thing.

--Malahal.