[linux-lvm] expanding physical disks

[linux-lvm] expanding physical disks

[Brian J. Murrell]

How does LVM deal with physical disks that can get bigger or smaller,
such as a hardware RAID device?  What happens to a PV on a hardware
RAID-5 device that is presented to the system as a single (say scsci)
target when you put a few more disks in it and tell the hardware raid
device to add them to the given (scsi) target that a PV was created
on?  (what a mouthful).

Is this really possible or can one only expand a hardware raid array
for LVM usage by adding additional raid targets and then adding the
raid targets as PVs to a VG?

b.

Re: [linux-lvm] expanding physical disks

[Andreas Dilger]

Brian Murrell writes:
> How does LVM deal with physical disks that can get bigger or smaller,
> such as a hardware RAID device?  What happens to a PV on a hardware
> RAID-5 device that is presented to the system as a single (say scsci)
> target when you put a few more disks in it and tell the hardware raid
> device to add them to the given (scsi) target that a PV was created
> on?  (what a mouthful).

Doesn't work at this time.  LVM will only see what was originally there
at the time pvcreate was run (or possibly vgcreate/vgextend).

It would not be impossible to fix this, however, depending on your needs.
The way the VGDA is layed out is as follows:

PV data
VG data
PV UUID list
LV data
PE map
(padding = original PV size % PE size (usually))
PE 0 = start of user data

When you increase the size of the PV itself, you are essentially adding
PEs to the end that need to be added to the PE map.  It is possible that
the PEs will fit into the padding space.  If that is the case, you don't
have many problems, and you are happy, assuming you know enough about
LVM user tools and metadata layout to write a tool to do this.

If the new PEs don't all fit into the padding space (it may happen if
your starting PV size is just a couple hundred kB over n * PE size),
then you will need to pvmove PE0 from its current location to somewhere
else in the VG, and then renumber all of the PEs on that PV as (PE# - 1).
That couble be ugly because I believe the PE number is stored in the
snapshot tables and such.

Probably not worth the effort.  Since each PE map entry is only 4 bytes
in size at this time, and you will normally have at least 4MB PEs, on
average you will be able to grow to 2MB/4*4MB=2TB (max ~4TB) without
any need for the complexity of the second option.  With 8MB PEs, you
get on average 8TB (max ~16TB).

The simple answer is don't do that.  If you write a tool to handle the
first case, then all you need to do is make the PE size sufficiently
large that you will "always" have enough padding to extend the PV to
any practical size.  You can easily hack the LVM code to ensure that
the pe_on_disk.size is always large enough to add an arbitrary number
of PEs, just by subtracting one or more PEs from the usable space at
the time it is initially set up.

> Is this really possible or can one only expand a hardware raid array
> for LVM usage by adding additional raid targets and then adding the
> raid targets as PVs to a VG?

This is the only currently available way to do it, the speculation
above notwithstanding.  Another solution (if you have the disk space)
is to pvmove all of the PEs from the PV you want to expand, vgreduce
the PV from the VG, extend the PV to the new size, pvcreate it again,
and then vgextend the VG with the "new" PV.  This requires 2xPV size
whenever you need to extend the PV, which is kind of counter productive.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] expanding physical disks

[Luca Berra]

On Wed, Jun 13, 2001 at 01:10:42AM -0600, Andreas Dilger wrote:
> Brian Murrell writes:
> > How does LVM deal with physical disks that can get bigger or smaller,
> > such as a hardware RAID device?  What happens to a PV on a hardware
> > RAID-5 device that is presented to the system as a single (say scsci)
> > target when you put a few more disks in it and tell the hardware raid
> > device to add them to the given (scsi) target that a PV was created
> > on?  (what a mouthful).
> 
> Doesn't work at this time.  LVM will only see what was originally there
> at the time pvcreate was run (or possibly vgcreate/vgextend).
At the moment your only chance is to create partitions on it and use those
partitions as PVs

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: [linux-lvm] expanding physical disks

[Hugo Lombard]

On Wed, Jun 13, 2001 at 03:20:32PM +0200, Luca Berra wrote:
> On Wed, Jun 13, 2001 at 01:10:42AM -0600, Andreas Dilger wrote:
> > Brian Murrell writes:
> > > How does LVM deal with physical disks that can get bigger or smaller,
> > > such as a hardware RAID device?  What happens to a PV on a hardware
> > > RAID-5 device that is presented to the system as a single (say scsci)
> > > target when you put a few more disks in it and tell the hardware raid
> > > device to add them to the given (scsi) target that a PV was created
> > > on?  (what a mouthful).
> > 
> > Doesn't work at this time.  LVM will only see what was originally there
> > at the time pvcreate was run (or possibly vgcreate/vgextend).
> At the moment your only chance is to create partitions on it and use those
> partitions as PVs
> 

Just a thought, a bit off topic perhaps.

If you're using LVM primarily as a way to make it possible for you to
expand your storage space on-line (and on-mount-point) wouldn't it be
"easier" to _just_ use the RAID? i.e. expand the RAID volume on the
controller, so basically the "disk" is bigger now, and the just "grow"
your partition, extend your filesystem, and that's it?

It's a thought I've been toying with since seeing the IBM ServeRAID's
capability to grow volumes.  I've not attempted any testing though, so
this is pure speculation...

(PS: All this means nothing if you're using LVM for it's other
features, like striping, but then the RAID can do that too...)




-- 
"You show me an American who can keep his mouth shut and I'll eat him."
-- Newspaperman from Frank Capra's _Meet_John_Doe_
---------------------------------------------------------------------------
 Hugo Lombard                    Infoline (Pty) Ltd
 System Administrator
---------------------------------------------------------------------------

Re: [linux-lvm] expanding physical disks

[Heinz J. Mauelshagen]

On Fri, Jun 15, 2001 at 04:11:51PM +0200, Hugo Lombard wrote:
> On Wed, Jun 13, 2001 at 03:20:32PM +0200, Luca Berra wrote:
> > On Wed, Jun 13, 2001 at 01:10:42AM -0600, Andreas Dilger wrote:
> > > Brian Murrell writes:
> > > > How does LVM deal with physical disks that can get bigger or smaller,
> > > > such as a hardware RAID device?  What happens to a PV on a hardware
> > > > RAID-5 device that is presented to the system as a single (say scsci)
> > > > target when you put a few more disks in it and tell the hardware raid
> > > > device to add them to the given (scsi) target that a PV was created
> > > > on?  (what a mouthful).
> > > 
> > > Doesn't work at this time.  LVM will only see what was originally there
> > > at the time pvcreate was run (or possibly vgcreate/vgextend).
> > At the moment your only chance is to create partitions on it and use those
> > partitions as PVs
> > 
> 
> Just a thought, a bit off topic perhaps.

;-)

> 
> If you're using LVM primarily as a way to make it possible for you to
> expand your storage space on-line (and on-mount-point) wouldn't it be
> "easier" to _just_ use the RAID? i.e. expand the RAID volume on the
> controller, so basically the "disk" is bigger now, and the just "grow"
> your partition, extend your filesystem, and that's it?

I think you're right in case you can live with the restrictions involved.

If the amount of resizable volumes a single smart controller can support is
enough, you can surely go with that solution. When it comes to more storage
you need a logical volume manager to group multiple disk subsystems together
in order to support large storage configurations.

> 
> It's a thought I've been toying with since seeing the IBM ServeRAID's
> capability to grow volumes.  I've not attempted any testing though, so
> this is pure speculation...
> 
> (PS: All this means nothing if you're using LVM for it's other
> features, like striping, but then the RAID can do that too...)

Plus LVM's features to move data around online in order to relocate it to
faster/bigger/newer disk subsystems, to have more that just a couple of 
resizable devices, support for hardware block device reconfiguration without
any changes in the namespace of the logical volumes, snapshot support etc.

Regards,
Heinz    -- The LVM Guy --

> 
> 
> 
> 
> -- 
> "You show me an American who can keep his mouth shut and I'll eat him."
> -- Newspaperman from Frank Capra's _Meet_John_Doe_
> ---------------------------------------------------------------------------
>  Hugo Lombard                    Infoline (Pty) Ltd
>  System Administrator
> ---------------------------------------------------------------------------
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] expanding physical disks

[Hugo Lombard]

On Fri, Jun 15, 2001 at 04:51:35PM +0200, Heinz J. Mauelshagen wrote:
> On Fri, Jun 15, 2001 at 04:11:51PM +0200, Hugo Lombard wrote:
> > 
> > If you're using LVM primarily as a way to make it possible for you to
> > expand your storage space on-line (and on-mount-point) wouldn't it be
> > "easier" to _just_ use the RAID? i.e. expand the RAID volume on the
> > controller, so basically the "disk" is bigger now, and the just "grow"
> > your partition, extend your filesystem, and that's it?
> 
> I think you're right in case you can live with the restrictions involved.
> 

I'm right!?!?! Mother would be so proud!! ;)

> If the amount of resizable volumes a single smart controller can support is
> enough, you can surely go with that solution. When it comes to more storage
> you need a logical volume manager to group multiple disk subsystems together
> in order to support large storage configurations.
> 

*nod*  As usual, there's no universal answer, and it depends on you
needs/configuration/preference/phase of moon...

> > 
> > It's a thought I've been toying with since seeing the IBM ServeRAID's
> > capability to grow volumes.  I've not attempted any testing though, so
> > this is pure speculation...
> > 
> > (PS: All this means nothing if you're using LVM for it's other
> > features, like striping, but then the RAID can do that too...)
> 
> Plus LVM's features to move data around online in order to relocate it to
> faster/bigger/newer disk subsystems, to have more that just a couple of 
> resizable devices, support for hardware block device reconfiguration without
> any changes in the namespace of the logical volumes, snapshot support etc.
> 

Very important features to consider, for sure.

-- 
Dorothy:	But how can you talk without a brain?
Scarecrow:	Well, I don't know... but some people without brains
		do an awful lot of talking.
		-- The Wizard of Oz
---------------------------------------------------------------------------
 Hugo Lombard                    Infoline (Pty) Ltd
 System Administrator
---------------------------------------------------------------------------

Re: [linux-lvm] expanding physical disks

[Paul Jarc]

Andreas Dilger <adilger@turbolinux.com> writes:
> Brian Murrell writes:
>> How does LVM deal with physical disks that can get bigger or smaller,
>> such as a hardware RAID device?
> 
> Doesn't work at this time.  LVM will only see what was originally there
> at the time pvcreate was run (or possibly vgcreate/vgextend).

But it won't do any harm either, right?

> If the new PEs don't all fit into the padding space (it may happen if
> your starting PV size is just a couple hundred kB over n * PE size),
> then you will need to pvmove PE0 from its current location to somewhere
> else in the VG, and then renumber all of the PEs on that PV as (PE# - 1).
> That couble be ugly because I believe the PE number is stored in the
> snapshot tables and such.

Since this is a case of making the PV bigger, all the PEs could simply
be shifted by one; there is, after all, more room at the end now.
That would still be ugly, though.


paul

Re: [linux-lvm] expanding physical disks

[Heinz J. Mauelshagen]

On Wed, Jun 13, 2001 at 01:22:26PM -0400, Paul Jarc wrote:
> Andreas Dilger <adilger@turbolinux.com> writes:
> > Brian Murrell writes:
> >> How does LVM deal with physical disks that can get bigger or smaller,
> >> such as a hardware RAID device?
> > 
> > Doesn't work at this time.  LVM will only see what was originally there
> > at the time pvcreate was run (or possibly vgcreate/vgextend).
> 
> But it won't do any harm either, right?
> 
> > If the new PEs don't all fit into the padding space (it may happen if
> > your starting PV size is just a couple hundred kB over n * PE size),
> > then you will need to pvmove PE0 from its current location to somewhere
> > else in the VG, and then renumber all of the PEs on that PV as (PE# - 1).
> > That couble be ugly because I believe the PE number is stored in the
> > snapshot tables and such.
> 
> Since this is a case of making the PV bigger, all the PEs could simply
> be shifted by one; there is, after all, more room at the end now.
> That would still be ugly, though.

Yes, it would :-(

Typical HW storage subsystems anable you to create additional IDs (for eg.
SCSI LUNs) to addrees the gained storage once you insert additional drives.

Do you have constraints to create those and just use the as new PVs or
is the maximum of 8 LUNs in the SCSI case insufficient for you?

That's the way we did it quite happily for multiple years in my previous job 
on HP, SUN and other platforms :-)

> 
> 
> paul
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-