[linux-lvm] LVM, partitions, RAID, and the Grand Scheme of Things TM

[linux-lvm] LVM, partitions, RAID, and the Grand Scheme of Things TM

[Andrew Clausen]

Greetings, LVM people :-)

I am the GNU Parted maintainer (www.gnu.org/software/parted), and I
work for Conectiva (www.conectiva.com).

First, we think it would be useful for the LVM tools to be able to
resize logical volumes, along with the file systems on it.  Perhaps
a --filesystem parameter to lvreduce, lvextend and lvcreate?  This
functionality should be fairly easy to add with the existing
libparted.  Although, we'd need to abstract out device I/O a bit
more... but this is trivial.

BTW: we are aware of Ted Tso's resizer.
(1) we like our's better :-P
(2) libparted can resize other file systems.  We can do ext2,
fat[16|32], linux-swap at the moment, and we're planning to do
others.  libparted provides an abstract interface, so you get new
features supported without any work :-)

More importantly:

We are interested in the future of partitions, LVM, RAID and Microsoft's
new system, which seems similar to LVM - although we haven't seen it
yet.  Also, AIX's and HPUX's LVM have some features that Linux LVM
doesn't have - you're working in that direction?

It seems, on the face of it, that LVM offers a strict superset of
functionality of partition tables and RAID (well, RAID-linear/0, anyway
- RAID-1/4/5 coming soon?).

However, it doesn't look like partition tables will disappear anytime
soon. Rather, it looks like LVM will have to cooperate with partition
tables.  And, perhaps Microsoft's new system as well (?).

Since LVM, partition tables, RAID, and M$'s thing are similar, we think
it might make sense to make a single tool (in addition to the LVM tools)
that can manipulate them all.  And in fact, a library (that wraps
libparted, liblvm, etc) to provide this abstracted functionality to
other programs.

Arguments for such a move:
* a lot of the complexity, WRT interaction between filesystems and
partition tables/LVM/* is similar.  Eg: making a partition/LV usually
happens at the same time as creating a file system on that partition.
Likewise for resizing.  This occurs right from the low-level (IO -
"which part of which disk do I read/write?") to the high-level (how
big can/should I make this partition?)

* may make life easier for boot-loader installers.  There has been a bit
of discussion between as and GRUB people about making a libbootload.
This is still a distant plan.  Basic idea: allow communication between
partition programs (or whatever) and boot loaders about information
like, where to find kernels, etc.

* users will love it :-)

* it's "philosophically" Right.

* may simplify automatic partitioning, since everything is in one place.

It may turn out that squeezing everything into a single interface will
complicate things too much, but we don't think this will happen.


Anyway, onto the NEXT part:  HOW.

First, I think it's desirable that we be independent of Linux, and
assume that there is no support for anything much in the kernel.  Is
this OK?  We can break this assumption, but we'd rather not...

Anyway, we are thinking about some rough correspondence like this:
* volume groups - equivalent to partition tables
* logical volumes - equivalent to partitions, RAID sets and raw devices.
* physical volumes - equivalent to a logical volume on the parent's
volume group.

Note: this "parent's volume group" thing.  Basically, partition tables
are nested.  So, LVM is like a partition table, nested inside, perhaps,
many MSDOS partition tables.

So, we'd provide some sort of API for manipulating these objects.

Comments...?

Thanks,
Andrew Clausen

Re: [linux-lvm] LVM, partitions, RAID, and the Grand Scheme of Things TM

[Andreas Dilger]

Andrew Clausen, you write:
> First, we think it would be useful for the LVM tools to be able to
> resize logical volumes, along with the file systems on it.  Perhaps
> a --filesystem parameter to lvreduce, lvextend and lvcreate?

The existing LVM user tools ship with something called "e2fsadm" which
does this for ext2 filesystems, first extending the LV and then the
filesystem.  I have updated e2fsadm to also handle online filesystem
resizing, but this is not in the official tools yet.  Yesterday, a tool
called lvmadm was announced which does this in a more generic way,
similar to fsck, by calling a fs-specific resizer depending on the fs
type.

> BTW: we are aware of Ted Tso's resizer.  We like our's better :-P

On a tangent, how different has your ext2 resizer become from the
ext2resize code?  You may want to reply in a separate note.

> We are interested in the future of partitions, LVM, RAID and Microsoft's
> new system, which seems similar to LVM - although we haven't seen it
> yet.  Also, AIX's and HPUX's LVM have some features that Linux LVM
> doesn't have - you're working in that direction?
> 
> It seems, on the face of it, that LVM offers a strict superset of
> functionality of partition tables and RAID (well, RAID-linear/0, anyway
> - RAID-1/4/5 coming soon?).

I think Heinz's stated direction is that LVM will stay layered on top
of MD/RAID rather than incorporating this functionality itself.

> However, it doesn't look like partition tables will disappear anytime
> soon. Rather, it looks like LVM will have to cooperate with partition
> tables.  And, perhaps Microsoft's new system as well (?).

Personally, I would rather get all knowledge of partition tables out of
LVM and have the existing partition table support in the kernel handle
everything.  Having a library to do all the partition-table parsing and
writing would be a boon not only to parted, but also if it was added to
fdisk, cfdisk, grub, et-al.

> * a lot of the complexity, WRT interaction between filesystems and
> partition tables/LVM/* is similar.  Eg: making a partition/LV usually
> happens at the same time as creating a file system on that partition.
> Likewise for resizing.  This occurs right from the low-level (IO -
> "which part of which disk do I read/write?") to the high-level (how
> big can/should I make this partition?)
>
> * users will love it :-)
> 
> * it's "philosophically" Right.
> 
> * may simplify automatic partitioning, since everything is in one place.

You should read the LVM mailing list archive for June.  IBM has already
proposed such a system, called LVMS, which would be the glue layer between
LVM, partitions, filesystems, RAID, etc.  From the sounds of it, they will
try and leverage as much of the existing code from the kernel as possible.

Cheers, Andreas

Re: [linux-lvm] LVM, partitions, RAID, and the Grand Scheme of Things TM

[Andrew Clausen]

Andreas Dilger wrote:
> 
> Andrew Clausen, you write:
> > First, we think it would be useful for the LVM tools to be able to
> > resize logical volumes, along with the file systems on it.  Perhaps
> > a --filesystem parameter to lvreduce, lvextend and lvcreate?
> 
> The existing LVM user tools ship with something called "e2fsadm" which
> does this for ext2 filesystems, first extending the LV and then the
> filesystem.  I have updated e2fsadm to also handle online filesystem
> resizing, but this is not in the official tools yet.

I'm not sure I like this idea of file system specific programs
doing LVM stuff.  What you describe here is better:

> Yesterday, a tool
> called lvmadm was announced which does this in a more generic way,
> similar to fsck, by calling a fs-specific resizer depending on the fs
> type.

It would be trivial (i.e. a bit of beaurocracy) to write a front end
to libparted, to do this (or even, a shell script wrapper of parted)

One thing: there is non-trivial interaction between the file system
and partition table handling (not with LVM), when resizing the start
of a partition.  Some resizers (including ext2resize, and reiserfs's
resizer) can't resize the start, and use the partition as the
address space.  When resizing the start, this doesn't make sense.

So, while it's rather cute to do:

	# ext2resize /dev/hda3 [new-size]

It generalizes to:

	# ext2resize /dev/hda3 [new-start] [new-end]

Doing this only makes sense if ext2resize (or whatever) updates the
partition table (as Parted does).

So, having /sbin/resize.* isn't as elegant as it looks.  Were
you suggesting have resize.* deal with partition table stuff?

> I think Heinz's stated direction is that LVM will stay layered on top
> of MD/RAID rather than incorporating this functionality itself.

OK.

> > However, it doesn't look like partition tables will disappear anytime
> > soon. Rather, it looks like LVM will have to cooperate with partition
> > tables.  And, perhaps Microsoft's new system as well (?).
> 
> Personally, I would rather get all knowledge of partition tables out of
> LVM and have the existing partition table support in the kernel handle
> everything.  Having a library to do all the partition-table parsing and
> writing would be a boon not only to parted, but also if it was added to
> fdisk, cfdisk, grub, et-al.

Yep.  OTOH, Parted already has lots of partition table parsing stuff
already.  It could be separated into a different library.

libparted's partition table API (version 1.3.x) is much cleaner than
anything else I've seen.  It supports msdos, mac and pc98 so far.

> You should read the LVM mailing list archive for June.  IBM has already
> proposed such a system, called LVMS, which would be the glue layer between
> LVM, partitions, filesystems, RAID, etc.  From the sounds of it, they will
> try and leverage as much of the existing code from the kernel as possible.

Looks like I have lots of reading to do!

Thanks,
Andrew Clausen

Re: [linux-lvm] LVM, partitions, RAID, and the Grand Scheme of Things TM

[Andreas Dilger]

Andrew Clausen writes:
> One thing: there is non-trivial interaction between the file system
> and partition table handling (not with LVM), when resizing the start
> of a partition.  Some resizers (including ext2resize, and reiserfs's
> resizer) can't resize the start, and use the partition as the
> address space.

When using LVM its not important to be able to move the beginning of
the filesystem, since you can always change the physical LE locations
without changing the filesystem itself.

What _is_ interesting, if you live in an LVM-centric world, is being
able to move the start of the filesystem a small amount into the
partition (after shrinking the end) so that you can turn the partition
into an PV/LV.  After such a conversion, you could do any normal LVM
handling of the LV/filesystem (e.g. shrink, grow, migrate).

It may just be easy enough to write a generic tool to shift the partition
contents X blocks via copying (after using an fs-specific tool to shrink
the end of the fs), and then handle the creation of the PV header,
and adding an LV of the appropriate size that covers the whole partition.
This is only really useful if you don't have enough space to duplicate
the whole partition.  Otherwise, you are just as well off creating a new
LV of the appropriate size, doing a raw copy of the filesystem to the new
LV, and deleting the old partition and/or turning it into a PV.

> So, having /sbin/resize.* isn't as elegant as it looks.  Were
> you suggesting have resize.* deal with partition table stuff?

Not at all.  The resize.* programs are filesystem specific, and only
resize from the end of the partition.  I suppose that they could be
generalized to accept an additional parameter (at the end) to give
the new start of the partition, e.g. "resize.ext2 /dev/hda1 [size] [start]"

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LVM, partitions, RAID, and the Grand Scheme of ThingsTM

[Andrew Clausen]

Andreas Dilger wrote:
> When using LVM its not important to be able to move the beginning of
> the filesystem, since you can always change the physical LE locations
> without changing the filesystem itself.

True.

> What _is_ interesting, if you live in an LVM-centric world, is being
> able to move the start of the filesystem a small amount into the
> partition (after shrinking the end) so that you can turn the partition
> into an PV/LV.  After such a conversion, you could do any normal LVM
> handling of the LV/filesystem (e.g. shrink, grow, migrate).

Yep :-)

> It may just be easy enough to write a generic tool to shift the partition
> contents X blocks via copying (after using an fs-specific tool to shrink
> the end of the fs), and then handle the creation of the PV header,
> and adding an LV of the appropriate size that covers the whole partition.
> This is only really useful if you don't have enough space to duplicate
> the whole partition.  Otherwise, you are just as well off creating a new
> LV of the appropriate size, doing a raw copy of the filesystem to the new
> LV, and deleting the old partition and/or turning it into a PV.

This is VERY slow (need to read & write the entire file system), and is
dangerous (requires journaling to prevent corruption on power failure,
or other interruptions).  Also: you may copy, at most, X bytes at a
time,
where X is the distance you move the start - unless you use an
intermediate copy area.  Either way (journaling, or intermediate
copying),
you're doing LOTS of seeks (if you're working with one disk) - which is
where IO gets really expensive.

We can do much better :-) 

I'll hopefully implement move-the-start for ext2 when I go to Brazil
(late Nov to late Feb)

> > So, having /sbin/resize.* isn't as elegant as it looks.  Were
> > you suggesting have resize.* deal with partition table stuff?
> 
> Not at all.  The resize.* programs are filesystem specific, and only
> resize from the end of the partition.  I suppose that they could be
> generalized to accept an additional parameter (at the end) to give
> the new start of the partition, e.g. "resize.ext2 /dev/hda1 [size] [start]"

"/dev/hda1" is not as simple as it seems.  It provides a linear address
space from the start of the old partition, to the end of the old
partition.  Where partition is defined by the kernel, BTW.  So, a
resizer
can use that linear address space the kernel provides.  If that is a bad
idea (which it is for move-the-start, and is somewhat dubious for
growth)
- then it must create it's own address space.  i.e. it must understand
partition tables. 

Andrew Clausen