FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

[Miguel Sousa Filipe]

Hello kernel hackers,

Some weeks ago, in a ZFS related thread, some kernel hackers asked the
user what did they liked in ZFS that linux didn't have, so that they
could (possibly) work on it.

So, here is my feature request:
- merge MD software raid framework and LVM in one unique
API/framework, to be controled/managed by one unique userland
toolset.


Any sysadmin which needs to do device & volume management and FS
management in Linux has do use three diferent tools, for 3 diferent
layers (that I think shouldn't be 3 layers.. but only 2).
- md :
   MD is a software raid framework, with raid0, 1, 4, 5, 6, append and others..
- lvm :
  The logical volume manager: this allows you to manage logical disks
(devices), grow a  device, add phisical devices to a existing volume,
snapshot.. etc..
- fs
 duh... the FileSystem level.


However, MD and LVM have feature replications:
- raid1 / mirroring
- raid0 / stripping
- append / concatenate

I find it high irritanting having two kernel interfaces and two
userland tools that provide the same funcionality, which one should I
use?

I also don't understand from the point of view of code maintenance..
shurely there is some unneeded duplicate code lying in the kernel
tree.

One of the  good things (my opinion) about ZFS is putting these things
in one unique place, with a good user interface (the zpool & other zfs
command tools are quite easy and simple to use).

On linux, I find myself having to setup each layer by hand, each with
its own peculiarities and command line tools.. I consede that maybe
the FS layer and the device management layer shouldn't be merged.
But clearly that in Linux we have too different layers (that aren't
really layers since they do not depend on one another) for the same
purpose:
- device manangement.

I'm no kernel hacker, I'm just a simple user, but this is my *pretty
please* that some kernel hackers work together to unite and simplify
the work that has to be made to use software raid and volume
management on Linux.

In essence, I believe that bringing the funcionality that is unique to
MD integrated in LVM would suffice (leaving MD there for legacy compat.. but
warning users about future improvements on LVM only)

Thank you very much,


PS: I'm not subscribed to lkml yet, please CC me any reply
-- 
Miguel Sousa Filipe

Re: FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

[Diego Calleja]

El Wed, 2 May 2007 20:18:55 +0100, "Miguel Sousa Filipe" <miguel.filipe@gmail.com> escribió:

> I find it high irritanting having two kernel interfaces and two
> userland tools that provide the same funcionality, which one should I
> use?

I doubt users care about kernel's design; however the lack of unification of
userspace tools is a real problem. Just my 2¢.

Re: FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

[Miguel Sousa Filipe]

On 5/2/07, Diego Calleja <diegocg@gmail.com> wrote:
> El Wed, 2 May 2007 20:18:55 +0100, "Miguel Sousa Filipe" <miguel.filipe@gmail.com> escribió:
>
> > I find it high irritanting having two kernel interfaces and two
> > userland tools that provide the same funcionality, which one should I
> > use?
>
> I doubt users care about kernel's design; however the lack of unification of
> userspace tools is a real problem. Just my 2¢.
>

I believe they do, since the kernels desing obviate the need and use
of several diferent tools (that shouldn't be needed) like for instance
for having raid5 with snapshot and dinamic partition resizing we will
allways need:

md-raid5
lvm
some FS.

This md-raid5 and lvm separation is the evidence of how the kernels
design & API affect usability and userspace tools that the user is
obliged to use. I cannot have those features without "bumping" into
this kernel design issue.
 This is also a problem for any developer who tries to improve
usability in this area by creating some unified userland tools to
manipulate MD & LVM. (Imagining myself implementing some userland tool
to create some "storage devices" + mount points.. doesn't  seem easy
nor fun..).


Best regards,

-- 
Miguel Sousa Filipe

Re: FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1688 bytes --]

On Wed, 2 May 2007, Miguel Sousa Filipe wrote:

> On 5/2/07, Diego Calleja <diegocg@gmail.com> wrote:
>>  El Wed, 2 May 2007 20:18:55 +0100, "Miguel Sousa Filipe"
>>  <miguel.filipe@gmail.com> escribió:
>> 
>> >  I find it high irritanting having two kernel interfaces and two
>> >  userland tools that provide the same funcionality, which one should I
>> >  use?
>>
>>  I doubt users care about kernel's design; however the lack of unification
>>  of
>>  userspace tools is a real problem. Just my 2¢.
>> 
>
> I believe they do, since the kernels desing obviate the need and use
> of several diferent tools (that shouldn't be needed) like for instance
> for having raid5 with snapshot and dinamic partition resizing we will
> allways need:
>
> md-raid5
> lvm
> some FS.
>
> This md-raid5 and lvm separation is the evidence of how the kernels
> design & API affect usability and userspace tools that the user is
> obliged to use. I cannot have those features without "bumping" into
> this kernel design issue.
> This is also a problem for any developer who tries to improve
> usability in this area by creating some unified userland tools to
> manipulate MD & LVM. (Imagining myself implementing some userland tool
> to create some "storage devices" + mount points.. doesn't  seem easy
> nor fun..).

why do you care if the userspace tool that does the resizing makes system 
calls to one layer or to two layers? how would you know?

currently you see that they are seperate becouse you use two different 
tools to manipulate them, but if it was one tool that manipulated both of 
them for the common cases you wouldn't know. (this is the point that Diego 
was trying to make)

David Lang

It's a bug of printk?

[gshan]

Hi,

I need 2 consoles for 2 individual serial ports that is registered by 
register_console(). The console for the 1st serial port is registered at 
first. I can see the output from the serial port #1 before the console 
for the 2nd serial port is registered. However, I saw duplicated output 
from serial port #1 again after the console for #2 serial port is 
registered.

The root cause is that there is only one con_start for all consoles. I 
think con_start need to be merged with struct console so that different 
console could have different start?

Thanks,
Gavin

Re: FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

[David Greaves]

david@lang.hm wrote:
> On Wed, 2 May 2007, Miguel Sousa Filipe wrote:
> 
>> On 5/2/07, Diego Calleja <diegocg@gmail.com> wrote:
>>>  El Wed, 2 May 2007 20:18:55 +0100, "Miguel Sousa Filipe"
>>>  <miguel.filipe@gmail.com> escribió:
>>>
>>> >  I find it high irritanting having two kernel interfaces and two
>>> >  userland tools that provide the same funcionality, which one should I
>>> >  use?
>>>
>>>  I doubt users care about kernel's design; however the lack of
>>> unification of  userspace tools is a real problem. Just my 2¢.

>> This is also a problem for any developer who tries to improve
>> usability in this area by creating some unified userland tools to
>> manipulate MD & LVM. (Imagining myself implementing some userland tool
>> to create some "storage devices" + mount points.. doesn't  seem easy
>> nor fun..).
> 
> why do you care if the userspace tool that does the resizing makes
> system calls to one layer or to two layers? how would you know?

Indeed!!

EVMS

http://evms.sourceforge.net/

Enterprise Volume Management System

In order to make the transition to EVMS as smooth as possible, EVMS includes
compatibility with a number of existing storage and volume management systems.
Currently, EVMS recognizes:

    * All locally attached disks
    * DOS-style disk partitions (used extensively on Linux systems)
    * GPT disk partitions (mainly used on IA-64)
    * S/390 disk partitions (CDL/LDL)
    * BSD disk partitions
    * Macintosh disk partitions
    * Linux MD/Software-RAID devices
    * Linux LVM volume groups and logical volumes (versions 1 and 2)

Anything else?

Oh... yes:

In addition to providing compatibility with these existing systems, EVMS also
provides new functionality that can be built on top of any of the above
"volumes" that EVMS already recognizes. Features that are currently included are:

    * Bad Block Relocation
    * Linear Drive Linking
    * Generic Snapshotting

Enough? or would you like:
In addition to these volume-level features, the EVMS tools provide convenient
integration with numerous filesystem tools, to allow tasks such as mkfs and fsck
directly from the EVMS user interfaces. Currently, the following filesystems are
supported:

    * Ext2/3
    * JFS
    * ReiserFS
    * XFS
    * Swap
    * OCFS2
    * NTFS
    * FAT

??
Oh, and for the l33t there's a GUI and screenshots...

Of course, in keeping with ZFS, this management layer is all proprietary and
costs megabucks - or is it GPL, can never remember...

Damn that "irritanting" architecture - keeps us from doing cool things...

Seriously - I hope this is useful ;)

David

Re: FEATURE REQUEST: merge MD software raid and LVM in one unique layer.

[Miguel Sousa Filipe]

On 5/3/07, David Greaves <david@dgreaves.com> wrote:
> david@lang.hm wrote:
> > On Wed, 2 May 2007, Miguel Sousa Filipe wrote:
> >
> >> On 5/2/07, Diego Calleja <diegocg@gmail.com> wrote:
> >>>  El Wed, 2 May 2007 20:18:55 +0100, "Miguel Sousa Filipe"
> >>>  <miguel.filipe@gmail.com> escribió:
> >>>
> >>> >  I find it high irritanting having two kernel interfaces and two
> >>> >  userland tools that provide the same funcionality, which one should I
> >>> >  use?
> >>>
> >>>  I doubt users care about kernel's design; however the lack of
> >>> unification of  userspace tools is a real problem. Just my 2¢.
>
> >> This is also a problem for any developer who tries to improve
> >> usability in this area by creating some unified userland tools to
> >> manipulate MD & LVM. (Imagining myself implementing some userland tool
> >> to create some "storage devices" + mount points.. doesn't  seem easy
> >> nor fun..).
> >
> > why do you care if the userspace tool that does the resizing makes
> > system calls to one layer or to two layers? how would you know?
>
> Indeed!!
>
> EVMS
>
> http://evms.sourceforge.net/
>
> Enterprise Volume Management System
>
> In order to make the transition to EVMS as smooth as possible, EVMS includes
> compatibility with a number of existing storage and volume management systems.
> Currently, EVMS recognizes:
>
>     * All locally attached disks
>     * DOS-style disk partitions (used extensively on Linux systems)
>     * GPT disk partitions (mainly used on IA-64)
>     * S/390 disk partitions (CDL/LDL)
>     * BSD disk partitions
>     * Macintosh disk partitions
>     * Linux MD/Software-RAID devices
>     * Linux LVM volume groups and logical volumes (versions 1 and 2)
>
> Anything else?
>
> Oh... yes:
>
> In addition to providing compatibility with these existing systems, EVMS also
> provides new functionality that can be built on top of any of the above
> "volumes" that EVMS already recognizes. Features that are currently included are:
>
>     * Bad Block Relocation
>     * Linear Drive Linking
>     * Generic Snapshotting
>
> Enough? or would you like:
> In addition to these volume-level features, the EVMS tools provide convenient
> integration with numerous filesystem tools, to allow tasks such as mkfs and fsck
> directly from the EVMS user interfaces. Currently, the following filesystems are
> supported:
>
>     * Ext2/3
>     * JFS
>     * ReiserFS
>     * XFS
>     * Swap
>     * OCFS2
>     * NTFS
>     * FAT
>
> ??
> Oh, and for the l33t there's a GUI and screenshots...
>
> Of course, in keeping with ZFS, this management layer is all proprietary and
> costs megabucks - or is it GPL, can never remember...
>
> Damn that "irritanting" architecture - keeps us from doing cool things...
>
> Seriously - I hope this is useful ;)
>
> David
>

That was indeed informative, thank you for your reply.
I'll try EVMS sometime soon then.. :D

-- 
Miguel Sousa Filipe