Re: Device-mapper submission for 2.4

Re: Device-mapper submission for 2.4

[Carl-Daniel Hailfinger]

Marcelo Tosatti wrote:
> On Tue, 9 Dec 2003, Joe Thornber wrote:
> 
>> On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
>> > I believe 2.6 is the right place for the device mapper.
>>
>> So what's the difference between a new filesystem like XFS and a new
>> device driver like dm ?
> 
> Expected question...
> 
> XFS is a totally different filesystem from the ones present in 2.4.

Please give me a pointer about what's so different about XFS. Last time I
looked, XFS features were mostly equivalent to those of other journaling
file systems.

This is a honest question, not a flamebait.


Thanks,
Carl-Daniel

Device-mapper submission for 2.4

[Joe Thornber]

Marcello,

This set of patches is the core of device mapper for 2.4.  I would
appreciate it if you could merge these into 2.4.24 please.

Thanks,

- Joe

Re: Device-mapper submission for 2.4

[Marcelo Tosatti]

On Tue, 9 Dec 2003, Joe Thornber wrote:

> Marcello,
> 
> This set of patches is the core of device mapper for 2.4.  I would
> appreciate it if you could merge these into 2.4.24 please.

Joe, 

I believe 2.6 is the right place for the device mapper. 

Re: Device-mapper submission for 2.4

[Joe Thornber]

On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
> I believe 2.6 is the right place for the device mapper. 

So what's the difference between a new filesystem like XFS and a new
device driver like dm ?

- Joe

Re: Device-mapper submission for 2.4

[Måns Rullgård]

Joe Thornber <thornber@sistina.com> writes:

>> I believe 2.6 is the right place for the device mapper. 
>
> So what's the difference between a new filesystem like XFS and a new
> device driver like dm ?

None.  Neither will go into 2.4, if I've understood things correctly.

-- 
Måns Rullgård
mru@kth.se

Re: Device-mapper submission for 2.4

[Muli Ben-Yehuda]

[-- Attachment #1: Type: text/plain, Size: 602 bytes --]

On Tue, Dec 09, 2003 at 03:00:15PM +0100, M?ns Rullg?rd wrote:
> Joe Thornber <thornber@sistina.com> writes:
> 
> >> I believe 2.6 is the right place for the device mapper. 
> >
> > So what's the difference between a new filesystem like XFS and a new
> > device driver like dm ?
> 
> None.  Neither will go into 2.4, if I've understood things
> correctly.

You haven't been following lkml, I guess. xfs has been merged to 2.4. 

Cheers, 
Muli 
-- 
Muli Ben-Yehuda
http://www.mulix.org | http://mulix.livejournal.com/

"the nucleus of linux oscillates my world" - gccbot@#offtopic


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Device-mapper submission for 2.4

[Måns Rullgård]

Muli Ben-Yehuda <mulix@mulix.org> writes:

>> >> I believe 2.6 is the right place for the device mapper. 
>> >
>> > So what's the difference between a new filesystem like XFS and a new
>> > device driver like dm ?
>> 
>> None.  Neither will go into 2.4, if I've understood things
>> correctly.
>
> You haven't been following lkml, I guess. xfs has been merged to 2.4. 

Yes, I see that now.  After Marcelo had said "no" about a hundred
times, I stopped reading anything with xfs in the subject.  I guess I
should have looked a little bit closer at that last subject.

-- 
Måns Rullgård
mru@kth.se

Re: Device-mapper submission for 2.4

[Joe Thornber]

On Tue, Dec 09, 2003 at 03:00:15PM +0100, M?ns Rullg?rd wrote:
> None.  Neither will go into 2.4, if I've understood things correctly.

http://kerneltrap.org/node/view/1751

Re: Device-mapper submission for 2.4

[Stefan Smietanowski]

Måns Rullgård wrote:

> Joe Thornber <thornber@sistina.com> writes:
> 
> 
>>>I believe 2.6 is the right place for the device mapper. 
>>
>>So what's the difference between a new filesystem like XFS and a new
>>device driver like dm ?
> 
> 
> None.  Neither will go into 2.4, if I've understood things correctly.
> 

XFS is in latest BK.

// Stefan

Re: Device-mapper submission for 2.4

[Marcelo Tosatti]

On Tue, 9 Dec 2003, Joe Thornber wrote:

> On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
> > I believe 2.6 is the right place for the device mapper. 
> 
> So what's the difference between a new filesystem like XFS and a new
> device driver like dm ?

Expected question... 

XFS is a totally different filesystem from the ones present in 2.4. 

As far as I know, we already have the similar functionality in 2.4 with
LVM. Device mapper provides the same functionality but in a much cleaner
way. Is that right?

Re: Device-mapper submission for 2.4

[Joe Thornber]

On Tue, Dec 09, 2003 at 12:10:06PM -0200, Marcelo Tosatti wrote:
> 
> 
> On Tue, 9 Dec 2003, Joe Thornber wrote:
> 
> > On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
> > > I believe 2.6 is the right place for the device mapper. 
> > 
> > So what's the difference between a new filesystem like XFS and a new
> > device driver like dm ?
> 
> Expected question... 
> 
> XFS is a totally different filesystem from the ones present in 2.4. 
> 
> As far as I know, we already have the similar functionality in 2.4 with
> LVM. Device mapper provides the same functionality but in a much cleaner
> way. Is that right?

Sort of, but please take into account the fact that the LVM1 driver
has bugs (particularly on the failure paths), and EVMS and other
volume managers dont use the LVM1 driver.  The snapshot target (which
I didn't include in the core patches) is hugely better performance
wise.

- Joe

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Tue, 9 Dec 2003, Joe Thornber wrote:

> Sort of, but please take into account the fact that the LVM1 driver
> has bugs (particularly on the failure paths), and EVMS and other
> volume managers dont use the LVM1 driver.  The snapshot target
> (which I didn't include in the core patches) is hugely better
> performance wise.

Would this be of any aid to 2.4 users to transition to DM, so that
they can then easily test 2.6 and boot back to 2.4 if needs be?

If so, my vote would be for it to be included in 2.4.

> - Joe

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
"The Right Honorable Gentleman is indebted to his memory for his jests
and to his imagination for his facts."
		-- Sheridan

Re: Device-mapper submission for 2.4

[Joe Thornber]

On Tue, Dec 09, 2003 at 09:07:49PM +0000, Paul Jakma wrote:
> On Tue, 9 Dec 2003, Joe Thornber wrote:
> Would this be of any aid to 2.4 users to transition to DM, so that
> they can then easily test 2.6 and boot back to 2.4 if needs be?
> 
> If so, my vote would be for it to be included in 2.4.

yes

Re: Device-mapper submission for 2.4

[Marcelo Tosatti]

On Tue, 9 Dec 2003, Joe Thornber wrote:

> On Tue, Dec 09, 2003 at 09:07:49PM +0000, Paul Jakma wrote:
> > On Tue, 9 Dec 2003, Joe Thornber wrote:
> > Would this be of any aid to 2.4 users to transition to DM, so that
> > they can then easily test 2.6 and boot back to 2.4 if needs be?
> > 
> > If so, my vote would be for it to be included in 2.4.
> 
> yes

I wont merge it Joe.

Its nothing against your or DM itself. 

Let DM be in 2.6. 

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Tue, 9 Dec 2003, Marcelo Tosatti wrote:

> Its nothing against your or DM itself.
> 
> Let DM be in 2.6. 

Well, how does this leave 2.4 LVM1 users? From my vague 
understanding:

- 2.6 DM does not support the LVM1 interface
- The DM tools library is dropping support for the LVM1 interface

This leaves 2.4 LVM1 users with a /huge/ leap to take if they wish to
test 2.6. Backward compatibility is awkward because of the DM tools
issue (need both old and new installed and some way to pick at boot,
or manually setup LVM), and you're ruling out the other option of
adding forwards compatibility to 2.4.

This isnt a new fs which 2.4 users wont be using, its an existing 
feature that has been reworked during 2.5 and is now incompatible in 
2.6 with 2.4. More over, its a feature on which access to data 
depends.

I'd really like to see one of:

- backwards compat: 2.6 have LVM1 support

- forward compat: 2.4 to have DM support to allow 2.4 users to 
migrate
LVM->DM first /before/ taking the risk on running 2.6.

- the DM tools to support both LVM1 and LVMx in 2.6, on a *long-term* 
  basis

I or others may not migrate to 2.6 for many a year, and when we do,
it'd nice to be able to migrate our data in place (not
backup&restore). Kernel interface compat at least tends to be the
most set in stone and is what I would prefer. Whether forward or
backward doesnt matter, adding compat cruft to a soon-to-be obsolete 
kernel is possibly better than weighing 2.6 down with it for the next 
3+ years.

There are people who store their data in LVM, we need compatibility,
and ideally we'd like to be able to migrate in small steps.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
You can write a small letter to Grandma in the filename.
		-- Forbes Burkowski, CS, University of Washington

Re: Device-mapper submission for 2.4

[William Lee Irwin III]

On Tue, Dec 09, 2003 at 11:46:13PM +0000, Paul Jakma wrote:
> This leaves 2.4 LVM1 users with a /huge/ leap to take if they wish to
> test 2.6. Backward compatibility is awkward because of the DM tools
> issue (need both old and new installed and some way to pick at boot,
> or manually setup LVM), and you're ruling out the other option of
> adding forwards compatibility to 2.4.
> This isnt a new fs which 2.4 users wont be using, its an existing 
> feature that has been reworked during 2.5 and is now incompatible in 
> 2.6 with 2.4. More over, its a feature on which access to data 
> depends.

Just apply the patch if you're for some reason terrified of 2.6.


-- wli

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Tue, 9 Dec 2003, William Lee Irwin III wrote:

> Just apply the patch if you're for some reason terrified of 2.6.

Or get RedHat or Fedora to apply the patch.

Its a slightly safer bet though to have it in stock 2.4, guarantees
it will be there if one needs it 2 years down the road when upgrading
some box. (and non-LVM users wont be compiling it in).

So personally I'd rather Marcelo included it, being paranoid about
having support for access to data.

> -- wli

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
"If you want to travel around the world and be invited to speak at a lot
of different places, just write a Unix operating system."
(By Linus Torvalds)

Re: Device-mapper submission for 2.4

[Stephan von Krawczynski]

On Wed, 10 Dec 2003 00:15:17 +0000 (GMT)
Paul Jakma <paul@clubi.ie> wrote:

> On Tue, 9 Dec 2003, William Lee Irwin III wrote:
> 
> > Just apply the patch if you're for some reason terrified of 2.6.
> 
> Or get RedHat or Fedora to apply the patch.

There it is again, this /dev/null argument.

"Multi-billion dollar companies" have gone bancrupt on the simple fact that
diversification of one product can rattle customers/users to a degree that they
in fact decide against the whole product range.

IOW go on with the idea to spread around an unknown number of kernel versions
and you can be sure that linux as a whole will greatly suffer.

This is a "user" issue, not a "developer" issue of course. Developers can apply
any kind of patches they like, but don't go and tell the vast user base to
"just apply patch xyz". They won't honor this at all, your level of acceptance
will dramatically drop.

Regards,
Stephan

Re: Device-mapper submission for 2.4

[Dave Jones]

On Wed, Dec 10, 2003 at 12:15:17AM +0000, Paul Jakma wrote:
 > > Just apply the patch if you're for some reason terrified of 2.6.
 > Or get RedHat or Fedora to apply the patch.

This isn't going to happen for Fedora.

		Dave

Re: Device-mapper submission for 2.4

[Jose Luis Domingo Lopez]

On Tuesday, 09 December 2003, at 23:46:13 +0000,
Paul Jakma wrote:

> There are people who store their data in LVM, we need compatibility,
> and ideally we'd like to be able to migrate in small steps.
> 
Install "module-init-tools", install "LVM2" (that can drive both LVM1
and DM Logical Volumes), compile a 2.6.x Linux kernel, reboot and you
should be done.

As far as I remember, migration is just that easy, and you can always go
back to plain 2.4.x while you don't update LVM metadata to newer version 2.

Greetings.

-- 
Jose Luis Domingo Lopez
Linux Registered User #189436     Debian Linux Sid (Linux 2.6.0-test10-mm1)

Re: Device-mapper submission for 2.4

[Tupshin Harper]

Jose Luis Domingo Lopez wrote:

>On Tuesday, 09 December 2003, at 23:46:13 +0000,
>Paul Jakma wrote:
>
>  
>
>>There are people who store their data in LVM, we need compatibility,
>>and ideally we'd like to be able to migrate in small steps.
>>
>>    
>>
>Install "module-init-tools", install "LVM2" (that can drive both LVM1
>and DM Logical Volumes), compile a 2.6.x Linux kernel, reboot and you
>should be done.
>
>As far as I remember, migration is just that easy, and you can always go
>back to plain 2.4.x while you don't update LVM metadata to newer version 2.
>
>Greetings.
>
>  
>
This is not true. LVM2 can read the LVM1 format, but it cannot 
communicate with non-dm interfaces in 2.4.x. This means that you need to 
run lvm1 on 2.4 and lvm2 on 2.6 unless you patch 2.4 with dm.

If this were the whole story, then it would be an amazingly painful 
transition to (safely) upgrade an lvm machine from 2.4 to 2.6 (upgrade 
to patched 2.4, then upgrade to 2.6). Luckily, debian has made the lvm1 
and lvm2 packages not conflict, and the correct ones runs at startup 
depending on which kernel you have. This is probably a feature that all 
distros will have to adopt to ease the upgrade cycle.

-Tupshin

Re: Device-mapper submission for 2.4

[Wichert Akkerman]

Previously Tupshin Harper wrote:
> This is not true. LVM2 can read the LVM1 format, but it cannot 
> communicate with non-dm interfaces in 2.4.x. This means that you need to 
> run lvm1 on 2.4 and lvm2 on 2.6 unless you patch 2.4 with dm.

And unless my memory is failing me all distros ship tools that will
detect which interface your system has and call the right tool for you.
That was already needed for lvm1 which went through several ioctl
interfaces and continues to work fine for lvm2. Which means this 
is pretty much a non-issue.

Wichert.

-- 
Wichert Akkerman <wichert@wiggy.net>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

Re: Device-mapper submission for 2.4

[Martin J. Bligh]

> I'd really like to see one of:
> 
> - backwards compat: 2.6 have LVM1 support
>
> - the DM tools to support both LVM1 and LVMx in 2.6, on a *long-term* 
>   basis

Some form of backward compatibility from 2.6 would seem a much more 
sensible thing to fight for. Foisting forward comaptibility on an 
older release seems like a bad road to go down.
 
M.

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Tue, 9 Dec 2003, Martin J. Bligh wrote:

> Some form of backward compatibility from 2.6 would seem a much more
> sensible thing to fight for. Foisting forward comaptibility on an
> older release seems like a bad road to go down.

I dont really care, but some kind of (long-term, ie lifetime of 
either 2.4 or 2.6) compatibility is needed.

LVM1 kernel support was recently removed from 2.6.0, so it would have 
to be added back in. 

One argument for adding forward compatibility in 2.4 is that it will 
/force/ people to move to DM before going to 2.6, which might be a 
good thing as, AIUI, LVM1 has problems. 

Its a choice between:

- 2.6 backwards compatibility, adding back in LVM1 support, LVM1 
users will then quite possibly continue to use the problematical LVM1 
interfaces even after migrating to 2.6.

- 2.4 forwards compatibility, add DM support - which appears (IMVU)  
to drop in cleanly alongside MD - and hence 2.6 can remain 'clean'.

I dont know, but it would be nice to have /something/ and to have it 
in stock kernel rather than /hope/ to have upstreams include the 
required backward or forward compatibility.

> M.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
But it does move!
		-- Galileo Galilei

Re: Device-mapper submission for 2.4

[venom]

DM is back compatible with LVM1, tested and runs well.

Of course LVM1 has problems, but should we consider the DM case as mutch the
same as XFS?

Luigi

On Wed, 10 Dec 2003, Paul Jakma wrote:

> Date: Wed, 10 Dec 2003 15:55:53 +0000 (GMT)
> From: Paul Jakma <paul@clubi.ie>
> To: Martin J. Bligh <mbligh@aracnet.com>
> Cc: Marcelo Tosatti <marcelo.tosatti@cyclades.com>,
>      Joe Thornber <thornber@sistina.com>, linux-kernel@vger.kernel.org
> Subject: Re: Device-mapper submission for 2.4
>
> On Tue, 9 Dec 2003, Martin J. Bligh wrote:
>
> > Some form of backward compatibility from 2.6 would seem a much more
> > sensible thing to fight for. Foisting forward comaptibility on an
> > older release seems like a bad road to go down.
>
> I dont really care, but some kind of (long-term, ie lifetime of
> either 2.4 or 2.6) compatibility is needed.
>
> LVM1 kernel support was recently removed from 2.6.0, so it would have
> to be added back in.
>
> One argument for adding forward compatibility in 2.4 is that it will
> /force/ people to move to DM before going to 2.6, which might be a
> good thing as, AIUI, LVM1 has problems.
>
> Its a choice between:
>
> - 2.6 backwards compatibility, adding back in LVM1 support, LVM1
> users will then quite possibly continue to use the problematical LVM1
> interfaces even after migrating to 2.6.
>
> - 2.4 forwards compatibility, add DM support - which appears (IMVU)
> to drop in cleanly alongside MD - and hence 2.6 can remain 'clean'.
>
> I dont know, but it would be nice to have /something/ and to have it
> in stock kernel rather than /hope/ to have upstreams include the
> required backward or forward compatibility.
>
> > M.
>
> regards,
> --
> Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
> 	warning: do not ever send email to spam@dishone.st
> Fortune:
> But it does move!
> 		-- Galileo Galilei
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Wed, 10 Dec 2003 venom@sns.it wrote:

> DM is back compatible with LVM1, tested and runs well.

What about the patches posted by Joe last (?) week which remove LVM1 
support from 2.6 DM? (if Linus hasnt picked them up, its surely an 
omen the support will go once the bug-only freeze is lifted (?)).

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
Someone is unenthusiastic about your work.

Re: Device-mapper submission for 2.4

[venom]

wow, I did not  notice that.

ok, if it is so, it's really different.

On Wed, 10 Dec 2003, Paul Jakma wrote:

> Date: Wed, 10 Dec 2003 17:00:43 +0000 (GMT)
> From: Paul Jakma <paul@clubi.ie>
> To: venom@sns.it
> Cc: Martin J. Bligh <mbligh@aracnet.com>,
>      Marcelo Tosatti <marcelo.tosatti@cyclades.com>,
>      Joe Thornber <thornber@sistina.com>, linux-kernel@vger.kernel.org
> Subject: Re: Device-mapper submission for 2.4
>
> On Wed, 10 Dec 2003 venom@sns.it wrote:
>
> > DM is back compatible with LVM1, tested and runs well.
>
> What about the patches posted by Joe last (?) week which remove LVM1
> support from 2.6 DM? (if Linus hasnt picked them up, its surely an
> omen the support will go once the bug-only freeze is lifted (?)).
>
> regards,
> --
> Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
> 	warning: do not ever send email to spam@dishone.st
> Fortune:
> Someone is unenthusiastic about your work.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Device-mapper submission for 2.4

[Mike Fedyk]

On Wed, Dec 10, 2003 at 05:00:43PM +0000, Paul Jakma wrote:
> On Wed, 10 Dec 2003 venom@sns.it wrote:
> 
> > DM is back compatible with LVM1, tested and runs well.
> 
> What about the patches posted by Joe last (?) week which remove LVM1 
> support from 2.6 DM? (if Linus hasnt picked them up, its surely an 

If this is what I was reading being discussed a few weeks ago, then the
support for the LVM1 sysctls/ioctls has/will be removed, so you will have to
use the DM utilities instead of the old LVM1 utilities.  LVM1 on-disk format
should still be supported.

Re: Device-mapper submission for 2.4

[Alasdair G Kergon]

On Wed, Dec 10, 2003 at 03:40:07PM -0800, Mike Fedyk wrote:
> On Wed, Dec 10, 2003 at 05:00:43PM +0000, Paul Jakma wrote:
> > On Wed, 10 Dec 2003 venom@sns.it wrote:
> > > DM is back compatible with LVM1, tested and runs well.
> > What about the patches posted by Joe last (?) week which remove LVM1 
> > support from 2.6 DM? 

They remove support for the broken version 1 of the device-mapper 
ioctl interface.  This is nothing to do with LVM1.
 
> If this is what I was reading being discussed a few weeks ago, then the
> support for the LVM1 sysctls/ioctls has/will be removed, so you will have to
> use the DM utilities instead of the old LVM1 utilities.  LVM1 on-disk format
> should still be supported.

2.6 does not support LVM1 ioctls.
LVM2 userspace tools and EVMS both support LVM1 on-disk format using
device-mapper.


Here's a reference sheet to help clarify the terminology and explain
what's happening.

LVM1 = Userspace tools + kernel ioctls included in marcelo's 2.4 tree
  - LVM1 kernel ioctls are *not* included in or available for 2.6
  - LVM1 userspace tools do *not* work with 2.6 kernels

dm = Kernel driver (GPL) for new volume managers to use.
  - Included in Linus's 2.6 kernels.
  - Available as a patch for 2.4 kernels from the Sistina website.
  - Knows *nothing* about volume manager's on-disk metadata layouts.
  - Userspace volume managers (e.g. EVMS and LVM2) communicate via a new 
    ioctl interface.
  - This ioctl interface is currently "version 4" and we regard it as
    stable.  [Some enhancements are on the horizon, but nothing that 
    breaks existing code/binaries.]
  - An old development version of this device-mapper ioctl interface known
    as "version 1" has problems with it, is deprecated and should be
    removed from kernel trees ASAP.  
    Always use "version 4" when building new kernels today.

libdevmapper = Userspace shared library (LGPL) which wraps a volume manager 
               application interface around the device-mapper ioctls
  - Can determine transparently whether the kernel device-mapper is using
    "version 4" dm ioctl interface or the deprecated "version 1" interface
    and adapt itself accordingly.  [configure --enable-compat]
  - Can only communicate with device-mapper: it cannot use LVM1 ioctls.
  - Designed primarily for use by LVM2 tools.  [EVMS does not use it]
  - Some parts of the libdevmapper API are not yet stable and are likely 
    to get changed.

dmsetup = Userspace utility (GPL) which provides full command-line access to
          the libdevmapper API.
  - Designed for use by shell scripts and for testing and debugging.
  - Command line interface may be considered stable.  New features may get 
    added, but we'll try not to break existing commands.

LVM2 = New Logical Volume Manager command line tools (GPL) designed to
       be backward-compatible with LVM1 and offering new features and
       more flexibility, configurability and stability.
  - Supports existing LVM1 on-disk metadata.
    This means you do *not* have to make changes to your existing on-disk 
    LVM1 volumes to switch between using LVM1 and LVM2.
  - Uses command lines similar to LVM1.
  - By default uses a new on-disk metadata format supporting more
    features than the original LVM1 version.
  - Communicates with the device-mapper kernel driver via libdevmapper's
    API.


Miscellaneous points:
  - LVM1 uses block major number 58: dm selects one or more major numbers
    dynamically as required instead.
  - LVM1 uses character major number 109: dm selects a misc minor number
    dynamically instead.
  - There's a (non-devfs) script for creating /dev/mapper/control at
    startup (or after dm module load).
  - You can use LVM1 tools with unpatched 2.4 kernels.
  - You can use LVM2 tools with patched 2.4 and unpatched 2.6 kernels.
  - Device-mapper support for snapshots and pvmove is so far released 
    only for 2.4.  Patches for 2.6 are being tested.
  - Multipath and mirror support are under development for 2.6.
    (Then get back-ported to 2.4.)

Web download page: http://www.sistina.com/products_lvm_download.htm

The device-mapper tarball contains: 
  device-mapper kernel patches - needed only for 2.4;
  userspace libdevmapper and dmsetup - needed with all dm kernels.
The LVM2 tarball contains the LVM2 command line tools.

Development code can be found via:
  http://people.sistina.com/~thornber/  (for kernel patches)
  http://www.sistina.com/products_CVS.htm  (for userspace code)

Device-mapper mailing list:
  http://lists.sistina.com/mailman/listinfo/dm-devel

Alasdair
-- 
agk@uk.sistina.com

Re: Device-mapper submission for 2.4

[bill davidsen]

In article <Pine.LNX.4.56.0312101547590.1218@fogarty.jakma.org>,
Paul Jakma  <paul@clubi.ie> wrote:
| On Tue, 9 Dec 2003, Martin J. Bligh wrote:
| 
| > Some form of backward compatibility from 2.6 would seem a much more
| > sensible thing to fight for. Foisting forward comaptibility on an
| > older release seems like a bad road to go down.
| 
| I dont really care, but some kind of (long-term, ie lifetime of 
| either 2.4 or 2.6) compatibility is needed.

Where on earth did you get that? Is this some new policy Linus has put
forth, or something you wish were real? It certainly wasn't the case for
2.2 => 2.4 conversion, where is it writ that LVM1 needs to get
conversion help?
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Device-mapper submission for 2.4

[bill davidsen]

In article <Pine.LNX.4.56.0312092329280.30298@fogarty.jakma.org>,
Paul Jakma  <paul@clubi.ie> wrote:

| I'd really like to see one of:
| 
| - backwards compat: 2.6 have LVM1 support
| 
| - forward compat: 2.4 to have DM support to allow 2.4 users to 
| migrate
| LVM->DM first /before/ taking the risk on running 2.6.

Hate to say it, but unlike XFS which has been available for 2.4 for ages
and very well tested, DM for 2.4 has all the joy of a newly posted
feature. I really think that you will find the *if* you want DM you will
be safer going to 2.6 and using the version which has been reasonably
well tested.

You will want to do a full backup before going to a new o/s in any case,
if your data is of value.
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Device-mapper submission for 2.4

[Jens Axboe]

On Tue, Dec 09 2003, Joe Thornber wrote:
> On Tue, Dec 09, 2003 at 09:07:49PM +0000, Paul Jakma wrote:
> > On Tue, 9 Dec 2003, Joe Thornber wrote:
> > Would this be of any aid to 2.4 users to transition to DM, so that
> > they can then easily test 2.6 and boot back to 2.4 if needs be?
> > 
> > If so, my vote would be for it to be included in 2.4.
> 
> yes

Seems to me, it's the lvm2 teams responsibility to provide easy
transition to 2.6 from 2.4. Merging dm in 2.4 right now looks like a
step in the wrong direction.

Arguments akin to "But XFS got merged, surely we can to" don't hold up
one bit. Should be obvious why.

-- 
Jens Axboe

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Wed, 10 Dec 2003, Jens Axboe wrote:

> Arguments akin to "But XFS got merged, surely we can to" don't hold
> up one bit. Should be obvious why.

Its not about a /new/ feature, its about an existing feature which is 
incompatible between 2.4 and 2.6.

I dont really care whether its done via forward or backware compat. 
(but why was LVM1 removed from 2.6?)

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
[Washington, D.C.] is the home of... taste for the people -- the big,
the bland and the banal.
		-- Ada Louise Huxtable

Re: Device-mapper submission for 2.4

[Joe Thornber]

On Wed, Dec 10, 2003 at 05:30:01PM +0000, Paul Jakma wrote:
> On Wed, 10 Dec 2003, Jens Axboe wrote:
> 
> > Arguments akin to "But XFS got merged, surely we can to" don't hold
> > up one bit. Should be obvious why.
> 
> Its not about a /new/ feature, its about an existing feature which is 
> incompatible between 2.4 and 2.6.
> 
> I dont really care whether its done via forward or backware compat. 
> (but why was LVM1 removed from 2.6?)

The LVM1 driver was removed because dm covered the same functionality
+ lots more, and is more flexible.  The LVM2 tools still understand
the LVM1 metadata format, so there is no problem about not being able
to read data in 2.6.  The main reason for submitting dm to 2.4 was
that there are a lot of people out there who want to use LVM2/EVMS
tools with 2.4, and kept asking me to do it.  If this is against
Marcelos current policy then so be it; I probably should have checked
with him before spamming lkml with the submission.  I don't want this
to degenerate into the old LVM1 vs dm argument; people can search the
archives for that.

- Joe

Re: Device-mapper submission for 2.4

[venom]

On Wed, 10 Dec 2003, Joe Thornber wrote:

>
> The LVM1 driver was removed because dm covered the same functionality
> + lots more, and is more flexible.  The LVM2 tools still understand
> the LVM1 metadata format, so there is no problem about not being able
> to read data in 2.6.

So I was right. Well, if back compatibility works, this solves most of the
problem.

> The main reason for submitting dm to 2.4 was
> that there are a lot of people out there who want to use LVM2/EVMS
> tools with 2.4, and kept asking me to do it.  If this is against
> Marcelos current policy then so be it; I probably should have checked
> with him before spamming lkml with the submission.

This is a good point, but patches are available, so those people can stil use
it, am I wrong?

Luigi

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Wed, 10 Dec 2003, Joe Thornber wrote:

> The LVM1 driver was removed because dm covered the same
> functionality + lots more, and is more flexible.

Yes, DM seems quite nice.

> The LVM2 tools still understand the LVM1 metadata format, so there
> is no problem about not being able to read data in 2.6.

Ah, and this capability is /not/ going away? If so, then that works 
for me. Its just i got the vague impression that support was going 
to be excised at some stage soonish, which is what worries me. If 
not, apologies, and then indeed there's no reason for DM in 2.4.

> - Joe

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
Will you loan me $20.00 and only give me ten of it?
That way, you will owe me ten, and I'll owe you ten, and we'll be even!

Re: Device-mapper submission for 2.4

[Jens Axboe]

On Wed, Dec 10 2003, Joe Thornber wrote:
> On Wed, Dec 10, 2003 at 05:30:01PM +0000, Paul Jakma wrote:
> > On Wed, 10 Dec 2003, Jens Axboe wrote:
> > 
> > > Arguments akin to "But XFS got merged, surely we can to" don't hold
> > > up one bit. Should be obvious why.
> > 
> > Its not about a /new/ feature, its about an existing feature which is 
> > incompatible between 2.4 and 2.6.
> > 
> > I dont really care whether its done via forward or backware compat. 
> > (but why was LVM1 removed from 2.6?)
> 
> The LVM1 driver was removed because dm covered the same functionality
> + lots more, and is more flexible.  The LVM2 tools still understand
> the LVM1 metadata format, so there is no problem about not being able
> to read data in 2.6.  The main reason for submitting dm to 2.4 was

Great, so then there's zero reason to merge it in 2.4.

Re: Device-mapper submission for 2.4

[Bill Rugolsky Jr.]

On Tue, Dec 09, 2003 at 12:10:06PM -0200, Marcelo Tosatti wrote:
> As far as I know, we already have the similar functionality in 2.4 with
> LVM. Device mapper provides the same functionality but in a much cleaner
> way. Is that right?

Yes.
 
And migration of root-on-LVM users to 2.6 will be *greatly* helped if users
can get LVM2/DM working on 2.4 (by upgrading lvm/initscripts/mkinitrd),
and then move to 2.6.

And LVM1 snapshots in 2.4 have limited value, due to the performance impact.

    Bill Rugolsky

Re: Device-mapper submission for 2.4

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]

On Tue, 9 Dec 2003 12:02:50 -0500 "Bill Rugolsky Jr."
<brugolsky@telemetry-investments.com> wrote:
| On Tue, Dec 09, 2003 at 12:10:06PM -0200, Marcelo Tosatti wrote:
| > As far as I know, we already have the similar functionality in 2.4
| > with LVM. Device mapper provides the same functionality but in a
| > much cleaner way. Is that right?
| 
| Yes.
|  
| And migration of root-on-LVM users to 2.6 will be *greatly* helped if
| users can get LVM2/DM working on 2.4 (by upgrading
| lvm/initscripts/mkinitrd), and then move to 2.6.

Agreed. Early 2.6test kernels had serious keyboard issues on my laptop,
and the effort of switching backwards and forwards between LVM1 and 2
has put me off trying again on that box. If I could run 2.4 with LVM2
easily without having to worry about yet another manual kernel patch I'd
be a lot more inclined to do testing.

-- 
Ciaran McCreesh
Mail:    ciaranm at gentoo.org
Web:     http://dev.gentoo.org/~ciaranm



[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Device-mapper submission for 2.4

[Lincoln Dale]

At 04:02 AM 10/12/2003, Bill Rugolsky Jr. wrote:
>On Tue, Dec 09, 2003 at 12:10:06PM -0200, Marcelo Tosatti wrote:
> > As far as I know, we already have the similar functionality in 2.4 with
> > LVM. Device mapper provides the same functionality but in a much cleaner
> > way. Is that right?
>
>Yes.
>
>And migration of root-on-LVM users to 2.6 will be *greatly* helped if users
>can get LVM2/DM working on 2.4 (by upgrading lvm/initscripts/mkinitrd),
>and then move to 2.6.

i concur with this.
Marcello: try to migrate from a root-on-LVM1/2.4 to LVM2/2.6; it is very 
painful.  major/minor # changes, more stuff required in initrd, "dm" 
doesn't appear in 2.6's /proc/partitions . . .

it is a painful upgrade - probably partly due to lack of 
tools/documentation on DMs part, but also equally because 2.4->2.6 is a bug 
jump in a kernel and its exacerbated by LVM1->LVM2 changes...


cheers,

lincoln.

Re: Device-mapper submission for 2.4

[Willy Tarreau]

On Wed, Dec 10, 2003 at 02:38:02PM +1100, Lincoln Dale wrote:
 
> i concur with this.
> Marcello: try to migrate from a root-on-LVM1/2.4 to LVM2/2.6; it is very 
> painful.  major/minor # changes, more stuff required in initrd, "dm" 
> doesn't appear in 2.6's /proc/partitions . . .
> 
> it is a painful upgrade - probably partly due to lack of 
> tools/documentation on DMs part, but also equally because 2.4->2.6 is a bug 
> jump in a kernel and its exacerbated by LVM1->LVM2 changes...

And what next ? people will ask "marcelo, please include initramfs support,
it will help us migrating", "marcelo, it's annoying to support both
module-init-tools and modutils, please accept this patch to change all modules
to 2.6 format", "marcelo, my usb memory stick is only supported in 2.6, please
include it in 2.4 so that I can use it to backup my system in case 2.6 crashes",
"marcelo, please include preempt, it's already in 2.6 and my desktop feels
smoother with it"...

If 2.6 breaks some backwards compatibility, which kernel do you think should
be changed ? Did anybody submit a patch to include netfilter support in 2.2
in case people would finally switch their firewall back to 2.2 when 2.4 was
unstable ? no.

I agree it's important to be able to upgrade and downgrade with a maximum
safety. But frankly, when you know that your data are so much important when
migrating to the new stable kernel, don't you believe you will backup them
first instead something weird happens ? Then they can be restored into a
common format. That's what I did when I used reiserfs 3.5 on raid5 in 2.2
when I switched to 2.4. Converting everything to ext2 was safer than risking
to rely on a not wide tested compatibility glue between the kernels.

It was the same for XFS imho. All XFS users once had the ability to patch
and install it themselves, and should still have the ability to continue
this way. OK this is annoying, and I too am happy that Marcelo makes it
easier now for them. There also are good reasons in case of DM. But we
should also consider that including any patch regularly breaks other
patches and makes it worse for many other people to include external
patches. So the question remains : what next ? 2.4 is definitely not what
I consider a "stable kernel", it's rather the "most stable actively
developped branch". Getting only bugfixes in it would be fairly simpler
for all people using it in production.

Cheers,
Willy

Re: Device-mapper submission for 2.4

[viro]

On Wed, Dec 10, 2003 at 07:12:13AM +0100, Willy Tarreau wrote:
 
> And what next ? people will ask "marcelo, please include initramfs support,
> it will help us migrating", "marcelo, it's annoying to support both
> module-init-tools and modutils, please accept this patch to change all modules
> to 2.6 format", "marcelo, my usb memory stick is only supported in 2.6, please
> include it in 2.4 so that I can use it to backup my system in case 2.6 crashes",
> "marcelo, please include preempt, it's already in 2.6 and my desktop feels
> smoother with it"...

Heh.  Actually, 99% of initramfs support is there - the only piece missing
is unpack_to_rootfs().  IOW, rootfs is there in the same way as on 2.6,
but it isn't pre-populated.  By now it's too late, but a couple of months
ago it would be a trivial enough for backport - init/initramfs.c is
self-contained and it would be a matter of copying several kilobytes of
stuff in 2.4 + adding a section to ld script (same on all architectures)
+ adding one line in init/main.c.  It's nowhere near as intrusive as other
changes on the list (including dm), but it *is* too late for any of that
stuff in 2.4.

Re: Device-mapper submission for 2.4

[Kevin Corry]

On Tuesday 09 December 2003 08:10, Marcelo Tosatti wrote:
> On Tue, 9 Dec 2003, Joe Thornber wrote:
> > On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
> > > I believe 2.6 is the right place for the device mapper.
> >
> > So what's the difference between a new filesystem like XFS and a new
> > device driver like dm ?
>
> Expected question...
>
> XFS is a totally different filesystem from the ones present in 2.4.
>
> As far as I know, we already have the similar functionality in 2.4 with
> LVM. Device mapper provides the same functionality but in a much cleaner
> way. Is that right?

Hi Marcelo,

With all due respect, I don't really agree with this assessment.

To the casual observer, XFS is just another filesystem. It's used to manage 
files, just like with ext3, Reiser, or JFS. However, XFS provides certain 
features and performance characteristics that may not be found in the other 
filesystems. For this reason, many people prefer XFS over the other 
filesystems, and have pushed for its inclusion in the 2.4 kernel. Of course, 
I'd argue that just as many (if not more) people have very little preference 
as to which filesystem they use. They're happy as long as their data doesn't 
get corrupted if their system crashes.

The situation with Device-Mapper is *very* similar. There are plenty of people 
that are happy using LVM1, and probably don't care much about Device-Mapper 
at this point. But there are also many people who prefer the improved 
features offered by using Device-Mapper. The two new volume management tools, 
LVM2 and EVMS, provide significant improvements over LVM1, such as improved 
metadata formats, more reliable metadata updates, better user interfaces, in 
addition to features that aren't available with LVM1, such as asynchronous 
snapshots. Device-Mapper also provides a modular interface for adding new 
functionality. For example, the EVMS project includes a module for performing 
block-level bad-block-relocation, and another developer has contributed a 
module for block-level encryption based on the crypto API. These new volume 
management tools only work with Device-Mapper, because LVM1 simply doesn't 
have the flexibility necessary to provide these capabilities. Again, this 
situation seems to closely mirror the situation with XFS vs. the existing 
filesystems.

Another compelling reason in my mind is that unlike the variety of filesystems 
that exist both in 2.4 and in 2.6, LVM1 is no longer available in 2.6. Many 
LVM1 users have been eager to try out 2.6 (and I certainly agree with you 
that we need to convince more people to make this switch) but the fact that 
their current tools are useless on 2.6 makes the transition far more painful. 
It would be a huge benefit if these folks were able to first transition to 
the new volume management tools while remaining on a 2.4 kernel. Then after 
they're comfortable with this first switch, they can begin transitioning to a 
2.6 kernel, where the new tools will work seemlessly.

I certainly understand your apprehension about accepting new drivers that 
modify common kernel code. As with XFS, nearly all of the submitted code sits 
in its own directory, and is only enabled if a user decides he needs it. And 
the common changes really are incredibly minimal.

Joe's first patch changes all of 8 lines in the JBD code, which is done to 
prevent JBD and Device-Mapper from stepping on each other's private data. The 
second patch (mempool) only adds new functionality that won't affect any 
existing code. (I'm actually suprised the mempool code hasn't been merged 
yet, since it would be quite useful for any number of drivers and/or 
filesystems besides Device-Mapper. It has certainly come in quite handy in 
2.6.) And the changes in arch/ are simply to support the Device-Mapper 
interface on 64-bit architectures.

I'd be happy to answer any questions or provide any other information that 
would help you with this decision. If you'd like additional review of the 
common code changes, I'll gladly look for volunteers to help with what should 
be a very simple review.

I truly believe that including Device-Mapper will not only benefit users who 
wish to continue on the 2.4 platform, but also those who are looking for an 
easier path to migrate to 2.6.

Thanks very much for your time, Marcelo!

-- 
Kevin Corry
kevcorry@us.ibm.com
http://evms.sourceforge.net/

Re: Device-mapper submission for 2.4

[Paul P Komkoff Jr]

Replying to Kevin Corry:
> prevent JBD and Device-Mapper from stepping on each other's private data. The 
> second patch (mempool) only adds new functionality that won't affect any 
> existing code. (I'm actually suprised the mempool code hasn't been merged 
> yet, since it would be quite useful for any number of drivers and/or 

alan had mempool in -ac for more than a year ;(

-- 
Paul P 'Stingray' Komkoff Jr // http://stingr.net/key <- my pgp key
 This message represents the official view of the voices in my head

Re: Device-mapper submission for 2.4

[Stefan Smietanowski]

Joe Thornber wrote:

> On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
> 
>>I believe 2.6 is the right place for the device mapper. 
> 
> 
> So what's the difference between a new filesystem like XFS and a new
> device driver like dm ?

One thing you're missing is that after all, XFS has existed longer than
dm. Hell, XFS existed before 2.4 did (in a Linux form, I'm not talking
IRIX now).

XFS is also a new filesystem as you said but DM is meant as a
replacement for other functions, not strictly as an additive.

// Stefan

Re: Device-mapper submission for 2.4

[Joe Thornber]

On Tue, Dec 09, 2003 at 03:23:50PM +0100, Stefan Smietanowski wrote:
> Joe Thornber wrote:
> 
> >On Tue, Dec 09, 2003 at 11:15:08AM -0200, Marcelo Tosatti wrote:
> >
> >>I believe 2.6 is the right place for the device mapper. 
> >
> >
> >So what's the difference between a new filesystem like XFS and a new
> >device driver like dm ?
> 
> One thing you're missing is that after all, XFS has existed longer than
> dm. Hell, XFS existed before 2.4 did (in a Linux form, I'm not talking
> IRIX now).

Correct, dm is only 2 and a half years old.

- Joe

Re: Device-mapper submission for 2.4

[William Lee Irwin III]

On Tue, Dec 09, 2003 at 11:58:06AM +0000, Joe Thornber wrote:
> This set of patches is the core of device mapper for 2.4.  I would
> appreciate it if you could merge these into 2.4.24 please.

You have *GOT* to be kidding.


-- wli

Re: Device-mapper submission for 2.4

[Paul Jakma]

On Tue, 9 Dec 2003, William Lee Irwin III wrote:

> You have *GOT* to be kidding.

considering the LVM1 tools interface no longer is supported by DM in
2.6, DM in 2.4 (presumably /with/ LVM1 support (i'd hope)) seems a 
sane way to give 2.4 LVM1 users an easy and reversable upgrade path 
to 2.6.

I know I would love to try out 2.6 on my NFS server, but OTOH, I much
prefer to have access to my data.

> -- wli

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
Neil Armstrong tripped.