device mapper integrated loops - and one more year !

device mapper integrated loops - and one more year !

[David Guyon Martin]

[-- Attachment #1.1: Type: text/plain, Size: 846 bytes --]

Hello,

Do you think dm could have a loop target instead of seting loops with
losetup ?
We allready have dm-crypt for the encryption stage of a "losetup -e
<crypt>", dm replaces losetup when source is a device but I understand that
we still need losetup if the source is a regular file.
Our problem is that we will need to map a great number of files, as if they
were PVs of a VG. Linux kernel can have up to 255 loops but we will reach
this limit.

The idea is to embed to loop itself in a device mapper add-on like dm-crypt,
so we won't use any /dev/loop* .

Do you think this is stupid or is there some possility out there ?
(we would like to have some advice before starting any project in this
direction)

Thank you,
David G.M.

PS:
Anniversary of this post, one year ago !
http://www.redhat.com/archives/dm-devel/2005-October/msg00031.html

[-- Attachment #1.2: Type: text/html, Size: 1011 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: device mapper integrated loops - and one more year !

[Alasdair G Kergon]

On Thu, Oct 05, 2006 at 04:26:54PM +0200, David Guyon Martin wrote:
> Do you think dm could have a loop target instead of seting loops with
> losetup ?

Something like:
  http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/editing/dm-loop.patch
now perhaps?

(And there are patches for dmsetup to become a drop-in losetup replacement.)

Alasdair
-- 
agk@redhat.com

Re: device mapper integrated loops - and one more year !

[David Guyon Martin]

On Thu, 5 Oct 2006 20:40:05 +0100 Alasdair G Kergon wrote:

>Something like:
>  http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/editing/dm-loop.patch
>now perhaps?

>(And there are patches for dmsetup to become a drop-in losetup replacement.)

Yes! This patch add a new file to the kernel tree, dm-loop.c:
"This implements a loopback target for device mapper allowing a regular
file to be treated as a block device."

Great !!! At last :)
It seems this dm-loop is quite new, less than a month...
As I understand it I need kernel 2.6.18-rc7 to use it, right ?

I really thank you a lot for your quick answer and link.

Does anyone has yet a feedback about this dm-loop ?

David G.M.

Re: device mapper integrated loops - and one more year !

[Jonathan E Brassow]

On Oct 6, 2006, at 4:38 AM, David Guyon Martin wrote:

> On Thu, 5 Oct 2006 20:40:05 +0100 Alasdair G Kergon wrote:
>
>> Something like:
>>   
>> http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/ 
>> editing/dm-loop.patch
>> now perhaps?
>
>> (And there are patches for dmsetup to become a drop-in losetup  
>> replacement.)
>
> Yes! This patch add a new file to the kernel tree, dm-loop.c:
> "This implements a loopback target for device mapper allowing a regular
> file to be treated as a block device."
>
> Great !!! At last :)
> It seems this dm-loop is quite new, less than a month...
> As I understand it I need kernel 2.6.18-rc7 to use it, right ?
>
> I really thank you a lot for your quick answer and link.
>
> Does anyone has yet a feedback about this dm-loop ?
>

I know Heinz Mauelshagen has been looking at it and already has some  
performance improvements.  Perhaps he'll post the updated patches soon.

  brassow

Re: device mapper integrated loops - and one more year !

[Heinz Mauelshagen]

On Mon, Oct 09, 2006 at 05:36:40PM -0500, Jonathan E Brassow wrote:
> 
> On Oct 6, 2006, at 4:38 AM, David Guyon Martin wrote:
> 
> >On Thu, 5 Oct 2006 20:40:05 +0100 Alasdair G Kergon wrote:
> >
> >>Something like:
> >>  
> >>http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/ 
> >>editing/dm-loop.patch
> >>now perhaps?
> >
> >>(And there are patches for dmsetup to become a drop-in losetup  
> >>replacement.)
> >
> >Yes! This patch add a new file to the kernel tree, dm-loop.c:
> >"This implements a loopback target for device mapper allowing a regular
> >file to be treated as a block device."
> >
> >Great !!! At last :)
> >It seems this dm-loop is quite new, less than a month...
> >As I understand it I need kernel 2.6.18-rc7 to use it, right ?
> >
> >I really thank you a lot for your quick answer and link.
> >
> >Does anyone has yet a feedback about this dm-loop ?
> >
> 
> I know Heinz Mauelshagen has been looking at it and already has some  
> performance improvements.  Perhaps he'll post the updated patches soon.

Yes, I'm settling the changes, which optimize the linear search
in the file extents table and the initial lookup of the extents
in the file to create the table with the dm-loop author Bryn Reeves
right now.

First benchmarks show promissing results :-)

We hope to post something shortly.

Regards,
Heinz    -- The LVM Guy --

> 
>  brassow
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Red Hat GmbH
Consulting Development Engineer                   Am Sonnenhang 11
Storage Development                               56242 Marienrachdorf
                                                  Germany
Mauelshagen@RedHat.com                            PHONE +49  171 7803392
                                                  FAX   +49 2626 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: device mapper integrated loops - and one more year !

[David Guyon Martin]

2006/10/10, Heinz Mauelshagen <mauelshagen@redhat.com>:
> On Mon, Oct 09, 2006 at 05:36:40PM -0500, Jonathan E Brassow wrote:
> >
> > On Oct 6, 2006, at 4:38 AM, David Guyon Martin wrote:
> >
> > >On Thu, 5 Oct 2006 20:40:05 +0100 Alasdair G Kergon wrote:
> > >
> > >>Something like:
> > >>
> > >>http://www.kernel.org/pub/linux/kernel/people/agk/patches/2.6/
> > >>editing/dm-loop.patch
> > >>now perhaps?
> > >
> > >>(And there are patches for dmsetup to become a drop-in losetup
> > >>replacement.)
> > >
> > >Yes! This patch add a new file to the kernel tree, dm-loop.c:
> > >"This implements a loopback target for device mapper allowing a regular
> > >file to be treated as a block device."
> > >
> > >Great !!! At last :)
> > >It seems this dm-loop is quite new, less than a month...
> > >As I understand it I need kernel 2.6.18-rc7 to use it, right ?
> > >
> > >I really thank you a lot for your quick answer and link.
> > >
> > >Does anyone has yet a feedback about this dm-loop ?
> > >
> >
> > I know Heinz Mauelshagen has been looking at it and already has some
> > performance improvements.  Perhaps he'll post the updated patches soon.
>
> Yes, I'm settling the changes, which optimize the linear search
> in the file extents table and the initial lookup of the extents
> in the file to create the table with the dm-loop author Bryn Reeves
> right now.
>
> First benchmarks show promissing results :-)
>
> We hope to post something shortly.
>
> Regards,
> Heinz    -- The LVM Guy --
>

Thanks a lot ! I stay tuned ( I can hardly wait the results ).
I think many sysadmins underestimate the power of loops, more loops
and greater performances will ease eyes opening.

From dm-loop, comes dm-stego. I didn't found anything similar in the dev tree.
Does anyone know if someone is working on a dm-stego filter ?

David G.M.

PS: Heinz, since you've written dm-delay, you may have a clue to
http://www.redhat.com/archives/dm-devel/2006-October/msg00048.html

Re: device mapper integrated loops - and one more year !

[devzero]

Hello !

after desparately seeking for a solution to have more than 256 loop devices (for a huge cd-rom server), i have recently come across dm-loop, which looks very promising (to replace loop.c)

can i have more than 256 devices which this?

our cd-rom server is going to have more than 256 iso-images soon and i need to upgrade to a more recent OS, anyway, so this won`t be a problem that dm-loop is so brand new.

but - any timeline for dm-loop mainline inclusion ?

should i wait for 2.6.19 or 2.6.20 ?

is it already useable with less recent kernels, i.e. can i download dm-loop patch and recent dmsetup tool and use with older kernel ?

i really would like to give it a try - no problem if there are still bugs inside, because i`d happily test and report if there are bugs left.

regards
Roland K.
systems engineer


_______________________________________________________________________
Viren-Scan für Ihren PC! Jetzt für jeden. Sofort, online und kostenlos.
Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=022222

Re: Re: device mapper integrated loops - and one more year !

[devzero]

Hello Bryn, 


>I'll see if we can get a revised version of the current patch for wider
>testing in the next couple of weeks - let me know if you would be
>interested in trying this out.

not sure if i sent a reply to this or have missed this -  but, yes - i`m still interested very much in using dm-loop !

is there anything new with this?
do i have to expect issues with files >2gb ? (i need to loopback mount dvd iso-images)

is there a download url for the latest dm-loop patch so i can do some tests and report feedback ?

i couldn`t find it at http://sources.redhat.com/cgi-bin/cvsweb.cgi/device-mapper/?cvsroot=dm  and i`m not sure if i should take that one from http://kernel.org/pub/linux/kernel/people/agk/patches/2.6/

TIA
roland



> Hello !
> 
> after desparately seeking for a solution to have more than 256 loop
> devices (for a huge cd-rom server), i have recently come across
> dm-loop, which looks very promising (to replace loop.c)
> 
> can i have more than 256 devices which this?

Hi Roland,

Yes, dm-loop will support as many loop devices as device-mapper can allow.

> our cd-rom server is going to have more than 256 iso-images soon and
> i need to upgrade to a more recent OS, anyway, so this won`t be a 
> problem that dm-loop is so brand new.
> 
> but - any timeline for dm-loop mainline inclusion ?
> 
> should i wait for 2.6.19 or 2.6.20 ?

There isn't a fixed timeline. We are currently working to improve the
lookup code - the prototype patch uses a linear search which doesn't
scale well for large/fragmented image files. I have some tests running
at the moment and we hope to have something soon - the results so far
are good, it's just a case of selecting between a couple of different
methods.

The changes dm-loop requires in device-mapper core are pretty minor and
have already been merged in 2.6.19, so it's just the dm-loop patch
itself remaining.

> is it already useable with less recent kernels, i.e. can i download
> dm-loop patch and recent dmsetup tool and use with older kernel ?

This should work fine - there are a couple of known issues with the
existing patch, but unless you are using large images (>2G), or have a
severely fragmented filesystem you shouldn't run into these.

I'll see if we can get a revised version of the current patch for wider
testing in the next couple of weeks - let me know if you would be
interested in trying this out.

Thanks,

Bryn.
__________________________________________________________________________
Erweitern Sie FreeMail zu einem noch leistungsstärkeren E-Mail-Postfach!		
Mehr Infos unter http://freemail.web.de/home/landingpad/?mc=021131

Re: device mapper integrated loops - and one more year !

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

devzero@web.de wrote:
> Hello Bryn, 
> 
> 
>> I'll see if we can get a revised version of the current patch for wider
>> testing in the next couple of weeks - let me know if you would be
>> interested in trying this out.
> 
> not sure if i sent a reply to this or have missed this -  but, yes - i`m still interested very much in using dm-loop !

Hi Roland,

No, you've not missed anything!

I've got an updated version of the patch with some enhancements Heinz
(lvmguy) has been working on to improve the driver's lookup performance.
This is still being worked on at the moment.

> is there anything new with this?
> do i have to expect issues with files >2gb ? (i need to loopback mount dvd iso-images)
> 
> is there a download url for the latest dm-loop patch so i can do some tests and report feedback ?

This patch is a little out of date now - I'll get a later version
(bugfixes mainly - the new lookup code is still being tested) made
available here.

Thanks,

Bryn.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFFtIcl6YSQoMYUY94RAsRqAJ4iKrXKW5IYe0S9NlUofBp1QpzoiQCgxxba
gfoC4jE1+anDhZwz/+MFtUs=
=IBSf
-----END PGP SIGNATURE-----