Re: bug in dm-loop? - was:Re: Re: device mapper

Re: bug in dm-loop? - was:Re: Re: device mapper

[devzero]

hi!

>With ~3000 of them active concurrently on a machine with 256M of RAM,
>I'm not that surprised that you hit this limit!

ok, not a problem - my cd-rom server has some more ram and i will probably never reach that big number of CD`s in the near future.

>Again, to keep things simple in the prototype, we grab the biggest
>possible table size when we create a new device and then re-allocate it
>to the correct size when we finish building the map - that's what the
>"Finalized extent map of 56 bytes, 2 entries." messages refer to.

may this be a reason why dmlosetup seems to be much slower than traditional losetup?
i did some tests with some larger images (100m, 1gig and 2,5gig) and dmlosetting them up took quite a while.....


>If debugging was enabled when the module was built, you should also see
>messages like:
>loop: [DEBUG] setup_loop_extents:Allocated initial extent map of 57352
>bytes, 2048 entries.
>
>It's these initial large allocations that are causing the errors you see.

ahh, good to know !

>The code's pretty dumb in this area at the moment - improving memory
>allocation and the time spent looking up entries in the table are our
>current goals for dm-loop.

what you tell is perfectly in sync with what i observed. fine that you want to adress this! :)


>the code will fail if it needs >2048 table entries to describe the file
>- - depending on the filesystem used and how fragmented it is this will
>limit you to files of a few gigabytes. This is another arbitrary limit
>in this version that will go away in later dm-loop releases.

indeed, it looks like there is a real hard 2gb limit - i wasn`t able to dmlosetup the 2,5gig files i created

anyway - good things will need some time.
so i`m just sitting here , waiting for the next versions(s) :)

regards
roland

_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=100071&distributionid=000000000066

Re: Re: bug in dm-loop? - was:Re: Re: device mapper

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

devzero@web.de wrote:
>> Again, to keep things simple in the prototype, we grab the biggest
>> possible table size when we create a new device and then re-allocate it
>> to the correct size when we finish building the map - that's what the
>> "Finalized extent map of 56 bytes, 2 entries." messages refer to.
> 
> may this be a reason why dmlosetup seems to be much slower than traditional losetup?
> i did some tests with some larger images (100m, 1gig and 2,5gig) and dmlosetting them up took quite a while.....
> 

Unfortunately, this is a more fundamental aspect of dm-loop. The way the
code achieves better performance than the regular loop driver is by
doing much of the work upfront when a device is first created. This does
mean that initial setup for a loop device takes longer with dm-loop, and
that time is again proportional to the size/fragmentation of the file.

For many filesystems, you can make 'guesses' that will speed things up,
but this comes at a heavy price. Guess wrong and it leads to data
corruption, so this isn't something we'd feel happy about implementing.

>> the code will fail if it needs >2048 table entries to describe the file
>> - - depending on the filesystem used and how fragmented it is this will
>> limit you to files of a few gigabytes. This is another arbitrary limit
>> in this version that will go away in later dm-loop releases.
> 
> indeed, it looks like there is a real hard 2gb limit - i wasn`t able to dmlosetup the 2,5gig files i created

It's not so much a  hard 2gb limit, but on a cleanly created ext3 or
similar fs, it tends to be around that number. Different filesystem
types or fragmentation levels change that number but they all stem from
the same limitation.

Kind regards,

Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFFt6ei6YSQoMYUY94RAu6VAJ0fOkdxJFs2bYZQspfDyJcFxxphLQCgiQN0
6VUI4+o6h/NtJ+bvAbnMjR0=
=aIWR
-----END PGP SIGNATURE-----