dd between thin provisioned devices

dd between thin provisioned devices

[devicemapper niko]

[-- Attachment #1.1: Type: text/plain, Size: 400 bytes --]

Hi,everyone

if we dd between thin provisioned devices,
can we just get allocated blocks copied ?
Will the output thin device get fully provisioned?

I glanced through the dm-thin code,
it turns out unmapped blocks return zeroed,am i right?

Is there a trick to tell upper applications that all u got are zeroes.
or should they scan zeroed blocks themselves to trigger WRITE_SAME to
reduce I/O load?

[-- Attachment #1.2: Type: text/html, Size: 540 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: dd between thin provisioned devices

[Zdenek Kabelac]

Dne 27.1.2013 11:01, devicemapper niko napsal(a):
> Hi,everyone
> if we dd between thin provisioned devices,
> can we just get allocated blocks copied ?

Nope - devices mapper target operates on block level - not on the filesystem
level - so it has no knowledge about file layout.

What I could eventually imagine is some kind of block compactification (just 
like kernel does for hugepages) - but it's currently not being developed....


> Will the output thin device get fully provisioned?
> I glanced through the dm-thin code,
> it turns out unmapped blocks return zeroed,am i right?

depends on settings - you could have either zeroed block or block
with content left on the disk - i.e. running thin with 1MB block/chunk size
means - before the first use whole 1MB must be zeroed - so it has its price.

> Is there a trick to tell upper applications that all u got are zeroes.
> or should they scan zeroed blocks themselves to trigger WRITE_SAME to reduce
> I/O load?


Documentation/device-mapper/thin-provisioning.txt
see flag: skip_block_zeroing

Eventually for lvm2   lvcreate --zero y|n for thin pool creation.

Zdenek

Re: dd between thin provisioned devices

[devicemapper niko]

[-- Attachment #1.1: Type: text/plain, Size: 1568 bytes --]

thanks for the reply. What i really wanna to ask is thin clone/mirror
supported or not.

I noticed the time later SCSI commad GET LBA STATUS can fullfil this
requiremnt .

Before block copy, send GET LBA STATUS to check the corresponding blocks
mapped or not.
if not, just skip copy.


On Mon, Jan 28, 2013 at 5:06 PM, Zdenek Kabelac <zdenek.kabelac@gmail.com>wrote:

> Dne 27.1.2013 11:01, devicemapper niko napsal(a):
>
>  Hi,everyone
>> if we dd between thin provisioned devices,
>> can we just get allocated blocks copied ?
>>
>
> Nope - devices mapper target operates on block level - not on the
> filesystem
> level - so it has no knowledge about file layout.
>
> What I could eventually imagine is some kind of block compactification
> (just like kernel does for hugepages) - but it's currently not being
> developed....
>
>
>
>  Will the output thin device get fully provisioned?
>> I glanced through the dm-thin code,
>> it turns out unmapped blocks return zeroed,am i right?
>>
>
> depends on settings - you could have either zeroed block or block
> with content left on the disk - i.e. running thin with 1MB block/chunk size
> means - before the first use whole 1MB must be zeroed - so it has its
> price.
>
>
>  Is there a trick to tell upper applications that all u got are zeroes.
>> or should they scan zeroed blocks themselves to trigger WRITE_SAME to
>> reduce
>> I/O load?
>>
>
>
> Documentation/device-mapper/**thin-provisioning.txt
> see flag: skip_block_zeroing
>
> Eventually for lvm2   lvcreate --zero y|n for thin pool creation.
>
> Zdenek
>
>

[-- Attachment #1.2: Type: text/html, Size: 2726 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: dd between thin provisioned devices

[thornber]

On Sun, Jan 27, 2013 at 06:01:25PM +0800, devicemapper niko wrote:
> Hi,everyone
> 
> if we dd between thin provisioned devices,
> can we just get allocated blocks copied ?
> Will the output thin device get fully provisioned?

I think what you want to do is take a thin snapshot.  This is a clone
of a thin device's mapping tree.  Initially all blocks will be
identical, but sharing will be broken as you write to either the
original thin device or the snapshot.

> I glanced through the dm-thin code,
> it turns out unmapped blocks return zeroed,am i right?

A read to an unprovisioned block will always return zeroes.  If you
write to an unprovisioned block any data that is not covered by the
write will be zeroed iff you have set the appropriate flag.

> Is there a trick to tell upper applications that all u got are zeroes.
> or should they scan zeroed blocks themselves to trigger WRITE_SAME to
> reduce I/O load?

Not sure what you're trying to do here.  You can use the thin tools to
grab a snapshot of the _metadata_ on a live system.  From this you can
work out what was shared at the time.  Of course there'll always be a
potential race with this kind of approach.  Why has your application
forgotten where it's written?

- Joe