Thinly-Provisioned Logical Volumes

Thinly-Provisioned Logical Volumes

[lilofile]

when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of  thin volume. how can i quickly understand dm-bufio.c?

Re: Thinly-Provisioned Logical Volumes

[Zdenek Kabelac]

Dne 20.9.2014 v 16:19 lilofile napsal(a):
> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of  thin volume. how can i quickly understand dm-bufio.c?
>


In general 2 major factors -

1. - size of chunk/(provisioned block)

the bigger the block is - the better usually filesystem performs - since the 
all the aggregation logic and optimization of fs works.

on the other - the bigger the block is - the less efficient snapshot you get
(since much bigger chunk needs to be copied for i.e. single byte write)


2. - zeroing on provisionsing.

when the block provisioned for the first time - it's (by default config) fully 
zeroes - so you don't get random disk content from unwritten areas.

If you don't care about this - you could safely disable zeroing - number of 
filesystem usually already manages unwritten space well so if you use block 
device for ext4 so shouldn't normally be able to read unwritten areas.


So please try i.e.  256KB chunk size with non-zeroing (-c 256 -Zn) when 
creating thin-pool  -  do you still fell you get poor performance ?

Zdenek


NOTE: you could change zeroing with existing pool, but not your thin-pool 
chunk size for now (though future version might be less restrictive here).

Re: Thinly-Provisioned Logical Volumes

[Mike Snitzer]

On Sat, Sep 20 2014 at 10:19am -0400,
lilofile <lilofile@aliyun.com> wrote:

> when I test Thinly-Provisioned Logical Volumes(use device thinprovison
> target),I found the random write performance is very low. What factors
> will influence the random write performance of  thin volume.

It is worth reading this blog post from Joe:
http://device-mapper.org/blog/2014/01/21/random-io-and-copy-on-write-schemes/

> how can i quickly understand dm-bufio.c?

What is it you'd like to know?  dm-bufio.c isn't the smoking gun for
random write performance being slow though.  It is more to do with the
natural impact random writes have on thinp (as discussed in the above
article).

reply： Thinly-Provisioned Logical Volumes

[lilofile]

when  I creating thin-pool with non-zeroing（-Zn),the ramdom performance can be greatly improved. what  is the difference in code  with non-zeroing and zeroing?


------------------------------------------------------------------
发件人：Zdenek Kabelac <zkabelac@redhat.com>
发送时间：2014年9月22日(星期一) 15:02
收件人：lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
主　题：Re: [dm-devel] Thinly-Provisioned Logical Volumes

Dne 20.9.2014 v 16:19 lilofile napsal(a):
> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of  thin volume. how can i quickly understand dm-bufio.c?
>


In general 2 major factors -

1. - size of chunk/(provisioned block)

the bigger the block is - the better usually filesystem performs - since the 
all the aggregation logic and optimization of fs works.

on the other - the bigger the block is - the less efficient snapshot you get
(since much bigger chunk needs to be copied for i.e. single byte write)


2. - zeroing on provisionsing.

when the block provisioned for the first time - it's (by default config) fully 
zeroes - so you don't get random disk content from unwritten areas.

If you don't care about this - you could safely disable zeroing - number of 
filesystem usually already manages unwritten space well so if you use block 
device for ext4 so shouldn't normally be able to read unwritten areas.


So please try i.e.  256KB chunk size with non-zeroing (-c 256 -Zn) when 
creating thin-pool  -  do you still fell you get poor performance ?

Zdenek


NOTE: you could change zeroing with existing pool, but not your thin-pool 
chunk size for now (though future version might be less restrictive here).


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: reply： Thinly-Provisioned Logical Volumes

[Zdenek Kabelac]

Dne 23.9.2014 v 15:51 lilofile napsal(a):
> when  I creating thin-pool with non-zeroing（-Zn),the ramdom performance can be greatly improved. what  is the difference in code  with non-zeroing and zeroing?
>
>

*Zeroing is set*

When you write to an unprovisioned chunk - certain amount of bytes in this 
chunk have known content (from your write BIO) and everything else gets zeroes
(that's zeroing on provisioning)

So when you read your thin device - you always get same content.
(since unprovisioned areas returns zero - and provisioned returns
what you have written - or zeros from unwritten part)
Of course zeroing i.e. 1MB chunks is quite slow compared with write of few 
single bytes on chunk provisioning.

*Zeroing is unset*

Unprovisioned data are still 'zeroes'

However when you first write to 'unprovisioned' chunk - then your data
are again in the place you wrote then - BUT remaining content is - what has 
been found on your disk in the provisioned chunk - so now when you read from 
'unwritten' areas in provisioned chunk - you read some random garbage left on 
your disk.
This may have i.e. security impact.


So you need to pick what suits your needs best.

For filesystem that keep map of written space - go with -Zn
for improved security stay with -Zy and be prepared to pay some price with 
initial provisioning - since repeated write has then regular performance.

Zdenek





> ------------------------------------------------------------------
> 发件人：Zdenek Kabelac <zkabelac@redhat.com>
> 发送时间：2014年9月22日(星期一) 15:02
> 收件人：lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
> 主　题：Re: [dm-devel] Thinly-Provisioned Logical Volumes
>
> Dne 20.9.2014 v 16:19 lilofile napsal(a):
>> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of  thin volume. how can i quickly understand dm-bufio.c?
>>
>
>
> In general 2 major factors -
>
> 1. - size of chunk/(provisioned block)
>
> the bigger the block is - the better usually filesystem performs - since the
> all the aggregation logic and optimization of fs works.
>
> on the other - the bigger the block is - the less efficient snapshot you get
> (since much bigger chunk needs to be copied for i.e. single byte write)
>
>
> 2. - zeroing on provisionsing.
>
> when the block provisioned for the first time - it's (by default config) fully
> zeroes - so you don't get random disk content from unwritten areas.
>
> If you don't care about this - you could safely disable zeroing - number of
> filesystem usually already manages unwritten space well so if you use block
> device for ext4 so shouldn't normally be able to read unwritten areas.
>
>
> So please try i.e.  256KB chunk size with non-zeroing (-c 256 -Zn) when
> creating thin-pool  -  do you still fell you get poor performance ?
>
> Zdenek
>
>
> NOTE: you could change zeroing with existing pool, but not your thin-pool
> chunk size for now (though future version might be less restrictive here).
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

reply： Thinly-Provisioned Logical Volumes

[lilofile]

When I test thinly-provisioned logical volumes,I use dd if=/dev/zero of=/dev/mapper/mythinvolume count=1 bs=4k
I only write 4k data in mythinvolume.but In thin_map functions, I found there are really  4K data written,but I found more read IO,probably dozens of read io. who issue these IO? 
 


------------------------------------------------------------------
发件人：Zdenek Kabelac <zkabelac@redhat.com>
发送时间：2014年9月23日(星期二) 22:07
收件人：lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
主　题：Re: [dm-devel] reply： Thinly-Provisioned Logical Volumes

Dne 23.9.2014 v 15:51 lilofile napsal(a):
> when  I creating thin-pool with non-zeroing（-Zn),the ramdom performance can be greatly improved. what  is the difference in code  with non-zeroing and zeroing?
>
>

*Zeroing is set*

When you write to an unprovisioned chunk - certain amount of bytes in this 
chunk have known content (from your write BIO) and everything else gets zeroes
(that's zeroing on provisioning)

So when you read your thin device - you always get same content.
(since unprovisioned areas returns zero - and provisioned returns
what you have written - or zeros from unwritten part)
Of course zeroing i.e. 1MB chunks is quite slow compared with write of few 
single bytes on chunk provisioning.

*Zeroing is unset*

Unprovisioned data are still 'zeroes'

However when you first write to 'unprovisioned' chunk - then your data
are again in the place you wrote then - BUT remaining content is - what has 
been found on your disk in the provisioned chunk - so now when you read from 
'unwritten' areas in provisioned chunk - you read some random garbage left on 
your disk.
This may have i.e. security impact.


So you need to pick what suits your needs best.

For filesystem that keep map of written space - go with -Zn
for improved security stay with -Zy and be prepared to pay some price with 
initial provisioning - since repeated write has then regular performance.

Zdenek





> ------------------------------------------------------------------
> 发件人：Zdenek Kabelac <zkabelac@redhat.com>
> 发送时间：2014年9月22日(星期一) 15:02
> 收件人：lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
> 主　题：Re: [dm-devel] Thinly-Provisioned Logical Volumes
>
> Dne 20.9.2014 v 16:19 lilofile napsal(a):
>> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of  thin volume. how can i quickly understand dm-bufio.c?
>>
>
>
> In general 2 major factors -
>
> 1. - size of chunk/(provisioned block)
>
> the bigger the block is - the better usually filesystem performs - since the
> all the aggregation logic and optimization of fs works.
>
> on the other - the bigger the block is - the less efficient snapshot you get
> (since much bigger chunk needs to be copied for i.e. single byte write)
>
>
> 2. - zeroing on provisionsing.
>
> when the block provisioned for the first time - it's (by default config) fully
> zeroes - so you don't get random disk content from unwritten areas.
>
> If you don't care about this - you could safely disable zeroing - number of
> filesystem usually already manages unwritten space well so if you use block
> device for ext4 so shouldn't normally be able to read unwritten areas.
>
>
> So please try i.e.  256KB chunk size with non-zeroing (-c 256 -Zn) when
> creating thin-pool  -  do you still fell you get poor performance ?
>
> Zdenek
>
>
> NOTE: you could change zeroing with existing pool, but not your thin-pool
> chunk size for now (though future version might be less restrictive here).
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: reply： Thinly-Provisioned Logical Volumes

[Zdenek Kabelac]

Dne 24.9.2014 v 04:11 lilofile napsal(a):
> When I test thinly-provisioned logical volumes,I use dd if=/dev/zero of=/dev/mapper/mythinvolume count=1 bs=4k
> I only write 4k data in mythinvolume.but In thin_map functions, I found there are really  4K data written,but I found more read IO,probably dozens of read io. who issue these IO?
>

Hi

I'd have guessed those I/O are likely metadata update related - at most every 
second metadata are synchronized to your metadata storage volume.
(Thin pool consists of _tdata data volume where provisioned chunks are located 
and _tmeta metadata volume).

And btw you could use tools like thin_dump  to explore how is your pool used,
which chunks belong to which volume, which are share and so on....

Regards

Zdenek

Re: reply： Thinly-Provisioned Logical Volumes

[Joe Thornber]

On Wed, Sep 24, 2014 at 10:11:00AM +0800, lilofile wrote:
> When I test thinly-provisioned logical volumes,I use dd if=/dev/zero of=/dev/mapper/mythinvolume count=1 bs=4k
> I only write 4k data in mythinvolume.but In thin_map functions, I found there are really  4K data written,but I found more read IO,probably dozens of read io. who issue these IO? 

Assuming the IO is going to the data device, and not the metadata
device, then the likely candidates would be read-ahead or udev.

- Joe