Re: [linux-lvm] Why is the performance of my lvmthin snapshot so poor

From: Demi Marie Obenour <demi@invisiblethingslab.com>
To: LVM general discussion and development <linux-lvm@redhat.com>,
	Gionatan Danti <g.danti@assyoma.it>
Subject: Re: [linux-lvm] Why is the performance of my lvmthin snapshot so poor
Date: Thu, 16 Jun 2022 03:53:23 -0400	[thread overview]
Message-ID: <Yqrhmwwbp5KfiEWb@itl-email> (raw)
In-Reply-To: <d354dfb0-c37e-8a85-6cf6-d4b3a7e9257d@bytedance.com>

[-- Attachment #1.1: Type: text/plain, Size: 1935 bytes --]

On Wed, Jun 15, 2022 at 03:42:17PM +0800, Zhiyong Ye wrote:
> 
> 
> 在 6/14/22 10:54 PM, Gionatan Danti 写道:
> > Il 2022-06-14 15:29 Zhiyong Ye ha scritto:
> > > The reason for this may be that when the volume creates a snapshot,
> > > each write to an existing block will cause a COW (Copy-on-write), and
> > > the COW is a copy of the entire data block in chunksize, for example,
> > > when the chunksize is 64k, even if only 4k of data is written, the
> > > entire 64k data block will be copied. I'm not sure if I understand
> > > this correctly.
> > 
> > Yes, in your case, the added copies are lowering total available IOPs.
> > But note how the decrease is sub-linear (from 64K to 1M you have a 16x
> > increase in chunk size but "only" a 10x hit in IOPs): this is due to the
> > lowered metadata overhead.
> 
> It seems that the consumption of COW copies when sending 4k requests is much
> greater than the loss from metadata.
> 
> > A last try: if you can, please regenerate your thin volume with 64K
> > chunks and set fio to execute 64K requests. Lets see if LVM is at least
> > smart enough to avoid coping a to-be-completely-overwritten chunks.
> 
> I regenerated the thin volume with the chunksize of 64K and the random write
> performance data tested with fio 64k requests is as follows:
> case                    iops
> thin lv                 9381
> snapshotted thin lv     8307

That seems reasonable.  My conclusion is that dm-thin (which is what LVM
uses) is not a good fit for workloads with a lot of small random writes
and frequent snapshots, due to the 64k minimum chunk size.  This also
explains why dm-thin does not allow smaller blocks: not only would it
only support very small thin pools, it would also have massive metadata
write overhead.  Hopefully dm-thin v2 will improve the situation.

-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 202 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/