Re: "rbd rm image" slow with big images ?

[Stefan Priebe <s.priebe@profihost.ag>]

One note:
he has written:
"then just delete it, without having writed nothing in image "


Am 31.05.2012 20:15, schrieb Wido den Hollander:
> Hi,
>
> On 05/31/2012 09:12 AM, Alexandre DERUMIER wrote:
>> Hi,
>>
>> I trying to delete some rbd images with rbd rm,
>> and it seem to be "slow" with big images.
>>
>>
>>
>> I'm testing it with just create a new image (1TB):
>>
>> # time rbd -p pool1 create --size 1000000 image2
>>
>> real 0m0.031s
>> user 0m0.015s
>> sys 0m0.010s
>>
>>
>> then just delete it, without having writed nothing in image
>>
>>
>> # time rbd -p pool1 rm image2
>> Removing image: 100% complete...done.
>>
>> real 1m45.558s
>> user 0m14.683s
>> sys 0m17.363s
>>
>>
>>
>> same test with 100GB
>>
>> # time rbd -p pool1 create --size 100000 image2
>>
>> real 0m0.032s
>> user 0m0.016s
>> sys 0m0.007s
>>
>> # time rbd -p pool1 rm image2
>> Removing image: 100% complete...done.
>>
>> real 0m10.499s
>> user 0m1.488s
>> sys 0m1.720s
>>
>>
>> I'm using journal in tmpfs, 3 servers, 15 osds with 1disk 15K (xfs)
>> network bandwith,diskio,cpu are low.
>>
>> Is it the normal behaviour ? Maybe some xfs tuning could help ?
>
> It's in the nature of RBD.
>
> A RBD image consists of multiple 4MB (default) RADOS objects.
>
> Let's say you have a disk of 40GB, that will contain 10.000 4MB RADOS
> objects, you can find those objects by doing: rados -p rbd ls
>
> Now, when you create a new image only the header is writting, but no
> object is written.
>
> When you start writing to a RBD image you will be writing to one of the
> 4MB objects. When it doesn't exist it will be created.
>
> So when you install your VM it will create objects, but not all of them.
>
> RBD knows which RADOS objects to access by three parameters:
>
> * Image name
> * Image size
> * Stripe size (4MB)
>
> So when your VM access for byte Y until Z on the disk, RBD knows which
> object to access by calculating this.
>
> Now, when you start removing the image there is no way of knowing which
> object exists and which doesn't, so RBD will try to remove all objects.
>
> In the case of a fresh image this results in 10.000 RADOS remove
> operations for non-existent objects and that is slow.
>
> Wido
>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html