[librbd] Add interface of get the snapshot size?

[librbd] Add interface of get the snapshot size?

[Haomai Wang]

Hi all,

As we know, snapshot is a lightweight resource in librbd and we
doesn't have any statistic informations about it. But it causes some
problems to the cloud management.

We can't measure the size of snapshot, different snapshot will occur
different space. So we don't have way to estimate the resource usage
of user.

Maybe we can have a counter to record space usage when volumn created.
When creating snapshot, the counter is freeze and store as the size of
snapshot. New counter will assign to zero for the volume.

Any feedback is appreciate!

-- 

Best Regards,

Wheat

Re: [librbd] Add interface of get the snapshot size?

[Wido den Hollander]

On 03/24/2014 02:30 PM, Haomai Wang wrote:
> Hi all,
>
> As we know, snapshot is a lightweight resource in librbd and we
> doesn't have any statistic informations about it. But it causes some
> problems to the cloud management.
>
> We can't measure the size of snapshot, different snapshot will occur
> different space. So we don't have way to estimate the resource usage
> of user.
>
> Maybe we can have a counter to record space usage when volumn created.

What do you mean with space usage? Cluster wide or pool usage?

> When creating snapshot, the counter is freeze and store as the size of
> snapshot. New counter will assign to zero for the volume.
>
> Any feedback is appreciate!
>


-- 
Wido den Hollander
42on B.V.

Phone: +31 (0)20 700 9902
Skype: contact42on

Re: [librbd] Add interface of get the snapshot size?

[Andrey Korolyov]

On 03/24/2014 05:30 PM, Haomai Wang wrote:
> Hi all,
> 
> As we know, snapshot is a lightweight resource in librbd and we
> doesn't have any statistic informations about it. But it causes some
> problems to the cloud management.
> 
> We can't measure the size of snapshot, different snapshot will occur
> different space. So we don't have way to estimate the resource usage
> of user.
> 
> Maybe we can have a counter to record space usage when volumn created.
> When creating snapshot, the counter is freeze and store as the size of
> snapshot. New counter will assign to zero for the volume.
> 
> Any feedback is appreciate!
> 

I believe that there is a rough estimation over 'rados df'. Per-image
statistics would be awesome, though precise stats will be neither rough
too(# of rbd object clones per volume) or introduce new counter
mechanism. Dealing with discard for the filestore, it looks even more
difficult to calculate right estimation, as with XFS preallocation feature.

Re: [librbd] Add interface of get the snapshot size?

[Josh Durgin]

On 03/24/2014 06:50 AM, Andrey Korolyov wrote:
> On 03/24/2014 05:30 PM, Haomai Wang wrote:
>> Hi all,
>>
>> As we know, snapshot is a lightweight resource in librbd and we
>> doesn't have any statistic informations about it. But it causes some
>> problems to the cloud management.
>>
>> We can't measure the size of snapshot, different snapshot will occur
>> different space. So we don't have way to estimate the resource usage
>> of user.
>>
>> Maybe we can have a counter to record space usage when volumn created.
>> When creating snapshot, the counter is freeze and store as the size of
>> snapshot. New counter will assign to zero for the volume.
>>
>> Any feedback is appreciate!
>>
>
> I believe that there is a rough estimation over 'rados df'. Per-image
> statistics would be awesome, though precise stats will be neither rough
> too(# of rbd object clones per volume) or introduce new counter
> mechanism. Dealing with discard for the filestore, it looks even more
> difficult to calculate right estimation, as with XFS preallocation feature.

diff_iterate() will let you see what extents exist in the image at a
given snapshot or between snapshots. It's not perfect, but it'll be more
accurate than other methods.

A better long term solution might be to leverage something like a
bitmap of object existence. This could be marked dirty (i.e.
unreliable) when opening an image, maintained in memory, and saved with
a snapshot and when closing an image as clean.

With xfs preallocation hints, the usage will be pretty close to object
size * num objects.

Josh