any chance for xfs shrinking?

any chance for xfs shrinking?

[Stefan Priebe - Profihost AG]

Hi,

while cloud / vms usage become more and more popular and qemu now also
offers memory hot add and unplug, cpu hot add and unplug, we still
suffer from a missing xfs shrink.

I would like to continue to use XFS as it is a rock solid base since
around 10 years for us.

But one missing piece in variable ressource usage for us is disk
shrinking. Is there any chance to get an xfs online shrinking?

Thanks!

Greets,
Stefan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: any chance for xfs shrinking?

[Dewangga Bachrul Alam]

Hello!

It's been discussed couple months ago,
Please check it here http://oss.sgi.com/archives/xfs/2015-02/msg00252.html

On 05/12/2015 07:01 PM, Stefan Priebe - Profihost AG wrote:
> Hi,
> 
> while cloud / vms usage become more and more popular and qemu now also
> offers memory hot add and unplug, cpu hot add and unplug, we still
> suffer from a missing xfs shrink.
> 
> I would like to continue to use XFS as it is a rock solid base since
> around 10 years for us.
> 
> But one missing piece in variable ressource usage for us is disk
> shrinking. Is there any chance to get an xfs online shrinking?
> 
> Thanks!
> 
> Greets,
> Stefan
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: any chance for xfs shrinking?

[Eric Sandeen]

On 5/12/15 7:01 AM, Stefan Priebe - Profihost AG wrote:
> Hi,
> 
> while cloud / vms usage become more and more popular and qemu now also
> offers memory hot add and unplug, cpu hot add and unplug, we still
> suffer from a missing xfs shrink.

Filesystem shrink is a very different scenario than "cloud / vms" needs
for hot memory & cpu plugging, IMHO.

> I would like to continue to use XFS as it is a rock solid base since
> around 10 years for us.
> 
> But one missing piece in variable ressource usage for us is disk
> shrinking. Is there any chance to get an xfs online shrinking?

Under what circumstance does your workflow require a filesystem shrink?

Honest question, I'm not challenging you, but I would like to understand
what it is about shrink that is so critical it may cause you to stop using
XFS.

One thing about shrink - while i.e. ext4 supports it, the end result of
a shrunk filesystem is a scrambled filesystem.  All those heuristics that
made the filesystem reasonably performant as it aged get thrown out the 
window as the fs scavenges for space into which to shrink the filesystem...

That said, another option is to use the dm-thinp target, and allocate /
de-allocate storage resources to the underlying block device as needed.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: any chance for xfs shrinking?

[Stefan Priebe - Profihost AG]

Am 12.05.2015 um 14:35 schrieb Eric Sandeen:
> On 5/12/15 7:01 AM, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> while cloud / vms usage become more and more popular and qemu now also
>> offers memory hot add and unplug, cpu hot add and unplug, we still
>> suffer from a missing xfs shrink.
> 
> Filesystem shrink is a very different scenario than "cloud / vms" needs
> for hot memory & cpu plugging, IMHO.
> 
>> I would like to continue to use XFS as it is a rock solid base since
>> around 10 years for us.
>>
>> But one missing piece in variable ressource usage for us is disk
>> shrinking. Is there any chance to get an xfs online shrinking?
> 
> Under what circumstance does your workflow require a filesystem shrink?

May be special but there are a lot of customers out there who do not
manager their ressources. So they've partners and partners of partners.
It happens quite often that the disk runs out of space and the customer
does not know what he can do as third parties control the waste of
space. So we are in the situation where we need to extent the partition
so the customer can continue his business. Later when the partner has
solved the issue (real world shows mostly 2 days - 3 weeks) the customer
wants to shrink again as he does not want to pay the space.

> Honest question, I'm not challenging you, but I would like to understand
> what it is about shrink that is so critical it may cause you to stop using
> XFS.
> 
> One thing about shrink - while i.e. ext4 supports it, the end result of
> a shrunk filesystem is a scrambled filesystem.  All those heuristics that
> made the filesystem reasonably performant as it aged get thrown out the 
> window as the fs scavenges for space into which to shrink the filesystem...
> 
> That said, another option is to use the dm-thinp target, and allocate /
> de-allocate storage resources to the underlying block device as needed.

We do so while using ceph and trim but the customer pays what he can
theoretically use and not what he uses in real. Ceph also does not
provide a way to show us the real usage of a rbd disk.

Greets,
Stefan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: any chance for xfs shrinking?

[Andrey Korolyov]

On Tue, May 12, 2015 at 3:41 PM, Stefan Priebe - Profihost AG
<s.priebe@profihost.ag> wrote:
>
> Am 12.05.2015 um 14:35 schrieb Eric Sandeen:
>> On 5/12/15 7:01 AM, Stefan Priebe - Profihost AG wrote:
>>> Hi,
>>>
>>> while cloud / vms usage become more and more popular and qemu now also
>>> offers memory hot add and unplug, cpu hot add and unplug, we still
>>> suffer from a missing xfs shrink.
>>
>> Filesystem shrink is a very different scenario than "cloud / vms" needs
>> for hot memory & cpu plugging, IMHO.
>>
>>> I would like to continue to use XFS as it is a rock solid base since
>>> around 10 years for us.
>>>
>>> But one missing piece in variable ressource usage for us is disk
>>> shrinking. Is there any chance to get an xfs online shrinking?
>>
>> Under what circumstance does your workflow require a filesystem shrink?
>
> May be special but there are a lot of customers out there who do not
> manager their ressources. So they've partners and partners of partners.
> It happens quite often that the disk runs out of space and the customer
> does not know what he can do as third parties control the waste of
> space. So we are in the situation where we need to extent the partition
> so the customer can continue his business. Later when the partner has
> solved the issue (real world shows mostly 2 days - 3 weeks) the customer
> wants to shrink again as he does not want to pay the space.
>
>> Honest question, I'm not challenging you, but I would like to understand
>> what it is about shrink that is so critical it may cause you to stop using
>> XFS.
>>
>> One thing about shrink - while i.e. ext4 supports it, the end result of
>> a shrunk filesystem is a scrambled filesystem.  All those heuristics that
>> made the filesystem reasonably performant as it aged get thrown out the
>> window as the fs scavenges for space into which to shrink the filesystem...
>>
>> That said, another option is to use the dm-thinp target, and allocate /
>> de-allocate storage resources to the underlying block device as needed.
>
> We do so while using ceph and trim but the customer pays what he can
> theoretically use and not what he uses in real. Ceph also does not
> provide a way to show us the real usage of a rbd disk.
>
> Greets,
> Stefan
>


Am I right in a guess that this indirect management issue is a primary
reason for having online shrinking as a feature? You can use BTRFS at
it is supporting in-place shrink and achieved enough stability with
plain use-cases, with enough necessity. Of course having simular thing
in an XFS would be quite awesome for a general purpose or for a
visionary case, but the real-world need in such a feature is a bit
limited, from at least cloud provider` point of view.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: any chance for xfs shrinking?

[greg.freemyer]

On May 12, 2015 2:22:16 PM EDT, Andrey Korolyov <andrey@xdel.ru> wrote:
>On Tue, May 12, 2015 at 3:41 PM, Stefan Priebe - Profihost AG
><s.priebe@profihost.ag> wrote:
>>
>> Am 12.05.2015 um 14:35 schrieb Eric Sandeen:
>>> On 5/12/15 7:01 AM, Stefan Priebe - Profihost AG wrote:
>>>> Hi,
>>>>
>>>> while cloud / vms usage become more and more popular and qemu now
>also
>>>> offers memory hot add and unplug, cpu hot add and unplug, we still
>>>> suffer from a missing xfs shrink.
>>>
>>> Filesystem shrink is a very different scenario than "cloud / vms"
>needs
>>> for hot memory & cpu plugging, IMHO.
>>>
>>>> I would like to continue to use XFS as it is a rock solid base
>since
>>>> around 10 years for us.
>>>>
>>>> But one missing piece in variable ressource usage for us is disk
>>>> shrinking. Is there any chance to get an xfs online shrinking?
>>>
>>> Under what circumstance does your workflow require a filesystem
>shrink?
>>
>> May be special but there are a lot of customers out there who do not
>> manager their ressources. So they've partners and partners of
>partners.
>> It happens quite often that the disk runs out of space and the
>customer
>> does not know what he can do as third parties control the waste of
>> space. So we are in the situation where we need to extent the
>partition
>> so the customer can continue his business. Later when the partner has
>> solved the issue (real world shows mostly 2 days - 3 weeks) the
>customer
>> wants to shrink again as he does not want to pay the space.
>>
>>> Honest question, I'm not challenging you, but I would like to
>understand
>>> what it is about shrink that is so critical it may cause you to stop
>using
>>> XFS.
>>>
>>> One thing about shrink - while i.e. ext4 supports it, the end result
>of
>>> a shrunk filesystem is a scrambled filesystem.  All those heuristics
>that
>>> made the filesystem reasonably performant as it aged get thrown out
>the
>>> window as the fs scavenges for space into which to shrink the
>filesystem...
>>>
>>> That said, another option is to use the dm-thinp target, and
>allocate /
>>> de-allocate storage resources to the underlying block device as
>needed.
>>
>> We do so while using ceph and trim but the customer pays what he can
>> theoretically use and not what he uses in real. Ceph also does not
>> provide a way to show us the real usage of a rbd disk.
>>
>> Greets,
>> Stefan
>>
>
>
>Am I right in a guess that this indirect management issue is a primary
>reason for having online shrinking as a feature? You can use BTRFS at
>it is supporting in-place shrink and achieved enough stability with
>plain use-cases, with enough necessity. Of course having simular thing
>in an XFS would be quite awesome for a general purpose or for a
>visionary case, but the real-world need in such a feature is a bit
>limited, from at least cloud provider` point of view.
>
>_______________________________________________
>xfs mailing list
>xfs@oss.sgi.com
>http://oss.sgi.com/mailman/listinfo/xfs

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: any chance for xfs shrinking?

[Dave Chinner]

On Tue, May 12, 2015 at 02:01:54PM +0200, Stefan Priebe - Profihost AG wrote:
> Hi,
> 
> while cloud / vms usage become more and more popular and qemu now also
> offers memory hot add and unplug, cpu hot add and unplug, we still
> suffer from a missing xfs shrink.

Over the years lots of people have come along and said "we want to
implement shrink" and we've pointed them at the pieces that need to
be put together to make it work. However, shrink is a terribly complex
operation which, as Eric has pointed out, scrambles the filesystem
up badly, and so for the most part it isn't a desired operation
to perform on a filesystem.

The reality is that nobody has needed shrink badly enough to push
such a complex set of operations through to completion, or fund a
developer to push it through to completion. Note that it's not just
code that needs to be written, the verification of shrink behaviour
and correctness is also difficult and time consuming.

> I would like to continue to use XFS as it is a rock solid base since
> around 10 years for us.
> 
> But one missing piece in variable ressource usage for us is disk
> shrinking. Is there any chance to get an xfs online shrinking?

In a world of infinite resources at my disposal, I'd say yes. But
when I already have a backlog of development work longer than my
arm, finding 3-6 months to implement shrink is not easy to
accomplish. And, really, reverse mapping btrees and parent pointers
come first so that shrink can be implemented efficiently (shrink
needs reverse block-to-path mapping), and that's another 6 months
worth of work before shrink....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs