Re: [TOPIC LPC] Filesystem Shrink

[Allison Henderson <allison.henderson@oracle.com>]

On 9/8/21 3:25 PM, Andreas Dilger wrote:
> On Sep 8, 2021, at 1:27 AM, Allison Henderson <allison.henderson@oracle.com> wrote:
>>
>> Hi All,
>>
>> Earlier this month I had sent out a lpc micro conference proposal for
>> file system shrink.  It sounds like the talk is of interest, but folks
>> recommended I forward the discussion to fsdevel for more feed back.
>> Below is the abstract for the talk:
>>
>>
>> File system shrink allows a file system to be reduced in size by some specified size blocks as long as the file system has enough unallocated space to do so.  This operation is currently unsupported in xfs.  Though a file system can be backed up and recreated in smaller sizes, this is not functionally the same as an in place resize.  Implementing this feature is costly in terms of developer time and resources, so it is important to consider the motivations to implement this feature.  This talk would aim to discuss any user stories for this feature.  What are the possible cases for a user needing to shrink the file system after creation, and by how much?  Can these requirements be satisfied with a simpler mkfs option to backup an existing file system into a new but smaller filesystem?  In the cases of creating a rootfs, will a protofile suffice?  If the shrink feature is needed, we should further discuss the APIs that users would need.
>>
>> Beyond the user stories, it is also worth discussing implementation challenges.  Reflink and parent pointers can assist in facilitating shrink operations, but is it reasonable to make them requirements for shrink?  Gathering feedback and addressing these challenges will help guide future development efforts for this feature.
>>
>>
>> Comments and feedback are appreciated!
> 
> This is an issue that has come up occasionally in the past, and more
> frequently these days because of virtualization. "Accidental resize"
> kind of mistakes, or an installer formatting a huge root filesystem
> but wanting to carve off separate filesystems for more robustness
> (e.g. so /var/log and /var/tmp don't fill the single root filesystem
> and cause the system to fail).
> 
> There was some prototype work for a "lazy" online shrink mechanism
> for ext4, that essentially just prevented block allocations at the
> end of the filesystem.  This required userspace to move any files
> and inodes that were beyond the high watermark, and then some time
> later either do the shrink offline once the end of the filesystem
> was empty, or later enhance the online resize code to remove unused
> block groups at the end of the filesystem.  This turns out to be not
> as complex as one expects, if the filesystem is already mostly empty,
> which is true in the majority of real use cases ("accidental resize",
> or "huge root partition" cases).
> 
> There is an old a patch available in Patchworks and some discussion
> about what would be needed to make it suitable for production use:
> 
> https://patchwork.ozlabs.org/project/linux-ext4/patch/9ba7e5de79b8b25e335026d57ec0640fc25e5ce0.1534905460.git.jaco@uls.co.za/
> 
> I don't think it would need a huge effort to update that patch and add
> the minor changes that are needed to make it really usable (stop inode
> allocations beyond the high watermark, add a group remove ioctl, etc.)
> 
> Cheers, Andreas
> 
I see, thanks for the link, I didn't know there had been effort made on 
the ext4 side, I had been looking more at the xfs implementation.  This 
certainly looks like it might be a good starting point for ext4, it 
seems both solutions need to limit user allocations one way or another. 
  And I suspect both will have to deal with similar statfs reporting 
challenges discussed for the xfs approach too.  Virtualization issues 
seem to be a common motivator for shrink support in both fs types, so 
it's a good indication that is a worthwhile pursuit.  Perhaps then this 
discussion topic will be of interest to both xfs and ext solutions.

Thanks for the feedback!
Allison


> 
> 
> 
>