btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

[Astro Xe]

On my box, the "used" value in the output of "btrfs filesystem df" is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute "btrfs filesystem sync", in order to update the value.

How do I fix this? Or, could someone fix this in btrfs-progs, please?

I suspect the cause is that subvolumes are marked immediately as deleted, but cleaned up at a later time, is this right?

Thanks,

Re: btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

[Russell Coker]

On Tue, 20 May 2014 14:50:10 Astro Xe wrote:
> On my box, the "used" value in the output of "btrfs filesystem df" is not
> updated in a timely manner, after that one or more subvolumes have been
> deleted. I need to execute "btrfs filesystem sync", in order to update the
> value.
> 
> How do I fix this? Or, could someone fix this in btrfs-progs, please?
> 
> I suspect the cause is that subvolumes are marked immediately as deleted,
> but cleaned up at a later time, is this right?

That appears to be the case.  I've deleted subvolumes and then had btrfs take 
10+ minutes to reclaim all the disk space.  I've even had the system crash 
during the process (due to btrfs bugs that seem fixed in 3.14) and the space 
keeps getting freed after the reboot.

Do we really want to make it the default to block on a btrfs sync?  In the 
case of manual sysadmin tasks it's probably going to be preferable to run the 
delete command and then move on to other things.

In the case of cron jobs we have to consider bulk performance, maybe it would 
be better for the authors of cron jobs to run a filesystem sync after deleting 
all snapshots to avoid driving the load average too high (I have had issues in 
the past when cron jobs to remove snapshots end up running at the same time as 
creating snapshots and scrubbing the filesystem).  If a cron job was to delete 
dozens of snapshots and sync after each one would that cause more disk access 
than syncing once after all deletes?

-- 
My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/

Re: btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

On Tue, May 20, 2014 at 02:50:10PM +0100, Astro Xe wrote:
> 
> On my box, the "used" value in the output of "btrfs filesystem df" is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute "btrfs filesystem sync", in order to update the value.
> 
> How do I fix this? Or, could someone fix this in btrfs-progs, please?

   I've not tested this, but I think you need one of the two -c/-C
options to btrfs sub del, which perform a synchronous delete. Once the
command returns, you can be assured that the subvolume has actually
been deleted and any extents freed up.

> I suspect the cause is that subvolumes are marked immediately as deleted, but cleaned up at a later time, is this right?

   Correct.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
      --- Great oxymorons of the world, no. 10: Business Ethics ---      

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

[Astro Xe]

On my box, the "used" value in the output of "btrfs filesystem df" is not updated in a timely manner, after that one or more subvolumes have been deleted. I need to execute "btrfs filesystem sync", in order to update the value. Could someone fix this, please? I suspect the cause is that subvolumes are marked immediately as deleted, but cleanedup at a later time, is this right?

Re: btrfs fi df output is not updated in a timely manner after subvolumes have been deleted

[Calvin Walton]

On Tue, 2014-05-20 at 14:46 +0100, Astro Xe wrote:
> On my box, the "used" value in the output of "btrfs filesystem df" is
> not updated in a timely manner, after that one or more subvolumes have
> been deleted. I need to execute "btrfs filesystem sync", in order to
> update the value. Could someone fix this, please? I suspect the cause
> is that subvolumes are marked immediately as deleted, but cleanedup at
> a later time, is this right?

Yes, this is exactly true. After you delete a subvolume on btrfs, the
"btrfs-cleaner" kernel process has to before the space is actually freed
up. This process goes through all the contents of the subvolume,
determines which files aren't being shared with other subvolumes or
reflinks, and frees the space for those.

The problem is that it doesn't make sense to update the free space
immediately - and it's not really possible to do it either.

It doesn't make sense to update the free space immediately, because the
space isn't actually usable immediately. Before you can create new files
that use this space, you have to wait for the cleaner process to run and
delete the old files.

It's not really possible to update the free space immediately, because
the filesystem doesn't actually know how much space would be freed! In
order to determine this number, the filesystem has to check all the
files in the subvolume - which is what btrfs-cleaner is doing.

-- 
Calvin Walton <calvin.walton@kepstin.ca>