* Re: Possible to wait for snapshot deletion?
2014-02-13 19:02 Possible to wait for snapshot deletion? Kai Krakow
@ 2014-02-13 19:29 ` Brendan Hide
2014-02-14 0:25 ` Kai Krakow
2014-02-13 19:57 ` Hugo Mills
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Brendan Hide @ 2014-02-13 19:29 UTC (permalink / raw)
To: Kai Krakow, linux-btrfs
On 2014/02/13 09:02 PM, Kai Krakow wrote:
> Hi!
>
> Is it technically possible to wait for a snapshot completely purged from
> disk? I imagine an option like "--wait" for btrfs delete subvolume.
>
> This would fit some purposes I'm planning to implement:
>
> * In a backup scenario
I have a similar use-case for this also involving backups. In my case I
have a script that uses a btrfs filesystem for the backup store using
snapshots. At the end of each run, if diskspace usage is below a
predefined threshold, it will delete old snapshots until the diskspace
usage is below that threshold again.
Of course, the first time I added the automatic deletion, it deleted far
more than was necessary due to the fact that the actual freeing of
diskspace is asynchronous from the command completion. I ended up
setting a small delay (of about 60 seconds) between each iteration and
also set it to monitor system load. If load is not low enough after the
delay then it waits another 60 seconds.
This complicated (frankly broken) workaround would be completely
unnecessary with a --wait switch.
Alternatively, perhaps a knob where we can see if a subvolume deletion
is in progress could help.
--
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Possible to wait for snapshot deletion?
2014-02-13 19:29 ` Brendan Hide
@ 2014-02-14 0:25 ` Kai Krakow
0 siblings, 0 replies; 10+ messages in thread
From: Kai Krakow @ 2014-02-14 0:25 UTC (permalink / raw)
To: linux-btrfs
Brendan Hide <brendan@swiftspirit.co.za> schrieb:
>> Is it technically possible to wait for a snapshot completely purged from
>> disk? I imagine an option like "--wait" for btrfs delete subvolume.
>>
>> This would fit some purposes I'm planning to implement:
>>
>> * In a backup scenario
>
> I have a similar use-case for this also involving backups. In my case I
> have a script that uses a btrfs filesystem for the backup store using
> snapshots. At the end of each run, if diskspace usage is below a
> predefined threshold, it will delete old snapshots until the diskspace
> usage is below that threshold again.
Yeah, I thought of that approach first, too... But:
> Of course, the first time I added the automatic deletion, it deleted far
> more than was necessary due to the fact that the actual freeing of
> diskspace is asynchronous from the command completion. I ended up
> setting a small delay (of about 60 seconds) between each iteration and
> also set it to monitor system load. If load is not low enough after the
> delay then it waits another 60 seconds.
Due to btrfs' behavior that cannot reliably work as you also figured out. It
will need quirky work-arounds totally dependent on system load.
> This complicated (frankly broken) workaround would be completely
> unnecessary with a --wait switch.
That's why I had the idea. ;-)
> Alternatively, perhaps a knob where we can see if a subvolume deletion
> is in progress could help.
Like "btrfs scrub status"...
If you're feeling curious, here's what I've implemented yet (snapshot
deletion still on the todo list):
https://gist.github.com/kakra/5520370
My idea is to fork-off a background process which constantly removes old
snapshots (within sane bounds yet to be defined) and then exits. Some
mechanism has to be found to not run into a deadlock situation here. Then,
at the end issue a bash "wait" to join the processes again and run a final
sync. However, I don't like to issue explicit syncs from within the
subprocess as the filesystem is busy with rsync at the same time.
PS: Yes, I know there's btrfs send/receive - but it doesn't seem ready for
the big show yet because it still has many strange quirks and should not be
run unattended yet.
--
Replies to list only preferred.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Possible to wait for snapshot deletion?
2014-02-13 19:02 Possible to wait for snapshot deletion? Kai Krakow
2014-02-13 19:29 ` Brendan Hide
@ 2014-02-13 19:57 ` Hugo Mills
2014-02-13 20:45 ` Garry T. Williams
` (2 subsequent siblings)
4 siblings, 0 replies; 10+ messages in thread
From: Hugo Mills @ 2014-02-13 19:57 UTC (permalink / raw)
To: Kai Krakow; +Cc: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1995 bytes --]
On Thu, Feb 13, 2014 at 08:02:43PM +0100, Kai Krakow wrote:
> Hi!
>
> Is it technically possible to wait for a snapshot completely purged from
> disk? I imagine an option like "--wait" for btrfs delete subvolume.
I'm sure there's been some patches for that around here somewhere,
some time ago. I guess they didn't get merged.
Hugo.
> This would fit some purposes I'm planning to implement:
>
> * In a backup scenario have a subprocess which deletes snapshots one by one,
> starting with the oldest. If free space raises above a certain threshold,
> pause the subprocess. If number of kept snapshots falls below a certain
> threshold, exit the subprocess. When the backup job finished, it joins the
> subprocess to wait for a pending subvolume deletion if any, then syncs the
> filesystem and waits some grace time for uncommitted writes, then shuts
> the system down or hibernates it.
>
> * Wait for pending background subvolume deletion before putting the system
> to sleep.
>
> * Get better control of cron jobs working with subvolumes so jobs either do
> not overlap or do not put too much parallel work on the file system.
>
> Scrubbing does allow that. Why don't snapshots do? At least I found no
> options for it.
>
> Background information: If btrfs is shut down improperly, already deleted
> subvolume entries may reappear in their directories. The deletion is not
> gracefully resumed. Since deletion of subvolumes/snapshots can take hours,
> this is a problem for systems that are not guaranteed to be up all the time
> or suffer from disconnected power, especially if multiple snapshots are
> being deleted at once. With an option to wait, a script managing snapshot
> deletion could more gracefully resume its job.
>
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- Gentlemen! You can't fight here! This is the War Room! ---
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Possible to wait for snapshot deletion?
2014-02-13 19:02 Possible to wait for snapshot deletion? Kai Krakow
2014-02-13 19:29 ` Brendan Hide
2014-02-13 19:57 ` Hugo Mills
@ 2014-02-13 20:45 ` Garry T. Williams
2014-02-14 0:12 ` Kai Krakow
2014-02-13 21:42 ` Holger Hoffstätte
2014-02-14 16:15 ` David Sterba
4 siblings, 1 reply; 10+ messages in thread
From: Garry T. Williams @ 2014-02-13 20:45 UTC (permalink / raw)
To: linux-btrfs
On 2-13-14 20:02:43 Kai Krakow wrote:
> Is it technically possible to wait for a snapshot completely purged
> from disk? I imagine an option like "--wait" for btrfs delete
> subvolume.
This may be what you're looking for:
http://www.spinics.net/lists/linux-btrfs/msg29833.html
--
Garry T. Williams
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Possible to wait for snapshot deletion?
2014-02-13 19:02 Possible to wait for snapshot deletion? Kai Krakow
` (2 preceding siblings ...)
2014-02-13 20:45 ` Garry T. Williams
@ 2014-02-13 21:42 ` Holger Hoffstätte
2014-02-14 0:15 ` Kai Krakow
2014-02-14 16:15 ` David Sterba
4 siblings, 1 reply; 10+ messages in thread
From: Holger Hoffstätte @ 2014-02-13 21:42 UTC (permalink / raw)
To: linux-btrfs
On Thu, 13 Feb 2014 20:02:43 +0100, Kai Krakow wrote:
> Is it technically possible to wait for a snapshot completely purged from
> disk? I imagine an option like "--wait" for btrfs delete subvolume.
That would indeed be sweet (see pointer to patch by Garry Williams).
In the meantime have you tried issuing a simple "sync" in your script
after every snapshot deletion? I've observed that this kicks the cleaner
into action.
-h
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Possible to wait for snapshot deletion?
2014-02-13 21:42 ` Holger Hoffstätte
@ 2014-02-14 0:15 ` Kai Krakow
0 siblings, 0 replies; 10+ messages in thread
From: Kai Krakow @ 2014-02-14 0:15 UTC (permalink / raw)
To: linux-btrfs
Holger Hoffstätte <holger.hoffstaette@googlemail.com> schrieb:
> On Thu, 13 Feb 2014 20:02:43 +0100, Kai Krakow wrote:
>
>> Is it technically possible to wait for a snapshot completely purged from
>> disk? I imagine an option like "--wait" for btrfs delete subvolume.
>
> That would indeed be sweet (see pointer to patch by Garry Williams).
>
> In the meantime have you tried issuing a simple "sync" in your script
> after every snapshot deletion? I've observed that this kicks the cleaner
> into action.
I was under the impression that a sync command just doesn't do the trick. It
kicks in the cleaner but returns to early. But I din't try that again
lately.
But I'm actually currently using "btrfs fs sync" to do it because a normal
"sync" command would affect all my filesystems and I'd like to keep it
isolated.
I'll post a link to my backup script for reference in another post in this
thread.
--
Replies to list only preferred.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Possible to wait for snapshot deletion?
2014-02-13 19:02 Possible to wait for snapshot deletion? Kai Krakow
` (3 preceding siblings ...)
2014-02-13 21:42 ` Holger Hoffstätte
@ 2014-02-14 16:15 ` David Sterba
4 siblings, 0 replies; 10+ messages in thread
From: David Sterba @ 2014-02-14 16:15 UTC (permalink / raw)
To: Kai Krakow; +Cc: linux-btrfs
On Thu, Feb 13, 2014 at 08:02:43PM +0100, Kai Krakow wrote:
> Is it technically possible to wait for a snapshot completely purged from
> disk? I imagine an option like "--wait" for btrfs delete subvolume.
I have the patch WIP, will look at it again.
> This would fit some purposes I'm planning to implement:
>
> * In a backup scenario have a subprocess which deletes snapshots one by one,
> starting with the oldest. If free space raises above a certain threshold,
> pause the subprocess. If number of kept snapshots falls below a certain
> threshold, exit the subprocess. When the backup job finished, it joins the
> subprocess to wait for a pending subvolume deletion if any, then syncs the
> filesystem and waits some grace time for uncommitted writes, then shuts
> the system down or hibernates it.
>
> * Wait for pending background subvolume deletion before putting the system
> to sleep.
>
> * Get better control of cron jobs working with subvolumes so jobs either do
> not overlap or do not put too much parallel work on the file system.
These usecases should be covered by the 'wait' command.
> If btrfs is shut down improperly, already deleted
> subvolume entries may reappear in their directories. The deletion is not
> gracefully resumed. Since deletion of subvolumes/snapshots can take hours,
> this is a problem for systems that are not guaranteed to be up all the time
> or suffer from disconnected power, especially if multiple snapshots are
> being deleted at once. With an option to wait, a script managing snapshot
> deletion could more gracefully resume its job.
As mentioned down the thread, next progs release will be able to force a
transaction commit when the subvoulme is deleted.
^ permalink raw reply [flat|nested] 10+ messages in thread