Re: freezes during snapshot creation/deletion -- to be expected? (Was: Re: btrfs based backup?)

[Christian Pernegger <pernegger@gmail.com>]

Am Mi., 20. Nov. 2019 um 18:59 Uhr schrieb Oliver Freyermuth
<o.freyermuth@googlemail.com>:
> So I'd say this is not normal.

Good to hear, that means it might be fixable. The alternative would be
to switch to Borg or restic, and I just don't feel comfortable with
deduplication relying solely on hashes, I'm a Luddite like that.

> The first thing you'd need to check is when exactly it happens

Currently 17 minutes past the hour, which is when my cron.hourly runs,
and that only runs btrbk. I can't say for certain if it happens every
hour, but I'm reasonably confident.

> btrbk logs the steps it is doing. Does it happen during the snapshotting, transferring, or deletion of snapshots?

It's just configured to snapshot & prune, no transfer. A central
backup server (grand name, for a white-box NAS) pulls the snapshots
each night and does its own pruning. I'm not sure how to tell when
exactly it happens, as I have not much agency while it is happening.

> Anything in the kernel log?

Nothing suspicious in btrbk.log, dmesg or the systemd journal. The
affected things just stop reacting, then continue as if nothing had
happened.

> Did you run a deduplication tool on the BTRFS volumes, or use quotas?

No to deduplication, maybe to quotas. It's possible that Timeshift
enables them, how can I check?

Just had another episode:
2019-11-21T17:17:01+0100 startup v0.26.0 - - - # btrbk command line
client, version 0.26.0
2019-11-21T17:17:01+0100 snapshot starting
/mnt/timeshift/backup/btrbk-snapshots/@.20191121T171701+0100
/mnt/timeshift/backup/@ - -
2019-11-21T17:17:01+0100 snapshot success
/mnt/timeshift/backup/btrbk-snapshots/@.20191121T171701+0100
/mnt/timeshift/backup/@ - -
2019-11-21T17:17:01+0100 snapshot starting
/mnt/timeshift/backup/btrbk-snapshots/@home.20191121T171701+0100
/mnt/timeshift/backup/@home - -
2019-11-21T17:17:01+0100 snapshot success
/mnt/timeshift/backup/btrbk-snapshots/@home.20191121T171701+0100
/mnt/timeshift/backup/@home - -
2019-11-21T17:17:01+0100 delete_snapshot starting
/mnt/timeshift/backup/btrbk-snapshots/@.20191119T161701+0100 - - -
2019-11-21T17:17:01+0100 delete_snapshot success
/mnt/timeshift/backup/btrbk-snapshots/@.20191119T161701+0100 - - -
2019-11-21T17:17:01+0100 delete_snapshot starting
/mnt/timeshift/backup/btrbk-snapshots/@home.20191119T161701+0100 - - -
2019-11-21T17:17:01+0100 delete_snapshot success
/mnt/timeshift/backup/btrbk-snapshots/@home.20191119T161701+0100 - - -
2019-11-21T17:17:01+0100 delete_snapshot starting
/mnt/timeshift/backup/btrbk-snapshots/@home-chris-.steam.20191119T161701+0100
- - -
2019-11-21T17:17:01+0100 delete_snapshot success
/mnt/timeshift/backup/btrbk-snapshots/@home-chris-.steam.20191119T161701+0100
- - -
2019-11-21T17:17:01+0100 finished success - - - -

I had a tail on the log, these came out in one go, no larger pauses.
At first I thought, just my luck, here I am lying in wait and of
course everything works, then the mini-freeze happened. CPU usage in
one core spiked during the freeze, but I couldn't switch tabs from the
graphs to the process list in gnome-system-monitor. Top it is, next
time.

Am Mi., 20. Nov. 2019 um 19:32 Uhr schrieb Chris Murphy
<lists@colorremedies.com>:
> What are the mount options?

defaults, which comes out as
rw,relatime,ssd,space_cache,subvolid=,subvol=, according to mount.

> And what's the workload immediate prior to the snapshot? Or does it always happen no matter the workload?

Can't guarantee "always", but ... This time I was in the process of
composing this e-Mail. A couple of things open, sure, Firefox, couple
of terminals, Signal, evince, deadbeat [stopped], but not doing
anything much. I'd call the workload "idle", especially fs-wise. Last
time I was typing at a bash prompt via gnome-terminal -- the input
wouldn't show or register until it was over. It's not only
i/o-intensive stuff that blocks.

Am Do., 21. Nov. 2019 um 02:51 Uhr schrieb Qu Wenruo <quwenruo.btrfs@gmx.com>:
> Are you using qgroup?

Not knowingly. If either Timeshift or btrbk enable them, it's possible.

Cheers,
C.