Freeing space over reboot question

Freeing space over reboot question

[Norbert Scheibner]

Gl=C3=BCck Auf!
I use now kernel 3.2. The filesystem was originally created under 2.6.3=
9 on 1 whole hdd, mounted with "noatime,nodiratime,inode_cache". I use =
it for backups: rsync the whole system to a subvolume, snapshot it and =
then delete some tempfiles in the snapshot, which are 90% of the full-b=
ackup, all once a day. In figures: on this 1 TB hdd is the full-backup =
with around 600 GiB and 10 to 20 snapshots with around 30 GiB each, all=
 together using around 700 GiB on disc.

What I did:
- I deleted (by accident) the big subvolume with the full-backup with "=
btrfs subvolume delete" and recreated it with the same name with a snap=
shot of the latest snapshot.
- During the deletion of this big subvolume in background I changed the=
 kernel from 3.1 to 3.2 and did a reboot.
- After that, the fs was operational, but the space was still used and =
the next system-backup onto this fs failed with no space left errors. "=
btrfs filesystem df" showed me that the fs used the whole hdd and that =
there were only some kB free, which fits to the errors from rsync durin=
g backup.

So the used space of subvolume I deleted, was not freed.

How to get the space back which should have been freed?
A balance did not help. What worked was the deletion of that half-fille=
d subvolume, which I use for the full backup. After that the space got =
freed and the next balance run shrinked the fs again, so that it uses o=
nly a part of the hdd.

What I wonder is: Couldn't this be a little bit more user-friendly?

If there is a background process running like this here, freeing some s=
pace, should the umount take as long as the background process or shoul=
d the background process stop immediately and restart after the next mo=
unt (if possible, especially with a kernel change in between or the pos=
sibility that the fs gets mounted read-only)?

=2E.. Or is this all nonsense and it happened here because I rebooted a=
nd after that used another kernel.

My best wishes
    Norbert
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Freeing space over reboot question

[Chester]

On Thu, Feb 9, 2012 at 11:42 AM, Norbert Scheibner <scno@gmx.net> wrote=
:
> Gl=FCck Auf!
> I use now kernel 3.2. The filesystem was originally created under 2.6=
=2E39 on 1 whole hdd, mounted with "noatime,nodiratime,inode_cache". I =
use it for backups: rsync the whole system to a subvolume, snapshot it =
and then delete some tempfiles in the snapshot, which are 90% of the fu=
ll-backup, all once a day. In figures: on this 1 TB hdd is the full-bac=
kup with around 600 GiB and 10 to 20 snapshots with around 30 GiB each,=
 all together using around 700 GiB on disc.
>
> What I did:
> - I deleted (by accident) the big subvolume with the full-backup with=
 "btrfs subvolume delete" and recreated it with the same name with a sn=
apshot of the latest snapshot.
> - During the deletion of this big subvolume in background I changed t=
he kernel from 3.1 to 3.2 and did a reboot.
> - After that, the fs was operational, but the space was still used an=
d the next system-backup onto this fs failed with no space left errors.=
 "btrfs filesystem df" showed me that the fs used the whole hdd and tha=
t there were only some kB free, which fits to the errors from rsync dur=
ing backup.
>
> So the used space of subvolume I deleted, was not freed.
>
> How to get the space back which should have been freed?
> A balance did not help. What worked was the deletion of that half-fil=
led subvolume, which I use for the full backup. After that the space go=
t freed and the next balance run shrinked the fs again, so that it uses=
 only a part of the hdd.
>
> What I wonder is: Couldn't this be a little bit more user-friendly?
>
> If there is a background process running like this here, freeing some=
 space, should the umount take as long as the background process or sho=
uld the background process stop immediately and restart after the next =
mount (if possible, especially with a kernel change in between or the p=
ossibility that the fs gets mounted read-only)?
>

A similar thing has happened to me recently. The snapshot deletion
happens asynchronously and should continue after a reboot (in my
case). If you boot up your system and leave it idle, take a look at
iotop. You might see a [btrfs-cleaner] doing some work.

> ... Or is this all nonsense and it happened here because I rebooted a=
nd after that used another kernel.
>
> My best wishes
> =A0 =A0Norbert
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Freeing space over reboot question

[Norbert Scheibner]

On Thu, 9 Feb 2012 12:11:19 -0600 Chester wrote:

> A similar thing has happened to me recently. The snapshot deletion
> happens asynchronously and should continue after a reboot (in my
> case). If you boot up your system and leave it idle, take a look at
> iotop. You might see a [btrfs-cleaner] doing some work.

No that didn't help. The backup did not start immediately after the reboot and 5 days later - still no change.

    Norbert

Re: Freeing space over reboot question

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

On Thu, 09 Feb 2012 18:42:32 +0100
"Norbert Scheibner" <scno@gmx.net> wrote:

> So the used space of subvolume I deleted, was not freed.

AFAIK the only reliable way currently to ensure the space after a subvolume
deletion is freed, is to remount the FS. 

In my opinion this is very inconvenient in many aspects, and I think "btrfs fi
sync" absolutely should sync all ongoing cleanerd operations too. Which it
currently doesn't, and in fact this makes me wonder, does "btrfs fi sync"
even do anything more than the standard "sync" program at all.

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Freeing space over reboot question

[cwillu]

On Thu, Feb 9, 2012 at 12:20 PM, Roman Mamedov <rm@romanrm.ru> wrote:
> On Thu, 09 Feb 2012 18:42:32 +0100
> "Norbert Scheibner" <scno@gmx.net> wrote:
>
>> So the used space of subvolume I deleted, was not freed.
>
> AFAIK the only reliable way currently to ensure the space after a subvolume
> deletion is freed, is to remount the FS.
>
> In my opinion this is very inconvenient in many aspects, and I think "btrfs fi
> sync" absolutely should sync all ongoing cleanerd operations too. Which it
> currently doesn't, and in fact this makes me wonder, does "btrfs fi sync"
> even do anything more than the standard "sync" program at all.

It doesn't.  It triggers the same function that the global sync would
(but doesn't also sync every other mounted filesystem).

Re: Freeing space over reboot question

[Norbert Scheibner]

On: Fri, 10 Feb 2012 00:20:55 +0600 Roman Mamedov <rm@romanrm.ru> wrote

> AFAIK the only reliable way currently to ensure the space after a
> subvolume
> deletion is freed, is to remount the FS. 

Have You tried it Yourself? I think the problem was the remount
before the space has been completely freed in background. It
left a valid and working fs, with still work to do.

In my opinion the kernel should either stall the umount till the
space is given free or restart the cleaner after the next mount.

To stall the umount could be annoying and make the user believe
something is broken, because the cleaner could take a while and
it feels to classic for modern fs.

Is there a way to leave an entry in the log to replay after the
next mount? Pardon me if I use the wrong terms here, I'm just
an interested user and not a fs-crack or even a programmer.

Regards
    Norbert

Re: Freeing space over reboot question

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 650 bytes --]

On Sat, 11 Feb 2012 14:47:15 +0100
"Norbert Scheibner" <scno@gmx.net> wrote:

> Have You tried it Yourself? I think the problem was the remount
> before the space has been completely freed in background. It
> left a valid and working fs, with still work to do.

Yes, after some snapshot deletions the umount takes a really long time.

> In my opinion the kernel should either stall the umount till the
> space is given free

In my experience it seemed to do just that.

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Freeing space over reboot question

[Norbert Scheibner]

On Sat, 11 Feb 2012 19:56:32 +0600 Roman Mamedov <rm@romanrm.ru> wrote:

> > Have You tried it Yourself? I think the problem was the remount
> > before the space has been completely freed in background. It
> > left a valid and working fs, with still work to do.
> 
> Yes, after some snapshot deletions the umount takes a really long time.

Ok, then the question is, why did this not happen here?
 
> > In my opinion the kernel should either stall the umount till the
> > space is given free
> 
> In my experience it seemed to do just that.

Hm, I did the umount under kernel 3.1 and after the reboot
I used kernel 3.2. Is that the reason?

Regards
    Norbert