syntax for deleting subvolumes?

syntax for deleting subvolumes?

[K Richard Pixley]

I'm having trouble deleting a subvolume.

[root@new-alfred ~]# uname -a
Linux new-alfred.corp.graphitesystems.com 3.10.0-123.el7.x86_64 #1 SMP 
Mon May 5 11:16:57 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@new-alfred ~]#   btrfs --version
Btrfs v3.16.2
[root@new-alfred ~]#   btrfs fi show
Label: 'rhel_bark'  uuid: d3c6c783-c190-4675-99f2-b83f3c462656
         Total devices 2 FS bytes used 14.25GiB
         devid    1 size 461.38GiB used 7.03GiB path /dev/sda2
         devid    2 size 465.76GiB used 12.01GiB path /dev/sdb1

Btrfs v3.16.2
[root@new-alfred ~]#   btrfs fi df /
Data, single: total=15.01GiB, used=13.19GiB
System, RAID1: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, RAID1: total=2.00GiB, used=1.07GiB
Metadata, single: total=8.00MiB, used=0.00
[root@new-alfred ~]# btrfs subvolume list /
ID 257 gen 1483 top level 5 path root
ID 258 gen 1466 top level 5 path home
[root@new-alfred ~]# btrfs subvolume delete home
Transaction commit: none (default)
ERROR: error accessing 'home'
[root@new-alfred ~]# btrfs subvolume delete /home
Transaction commit: none (default)
ERROR: error accessing '/home'

What format do I need to use to delete the "home" subvolume?

--rich

Re: syntax for deleting subvolumes?

[Chris Murphy]

On Wed, Mar 18, 2015 at 2:42 PM, K Richard Pixley
<rpixley@graphitesystems.com> wrote:
> I'm having trouble deleting a subvolume.
>
> [root@new-alfred ~]# uname -a
> Linux new-alfred.corp.graphitesystems.com 3.10.0-123.el7.x86_64 #1 SMP Mon

If highly recommend that you check out elrepo.org for a newer kernel
than this. Currently they have
kernel-ml-3.19.1-1.el7.elrepo.x86_64.rpm available.

The Fedora 21 package for btrfs-progs-3.19 also works on RHEL 7/CentOS
7, which you can find in koji.fedoraproject.org, or build it from
source if you prefer.



> [root@new-alfred ~]#   btrfs fi show
> Label: 'rhel_bark'  uuid: d3c6c783-c190-4675-99f2-b83f3c462656
>         Total devices 2 FS bytes used 14.25GiB
>         devid    1 size 461.38GiB used 7.03GiB path /dev/sda2
>         devid    2 size 465.76GiB used 12.01GiB path /dev/sdb1
>
> Btrfs v3.16.2
> [root@new-alfred ~]#   btrfs fi df /
> Data, single: total=15.01GiB, used=13.19GiB
> System, RAID1: total=8.00MiB, used=16.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, RAID1: total=2.00GiB, used=1.07GiB
> Metadata, single: total=8.00MiB, used=0.00
> [root@new-alfred ~]# btrfs subvolume list /
> ID 257 gen 1483 top level 5 path root
> ID 258 gen 1466 top level 5 path home
> [root@new-alfred ~]# btrfs subvolume delete home
> Transaction commit: none (default)
> ERROR: error accessing 'home'
> [root@new-alfred ~]# btrfs subvolume delete /home
> Transaction commit: none (default)
> ERROR: error accessing '/home'
>
> What format do I need to use to delete the "home" subvolume?

# mkdir  /mnt/btr
# mount /dev/sda2 /mnt/btr
# btrfs subvolume delete /mnt/btr/home

The Fedora/RHEL/CentOS installer creates two subvolumes: root and
home. If you check out fstab, those subvolumes are mounted at / and
/home. Therefore the top level subvolume (id 5) is not mounted by
default, so there's no way to delete subvolumes in the top level.



-- 
Chris Murphy

Re: syntax for deleting subvolumes?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2499 bytes --]

On Wed, Mar 18, 2015 at 01:42:55PM -0700, K Richard Pixley wrote:
> I'm having trouble deleting a subvolume.
> 
> [root@new-alfred ~]# uname -a
> Linux new-alfred.corp.graphitesystems.com 3.10.0-123.el7.x86_64 #1
> SMP Mon May 5 11:16:57 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
> [root@new-alfred ~]#   btrfs --version
> Btrfs v3.16.2
> [root@new-alfred ~]#   btrfs fi show
> Label: 'rhel_bark'  uuid: d3c6c783-c190-4675-99f2-b83f3c462656
>         Total devices 2 FS bytes used 14.25GiB
>         devid    1 size 461.38GiB used 7.03GiB path /dev/sda2
>         devid    2 size 465.76GiB used 12.01GiB path /dev/sdb1
> 
> Btrfs v3.16.2
> [root@new-alfred ~]#   btrfs fi df /
> Data, single: total=15.01GiB, used=13.19GiB
> System, RAID1: total=8.00MiB, used=16.00KiB
> System, single: total=4.00MiB, used=0.00
> Metadata, RAID1: total=2.00GiB, used=1.07GiB
> Metadata, single: total=8.00MiB, used=0.00
> [root@new-alfred ~]# btrfs subvolume list /
> ID 257 gen 1483 top level 5 path root
> ID 258 gen 1466 top level 5 path home
> [root@new-alfred ~]# btrfs subvolume delete home
> Transaction commit: none (default)
> ERROR: error accessing 'home'
> [root@new-alfred ~]# btrfs subvolume delete /home
> Transaction commit: none (default)
> ERROR: error accessing '/home'
> 
> What format do I need to use to delete the "home" subvolume?

  I'm guessing you've got the home subvolume mounted, something like:

# mount /dev/sda2 /home -o subvol=home

   You can't delete that while it's mounted, because you'd be trying
to delete the subvolume out from under the mountpoint. So the first
thing is to unmount it.

   Then the syntax to delete a subvolume is:

# btrfs sub del /path/to/subvolume

... but at the moment, you can't see the subvolume, because it's not
mounted, and there's no way to see it. The trick here it to mount
something higher than the subvolume you want to operate on, and access
it through that mount path. So, mount the top level subvolume somewhere:

# mount /dev/sda2 /media/btrfs -o subvolid=0

then delete the subvolume through that:

# btrfs sub del /media/btrfs/home

   In fact, I'd recommend doing _all_ subvolume management operations
(and nothing else) through the top-level mount at /media/btrfs.

   Hugo.

-- 
Hugo Mills             | "There's a Martian war machine outside -- they want
hugo@... carfax.org.uk | to talk to you about a cure for the common cold."
http://carfax.org.uk/  |
PGP: 65E74AC0          |                           Stephen Franklin, Babylon 5

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: syntax for deleting subvolumes?

[K Richard Pixley]

On 3/18/15 14:06 , Chris Murphy wrote:
> The Fedora/RHEL/CentOS installer creates two subvolumes: root and 
> home. If you check out fstab, those subvolumes are mounted at / and 
> /home. Therefore the top level subvolume (id 5) is not mounted by 
> default, so there's no way to delete subvolumes in the top level. 
Ah!  Thank you.  That's the piece I was missing.

IMO, someone needs to take a clue-by-four to the heads of the 
Fedora/RHEL/CentOS installer folks.  I see no reason for this with btrfs.

--rich

Re: syntax for deleting subvolumes?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1518 bytes --]

On Wed, Mar 18, 2015 at 02:39:26PM -0700, K Richard Pixley wrote:
> On 3/18/15 14:06 , Chris Murphy wrote:
> >The Fedora/RHEL/CentOS installer creates two subvolumes: root and
> >home. If you check out fstab, those subvolumes are mounted at /
> >and /home. Therefore the top level subvolume (id 5) is not mounted
> >by default, so there's no way to delete subvolumes in the top
> >level.
> Ah!  Thank you.  That's the piece I was missing.
> 
> IMO, someone needs to take a clue-by-four to the heads of the
> Fedora/RHEL/CentOS installer folks.  I see no reason for this with
> btrfs.

   Actually, it's the recommended approach.

   We've found over time that nesting subvolumes inside each other is,
in general, more trouble than it's worth. It makes it harder to do
things like take snapshots and then replace the original with a
snapshot (e.g. rollback to an earlier state).

   Arranging the subvolumes with the top level (subvolid=0) as nothing
but a store for subvolumes means that you don't get the subvolumes
tangled up with each other, so it's much easier to manage them in
general.

   There's a few cases where nesting subvolumes is helpful, but that's
more to do with preventing snapshots being taken of some pieces of the
filesystem.

   Hugo.

-- 
Hugo Mills             | "There's a Martian war machine outside -- they want
hugo@... carfax.org.uk | to talk to you about a cure for the common cold."
http://carfax.org.uk/  |
PGP: 65E74AC0          |                           Stephen Franklin, Babylon 5

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: syntax for deleting subvolumes?

[Chris Murphy]

On Wed, Mar 18, 2015 at 3:39 PM, K Richard Pixley
<rpixley@graphitesystems.com> wrote:

> Ah!  Thank you.  That's the piece I was missing.
>
> IMO, someone needs to take a clue-by-four to the heads of the
> Fedora/RHEL/CentOS installer folks.  I see no reason for this with btrfs.

Other than the technical reasons Hugo mentions regarding nesting...

The problem with the "install normally to top level with Linux FHS"
approach like Ubuntu and openSUSE follow now, is that snapshots then
have to go in the mounted path. This arguably exposes old binaries in
that mounted path and is a possible security risk. There are some ways
to mitigate that, but better when it's simply not in the mounted path,
sorta like a chroot.

It's also a better way to organize stateless systems. Myriad trees
that can be used to form a stateless system existing "out of tree" and
mounted either by path or subvolid is more sane (or at least less
madness inducing) than alternatives. See under "what we propose" for
the subvol naming convention:
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html

This is also compatible with delivery of such systems with a btrfs seed device.


-- 
Chris Murphy

Re: syntax for deleting subvolumes?

[K Richard Pixley]

On 3/18/15 14:55 , Hugo Mills wrote:
> On Wed, Mar 18, 2015 at 02:39:26PM -0700, K Richard Pixley wrote:
>> On 3/18/15 14:06 , Chris Murphy wrote:
>>> The Fedora/RHEL/CentOS installer creates two subvolumes: root and
>>> home. If you check out fstab, those subvolumes are mounted at /
>>> and /home. Therefore the top level subvolume (id 5) is not mounted
>>> by default, so there's no way to delete subvolumes in the top
>>> level.
>> Ah!  Thank you.  That's the piece I was missing.
>>
>> IMO, someone needs to take a clue-by-four to the heads of the
>> Fedora/RHEL/CentOS installer folks.  I see no reason for this with
>> btrfs.
>     Actually, it's the recommended approach.
>
>     We've found over time that nesting subvolumes inside each other is,
> in general, more trouble than it's worth. It makes it harder to do
> things like take snapshots and then replace the original with a
> snapshot (e.g. rollback to an earlier state).
>
>     Arranging the subvolumes with the top level (subvolid=0) as nothing
> but a store for subvolumes means that you don't get the subvolumes
> tangled up with each other, so it's much easier to manage them in
> general.
>
>     There's a few cases where nesting subvolumes is helpful, but that's
> more to do with preventing snapshots being taken of some pieces of the
> filesystem.
Yes, I understand about the flat arrangement.  It's been that way for 
years now.

My complaint is a) about multiple subvols and b) about an unnecessary 
and redundant subvol for the top level file system.

The redundancy should be obvious.  And the problem with multiple subvols 
is that they aren't needed and make snapshotting an entire system more 
difficult than it needs to be.  I want to be able to snapshot and 
chroot.  The traditional reason for partitioning a disk was to isolate 
errors and to firewall "out of space" usage.  Neither of those really 
apply to btrfs as far as I can see.

I would much rather the installer simply create a single file system and 
mount the partition directly rather than through a superfluous subvol.  
I've never seen it done any other way as you might guess from my 
confusion above.

--rich

Re: syntax for deleting subvolumes?

[K Richard Pixley]

On 3/18/15 15:15 , Chris Murphy wrote:
> On Wed, Mar 18, 2015 at 3:39 PM, K Richard Pixley
> <rpixley@graphitesystems.com> wrote:
>
>> Ah!  Thank you.  That's the piece I was missing.
>>
>> IMO, someone needs to take a clue-by-four to the heads of the
>> Fedora/RHEL/CentOS installer folks.  I see no reason for this with btrfs.
> Other than the technical reasons Hugo mentions regarding nesting...
>
> The problem with the "install normally to top level with Linux FHS"
> approach like Ubuntu and openSUSE follow now, is that snapshots then
> have to go in the mounted path. This arguably exposes old binaries in
> that mounted path and is a possible security risk. There are some ways
> to mitigate that, but better when it's simply not in the mounted path,
> sorta like a chroot.
>
> It's also a better way to organize stateless systems. Myriad trees
> that can be used to form a stateless system existing "out of tree" and
> mounted either by path or subvolid is more sane (or at least less
> madness inducing) than alternatives. See under "what we propose" for
> the subvol naming convention:
> http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html
>
> This is also compatible with delivery of such systems with a btrfs seed device.
I see.  Thanks for the education.

I'm not sure what I think about the possible security risk, but I hear 
the concern.

Most of the uses I have for btrfs involve fairly dynamic use of 
snapshots, typically by non-root users. That's what brought me to btrfs 
in the first place and continues to be the biggest driver for me.

Because of this, the top level file system would need to be mounted 
pretty much constantly, which essentially removes any benefit from the 
redundant top level subvol.  It's just a nuisance for my applications.  
And most of my applications try very hard to avoid mounting the 
snapshots.  That takes too much time and isn't reentrant.

It seems to me that it depends on whether you think of snapshots as a 
system admin sort of facility or as a user facility.  As a system admin 
facility, you're probably right.  But as a user level facility, I want 
to be able to snapshot before making a change to a tree full of source 
code and (re)building it all over again.  I may want to keep my new 
build, but I may want to flush it and return to known good state.  It's 
pretty easy to open that facility up to non-root users but the easiest 
way to do that that I've found is to use a single file system on root 
mounted directly.  For an individual user, this can easily save hours 
and hundreds of gigabytes.  For automated build systems, it can mean a 
few orders of magnitude difference in typical build times.

It's not clear to me yet how to set machines up this way from kickstart, 
which makes this scheme look like an impediment, rather than a feature.  
But maybe all I need is an easy way to shut it off and get the more 
familiar arrangement.

--rich

Re: syntax for deleting subvolumes?

[Chris Murphy]

On Wed, Mar 18, 2015 at 5:01 PM, K Richard Pixley
<rpixley@graphitesystems.com> wrote:

> My complaint is a) about multiple subvols and b) about an unnecessary and
> redundant subvol for the top level file system.

The current granularity supplied by root and home subvolumes is minor.
Eventually there'd also be a boot subvolume too, but that's not
supported yet due to a very old grubby bug that's like a booger that
can't be flicked off.

openSUSE uses ~13 subvolumes by default, for an idea of much finer
granularity (which I don't like, personally, it's too much and really
is unnecessary).

There's rarely a use case where you want to rollback root and home at
the same time anyway, therefore making them separate subvolumes makes
sense. There could be an argument that /boot can be a directory rather
than a subvolume since it probably shouldn't get separated from its
root anyway due to kernel+kernel-module pairing and bootloader stuff.

A separate home subvolume is consistent with the RHEL/CentOS/Fedora
installer (Anaconda) default of putting home on a separate volume. It
makes home easier to backup, restore, migrate, and upgrade systems.
Btrfs support is uniquely handled by Anaconda too, in that it doesn't
require Btrfs volumes be reformat to do a clean install like for other
file systems. It does require a new subvolume, however, for the
"rootxx" subvolume.


> The redundancy should be obvious.  And the problem with multiple subvols is
> that they aren't needed and make snapshotting an entire system more
> difficult than it needs to be.  I want to be able to snapshot and chroot.
> The traditional reason for partitioning a disk was to isolate errors and to
> firewall "out of space" usage.  Neither of those really apply to btrfs as
> far as I can see.

It's the same on Btrfs in that regard as any other. Some people prefer
partitions, others prefer quotas.


> I would much rather the installer simply create a single file system and
> mount the partition directly rather than through a superfluous subvol.  I've
> never seen it done any other way as you might guess from my confusion above.

The confusion is understandable. But there's an express intent behind
the layout. Maybe the installer should include a brief explanatory
readme.txt somewhere? Or maybe integrate it as comments in /etc/fstab?

-- 
Chris Murphy

Re: syntax for deleting subvolumes?

[Chris Murphy]

On Wed, Mar 18, 2015 at 5:22 PM, K Richard Pixley
<rpixley@graphitesystems.com> wrote:
> Most of the uses I have for btrfs involve fairly dynamic use of snapshots,
> typically by non-root users. That's what brought me to btrfs in the first
> place and continues to be the biggest driver for me.

By default, none privileged users can create subvolumes and snapshots,
but they can't delete them. User deletion of subvolumes requires a
mount option to enable. I'm not sure if this mount option exposes all
subvolumes to none privileged users' ability to delete. Eventually, I
think the idea is to make subvolumes behave even more like
directories. Right now that's largely the case but the line needs to
be blurred a bit more.


> Because of this, the top level file system would need to be mounted pretty
> much constantly, which essentially removes any benefit from the redundant
> top level subvol.  It's just a nuisance for my applications.  And most of my
> applications try very hard to avoid mounting the snapshots.  That takes too
> much time and isn't reentrant.

User subvolumes/snapshots nested under the home subvolume aren't being
proscribed. It's just that doing this out of the gate with an
installation could lead to unexpected results.

Realize that snapshots are non-recursive for any nested
subvol/snapshots. So by virtue of your users creating subvols or
snapshots in their own home directory, if you snapshot home, *none* of
their subvolumes (including their snapshots) will be snapshot. And
that means they're excluded from btrfs send/receive as well. So long
as you and users know this, it's fine to nest them.


>
> It seems to me that it depends on whether you think of snapshots as a system
> admin sort of facility or as a user facility.  As a system admin facility,
> you're probably right.  But as a user level facility, I want to be able to
> snapshot before making a change to a tree full of source code and
> (re)building it all over again.  I may want to keep my new build, but I may
> want to flush it and return to known good state.  It's pretty easy to open
> that facility up to non-root users but the easiest way to do that that I've
> found is to use a single file system on root mounted directly.  For an
> individual user, this can easily save hours and hundreds of gigabytes.  For
> automated build systems, it can mean a few orders of magnitude difference in
> typical build times.

Sure. The near equivalent to a btrfs subvolume is a new file system,
and creating new file systems is not typically a user facility. A
btrfs subvolume is a separate fs tree, not a completely separate file
system, so it's sorta something in between that of a directory and a
new separate file system. For example inode counts start over. So two
different files in two subvolumes can have the same inode number.

So I'm not disputing the use case you've come up with, just that it's
going to take some consideration of all the myriad consequences of the
current state.

And for that matter, definitely don't depend on EL kernels for serious
Btrfs use. Btrfs is a tech preview. The latest features and bug fixes
will be in much newer stable kernels, and there's been a metric
shitton of changes since 3.10.


> It's not clear to me yet how to set machines up this way from kickstart,
> which makes this scheme look like an impediment, rather than a feature.  But
> maybe all I need is an easy way to shut it off and get the more familiar
> arrangement.

I'm fairly certain Anaconda considers any defined mount point to be
cause to create a subvolume. Just like for on LVM, a specific mount
point causes an LV to be created; and for Standard Partitions each
mount point is a separate partition.



-- 
Chris Murphy

Re: syntax for deleting subvolumes?

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 3/18/15 7:33 PM, Chris Murphy wrote:
> On Wed, Mar 18, 2015 at 5:01 PM, K Richard Pixley 
> <rpixley@graphitesystems.com> wrote:
> 
>> My complaint is a) about multiple subvols and b) about an
>> unnecessary and redundant subvol for the top level file system.
> 
> The current granularity supplied by root and home subvolumes is
> minor. Eventually there'd also be a boot subvolume too, but that's
> not supported yet due to a very old grubby bug that's like a booger
> that can't be flicked off.
> 
> openSUSE uses ~13 subvolumes by default, for an idea of much finer 
> granularity (which I don't like, personally, it's too much and
> really is unnecessary).

We use this layout in SLES too and it's necessary for both compliance
and principle-of-lease-surprise purposes in concert with our
snapshot-rollback facility. If you roll back your root file system,
would you really expect to lose all your logs? How about your mail? Do
you want snapshots of /tmp to hang around eating up disk space,
causing you to reduce the number of snapshots you can retain? Keeping
these as separate subvolumes also allows us to mount them seamlessly
when you boot from a snapshot to either recover or reset your system.

Does it look nice and neat? That's a matter of style. But the reasons
for having separate subvolumes are well thought out and completely
necessary.

BTW, going back to the original issue of why you'd even have a
separate non-top-level subvolume for root - if you boot from a
snapshot and choose to continue from there, you wouldn't be able to
remove the old "root" if it weren't a separate subvolume.

- -Jeff

- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQIcBAEBAgAGBQJVChyhAAoJEB57S2MheeWyQoAQAKVJmrh+BA8MBApVqmpOADUi
o7RvsSx0D/2lReqep9p2zCyLq0RCorClP7ESKqu+VXCl4gLkZmLszr1G5btTthk8
524vN0ErR5vb2L7hbZ/Rexkd5+H3CRmKsgLPeHeO8ladx57k1lF3bDEX4J6ssfd/
4MbIhZ/B3Xu8kxe7Te69T/86kL8A0teVesBARRlxewtm1AzMn6ABXVrAIWhJU9rk
vOe+8L/kMXTqOod0cv9wQNKWu5y587dHrrjGsTQ9qcrdTrudbFq0GKzO5gDaFTm+
RZ8LWucjMKdTDzLVxdmt0KxcBCS/fLTWjSoKeVjHk3If/ciFRYM7DZeCxKsmLU1j
5Fwtlm3VjcquzLRyAZST3buJ0atuP9w1Onu2LNKn4xymQlLvoF2U/Fp/gerNiDxe
ZxVBA6JuwG8PktAFqKNYZC9Wd3+BBmYUTyV7JXM5qXbuXZ8uN1O0e0I3XcN3zgh/
56FPQ7cKIe9AxCjxJzlho8ZDQ8kpYB1p+NiGAs7qiCIfsNKwH3ZfuwoBizdagM8M
KmFUVLC/y7b9vVQW5bQX+4NPJU9dmAYe0/dVD68ZK6ROF4b+pV5baP+qlFu8Hp1+
8DFxlG3wJ51WSoZuIvAWV+9y0drhamOONAQ0D1smddnbWvG1HoiL+Z4v36iYc9+3
kGXR3LtcuNWaV5rCjZ3i
=dToy
-----END PGP SIGNATURE-----

Re: syntax for deleting subvolumes?

[Chris Murphy]

On Wed, Mar 18, 2015 at 6:47 PM, Jeff Mahoney <jeffm@suse.com> wrote:
> We use this layout in SLES too and it's necessary for both compliance
> and principle-of-lease-surprise purposes in concert with our
> snapshot-rollback facility.

I understand the logic, I'm just not convinced it's the only way to
achieve that. It's kinda like 28 GPT partitions on the typical
Android/Cyanogen mobile phone — as long as everything works and you
don't interact with it, it's fine. But as soon as you need to
understand it, I think this layout and what's going on is non-obvious.

After a clean installation, and updates, it looks like this:

# btrfs sub list /
ID 257 gen 36 top level 5 path boot/grub2/i386-pc
ID 258 gen 36 top level 5 path boot/grub2/x86_64-efi
ID 259 gen 24 top level 5 path opt
ID 260 gen 36 top level 5 path srv
ID 261 gen 101 top level 5 path tmp
ID 262 gen 36 top level 5 path usr/local
ID 263 gen 36 top level 5 path var/crash
ID 264 gen 36 top level 5 path var/lib/mailman
ID 265 gen 36 top level 5 path var/lib/named
ID 266 gen 36 top level 5 path var/lib/pgsql
ID 267 gen 101 top level 5 path var/log
ID 268 gen 36 top level 5 path var/opt
ID 269 gen 101 top level 5 path var/spool
ID 270 gen 101 top level 5 path var/tmp
ID 275 gen 101 top level 5 path .snapshots
ID 276 gen 38 top level 275 path .snapshots/1/snapshot
ID 277 gen 41 top level 275 path .snapshots/2/snapshot
ID 278 gen 43 top level 275 path .snapshots/3/snapshot
ID 279 gen 46 top level 275 path .snapshots/4/snapshot
ID 282 gen 81 top level 275 path .snapshots/5/snapshot
ID 283 gen 97 top level 275 path .snapshots/6/snapshot

I reckognize that these subvolumes exist to *exclude* them from the
snapshotting policies, rather than to include them. That's part of the
typical user confusion, since subvolumes are associated with making
snapshots rather than avoiding them.

Plus it also confuses the hell out of the RH/Fedora installer which
ends up seeing every snapshot as a completely separate unique openSUSE
installation - because the distros haven't had any sort of
conversation about interoperability in this brave new world.
https://bugzilla.redhat.com/attachment.cgi?id=983116



> If you roll back your root file system,
> would you really expect to lose all your logs? How about your mail?

Of course not. I just disagree that there's only one way to manage this.

For the most part the "OS" is in /usr, and users' stuff is in /home,
and then there's everything else which includes /etc, /var, and /boot
being the things that change state the most at a system level. I would
like to see applications have an explicit home. This is a huge problem
on linux as they spread themselves throughout the filesystem. And
mobile platforms, apps are in a separate place, on Linux
desktop/server I should be able to revert applications separate from
an OS update, those two things aren't the same thing.

I'd also prefer seeing updates/upgrades handled differently. Create
the proper snapshots, upgrade the snapshots in a container, boot the
upgraded snapshot in a container to test its viability, and if
successful then make it a new boot option at next boot; if it fails
then obliterate the snapshot. This makes for atomic upgrades that
aren't yanking or modifying files out from running processes, and
permits indefinitely delayed reboots.

And eventually I really want to get to stateless systems on Linux.
Mobile platforms have surpassed desktop/server Linux in this regard.
Even Windows 8+ have surpassed it, with system refresh and reset
options.

I'm all for refining the granularity, but I think the subvolume as
snapshot exclusion is messy and confusing. I have to use snapper to
figure out the relationship of anything in .snapshot. And anything in
.snapshot is accessible from the normally mounted fs hierarchy. AND
when doing backups I have to make really really sure that I exclude
.snapshot or I've got a very good chance of encountering an ENOSPC on
the backup target due to what will appear to be dozens or hundreds of
OS's - the deduplication aspect of those snapshots is lost


>Do
> you want snapshots of /tmp to hang around eating up disk space,
> causing you to reduce the number of snapshots you can retain?

Well I put /tmp on tmpfs so I don't worry about that.

> Keeping
> these as separate subvolumes also allows us to mount them seamlessly
> when you boot from a snapshot to either recover or reset your system.
>
> Does it look nice and neat? That's a matter of style. But the reasons
> for having separate subvolumes are well thought out and completely
> necessary.

I understand the logic, I just happen to disagree this is the only way
to achieve the desired outcome. And I also think it's possible to come
up with one layout that is compatible with both snapshot+rollback, and
stateless systems paradigms.

-- 
Chris Murphy

Re: syntax for deleting subvolumes?

[Erkki Seppala]

K Richard Pixley <rpixley@graphitesystems.com> writes:

> But as a user level facility, I want to be able to snapshot before
> making a change to a tree full of source code and (re)building it all
> over again.  I may want to keep my new build, but I may want to flush
> it and return to known good state.

You may want to check out cp --reflink=always (different from cp
--link), which creates copy-on-write copy of the data. It isn't quite as
fast as snapshots to create, but it's still plenty fast and without the
downsides of subvolumes.

-- 
  _____________________________________________________________________
     / __// /__ ____  __               http://www.modeemi.fi/~flux/\   \
    / /_ / // // /\ \/ /                                            \  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi                                  \/

Re: syntax for deleting subvolumes?

[Chris Murphy]

On Wed, Mar 18, 2015 at 5:22 PM, K Richard Pixley
<rpixley@graphitesystems.com> wrote:
> Most of the uses I have for btrfs involve fairly dynamic use of snapshots,
> typically by non-root users.

Another thing. Some distros behave this way:

chris@linux-6gc0:~> btrfs sub list /
Absolute path to 'btrfs' is '/usr/sbin/btrfs', so running it may
require superuser privileges (eg. root).

As is the case on openSUSE. On Fedora, any user can use the btrfs
command, without sudo, and even without being in group wheel. Kindof a
stark difference, I'm vaguely curious about the security implications.
So this normal/non-admin user I'm logged in right now can create
subvolumes and snapshots at will, but I can't delete them (as
previously mentioned).

Erkki's cp --reflink idea is a good one. I've often wondered if it's a
good idea, and possible, to eventually make --reflink the default
behavior with Btrfs copies (I think some things probably first need
enhancement like the cross subvolume stuff, because otherwise cp could
fail inexplicably).

-- 
Chris Murphy

Re: syntax for deleting subvolumes?

[Duncan]

Chris Murphy posted on Thu, 19 Mar 2015 13:09:06 -0600 as excerpted:

> Erkki's cp --reflink idea is a good one. I've often wondered if it's a
> good idea, and possible, to eventually make --reflink the default
> behavior with Btrfs copies (I think some things probably first need
> enhancement like the cross subvolume stuff, because otherwise cp could
> fail inexplicably).

If cp --reflink=auto was the default, it'd "just work", making a reflink 
where possible, falling back to a normal copy where not possible to 
reflink.

However, I'd be wary of such a change, because admins are used to cp 
creating a separate copy which may well be intended as a backup, guarding 
against I/O errors on the original.  Btrfs does do checksum checking to 
ensure validity, but if that fails, do you really want it failing for 
BOTH copies, including the one the admin made specifically to AVOID such 
a problem, because he knows the file is likely to be open at the time of 
a crash or some such and did a cp to have a backup, just in case?  I know 
as an admin, *I* wouldn't like such behavior.  Far better to simply have 
the option there should I deliberately invoke it, but not invoke it by 
default, in ordered to avoid unhappy surprises!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: syntax for deleting subvolumes?

[Russell Coker]

On Fri, 20 Mar 2015 04:18:38 AM Duncan wrote:
> If cp --reflink=auto was the default, it'd "just work", making a reflink 
> where possible, falling back to a normal copy where not possible to 
> reflink.
> 
> However, I'd be wary of such a change, because admins are used to cp 
> creating a separate copy which may well be intended as a backup, guarding 
> against I/O errors on the original.  Btrfs does do checksum checking to 
> ensure validity, but if that fails, do you really want it failing for 
> BOTH copies, including the one the admin made specifically to AVOID such 

If you have a second copy on the same filesystem then an error on any of the 
metadata that is a parent will corrupt both.  EG if you have 2 copies under 
your home directory then a directory error for /, /home, or /home/$USER will 
make both copies unavailable.

Also any errors on superblocks or other essential filesystem metadata risks 
losing both copies.

If BTRFS was to adopt something equivalent to the ZFS "copies=" feature then 
the sysadmin could specify that certain subtrees would have extra copies of 
data AND each metadata block would have 1 more copy than the data blocks it 
refers to.

In conclusion I think that the ZFS copies= feature is the correct solution to 
this problem.  Until/unless the BFTRFS developers copy that design concept the 
thing to do if you want backups on the same storage hardware is to copy the 
data to a filesystem on a different partition - the NetApp research shows that 
disk read errors tend to be correlated by location on disk.

-- 
My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/

Re: syntax for deleting subvolumes?

[Russell Coker]

On Thu, 19 Mar 2015 07:18:29 AM Erkki Seppala wrote:
> > But as a user level facility, I want to be able to snapshot before
> > making a change to a tree full of source code and (re)building it all
> > over again.  I may want to keep my new build, but I may want to flush
> > it and return to known good state.
> 
> You may want to check out cp --reflink=always (different from cp
> --link), which creates copy-on-write copy of the data. It isn't quite as
> fast as snapshots to create, but it's still plenty fast and without the
> downsides of subvolumes.

In which situations is --reflink=always expected to fail on BTRFS?

I recently needed to move /home on one system to a subvol (instead of being in 
the root subvol) and used cp --reflink=always which gave a few errors when 
running with BTRFS on kernel 3.16.7-ckt7-1.  I was in a hurry and didn't try 
to track down the caue, I just ran it again with --reflink=auto which gave no 
errors and completed very quickly (obviously the files which couldn't be 
reflinked were small).

-- 
My Main Blog         http://etbe.coker.com.au/
My Documents Blog    http://doc.coker.com.au/