btrfs subvolume clone or fork (btrfs-progs feature request)

btrfs subvolume clone or fork (btrfs-progs feature request)

[james harvey]

Request for new btrfs subvolume subcommand:

clone or fork [-i <qgroupid] <source> [<dest>]<name>
   Create a subvolume <name> in <dest>, which is a clone or fork of source.
   If <dest> is not given, subvolume <name> will be created in the
current directory.
   Options
   -i <qgroupid>
      Add the newly created subvolume to a qgroup.  This option can be
given multiple times.

Would (I think):
* btrfs subvolume create <dest-subvolume>
* cp -ax --reflink=always <source-subvolume>/* <dest-subvolume>/


Particularly useful to avoid "Invalid cross-device link" when using cp
-ax, when source and dest are seen as different devices.

Such as if there is a top-level subvolume mounted as /, with others
mounted elsewhere.  Required workaround is to mount with subvolid=0 to
/mnt, cp -ax --reflink=always /mnt/...source /mnt/...dest

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Fajar A. Nugraha]

On Thu, Jul 9, 2015 at 8:20 AM, james harvey <jamespharvey20@gmail.com> wrote:
> Request for new btrfs subvolume subcommand:
>
> clone or fork [-i <qgroupid] <source> [<dest>]<name>
>    Create a subvolume <name> in <dest>, which is a clone or fork of source.
>    If <dest> is not given, subvolume <name> will be created in the
> current directory.
>    Options
>    -i <qgroupid>
>       Add the newly created subvolume to a qgroup.  This option can be
> given multiple times.
>
> Would (I think):
> * btrfs subvolume create <dest-subvolume>
> * cp -ax --reflink=always <source-subvolume>/* <dest-subvolume>/

What's wrong with "btrfs subvolume snapshot"?

-- 
Fajar

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

On 2015-07-09 02:22, Fajar A. Nugraha wrote:
> On Thu, Jul 9, 2015 at 8:20 AM, james harvey <jamespharvey20@gmail.com> wrote:
>> Request for new btrfs subvolume subcommand:
>>
>> clone or fork [-i <qgroupid] <source> [<dest>]<name>
>>     Create a subvolume <name> in <dest>, which is a clone or fork of source.
>>     If <dest> is not given, subvolume <name> will be created in the
>> current directory.
>>     Options
>>     -i <qgroupid>
>>        Add the newly created subvolume to a qgroup.  This option can be
>> given multiple times.
>>
>> Would (I think):
>> * btrfs subvolume create <dest-subvolume>
>> * cp -ax --reflink=always <source-subvolume>/* <dest-subvolume>/
>
> What's wrong with "btrfs subvolume snapshot"?
>
Well, personally I would say the fact that once something is tagged as a 
snapshot, you can't change it to a regular subvolume without doing a 
non-incremental send/receive.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Sander]

Austin S Hemmelgarn wrote (ao):
> >What's wrong with "btrfs subvolume snapshot"?
>
> Well, personally I would say the fact that once something is tagged as
> a snapshot, you can't change it to a regular subvolume without doing a
> non-incremental send/receive.

A snapshot is a subvolume. There is no such thing as "tagged as a
snapshot".

	Sander

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]

On 2015-07-09 08:41, Sander wrote:
> Austin S Hemmelgarn wrote (ao):
>>> What's wrong with "btrfs subvolume snapshot"?
>>
>> Well, personally I would say the fact that once something is tagged as
>> a snapshot, you can't change it to a regular subvolume without doing a
>> non-incremental send/receive.
>
> A snapshot is a subvolume. There is no such thing as "tagged as a
> snapshot".
>
> 	Sander
>
No, there is a bit in the subvolume metadata that says whether it's 
considered a snapshot or not.  Internally, they are handled identically, 
but it does come into play when you consider things like btrfs subvolume 
show -s (which only lists snapshots), which in turn means that certain 
tasks are more difficult to script robustly.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Sander]

Austin S Hemmelgarn wrote (ao):
> On 2015-07-09 08:41, Sander wrote:
> >Austin S Hemmelgarn wrote (ao):
> >>>What's wrong with "btrfs subvolume snapshot"?
> >>
> >>Well, personally I would say the fact that once something is tagged as
> >>a snapshot, you can't change it to a regular subvolume without doing a
> >>non-incremental send/receive.
> >
> >A snapshot is a subvolume. There is no such thing as "tagged as a
> >snapshot".
> >
> >	Sander
> >
> No, there is a bit in the subvolume metadata that says whether it's
> considered a snapshot or not.  Internally, they are handled identically, but
> it does come into play when you consider things like btrfs subvolume show -s
> (which only lists snapshots), which in turn means that certain tasks are
> more difficult to script robustly.

I stand corrected. Thanks for the info.

	Sander

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Duncan]

Austin S Hemmelgarn posted on Thu, 09 Jul 2015 08:48:00 -0400 as
excerpted:

> On 2015-07-09 08:41, Sander wrote:
>> Austin S Hemmelgarn wrote (ao):
>>>> What's wrong with "btrfs subvolume snapshot"?
>>>
>>> Well, personally I would say the fact that once something is tagged as
>>> a snapshot, you can't change it to a regular subvolume without doing a
>>> non-incremental send/receive.
>>
>> A snapshot is a subvolume. There is no such thing as "tagged as a
>> snapshot".
>>
>> 	Sander
>>
> No, there is a bit in the subvolume metadata that says whether it's
> considered a snapshot or not.  Internally, they are handled identically,
> but it does come into play when you consider things like btrfs subvolume
> show -s (which only lists snapshots), which in turn means that certain
> tasks are more difficult to script robustly.

My use-case doesn't involve subvolumes or snapshots so I can't check for 
sure, but...

I could have sworn btrfs property -t subvolume can get/set that snapshot 
bit.  I know I saw the discussion and I think patch for it go by, but 
again, as I don't use them, I haven't tracked closely enough to see if it 
ever got in.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

On Thu, Jul 09, 2015 at 01:43:53PM +0000, Duncan wrote:
> I could have sworn btrfs property -t subvolume can get/set that snapshot 
> bit.  I know I saw the discussion and I think patch for it go by, but 
> again, as I don't use them, I haven't tracked closely enough to see if it 
> ever got in.

   Are you thinking of the read-only flag? That's not the same thing
as the various UUID properties (e.g. parent) which can be used to
detemine if a subvolume was made using a snapshot.

   Hugo.

-- 
Hugo Mills             | Someone's been throwing dead sheep down my Fun Well
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                          Nick Gibbins

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Duncan]

Hugo Mills posted on Thu, 09 Jul 2015 13:54:48 +0000 as excerpted:

> On Thu, Jul 09, 2015 at 01:43:53PM +0000, Duncan wrote:
>> I could have sworn btrfs property -t subvolume can get/set that
>> snapshot bit.  I know I saw the discussion and I think patch for it go
>> by, but again, as I don't use them, I haven't tracked closely enough to
>> see if it ever got in.
> 
> Are you thinking of the read-only flag? That's not the same thing
> as the various UUID properties (e.g. parent) which can be used to
> detemine if a subvolume was made using a snapshot.

Perhaps, but I was sure there was a snapshot property too, because I 
remember discussion of being able to unset it in ordered to remove it 
from the snapshot (only) list.

But maybe that's all it was, discussion, it wasn't implemented, and I 
ended up conflating it with the read-only bit, which /can/ be set/unset 
that way.  Like I said I can't check as I don't have any subvolumes/
snapshots available to do a listing on and see, and the property manpage 
doesn't have a properties list to check on, it wants you to use the list 
option to get the list.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

On Thu, 09 Jul 2015 08:48:00 -0400
Austin S Hemmelgarn <ahferroin7@gmail.com> wrote:

> On 2015-07-09 08:41, Sander wrote:
> > Austin S Hemmelgarn wrote (ao):
> >>> What's wrong with "btrfs subvolume snapshot"?
> >>
> >> Well, personally I would say the fact that once something is tagged as
> >> a snapshot, you can't change it to a regular subvolume without doing a
> >> non-incremental send/receive.
> >
> > A snapshot is a subvolume. There is no such thing as "tagged as a
> > snapshot".
> >
> > 	Sander
> >
> No, there is a bit in the subvolume metadata that says whether it's 
> considered a snapshot or not.  Internally, they are handled identically, 
> but it does come into play when you consider things like btrfs subvolume 
> show -s (which only lists snapshots), which in turn means that certain 
> tasks are more difficult to script robustly.

This sounds like a vestigial leftover from back when snapshots were
conceptualized to be somehow functionally different from subvolumes... But as
you said, now there is effectively no difference, so that bit is used for
what, only to track how a subvolume was created? And to output in the
subvolume list if the user passes "-s"? I'd say that's a pretty oddball feature
to even have, since in any case if you want to distinguish and list only your
snapshots, you would typically just name them in a certain way, e.g.
"/snaps/originalname/<datetime>".

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[David Sterba]

On Thu, Jul 09, 2015 at 08:48:00AM -0400, Austin S Hemmelgarn wrote:
> On 2015-07-09 08:41, Sander wrote:
> > Austin S Hemmelgarn wrote (ao):
> >>> What's wrong with "btrfs subvolume snapshot"?
> >>
> >> Well, personally I would say the fact that once something is tagged as
> >> a snapshot, you can't change it to a regular subvolume without doing a
> >> non-incremental send/receive.
> >
> > A snapshot is a subvolume. There is no such thing as "tagged as a
> > snapshot".
> >
> No, there is a bit in the subvolume metadata that says whether it's 
> considered a snapshot or not.

Technically it's not really a bit. The snapshot relation is determined
by the parent uuid value of a subvolume.

> Internally, they are handled identically, 
> but it does come into play when you consider things like btrfs subvolume 
> show -s (which only lists snapshots),

That was probably 'btrfs subvol list -s', though the 'subvol show'
command prints all snapshots of a given subvolume.

> which in turn means that certain 
> tasks are more difficult to script robustly.

I don't deny the interface/output is imperfect for scripting purposes,
maybe we can provide filters that would satisfy your usecase.

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 1818 bytes --]

On 2015-07-09 14:33, David Sterba wrote:
> On Thu, Jul 09, 2015 at 08:48:00AM -0400, Austin S Hemmelgarn wrote:
>> On 2015-07-09 08:41, Sander wrote:
>>> Austin S Hemmelgarn wrote (ao):
>>>>> What's wrong with "btrfs subvolume snapshot"?
>>>>
>>>> Well, personally I would say the fact that once something is tagged as
>>>> a snapshot, you can't change it to a regular subvolume without doing a
>>>> non-incremental send/receive.
>>>
>>> A snapshot is a subvolume. There is no such thing as "tagged as a
>>> snapshot".
>>>
>> No, there is a bit in the subvolume metadata that says whether it's
>> considered a snapshot or not.
>
> Technically it's not really a bit. The snapshot relation is determined
> by the parent uuid value of a subvolume.
I'm actually kind of curious, is the parent UUID actually used for 
anything outside of send/receive?
>
>> Internally, they are handled identically,
>> but it does come into play when you consider things like btrfs subvolume
>> show -s (which only lists snapshots),
>
> That was probably 'btrfs subvol list -s', though the 'subvol show'
> command prints all snapshots of a given subvolume.
You're right, I just have a tendency to get the two confused because my 
workflow means that I don't use either very frequently.
>
>> which in turn means that certain
>> tasks are more difficult to script robustly.
>
> I don't deny the interface/output is imperfect for scripting purposes,
> maybe we can provide filters that would satisfy your usecase.
>
Personally, I don't really do much direct scripting of BTRFS related 
tasks (although that might change if I can convince my boss that we 
should move to BTRFS for our server systems).  Most of my complaint with 
the current arrangement is primarily aesthetic more than anything else.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[David Sterba]

On Fri, Jul 10, 2015 at 09:36:45AM -0400, Austin S Hemmelgarn wrote:
> > Technically it's not really a bit. The snapshot relation is determined
> > by the parent uuid value of a subvolume.
> I'm actually kind of curious, is the parent UUID actually used for 
> anything outside of send/receive?

AFAIK no.

> >> which in turn means that certain
> >> tasks are more difficult to script robustly.
> >
> > I don't deny the interface/output is imperfect for scripting purposes,
> > maybe we can provide filters that would satisfy your usecase.
> >
> Personally, I don't really do much direct scripting of BTRFS related 
> tasks (although that might change if I can convince my boss that we 
> should move to BTRFS for our server systems).  Most of my complaint with 
> the current arrangement is primarily aesthetic more than anything else.

Ok understood, thanks.

Re: btrfs subvolume clone or fork (btrfs-progs feature request)

[David Sterba]

On Wed, Jul 08, 2015 at 09:20:46PM -0400, james harvey wrote:
> Would (I think):
> * btrfs subvolume create <dest-subvolume>
> * cp -ax --reflink=always <source-subvolume>/* <dest-subvolume>/
> 
> Particularly useful to avoid "Invalid cross-device link" when using cp
> -ax, when source and dest are seen as different devices.
> 
> Such as if there is a top-level subvolume mounted as /, with others
> mounted elsewhere.  Required workaround is to mount with subvolid=0 to
> /mnt, cp -ax --reflink=always /mnt/...source /mnt/...dest

This is a known limitation of reflinks and intentional, ie. no
cross-mountpoint operations are allowed to a single filesystem. The same
holds for hardlinks and the reflink duplicates the restrictions.

btrfs subvolume clone or fork (btrfs-progs feature request)

[james harvey]