* Snapshots – noob questions
@ 2014-01-27 13:53 KC
2014-01-27 14:03 ` Hugo Mills
2014-01-27 18:44 ` Chris Murphy
0 siblings, 2 replies; 14+ messages in thread
From: KC @ 2014-01-27 13:53 UTC (permalink / raw)
To: linux-btrfs
I have been trying to understand how snapshots work (in BRTFS and in
general), but I still have some questions, and would appreciate if
someone could clear them for me.
To make things easier, I tried to make most of them Yes/No questions:
1. When creating the filesystem, I only made btrfs partition with no
subvolumes. Do I have to use subvolumes, or can I snapshot entire “/“?
2. Can I take a snapshot of a working / root filesystem?
3. If I make a snapshot of / and there are some separate partitions
mounted under /mnt/ or /home/, will snapshot skip them?
4. If I have a snapshot of /, can I completely erase this partition and
later restore it in full form that snapshot, or do snapshots work only
if a limited number of files has been changed?
If the former, then does it mean that snapshot size will be comparable
to the original data size?
5. Can a snapshot be stored on a different partition? And if so, does
that partition have to be BTRFS too? I ask, because I would also want to
store snapshots in a cloud storage.
Thanks.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 13:53 Snapshots – noob questions KC
@ 2014-01-27 14:03 ` Hugo Mills
2014-01-27 14:14 ` Jim Salter
2014-01-27 18:35 ` Chris Murphy
2014-01-27 18:44 ` Chris Murphy
1 sibling, 2 replies; 14+ messages in thread
From: Hugo Mills @ 2014-01-27 14:03 UTC (permalink / raw)
To: KC; +Cc: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 3123 bytes --]
On Mon, Jan 27, 2014 at 02:53:36PM +0100, KC wrote:
> I have been trying to understand how snapshots work (in BRTFS and in
> general), but I still have some questions, and would appreciate if
> someone could clear them for me.
> To make things easier, I tried to make most of them Yes/No questions:
>
> 1. When creating the filesystem, I only made btrfs partition with no
> subvolumes. Do I have to use subvolumes, or can I snapshot entire
> “/“?
/ (the top level of the FS) is a subvolume as well -- it's the only
one that can't be renamed or deleted, though.
> 2. Can I take a snapshot of a working / root filesystem?
Yes.
> 3. If I make a snapshot of / and there are some separate partitions
> mounted under /mnt/ or /home/, will snapshot skip them?
Yes. Snapshots work on a single subvolume at a time.
> 4. If I have a snapshot of /, can I completely erase this partition
> and later restore it in full form that snapshot,
If you have your root filesystem as the top level of the btrfs
filesystem, then no, you can't. I would recommend using the filesystem
structure shown here
https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Managing_snapshots
because it makes making, managing and rolling back the snapshots much
easier.
> or do snapshots
> work only if a limited number of files has been changed?
Snapshots will continue to work regardless of how many files have
been changed. The only thing to note is that as you make more
modifications (either to the snapshot or the original, or both), the
amount of storage requires goes up. Note that once you've made a
snapshot, there's no distinction between the snapshot and the original
-- they're completely equal partners. This means that you can delete
either one and the other one will remain.
> If the former, then does it mean that snapshot size will be
> comparable to the original data size?
When you first make a snapshot, the total storage space required is
unchanged(*). As you make modifications, the two subvolumes will
diverge from each other, and more and more space is needed.
Eventually, when you've modified every file (on one side or the
other), you will end up needing enough space to store each one
separately. Basically, they share as many data extents as possible,
and use separate ones only when needed.
(*) OK, that's a lie -- you need an extra tree root for the snapshot,
but that's a whole 4k-16k, depending on the size of your tree nodes.
> 5. Can a snapshot be stored on a different partition? And if so,
> does that partition have to be BTRFS too? I ask, because I would
> also want to store snapshots in a cloud storage.
No, they can't.
Hugo.
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- For months now, we have been making triumphant retreats ---
before a demoralised enemy who is advancing
in utter disorder.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 14:03 ` Hugo Mills
@ 2014-01-27 14:14 ` Jim Salter
2014-01-27 18:35 ` Chris Murphy
1 sibling, 0 replies; 14+ messages in thread
From: Jim Salter @ 2014-01-27 14:14 UTC (permalink / raw)
To: Hugo Mills, KC, linux-btrfs
Well... you can't /create/ snapshots on different partitions / cloud
storage / whatever, but you can certainly /send/ them there once you've
created them.
Generally the best way to do this kind of thing is with incremental
replication of snapshots to another btrfs filesystem, which may be local
or cloud.
you@box1:~$ sudo btrfs send /.snapshots/snapshot1 | ssh you@box2
sudo btrfs receive /.snapshots
This sends the snapshot to the remote machine, where it will end up in
/.snapshots. This is a FULL send - you'll be sending a MB of data for
every MB stored on disk locally. But the next time:
you@box1:~$ sudo btrfs send -p /.snapshots/snapshot1
/.snapshots/snapshot2 | ssh you@box2 sudo btrfs receive /.snapshots
Assuming you still have a copy of snapshot1 on box2, the next time, you
can do an INCREMENTAL send as shown above. This tells btrfs send to use
snapshot1 as a "parent" - which means it only sends the blocks in
snapshot2 that are different from the blocks in snapshot1. For example,
this might be a filesystem with 500GB of data - but if you only changed
10MB of that data between snapshot1 and snapshot2, you only have to send
that 10MB of data to box2 with this incremental send.
Now you have snapshot1 and snapshot2 on both the local and the remote
machine - and, best part, they're deduplicated, meaning that even though
each snapshot is a fully browseable copy of your (example) 500GB of
data, they only require (example) 500.01GB of storage - the 500GB of
data they share in common, plus the 10MB of stuff that's different
between snapshot1 and snapshot2.
On 01/27/2014 09:03 AM, Hugo Mills wrote:
>> 5. Can a snapshot be stored on a different partition? And if so,
>> does that partition have to be BTRFS too? I ask, because I would
>> also want to store snapshots in a cloud storage.
> No, they can't.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 14:03 ` Hugo Mills
2014-01-27 14:14 ` Jim Salter
@ 2014-01-27 18:35 ` Chris Murphy
2014-01-27 18:42 ` Hugo Mills
1 sibling, 1 reply; 14+ messages in thread
From: Chris Murphy @ 2014-01-27 18:35 UTC (permalink / raw)
To: Hugo Mills; +Cc: KC, linux-btrfs
On Jan 27, 2014, at 7:03 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
> On Mon, Jan 27, 2014 at 02:53:36PM +0100, KC wrote:
>> I have been trying to understand how snapshots work (in BRTFS and in
>> general), but I still have some questions, and would appreciate if
>> someone could clear them for me.
>> To make things easier, I tried to make most of them Yes/No questions:
>>
>> 1. When creating the filesystem, I only made btrfs partition with no
>> subvolumes. Do I have to use subvolumes, or can I snapshot entire
>> “/“?
>
> / (the top level of the FS) is a subvolume as well -- it's the only
> one that can't be renamed or deleted, though.
I'll point out a source of confusion I've had, and see new users have the same confusion quite often also. Within the file system hierarchy, / is a mount point. It could be that subvolume ID 5 is mounted there, which is how opensuse's default rootfs on Btrfs works. However, Fedora's / is a subvolume named root, the top level ID 5 subvolume isn't mounted by default.
So / is not necessarily the top level of the file system. I'm not sure what to call the undeleteable ID 5 subvolume. It's the default subvolume, but if the user changes the default subvolume then it isn't. Calling it the default default is maybe more confusing. And calling it subvolume ID 5 is a bit confusing because subvolume ID 0 maps to 5, and many users are more familiar with subvol ID 0 than ID 5. And then if I have a subvolume within a subvolume mounted and do a btrfs list, it shows top level ID with a value other than 5.
Anyway I really don't know what else to call the undeletable first subvolume.
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 18:35 ` Chris Murphy
@ 2014-01-27 18:42 ` Hugo Mills
0 siblings, 0 replies; 14+ messages in thread
From: Hugo Mills @ 2014-01-27 18:42 UTC (permalink / raw)
To: Chris Murphy; +Cc: KC, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 2641 bytes --]
On Mon, Jan 27, 2014 at 11:35:03AM -0700, Chris Murphy wrote:
>
> On Jan 27, 2014, at 7:03 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
>
> > On Mon, Jan 27, 2014 at 02:53:36PM +0100, KC wrote:
> >> I have been trying to understand how snapshots work (in BRTFS and in
> >> general), but I still have some questions, and would appreciate if
> >> someone could clear them for me.
> >> To make things easier, I tried to make most of them Yes/No questions:
> >>
> >> 1. When creating the filesystem, I only made btrfs partition with no
> >> subvolumes. Do I have to use subvolumes, or can I snapshot entire
> >> “/“?
> >
> > / (the top level of the FS) is a subvolume as well -- it's the only
> > one that can't be renamed or deleted, though.
>
>
> I'll point out a source of confusion I've had, and see new users have the same confusion quite often also. Within the file system hierarchy, / is a mount point. It could be that subvolume ID 5 is mounted there, which is how opensuse's default rootfs on Btrfs works. However, Fedora's / is a subvolume named root, the top level ID 5 subvolume isn't mounted by default.
I agree the terminology gets confusing. I try to be consistent with
the set of words used in the SysadminGuide and (at least some of) the
rest of the website:
Subvolume id=5: "top level of the FS"
The top of the current visible UNIX filesystem structure: "/"
Root's home directory: "~root"
I always try to avoid using the word "root" for any of them, as
it's rather heavily overloaded at this point.
> So / is not necessarily the top level of the file system. I'm not sure what to call the undeleteable ID 5 subvolume. It's the default subvolume, but if the user changes the default subvolume then it isn't. Calling it the default default is maybe more confusing. And calling it subvolume ID 5 is a bit confusing because subvolume ID 0 maps to 5, and many users are more familiar with subvol ID 0 than ID 5. And then if I have a subvolume within a subvolume mounted and do a btrfs list, it shows top level ID with a value other than 5.
>
> Anyway I really don't know what else to call the undeletable first subvolume.
I call it "the top level of the btrfs filesystem", as that's what
it is. (Even if you don't always mount the top level of the FS).
Hugo.
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- I always felt that as a C programmer, I ---
was becoming typecast.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 13:53 Snapshots – noob questions KC
2014-01-27 14:03 ` Hugo Mills
@ 2014-01-27 18:44 ` Chris Murphy
2014-01-27 18:50 ` Hugo Mills
` (2 more replies)
1 sibling, 3 replies; 14+ messages in thread
From: Chris Murphy @ 2014-01-27 18:44 UTC (permalink / raw)
To: KC; +Cc: linux-btrfs
On Jan 27, 2014, at 6:53 AM, KC <impactoria@googlemail.com> wrote:
>
> 3. If I make a snapshot of / and there are some separate partitions mounted under /mnt/ or /home/, will snapshot skip them?
A snapshot of a subvolume only snapshots that subvolume, not other subvolumes contained in it or other partitions/file systems.
>
> 4. If I have a snapshot of /, can I completely erase this partition and later restore it in full form that snapshot, or do snapshots work only if a limited number of files has been changed?
> If the former, then does it mean that snapshot size will be comparable to the original data size?
I'm confused by what you mean by completely erase this partitions and later restore it. What do you mean by partition? Taken literally I'd say no because a snapshot is a subvolume, it's a function of a Btrfs file system, and if you erase the partition you've erased the file system and everything on it.
Otherwise, either the snapshot or its parent subvolume can be deleted at any time. At the moment the snapshot is taken, they are clones, but they behave as completely separate file systems. Any changes to one do not affect the other.
> 5. Can a snapshot be stored on a different partition?
Yes Btrfs send/receive.
> And if so, does that partition have to be BTRFS too?
You can send a snapshot with -f <filename> to a non-Btrfs file system. But it's a file. You can't interact with it, you can only restore it using btrfs receive, to another Btrfs file system.
> I ask, because I would also want to store snapshots in a cloud storage.
Yes although a possible drawback is that I think the resulting snapshot as a file, doesn't contain checksums. So silent data corruption detection isn't possible. Yes you can and should shasum the resulting file but if the sums don't match you don't necessarily know what files are affected.
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 18:44 ` Chris Murphy
@ 2014-01-27 18:50 ` Hugo Mills
2014-01-27 19:49 ` Chris Murphy
2014-01-28 18:39 ` Martin
[not found] ` <52E808CB.9020005@gmail.com>
2 siblings, 1 reply; 14+ messages in thread
From: Hugo Mills @ 2014-01-27 18:50 UTC (permalink / raw)
To: Chris Murphy; +Cc: KC, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1759 bytes --]
On Mon, Jan 27, 2014 at 11:44:24AM -0700, Chris Murphy wrote:
>
> On Jan 27, 2014, at 6:53 AM, KC <impactoria@googlemail.com> wrote:
> > 4. If I have a snapshot of /, can I completely erase this partition and later restore it in full form that snapshot, or do snapshots work only if a limited number of files has been changed?
> > If the former, then does it mean that snapshot size will be comparable to the original data size?
>
> I'm confused by what you mean by completely erase this partitions and later restore it. What do you mean by partition? Taken literally I'd say no because a snapshot is a subvolume, it's a function of a Btrfs file system, and if you erase the partition you've erased the file system and everything on it.
>
> Otherwise, either the snapshot or its parent subvolume can be deleted at any time. At the moment the snapshot is taken, they are clones, but they behave as completely separate file systems. Any changes to one do not affect the other.
Probably slightly more accurate to say "completely different file
trees", as they're part of the same filesystem (the single btrfs FS
that contains all of the subvolumes in this example).
> > 5. Can a snapshot be stored on a different partition?
>
> Yes Btrfs send/receive.
Mmmm... debatable semantics again. You can transfer the data with
btrfs send/receive, but after that it's not really a snapshot in the
sense of being a CoW copy on the same filesystem...
Hugo.
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- I always felt that as a C programmer, I ---
was becoming typecast.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 18:50 ` Hugo Mills
@ 2014-01-27 19:49 ` Chris Murphy
0 siblings, 0 replies; 14+ messages in thread
From: Chris Murphy @ 2014-01-27 19:49 UTC (permalink / raw)
To: Hugo Mills; +Cc: KC, linux-btrfs
On Jan 27, 2014, at 11:50 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
> On Mon, Jan 27, 2014 at 11:44:24AM -0700, Chris Murphy wrote:
>>
>> On Jan 27, 2014, at 6:53 AM, KC <impactoria@googlemail.com> wrote:
>>> 4. If I have a snapshot of /, can I completely erase this partition and later restore it in full form that snapshot, or do snapshots work only if a limited number of files has been changed?
>>> If the former, then does it mean that snapshot size will be comparable to the original data size?
>>
>> I'm confused by what you mean by completely erase this partitions and later restore it. What do you mean by partition? Taken literally I'd say no because a snapshot is a subvolume, it's a function of a Btrfs file system, and if you erase the partition you've erased the file system and everything on it.
>>
>> Otherwise, either the snapshot or its parent subvolume can be deleted at any time. At the moment the snapshot is taken, they are clones, but they behave as completely separate file systems. Any changes to one do not affect the other.
>
> Probably slightly more accurate to say "completely different file
> trees", as they're part of the same filesystem (the single btrfs FS
> that contains all of the subvolumes in this example).
Yes good point. File system = volume; file tree = subvolume.
>
>>> 5. Can a snapshot be stored on a different partition?
>>
>> Yes Btrfs send/receive.
>
> Mmmm... debatable semantics again. You can transfer the data with
> btrfs send/receive, but after that it's not really a snapshot in the
> sense of being a CoW copy on the same filesystem…
Ahh yes I was being very literal rather than thinking of a magical hard link that can work across file system boundaries.
Maybe the question needs to be phrased in terms of a use case question, and possibly a seed-device would fit?
https://btrfs.wiki.kernel.org/index.php/Seed-device
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-27 18:44 ` Chris Murphy
2014-01-27 18:50 ` Hugo Mills
@ 2014-01-28 18:39 ` Martin
2014-01-28 19:16 ` Hugo Mills
[not found] ` <52E808CB.9020005@gmail.com>
2 siblings, 1 reply; 14+ messages in thread
From: Martin @ 2014-01-28 18:39 UTC (permalink / raw)
To: linux-btrfs
On 27/01/14 18:44, Chris Murphy wrote:
>
> On Jan 27, 2014, at 6:53 AM, KC <impactoria@googlemail.com> wrote:
>>
>> 3. If I make a snapshot of / and there are some separate partitions mounted under /mnt/ or /home/, will snapshot skip them?
>
> A snapshot of a subvolume only snapshots that subvolume, not other subvolumes contained in it or other partitions/file systems.
Thanks. That's a critical detail to know.
A few follow-on questions... Are taking snapshots:
"atomic"?
Do they hold up or lock out other disk activity?
Is there an overhead to keeping many snapshots other than the storage
any changed files take?
Can we now consider making and deleting snapshots a "debugged stable
feature"?
Thanks,
Martin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-28 18:39 ` Martin
@ 2014-01-28 19:16 ` Hugo Mills
2014-01-29 1:21 ` Duncan
0 siblings, 1 reply; 14+ messages in thread
From: Hugo Mills @ 2014-01-28 19:16 UTC (permalink / raw)
To: Martin; +Cc: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]
On Tue, Jan 28, 2014 at 06:39:45PM +0000, Martin wrote:
> On 27/01/14 18:44, Chris Murphy wrote:
> >
> > On Jan 27, 2014, at 6:53 AM, KC <impactoria@googlemail.com> wrote:
> >>
> >> 3. If I make a snapshot of / and there are some separate partitions mounted under /mnt/ or /home/, will snapshot skip them?
> >
> > A snapshot of a subvolume only snapshots that subvolume, not other subvolumes contained in it or other partitions/file systems.
>
> Thanks. That's a critical detail to know.
>
>
> A few follow-on questions... Are taking snapshots:
>
>
> "atomic"?
Yes. The state of the snapshot is that of a single point in the
filesystem's history.
> Do they hold up or lock out other disk activity?
Yes, but only very briefly -- no more than any other simple
filesystem operation.
> Is there an overhead to keeping many snapshots other than the storage
> any changed files take?
The storage the changed metadata takes, but that's all.
> Can we now consider making and deleting snapshots a "debugged stable
> feature"?
I think so, yes, although there are still cases where having lots
of snapshots (thousands and upwards) can cause problems. I'm thinking
specifically of deletion of snapshots, where the system can bog down
quite heavily doing all the deletions.
Hugo.
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- Great oxymorons of the world, no. 10: Business Ethics ---
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
[not found] ` <52E808CB.9020005@gmail.com>
@ 2014-01-28 20:16 ` Chris Murphy
[not found] ` <52EABE29.6080703@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Chris Murphy @ 2014-01-28 20:16 UTC (permalink / raw)
To: KC, Btrfs BTRFS
On Jan 28, 2014, at 12:45 PM, KC <impactoria@googlemail.com> wrote:
> On 27/01/14 19:44, Chris Murphy wrote:
>>
>> On Jan 27, 2014, at 6:53 AM, KC <impactoria@googlemail.com> wrote:
>>>
>>> 3. If I make a snapshot of / and there are some separate partitions mounted under /mnt/ or /home/, will snapshot skip them?
>>
>> A snapshot of a subvolume only snapshots that subvolume, not other subvolumes contained in it or other partitions/file systems.
>>
>
> Excellent, thank you.
>
> If currently my /home is just a folder on the / subvolume, is there any way to promote existing /home to a subvolume status so that if I take a snapshot of /, the /home gets skipped?
Multiple ways. I think the easiest is for the subvolumes to be at the top level of the Btrfs volume, and mount them where you want with fstab. So for example on Fedora 20 my top level subvolumes are:
boot
root
home
root is mounted at / using mount option subvol=root
boot is mounted at /boot using mount option subvol=boot
home is mounted at /home using mount option subvol=home
I've also decided to put systemd-journals on a separate subvolume:
journald is mounted at /var/log/journal using mount option subvol=journald
That way I can snapshot root, and rollback but have a persistent journal rather than a snapshot specific journal. Of course /home is also persistent because I don't roll it back.
You could also rename /home, create a subvolume home within current subvolume root, then move the files from directory /home_old to the subvolume /home. Now when you snapshot rootfs, /home isn't snapshot. In this case you don't need to separately mount the home subvolume because it's available as if it were a directory. It's an organizational thing I don't know that there's a right or wrong here.
One thing that came up recently on the Fedora devel list though, with respect to roofs snapshots available in the mounted fs hierarchy, is that an old rootfs snapshot could contain vulnerabilities. So two ways to handle it are to keep snapshots in the top level of the fs outside of the mounted fs hierarchy (only mount it as needed when taking snapshots); and/or mount that snapshots subvolume with noexec or nosuid mount option. That way its executables aren't root executable.
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
2014-01-28 19:16 ` Hugo Mills
@ 2014-01-29 1:21 ` Duncan
0 siblings, 0 replies; 14+ messages in thread
From: Duncan @ 2014-01-29 1:21 UTC (permalink / raw)
To: linux-btrfs
Hugo Mills posted on Tue, 28 Jan 2014 19:16:16 +0000 as excerpted:
>> Can we now consider making and deleting snapshots a "debugged stable
>> feature"?
>
> I think so, yes, although there are still cases where having lots
> of snapshots (thousands and upwards) can cause problems. I'm thinking
> specifically of deletion of snapshots, where the system can bog down
> quite heavily doing all the deletions.
What about the recent pathologic large (half-gig-plus) frequently-
internally-rewritten and thus without NOCOW highly fragmented file case,
made pathologic (even on SSD, it's a CPU not an IO bottleneck) by
thousands of snapshots?
To my knowledge that's speculatively pinned down to the not all that long
ago snapshot-aware-defrag code (which itself was in response to the
problem of defrag not being snapshot aware before that), but while I'm
not a dev and thus don't/can't-easily follow developing patches in code-
level detail (tho I routinely read both intros and followups), I'm not
aware of pending patches that address that yet.
Altho recommended procedure is to NOCOW such files at creation (generally
by setting NOCOW on the directory they're to be placed in, before the
file is written at all), based on reports here, quite a few people get
caught in the trap of not knowing about that until it's too late, and
then are stuck with a file that it can literally take *DAYS* to do
anything with due to the pathelogic lack of btrfs ability to scale or
make pretty much any progress at all in attempting to move/defrag/delete
such files (altho I believe they can be renamed in-place on the same
snapshot).
That said, in general I'd agree, snapshots can practically be considered
/reasonably/ debugged-stable, as long as you're doing reasonable snapshot
management and don't get thousands (and preferably not too many hundreds)
of them to deal with on the same filesystem.
An automated script that say takes a snapshot a minute, but keeps only
say 30 one-minute snapshots, keeping a snapshot every half hour for a day
(47, 30+47=77), keeping a snapshot every hour for another two days (48,
total three days covered, 77+48=125), keeping a snapshot every 6 hours to
fill out the week ((7-3)*4=16, 125+16=141), keeping a snapshot a day to
fill out the month (31-7=24, 142+24=165), keeping a snapshot a week to
fill out the year (52-4=48, 166+48=213), and finally keeping a snapshot a
quarter (13 weeks) out to say a decade (40-4=36, 213+36=249), if the
filesystem remains in use that long... should be reasonable, and assuming
the math is correct, that's under 250 snapshots total, with coverage of a
snapshot a minute out to a half hour, a snapshot a half-hour out to a
day, a snapshot a day out to a month, a snapshot a week out to a year,
and a snapshot a quarter out to a decade (!!). If you need history back
more than a decade and you don't have some sort of longer-term storage
archiving involved... well let's just say I seriously doubt the quality
of your decisions in that case.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
[not found] ` <52EABE29.6080703@gmail.com>
@ 2014-01-31 4:05 ` Chris Murphy
2014-01-31 4:19 ` Chris Murphy
1 sibling, 0 replies; 14+ messages in thread
From: Chris Murphy @ 2014-01-31 4:05 UTC (permalink / raw)
To: KC; +Cc: Btrfs BTRFS
readded list
On Jan 30, 2014, at 2:03 PM, KC <impactoria@googlemail.com> wrote:
> On 28/01/14 21:16, Chris Murphy wrote:
>> You could also rename /home, create a subvolume home within current subvolume root, then move the files from directory /home_old to the subvolume /home. Now when you snapshot rootfs, /home isn't snapshot. In this case you don't need to separately mount the home subvolume because it's available as if it were a directory. It's an organizational thing I don't know that there's a right or wrong here.
>>
>
> So I did
> btrfs subvolume create /home
>
> and rsynced all the files from old home into it. The system boots and works fine. Now I get:
>
> $ sudo btrfs subvolume list .
> ID 351 gen 25880 top level 5 path home
>
> Does this mean that /home is now subvolume of root?
Yes, but more correctly it means it's in the top level of the file system. This also means your layout is mounting the top level of the file system, rather than some other subvolume. I find this layout confusing now that I'm used to a different layout: by default Fedora creates boot [1], root and home subvolumes, which then in fstab are mounted using subvol=. Therefore the top level of the file system is not mounted.
Naturally I found this quite confusing when I first encountered it, so it's understandable why other distros do things differently. It's perhaps easier to see the fstab, sub list, and the kernel boot param.
UUID=xxxxxx / btrfs subvol=root,compress=lzo,ssd 0 0
UUID=xxxxxx /boot btrfs subvol=boot,compress=lzo,ssd 0 0
UUID=yy /boot/efi vfat umask=0077,shortname=winnt 0 0
UUID=xxxxxx /home btrfs subvol=home,compress=lzo,ssd 0 0
UUID=xxxxxx /var/log/journal btrfs subvol=journald,compress=lzo,ssd 0 0
# btrfs sub list /
ID 256 gen 505 top level 5 path home
ID 258 gen 510 top level 5 path root
ID 259 gen 511 top level 5 path journald
ID 263 gen 507 top level 5 path boot
ID 264 gen 192 top level 5 path boot.clean
ID 265 gen 193 top level 5 path root.clean
ID 266 gen 194 top level 5 path home.clean
ID 269 gen 189 top level 5 path boot.1
ID 270 gen 190 top level 5 path root.1
ID 271 gen 191 top level 5 path home.1
ID 272 gen 192 top level 5 path boot.clean_ro
ID 273 gen 193 top level 5 path root.clean_ro
ID 274 gen 194 top level 5 path home.clean_ro
linuxefi /boot/vmlinuz-3.12.8-300.fc20.x86_64 root=UUID=xxxxxx ro rootflags=subvol=root vconsole.font=latarcyrheb-sun16 elevator=noop 3
The boot param "rootflags=subvol=root" is the same as fstab's mount option subvol=root. This tells the kernel that the root file system isn't merely at UUID=xxxxxx, but it's within subvolume root.
> And how do I take a snapshot of root, if it is not listed in this command?
btrfs sub snaps -r / root.1
At the top level of the file system is a subvolume, ID 5. You can't rename it, and you can't delete it. If the top level subvolume is mounted, then you are snapshotting it by referring to the mount point, in this case /. And in this example root.1 will be a read-only snapshot of /.
I think snapper on opensuse puts snapshots into a subvolume called .snapshots. You could do the same thing, and then with a file listing of / you won't see a bunch of accumulated snapshots. They could just as well go in a directory called .snapshots, off hand I'm not thinking of an advantage of putting snapshots into a subvolume.
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Snapshots – noob questions
[not found] ` <52EABE29.6080703@gmail.com>
2014-01-31 4:05 ` Chris Murphy
@ 2014-01-31 4:19 ` Chris Murphy
1 sibling, 0 replies; 14+ messages in thread
From: Chris Murphy @ 2014-01-31 4:19 UTC (permalink / raw)
To: KC; +Cc: Btrfs BTRFS
I wrote:
> They could just as well go in a directory called .snapshots, off hand I'm not thinking of an advantage of putting snapshots into a subvolume.
There isn't an advantage with a top level 5 subvolume being used for rootfs. But if it's not being mounted, rather other named subvolumes are, then the advantage of a subvolume for snapshots is that it can be independently mounted to provide access to all snapshots within from which to create derivative snapshots. And it can also have a nosuid or noexec mount option to prevent old vulnerabilities from being root executable.
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2014-01-31 4:19 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-01-27 13:53 Snapshots – noob questions KC
2014-01-27 14:03 ` Hugo Mills
2014-01-27 14:14 ` Jim Salter
2014-01-27 18:35 ` Chris Murphy
2014-01-27 18:42 ` Hugo Mills
2014-01-27 18:44 ` Chris Murphy
2014-01-27 18:50 ` Hugo Mills
2014-01-27 19:49 ` Chris Murphy
2014-01-28 18:39 ` Martin
2014-01-28 19:16 ` Hugo Mills
2014-01-29 1:21 ` Duncan
[not found] ` <52E808CB.9020005@gmail.com>
2014-01-28 20:16 ` Chris Murphy
[not found] ` <52EABE29.6080703@gmail.com>
2014-01-31 4:05 ` Chris Murphy
2014-01-31 4:19 ` Chris Murphy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.