Btrfs default on Fedora?

Btrfs default on Fedora?

[Ken D'Ambrosio]

Hi!  I just saw this mentioned on a BBS I'm on:
https://fedoraproject.org/wiki/Changes/BtrfsByDefault

I'll admit, I'm incredibly surprised, and pleased, to see that this 
might happen.  I do have three items of concern as Joe End User.  (Do 
note that for home use (where I use btrfs) I'm usually on Ubuntu or a 
variant and not RH, if that matters.)

* Swap files.  At least last time I checked, it was a PITA to take a 
snapshot of a volume that had a swapfile on it -- I wound up writing a 
wrapper that goes, does a swapoff, removes the file, creates the 
snapshot, and then re-creates the file.   Is this still "a thing"?  Or 
is there a way to work around that that isn't kludgey?

* When Stuff Goes Wrong(tm).  Again, my experience is not terribly 
current, but when things hit the fan, for most FSes, you do an
fsck -y /path/to/dev
and hope things come together.  But with btrfs, it seems that it's 
substantially more complicated to figure out what to do.  Have the 
tools, perhaps, been updated to help end users figure out what choices 
to make, etc., when dealing with an issue?

* RAID 5/6.  Last time I looked, that was in an unhappy state, so I just 
set up a RAID with mdadm, lay btrfs on top of that, and call it good.  
That seems to do the job, though it loses lots of smarts that would be 
had with btrfs running the RAID.  I see discussion on the wiki 
(https://btrfs.wiki.kernel.org/index.php/RAID56) talking about an RFC 
submitted to address the underlying issues; is this still broken?

Thanks much,

-Ken

P.S.  If there's a better mailing list, e.g., "end user questions" or 
something, please feel free to point me to it.

Re: Btrfs default on Fedora?

[Chris Murphy]

On Sat, Jul 11, 2020 at 12:44 PM Ken D'Ambrosio <ken@jots.org> wrote:

> * Swap files.  At least last time I checked, it was a PITA to take a
> snapshot of a volume that had a swapfile on it -- I wound up writing a
> wrapper that goes, does a swapoff, removes the file, creates the
> snapshot, and then re-creates the file.   Is this still "a thing"?  Or
> is there a way to work around that that isn't kludgey?

Put the swapfile in its own subvolume and don't snapshot it. One way
is to create a (nested) subvolume named "swap" inside of the "root"
subvolume created at installation time; use chattr +C on it; now
create the swapfile per 'man 5 btrfs'.

Since btrfs snapshots aren't recursive, making a snapshot of 'root'
will not cause a snapshot to be taken of 'swap' or its swapfiles.


> * When Stuff Goes Wrong(tm).  Again, my experience is not terribly
> current, but when things hit the fan, for most FSes, you do an
> fsck -y /path/to/dev
> and hope things come together.  But with btrfs, it seems that it's
> substantially more complicated to figure out what to do.  Have the
> tools, perhaps, been updated to help end users figure out what choices
> to make, etc., when dealing with an issue?

UX of the tools needs improvement. But for various reasons, it's
difficult to repair a Btrfs file system, so the emphasis is on taking
advantage of more tolerant read-only mount to freshen backups. Also,
stuff going wrong implies some sort of hardware/firmware problem, not
just Btrfs sensitivity to critical areas being damaged as a result.
The offline scrape tool is hard to use but really effective if you
stick with it.


>
> * RAID 5/6.  Last time I looked, that was in an unhappy state, so I just
> set up a RAID with mdadm, lay btrfs on top of that, and call it good.

That's fine. You don't get btrfs self healing, except for DUP
metadata. But you still get error detection, with path to damaged
file.

> That seems to do the job, though it loses lots of smarts that would be
> had with btrfs running the RAID.  I see discussion on the wiki
> (https://btrfs.wiki.kernel.org/index.php/RAID56) talking about an RFC
> submitted to address the underlying issues; is this still broken?

You should read Zygo's recent write up on raid5, most of which applies to raid6.

https://lore.kernel.org/linux-btrfs/20200627032414.GX10769@hungrycats.org/

-- 
Chris Murphy

Re: Btrfs default on Fedora?

[Andrei Borzenkov]

12.07.2020 00:18, Chris Murphy пишет:
> On Sat, Jul 11, 2020 at 12:44 PM Ken D'Ambrosio <ken@jots.org> wrote:
> 
>> * Swap files.  At least last time I checked, it was a PITA to take a
>> snapshot of a volume that had a swapfile on it -- I wound up writing a
>> wrapper that goes, does a swapoff, removes the file, creates the
>> snapshot, and then re-creates the file.   Is this still "a thing"?  Or
>> is there a way to work around that that isn't kludgey?
> 
> Put the swapfile in its own subvolume and don't snapshot it. One way
> is to create a (nested) subvolume named "swap" inside of the "root"
> subvolume created at installation time; use chattr +C on it; now
> create the swapfile per 'man 5 btrfs'.
> 
> Since btrfs snapshots aren't recursive, making a snapshot of 'root'
> will not cause a snapshot to be taken of 'swap' or its swapfiles.
> 

swap file requires calculation of absolute disk offset (that is what
kernel works with) and this is not supported on btrfs with by systemd:

https://github.com/systemd/systemd/blob/d67b1d18fcda2c8c5aacfc50f9591c8dc7a4a8a1/src/shared/sleep-config.c#L240

I believe with current systemd if you manually compute and provide
resume=btrfs-partition and resume_offset=absolute-offset-of-swapfile it
should take it, but using resume=file-on-btrfs will fail.

Re: Btrfs default on Fedora?

[David Sterba]

On Sun, Jul 12, 2020 at 08:28:32AM +0300, Andrei Borzenkov wrote:
> 12.07.2020 00:18, Chris Murphy пишет:
> > On Sat, Jul 11, 2020 at 12:44 PM Ken D'Ambrosio <ken@jots.org> wrote:
> > 
> >> * Swap files.  At least last time I checked, it was a PITA to take a
> >> snapshot of a volume that had a swapfile on it -- I wound up writing a
> >> wrapper that goes, does a swapoff, removes the file, creates the
> >> snapshot, and then re-creates the file.   Is this still "a thing"?  Or
> >> is there a way to work around that that isn't kludgey?
> > 
> > Put the swapfile in its own subvolume and don't snapshot it. One way
> > is to create a (nested) subvolume named "swap" inside of the "root"
> > subvolume created at installation time; use chattr +C on it; now
> > create the swapfile per 'man 5 btrfs'.
> > 
> > Since btrfs snapshots aren't recursive, making a snapshot of 'root'
> > will not cause a snapshot to be taken of 'swap' or its swapfiles.
> 
> swap file requires calculation of absolute disk offset (that is what
> kernel works with) and this is not supported on btrfs with by systemd:

When the swapfile is not used for suspend/resume, the offsets don't
matter. The reads and writes build on generic DIO infrastructure, in the
same way swap-over-NFS works that also does not have any physical
offsets in general.

> https://github.com/systemd/systemd/blob/d67b1d18fcda2c8c5aacfc50f9591c8dc7a4a8a1/src/shared/sleep-config.c#L240
> 
> I believe with current systemd if you manually compute and provide
> resume=btrfs-partition and resume_offset=absolute-offset-of-swapfile it
> should take it, but using resume=file-on-btrfs will fail.

Yeah, that's where the swapfile is not accessed through the filesystem,
so the block offsets need to be calculated externally.

Re: Btrfs default on Fedora?

[David Sterba]

On Sat, Jul 11, 2020 at 02:38:26PM -0400, Ken D'Ambrosio wrote:
> Hi!  I just saw this mentioned on a BBS I'm on:
> https://fedoraproject.org/wiki/Changes/BtrfsByDefault
> 
> I'll admit, I'm incredibly surprised, and pleased, to see that this 
> might happen.  I do have three items of concern as Joe End User.  (Do 
> note that for home use (where I use btrfs) I'm usually on Ubuntu or a 
> variant and not RH, if that matters.)
> 
> * Swap files.  At least last time I checked, it was a PITA to take a 
> snapshot of a volume that had a swapfile on it -- I wound up writing a 
> wrapper that goes, does a swapoff, removes the file, creates the 
> snapshot, and then re-creates the file.   Is this still "a thing"?  Or 
> is there a way to work around that that isn't kludgey?

The workaround is to use separate subvolume for swapfile. Anything else
would cause only problems. Imagine you have a swapfile on root
partition, take frequent snapshots. As the swapfile contents change
during system use, the shared blocks get unshared and occupy more space.
In the end, each rootfs snapshot could have it's own swapfile with
useless data.

Requiring the separate subvolume was a compromise, it's not ideal but
IMHO at least makes things clear. Deleting a swapfile will release the
blocks, there's only one instance of the swapfile (for swapon/swapoff).