Small fs

Small fs

[Imran Geriskovan]

What is the smallest recommended fs size for btrfs?

- There are mentions of 256MB around the net.
- Gparted reserves minimum of 256MB for btrfs.

With an ordinary partition on a single disk,
fs created with just "mkfs.btrfs /dev/sdxx":
- 128MB works fine.
- 127MB works but as if it is 64MB.

Can we say size should be in multiples of 64MB?

Re: Small fs

[Martin Steigerwald]

Am Sonntag, 11. September 2016, 18:27:30 CEST schrieben Sie:
> What is the smallest recommended fs size for btrfs?
> 
> - There are mentions of 256MB around the net.
> - Gparted reserves minimum of 256MB for btrfs.
> 
> With an ordinary partition on a single disk,
> fs created with just "mkfs.btrfs /dev/sdxx":
> - 128MB works fine.
> - 127MB works but as if it is 64MB.
> 
> Can we say size should be in multiples of 64MB?

Do you want to know the smalled *recommended* or the smallest *possible* size?

I personally wouldn´t go below one or two GiB or or so with BTRFS. On small 
filesystems, I don´t know the treshold right now it uses a mixed metadata/data 
format. And I think using smaller BTRFS filesystem invited any left over 
"filesystem is full while it isn´t" issues.

Well there we go. Excerpt from mkfs.btrfs(8) manpage:

       -M|--mixed
           Normally the data and metadata block groups are isolated.
           The mixed mode will remove the isolation and store both
           types in the same block group type. This helps to utilize
           the free space regardless of the purpose and is suitable
           for small devices. The separate allocation of block groups
           leads to a situation where the space is reserved for the
           other block group type, is not available for allocation and
           can lead to ENOSPC state.

           The recommended size for the mixed mode is for filesystems
           less than 1GiB. The soft recommendation is to use it for
           filesystems smaller than 5GiB. The mixed mode may lead to
           degraded performance on larger filesystems, but is
           otherwise usable, even on multiple devices.

           The nodesize and sectorsize must be equal, and the block
           group types must match.

               Note
               versions up to 4.2.x forced the mixed mode for devices
               smaller than 1GiB. This has been removed in 4.3+ as it
               caused some usability issues.

Thanks
-- 
Martin

Re: Small fs

[Duncan]

Martin Steigerwald posted on Sun, 11 Sep 2016 17:32:44 +0200 as excerpted:

> Am Sonntag, 11. September 2016, 18:27:30 CEST schrieben Sie:
>> What is the smallest recommended fs size for btrfs?
>> 
>> - There are mentions of 256MB around the net.
>> - Gparted reserves minimum of 256MB for btrfs.
>> 
>> With an ordinary partition on a single disk,
>> fs created with just "mkfs.btrfs /dev/sdxx":
>> - 128MB works fine.
>> - 127MB works but as if it is 64MB.
>> 
>> Can we say size should be in multiples of 64MB?
> 
> Do you want to know the smalled *recommended* or the smallest *possible*
> size?
> 
> I personally wouldn´t go below one or two GiB or or so with BTRFS. On
> small filesystems, I don´t know the treshold right now it uses a mixed
> metadata/data format. And I think using smaller BTRFS filesystem invited
> any left over "filesystem is full while it isn´t" issues.

I agree with the general recommendations and *strongly* recommend (and 
use) mixed-mode below say 2 GiB myself, but I don't necessarily agree 
that the practical bottom limit is 1 GiB, as I partition heavily and have 
a couple smaller btrfs myself.

There are some limits that should be kept in mind, however.

* Metadata, and thus mixed-bg, defaults to DUP mode on a single-device 
filesystem (except on ssd where I actually still use it myself, and 
recommend it except for ssds that do firmware dedupe).  In mixed-mode 
this means two copies of data as well, which halves the usable space.

IOW, when using mixed-mode, which is recommended under a gig, and dup 
replication which is then the single-device default, effective usable 
space is **HALVED**, so 256 MiB btrfs size becomes 128 MiB usable. (!!)

When the filesystem is that small, space tends to be at a premium in any 
case, and halving the available space can take a big bite out of things, 
so one considering such a btrfs should consider carefully, and possibly 
specify single mode if the loss of the data on the filesystem won't be a 
big deal, or double the filesystem size to account for dup data, if it 
would be more appropriate.

Or do pair-device which defaults to raid1 mode for metadata and mixed.  
Or choose a more traditional filesystem, perhaps ext* based, perhaps even 
without a journal if size is that constrained, because the journal takes 
space as well, because fscking that small a filesystem in the event of a 
crash shouldn't take that long anyway.

* There's also a system chunk to consider.  This too is normally dup mode 
on single device, raid1 on multi.  While it shrinks to some extent with 
size of filesystem, my 256 MiB /boot still has a 15.5 MiB system chunk, 
doubled due to dup mode to 31 MiB.  This is overhead you won't be able to 
use for anything else.

* There's also global reserve.  This is a part of metadata (and thus 
mixed-mode) that cannot be used under normal circumstances either.  
However, unlike normal metadata, the accounting here is single -- it's X 
space reserved no matter the replication type.  On my 256 MiB /boot, it's 
4 MiB.

So of my 256 MiB btrfs mixed-mode /boot, 31+4=35 MiB is overhead, leaving 
221 MiB for actual data and metadata.  But due to dup mode that's halved, 
to 110.5 MiB usable space.

For something like a dedicated /boot that may be acceptable.  For many 
applications it wouldn't be, and either the target size would need 
doubled, or filesystem robustness would need compromised by switching to 
single mode instead of dup.  Or use a different filesystem.

Tho to compare apples to apples, most other filesystems are only single 
mode anyway, and if it's an acceptable usage for them, it may well be an 
acceptable usage for btrfs as well.  It's just that btrfs likes a bit 
more metadata robustness than that.  But with such limited sizes, backing 
up the data if it's worth the hassle shouldn't be horribly time or 
resource consuming, and single mode can then make a bit more sense.

FWIW, in addition to my previously mentioned 256 MiB /boot, and a backup 
/boot (selectable via bios) on another device, I have a 640 MiB btrfs 
pair-device raid1 /var/log.  It's mixed-mode too, 640 MiB per device, but 
pair-device raid1, so I don't have to worry about the 2X data factor on a 
single device.

All my other "system" partitions are btrfs raid1 as well, including /, 
/home, and the packages/build partition.  But those are all over 2 GiB 
each.  All my btrfs are on ssd, with only my media partitions and backups 
on spinning rust (reiserfs).

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Small fs

[Imran Geriskovan]

On 9/11/16, Duncan <1i5t5.duncan@cox.net> wrote:
> Martin Steigerwald posted on Sun, 11 Sep 2016 17:32:44 +0200 as excerpted:
>>> What is the smallest recommended fs size for btrfs?
>>> Can we say size should be in multiples of 64MB?
>> Do you want to know the smalled *recommended* or the smallest *possible*
>> size?

In fact both.
I'm reconsidering my options for /boot

> * Metadata, and thus mixed-bg, defaults to DUP mode on a single-device
> filesystem (except on ssd where I actually still use it myself, and
> recommend it except for ssds that do firmware dedupe).  In mixed-mode
> this means two copies of data as well, which halves the usable space.

> IOW, when using mixed-mode, which is recommended under a gig, and dup
> replication which is then the single-device default, effective usable
> space is **HALVED**, so 256 MiB btrfs size becomes 128 MiB usable. (!!)

> * There's also a system chunk to consider.  This too is normally dup mode
> on single device, raid1 on multi.  While it shrinks to some extent with
> size of filesystem, my 256 MiB /boot still has a 15.5 MiB system chunk,
> doubled due to dup mode to 31 MiB.  This is overhead you won't be able to
> use for anything else.
>
> * There's also global reserve.  This is a part of metadata (and thus
> mixed-mode) that cannot be used under normal circumstances either.
> However, unlike normal metadata, the accounting here is single -- it's X
> space reserved no matter the replication type.  On my 256 MiB /boot, it's
> 4 MiB.
>
> So of my 256 MiB btrfs mixed-mode /boot, 31+4=35 MiB is overhead, leaving
> 221 MiB for actual data and metadata.  But due to dup mode that's halved,
> to 110.5 MiB usable space.

That's quite an info.. Thanks a lot..

Just to note again:
Ordinary 127MB btrfs gives "Out of space" around
64MB payload. 128MB is usable to the end.

I'm experimenting with an extracted (and customized) initrd
on /boot. That is, /boot is a "minimal root on its own" which
can switch to real root or do other things. Kernel and modules
at /boot will not support any fs other than btrfs. (Or it may)

It seems a minimally usable root around 10MB is possible.
And it is free of udev and systemd..

Re: Small fs

[Martin Steigerwald]

Am Sonntag, 11. September 2016, 21:56:07 CEST schrieb Imran Geriskovan:
> On 9/11/16, Duncan <1i5t5.duncan@cox.net> wrote:
> > Martin Steigerwald posted on Sun, 11 Sep 2016 17:32:44 +0200 as excerpted:
> >>> What is the smallest recommended fs size for btrfs?
> >>> Can we say size should be in multiples of 64MB?
> >> 
> >> Do you want to know the smalled *recommended* or the smallest *possible*
> >> size?
> 
> In fact both.
> I'm reconsidering my options for /boot

Well my stance on boot still is: Ext4. Done.

:)

It just does not bother me. It practically makes no difference at all. It has 
no visible effect on my user experience and I never saw the need to snapshot /
boot.

But another approach in case you want to use BTRFS for /boot is to use a 
subvolume. Thats IMHO the SLES 12 default setup. They basically create 
subvolumes for /boot, /var, /var/lib/mysql – you name it. Big advantage: You 
have one big FS and do not need to plan space for partitions or LVs. 
Disadvantage: If it breaks, it breaks.

That said, I think at a new installation I may do this for /boot. Just put it 
inside a subvolume.

>From my experiences at work with customer systems and even some systems I 
setup myself, I often do not use little partitions anymore. I did so for a 
CentOS 7 training VM, just 2 GiB XFS for /var. Guess what happened? Last 
update was too long ago, so… yum tried to download a ton of packages and then 
complained it has not enough space in /var. Luckily I used LVM, so I enlarged 
partition LVM resides on, enlarged PV and then enlarged /var. There may be 
valid reasons to split things up, and I am quite comfortable with splitting /
boot out, cause its, well, plannable easily enough. And it may make sense to 
split /var or /var/log out. But on BTRFS I would likely use subvolumes. Only 
thing I may separate would be /home to make it easier on a re-installation of 
the OS to keep it around. That said, I never ever reinstalled the Debian on 
this ThinkPad T520 since I initially installed it. And on previous laptops I 
even copied the Debian on the older laptop onto the newer laptop. With the 
T520 I reinstalled, cause I wanted to switch to 64 bit cleanly.

-- 
Martin

Re: Small fs

[Austin S. Hemmelgarn]

On 2016-09-11 15:21, Martin Steigerwald wrote:
> Am Sonntag, 11. September 2016, 21:56:07 CEST schrieb Imran Geriskovan:
>> On 9/11/16, Duncan <1i5t5.duncan@cox.net> wrote:
>>> Martin Steigerwald posted on Sun, 11 Sep 2016 17:32:44 +0200 as excerpted:
>>>>> What is the smallest recommended fs size for btrfs?
>>>>> Can we say size should be in multiples of 64MB?
>>>>
>>>> Do you want to know the smalled *recommended* or the smallest *possible*
>>>> size?
>>
>> In fact both.
>> I'm reconsidering my options for /boot
>
> Well my stance on boot still is: Ext4. Done.
>
> :)
>
> It just does not bother me. It practically makes no difference at all. It has
> no visible effect on my user experience and I never saw the need to snapshot /
> boot.
FWIW, I use BTRFS for /boot, but it's not for snapshotting or even the 
COW, it's for DUP mode and the error recovery it provides.  Most people 
don't think about this if it hasn't happened to them, but if you get a 
bad read from /boot when loading the kernel or initrd, it can 
essentially nuke your whole system.  I run BTRFS for /boot in DUP mode 
with mixed-bg (because I only use 512MB for boot) to mitigate the chance 
that a failed read has any impact, and ensure that if it does, it will 
refuse to boot instead of booting with a corrupted kernel or initrd.
>
> But another approach in case you want to use BTRFS for /boot is to use a
> subvolume. Thats IMHO the SLES 12 default setup. They basically create
> subvolumes for /boot, /var, /var/lib/mysql – you name it. Big advantage: You
> have one big FS and do not need to plan space for partitions or LVs.
> Disadvantage: If it breaks, it breaks.
>
> That said, I think at a new installation I may do this for /boot. Just put it
> inside a subvolume.
>
> From my experiences at work with customer systems and even some systems I
> setup myself, I often do not use little partitions anymore. I did so for a
> CentOS 7 training VM, just 2 GiB XFS for /var. Guess what happened? Last
> update was too long ago, so… yum tried to download a ton of packages and then
> complained it has not enough space in /var. Luckily I used LVM, so I enlarged
> partition LVM resides on, enlarged PV and then enlarged /var. There may be
> valid reasons to split things up, and I am quite comfortable with splitting /
> boot out, cause its, well, plannable easily enough. And it may make sense to
> split /var or /var/log out. But on BTRFS I would likely use subvolumes. Only
> thing I may separate would be /home to make it easier on a re-installation of
> the OS to keep it around. That said, I never ever reinstalled the Debian on
> this ThinkPad T520 since I initially installed it. And on previous laptops I
> even copied the Debian on the older laptop onto the newer laptop. With the
> T520 I reinstalled, cause I wanted to switch to 64 bit cleanly.
>
Yeah, small partitions make sense in some cases, but in a lot of places 
they're typically used, there are perfectly legitimate reasons to 
suddenly need multiple gigabytes of extra space for a short period of 
time.  In my case, the only partition that I have split out that's less 
than a few GB is /boot, but I usually split out /usr/src and 
/usr/portage, because I need them to be fast, and run /tmp and /var/tmp 
on tmpfs (so technically split out).  I personally see little value in 
splitting out /home on a single user system unless you're multi-booting 
or planning to switch distros at some point (or are running a distro 
that needs special handling to do release upgrades).

Re: Small fs

[Henk Slager]

> FWIW, I use BTRFS for /boot, but it's not for snapshotting or even the COW,
> it's for DUP mode and the error recovery it provides.  Most people don't
> think about this if it hasn't happened to them, but if you get a bad read
> from /boot when loading the kernel or initrd, it can essentially nuke your
> whole system.  I run BTRFS for /boot in DUP mode with mixed-bg (because I
> only use 512MB for boot) to mitigate the chance that a failed read has any
> impact, and ensure that if it does, it will refuse to boot instead of
> booting with a corrupted kernel or initrd.

Suppose kernel and initrd are on a BTRFS fs with data, metadata and
system all single profile. Will a bootloader then just continue
booting up a system even when there are csum errors in kernel and/or
initrd files?  Suppose the bootloader is grub2.

Re: Small fs

[Austin S. Hemmelgarn]

On 2016-09-12 10:09, Henk Slager wrote:
>> FWIW, I use BTRFS for /boot, but it's not for snapshotting or even the COW,
>> it's for DUP mode and the error recovery it provides.  Most people don't
>> think about this if it hasn't happened to them, but if you get a bad read
>> from /boot when loading the kernel or initrd, it can essentially nuke your
>> whole system.  I run BTRFS for /boot in DUP mode with mixed-bg (because I
>> only use 512MB for boot) to mitigate the chance that a failed read has any
>> impact, and ensure that if it does, it will refuse to boot instead of
>> booting with a corrupted kernel or initrd.
>
> Suppose kernel and initrd are on a BTRFS fs with data, metadata and
> system all single profile. Will a bootloader then just continue
> booting up a system even when there are csum errors in kernel and/or
> initrd files?  Suppose the bootloader is grub2.
>
Assuming I understand how they implemented their BTRFS support in GRUB, 
it should still be checking csums just like the kernel implementation, 
and thus should throw a read error when a block is corrupted.  For a 
stock upstream version of GRUB2, this would mean that it will refuse to 
boot that entry, and may try to auto-boot a fallback entry (which will 
probably fail given how most distros set up their fallback entries).

Re: Small fs

[Chris Murphy]

On Mon, Sep 12, 2016 at 8:09 AM, Henk Slager <eye1tm@gmail.com> wrote:
>> FWIW, I use BTRFS for /boot, but it's not for snapshotting or even the COW,
>> it's for DUP mode and the error recovery it provides.  Most people don't
>> think about this if it hasn't happened to them, but if you get a bad read
>> from /boot when loading the kernel or initrd, it can essentially nuke your
>> whole system.  I run BTRFS for /boot in DUP mode with mixed-bg (because I
>> only use 512MB for boot) to mitigate the chance that a failed read has any
>> impact, and ensure that if it does, it will refuse to boot instead of
>> booting with a corrupted kernel or initrd.
>
> Suppose kernel and initrd are on a BTRFS fs with data, metadata and
> system all single profile. Will a bootloader then just continue
> booting up a system even when there are csum errors in kernel and/or
> initrd files?  Suppose the bootloader is grub2.

I"m wondering the same thing. I don't know if GRUB's Btrfs code checks
for csum matches, and on error whether it knows to retry from some
other block group.


-- 
Chris Murphy

Re: Small fs

[Austin S. Hemmelgarn]

On 2016-09-12 10:51, Chris Murphy wrote:
> On Mon, Sep 12, 2016 at 8:09 AM, Henk Slager <eye1tm@gmail.com> wrote:
>>> FWIW, I use BTRFS for /boot, but it's not for snapshotting or even the COW,
>>> it's for DUP mode and the error recovery it provides.  Most people don't
>>> think about this if it hasn't happened to them, but if you get a bad read
>>> from /boot when loading the kernel or initrd, it can essentially nuke your
>>> whole system.  I run BTRFS for /boot in DUP mode with mixed-bg (because I
>>> only use 512MB for boot) to mitigate the chance that a failed read has any
>>> impact, and ensure that if it does, it will refuse to boot instead of
>>> booting with a corrupted kernel or initrd.
>>
>> Suppose kernel and initrd are on a BTRFS fs with data, metadata and
>> system all single profile. Will a bootloader then just continue
>> booting up a system even when there are csum errors in kernel and/or
>> initrd files?  Suppose the bootloader is grub2.
>
> I"m wondering the same thing. I don't know if GRUB's Btrfs code checks
> for csum matches, and on error whether it knows to retry from some
> other block group.
I can test this and report results (I've got GRUB built with the FUSE 
based mount tool, which lets you use GRUB's FS modules from regular 
Linux userspace), but it may be a while before I can get things set up 
to properly test it.

Re: Small fs

[Duncan]

Imran Geriskovan posted on Sun, 11 Sep 2016 21:56:07 +0300 as excerpted:

> On 9/11/16, Duncan <1i5t5.duncan@cox.net> wrote:
>> Martin Steigerwald posted on Sun, 11 Sep 2016 17:32:44 +0200 as
>> excerpted:
>>>> What is the smallest recommended fs size for btrfs?
>>>> Can we say size should be in multiples of 64MB?
>>> Do you want to know the smalled *recommended* or the smallest
>>> *possible*
>>> size?
> 
> In fact both.
> I'm reconsidering my options for /boot

>> [Snip my detail points, leaving just the summary...]
 
>> So of my 256 MiB btrfs mixed-mode /boot, 31+4=35 MiB is overhead,
>> leaving 221 MiB for actual data and metadata.  But due to dup mode
>> that's halved, to 110.5 MiB usable space.
> 
> That's quite an info.. Thanks a lot..
> 
> Just to note again:
> Ordinary 127MB btrfs gives "Out of space" around 64MB payload. 128MB is
> usable to the end.

Thanks, and just to clarify for others possibly following along or 
googling it up later, that's single mode (as opposed to dup mode) for at 
least data, if in normal separate data/metadata mode, and single for the 
combined mixed-mode chunks if in mixed-bg mode, correct?

Because if the data is dup mode as well, as it would be by default in 
mixed-bg mode (unless on ssd), 128 MiB should allow storing only 64 MiB 
(and that's not accounting for the system chunk or global reserve 
metadata, so it'd be less than that) data.

> I'm experimenting with an extracted (and customized) initrd on /boot.
> That is, /boot is a "minimal root on its own" which can switch to real
> root or do other things. Kernel and modules at /boot will not support
> any fs other than btrfs. (Or it may)
> 
> It seems a minimally usable root around 10MB is possible.
> And it is free of udev and systemd..

You don't specifically mention the distro, but given that gentoo's one of 
the only general-purpose distros that hasn't switched to systemd yet (tho 
it offers the choice for those who want it, and I've taken that choice 
here), there's a fair chance that's what you're running, as both I and 
Martin Steigerwald (based on earlier threads) are.

FWIW, a bit more about my /boot (and kernel/initr*) layout, then, since 
it may be apropos... both because you're dealing with /boot, and because 
you might be on gentoo, as well, in which case you may find even more 
useful hints in the below.  Of course you can set it up how you like, but 
this is what I've found works very well for me. =:^)

1) Since I custom-configure and build my own kernels anyway, I build 
monolithic, no kernel modules, and indeed, the kernel options allowing 
loading modules are turned off -- more secure that way since I don't load 
modules anyway.

That simplifies a lot of stuff, including no longer needing module-init-
tools or whatever installed at all.  So it's not. (I negated the @system 
listing for it; actually I negated @system entirely and added what I 
actually needed to my @world via world-sets, but that's beside the point.)

2) Since a multi-device btrfs / effectively requires an initr*, I do run 
one, using dracut to generate it, but I don't boot it as a separate 
file.  Instead I have the kernel configured to pull it in and append it 
as an initramfs to each built kernel.

The idea here is that I keep tested-bootable kernels along with their 
tested-bootable initr*s around, so whatever I may be upgrading, kernel, 
dracut, the initr* itself, or some package with a binary in the initr*, I 
always have a fallback kernel and its attached initramfs that's tested 
bootable.  That way, should some kernel/initr* component fail, I know I 
can simply pick an older/tested kernel/initramfs file, since they're a 
single file, and boot with it, using that boot to investigate and fix the 
problem with the kernel/initramfs.  And I also know that once I've tested 
a kernel/initramfs to work, if it suddenly fails, it can't be something 
in the initramfs or kernel itself, as that is known to work, it must 
instead be some problem with the hardware or in the boot after handover 
to the main /.

One additional detail here regarding dracut:  Even if you've configured 
dracut to not worry about modules as you run a monolithic kernel, for 
some reason (bug?) it still requires (as opposed to uses if installed) 
rmmod for an initr* build.  Why it requires rmmod but not modprobe is 
beyond me, but it does.  So now I have a no-rmmod.patch in
/etc/portage/patches/sys-kernel/dracut , and I can use dracut setup for a 
monolithic kernel initr*, without having to have the useless rmmod from 
module-init-tools around for it to pull in when I build a new initr*.

3) Due to btrfs dup-mode halving my usable space on /boot, something I 
should have anticipated but didn't, I have only ~110 MiB of space on /
boot, and the grub2 installation eats up some of it.  Of course that 
leaves me way less room for kernels than I expected, particularly when I 
had the uncompressed initr* (that the kernel build compresses and appends 
as an initramfs to every built kernel) stored on /boot as well.

Eventually that crimped my style sufficiently that I moved the 
uncompressed initr* elsewhere.  I ended up with it on /p, my packages/
build partition, which has the kernel sources on it too, so it'd need to 
be mounted to build the kernel anyway.  Without the uncompressed initr* 
on /boot, the 256 MiB btrfs dup-mode, thus ~110 MiB usable, has been fine 
-- I don't find myself running out of space for new kernel builds as much 
any more, making it much easier to git-bisect a kernel bug, for instance. 
=:^)

So if you're going to do an uncompressed initr* on /boot, if you're using 
btrfs in mixed-bg dup mode as I am, better make it 384 MiB or even 512 
MiB, because 256 MiB will definitely crimp your style.  If OTOH you're 
using single mode data, presumably because you're doing mixed-mode but 
only single (I still wouldn't recommend non-mixed-mode at that size), 256 
MiB ends up being much more reasonable.

4) I use gpt partition tables on everything.  They're far more flexible 
and reliable than the old MBR, and because they have a checksummed second 
copy at the end of the device as well, they're far easier to recover if 
the primary copy gets corrupted (plus with the checksumming the 
corruption is actually detected).

In gpt, I actually setup both an EFI partition and a legacy BIOS 
partition.  They're both very small and it increases your flexibility 
dramatically.  As it happens, my current main system was purchased just a 
bit before EFI replaced BIOS, and I have grub2 installed to the legacy 
BIOS area and the EFI unused, but both are available should that change 
and I decide to use EFI booting.  (I may not, it sounds like more trouble 
than it's worth for those building their own grub, etc, but the 
flexibility is there should I decide to use it.)

5) The BIOS does have a hotkey to select the boot device, however, and I 
keep a backup /boot installed to the other ssd (primary backup) and to 
the spinning rust (secondary backup), just in case.

That allows me to upgrade grub without worrying about killing bootability 
due to a corrupted grub install.  I upgrade the package, then install it 
to one device first, then test booting to it before trying to install to 
the backups.

6) In ordered to facilitate installing grub2 to the backups as well as my 
working /boot, /boot itself is actually a symlink.  By default, it points 
to /bt, the mountpoint for my working /boot.  That way I can tell grub2 
to install to /dev/sda, and it installs grub2-core to the dedicated BIOS 
boot partition on that device, while installing the various other grub 
files to the /boot, which is pointed at the mounted /bt.

When I want to install grub2 to a backup, I point the /boot symlink at 
/bk/bt, the back-boot mountpoint, instead.  Then with the appropriate 
backup boot mounted, I can install grub2 to either sdb or sdc as 
appropriate, and again have it install both grub-core to the appropriate 
device's dedicated BIOS boot partition, and the various grub files to /
boot, which is now pointing at /bk/bt, which is where the backup /boot 
for the appropriate device is mounted.

Because it's far easier and easier to keep track of without getting 
confused, to simply switch the symlink pointer and keep the backup /boots 
mounting in the backup location, than it would be to switch the 
mountpoints themselves around.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Small fs

[Imran Geriskovan]

>> Just to note again:
>> Ordinary 127MB btrfs gives "Out of space" around 64MB payload. 128MB is
>> usable to the end.

> Thanks, and just to clarify for others possibly following along or
> googling it up later, that's single mode (as opposed to dup mode) for at
> least data, if in normal separate data/metadata mode, and single for the
> combined mixed-mode chunks if in mixed-bg mode, correct?
>
> Because if the data is dup mode as well, as it would be by default in
> mixed-bg mode (unless on ssd), 128 MiB should allow storing only 64 MiB
> (and that's not accounting for the system chunk or global reserve
> metadata, so it'd be less than that) data.


That's /boot on my laptop. Its fairly old fs
which is created about 3-4 years. May be 5.
I'm not so sure..

btrfs filesystem df /boot
System, single: total=4.00MiB, used=4.00KiB
Data+Metadata, single: total=124.00MiB, used=103.88MiB
GlobalReserve, single: total=4.00MiB, used=0.00B
IT IS SINGLE!!

The 128/64Mb thing happened when I created a backup
usb drive for mirroring it. Then make it 256MB.

btrfs filesystem df /mnt/back/boot
Data, single: total=8.00MiB, used=0.00B
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=32.00MiB, used=112.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
IT IS DUP!!

So Duncan you got it:
I'm comparing the SINGLE to DUP...
I'm not sure how I got that SINGLE though...


>> I'm experimenting with an extracted (and customized) initrd on /boot.
>> That is, /boot is a "minimal root on its own" which can switch to real
>> root or do other things. Kernel and modules at /boot will not support
>> any fs other than btrfs. (Or it may)
>> It seems a minimally usable root around 10MB is possible.
>> And it is free of udev and systemd..
>
> You don't specifically mention the distro, but given that gentoo's one of
> the only general-purpose distros that hasn't switched to systemd yet (tho
> it offers the choice for those who want it, and I've taken that choice
> here), there's a fair chance that's what you're running, as both I and
> Martin Steigerwald (based on earlier threads) are.

Now the offtopic:

- I'm on Arch. Considering to switch to Void.

- Using syslinux as bootmanager. It is rock solid and compact.
Just edit syslinux.cfg and reboot to any configuration
without any hassle.

- Also using runit side stepping systemd. It is also rock solid
and compact.

- Know your hardware and thus the Kernel.
And get rid of and middleware (all udev derivatives) playing
guess games on your machine.

- Two custom kernel configurations:
All built-in (Was using until now.)
All modules. (Now switching to this.)
Load all your modules on exactly known locations
when booting. To do this customize your
runit start up scripts.
I'm switching to "All modules" approach
after I determined and understand the key modules.
Don't affaid. Its accually very doable.
You need at most 8-9 modprobes
to get them all. If you know your HW and
kernel it is no big deal.

- Configuring kernel teaches alot.

- Compiling a kernel with only required modules
takes about 15 minutes if you got your .config
file right. Play with "make xconfig". Alot...

- Keep all your previous .config files as record.
When something goes wrong diff them for
throubleshooting.

- Dracut is a dirty hack. mkinitcpio is great.

- Compressed initrd is bad. It hides what is going
on. Extract it. Play with it. Than make it the root fs
of your /boot. (I'm now here)

- Starting X without udev is simple.
Disable hotplug.
5-6 lines of Manual entries for keyboard, mouse and
touchpad is sufficient in your /etc/X11/xorg.conf.d

Re: Small fs

[Imran Geriskovan]

> btrfs filesystem df /mnt/back/boot
> Data, single: total=8.00MiB, used=0.00B
> System, DUP: total=8.00MiB, used=16.00KiB
> Metadata, DUP: total=32.00MiB, used=112.00KiB
> GlobalReserve, single: total=16.00MiB, used=0.00B
> IT IS DUP!!

Wait wait wait a second:
This is 256 MB SINGLE created
by GPARTED, which is the replacement of MANUALLY
CREATED 127MB DUP which is now non-existant..
Which I was not aware it was a DUP at the time..
Peeww... Small btrfs is full of surprises.. ;)

Re: Small fs

[Imran Geriskovan]

> Wait wait wait a second:
> This is 256 MB SINGLE created
> by GPARTED, which is the replacement of MANUALLY
> CREATED 127MB DUP which is now non-existant..
> Which I was not aware it was a DUP at the time..
> Peeww... Small btrfs is full of surprises.. ;)

What's more, I also have another 128MB SINGLE
which I've been using for some years and did not
bother with its DUP/SINGLENESS. And I compared
them all to draw some conclusions. Heh..
That's the story.

Verdict is: DUP/SINGLE is a very serious fun stuff
when used unknowingly. Small btrfs is such a case.
And third party tools (ex: gparted) plays with it.

Let's warn users with some documentation,
together with "A formal small fs" behaviour..

Re: Small fs

[Austin S. Hemmelgarn]

On 2016-09-12 14:46, Imran Geriskovan wrote:
>> Wait wait wait a second:
>> This is 256 MB SINGLE created
>> by GPARTED, which is the replacement of MANUALLY
>> CREATED 127MB DUP which is now non-existant..
>> Which I was not aware it was a DUP at the time..
>> Peeww... Small btrfs is full of surprises.. ;)
>
> What's more, I also have another 128MB SINGLE
> which I've been using for some years and did not
> bother with its DUP/SINGLENESS. And I compared
> them all to draw some conclusions. Heh..
> That's the story.
>
> Verdict is: DUP/SINGLE is a very serious fun stuff
> when used unknowingly. Small btrfs is such a case.
> And third party tools (ex: gparted) plays with it.
>
> Let's warn users with some documentation,
> together with "A formal small fs" behaviour..
I'm not sure about gparted, but the default behavior for mkfs is as follows:
1. Is the device rotational? (check /sys/block/<device>/rotational).  If 
not, do some extra stuff to try and ID it as an SSD.  If it is an SSD, 
use SINGLE mode for metadata, otherwise use DUP mode for metadata.
2. Is the FS set for mixed-bg?  If so, use the same profile for data as 
metadata, otherwise use SINGLE mode for data.

It would not surprise me if gparted switches to single metadata mode for 
a small enough FS, but I'm not certain.  I do think that they just use 
the default selection for mixed-bg though, which means not using it in 
current btrfs-progs versions.

Re: Small fs

[Mike Fleetwood]

On 12 September 2016 at 19:55, Austin S. Hemmelgarn
<ahferroin7@gmail.com> wrote:
> I'm not sure about gparted, but the default behavior for mkfs is as follows:
> 1. Is the device rotational? (check /sys/block/<device>/rotational).  If
> not, do some extra stuff to try and ID it as an SSD.  If it is an SSD, use
> SINGLE mode for metadata, otherwise use DUP mode for metadata.
> 2. Is the FS set for mixed-bg?  If so, use the same profile for data as
> metadata, otherwise use SINGLE mode for data.
>
> It would not surprise me if gparted switches to single metadata mode for a
> small enough FS, but I'm not certain.  I do think that they just use the
> default selection for mixed-bg though, which means not using it in current
> btrfs-progs versions.

GParted always just takes the mkfs.btrfs defaults.

https://git.gnome.org/browse/gparted/tree/src/btrfs.cc?h=GPARTED_0_26_1#n154

Re: Small fs

[Martin Steigerwald]

Am Sonntag, 11. September 2016, 16:44:23 CEST schrieb Duncan:
> * Metadata, and thus mixed-bg, defaults to DUP mode on a single-device 
> filesystem (except on ssd where I actually still use it myself, and 
> recommend it except for ssds that do firmware dedupe).  In mixed-mode 
> this means two copies of data as well, which halves the usable space.
> 
> IOW, when using mixed-mode, which is recommended under a gig, and dup 
> replication which is then the single-device default, effective usable 
> space is **HALVED**, so 256 MiB btrfs size becomes 128 MiB usable. (!!)

I don´t get this part. That is just *metadata* being duplicated, not the 
actual *data* inside the files. Or am I missing something here?

-- 
Martin

Re: Small fs

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]

On Sun, Sep 11, 2016 at 09:13:28PM +0200, Martin Steigerwald wrote:
> Am Sonntag, 11. September 2016, 16:44:23 CEST schrieb Duncan:
> > * Metadata, and thus mixed-bg, defaults to DUP mode on a single-device 
> > filesystem (except on ssd where I actually still use it myself, and 
> > recommend it except for ssds that do firmware dedupe).  In mixed-mode 
> > this means two copies of data as well, which halves the usable space.
> > 
> > IOW, when using mixed-mode, which is recommended under a gig, and dup 
> > replication which is then the single-device default, effective usable 
> > space is **HALVED**, so 256 MiB btrfs size becomes 128 MiB usable. (!!)
> 
> I don´t get this part. That is just *metadata* being duplicated, not the 
> actual *data* inside the files. Or am I missing something here?

   In mixed mode, there's no distinction: Data and metadata both use
the same chunks. If those chunks are DUP, then both data and metadata
are duplicated, and you get half the space available.

   Hugo.

-- 
Hugo Mills             | Questions are a burden, and answers a prison for
hugo@... carfax.org.uk | oneself
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                          The Prisoner

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Small fs

[Martin Steigerwald]

Am Sonntag, 11. September 2016, 19:46:32 CEST schrieb Hugo Mills:
> On Sun, Sep 11, 2016 at 09:13:28PM +0200, Martin Steigerwald wrote:
> > Am Sonntag, 11. September 2016, 16:44:23 CEST schrieb Duncan:
> > > * Metadata, and thus mixed-bg, defaults to DUP mode on a single-device
> > > filesystem (except on ssd where I actually still use it myself, and
> > > recommend it except for ssds that do firmware dedupe).  In mixed-mode
> > > this means two copies of data as well, which halves the usable space.
> > > 
> > > IOW, when using mixed-mode, which is recommended under a gig, and dup
> > > replication which is then the single-device default, effective usable
> > > space is **HALVED**, so 256 MiB btrfs size becomes 128 MiB usable. (!!)
> > 
> > I don´t get this part. That is just *metadata* being duplicated, not the
> > actual *data* inside the files. Or am I missing something here?
> 
>    In mixed mode, there's no distinction: Data and metadata both use
> the same chunks. If those chunks are DUP, then both data and metadata
> are duplicated, and you get half the space available.

In german I´d say "autsch", in english according to pda.leo.org "ouch", to 
this.

Okay, I just erased using mixed mode as an idea from my mind altogether :).

Just like I think I will never use a BTRFS below 5 GiB. Well, with one 
exception, maybe on the eMMC flash of the new Omnia Turris router that I hope 
will arrive soon at my place.

-- 
Martin

Re: Small fs

[Austin S. Hemmelgarn]

On 2016-09-11 15:51, Martin Steigerwald wrote:
> Am Sonntag, 11. September 2016, 19:46:32 CEST schrieb Hugo Mills:
>> On Sun, Sep 11, 2016 at 09:13:28PM +0200, Martin Steigerwald wrote:
>>> Am Sonntag, 11. September 2016, 16:44:23 CEST schrieb Duncan:
>>>> * Metadata, and thus mixed-bg, defaults to DUP mode on a single-device
>>>> filesystem (except on ssd where I actually still use it myself, and
>>>> recommend it except for ssds that do firmware dedupe).  In mixed-mode
>>>> this means two copies of data as well, which halves the usable space.
>>>>
>>>> IOW, when using mixed-mode, which is recommended under a gig, and dup
>>>> replication which is then the single-device default, effective usable
>>>> space is **HALVED**, so 256 MiB btrfs size becomes 128 MiB usable. (!!)
>>>
>>> I don´t get this part. That is just *metadata* being duplicated, not the
>>> actual *data* inside the files. Or am I missing something here?
>>
>>    In mixed mode, there's no distinction: Data and metadata both use
>> the same chunks. If those chunks are DUP, then both data and metadata
>> are duplicated, and you get half the space available.
>
> In german I´d say "autsch", in english according to pda.leo.org "ouch", to
> this.
>
> Okay, I just erased using mixed mode as an idea from my mind altogether :).
Until recently (when DUP mode for data on a single device got added), it 
did have advantages.  If you're running on a single disk and need 
replication, it's a lot more efficient to just run a single partition in 
DUP mode than two in RAID1 mode.  It also makes it a bit less likely to 
hit ENOSPC issues on small filesystems.
>
> Just like I think I will never use a BTRFS below 5 GiB. Well, with one
> exception, maybe on the eMMC flash of the new Omnia Turris router that I hope
> will arrive soon at my place.
>
FWIW, I don't use anything smaller than 8G unless I have to (with the 
notable exception of /boot), and when I do need to, I usually just use 
mixed mode because once you get that small, you need all the help you 
can get to keep from the whole FS being allocated to chunks.

Re: Small fs

[Chris Murphy]

On Sun, Sep 11, 2016 at 9:27 AM, Imran Geriskovan
<imran.geriskovan@gmail.com> wrote:
> What is the smallest recommended fs size for btrfs?

It depends on the layout. And there is some confusion about the mkfs
command message it returns when it doesn't work out.

https://bugzilla.kernel.org/show_bug.cgi?id=112461

Something else that's screwy in that bug that I just realized, why is
it not defaulting to mixed-block groups on a 100MiB fallocated file? I
thought mixed-bg was the default below a certain size like 2GiB or
whatever?


>
> - There are mentions of 256MB around the net.
> - Gparted reserves minimum of 256MB for btrfs.
>
> With an ordinary partition on a single disk,
> fs created with just "mkfs.btrfs /dev/sdxx":
> - 128MB works fine.
> - 127MB works but as if it is 64MB.
>
> Can we say size should be in multiples of 64MB?

Why should it be in multiples?  I think what you're describing is part
of the bug above that just needs to be fixed. Btrfs itself internally
uses bytes, so multiples of 64MiB is OK but I wouldn't  use the word
"should" with it.

-- 
Chris Murphy

Re: Small fs

[Duncan]

Chris Murphy posted on Sun, 11 Sep 2016 14:33:18 -0600 as excerpted:

> Something else that's screwy in that bug that I just realized, why is it
> not defaulting to mixed-block groups on a 100MiB fallocated file? I
> thought mixed-bg was the default below a certain size like 2GiB or
> whatever?

You apparently missed the memo...

Newer btrfs-progs mkfs.btrfs no longer defaults under-1-GiB to mixed-bg 
mode, tho it remains very strongly recommended below 1 GiB, and soft-
recommended to somewhere between 4 and 32 GiB (I believe the wiki says 5 
GiB at this point but don't know how it arrived at that, but the numbers 
I've seen suggested on-list range between 4 and 32 GiB, as above).

The explanation of why, based on the thread where I remember it coming 
up, was because defaulting to mixed-mode was making testing more 
complex.  Don't ask me to agree with that because I most certainly don't; 
IMO sane defaults for normal use, which everyone seems to agree mixed-
mode for under a GiB is, should apply, and if testing needs special-
cased, well, special-case it.  But none-the-less, that's the context in 
which it was agreed to do away with the mixed-mode default, despite it 
still being extremely strongly recommended for under a GiB.  <shrug>

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Small fs

[Chris Murphy]

On Sun, Sep 11, 2016 at 8:00 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> Chris Murphy posted on Sun, 11 Sep 2016 14:33:18 -0600 as excerpted:
>
>> Something else that's screwy in that bug that I just realized, why is it
>> not defaulting to mixed-block groups on a 100MiB fallocated file? I
>> thought mixed-bg was the default below a certain size like 2GiB or
>> whatever?
>
> You apparently missed the memo...

I got the memo right after I clicked send and read the thread.

> Newer btrfs-progs mkfs.btrfs no longer defaults under-1-GiB to mixed-bg
> mode, tho it remains very strongly recommended below 1 GiB, and soft-
> recommended to somewhere between 4 and 32 GiB (I believe the wiki says 5
> GiB at this point but don't know how it arrived at that, but the numbers
> I've seen suggested on-list range between 4 and 32 GiB, as above).
>
> The explanation of why, based on the thread where I remember it coming
> up, was because defaulting to mixed-mode was making testing more
> complex.  Don't ask me to agree with that because I most certainly don't;
> IMO sane defaults for normal use, which everyone seems to agree mixed-
> mode for under a GiB is, should apply, and if testing needs special-
> cased, well, special-case it.  But none-the-less, that's the context in
> which it was agreed to do away with the mixed-mode default, despite it
> still being extremely strongly recommended for under a GiB.  <shrug>

The man page says:
"The recommended size for the mixed mode is for filesystems less than
1GiB." But in this case recommended !=default which requires some
mental gymnastics to rectify. If mixed-bg becomes obsolete upon enospc
being no more likely with isolated block groups, then OK fine, but in
the meantime...


-- 
Chris Murphy

Re: Small fs

[Duncan]

Chris Murphy posted on Sun, 11 Sep 2016 21:03:04 -0600 as excerpted:

> The man page says:
> "The recommended size for the mixed mode is for filesystems less than
> 1GiB." But in this case recommended !=default which requires some mental
> gymnastics to rectify. If mixed-bg becomes obsolete upon enospc being no
> more likely with isolated block groups, then OK fine, but in the
> meantime...

On the bright side, the double-whammy of being under such tight 
filesystem size constraints, coupled with finding out you have less than 
half the space of the filesystem actually available due to default-mixed-
mode AND default dup-metadata (thus dup everything), gets eliminated, and 
barring problems with unbalanced chunk-sizes, you actually get to use 
most of capacity of the filesystem for actual files, instead of less than 
half of it.

=:^)

But I remain unconvinced that benefit outweighs the serious 
administrative headaches trying to run without mixed-mode on such small 
btrfs is likely to generate.  And what's worse, it's the same folks that 
are likely to have problems coping with either issue, but fixing the 
under-half-available-for-use (at the cost of filesystem resiliency) is a 
one-time thing, while the administrative issue of unbalanced chunks is 
likely to come back to bite them again and again.

But still, having people find they can fit only ~110 MiB in a 256 MiB 
btrfs, while with ext*, they can fit say 240 MiB in the same size 
filesystem, could be a bit more to try to explain to the technically 
under-literate, than the devs decided they were willing to deal with.  
Just saying it'd divided in chunks and another chunk won't fit is 
arguably easier than trying to explain why the filesystem will fit less 
than half of what the size of it suggests it should fit, IOW.

And I think that argument /was/ made, to some extent.  But the whole 
thing only came up because they found testing with small filesystems 
inconvenient due to the mixed-bg default, so rather than fix that by 
fixing the tests, they broke the previously sane defaults, instead.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Small fs

[Chris Murphy]

On Sun, Sep 11, 2016 at 10:54 PM, Duncan <1i5t5.duncan@cox.net> wrote:

> On the bright side, the double-whammy of being under such tight
> filesystem size constraints, coupled with finding out you have less than
> half the space of the filesystem actually available due to default-mixed-
> mode AND default dup-metadata (thus dup everything),

I'm not following what you mean by default dup everything. You mean
that's how it used to work? Because on a real USB stick, 2GiB:


[chris@f24m ~]$ mkfs.btrfs -M /dev/sdb
btrfs-progs v4.7.2
See http://btrfs.wiki.kernel.org for more information.

probe of /dev/sdb failed, cannot detect existing filesystem.
ERROR: use the -f option to force overwrite of /dev/sdb
[chris@f24m ~]$ sudo mkfs.btrfs -M /dev/sdb
btrfs-progs v4.7.2
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               237d8ab9-b22f-435c-bbd8-2117afa1760c
Node size:          4096
Sector size:        4096
Filesystem size:    1.87GiB
Block group profiles:
  Data+Metadata:    single            8.00MiB
  System:           single            4.00MiB
SSD detected:       no
Incompat features:  mixed-bg, extref, skinny-metadata
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1     1.87GiB  /dev/sdb



Whereas without the -M


[chris@f24m ~]$ sudo mkfs.btrfs -f /dev/sdb
[sudo] password for chris:
btrfs-progs v4.7.2
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               a7c79550-10b8-4a4c-9b23-9c9110e30121
Node size:          16384
Sector size:        4096
Filesystem size:    1.87GiB
Block group profiles:
  Data:             single            8.00MiB
  Metadata:         DUP              95.56MiB
  System:           DUP               8.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1     1.87GiB  /dev/sdb



So with -M, it's single by default..


-- 
Chris Murphy

Re: Small fs

[Duncan]

Chris Murphy posted on Mon, 12 Sep 2016 08:48:49 -0600 as excerpted:

> On Sun, Sep 11, 2016 at 10:54 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> 
>> On the bright side, the double-whammy of being under such tight
>> filesystem size constraints, coupled with finding out you have less
>> than half the space of the filesystem actually available due to
>> default-mixed-mode AND default dup-metadata (thus dup everything),
> 
> I'm not following what you mean by default dup everything. You mean
> that's how it used to work? Because on a real USB stick, 2GiB:


Yes.  What I meant is that now that --mixed isn't default, it avoids auto-
halving the space due to duping the data as well (which AFAIK it used to 
do by default since mixed-mode treated everything as metadata, and 
metadata defaults are dup except on ssd), in mixed-mode.

Tho now that I think of it, it's possible I'm mistaken, since I always 
specify single/dup/raid1/whatever, as well as mixed if I want it, 
choosing not to rely on the defaults, myself.

Which seems a pretty wise choice, with the defaults changing over time, 
sometimes, as with removal of the old default mixed under a GiB, to 
strongly dis-recommended defaults. =:^(

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Small fs

[Imran Geriskovan]

On 9/11/16, Chris Murphy <lists@colorremedies.com> wrote:
> Something else that's screwy in that bug that I just realized, why is
> it not defaulting to mixed-block groups on a 100MiB fallocated file? I
> thought mixed-bg was the default below a certain size like 2GiB or
> whatever?

>> With an ordinary partition on a single disk,
>> fs created with just "mkfs.btrfs /dev/sdxx":
>> - 128MB works fine.
>> - 127MB works but as if it is 64MB.
>> Can we say size should be in multiples of 64MB?

> Why should it be in multiples?  I think what you're describing is part
> of the bug above that just needs to be fixed. Btrfs itself internally
> uses bytes, so multiples of 64MiB is OK but I wouldn't  use the word
> "should" with it.

I'm not suggesting anything. I'm just describing the behaviour
we've seen. If it is (or will be) something different its all ok for me.

But, what is that "formal behaviour" at the low end?
That is the discussion..

Re: Small fs

[Austin S. Hemmelgarn]

On 2016-09-12 08:54, Imran Geriskovan wrote:
> On 9/11/16, Chris Murphy <lists@colorremedies.com> wrote:
>> Something else that's screwy in that bug that I just realized, why is
>> it not defaulting to mixed-block groups on a 100MiB fallocated file? I
>> thought mixed-bg was the default below a certain size like 2GiB or
>> whatever?
>
>>> With an ordinary partition on a single disk,
>>> fs created with just "mkfs.btrfs /dev/sdxx":
>>> - 128MB works fine.
>>> - 127MB works but as if it is 64MB.
>>> Can we say size should be in multiples of 64MB?
>
>> Why should it be in multiples?  I think what you're describing is part
>> of the bug above that just needs to be fixed. Btrfs itself internally
>> uses bytes, so multiples of 64MiB is OK but I wouldn't  use the word
>> "should" with it.
>
> I'm not suggesting anything. I'm just describing the behaviour
> we've seen. If it is (or will be) something different its all ok for me.
>
> But, what is that "formal behaviour" at the low end?
> That is the discussion..
Formally, with mixed-bg and otherwise default options, you should be 
able to get to about 48-49% of the total size of the FS worth of data 
stored.  In practice, I see such filesystems start having issues at 
about 45% of the total size.  Note that this is not the percentages 
reported by regular df (which should say about 90% full in this case), b 
ut a comparison based on du -sh and the size of the partition.