test btrfs scrubbing/get system's total capacity

test btrfs scrubbing/get system's total capacity

[ST]

Hello,

1. is there a tool that I can use to test btrfs scrubbing? Namely I need
to "damage" a file, i.e. change it without changing its meta
information, so I can see how scrubbing performs... Is there something
like this?


2. I want to use Ansible to set btrfs quota on a /home subvolume as say
0.8*TotalCapacityOfRoot and then assign to it individual users'
subvolumes with certain (smaller) quatos.
2.1 Is it OK to rely on ansible_facts? Does it has correct information
about btrfs-based "/" total capacity?
2.2 If not, is there a raw command to get TotalCapacityOfRoot in -TB so
I can set it? Is there something like btrfs filesystem usage / -t
--device_size? to get only the total Device Size information?

Thank you!
ST

Re: test btrfs scrubbing/get system's total capacity

[Chris Murphy]

On Mon, Nov 20, 2017 at 9:50 AM, ST <smntov@gmail.com> wrote:
> Hello,
>
> 1. is there a tool that I can use to test btrfs scrubbing? Namely I need
> to "damage" a file, i.e. change it without changing its meta
> information, so I can see how scrubbing performs... Is there something
> like this?


btrfs-corrupt-block.c

It's part of btrfs-progs but many distributions do not include it by
default, but is easy to build it stand alone.



> 2. I want to use Ansible to set btrfs quota on a /home subvolume as say
> 0.8*TotalCapacityOfRoot and then assign to it individual users'
> subvolumes with certain (smaller) quatos.
> 2.1 Is it OK to rely on ansible_facts? Does it has correct information
> about btrfs-based "/" total capacity?
> 2.2 If not, is there a raw command to get TotalCapacityOfRoot in -TB so
> I can set it? Is there something like btrfs filesystem usage / -t
> --device_size? to get only the total Device Size information?

No idea how ansible is manipulating quotas on Btrfs.


-- 
Chris Murphy

Re: test btrfs scrubbing/get system's total capacity

[Duncan]

ST posted on Mon, 20 Nov 2017 18:50:26 +0200 as excerpted:

> [...] to set btrfs quota on a /home subvolume as say
> 0.8*TotalCapacityOfRoot and then assign to it individual users'
> subvolumes with certain (smaller) quatos.

On the narrow subject of btrfs quotas...

Just be sure you're aware of the effect of btrfs quotas on btrfs 
maintenance command times, and are using a "newish" kernel, as...

* For years quotas were so buggy that they simply weren't sanely reliable 
(negative numbers, anyone?).  In general those bugs are fixed in newer 
kernels, but there's still minor tweaks going in.  I'd for sure want at 
/least/ 4.9-LTS series if I were relying on kernels (and I'm not 
/entirely/ sure the appropriate patches have all made it there), and 
would want to move to 4.14-LTS ASAP.  Or just use the current kernel 
series and upgrade with it.

IOW, don't even consider quotas pre-4.9.  They're too buggy to be worth 
the hassle.

* While the worst quota bugs now appear to be quashed and it's at least 
usable (the numbers come out correct), quotas continue to have scaling 
issues and likely will for some time to come.

Generally these scaling issues don't affect generic filesystem (or quota) 
operation, but they *do* affect various btrfs maintenance commands where 
they compound already severe scaling interactions with snapshots and 
reflinks in commands such as btrfs subvol delete, btrfs balance (and thus 
btrfs device remove due to its implicit balance), and btrfs check, to the 
point that the commands cannot complete in practical time (days to weeks, 
even months, instead of hours).

For commands such as btrfs balance, temporarily disabling quotas (renable 
and rescan if necessary when done) speeds things up dramatically, but 
that's not likely to be possible in many of the scenarios one would find 
themselves wanting to run btrfs check in.


If you're already aware of and prepared to work with those caveats, 
great, but I've seen enough people complaining about slow balance, etc, 
due to having quotas enabled (and being thrilled with the actually 
workable speed once they turn them off) when they didn't really need 
them, that I tend to cringe inside every time I see people saying they 
plan on using them, without mentioning that they're aware of the issues 
and find them an acceptable tradeoff for their use-case.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: test btrfs scrubbing/get system's total capacity

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 1636 bytes --]



On 2017年11月21日 00:50, ST wrote:
> Hello,
> 
> 1. is there a tool that I can use to test btrfs scrubbing? Namely I need
> to "damage" a file, i.e. change it without changing its meta
> information, so I can see how scrubbing performs... Is there something
> like this?

Btrfs-corrupt-block, which is not compiled by default.
And not well documented.

> 
> 
> 2. I want to use Ansible to set btrfs quota on a /home subvolume as say
> 0.8*TotalCapacityOfRoot and then assign to it individual users'
> subvolumes with certain (smaller) quatos.

Quota only works on subvolumes.
Although you could do it using multi level qgroups.

Using higher level qgroup, 1/0 for example, as a big pool and then
assign real subvolume qgroup to that pool.

However, current qgroup will bring big performance overhead, especially
for balance and snapshot deletion.

Although not sure what Ansible is and if it uses btrfs quota.


> 2.1 Is it OK to rely on ansible_facts? Does it has correct information
> about btrfs-based "/" total capacity?

Not sure about ansible at all.

> 2.2 If not, is there a raw command to get TotalCapacityOfRoot in -TB so
> I can set it? Is there something like btrfs filesystem usage / -t
> --device_size? to get only the total Device Size information?\

What is the meaning of TotalCapacityOfRoot? Device size? Fs size?

Thanks,
Qu

> 
> Thank you!
> ST
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]