Re: Do quota groups cost noticeable performance in 3.14?

[Duncan <1i5t5.duncan@cox.net>]

Duncan posted on Mon, 21 Apr 2014 05:44:54 +0000 as excerpted:

> Marc MERLIN posted on Sun, 20 Apr 2014 12:59:01 -0700 as excerpted:
> 
>> I was looking at using qgroups for my backup server, which will be
>> filled with millions of files in subvolumes with snapshots.
>> 
>> I read a warning that quota groups had performance issues, at least in
>> the past.
> 
> Yes.  Additionally, there were serious bugs [...]
> 
>> Is it still true?
> 
> Very good question.

New information.  See Josef Bacik's new thread:

Snapshot aware defrag and qgroups thoughts
Monday, 21 April, 7:55:46 -0700

http://permalink.gmane.org/gmane.comp.file-systems.btrfs/34405

Looks like he's going to be rewriting qgroups accounting to rework 
sequence numbers, as part of his work to get a reasonable scalable 
snapshot-aware-defrag.  But he's on paternity leave ATM, so my guess is 
that's iffy for the 3.16 commit-window and thus may not make it until 
3.17, which @ ~10 weeks a kernel cycle and being ~2 weeks past the 3.15 
commit window (we're on rc2), leaves us ~18 weeks until 3.17-rc1, early 
September.

Anyway, with that rewrite coming, unless you're really itchy to get into 
qgroups now, I'd wait until after that to dive in.

Meanwhile, his explanation of the present interaction between qgroups and 
the (currently disabled) snapshot-aware-defrag was an entirely new thing 
for me.  As I haven't any current need for qgroups I had somewhat walled 
that area off as something I didn't mess with or need to know about at 
this time, but his explanation certainly goes quite some way to 
explaining why snapshot-aware-defrag was so horribly bad for some people, 
those unlucky enough to be doing heavy snapshotting, with qgroups active, 
on very active heavy-internal-rewrite-pattern files.

I was already (and still) recommending a good snapshot thinning program 
for those doing automated snapshotting, keeping the number of snapshots 
per subvolume under 500 and preferably 200-300 max (quite reasonable with 
a good thinning setup, even with originally per-minute snapshots).  And I 
was already recommending that people keep large (>1 GiB) heavy-internal-
rewrite-pattern files NOCOW, on dedicated subvolumes to avoid 
snapshotting (using conventional backup for them).  But I had no /idea/ 
qgroups threw another geometric-scaling factor into the mix!

That definitely adds a new recommendation to the set -- avoid qgroups on 
subvolumes with heavy-internal-rewrite-pattern files.  And if you MUST 
qgroup OR heavily snapshot, choose one OR the other, DEFINITELY NOT 
BOTH!  At least until that qgroups accounting rewrite gets done.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman