Re: [PATCH v2 1/3] quota: add quota in-memory format support

["Darrick J. Wong" <djwong@kernel.org>]

On Wed, Nov 23, 2022 at 09:36:15AM +0100, Lukas Czerner wrote:
> On Tue, Nov 22, 2022 at 11:58:33PM -0800, Christoph Hellwig wrote:
> > On Tue, Nov 22, 2022 at 03:21:17PM +0100, Lukas Czerner wrote:
> > > > That seems like a good idea for memory usage, but I think this might
> > > > also make the code much simpler, as that just requires fairly trivial
> > > > quota_read and quota_write methods in the shmem code instead of new
> > > > support for an in-memory quota file.
> > > 
> > > You mean like the implementation in the v1 ?
> > 
> > Having now found it: yes.
> > 
> 
> Jan,
> 
> do you have any argument for this, since it was your suggestion?
> 
> I also think that the implementation is much simpler with in-memory
> dquots because we will avoid all the hassle with creating and
> maintaining quota file in a proper format. It's not just reads and
> writes it's the entire machinery befind it in quota_v2.c and quota_tree.c.
> 
> But it is true that even with only user modified dquots being
> non-reclaimable until unmount it could theoreticaly represent a
> substantial memory consumption. Although I do wonder if this problem
> is even real. How many user/group ids would you expect extremely heavy
> quota user would have the limits set for? 1k, 10k, million, or even
> more? Do you know?

The last time I checked, some of our container schedulers will heap
~1000 containers onto a single host(!!) at a time.  Assuming that a
container with a single container might map ~10 uids from the global
namespace, that's easily 10,000 at a time.  If the container runtime
only reuses global uid namespace when it runs out of namespace (i.e. it
doesn't immediately recycle them) then you could actually get up in the
millions or billions pretty easily.  The dquot counters would drop to
zero so you might still be able to reclaim the old ones, though it
sounds like you'd have to unset any per-dquot limits to get it to do
that.

That said, fsx in fstests will make all sorts of chown/chgrp calls,
which has lead to problems with the XFS quota files reaching their
maximum size (~580M per quota type) and filling up the whole fs.

--D

> -Lukas
>