What is the long term fix for the idmapper key-quota problem.

What is the long term fix for the idmapper key-quota problem.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]


Hi,
 as you probably know, request_key() imposes a quota on
the number of keys that can be requested, which by default is quite low (200).
When idmap exceeds this quota, request_key() returns an error and
the result is userspace sees "nobody" as the owner.

A short term fix is to fiddle some sysctl values, but I wonder if any long
term fix is being planned.

Probably the sensible thing would be for nfs-idmap to bypass the quota.
As keys are not held active for very long at a time, they should be garbage
collected in due course.
There is currently no interface to request this but I suspect one could be
added.

(I want to fixed this for openSUSE-13.1, and don't want to diverge too far
from mainline).

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: What is the long term fix for the idmapper key-quota problem.

[Anna Schumaker]

On Mon 21 Oct 2013 01:05:14 AM EDT, NeilBrown wrote:
>
> Hi,
>  as you probably know, request_key() imposes a quota on
> the number of keys that can be requested, which by default is quite low (200).
> When idmap exceeds this quota, request_key() returns an error and
> the result is userspace sees "nobody" as the owner.
>
> A short term fix is to fiddle some sysctl values, but I wonder if any long
> term fix is being planned.

I don't have anything planned.

>
> Probably the sensible thing would be for nfs-idmap to bypass the quota.
> As keys are not held active for very long at a time, they should be garbage
> collected in due course.
> There is currently no interface to request this but I suspect one could be
> added.

Adding a new keyring interface might be a better question for David 
Howells, since he's done a lot of keyring work.

How many idmap keys do you have active at once?  I'm guessing the quota 
is there for a reason, and increasing it in-kernel would probably be 
much easier than adding a new interface for short-lived keys.

>
> (I want to fixed this for openSUSE-13.1, and don't want to diverge too far
> from mainline).
>
> Thanks,
> NeilBrown

Re: What is the long term fix for the idmapper key-quota problem.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2537 bytes --]

On Tue, 22 Oct 2013 10:49:13 -0400 Anna Schumaker <bjschuma@netapp.com> wrote:

> On Mon 21 Oct 2013 01:05:14 AM EDT, NeilBrown wrote:
> >
> > Hi,
> >  as you probably know, request_key() imposes a quota on
> > the number of keys that can be requested, which by default is quite low (200).
> > When idmap exceeds this quota, request_key() returns an error and
> > the result is userspace sees "nobody" as the owner.
> >
> > A short term fix is to fiddle some sysctl values, but I wonder if any long
> > term fix is being planned.
> 
> I don't have anything planned.

Maybe it is time to start planning then.  This is a very real regression that
needs to be fixed.


> 
> >
> > Probably the sensible thing would be for nfs-idmap to bypass the quota.
> > As keys are not held active for very long at a time, they should be garbage
> > collected in due course.
> > There is currently no interface to request this but I suspect one could be
> > added.
> 
> Adding a new keyring interface might be a better question for David 
> Howells, since he's done a lot of keyring work.

Let's add David to Cc then.  Hi David!

> 
> How many idmap keys do you have active at once?  I'm guessing the quota 
> is there for a reason, and increasing it in-kernel would probably be 
> much easier than adding a new interface for short-lived keys.

Only 1 or 2 active at once.  Lots active sequentially in a short period of
time.

The problem, as I see it, is that you are trying to use a keyring like a
cache, and it doesn't seem to be designed for that purpose.

It certainly has some of the attributes of a cache, like an LRU list and a
garbage collector.  However the garbage collector doesn't discard entries
simply because they are old (as you would expect for a cache).  It discards
them because they have expired or have been marked as dead.

To get the keyring to act like a cache you need some way to discard the
"oldest" when the keyring is full.

Though as the quotas are per-user rather than per-keyring I'm not sure that
is really a well defined concept.

I was imagining something like
  while (request_key(&key_type_id_resolver, ...) == -EDQUOT)
        discard_oldest_key(&key_type_id_resolver);

but it is entirely possible that root's entire quota is used for some other
key_type so this could fail.

David: is there some way that keyrings can be used as a simple FIFO cache
with some minimum number of entries?  If not, this might require a major
redesign.


NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]