* [PATCH] gssd: validate cred in gssd_acquire_user_cred
@ 2013-10-25 17:09 Weston Andros Adamson
2013-10-28 12:44 ` Steve Dickson
0 siblings, 1 reply; 8+ messages in thread
From: Weston Andros Adamson @ 2013-10-25 17:09 UTC (permalink / raw)
To: steved; +Cc: linux-nfs, Weston Andros Adamson
Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
credentials.
This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
that causes the user's ticket cache to grow unbounded with expired service
tickets when the user's credentials expire.
To reproduce this issue:
- mount kerberos nfs export
- kinit for a short lifetime (ie "kinit -l 1m")
- run a job that opens a file and writes for more than the lifetime
- run klist a few times after expiry and see the list grow, ie:
Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
Default principal: dros@APIKIA.FAKE
Valid starting Expires Service principal
10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
Signed-off-by: Weston Andros Adamson <dros@netapp.com>
Reviewed-by: Simo Sorce <simo@redhat.com>
---
Version 2 of the patch - version 1 had compiler warnings I completely missed!
I wonder how it worked before... but this now compiles cleanly and I verified
that it fixes the growing tkt cache problem.
Thanks,
-dros
utils/gssd/krb5_util.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
index c6e52fd..697d1d2 100644
--- a/utils/gssd/krb5_util.c
+++ b/utils/gssd/krb5_util.c
@@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
ret = gssd_acquire_krb5_cred(name, gss_cred);
+ /* force validation of cred to check for expiry */
+ if (ret == 0) {
+ if (gss_inquire_cred(&min_stat, *gss_cred, NULL, NULL,
+ NULL, NULL) != GSS_S_COMPLETE)
+ ret = -1;
+ }
+
maj_stat = gss_release_name(&min_stat, &name);
return ret;
}
--
1.8.3.1 (Apple Git-46)
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH] gssd: validate cred in gssd_acquire_user_cred
2013-10-25 17:09 [PATCH] gssd: validate cred in gssd_acquire_user_cred Weston Andros Adamson
@ 2013-10-28 12:44 ` Steve Dickson
0 siblings, 0 replies; 8+ messages in thread
From: Steve Dickson @ 2013-10-28 12:44 UTC (permalink / raw)
To: Weston Andros Adamson; +Cc: linux-nfs
On 25/10/13 13:09, Weston Andros Adamson wrote:
> Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
> credentials.
>
> This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
> that causes the user's ticket cache to grow unbounded with expired service
> tickets when the user's credentials expire.
>
> To reproduce this issue:
>
> - mount kerberos nfs export
> - kinit for a short lifetime (ie "kinit -l 1m")
> - run a job that opens a file and writes for more than the lifetime
> - run klist a few times after expiry and see the list grow, ie:
>
> Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
> Default principal: dros@APIKIA.FAKE
>
> Valid starting Expires Service principal
> 10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
> 10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>
> Signed-off-by: Weston Andros Adamson <dros@netapp.com>
> Reviewed-by: Simo Sorce <simo@redhat.com>
Committed...
steved.
>
> ---
>
> Version 2 of the patch - version 1 had compiler warnings I completely missed!
> I wonder how it worked before... but this now compiles cleanly and I verified
> that it fixes the growing tkt cache problem.
>
> Thanks,
> -dros
>
>
> utils/gssd/krb5_util.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
> index c6e52fd..697d1d2 100644
> --- a/utils/gssd/krb5_util.c
> +++ b/utils/gssd/krb5_util.c
> @@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
>
> ret = gssd_acquire_krb5_cred(name, gss_cred);
>
> + /* force validation of cred to check for expiry */
> + if (ret == 0) {
> + if (gss_inquire_cred(&min_stat, *gss_cred, NULL, NULL,
> + NULL, NULL) != GSS_S_COMPLETE)
> + ret = -1;
> + }
> +
> maj_stat = gss_release_name(&min_stat, &name);
> return ret;
> }
> -- 1.8.3.1 (Apple Git-46)
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] gssd: validate cred in gssd_acquire_user_cred
@ 2013-10-22 14:04 Weston Andros Adamson
2013-10-22 14:41 ` Simo Sorce
0 siblings, 1 reply; 8+ messages in thread
From: Weston Andros Adamson @ 2013-10-22 14:04 UTC (permalink / raw)
To: SteveD; +Cc: simo, linux-nfs, Weston Andros Adamson
Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
credentials.
This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
that causes the user's ticket cache to grow unbounded with expired service
tickets when the user's credentials expire.
To reproduce this issue:
- mount kerberos nfs export
- kinit for a short lifetime (ie "kinit -l 1m")
- run a job that opens a file and writes for more than the lifetime
- run klist a few times after expiry and see the list grow, ie:
Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
Default principal: dros@APIKIA.FAKE
Valid starting Expires Service principal
10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
Signed-off-by: Weston Andros Adamson <dros@netapp.com>
---
utils/gssd/krb5_util.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
index c6e52fd..ec5db83 100644
--- a/utils/gssd/krb5_util.c
+++ b/utils/gssd/krb5_util.c
@@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
ret = gssd_acquire_krb5_cred(name, gss_cred);
+ /* force validation of cred to check for expiry */
+ if (ret == 0) {
+ if (gss_inquire_cred(min_stat, gss_cred, NULL, NULL,
+ NULL, NULL) != GSS_S_COMPLETE)
+ ret = -1;
+ }
+
maj_stat = gss_release_name(&min_stat, &name);
return ret;
}
--
1.7.12.4 (Apple Git-37)
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH] gssd: validate cred in gssd_acquire_user_cred
2013-10-22 14:04 Weston Andros Adamson
@ 2013-10-22 14:41 ` Simo Sorce
2013-10-22 16:03 ` Weston Andros Adamson
0 siblings, 1 reply; 8+ messages in thread
From: Simo Sorce @ 2013-10-22 14:41 UTC (permalink / raw)
To: Weston Andros Adamson; +Cc: SteveD, linux-nfs
On Tue, 2013-10-22 at 10:04 -0400, Weston Andros Adamson wrote:
> Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
> credentials.
>
> This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
> that causes the user's ticket cache to grow unbounded with expired service
> tickets when the user's credentials expire.
>
> To reproduce this issue:
>
> - mount kerberos nfs export
> - kinit for a short lifetime (ie "kinit -l 1m")
> - run a job that opens a file and writes for more than the lifetime
> - run klist a few times after expiry and see the list grow, ie:
>
> Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
> Default principal: dros@APIKIA.FAKE
>
> Valid starting Expires Service principal
> 10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
> 10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>
> Signed-off-by: Weston Andros Adamson <dros@netapp.com>
> ---
> utils/gssd/krb5_util.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
> index c6e52fd..ec5db83 100644
> --- a/utils/gssd/krb5_util.c
> +++ b/utils/gssd/krb5_util.c
> @@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
>
> ret = gssd_acquire_krb5_cred(name, gss_cred);
>
> + /* force validation of cred to check for expiry */
> + if (ret == 0) {
> + if (gss_inquire_cred(min_stat, gss_cred, NULL, NULL,
> + NULL, NULL) != GSS_S_COMPLETE)
> + ret = -1;
> + }
> +
> maj_stat = gss_release_name(&min_stat, &name);
> return ret;
> }
A good start, but given you are inquiring creds, then I think it would
totally make sense to pass in a uint32_t for the 4th argument
(lifetime), and check if there is "enough".
For example, if it returns a lifetime of 1 second should we continue ?
There is a fat chance that it will fail later on.
I think we should at least log it if the credential we are trying to use
turns out to be really close to expiring, no ? It may save some gray
hairs to server administrators trying to find out what is going wrong
(like it happened to you :)
Simo.
--
Simo Sorce * Red Hat, Inc * New York
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gssd: validate cred in gssd_acquire_user_cred
2013-10-22 14:41 ` Simo Sorce
@ 2013-10-22 16:03 ` Weston Andros Adamson
2013-10-22 16:11 ` Weston Andros Adamson
2013-10-22 16:13 ` Simo Sorce
0 siblings, 2 replies; 8+ messages in thread
From: Weston Andros Adamson @ 2013-10-22 16:03 UTC (permalink / raw)
To: Simo Sorce; +Cc: <SteveD@redhat.com>, <linux-nfs@vger.kernel.org>
On Oct 22, 2013, at 10:41 AM, Simo Sorce <simo@redhat.com> wrote:
> On Tue, 2013-10-22 at 10:04 -0400, Weston Andros Adamson wrote:
>> Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
>> credentials.
>>
>> This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
>> that causes the user's ticket cache to grow unbounded with expired service
>> tickets when the user's credentials expire.
>>
>> To reproduce this issue:
>>
>> - mount kerberos nfs export
>> - kinit for a short lifetime (ie "kinit -l 1m")
>> - run a job that opens a file and writes for more than the lifetime
>> - run klist a few times after expiry and see the list grow, ie:
>>
>> Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
>> Default principal: dros@APIKIA.FAKE
>>
>> Valid starting Expires Service principal
>> 10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
>> 10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>
>> Signed-off-by: Weston Andros Adamson <dros@netapp.com>
>> ---
>> utils/gssd/krb5_util.c | 7 +++++++
>> 1 file changed, 7 insertions(+)
>>
>> diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
>> index c6e52fd..ec5db83 100644
>> --- a/utils/gssd/krb5_util.c
>> +++ b/utils/gssd/krb5_util.c
>> @@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
>>
>> ret = gssd_acquire_krb5_cred(name, gss_cred);
>>
>> + /* force validation of cred to check for expiry */
>> + if (ret == 0) {
>> + if (gss_inquire_cred(min_stat, gss_cred, NULL, NULL,
>> + NULL, NULL) != GSS_S_COMPLETE)
>> + ret = -1;
>> + }
>> +
>> maj_stat = gss_release_name(&min_stat, &name);
>> return ret;
>> }
>
> A good start, but given you are inquiring creds, then I think it would
> totally make sense to pass in a uint32_t for the 4th argument
> (lifetime), and check if there is "enough".
Well, from my perspective gssd already gives enough info for you to know what's going on, i.e.:
getting credentials for client with uid 1749600001 for server zero.apikia.fake
CC '/run/user/1749600001/krb5cc' being considered, with preferred realm 'APIKIA.FAKE'
CC 'DIR:/run/user/1749600001/krb5cc' is expired or corrupt
WARNING: Failed to create krb5 context for user with uid 1749600001 for server zero.apikia.fake
doing error downfall
And it's obvious from the NFS layer as EKEYEXPIRED gets passed through as the error.
>
> For example, if it returns a lifetime of 1 second should we continue ?
> There is a fat chance that it will fail later on.
I think how we handle this now (without the regression this fixes) is fine. I really don't think users/ admins want to enable verbose debugging in gssd to see this - they should be able to tell what happened when the caller gets an expired cred.
>
> I think we should at least log it if the credential we are trying to use
> turns out to be really close to expiring, no ? It may save some gray
> hairs to server administrators trying to find out what is going wrong
> (like it happened to you :)
Well, it was obvious to me what was happening when the keys expired - what was not obvious was why the tkt cache was growing unbounded with service tickets that were *never* valid (the regression this patch fixes).
I suppose we could do something more, but I don't really see a reason to.
-dros
>
> Simo.
>
> --
> Simo Sorce * Red Hat, Inc * New York
>
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gssd: validate cred in gssd_acquire_user_cred
2013-10-22 16:03 ` Weston Andros Adamson
@ 2013-10-22 16:11 ` Weston Andros Adamson
2013-10-22 16:13 ` Simo Sorce
1 sibling, 0 replies; 8+ messages in thread
From: Weston Andros Adamson @ 2013-10-22 16:11 UTC (permalink / raw)
To: Simo Sorce; +Cc: <SteveD@redhat.com>, <linux-nfs@vger.kernel.org>
On Oct 22, 2013, at 12:03 PM, Weston Andros Adamson <dros@netapp.com> wrote:
>
> On Oct 22, 2013, at 10:41 AM, Simo Sorce <simo@redhat.com> wrote:
>
>> On Tue, 2013-10-22 at 10:04 -0400, Weston Andros Adamson wrote:
>>> Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
>>> credentials.
>>>
>>> This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
>>> that causes the user's ticket cache to grow unbounded with expired service
>>> tickets when the user's credentials expire.
>>>
>>> To reproduce this issue:
>>>
>>> - mount kerberos nfs export
>>> - kinit for a short lifetime (ie "kinit -l 1m")
>>> - run a job that opens a file and writes for more than the lifetime
>>> - run klist a few times after expiry and see the list grow, ie:
>>>
>>> Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
>>> Default principal: dros@APIKIA.FAKE
>>>
>>> Valid starting Expires Service principal
>>> 10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
>>> 10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>
>>> Signed-off-by: Weston Andros Adamson <dros@netapp.com>
>>> ---
>>> utils/gssd/krb5_util.c | 7 +++++++
>>> 1 file changed, 7 insertions(+)
>>>
>>> diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
>>> index c6e52fd..ec5db83 100644
>>> --- a/utils/gssd/krb5_util.c
>>> +++ b/utils/gssd/krb5_util.c
>>> @@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
>>>
>>> ret = gssd_acquire_krb5_cred(name, gss_cred);
>>>
>>> + /* force validation of cred to check for expiry */
>>> + if (ret == 0) {
>>> + if (gss_inquire_cred(min_stat, gss_cred, NULL, NULL,
>>> + NULL, NULL) != GSS_S_COMPLETE)
>>> + ret = -1;
>>> + }
>>> +
>>> maj_stat = gss_release_name(&min_stat, &name);
>>> return ret;
>>> }
>>
>> A good start, but given you are inquiring creds, then I think it would
>> totally make sense to pass in a uint32_t for the 4th argument
>> (lifetime), and check if there is "enough".
>
> Well, from my perspective gssd already gives enough info for you to know what's going on, i.e.:
>
> getting credentials for client with uid 1749600001 for server zero.apikia.fake
> CC '/run/user/1749600001/krb5cc' being considered, with preferred realm 'APIKIA.FAKE'
> CC 'DIR:/run/user/1749600001/krb5cc' is expired or corrupt
> WARNING: Failed to create krb5 context for user with uid 1749600001 for server zero.apikia.fake
> doing error downfall
>
> And it's obvious from the NFS layer as EKEYEXPIRED gets passed through as the error.
>
>>
>> For example, if it returns a lifetime of 1 second should we continue ?
>> There is a fat chance that it will fail later on.
>
> I think how we handle this now (without the regression this fixes) is fine. I really don't think users/ admins want to enable verbose debugging in gssd to see this - they should be able to tell what happened when the caller gets an expired cred.
>
>>
>> I think we should at least log it if the credential we are trying to use
>> turns out to be really close to expiring, no ? It may save some gray
>> hairs to server administrators trying to find out what is going wrong
>> (like it happened to you :)
>
> Well, it was obvious to me what was happening when the keys expired - what was not obvious was why the tkt cache was growing unbounded with service tickets that were *never* valid (the regression this patch fixes).
>
> I suppose we could do something more, but I don't really see a reason to.
.. and if we did, it would be a new patch (/patchset) - this patch fixes the regression to make gssd behave as it did before 302de786930a2c533068f9d8909a817b40f07c32.
-dros
>
> -dros
>
>>
>> Simo.
>>
>> --
>> Simo Sorce * Red Hat, Inc * New York
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gssd: validate cred in gssd_acquire_user_cred
2013-10-22 16:03 ` Weston Andros Adamson
2013-10-22 16:11 ` Weston Andros Adamson
@ 2013-10-22 16:13 ` Simo Sorce
2013-10-22 16:22 ` Weston Andros Adamson
1 sibling, 1 reply; 8+ messages in thread
From: Simo Sorce @ 2013-10-22 16:13 UTC (permalink / raw)
To: Weston Andros Adamson
Cc: <SteveD@redhat.com>, <linux-nfs@vger.kernel.org>
On Tue, 2013-10-22 at 16:03 +0000, Weston Andros Adamson wrote:
> On Oct 22, 2013, at 10:41 AM, Simo Sorce <simo@redhat.com> wrote:
>
> > On Tue, 2013-10-22 at 10:04 -0400, Weston Andros Adamson wrote:
> >> Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
> >> credentials.
> >>
> >> This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
> >> that causes the user's ticket cache to grow unbounded with expired service
> >> tickets when the user's credentials expire.
> >>
> >> To reproduce this issue:
> >>
> >> - mount kerberos nfs export
> >> - kinit for a short lifetime (ie "kinit -l 1m")
> >> - run a job that opens a file and writes for more than the lifetime
> >> - run klist a few times after expiry and see the list grow, ie:
> >>
> >> Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
> >> Default principal: dros@APIKIA.FAKE
> >>
> >> Valid starting Expires Service principal
> >> 10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
> >> 10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
> >>
> >> Signed-off-by: Weston Andros Adamson <dros@netapp.com>
> >> ---
> >> utils/gssd/krb5_util.c | 7 +++++++
> >> 1 file changed, 7 insertions(+)
> >>
> >> diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
> >> index c6e52fd..ec5db83 100644
> >> --- a/utils/gssd/krb5_util.c
> >> +++ b/utils/gssd/krb5_util.c
> >> @@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
> >>
> >> ret = gssd_acquire_krb5_cred(name, gss_cred);
> >>
> >> + /* force validation of cred to check for expiry */
> >> + if (ret == 0) {
> >> + if (gss_inquire_cred(min_stat, gss_cred, NULL, NULL,
> >> + NULL, NULL) != GSS_S_COMPLETE)
> >> + ret = -1;
> >> + }
> >> +
> >> maj_stat = gss_release_name(&min_stat, &name);
> >> return ret;
> >> }
> >
> > A good start, but given you are inquiring creds, then I think it would
> > totally make sense to pass in a uint32_t for the 4th argument
> > (lifetime), and check if there is "enough".
>
> Well, from my perspective gssd already gives enough info for you to know what's going on, i.e.:
>
> getting credentials for client with uid 1749600001 for server zero.apikia.fake
> CC '/run/user/1749600001/krb5cc' being considered, with preferred realm 'APIKIA.FAKE'
> CC 'DIR:/run/user/1749600001/krb5cc' is expired or corrupt
> WARNING: Failed to create krb5 context for user with uid 1749600001 for server zero.apikia.fake
> doing error downfall
>
> And it's obvious from the NFS layer as EKEYEXPIRED gets passed through as the error.
>
> >
> > For example, if it returns a lifetime of 1 second should we continue ?
> > There is a fat chance that it will fail later on.
>
> I think how we handle this now (without the regression this fixes) is fine. I really don't think users/ admins want to enable verbose debugging in gssd to see this - they should be able to tell what happened when the caller gets an expired cred.
>
> >
> > I think we should at least log it if the credential we are trying to use
> > turns out to be really close to expiring, no ? It may save some gray
> > hairs to server administrators trying to find out what is going wrong
> > (like it happened to you :)
>
> Well, it was obvious to me what was happening when the keys expired - what was not obvious was why the tkt cache was growing unbounded with service tickets that were *never* valid (the regression this patch fixes).
>
> I suppose we could do something more, but I don't really see a reason to.
you are probably right,
Reviewed-by: Simo Sorce <simo@redhat.com>
Simo.
--
Simo Sorce * Red Hat, Inc * New York
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gssd: validate cred in gssd_acquire_user_cred
2013-10-22 16:13 ` Simo Sorce
@ 2013-10-22 16:22 ` Weston Andros Adamson
0 siblings, 0 replies; 8+ messages in thread
From: Weston Andros Adamson @ 2013-10-22 16:22 UTC (permalink / raw)
To: Simo Sorce; +Cc: <SteveD@redhat.com>, <linux-nfs@vger.kernel.org>
On Oct 22, 2013, at 12:13 PM, Simo Sorce <simo@redhat.com> wrote:
> On Tue, 2013-10-22 at 16:03 +0000, Weston Andros Adamson wrote:
>> On Oct 22, 2013, at 10:41 AM, Simo Sorce <simo@redhat.com> wrote:
>>
>>> On Tue, 2013-10-22 at 10:04 -0400, Weston Andros Adamson wrote:
>>>> Call gss_inquire_cred after gssd_acquire_krb5_cred check for expired
>>>> credentials.
>>>>
>>>> This fixes a recent regression (since 302de786930a2c533068f9d8909a817b40f07c32)
>>>> that causes the user's ticket cache to grow unbounded with expired service
>>>> tickets when the user's credentials expire.
>>>>
>>>> To reproduce this issue:
>>>>
>>>> - mount kerberos nfs export
>>>> - kinit for a short lifetime (ie "kinit -l 1m")
>>>> - run a job that opens a file and writes for more than the lifetime
>>>> - run klist a few times after expiry and see the list grow, ie:
>>>>
>>>> Ticket cache: DIR::/run/user/1749600001/krb5cc/tktYmpGlX
>>>> Default principal: dros@APIKIA.FAKE
>>>>
>>>> Valid starting Expires Service principal
>>>> 10/21/2013 15:39:38 10/21/2013 15:40:35 krbtgt/APIKIA.FAKE@APIKIA.FAKE
>>>> 10/21/2013 15:39:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:35 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:36 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:37 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:38 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:39 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:40 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:41 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>> 10/21/2013 15:40:42 10/21/2013 15:40:35 nfs/zero.apikia.fake@APIKIA.FAKE
>>>>
>>>> Signed-off-by: Weston Andros Adamson <dros@netapp.com>
>>>> ---
>>>> utils/gssd/krb5_util.c | 7 +++++++
>>>> 1 file changed, 7 insertions(+)
>>>>
>>>> diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c
>>>> index c6e52fd..ec5db83 100644
>>>> --- a/utils/gssd/krb5_util.c
>>>> +++ b/utils/gssd/krb5_util.c
>>>> @@ -1405,6 +1405,13 @@ gssd_acquire_user_cred(uid_t uid, gss_cred_id_t *gss_cred)
>>>>
>>>> ret = gssd_acquire_krb5_cred(name, gss_cred);
>>>>
>>>> + /* force validation of cred to check for expiry */
>>>> + if (ret == 0) {
>>>> + if (gss_inquire_cred(min_stat, gss_cred, NULL, NULL,
>>>> + NULL, NULL) != GSS_S_COMPLETE)
>>>> + ret = -1;
>>>> + }
>>>> +
>>>> maj_stat = gss_release_name(&min_stat, &name);
>>>> return ret;
>>>> }
>>>
>>> A good start, but given you are inquiring creds, then I think it would
>>> totally make sense to pass in a uint32_t for the 4th argument
>>> (lifetime), and check if there is "enough".
>>
>> Well, from my perspective gssd already gives enough info for you to know what's going on, i.e.:
>>
>> getting credentials for client with uid 1749600001 for server zero.apikia.fake
>> CC '/run/user/1749600001/krb5cc' being considered, with preferred realm 'APIKIA.FAKE'
>> CC 'DIR:/run/user/1749600001/krb5cc' is expired or corrupt
>> WARNING: Failed to create krb5 context for user with uid 1749600001 for server zero.apikia.fake
>> doing error downfall
>>
>> And it's obvious from the NFS layer as EKEYEXPIRED gets passed through as the error.
>>
>>>
>>> For example, if it returns a lifetime of 1 second should we continue ?
>>> There is a fat chance that it will fail later on.
>>
>> I think how we handle this now (without the regression this fixes) is fine. I really don't think users/ admins want to enable verbose debugging in gssd to see this - they should be able to tell what happened when the caller gets an expired cred.
>>
>>>
>>> I think we should at least log it if the credential we are trying to use
>>> turns out to be really close to expiring, no ? It may save some gray
>>> hairs to server administrators trying to find out what is going wrong
>>> (like it happened to you :)
>>
>> Well, it was obvious to me what was happening when the keys expired - what was not obvious was why the tkt cache was growing unbounded with service tickets that were *never* valid (the regression this patch fixes).
>>
>> I suppose we could do something more, but I don't really see a reason to.
>
>
> you are probably right,
>
> Reviewed-by: Simo Sorce <simo@redhat.com>
Cool, thanks for the help! I'm sure it would have taken me a long time to track down gss_inquire_cred() on my own.
I'm definitely open to improving the debuggability of gssd, so if you really think we need some warning during some window before the cred expires, let's work with the list to figure out how that should look.
-dros
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2013-10-28 12:43 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-10-25 17:09 [PATCH] gssd: validate cred in gssd_acquire_user_cred Weston Andros Adamson
2013-10-28 12:44 ` Steve Dickson
-- strict thread matches above, loose matches on Subject: below --
2013-10-22 14:04 Weston Andros Adamson
2013-10-22 14:41 ` Simo Sorce
2013-10-22 16:03 ` Weston Andros Adamson
2013-10-22 16:11 ` Weston Andros Adamson
2013-10-22 16:13 ` Simo Sorce
2013-10-22 16:22 ` Weston Andros Adamson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).