* Linux NFSv4 security issue: client presents wrong user's credentials to NFS server @ 2014-10-07 2:47 Andrew J. Romero 2014-10-07 13:10 ` Benjamin Coddington 0 siblings, 1 reply; 6+ messages in thread From: Andrew J. Romero @ 2014-10-07 2:47 UTC (permalink / raw) To: linux-nfs@vger.kernel.org Cc: Michael K. Rosier, Briant S Lawson, Joseph W Klemencic, Lynn A Garren Hi It is common in certain cases for a Linux workstation administrator to create a shared account with a corresponding .k5login file. Kerberos principal strings for users are added to the .k5login and the corresponding users can logon as the local shared account using their Kerberos credentials. Once logged on to the workstation, the user's local identity (local to the workstation) is the UID of the local shared account; however, when accessing kerberized resources external to the workstation (for example NFSv4 volumes), the user's identity is expected to be the individual user's kerberos identity. I am seeing the following serious security issue: If N users (user1@REALM , user2@REALM ...userN@REALM) (who are listed sharusr's .k5login) all logon to workstation1 as sharusr and are all running sessions simultaneously, then, the GSS Context / kerberos credentials stored in and used by the NFS-client kernel code when processing NFS requests on behalf of a user, will be correct for 1 user and incorrect for N-1 users. Each of the N-1 users will not be able to access NFS server resources that only they have been granted access to; but, they will be able to access all NFS resources that the 1 user has been granted access to. (even if their kerberos identity has not been granted access). Even after each of the N-1 users destroys their credentials they still retain access to all NFS resources that the 1 user has been granted access to. Access will not be denied until the 1 user also destroys his / her credentials. It is my understanding that the RPC GSS kernel module on an NFS client system, keeps a GSS security context (copy of a user's credentials) that it uses when it communicates with the NFS server on the user's behalf. I believe that the kernel security context relates back to the user mode process solely by UID number. Would adding session ID to the relation resolve the issue ? Each Session for each user would then have a unique tuple (UID,SessionID) that would map to the *correct* GSS Security context in the kernel. Thanks Andy Romero romero@fnal.gov NAS / SAN Administrator, Fermilab ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Linux NFSv4 security issue: client presents wrong user's credentials to NFS server 2014-10-07 2:47 Linux NFSv4 security issue: client presents wrong user's credentials to NFS server Andrew J. Romero @ 2014-10-07 13:10 ` Benjamin Coddington 2014-10-07 13:36 ` Trond Myklebust 0 siblings, 1 reply; 6+ messages in thread From: Benjamin Coddington @ 2014-10-07 13:10 UTC (permalink / raw) To: Andrew J. Romero Cc: linux-nfs@vger.kernel.org, Michael K. Rosier, Briant S Lawson, Joseph W Klemencic, Lynn A Garren On Tue, 7 Oct 2014, Andrew J. Romero wrote: > Hi > > It is common in certain cases for > a Linux workstation administrator to > create a shared account with a > corresponding .k5login file. > > Kerberos principal strings for users > are added to the .k5login and the corresponding > users can logon as the local shared account using > their Kerberos credentials. > > Once logged on to the workstation, the user's > local identity (local to the workstation) is > the UID of the local shared account; however, > when accessing kerberized resources external > to the workstation (for example NFSv4 volumes), > the user's identity is expected to be the > individual user's kerberos identity. > > I am seeing the following serious security issue: > > If N users (user1@REALM , user2@REALM ...userN@REALM) > (who are listed sharusr's .k5login) all logon to > workstation1 as sharusr and are all running sessions > simultaneously, then, the GSS Context / kerberos credentials > stored in and used by the NFS-client kernel code when > processing NFS requests on behalf of a user, > will be correct for 1 user and incorrect for N-1 users. > > Each of the N-1 users will not be able to > access NFS server resources that only they have been > granted access to; but, they will be able > to access all NFS resources that the 1 user > has been granted access to. (even if their > kerberos identity has not been granted access). > > Even after each of the N-1 users destroys their credentials > they still retain access to all NFS resources > that the 1 user has been granted access to. > Access will not be denied until the 1 user also destroys > his / her credentials. > > It is my understanding that the RPC GSS kernel module > on an NFS client system, keeps a GSS security context > (copy of a user's credentials) that it uses when it > communicates with the NFS server on the user's behalf. > > I believe that the kernel security context > relates back to the user mode process solely by UID number. That is correct. > Would adding session ID to the relation resolve the issue ? > Each Session for each user would then have a unique tuple > (UID,SessionID) that would map to the *correct* GSS Security > context in the kernel. I think what could be done is to hang gss contexts off either session or process keyrings - then each process group would be able to own an identity independently of other processes running with the same UID. To do this the gssd upcall could be moved to a request-key mechanism, as there would need to be a way to pass along which krb5 credentials ought to be used to establish the context. I suppose that if the process already has krb5 credentials in a keyring ccache, those could be used - but I don't think that we could count on always having a keyring ccache available, and the fallback would be to try to guess which ccache to use based on UID as things are today. There would additionally be the situation where keyring ccaches cannot be used, and the main problem becomes how to have a process tell gssd which credential cache to use. Maybe instead of establishing contexts at file access time, they could be established deliberately beforehand with a userspace command that would specify which krb5 credential to use to create the context, and then the context would hang off the process keyrings. The client could then check to see if there's an existing keyring context, and if so use that, and if not do the upcall. Ben ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Linux NFSv4 security issue: client presents wrong user's credentials to NFS server 2014-10-07 13:10 ` Benjamin Coddington @ 2014-10-07 13:36 ` Trond Myklebust 2014-10-07 14:29 ` J. Bruce Fields 2014-10-08 19:39 ` Benjamin Coddington 0 siblings, 2 replies; 6+ messages in thread From: Trond Myklebust @ 2014-10-07 13:36 UTC (permalink / raw) To: Benjamin Coddington Cc: Andrew J. Romero, linux-nfs@vger.kernel.org, Michael K. Rosier, Briant S Lawson, Joseph W Klemencic, Lynn A Garren On Tue, Oct 7, 2014 at 9:10 AM, Benjamin Coddington <bcodding@redhat.com> wrote: > On Tue, 7 Oct 2014, Andrew J. Romero wrote: > >> Hi >> >> It is common in certain cases for >> a Linux workstation administrator to >> create a shared account with a >> corresponding .k5login file. >> >> Kerberos principal strings for users >> are added to the .k5login and the corresponding >> users can logon as the local shared account using >> their Kerberos credentials. >> >> Once logged on to the workstation, the user's >> local identity (local to the workstation) is >> the UID of the local shared account; however, >> when accessing kerberized resources external >> to the workstation (for example NFSv4 volumes), >> the user's identity is expected to be the >> individual user's kerberos identity. >> >> I am seeing the following serious security issue: >> >> If N users (user1@REALM , user2@REALM ...userN@REALM) >> (who are listed sharusr's .k5login) all logon to >> workstation1 as sharusr and are all running sessions >> simultaneously, then, the GSS Context / kerberos credentials >> stored in and used by the NFS-client kernel code when >> processing NFS requests on behalf of a user, >> will be correct for 1 user and incorrect for N-1 users. >> >> Each of the N-1 users will not be able to >> access NFS server resources that only they have been >> granted access to; but, they will be able >> to access all NFS resources that the 1 user >> has been granted access to. (even if their >> kerberos identity has not been granted access). >> >> Even after each of the N-1 users destroys their credentials >> they still retain access to all NFS resources >> that the 1 user has been granted access to. >> Access will not be denied until the 1 user also destroys >> his / her credentials. >> >> It is my understanding that the RPC GSS kernel module >> on an NFS client system, keeps a GSS security context >> (copy of a user's credentials) that it uses when it >> communicates with the NFS server on the user's behalf. >> >> I believe that the kernel security context >> relates back to the user mode process solely by UID number. > > > That is correct. > >> Would adding session ID to the relation resolve the issue ? >> Each Session for each user would then have a unique tuple >> (UID,SessionID) that would map to the *correct* GSS Security >> context in the kernel. > > > I think what could be done is to hang gss contexts off either session > or process keyrings - then each process group would be able to own an > identity independently of other processes running with the same UID. > > To do this the gssd upcall could be moved to a request-key mechanism, as > there would need to be a way to pass along which krb5 credentials ought > to be used to establish the context. I suppose that if the process > already has krb5 credentials in a keyring ccache, those could be used - > but I don't think that we could count on always having a keyring ccache > available, and the fallback would be to try to guess which ccache to use > based on UID as things are today. There would additionally be the > situation where keyring ccaches cannot be used, and the main problem > becomes how to have a process tell gssd which credential cache to use. The problem with the request key interface is that it is completely broken when applied to containers, since it only runs in the global init namespace context. Fixing that is a non-trivial exercise; you'd have to not only carry a full namespace context with the RPC credential, but also somehow apply it to the upcall thread. The rpc.gssd model is much easier to manage, since the daemon itself is started from the container's local init process. > Maybe instead of establishing contexts at file access time, they could > be established deliberately beforehand with a userspace command that > would specify which krb5 credential to use to create the context, and > then the context would hang off the process keyrings. The client could > then check to see if there's an existing keyring context, and if so use > that, and if not do the upcall. > The server is free to drop any GSS session at any time, depending on internal resource management requirements. A client must be able to recover from that scenario or risk being unable to write back data. Cheers Trond -- Trond Myklebust Linux NFS client maintainer, PrimaryData trond.myklebust@primarydata.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Linux NFSv4 security issue: client presents wrong user's credentials to NFS server 2014-10-07 13:36 ` Trond Myklebust @ 2014-10-07 14:29 ` J. Bruce Fields 2014-10-08 19:39 ` Benjamin Coddington 1 sibling, 0 replies; 6+ messages in thread From: J. Bruce Fields @ 2014-10-07 14:29 UTC (permalink / raw) To: Trond Myklebust Cc: Benjamin Coddington, Andrew J. Romero, linux-nfs@vger.kernel.org, Michael K. Rosier, Briant S Lawson, Joseph W Klemencic, Lynn A Garren, Jeff Layton On Tue, Oct 07, 2014 at 09:36:12AM -0400, Trond Myklebust wrote: > The problem with the request key interface is that it is completely > broken when applied to containers, since it only runs in the global > init namespace context. Fixing that is a non-trivial exercise; you'd > have to not only carry a full namespace context with the RPC > credential, but also somehow apply it to the upcall thread. This one's really keeping me up at night. Mainly because of the server reboot recovery stuff. The client v4 idmapping has this problem too, doesn't it? How are we going to decide if the user-helper containerization is doable or if it's just a hopeless case? Jeff seems optimistic about fixing it, and that'd be great, but if it's not going to happen then we need to give up and use daemons for this stuff. --b. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Linux NFSv4 security issue: client presents wrong user's credentials to NFS server 2014-10-07 13:36 ` Trond Myklebust 2014-10-07 14:29 ` J. Bruce Fields @ 2014-10-08 19:39 ` Benjamin Coddington 2014-10-08 19:47 ` Trond Myklebust 1 sibling, 1 reply; 6+ messages in thread From: Benjamin Coddington @ 2014-10-08 19:39 UTC (permalink / raw) To: Trond Myklebust Cc: Andrew J. Romero, linux-nfs@vger.kernel.org, Michael K. Rosier, Briant S Lawson, Joseph W Klemencic, Lynn A Garren On Tue, 7 Oct 2014, Trond Myklebust wrote: > On Tue, Oct 7, 2014 at 9:10 AM, Benjamin Coddington <bcodding@redhat.com> wrote: >> On Tue, 7 Oct 2014, Andrew J. Romero wrote: >> >>> Hi >>> >>> It is common in certain cases for >>> a Linux workstation administrator to >>> create a shared account with a >>> corresponding .k5login file. >>> >>> Kerberos principal strings for users >>> are added to the .k5login and the corresponding >>> users can logon as the local shared account using >>> their Kerberos credentials. >>> >>> Once logged on to the workstation, the user's >>> local identity (local to the workstation) is >>> the UID of the local shared account; however, >>> when accessing kerberized resources external >>> to the workstation (for example NFSv4 volumes), >>> the user's identity is expected to be the >>> individual user's kerberos identity. >>> >>> I am seeing the following serious security issue: >>> >>> If N users (user1@REALM , user2@REALM ...userN@REALM) >>> (who are listed sharusr's .k5login) all logon to >>> workstation1 as sharusr and are all running sessions >>> simultaneously, then, the GSS Context / kerberos credentials >>> stored in and used by the NFS-client kernel code when >>> processing NFS requests on behalf of a user, >>> will be correct for 1 user and incorrect for N-1 users. >>> >>> Each of the N-1 users will not be able to >>> access NFS server resources that only they have been >>> granted access to; but, they will be able >>> to access all NFS resources that the 1 user >>> has been granted access to. (even if their >>> kerberos identity has not been granted access). >>> >>> Even after each of the N-1 users destroys their credentials >>> they still retain access to all NFS resources >>> that the 1 user has been granted access to. >>> Access will not be denied until the 1 user also destroys >>> his / her credentials. >>> >>> It is my understanding that the RPC GSS kernel module >>> on an NFS client system, keeps a GSS security context >>> (copy of a user's credentials) that it uses when it >>> communicates with the NFS server on the user's behalf. >>> >>> I believe that the kernel security context >>> relates back to the user mode process solely by UID number. >> >> >> That is correct. >> >>> Would adding session ID to the relation resolve the issue ? >>> Each Session for each user would then have a unique tuple >>> (UID,SessionID) that would map to the *correct* GSS Security >>> context in the kernel. >> >> >> I think what could be done is to hang gss contexts off either session >> or process keyrings - then each process group would be able to own an >> identity independently of other processes running with the same UID. >> >> To do this the gssd upcall could be moved to a request-key mechanism, as >> there would need to be a way to pass along which krb5 credentials ought >> to be used to establish the context. I suppose that if the process >> already has krb5 credentials in a keyring ccache, those could be used - >> but I don't think that we could count on always having a keyring ccache >> available, and the fallback would be to try to guess which ccache to use >> based on UID as things are today. There would additionally be the >> situation where keyring ccaches cannot be used, and the main problem >> becomes how to have a process tell gssd which credential cache to use. > > The problem with the request key interface is that it is completely > broken when applied to containers, since it only runs in the global > init namespace context. Fixing that is a non-trivial exercise; you'd > have to not only carry a full namespace context with the RPC > credential, but also somehow apply it to the upcall thread. > > The rpc.gssd model is much easier to manage, since the daemon itself > is started from the container's local init process. That seems like something that might be fixed in the keys stuff instead of dragging the namespace along on the rpc cred, because it seems like a more general problem that request-key upcalls can execute outside their container.. but maybe that's exactly what request-key is supposed to do. I'll ask David what he thinks about it. Alternatively, the contexts could still hang off session or process keyrings but still use the existing upcall... How did the idmap upcall get moved over to the request-key interface without taking the container problem into account? >> Maybe instead of establishing contexts at file access time, they could >> be established deliberately beforehand with a userspace command that >> would specify which krb5 credential to use to create the context, and >> then the context would hang off the process keyrings. The client could >> then check to see if there's an existing keyring context, and if so use >> that, and if not do the upcall. >> > > The server is free to drop any GSS session at any time, depending on > internal resource management requirements. A client must be able to > recover from that scenario or risk being unable to write back data. Ah, right. Thank you for pointing that out. Ben ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Linux NFSv4 security issue: client presents wrong user's credentials to NFS server 2014-10-08 19:39 ` Benjamin Coddington @ 2014-10-08 19:47 ` Trond Myklebust 0 siblings, 0 replies; 6+ messages in thread From: Trond Myklebust @ 2014-10-08 19:47 UTC (permalink / raw) To: Benjamin Coddington Cc: Andrew J. Romero, linux-nfs@vger.kernel.org, Michael K. Rosier, Briant S Lawson, Joseph W Klemencic, Lynn A Garren On Wed, Oct 8, 2014 at 3:39 PM, Benjamin Coddington <bcodding@redhat.com> wrote: > > On Tue, 7 Oct 2014, Trond Myklebust wrote: > >> On Tue, Oct 7, 2014 at 9:10 AM, Benjamin Coddington <bcodding@redhat.com> >> wrote: >>> >>> On Tue, 7 Oct 2014, Andrew J. Romero wrote: >>> >>>> Hi >>>> >>>> It is common in certain cases for >>>> a Linux workstation administrator to >>>> create a shared account with a >>>> corresponding .k5login file. >>>> >>>> Kerberos principal strings for users >>>> are added to the .k5login and the corresponding >>>> users can logon as the local shared account using >>>> their Kerberos credentials. >>>> >>>> Once logged on to the workstation, the user's >>>> local identity (local to the workstation) is >>>> the UID of the local shared account; however, >>>> when accessing kerberized resources external >>>> to the workstation (for example NFSv4 volumes), >>>> the user's identity is expected to be the >>>> individual user's kerberos identity. >>>> >>>> I am seeing the following serious security issue: >>>> >>>> If N users (user1@REALM , user2@REALM ...userN@REALM) >>>> (who are listed sharusr's .k5login) all logon to >>>> workstation1 as sharusr and are all running sessions >>>> simultaneously, then, the GSS Context / kerberos credentials >>>> stored in and used by the NFS-client kernel code when >>>> processing NFS requests on behalf of a user, >>>> will be correct for 1 user and incorrect for N-1 users. >>>> >>>> Each of the N-1 users will not be able to >>>> access NFS server resources that only they have been >>>> granted access to; but, they will be able >>>> to access all NFS resources that the 1 user >>>> has been granted access to. (even if their >>>> kerberos identity has not been granted access). >>>> >>>> Even after each of the N-1 users destroys their credentials >>>> they still retain access to all NFS resources >>>> that the 1 user has been granted access to. >>>> Access will not be denied until the 1 user also destroys >>>> his / her credentials. >>>> >>>> It is my understanding that the RPC GSS kernel module >>>> on an NFS client system, keeps a GSS security context >>>> (copy of a user's credentials) that it uses when it >>>> communicates with the NFS server on the user's behalf. >>>> >>>> I believe that the kernel security context >>>> relates back to the user mode process solely by UID number. >>> >>> >>> >>> That is correct. >>> >>>> Would adding session ID to the relation resolve the issue ? >>>> Each Session for each user would then have a unique tuple >>>> (UID,SessionID) that would map to the *correct* GSS Security >>>> context in the kernel. >>> >>> >>> >>> I think what could be done is to hang gss contexts off either session >>> or process keyrings - then each process group would be able to own an >>> identity independently of other processes running with the same UID. >>> >>> To do this the gssd upcall could be moved to a request-key mechanism, as >>> there would need to be a way to pass along which krb5 credentials ought >>> to be used to establish the context. I suppose that if the process >>> already has krb5 credentials in a keyring ccache, those could be used - >>> but I don't think that we could count on always having a keyring ccache >>> available, and the fallback would be to try to guess which ccache to use >>> based on UID as things are today. There would additionally be the >>> situation where keyring ccaches cannot be used, and the main problem >>> becomes how to have a process tell gssd which credential cache to use. >> >> >> The problem with the request key interface is that it is completely >> broken when applied to containers, since it only runs in the global >> init namespace context. Fixing that is a non-trivial exercise; you'd >> have to not only carry a full namespace context with the RPC >> credential, but also somehow apply it to the upcall thread. >> >> The rpc.gssd model is much easier to manage, since the daemon itself >> is started from the container's local init process. > > > That seems like something that might be fixed in the keys stuff instead > of dragging the namespace along on the rpc cred, because it seems like a > more general problem that request-key upcalls can execute outside their > container.. but maybe that's exactly what request-key is supposed to > do. I'll ask David what he thinks about it. > > Alternatively, the contexts could still hang off session or process > keyrings but still use the existing upcall... > > How did the idmap upcall get moved over to the request-key interface > without taking the container problem into account? Timing is everything. We converted idmapper in 2010 just before the Parallels people came and wanted to do net namespaces in 2011. However you will note that we still do support the rpc.idmapd daemon, and that the interfaces it uses have been converted to support net namespace. >>> Maybe instead of establishing contexts at file access time, they could >>> be established deliberately beforehand with a userspace command that >>> would specify which krb5 credential to use to create the context, and >>> then the context would hang off the process keyrings. The client could >>> then check to see if there's an existing keyring context, and if so use >>> that, and if not do the upcall. >>> >> >> The server is free to drop any GSS session at any time, depending on >> internal resource management requirements. A client must be able to >> recover from that scenario or risk being unable to write back data. > > > Ah, right. Thank you for pointing that out. > > Ben -- Trond Myklebust Linux NFS client maintainer, PrimaryData trond.myklebust@primarydata.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-10-08 19:47 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-10-07 2:47 Linux NFSv4 security issue: client presents wrong user's credentials to NFS server Andrew J. Romero 2014-10-07 13:10 ` Benjamin Coddington 2014-10-07 13:36 ` Trond Myklebust 2014-10-07 14:29 ` J. Bruce Fields 2014-10-08 19:39 ` Benjamin Coddington 2014-10-08 19:47 ` Trond Myklebust
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.