kerberos / AD requirements, blueprint

kerberos / AD requirements, blueprint

[Sage Weil]

I started to write up a blueprint for kerberos / LDAP / AD support:

	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz

In case it isn't clear from that document, I only sort of know what I'm 
talking about here.  (This is largely based on what I managed to retain 
from a conversation with Andrew, but I suspect I got some of it wrong.)

For anyone working in environments where kerberos is in use, I am *very* 
interested in hearing what your requirements are for your environment.  In 
particular, are you using AD or LDAP or something else?  How do you expect 
RBAC to work for you?

We'll definitely have a slot for this at the upcoming CDS, but anything we 
can hash out before then will make that conversation more productive.

Thanks!
sage

Re: kerberos / AD requirements, blueprint

[mdw]

On Thu, Oct 16, 2014 at 12:57:12PM -0700, Sage Weil wrote:
> I started to write up a blueprint for kerberos / LDAP / AD support:
> 
> 	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> 
> In case it isn't clear from that document, I only sort of know what I'm 
> talking about here.  (This is largely based on what I managed to retain 
> from a conversation with Andrew, but I suspect I got some of it wrong.)
> 
> For anyone working in environments where kerberos is in use, I am *very* 
> interested in hearing what your requirements are for your environment.  In 
> particular, are you using AD or LDAP or something else?  How do you expect 
> RBAC to work for you?
> 
> We'll definitely have a slot for this at the upcoming CDS, but anything we 
> can hash out before then will make that conversation more productive.
> 
> Thanks!
> sage
> 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

I don't know any details about what you discussed (other than what
you have in the document), but I do know kerberos very well, and
perhaps I can provide background that will help you or others,
(Some of this is in your document w/ slightly less detail.)

1. iaa
2. kerberos implementations
3. name/id/group lookup
4. for ceph

____ 1. iaa

So, you mention kerberos, AD, LDAP -- these are somewhat separable,
and probably ought to be separated for what you do:

general acronym for all three is IAA:
	I	-identity
	A	-authentication		(N)
	A	-authorization		(Z)
kerberos does identity/authentication, period.  Does not do authorization
at all, period, full stop.  ( Yes, "AD" makes a "pac" inside a kerberos
ticket, that can provide authorization data.  More on that in a bit. )

Ldap is a protocol to access a database that can contain identities,
group information, + other stuff.  Does not really do authentication at
all, can do half of the authorization job.

AD is a specific product from microsoft.  It provides both kerberos
infrastructure (server, libraries) + ldap.  There are other products that
provide various parts of this - including of course freeipa from redhat.
AD certainly has a lot more industry uptake than freeipa.  But most
of that is microsoft-centric, with all the caveats that entails.
Some places may have nds which does ldap and has kerberos support.
There are also plenty of people, including some large sites, who run
MIT kerberos and openldap.

At a large site, I think you can confidently expect:
	or or more large scale deployments of AD,MIT,openldap,etc.
	homebrew management system to provision and manage accounts
	the filesystem group(s) and the identity management group are separate

____ 2. kerberos implementations

For kerberos proper, there are about 4 choices (library-wise):
	1. MIT kerberos
	2. Heimdal.
	3. gssapi
	4. microsoft...

MIT and Heimdal have libraries with many of the same entry point names,
but with structures that are different and have different field names.
It's mostly possible but not especially attractive to code for both.
Having said that, there are attractions to coding directly for this.

gssapi is the usual community response to the "but you have 2 api's".
It does give you a single common api, and a higher level view of things.

Microsoft has its own api.  That's one of those "caveats", above.
Fortunately you probably don't care.

____ 3. name/id/group lookup

Since kerberos "just" does identity/authentication; - you generally need
a separate id/group lookup function.  Since you are probably going to be
hooking into somebody else's infrastructure, I think instead of assuming a pac
or ldap, this is something you want to make at least customizable,
probably configurable, and possibly pluggable.  Kerberos per se provides
a name, such as "sageweil@EXAMPLE.COM".  For posix file semantics, you
would want to map this to a uid=123, group=456, groups=7 8 9.

One common way to resolve that mapping of name->id info is via ldap,
and there are standard ldap schemas that let you do that. (posixAccount,
also posixGroup).  Freeipa almost certainly gives you this "for free".
For openldap (and nds) this is something you can choose to add.

AD, being microsoft, -- apparently the samba people have to deal with
this, and they say that one solution is to install the AD4UNIX schema
(there's lots more in their samba howto idmapper document,
https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/idmapper.html )

AD natively provides group information to microsoft applications.
This is encoded in the "PAC" structure of service tickets issued
by AD.  MicroSoft group information does not exactly translate
to posix uid/gid type information, so this is not necessarily
exactly what you want, especially in a Unix environment.
More about PAC's here,
http://blogs.msdn.com/b/openspecification/archive/2009/04/24/understanding-microsoft-kerberos-pac-validation.aspx

Other filesystems: nfsv4 has something called "idmap" that often
resolves this from the local host /etc/passwd /etc/group.  afs has the
"protection service", protocol is called "prot", but most people know
this as pts or ptserver.  And then there are always other "homebrew"
choices such as mysql etc.

And lastly, since you have a "user db" - working with kerberos
*without* ldap/ad/whatever - should be doable too.

____ 4. for ceph

So for ceph, you appear to be proposing a "cephx" ticket issuing service
that relies on kerberos tickets.  That should be easy -- gssapi or mit lib
are probably your obvious choices here; either will work.

The "k5/cephx" service has a choice of ways to resolve group/id information.
An ldap lookup for posixaccount/group, maybe ad4unix seem reasonable.
MicroSoft pac is harder and has tradeoffs, but probably doable.
Providing this as a pluggable interface with examples could provide
people with additional choices and the option for whatever.

You mention the ceph user db -- one option you probably also want
here is an "autoregistration" mechanism to add people to the userdb - for
people coming in via kerberos who weren't seen before.  Probably you
want to worry about membership changes too if you do this.

I haven't gone through cephx in detail, but my naive impression is that
it does not provide a particularly robust assurance that user id 123
is really user id 123.  You might want ways to turn off the ability of
random clients/servers from creating cephx tickets without going through
kerberos or some other stronger means of identity assurance.

				- Marcus Watts

Re: kerberos / AD requirements, blueprint

[Andrew Bartlett]

Thanks Sage and Marcus for writing this up.  I've got a few comments inline:

On Thu, 2014-10-16 at 20:52 -0400, mdw@linuxbox.com wrote:
> On Thu, Oct 16, 2014 at 12:57:12PM -0700, Sage Weil wrote:
> > I started to write up a blueprint for kerberos / LDAP / AD support:
> > 
> > 	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> > 
> > In case it isn't clear from that document, I only sort of know what I'm 
> > talking about here.  (This is largely based on what I managed to retain 
> > from a conversation with Andrew, but I suspect I got some of it wrong.)
> > 
> > For anyone working in environments where kerberos is in use, I am *very* 
> > interested in hearing what your requirements are for your environment.  In 
> > particular, are you using AD or LDAP or something else?  How do you expect 
> > RBAC to work for you?
> > 

> I don't know any details about what you discussed (other than what
> you have in the document), but I do know kerberos very well, and
> perhaps I can provide background that will help you or others,
> (Some of this is in your document w/ slightly less detail.)
> 
> 1. iaa
> 2. kerberos implementations
> 3. name/id/group lookup
> 4. for ceph
> 

> At a large site, I think you can confidently expect:
> 	or or more large scale deployments of AD,MIT,openldap,etc.
> 	homebrew management system to provision and manage accounts
> 	the filesystem group(s) and the identity management group are separate

Do you see a lot of this beyond major universities and similar sites?  I
ask because at least from my Samba background, I feel like we lost out
in the 2000's against AD, with a good number of very passionate users
waiting patiently for Samba4, but so, so many just running AD from
Microsoft.  The number with OpenLDAP and Kerberos that come past the
Samba lists seemed vanishingly small. 

(I've very glad for our passionate OpenLDAP and Kerberos users, I just
don't see so many of them these days).
 
> ____ 2. kerberos implementations
> 
> For kerberos proper, there are about 4 choices (library-wise):
> 	1. MIT kerberos
> 	2. Heimdal.
> 	3. gssapi
> 	4. microsoft...
> 
> MIT and Heimdal have libraries with many of the same entry point names,
> but with structures that are different and have different field names.
> It's mostly possible but not especially attractive to code for both.
> Having said that, there are attractions to coding directly for this.
> 
> gssapi is the usual community response to the "but you have 2 api's".
> It does give you a single common api, and a higher level view of things.

In my experience, you need to code to both.  You should use GSSAPI, it
is more than just a wrapper, it actually adds a protocol layer, but some
of the things you need to do at the krb5 layer. 

Certainly there are differences, but they have been getting better in
recent years.  The main trick is to require a modern version of MIT
Kerberos or Heimdal, and that means most of the missing functions are
implemented. 

> Microsoft has its own api.  That's one of those "caveats", above.
> Fortunately you probably don't care.
> 
> ____ 3. name/id/group lookup
> 
> Since kerberos "just" does identity/authentication; - you generally need
> a separate id/group lookup function.  Since you are probably going to be
> hooking into somebody else's infrastructure, I think instead of assuming a pac
> or ldap, this is something you want to make at least customizable,
> probably configurable, and possibly pluggable.  Kerberos per se provides
> a name, such as "sageweil@EXAMPLE.COM".  For posix file semantics, you
> would want to map this to a uid=123, group=456, groups=7 8 9.
> 
> One common way to resolve that mapping of name->id info is via ldap,
> and there are standard ldap schemas that let you do that. (posixAccount,
> also posixGroup).  Freeipa almost certainly gives you this "for free".
> For openldap (and nds) this is something you can choose to add.
> 
> AD, being microsoft, -- apparently the samba people have to deal with
> this, and they say that one solution is to install the AD4UNIX schema
> (there's lots more in their samba howto idmapper document,
> https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/idmapper.html )

I really should help clean up some of those docs, but for this, we don't
need to worry about idmap, because we don't need the users to be in a
posix group, just to have some ceph permissions.  So we just need to
specify a group or a group SID in a config file, and say that these
folks have certain rights, we don't need to do anything related to
POSIX.  Samba can extract the SIDs from the PAC, which helps a lot,
particularly with performance and accuracy. 

I do appreciate you putting some more of the background in writing here.
This is a difficult area with a lot of history, and much folklore :-)

Thanks,

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba




-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

Re: kerberos / AD requirements, blueprint

[mdw]

On Wed, Oct 22, 2014 at 05:21:29PM +1300, Andrew Bartlett wrote:
...
> > At a large site, I think you can confidently expect:
> > 	or or more large scale deployments of AD,MIT,openldap,etc.
> > 	homebrew management system to provision and manage accounts
> > 	the filesystem group(s) and the identity management group are separate
> 
> Do you see a lot of this beyond major universities and similar sites?  I
> ask because at least from my Samba background, I feel like we lost out
> in the 2000's against AD, with a good number of very passionate users
> waiting patiently for Samba4, but so, so many just running AD from
> Microsoft.  The number with OpenLDAP and Kerberos that come past the
> Samba lists seemed vanishingly small. 
> 
> (I've very glad for our passionate OpenLDAP and Kerberos users, I just
> don't see so many of them these days).

I think the overwhelming common implementation is AD - at all sizes
of organizations from small to large.  But most of those will be
microsoft-only environments, so aren't particularly relevant to ceph.
I don't have good stats on the # of openldap/mit sites - but I imagine
many of them either don't care about samba, or have already invested
effort in a more or less parallel AD setup.  If you're running a lot
of microsoft desktops already, you'd have to be pretty passionate
to not just run AD and call it a day.  For ceph, though, you're
talking about linux machines - and there, the attraction for AD
is underwhelming.

					-Marcus Watts

Re: kerberos / AD requirements, blueprint

[Christoph Hellwig]

On Wed, Oct 22, 2014 at 06:46:06PM -0400, mdw@linuxbox.com wrote:
> I think the overwhelming common implementation is AD - at all sizes
> of organizations from small to large.  But most of those will be
> microsoft-only environments, so aren't particularly relevant to ceph.
> I don't have good stats on the # of openldap/mit sites - but I imagine
> many of them either don't care about samba, or have already invested
> effort in a more or less parallel AD setup.  If you're running a lot
> of microsoft desktops already, you'd have to be pretty passionate
> to not just run AD and call it a day.  For ceph, though, you're
> talking about linux machines - and there, the attraction for AD
> is underwhelming.

I know enough large sites using AD for their Linux nodes as well. So
far I've not seen an overlap with Ceph deployments, though.

Re: kerberos / AD requirements, blueprint

[Dan Van Der Ster]

Hi Sage,

> On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
> 
> I started to write up a blueprint for kerberos / LDAP / AD support:
> 
> 	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> 
> In case it isn't clear from that document, I only sort of know what I'm 
> talking about here.  (This is largely based on what I managed to retain 
> from a conversation with Andrew, but I suspect I got some of it wrong.)
> 
> For anyone working in environments where kerberos is in use, I am *very* 
> interested in hearing what your requirements are for your environment.  In 
> particular, are you using AD or LDAP or something else?  How do you expect 
> RBAC to work for you?

Let’s talk about requirements then. (Anyway, I’m not in any way a kerberos expert and it is not 100% clear to me what you are trying to achieve.)

Firstly, our environment is almost identical to that described by Marcus (we use AD). For filesystems, our user communities are used to the way OpenAFS works with kerberos. Generally, that means they use kinit && aklog to authenticate and they understand (mostly) how to use AFS groups and AFS ACLs for authz.

Next I basically have a bunch of questions to clarify your goals. The first part of your blueprint seems to be a convenient way to distribute cephx keyrings. To prevent harvesting/reuse, would you intend those to not be “normal” cephx keyrings as such, but rather to be keyrings encrypted by the mon/whatever and time limited?

For authz, is this mechanism meant to be used for RADOS only, or also for CephFS?

If RADOS only, then as I understand it you could basically add the current cephx capabilities (with pool granularity) to the proposed ceph user db (perhaps scoped down to a few limited roles). I wouldn’t expect to be able to manipulate ceph capabilities via some AD blob (due mostly to my AD ignorance). I would probably expect to add our authorised ceph users to an AD group, then keep that in sync with your ceph user/role/capability db.

If CephFS, then the problem I see is that the POSIX permissions/ACLs are being evaluated only on the client side. Were you thinking to change that, i.e. to add some notion of CephFS file-granular cephx capabilities? We thought a little bit about this before:
  https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS
The basic idea was to have an MDS or other service that evaluates permissions/ACLs on the server side then hands out an extra signed capability to be used by the client for IOs with an OSD. It seems heavyweight, however, so I’m not sure how this would perform in practise.

Hope that helps,
Dan




> 
> We'll definitely have a slot for this at the upcoming CDS, but anything we 
> can hash out before then will make that conversation more productive.
> 
> Thanks!
> sage
> 

Re: kerberos / AD requirements, blueprint

[Sage Weil]

On Wed, 22 Oct 2014, Dan Van Der Ster wrote:
> Hi Sage,
> 
> > On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
> > 
> > I started to write up a blueprint for kerberos / LDAP / AD support:
> > 
> > 	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> > 
> > In case it isn't clear from that document, I only sort of know what I'm 
> > talking about here.  (This is largely based on what I managed to retain 
> > from a conversation with Andrew, but I suspect I got some of it wrong.)
> > 
> > For anyone working in environments where kerberos is in use, I am *very* 
> > interested in hearing what your requirements are for your environment.  In 
> > particular, are you using AD or LDAP or something else?  How do you expect 
> > RBAC to work for you?
> 
> Let?s talk about requirements then. (Anyway, I?m not in any way a 
> kerberos expert and it is not 100% clear to me what you are trying to 
> achieve.)
> 
> Firstly, our environment is almost identical to that described by Marcus 
> (we use AD). For filesystems, our user communities are used to the way 
> OpenAFS works with kerberos. Generally, that means they use kinit && 
> aklog to authenticate and they understand (mostly) how to use AFS groups 
> and AFS ACLs for authz.

I take it that the kinit + aklog use-case is about a user authenticating 
their workstation to mount AFS?

Just to be clear, the user auth issues for fs access once things are 
mounted is still out of scope for the current effort.  But mounting itself 
means the client (host) has to authenticate against the cluster and that 
is probably in scope.

> Next I basically have a bunch of questions to clarify your goals. The 
> first part of your blueprint seems to be a convenient way to distribute 
> cephx keyrings. To prevent harvesting/reuse, would you intend those to 
> not be ?normal? cephx keyrings as such, but rather to be keyrings 
> encrypted by the mon/whatever and time limited?

It's not so much about distributing keyrings, and using the existing 
kerberos auth in place of cephx keys.  With just the first part, the 
workflow might go something like:

 - define client.user1 and its caps
    - add a property that specifies a kerberos user/domain to map to
    - but do not generate a cephx key
 - kinit user1
 - ceph -n client.user1 ...
    - would use kerberos ticket to authenticate as client.user1 on mon
 ...

> For authz, is this mechanism meant to be used for RADOS only, or also 
> for CephFS?

Initially I'm just concerned with RADOS authentication.  And the main 
use-case is cluster administration (ceph and rbd CLI commands).

Step 2 (authz) would be to tag the kerberos users with some metadata like 
'ceph rw' and map to a generic ceph user (say, client.krb-rw).  The caps 
would be defined once per role in ceph as opposed to once per user (which 
is an administrative headache).  Granting a new kerberos user readonly or 
readwrite access to the ceph/rbd CLI could then be done exclusively from 
AD (or whatever) infrastructure.

> If RADOS only, then as I understand it you could basically add the 
> current cephx capabilities (with pool granularity) to the proposed ceph 
> user db (perhaps scoped down to a few limited roles). I wouldn?t expect 
> to be able to manipulate ceph capabilities via some AD blob (due mostly 
> to my AD ignorance). I would probably expect to add our authorised ceph 
> users to an AD group, then keep that in sync with your ceph 
> user/role/capability db.

Yeah, that is probably the most sensible.  Perhaps the ceph role 
definition would be something like

[client.krb-rw]
	mon = allow rw
	osd = allow *
	mds = allow *
	krb5_group = ceph_rw

or something along those lines.

> If CephFS, then the problem I see is that the POSIX permissions/ACLs are 
> being evaluated only on the client side. Were you thinking to change 
> that, i.e. to add some notion of CephFS file-granular cephx 
> capabilities? We thought a little bit about this before:
>   
> https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS 
> The basic idea was to have an MDS or other service that evaluates 
> permissions/ACLs on the server side then hands out an extra signed 
> capability to be used by the client for IOs with an OSD. It seems 
> heavyweight, however, so I?m not sure how this would perform in 
> practise.

Yeah, this is a whole other ball of wax that I'd prefer to ignore for the 
time being... :)

Thanks!
sage

Re: kerberos / AD requirements, blueprint

[Dan Van Der Ster]

Hi Sage,

> On 22 Oct 2014, at 19:08, Sage Weil <sage@newdream.net> wrote:
> 
> On Wed, 22 Oct 2014, Dan Van Der Ster wrote:
>> Hi Sage,
>> 
>>> On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
>>> 
>>> I started to write up a blueprint for kerberos / LDAP / AD support:
>>> 
>>> 	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
>>> 
>>> In case it isn't clear from that document, I only sort of know what I'm 
>>> talking about here.  (This is largely based on what I managed to retain 
>>> from a conversation with Andrew, but I suspect I got some of it wrong.)
>>> 
>>> For anyone working in environments where kerberos is in use, I am *very* 
>>> interested in hearing what your requirements are for your environment.  In 
>>> particular, are you using AD or LDAP or something else?  How do you expect 
>>> RBAC to work for you?
>> 
>> Let?s talk about requirements then. (Anyway, I?m not in any way a 
>> kerberos expert and it is not 100% clear to me what you are trying to 
>> achieve.)
>> 
>> Firstly, our environment is almost identical to that described by Marcus 
>> (we use AD). For filesystems, our user communities are used to the way 
>> OpenAFS works with kerberos. Generally, that means they use kinit && 
>> aklog to authenticate and they understand (mostly) how to use AFS groups 
>> and AFS ACLs for authz.
> 
> I take it that the kinit + aklog use-case is about a user authenticating 
> their workstation to mount AFS?

AFS mounts (via a kernel module) without any authentication. Their token is used to authorize in the context of their file access.


> 
> Just to be clear, the user auth issues for fs access once things are 
> mounted is still out of scope for the current effort.  But mounting itself 
> means the client (host) has to authenticate against the cluster and that 
> is probably in scope.

OK that simplifies things ;)

> 
>> Next I basically have a bunch of questions to clarify your goals. The 
>> first part of your blueprint seems to be a convenient way to distribute 
>> cephx keyrings. To prevent harvesting/reuse, would you intend those to 
>> not be ?normal? cephx keyrings as such, but rather to be keyrings 
>> encrypted by the mon/whatever and time limited?
> 
> It's not so much about distributing keyrings, and using the existing 
> kerberos auth in place of cephx keys.  With just the first part, the 
> workflow might go something like:
> 
> - define client.user1 and its caps
>    - add a property that specifies a kerberos user/domain to map to
>    - but do not generate a cephx key
> - kinit user1
> - ceph -n client.user1 ...
>    - would use kerberos ticket to authenticate as client.user1 on mon
> …
> 

OK I had misunderstood "If authentic, everything else proceeds as before (it provides the user with a cephx ticket to do real work)."


>> For authz, is this mechanism meant to be used for RADOS only, or also 
>> for CephFS?
> 
> Initially I'm just concerned with RADOS authentication.  And the main 
> use-case is cluster administration (ceph and rbd CLI commands).
> 
> Step 2 (authz) would be to tag the kerberos users with some metadata like 
> 'ceph rw' and map to a generic ceph user (say, client.krb-rw).  The caps 
> would be defined once per role in ceph as opposed to once per user (which 
> is an administrative headache).  Granting a new kerberos user readonly or 
> readwrite access to the ceph/rbd CLI could then be done exclusively from 
> AD (or whatever) infrastructure.
> 
>> If RADOS only, then as I understand it you could basically add the 
>> current cephx capabilities (with pool granularity) to the proposed ceph 
>> user db (perhaps scoped down to a few limited roles). I wouldn?t expect 
>> to be able to manipulate ceph capabilities via some AD blob (due mostly 
>> to my AD ignorance). I would probably expect to add our authorised ceph 
>> users to an AD group, then keep that in sync with your ceph 
>> user/role/capability db.
> 
> Yeah, that is probably the most sensible.  Perhaps the ceph role 
> definition would be something like
> 
> [client.krb-rw]
> 	mon = allow rw
> 	osd = allow *
> 	mds = allow *
> 	krb5_group = ceph_rw
> 
> or something along those lines.

Yeah that sounds good (though maybe ldap_group?). Then you can query ldap to see if authenticating users are  part of our ceph_rw group and give authz.

> 
>> If CephFS, then the problem I see is that the POSIX permissions/ACLs are 
>> being evaluated only on the client side. Were you thinking to change 
>> that, i.e. to add some notion of CephFS file-granular cephx 
>> capabilities? We thought a little bit about this before:
>> 
>> https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS 
>> The basic idea was to have an MDS or other service that evaluates 
>> permissions/ACLs on the server side then hands out an extra signed 
>> capability to be used by the client for IOs with an OSD. It seems 
>> heavyweight, however, so I?m not sure how this would perform in 
>> practise.
> 
> Yeah, this is a whole other ball of wax that I'd prefer to ignore for the 
> time being... :)


GOod ;)

Cheers, Dan

> 
> Thanks!
> sage

Re: kerberos / AD requirements, blueprint

[Andrea Ieri]

[-- Attachment #1: Type: text/plain, Size: 4030 bytes --]

Hi Sage,

It's perhaps useful to clear up that Kerberos as a protocol provides your service with the following information (at the end of the initial exchange):
1. a string representing a Kerberos principal (user@REALM)
2. a session key, known only to you and the principal above
3. a reasonable assurance that whoever sent you the ticket that contains the data above controls the principal mentioned (as in "knows its password")

Now, you can choose what to do with the information above.
- you'll most likely need to map the "principal" to a "user", whatever that means for your system (a UID/GID tuple obtained via LDAP? A user string in your own local database?)
- you'll have to decide what to do with the session key. Do you want to encrypt all communication? Implement a MAC? Throw it away? See for example the different Kerberos options in NFSv4

In any case as you can see Kerberos is not concerned with authorization at all. How you enforce permissions is out of its scope.

Cheers,
Andrea

On 22/10/2014 18:39, Dan Van Der Ster wrote:
> Hi Sage,
>
>> On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
>>
>> I started to write up a blueprint for kerberos / LDAP / AD support:
>>
>> 	https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
>>
>> In case it isn't clear from that document, I only sort of know what I'm 
>> talking about here.  (This is largely based on what I managed to retain 
>> from a conversation with Andrew, but I suspect I got some of it wrong.)
>>
>> For anyone working in environments where kerberos is in use, I am *very* 
>> interested in hearing what your requirements are for your environment.  In 
>> particular, are you using AD or LDAP or something else?  How do you expect 
>> RBAC to work for you?
> Let’s talk about requirements then. (Anyway, I’m not in any way a kerberos expert and it is not 100% clear to me what you are trying to achieve.)
>
> Firstly, our environment is almost identical to that described by Marcus (we use AD). For filesystems, our user communities are used to the way OpenAFS works with kerberos. Generally, that means they use kinit && aklog to authenticate and they understand (mostly) how to use AFS groups and AFS ACLs for authz.
>
> Next I basically have a bunch of questions to clarify your goals. The first part of your blueprint seems to be a convenient way to distribute cephx keyrings. To prevent harvesting/reuse, would you intend those to not be “normal” cephx keyrings as such, but rather to be keyrings encrypted by the mon/whatever and time limited?
>
> For authz, is this mechanism meant to be used for RADOS only, or also for CephFS?
>
> If RADOS only, then as I understand it you could basically add the current cephx capabilities (with pool granularity) to the proposed ceph user db (perhaps scoped down to a few limited roles). I wouldn’t expect to be able to manipulate ceph capabilities via some AD blob (due mostly to my AD ignorance). I would probably expect to add our authorised ceph users to an AD group, then keep that in sync with your ceph user/role/capability db.
>
> If CephFS, then the problem I see is that the POSIX permissions/ACLs are being evaluated only on the client side. Were you thinking to change that, i.e. to add some notion of CephFS file-granular cephx capabilities? We thought a little bit about this before:
>   https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS
> The basic idea was to have an MDS or other service that evaluates permissions/ACLs on the server side then hands out an extra signed capability to be used by the client for IOs with an OSD. It seems heavyweight, however, so I’m not sure how this would perform in practise.
>
> Hope that helps,
> Dan
>
>
>
>
>> We'll definitely have a slot for this at the upcoming CDS, but anything we 
>> can hash out before then will make that conversation more productive.
>>
>> Thanks!
>> sage
>>



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5620 bytes --]

Re: kerberos / AD requirements, blueprint

[Matt W. Benjamin]

Hi,

Is the goal then NOT to provide per-user authn (etc) as AFS or DCE would do, when a user accesses files using the kernel client (and provide for that, in general)?  I would have thought that was a lot of the point.

It seems like that is the well trodden path, for AFS, DFS, NFSv4, it's basically similar, varying only in that NFSv4's per-user token propagation is a bit better integrated on Linux (but doesn't yet provide the PAG concept).

Matt

----- "Andrea Ieri" <andrea.ieri@cern.ch> wrote:

> Hi Sage,
> 
> It's perhaps useful to clear up that Kerberos as a protocol provides
> your service with the following information (at the end of the initial
> exchange):
> 1. a string representing a Kerberos principal (user@REALM)
> 2. a session key, known only to you and the principal above
> 3. a reasonable assurance that whoever sent you the ticket that
> contains the data above controls the principal mentioned (as in "knows
> its password")
> 
> Now, you can choose what to do with the information above.
> - you'll most likely need to map the "principal" to a "user", whatever
> that means for your system (a UID/GID tuple obtained via LDAP? A user
> string in your own local database?)
> - you'll have to decide what to do with the session key. Do you want
> to encrypt all communication? Implement a MAC? Throw it away? See for
> example the different Kerberos options in NFSv4
> 
> In any case as you can see Kerberos is not concerned with
> authorization at all. How you enforce permissions is out of its
> scope.
> 
> Cheers,
> Andrea
> 
> On 22/10/2014 18:39, Dan Van Der Ster wrote:
> > Hi Sage,
> >
> >> On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
> >>
> >> I started to write up a blueprint for kerberos / LDAP / AD
> support:
> >>
> >> 
> https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> >>
> >> In case it isn't clear from that document, I only sort of know what
> I'm 
> >> talking about here.  (This is largely based on what I managed to
> retain 
> >> from a conversation with Andrew, but I suspect I got some of it
> wrong.)
> >>
> >> For anyone working in environments where kerberos is in use, I am
> *very* 
> >> interested in hearing what your requirements are for your
> environment.  In 
> >> particular, are you using AD or LDAP or something else?  How do you
> expect 
> >> RBAC to work for you?
> > Let’s talk about requirements then. (Anyway, I’m not in any way a
> kerberos expert and it is not 100% clear to me what you are trying to
> achieve.)
> >
> > Firstly, our environment is almost identical to that described by
> Marcus (we use AD). For filesystems, our user communities are used to
> the way OpenAFS works with kerberos. Generally, that means they use
> kinit && aklog to authenticate and they understand (mostly) how to use
> AFS groups and AFS ACLs for authz.
> >
> > Next I basically have a bunch of questions to clarify your goals.
> The first part of your blueprint seems to be a convenient way to
> distribute cephx keyrings. To prevent harvesting/reuse, would you
> intend those to not be “normal” cephx keyrings as such, but rather to
> be keyrings encrypted by the mon/whatever and time limited?
> >
> > For authz, is this mechanism meant to be used for RADOS only, or
> also for CephFS?
> >
> > If RADOS only, then as I understand it you could basically add the
> current cephx capabilities (with pool granularity) to the proposed
> ceph user db (perhaps scoped down to a few limited roles). I wouldn’t
> expect to be able to manipulate ceph capabilities via some AD blob
> (due mostly to my AD ignorance). I would probably expect to add our
> authorised ceph users to an AD group, then keep that in sync with your
> ceph user/role/capability db.
> >
> > If CephFS, then the problem I see is that the POSIX permissions/ACLs
> are being evaluated only on the client side. Were you thinking to
> change that, i.e. to add some notion of CephFS file-granular cephx
> capabilities? We thought a little bit about this before:
> >  
> https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS
> > The basic idea was to have an MDS or other service that evaluates
> permissions/ACLs on the server side then hands out an extra signed
> capability to be used by the client for IOs with an OSD. It seems
> heavyweight, however, so I’m not sure how this would perform in
> practise.
> >
> > Hope that helps,
> > Dan
> >
> >
> >
> >
> >> We'll definitely have a slot for this at the upcoming CDS, but
> anything we 
> >> can hash out before then will make that conversation more
> productive.
> >>
> >> Thanks!
> >> sage
> >>

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kerberos / AD requirements, blueprint

[Matt W. Benjamin]

I meant to say, per-user protection.

Matt

----- "Matt W. Benjamin" <matt@linuxbox.com> wrote:

> Hi,
> 
> Is the goal then NOT to provide per-user authn (etc) as AFS or DCE
> would do, when a user accesses files using the kernel client (and
> provide for that, in general)?  I would have thought that was a lot of
> the point.
> 
> It seems like that is the well trodden path, for AFS, DFS, NFSv4, it's
> basically similar, varying only in that NFSv4's per-user token
> propagation is a bit better integrated on Linux (but doesn't yet
> provide the PAG concept).
> 
> Matt
> 
> ----- "Andrea Ieri" <andrea.ieri@cern.ch> wrote:
> 
> > Hi Sage,
> > 
> > It's perhaps useful to clear up that Kerberos as a protocol
> provides
> > your service with the following information (at the end of the
> initial
> > exchange):
> > 1. a string representing a Kerberos principal (user@REALM)
> > 2. a session key, known only to you and the principal above
> > 3. a reasonable assurance that whoever sent you the ticket that
> > contains the data above controls the principal mentioned (as in
> "knows
> > its password")
> > 
> > Now, you can choose what to do with the information above.
> > - you'll most likely need to map the "principal" to a "user",
> whatever
> > that means for your system (a UID/GID tuple obtained via LDAP? A
> user
> > string in your own local database?)
> > - you'll have to decide what to do with the session key. Do you
> want
> > to encrypt all communication? Implement a MAC? Throw it away? See
> for
> > example the different Kerberos options in NFSv4
> > 
> > In any case as you can see Kerberos is not concerned with
> > authorization at all. How you enforce permissions is out of its
> > scope.
> > 
> > Cheers,
> > Andrea
> > 
> > On 22/10/2014 18:39, Dan Van Der Ster wrote:
> > > Hi Sage,
> > >
> > >> On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
> > >>
> > >> I started to write up a blueprint for kerberos / LDAP / AD
> > support:
> > >>
> > >> 
> >
> https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> > >>
> > >> In case it isn't clear from that document, I only sort of know
> what
> > I'm 
> > >> talking about here.  (This is largely based on what I managed to
> > retain 
> > >> from a conversation with Andrew, but I suspect I got some of it
> > wrong.)
> > >>
> > >> For anyone working in environments where kerberos is in use, I
> am
> > *very* 
> > >> interested in hearing what your requirements are for your
> > environment.  In 
> > >> particular, are you using AD or LDAP or something else?  How do
> you
> > expect 
> > >> RBAC to work for you?
> > > Let’s talk about requirements then. (Anyway, I’m not in any way a
> > kerberos expert and it is not 100% clear to me what you are trying
> to
> > achieve.)
> > >
> > > Firstly, our environment is almost identical to that described by
> > Marcus (we use AD). For filesystems, our user communities are used
> to
> > the way OpenAFS works with kerberos. Generally, that means they use
> > kinit && aklog to authenticate and they understand (mostly) how to
> use
> > AFS groups and AFS ACLs for authz.
> > >
> > > Next I basically have a bunch of questions to clarify your goals.
> > The first part of your blueprint seems to be a convenient way to
> > distribute cephx keyrings. To prevent harvesting/reuse, would you
> > intend those to not be “normal” cephx keyrings as such, but rather
> to
> > be keyrings encrypted by the mon/whatever and time limited?
> > >
> > > For authz, is this mechanism meant to be used for RADOS only, or
> > also for CephFS?
> > >
> > > If RADOS only, then as I understand it you could basically add
> the
> > current cephx capabilities (with pool granularity) to the proposed
> > ceph user db (perhaps scoped down to a few limited roles). I
> wouldn’t
> > expect to be able to manipulate ceph capabilities via some AD blob
> > (due mostly to my AD ignorance). I would probably expect to add our
> > authorised ceph users to an AD group, then keep that in sync with
> your
> > ceph user/role/capability db.
> > >
> > > If CephFS, then the problem I see is that the POSIX
> permissions/ACLs
> > are being evaluated only on the client side. Were you thinking to
> > change that, i.e. to add some notion of CephFS file-granular cephx
> > capabilities? We thought a little bit about this before:
> > >  
> >
> https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS
> > > The basic idea was to have an MDS or other service that evaluates
> > permissions/ACLs on the server side then hands out an extra signed
> > capability to be used by the client for IOs with an OSD. It seems
> > heavyweight, however, so I’m not sure how this would perform in
> > practise.
> > >
> > > Hope that helps,
> > > Dan
> > >
> > >
> > >
> > >
> > >> We'll definitely have a slot for this at the upcoming CDS, but
> > anything we 
> > >> can hash out before then will make that conversation more
> > productive.
> > >>
> > >> Thanks!
> > >> sage
> > >>
> 
> -- 
> Matt Benjamin
> The Linux Box
> 206 South Fifth Ave. Suite 150
> Ann Arbor, MI  48104
> 
> http://linuxbox.com
> 
> tel.  734-761-4689 
> fax.  734-769-8938 
> cel.  734-216-5309 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kerberos / AD requirements, blueprint

[Sage Weil]

On Thu, 23 Oct 2014, Matt W. Benjamin wrote:
> Hi,
> 
> Is the goal then NOT to provide per-user authn (etc) as AFS or DCE would 
> do, when a user accesses files using the kernel client (and provide for 
> that, in general)?  I would have thought that was a lot of the point.

Eventually, yes.  The baby step is just use AD/LDAP for the ceph/rbd CLIs 
and for authn/z at mount time.

sage


> 
> Matt
> 
> ----- "Andrea Ieri" <andrea.ieri@cern.ch> wrote:
> 
> > Hi Sage,
> > 
> > It's perhaps useful to clear up that Kerberos as a protocol provides
> > your service with the following information (at the end of the initial
> > exchange):
> > 1. a string representing a Kerberos principal (user@REALM)
> > 2. a session key, known only to you and the principal above
> > 3. a reasonable assurance that whoever sent you the ticket that
> > contains the data above controls the principal mentioned (as in "knows
> > its password")
> > 
> > Now, you can choose what to do with the information above.
> > - you'll most likely need to map the "principal" to a "user", whatever
> > that means for your system (a UID/GID tuple obtained via LDAP? A user
> > string in your own local database?)
> > - you'll have to decide what to do with the session key. Do you want
> > to encrypt all communication? Implement a MAC? Throw it away? See for
> > example the different Kerberos options in NFSv4
> > 
> > In any case as you can see Kerberos is not concerned with
> > authorization at all. How you enforce permissions is out of its
> > scope.
> > 
> > Cheers,
> > Andrea
> > 
> > On 22/10/2014 18:39, Dan Van Der Ster wrote:
> > > Hi Sage,
> > >
> > >> On 16 Oct 2014, at 21:57, Sage Weil <sage@newdream.net> wrote:
> > >>
> > >> I started to write up a blueprint for kerberos / LDAP / AD
> > support:
> > >>
> > >> 
> > https://wiki.ceph.com/Planning/Blueprints/Hammer/kerberos_authn%2C_AD_authn%2F%2Fauthz
> > >>
> > >> In case it isn't clear from that document, I only sort of know what
> > I'm 
> > >> talking about here.  (This is largely based on what I managed to
> > retain 
> > >> from a conversation with Andrew, but I suspect I got some of it
> > wrong.)
> > >>
> > >> For anyone working in environments where kerberos is in use, I am
> > *very* 
> > >> interested in hearing what your requirements are for your
> > environment.  In 
> > >> particular, are you using AD or LDAP or something else?  How do you
> > expect 
> > >> RBAC to work for you?
> > > Let?s talk about requirements then. (Anyway, I?m not in any way a
> > kerberos expert and it is not 100% clear to me what you are trying to
> > achieve.)
> > >
> > > Firstly, our environment is almost identical to that described by
> > Marcus (we use AD). For filesystems, our user communities are used to
> > the way OpenAFS works with kerberos. Generally, that means they use
> > kinit && aklog to authenticate and they understand (mostly) how to use
> > AFS groups and AFS ACLs for authz.
> > >
> > > Next I basically have a bunch of questions to clarify your goals.
> > The first part of your blueprint seems to be a convenient way to
> > distribute cephx keyrings. To prevent harvesting/reuse, would you
> > intend those to not be ?normal? cephx keyrings as such, but rather to
> > be keyrings encrypted by the mon/whatever and time limited?
> > >
> > > For authz, is this mechanism meant to be used for RADOS only, or
> > also for CephFS?
> > >
> > > If RADOS only, then as I understand it you could basically add the
> > current cephx capabilities (with pool granularity) to the proposed
> > ceph user db (perhaps scoped down to a few limited roles). I wouldn?t
> > expect to be able to manipulate ceph capabilities via some AD blob
> > (due mostly to my AD ignorance). I would probably expect to add our
> > authorised ceph users to an AD group, then keep that in sync with your
> > ceph user/role/capability db.
> > >
> > > If CephFS, then the problem I see is that the POSIX permissions/ACLs
> > are being evaluated only on the client side. Were you thinking to
> > change that, i.e. to add some notion of CephFS file-granular cephx
> > capabilities? We thought a little bit about this before:
> > >  
> > https://wiki.ceph.com/Planning/Blueprints/%3CSIDEBOARD%3E/Strong_AuthN_and_AuthZ_for_CephFS
> > > The basic idea was to have an MDS or other service that evaluates
> > permissions/ACLs on the server side then hands out an extra signed
> > capability to be used by the client for IOs with an OSD. It seems
> > heavyweight, however, so I?m not sure how this would perform in
> > practise.
> > >
> > > Hope that helps,
> > > Dan
> > >
> > >
> > >
> > >
> > >> We'll definitely have a slot for this at the upcoming CDS, but
> > anything we 
> > >> can hash out before then will make that conversation more
> > productive.
> > >>
> > >> Thanks!
> > >> sage
> > >>
> 
> -- 
> Matt Benjamin
> The Linux Box
> 206 South Fifth Ave. Suite 150
> Ann Arbor, MI  48104
> 
> http://linuxbox.com
> 
> tel.  734-761-4689 
> fax.  734-769-8938 
> cel.  734-216-5309 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
>