CDS blueprint: strong auth for cephfs

CDS blueprint: strong auth for cephfs

[Dan van der Ster]

Hi all,
This mail is just to let you know that we've prepared a draft
blueprint related to adding strong(er) authn/authz to cephfs:

http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Strong_AuthN_and_AuthZ_for_CephFS

The main goal of the idea is that we'd like to be able to use CephFS
from untrusted clients:
  - the CephX key gives full rw access to pools (e.g. data/metadata)
via rados; we cannot distribute this key to untrusted hosts.
  - root on untrusted clients can forge their uid/gid and rm -rf /cephfs/*.

In the doc we've proposed one way to add authn/authz to the ceph
server side, but perhaps there is a simpler (more feasible in the
short term) solution which would enable us to allow untrusted cephfs
clients.

Best Regards,
Arne & Andreas & Andrea & Dan
CERN IT

Re: CDS blueprint: strong auth for cephfs

[Gregory Farnum]

On Wed, Nov 13, 2013 at 8:05 AM, Dan van der Ster <dan@vanderster.com> wrote:
> Hi all,
> This mail is just to let you know that we've prepared a draft
> blueprint related to adding strong(er) authn/authz to cephfs:
>
> http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Strong_AuthN_and_AuthZ_for_CephFS
>
> The main goal of the idea is that we'd like to be able to use CephFS
> from untrusted clients:
>   - the CephX key gives full rw access to pools (e.g. data/metadata)
> via rados; we cannot distribute this key to untrusted hosts.
>   - root on untrusted clients can forge their uid/gid and rm -rf /cephfs/*.
>
> In the doc we've proposed one way to add authn/authz to the ceph
> server side, but perhaps there is a simpler (more feasible in the
> short term) solution which would enable us to allow untrusted cephfs
> clients.

You want to check out:
http://wiki.ceph.com/01Planning/Sideboard/Client_Security_for_CephFS,
the previous edition of the client security blueprint
http://www.ssrc.ucsc.edu/Papers/leung-sc07.pdf, "Scalable Security for
Petascale Parallel File Systems", which discusses one way of using
MDS-granted permissions
http://www.ssrc.ucsc.edu/Papers/li-fast13.pdf, "Horus: Fine-Grained
Encryption-Based Security for Large-Scale Storage" is largely
orthogonal in specifics but might inspire some other ides

It looks to me like you've actually got two different blueprints
there, though – "Allow Ceph clients to use Kerberos as an entry point"
and "Enable stronger CephFS security". You're going to need to discuss
the MDS tickets in a lot more detail than you have there, though — for
instance, with your current description you'll need a shared secret
between the OSD and MDS, and you'll need to expire the client extra
keys, etc. You may want to consider skipping this in the first
iteration and just implementing proper security on the MDS while
relying on RADOS-level options to keep data secure (separate
namespaces or pools for different users, and providing narrow caps to
clients). That would also leave more time for some serious research
into how to do a scalable implementation of any ticketing system we
create.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Dan van der Ster]

Hi Greg,

On Wed, Nov 13, 2013 at 6:45 PM, Gregory Farnum <greg@inktank.com> wrote:
> On Wed, Nov 13, 2013 at 8:05 AM, Dan van der Ster <dan@vanderster.com> wrote:
>> Hi all,
>> This mail is just to let you know that we've prepared a draft
>> blueprint related to adding strong(er) authn/authz to cephfs:
>>
>> http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Strong_AuthN_and_AuthZ_for_CephFS
>>
>> The main goal of the idea is that we'd like to be able to use CephFS
>> from untrusted clients:
>>   - the CephX key gives full rw access to pools (e.g. data/metadata)
>> via rados; we cannot distribute this key to untrusted hosts.
>>   - root on untrusted clients can forge their uid/gid and rm -rf /cephfs/*.
>>
>> In the doc we've proposed one way to add authn/authz to the ceph
>> server side, but perhaps there is a simpler (more feasible in the
>> short term) solution which would enable us to allow untrusted cephfs
>> clients.
>
> You want to check out:
> http://wiki.ceph.com/01Planning/Sideboard/Client_Security_for_CephFS,
> the previous edition of the client security blueprint

AFAICT, this would allow single users to mount a subtree on CephFS and
prevent them from writing where they are not permitted. But our
use-case is different: we want to mount /cephfs/ from shared
workstations and batch nodes to store personal home directories,
shared group areas, and read-only software areas. Managing per-user
keyrings and per-user mountpoints would become quickly unmanageable.

> http://www.ssrc.ucsc.edu/Papers/leung-sc07.pdf, "Scalable Security for
> Petascale Parallel File Systems", which discusses one way of using
> MDS-granted permissions
> http://www.ssrc.ucsc.edu/Papers/li-fast13.pdf, "Horus: Fine-Grained
> Encryption-Based Security for Large-Scale Storage" is largely
> orthogonal in specifics but might inspire some other ides
>

Thanks! I'll take a look.

> It looks to me like you've actually got two different blueprints
> there, though – "Allow Ceph clients to use Kerberos as an entry point"
> and "Enable stronger CephFS security". You're going to need to discuss

It's true, they are separable, but strong authn would have little
utility without strong authz, and vice versa.

> the MDS tickets in a lot more detail than you have there, though — for
> instance, with your current description you'll need a shared secret
> between the OSD and MDS, and you'll need to expire the client extra

Right, the shared secret is the kerberos service key in our model.

> keys, etc. You may want to consider skipping this in the first
> iteration and just implementing proper security on the MDS while
> relying on RADOS-level options to keep data secure (separate
> namespaces or pools for different users, and providing narrow caps to
> clients). That would also leave more time for some serious research

It sounds like like you're referring to the Sideboard above, which
unless I'm wrong won't help for our (not uncommon?) use-case.

Cheers, Dan

> into how to do a scalable implementation of any ticketing system we
> create.
> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Gregory Farnum]

On Thu, Nov 14, 2013 at 2:00 AM, Dan van der Ster <dan@vanderster.com> wrote:
> Hi Greg,
>
> On Wed, Nov 13, 2013 at 6:45 PM, Gregory Farnum <greg@inktank.com> wrote:
>> On Wed, Nov 13, 2013 at 8:05 AM, Dan van der Ster <dan@vanderster.com> wrote:
>>> Hi all,
>>> This mail is just to let you know that we've prepared a draft
>>> blueprint related to adding strong(er) authn/authz to cephfs:
>>>
>>> http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Strong_AuthN_and_AuthZ_for_CephFS
>>>
>>> The main goal of the idea is that we'd like to be able to use CephFS
>>> from untrusted clients:
>>>   - the CephX key gives full rw access to pools (e.g. data/metadata)
>>> via rados; we cannot distribute this key to untrusted hosts.
>>>   - root on untrusted clients can forge their uid/gid and rm -rf /cephfs/*.
>>>
>>> In the doc we've proposed one way to add authn/authz to the ceph
>>> server side, but perhaps there is a simpler (more feasible in the
>>> short term) solution which would enable us to allow untrusted cephfs
>>> clients.
>>
>> You want to check out:
>> http://wiki.ceph.com/01Planning/Sideboard/Client_Security_for_CephFS,
>> the previous edition of the client security blueprint
>
> AFAICT, this would allow single users to mount a subtree on CephFS and
> prevent them from writing where they are not permitted. But our
> use-case is different: we want to mount /cephfs/ from shared
> workstations and batch nodes to store personal home directories,
> shared group areas, and read-only software areas. Managing per-user
> keyrings and per-user mountpoints would become quickly unmanageable.

Ah, I hadn't gotten that from skimming the blueprint. So this isn't
just about hooking into Kerberos when we do a mount, you want to
dynamically retrieve different caps from the monitors as users log on,
and have that single client choose between them based on who's making
the request?

>> the MDS tickets in a lot more detail than you have there, though — for
>> instance, with your current description you'll need a shared secret
>> between the OSD and MDS, and you'll need to expire the client extra
>
> Right, the shared secret is the kerberos service key in our model.

So the whole Ceph cluster has one shared service key, and they use
that for securing everything that anybody does? (Sorry, that's
probably an elementary question, but I haven't looked at Kerberos
since Yehuda implemented the initial CephX auth.)

>> keys, etc. You may want to consider skipping this in the first
>> iteration and just implementing proper security on the MDS while
>> relying on RADOS-level options to keep data secure (separate
>> namespaces or pools for different users, and providing narrow caps to
>> clients). That would also leave more time for some serious research
>
> It sounds like like you're referring to the Sideboard above, which
> unless I'm wrong won't help for our (not uncommon?) use-case.

Yeah. :/

On Thu, Nov 14, 2013 at 5:13 AM, Arne Wiebalck <Arne.Wiebalck@cern.ch> wrote:
> It looks to me like Maat is already pretty close to what the use of CephFS on untrusted
> clients would require.
>
> Any idea why Maat does not use a kerberized Ceph service and the corresponding
> shared session keys? I think that could avoid the generation and distribution of per
> client keys and outsource the shared key negotation to a standardized mechanism
> (and allow for integration with existing authN service infrastructures, of course).

I do not. If I were placing bets, because it was a research project
and they didn't have Kerberos set up already. ;) But they also discuss
the security implications of using a single shared key, and discard it
because they want the system to remain mostly secure if an attacker
compromises a machine (and gets access to its keys).

> Also, why was the Maat idea (which was apparently implemented, tested and
> benchmarked) not integrated into mainline Ceph?

By the time I arrived this was all history; you'll have to wait until
Sage gets back for that one. I suspect he didn't like the performance
enough and the trees had diverged non-trivially by the time Maat was
in a useful state. But if you are brave, you can go back in time
(couple years ago) and look for the "security" folder, which I think
contained the Maat changes, and see what would be involved in
extracting them. :)
-Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Dan van der Ster]

On Thu, Nov 14, 2013 at 4:55 PM, Gregory Farnum <greg@inktank.com> wrote:
> On Thu, Nov 14, 2013 at 2:00 AM, Dan van der Ster <dan@vanderster.com> wrote:
>> Hi Greg,
>>
>> On Wed, Nov 13, 2013 at 6:45 PM, Gregory Farnum <greg@inktank.com> wrote:
>>> On Wed, Nov 13, 2013 at 8:05 AM, Dan van der Ster <dan@vanderster.com> wrote:
>>>> Hi all,
>>>> This mail is just to let you know that we've prepared a draft
>>>> blueprint related to adding strong(er) authn/authz to cephfs:
>>>>
>>>> http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Strong_AuthN_and_AuthZ_for_CephFS
>>>>
>>>> The main goal of the idea is that we'd like to be able to use CephFS
>>>> from untrusted clients:
>>>>   - the CephX key gives full rw access to pools (e.g. data/metadata)
>>>> via rados; we cannot distribute this key to untrusted hosts.
>>>>   - root on untrusted clients can forge their uid/gid and rm -rf /cephfs/*.
>>>>
>>>> In the doc we've proposed one way to add authn/authz to the ceph
>>>> server side, but perhaps there is a simpler (more feasible in the
>>>> short term) solution which would enable us to allow untrusted cephfs
>>>> clients.
>>>
>>> You want to check out:
>>> http://wiki.ceph.com/01Planning/Sideboard/Client_Security_for_CephFS,
>>> the previous edition of the client security blueprint
>>
>> AFAICT, this would allow single users to mount a subtree on CephFS and
>> prevent them from writing where they are not permitted. But our
>> use-case is different: we want to mount /cephfs/ from shared
>> workstations and batch nodes to store personal home directories,
>> shared group areas, and read-only software areas. Managing per-user
>> keyrings and per-user mountpoints would become quickly unmanageable.
>
> Ah, I hadn't gotten that from skimming the blueprint. So this isn't
> just about hooking into Kerberos when we do a mount, you want to
> dynamically retrieve different caps from the monitors as users log on,
> and have that single client choose between them based on who's making
> the request?

In our proposal the "additional capabilities" (which allow IOs with a
particular file/object) are encrypted and sent by the MDS to the
client who passes it on to the OSD for object IOs.

>
>>> the MDS tickets in a lot more detail than you have there, though — for
>>> instance, with your current description you'll need a shared secret
>>> between the OSD and MDS, and you'll need to expire the client extra
>>
>> Right, the shared secret is the kerberos service key in our model.
>
> So the whole Ceph cluster has one shared service key, and they use
> that for securing everything that anybody does? (Sorry, that's
> probably an elementary question, but I haven't looked at Kerberos
> since Yehuda implemented the initial CephX auth.)

We've used the shared service key in the past with other distributed
file systems. But perhaps the model can be changed to per-OSD/MDS
keys.

>
>>> keys, etc. You may want to consider skipping this in the first
>>> iteration and just implementing proper security on the MDS while
>>> relying on RADOS-level options to keep data secure (separate
>>> namespaces or pools for different users, and providing narrow caps to
>>> clients). That would also leave more time for some serious research
>>
>> It sounds like like you're referring to the Sideboard above, which
>> unless I'm wrong won't help for our (not uncommon?) use-case.
>
> Yeah. :/
>
> On Thu, Nov 14, 2013 at 5:13 AM, Arne Wiebalck <Arne.Wiebalck@cern.ch> wrote:
>> It looks to me like Maat is already pretty close to what the use of CephFS on untrusted
>> clients would require.
>>
>> Any idea why Maat does not use a kerberized Ceph service and the corresponding
>> shared session keys? I think that could avoid the generation and distribution of per
>> client keys and outsource the shared key negotation to a standardized mechanism
>> (and allow for integration with existing authN service infrastructures, of course).
>
> I do not. If I were placing bets, because it was a research project
> and they didn't have Kerberos set up already. ;) But they also discuss
> the security implications of using a single shared key, and discard it
> because they want the system to remain mostly secure if an attacker
> compromises a machine (and gets access to its keys).
>
>> Also, why was the Maat idea (which was apparently implemented, tested and
>> benchmarked) not integrated into mainline Ceph?
>
> By the time I arrived this was all history; you'll have to wait until
> Sage gets back for that one. I suspect he didn't like the performance
> enough and the trees had diverged non-trivially by the time Maat was
> in a useful state. But if you are brave, you can go back in time
> (couple years ago) and look for the "security" folder, which I think
> contained the Maat changes, and see what would be involved in
> extracting them. :)

Is it this one?
   https://github.com/ceph/ceph/tree/historic/aleung_mds_security/


Cheers, Dan


> -Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Gregory Farnum]

On Thu, Nov 14, 2013 at 8:21 AM, Dan van der Ster <dan@vanderster.com> wrote:
> On Thu, Nov 14, 2013 at 4:55 PM, Gregory Farnum <greg@inktank.com> wrote:
>> Ah, I hadn't gotten that from skimming the blueprint. So this isn't
>> just about hooking into Kerberos when we do a mount, you want to
>> dynamically retrieve different caps from the monitors as users log on,
>> and have that single client choose between them based on who's making
>> the request?
>
> In our proposal the "additional capabilities" (which allow IOs with a
> particular file/object) are encrypted and sent by the MDS to the
> client who passes it on to the OSD for object IOs.

I'm confused; if two different people are using the same client mount,
the MDS can't distinguish between them unless the client is
cooperating. Isn't that what you were saying it needs to do?

>>>> the MDS tickets in a lot more detail than you have there, though — for
>>>> instance, with your current description you'll need a shared secret
>>>> between the OSD and MDS, and you'll need to expire the client extra
>>>
>>> Right, the shared secret is the kerberos service key in our model.
>>
>> So the whole Ceph cluster has one shared service key, and they use
>> that for securing everything that anybody does? (Sorry, that's
>> probably an elementary question, but I haven't looked at Kerberos
>> since Yehuda implemented the initial CephX auth.)
>
> We've used the shared service key in the past with other distributed
> file systems. But perhaps the model can be changed to per-OSD/MDS
> keys.
>
>> By the time I arrived this was all history; you'll have to wait until
>> Sage gets back for that one. I suspect he didn't like the performance
>> enough and the trees had diverged non-trivially by the time Maat was
>> in a useful state. But if you are brave, you can go back in time
>> (couple years ago) and look for the "security" folder, which I think
>> contained the Maat changes, and see what would be involved in
>> extracting them. :)
>
> Is it this one?
>    https://github.com/ceph/ceph/tree/historic/aleung_mds_security/

Specifically the branches/aleung/security1/ folder in there. (This is
all svn branches converted to a git repository, etc, so the
organization is a bit weird.)
-Greg
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Arne Wiebalck]

On Nov 14, 2013, at 5:37 PM, Gregory Farnum <greg@inktank.com>
 wrote:

> On Thu, Nov 14, 2013 at 8:21 AM, Dan van der Ster <dan@vanderster.com> wrote:
>> On Thu, Nov 14, 2013 at 4:55 PM, Gregory Farnum <greg@inktank.com> wrote:
>>> Ah, I hadn't gotten that from skimming the blueprint. So this isn't
>>> just about hooking into Kerberos when we do a mount, you want to
>>> dynamically retrieve different caps from the monitors as users log on,
>>> and have that single client choose between them based on who's making
>>> the request?
>> 
>> In our proposal the "additional capabilities" (which allow IOs with a
>> particular file/object) are encrypted and sent by the MDS to the
>> client who passes it on to the OSD for object IOs.
> 
> I'm confused; if two different people are using the same client mount,
> the MDS can't distinguish between them unless the client is
> cooperating. Isn't that what you were saying it needs to do?

It's the "who passes it on" which created the confusion? This was to say that
the client kernel module can't decrypt the additional capability (as it does not
have the corresponding key). That does however not mean that the MDS or
the CephFS client module can't distinguish between users as all communication
is done using session keys. That's at least how I understood it :)

>>>>> the MDS tickets in a lot more detail than you have there, though — for
>>>>> instance, with your current description you'll need a shared secret
>>>>> between the OSD and MDS, and you'll need to expire the client extra
>>>> 
>>>> Right, the shared secret is the kerberos service key in our model.
>>> 
>>> So the whole Ceph cluster has one shared service key, and they use
>>> that for securing everything that anybody does? (Sorry, that's
>>> probably an elementary question, but I haven't looked at Kerberos
>>> since Yehuda implemented the initial CephX auth.)
>> 
>> We've used the shared service key in the past with other distributed
>> file systems. But perhaps the model can be changed to per-OSD/MDS
>> keys.
>> 
>>> By the time I arrived this was all history; you'll have to wait until
>>> Sage gets back for that one. I suspect he didn't like the performance
>>> enough and the trees had diverged non-trivially by the time Maat was
>>> in a useful state. But if you are brave, you can go back in time
>>> (couple years ago) and look for the "security" folder, which I think
>>> contained the Maat changes, and see what would be involved in
>>> extracting them. :)
>> 
>> Is it this one?
>>   https://github.com/ceph/ceph/tree/historic/aleung_mds_security/
> 
> Specifically the branches/aleung/security1/ folder in there. (This is
> all svn branches converted to a git repository, etc, so the
> organization is a bit weird.)

Cheers,
 Arne




--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Gregory Farnum]

On Thu, Nov 14, 2013 at 12:30 PM, Arne Wiebalck <Arne.Wiebalck@cern.ch> wrote:
>
> On Nov 14, 2013, at 5:37 PM, Gregory Farnum <greg@inktank.com>
>  wrote:
>
>> On Thu, Nov 14, 2013 at 8:21 AM, Dan van der Ster <dan@vanderster.com> wrote:
>>> On Thu, Nov 14, 2013 at 4:55 PM, Gregory Farnum <greg@inktank.com> wrote:
>>>> Ah, I hadn't gotten that from skimming the blueprint. So this isn't
>>>> just about hooking into Kerberos when we do a mount, you want to
>>>> dynamically retrieve different caps from the monitors as users log on,
>>>> and have that single client choose between them based on who's making
>>>> the request?
>>>
>>> In our proposal the "additional capabilities" (which allow IOs with a
>>> particular file/object) are encrypted and sent by the MDS to the
>>> client who passes it on to the OSD for object IOs.
>>
>> I'm confused; if two different people are using the same client mount,
>> the MDS can't distinguish between them unless the client is
>> cooperating. Isn't that what you were saying it needs to do?
>
> It's the "who passes it on" which created the confusion? This was to say that
> the client kernel module can't decrypt the additional capability (as it does not
> have the corresponding key). That does however not mean that the MDS or
> the CephFS client module can't distinguish between users as all communication
> is done using session keys. That's at least how I understood it :)

Yeah, this sequence is what I'm confused about. As I understand it,
you're going to have an existing client with existing sessions to the
monitor and MDS, and then a user is going to log in and the client is
going to add additional keys to the monitor session, and then switch
back and forth between each user's tickets for different messages to
the MDS? And the MDS will send back "data access" tickets which the
client will further switch between for each OSD op?
-Greg

Re: CDS blueprint: strong auth for cephfs

[Alex Elsayed]

Dan van der Ster wrote:

> Hi Greg,
<snip>
> AFAICT, this would allow single users to mount a subtree on CephFS and
> prevent them from writing where they are not permitted. But our
> use-case is different: we want to mount /cephfs/ from shared
> workstations and batch nodes to store personal home directories,
> shared group areas, and read-only software areas. Managing per-user
> keyrings and per-user mountpoints would become quickly unmanageable.
<snip>

Actually, there are a number of ways this could be managed pretty reasonably 
- eCryptfs, in particular, may be worth examining. It ships with a 
pam_ecryptfs module that (on login) uses the login password to produce the 
decryption key, and load it into the user's session keyring (which is then 
associated with every process in that session, although I don't think 
eCryptfs uses that). It then mounts the eCryptfs filesystem, and the kernel 
uses the loaded key for decryption.

If the credentials are only used on mount, something similar to pam_ecryptfs 
could work, or a minimal module that _only_ loads the key followed by 
pam_mount.

If the credentials are handled on FS operations, then you'd be able to have 
a single top-level mount with minimal privileges and then use the kernel 
keyring on access.

That also opens up using 'keyctl' to manually load the key into the user or 
session keyring, or using trusted/encrypted keys to allow secure automated 
mounts.

In fact, trusted or encrypted keys could be very useful if someone wanted to 
securely use Ceph as a rootfs - they'd make it possible to authenticate 
mounts based on the hardware, not just the configuration.

Re: CDS blueprint: strong auth for cephfs

[Dan van der Ster]

Hi Alex,
The problem is that the cephx keyring would still be needed on
untrusted hosts. With that, anyone can delete/corrupt the underlying
objects (though they may be encrypted) using rados.
Cheers, Dan

On Thu, Nov 14, 2013 at 9:09 PM, Alex Elsayed <eternaleye@gmail.com> wrote:
> Dan van der Ster wrote:
>
>> Hi Greg,
> <snip>
>> AFAICT, this would allow single users to mount a subtree on CephFS and
>> prevent them from writing where they are not permitted. But our
>> use-case is different: we want to mount /cephfs/ from shared
>> workstations and batch nodes to store personal home directories,
>> shared group areas, and read-only software areas. Managing per-user
>> keyrings and per-user mountpoints would become quickly unmanageable.
> <snip>
>
> Actually, there are a number of ways this could be managed pretty reasonably
> - eCryptfs, in particular, may be worth examining. It ships with a
> pam_ecryptfs module that (on login) uses the login password to produce the
> decryption key, and load it into the user's session keyring (which is then
> associated with every process in that session, although I don't think
> eCryptfs uses that). It then mounts the eCryptfs filesystem, and the kernel
> uses the loaded key for decryption.
>
> If the credentials are only used on mount, something similar to pam_ecryptfs
> could work, or a minimal module that _only_ loads the key followed by
> pam_mount.
>
> If the credentials are handled on FS operations, then you'd be able to have
> a single top-level mount with minimal privileges and then use the kernel
> keyring on access.
>
> That also opens up using 'keyctl' to manually load the key into the user or
> session keyring, or using trusted/encrypted keys to allow secure automated
> mounts.
>
> In fact, trusted or encrypted keys could be very useful if someone wanted to
> securely use Ceph as a rootfs - they'd make it possible to authenticate
> mounts based on the hardware, not just the configuration.
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: CDS blueprint: strong auth for cephfs

[Arne Wiebalck]

[-- Attachment #1: Type: text/plain, Size: 2931 bytes --]


On Nov 13, 2013, at 6:45 PM, Gregory Farnum <greg@inktank.com> wrote:

> On Wed, Nov 13, 2013 at 8:05 AM, Dan van der Ster <dan@vanderster.com> wrote:
>> Hi all,
>> This mail is just to let you know that we've prepared a draft
>> blueprint related to adding strong(er) authn/authz to cephfs:
>> 
>> http://wiki.ceph.com/01Planning/02Blueprints/Firefly/Strong_AuthN_and_AuthZ_for_CephFS
>> 
>> The main goal of the idea is that we'd like to be able to use CephFS
>> from untrusted clients:
>>  - the CephX key gives full rw access to pools (e.g. data/metadata)
>> via rados; we cannot distribute this key to untrusted hosts.
>>  - root on untrusted clients can forge their uid/gid and rm -rf /cephfs/*.
>> 
>> In the doc we've proposed one way to add authn/authz to the ceph
>> server side, but perhaps there is a simpler (more feasible in the
>> short term) solution which would enable us to allow untrusted cephfs
>> clients.
> 
> You want to check out:
> http://wiki.ceph.com/01Planning/Sideboard/Client_Security_for_CephFS,
> the previous edition of the client security blueprint
> http://www.ssrc.ucsc.edu/Papers/leung-sc07.pdf, "Scalable Security for
> Petascale Parallel File Systems", which discusses one way of using
> MDS-granted permissions
> http://www.ssrc.ucsc.edu/Papers/li-fast13.pdf, "Horus: Fine-Grained
> Encryption-Based Security for Large-Scale Storage" is largely
> orthogonal in specifics but might inspire some other ides
> 
> It looks to me like you've actually got two different blueprints
> there, though – "Allow Ceph clients to use Kerberos as an entry point"
> and "Enable stronger CephFS security". You're going to need to discuss
> the MDS tickets in a lot more detail than you have there, though — for
> instance, with your current description you'll need a shared secret
> between the OSD and MDS, and you'll need to expire the client extra
> keys, etc. You may want to consider skipping this in the first
> iteration and just implementing proper security on the MDS while
> relying on RADOS-level options to keep data secure (separate
> namespaces or pools for different users, and providing narrow caps to
> clients). That would also leave more time for some serious research
> into how to do a scalable implementation of any ticketing system we
> create.

It looks to me like Maat is already pretty close to what the use of CephFS on untrusted
clients would require.

Any idea why Maat does not use a kerberized Ceph service and the corresponding
shared session keys? I think that could avoid the generation and distribution of per
client keys and outsource the shared key negotation to a standardized mechanism
(and allow for integration with existing authN service infrastructures, of course).

Also, why was the Maat idea (which was apparently implemented, tested and
benchmarked) not integrated into mainline Ceph?

Thanks!
 Arne


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4311 bytes --]