[NFS] re-exporting NFS-mounted dir over NFS

[NFS] re-exporting NFS-mounted dir over NFS

[Martin Schuster (IFKL IT OS DSM CD)]

I am in the process of setting up an "NFS-proxy", a machine which
mounts directories from an NetApp-filer using NFS3, and should re-export
them using NFS4 (with Kerberos).

But apparently nfsd doesn't support re-exporting NFS-mounted dirs:
# mount | grep home
netapp.example.com:/vol/home/schumar on /srv/nfs4/home/schumar type nfs
# cat /etc/exports
/srv/nfs4/home/schumar  gss/krb5(rw,sync,fsid=0,secure,no_subtree_check)
# exportfs -r
exportfs: Warning: /srv/nfs4/home/schumar does not support NFS export.

I traced this back to linux-2.6.25.1/fs/nfsd/export.c, where it says, starting
in line 386:
        if (!inode->i_sb->s_export_op ||
            !inode->i_sb->s_export_op->fh_to_dentry) {
                dprintk("exp_export: export of invalid fs type.\n");
                return -EINVAL;
        }

(and a quick look in fs/nfs/super.c confirmed that the nfs-client never
sets an export_op)

Is there a technical reason for this (i.e. is it simply theoretically
impossible to re-export an NFS-mount), or is my use-case so strange that
nobody has ever needed this until now, and thus it just wasn't coded?
Or am I just stupid/blind? (wouldn't be the first time :)

Thanks in advance,
-- 
Infineon Technologies IT-Services GmbH   Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/w@public.gmane.org
Lakeside B05, 9020 Klagenfurt, Austria   Martin Schuster
         FB: LG Klagenfurt, FN 246787y   +43 5 1777 3517

Re: [NFS] re-exporting NFS-mounted dir over NFS

[Peter Staubach]

Martin Schuster (IFKL IT OS DSM CD) wrote:
> I am in the process of setting up an "NFS-proxy", a machine which
> mounts directories from an NetApp-filer using NFS3, and should re-export
> them using NFS4 (with Kerberos).
>
> But apparently nfsd doesn't support re-exporting NFS-mounted dirs:
> # mount | grep home
> netapp.example.com:/vol/home/schumar on /srv/nfs4/home/schumar type nfs
> # cat /etc/exports
> /srv/nfs4/home/schumar  gss/krb5(rw,sync,fsid=0,secure,no_subtree_check)
> # exportfs -r
> exportfs: Warning: /srv/nfs4/home/schumar does not support NFS export.
>
> I traced this back to linux-2.6.25.1/fs/nfsd/export.c, where it says, starting
> in line 386:
>         if (!inode->i_sb->s_export_op ||
>             !inode->i_sb->s_export_op->fh_to_dentry) {
>                 dprintk("exp_export: export of invalid fs type.\n");
>                 return -EINVAL;
>         }
>
> (and a quick look in fs/nfs/super.c confirmed that the nfs-client never
> sets an export_op)
>
> Is there a technical reason for this (i.e. is it simply theoretically
> impossible to re-export an NFS-mount), or is my use-case so strange that
> nobody has ever needed this until now, and thus it just wasn't coded?
> Or am I just stupid/blind? (wouldn't be the first time :)
>
> Thanks in advance,
>   
Is the real goal to be able to export the files using krb5
authentication or the use of NFSv4?

If the former, then why not just export the files from the
NetApp using Kerberos?

If the latter, then I suspect that it won't provide much, if
any, benefit.  It would still be limited to the NFSv3 semantics
of the file system.

    Thanx...

       ps

Re: [NFS] re-exporting NFS-mounted dir over NFS

[Martin Schuster (IFKL IT OS DSM CD)]

Thanks for your thoughts about this.

Peter Staubach wrote:
> Is the real goal to be able to export the files using krb5
> authentication or the use of NFSv4?
> 
Both, I fear.

> If the former, then why not just export the files from the
> NetApp using Kerberos?
> 
> If the latter, then I suspect that it won't provide much, if
> any, benefit.  It would still be limited to the NFSv3 semantics
> of the file system.
>
The current NFS4-support in NetApps OnTap is afaik quite new,
so our filer administrator doesn't want to enable it in the
near future; he prefers waiting until the issues that are likely
to come up are solved before allowing it on a productive machine.

But mounting directly from the filer using NFS3+Kerberos would
allow the following attack vector, as the clients are in an
unsecure network (i.e. could get root access on their machines):
 User mounts an directory using his Kerberos-credentials
 User gets root, then changes w/o password to another user
 User can now read the files of that other user, as the NFS3-server
     doesn't check the permissions

(at least, that's how I understood the difference between NFS3
 and NFS4 -- please correct me if I'm wrong)

So my question still is: Is re-exporting an NFS-mount technically
impossible, or does it just need some coding to get it working?

Thanks in advance,
-- 
Infineon Technologies IT-Services GmbH   Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/w@public.gmane.org
Lakeside B05, 9020 Klagenfurt, Austria   Martin Schuster
         FB: LG Klagenfurt, FN 246787y   +43 5 1777 3517

Re: [NFS] re-exporting NFS-mounted dir over NFS

[Peter Staubach]

Martin Schuster (IFKL IT OS DSM CD) wrote:
> Thanks for your thoughts about this.
>
> Peter Staubach wrote:
>   
>> Is the real goal to be able to export the files using krb5
>> authentication or the use of NFSv4?
>>
>>     
> Both, I fear.
>
>   
>> If the former, then why not just export the files from the
>> NetApp using Kerberos?
>>
>> If the latter, then I suspect that it won't provide much, if
>> any, benefit.  It would still be limited to the NFSv3 semantics
>> of the file system.
>>
>>     
> The current NFS4-support in NetApps OnTap is afaik quite new,
> so our filer administrator doesn't want to enable it in the
> near future; he prefers waiting until the issues that are likely
> to come up are solved before allowing it on a productive machine.
>
> But mounting directly from the filer using NFS3+Kerberos would
> allow the following attack vector, as the clients are in an
> unsecure network (i.e. could get root access on their machines):
>  User mounts an directory using his Kerberos-credentials
>  User gets root, then changes w/o password to another user
>  User can now read the files of that other user, as the NFS3-server
>      doesn't check the permissions
>
> (at least, that's how I understood the difference between NFS3
>  and NFS4 -- please correct me if I'm wrong)
>
>   

Ahh, no.  All versions of the NFS servers check permissions on each
and every file access.  Even NFSv2.  NFSv3 and NFSv4 support an
ACCESS protocol operation which allows the client to ask the server
for which file access permissions that the user would be allowed to
have.

When a file system is exported using krb5, then all file accesses
must be made with the right kerberos credential or access will be
denied.  The attack that you described, while working for AUTH_SYS,
does not work for RPCSEC_GSS with krb5, no matter which version of
the NFS protocol that you are using.


> So my question still is: Is re-exporting an NFS-mount technically
> impossible, or does it just need some coding to get it working?

It may be technically possible, in some situations, but is not
something that is always possible or has ever been supported.
It could easily be used to thwart security.  A compromised
client, which was allowed access to file systems from the server,
could then re-export those file systems to other clients which
should not have been allowed access.

----

The bottom line is that 1) I don't think that the NFSv4
implementation from NetApp is as bad as feared and 2) that
using NFSv3 with krb5 should be as secure as NFSv4 with krb5.
Give either or both a shot.  I think that you will be pleasantly
surprised.

       ps

Re: [NFS] re-exporting NFS-mounted dir over NFS

[J. Bruce Fields]

On Thu, Jun 05, 2008 at 07:47:35AM -0400, Peter Staubach wrote:
> The bottom line is that 1) I don't think that the NFSv4
> implementation from NetApp is as bad as feared and 2) that
> using NFSv3 with krb5 should be as secure as NFSv4 with krb5.

That's true for the protocol itself, though depending on the threat
you're worried about, the fact that NFSv4 allows the equivalent of mount
calls to be done with krb5 security (and thus thwarts spoofing of the
replies) may be an advantage for NFSv4.

--b.

> Give either or both a shot.  I think that you will be pleasantly
> surprised.

Re: [NFS] re-exporting NFS-mounted dir over NFS

[Chuck Lever]

On Thu, Jun 5, 2008 at 2:26 AM, Martin Schuster (IFKL IT OS DSM CD)
<Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/w@public.gmane.org> wrote:
> Thanks for your thoughts about this.
>
> Peter Staubach wrote:
>> Is the real goal to be able to export the files using krb5
>> authentication or the use of NFSv4?
>>
> Both, I fear.
>
>> If the former, then why not just export the files from the
>> NetApp using Kerberos?
>>
>> If the latter, then I suspect that it won't provide much, if
>> any, benefit.  It would still be limited to the NFSv3 semantics
>> of the file system.
>>
> The current NFS4-support in NetApps OnTap is afaik quite new,
> so our filer administrator doesn't want to enable it in the
> near future; he prefers waiting until the issues that are likely
> to come up are solved before allowing it on a productive machine.

The NFSv4 server in OnTAP is several years old, actually.  There were
some problems with it in the 6.5 and 7.0 time frame, but if your filer
is running a current OnTAP release (7.2.2, I think, is the most recent
GD release) you should be fine.  The "early adopter period" is long
over for NFSv4 support in OnTAP.

Mounting the filer directly will definitely be more secure (and
perform better) than going through an NFSv3 gateway.  I think Peter
has covered most of the details already.

> So my question still is: Is re-exporting an NFS-mount technically
> impossible, or does it just need some coding to get it working?

Aside from the fact that it is not advisable to do, I don't think it
is technically possible on Linux with the kernel NFS server.
NFS-exportable file systems must have special hooks to construct NFS
file handles from on-disk inodes correctly, and I don't think the
Linux NFS client provides those hooks.  You might have better luck
using the user-space NFS server on your gateway system, but it has
plenty of known issues.

-- 
Chuck Lever

Re: [NFS] re-exporting NFS-mounted dir over NFS

[J. Bruce Fields]

On Wed, Jun 04, 2008 at 04:10:58PM +0200, Martin Schuster (IFKL IT OS DSM CD) wrote:
> I am in the process of setting up an "NFS-proxy", a machine which
> mounts directories from an NetApp-filer using NFS3, and should re-export
> them using NFS4 (with Kerberos).
> 
> But apparently nfsd doesn't support re-exporting NFS-mounted dirs:
> # mount | grep home
> netapp.example.com:/vol/home/schumar on /srv/nfs4/home/schumar type nfs
> # cat /etc/exports
> /srv/nfs4/home/schumar  gss/krb5(rw,sync,fsid=0,secure,no_subtree_check)
> # exportfs -r
> exportfs: Warning: /srv/nfs4/home/schumar does not support NFS export.
> 
> I traced this back to linux-2.6.25.1/fs/nfsd/export.c, where it says, starting
> in line 386:
>         if (!inode->i_sb->s_export_op ||
>             !inode->i_sb->s_export_op->fh_to_dentry) {
>                 dprintk("exp_export: export of invalid fs type.\n");
>                 return -EINVAL;
>         }
> 
> (and a quick look in fs/nfs/super.c confirmed that the nfs-client never
> sets an export_op)
> 
> Is there a technical reason for this (i.e. is it simply theoretically
> impossible to re-export an NFS-mount), or is my use-case so strange that
> nobody has ever needed this until now, and thus it just wasn't coded?

It's certainly not trivial to implement re-export.  It's probably also
possible, at least in theory, but would be difficult (and probably
wouldn't work terribly well).  So, like Peter Staubach says, I'd try to
talk the server administrator into turning on krb5 on the filer.

--b.

> Or am I just stupid/blind? (wouldn't be the first time :)
> 
> Thanks in advance,
> -- 
> Infineon Technologies IT-Services GmbH   Martin.Schuster1-d0qZbvYSIPpWk0Htik3J/w@public.gmane.org
> Lakeside B05, 9020 Klagenfurt, Austria   Martin Schuster
>          FB: LG Klagenfurt, FN 246787y   +43 5 1777 3517
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html