Re: some problems about permission of subdirectory

Re: some problems about permission of subdirectory

[houlinfei]

aGkgS2luZ2xvbmcsCiBJIGRvd25sb2FkIG5mcy11dGlscy0xLjMuNCwgYnV0IGl0IGlzbid0IGhh
cyBkZWJpYW4gZGlyZWN0b3J5LiBTbyBpIGNhbid0IGJ1aWxkIGRlYiBwYWNrYWdlIG9uIHVidW50
dSBzeXN0ZW0uIE5vdywgaWYgbmZzLXV0aWxzLTEuMy40IHN1cHBvcnQgdWJ1bnR1IHN5c3RlbT8K
dGhhbmtzIHZlcnkgbXVjaCwKaG91bGluZmVp

Re: some problems about permission of subdirectory

[Kinglong Mee]

On 2/25/2017 17:44, houlinfei wrote:
> hi Kinglong,
>  I download nfs-utils-1.3.4, but it isn't has debian directory. So i can't build deb package on ubuntu system. Now, if nfs-utils-1.3.4 support ubuntu system?

I'm not familiar with ubuntu/debian,
but I think nfs-utils is supported on them.
Without debain directory, you can use the make/Makefile to install it
following the README.

# sh autogen.sh
# ./confiugre
# make
# make install 

thanks,
Kinglong Mee

Re: some problems about permission of subdirectory

[houlinfei]

aGkgS2luZ2xvbmcsIEJydWNlLApUd28gc3VnZ2VzdGlvbnMgYWxsIGNhbiByZXNvbHZlIHRoZSBw
cm9ibGVtLGJ1dCB3aGVuIGkgdXNlIHRoZSB3YXkgb2YgbW91bnQgLS1iaW5kIC9yb290L2hoL2ho
MSAvcm9vdC9oaC9oaDEgLCBhbmQgY2xpZW50IGhhcyBtb3VudCB0aGlzIG5mcyBzaGFyZS4gV2hl
biBpIHVtb3VudCAvcm9vdC9oaC9oaDEuSXQgd2lsbCBidXN5LiB3aGVuIGkgdW1vdW50IHNoYXJl
IG9uIGNsaWVudCBhbmQgc3RvcCBuZnNkIG9uIHNlcnZlci4gVGhlbiBpIGNhbiB1bW91bnQgL3Jv
b3QvaGgvaGgxLiBXaHk/CnRoYW5rcyB2ZXJ5IG11Y2gsCmhvdWxpbmZlaQ==

Re: some problems about permission of subdirectory

[Kinglong Mee]

On 2/25/2017 21:26, houlinfei wrote:
> hi Kinglong, Bruce,
> Two suggestions all can resolve the problem,but when i use the way of mount --bind /root/hh/hh1 /root/hh/hh1 , and client has mount this nfs share. When i umount /root/hh/hh1.It will busy. when i umount share on client and stop nfsd on server. Then i can umount /root/hh/hh1. Why?

Yes, that's a old problem.

For each export entry, nfsd has some cache in kernel,each valid cache
has a reference to the export directory.
The umount operation finds that and report busy, after stop the service,
all cache will be dropped(no any reference to export entry), umount success.

I have try to fix the problem at 2015, 
http://www.spinics.net/lists/linux-nfs/msg52646.html

Ps, if someone think it's really a requirement, I will pick it up?

thanks,
Kinglong Mee

Re: some problems about permission of subdirectory

[houlinfei]

aGkgS2luZ2xvbmcsCkkgbW9kaWZ5IHRoZSBjb2RlIGFuZCB0ZXN0LCBpdCdzIG9rYXkuCnRoYW5r
cyB2ZXJ5IG11Y2gsCmhvdWxpbmZlaQ==

Re: some problems about permission of subdirectory

[houlinfei]

aGkgS2luZ2xvbmcsCkkgdHJ5IGFnYWluLCBpdCdzIG9rYXkuIGJ1dCB3ZSB3YW50IHRvIG1vZGlm
eSB0aGUgY29kZSwgdGlsbCBub3cgd2UgYXJlIG5vdCBmYW1pbGlhciB3aXRoIG5mcywgY2FuIHlv
dSB0ZWxsIHVzIHdoZXJlIHNob3VsZCB3ZSBtb2RpZnk/CnRoYW5rcywKaG91bGluZmVp

Re: some problems about permission of subdirectory

[Kinglong Mee]

Cc Bruce, Steven,

On 2/23/2017 11:53, houlinfei wrote:
> hi Kinglong,
> I try again, it's okay. but we want to modify the code, till now we are not familiar with nfs, can you tell us where should we modify?

The logic is set a mark on the exported dir when exporting dir,
nfsd kernel checks the mark when finding export entry.

There are two patches I have a little test,
fs/nfsd/vfs.c is the kernel source, and 
utils/exportfs/exportfs.c is the nfs-utils source.

diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 26c6fdb..c03566c 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -456,6 +456,7 @@ nfsd_setattr(struct svc_rqst *rqstp, struct svc_fh *fhp, struct iattr *iap,
  * NFS junction information is stored in an extended attribute.
  */
 #define NFSD_JUNCTION_XATTR_NAME	XATTR_TRUSTED_PREFIX "junction.nfs"
+#define NFSD_EXPORTED_XATTR_NAME	NFSD_JUNCTION_XATTR_NAME ".exported"
 
 /**
  * nfsd4_is_junction - Test if an object could be an NFS junction
@@ -471,6 +472,8 @@ int nfsd4_is_junction(struct dentry *dentry)
 
 	if (inode == NULL)
 		return 0;
+	if (vfs_getxattr(dentry, NFSD_EXPORTED_XATTR_NAME, NULL, 0) > 0)
+		return 1;
 	if (inode->i_mode & S_IXUGO)
 		return 0;
 	if (!(inode->i_mode & S_ISVTX))

diff --git a/utils/exportfs/exportfs.c b/utils/exportfs/exportfs.c
index 02d5b6d..3c6e36b 100644
--- a/utils/exportfs/exportfs.c
+++ b/utils/exportfs/exportfs.c
@@ -28,6 +28,7 @@
 #include <errno.h>
 #include <limits.h>
 #include <time.h>
+#include <linux/xattr.h>
 
 #define INT_TO_LONG_THRESHOLD_SECS (INT_MAX - (60 * 60 * 24))
 
@@ -556,6 +557,9 @@ validate_export(nfs_export *exp)
 		return;
 
 	}
+
+#define NFSD_EXPORTED_XATTR_NAME  XATTR_TRUSTED_PREFIX "junction.nfs.exported"
+	setxattr(path, NFSD_EXPORTED_XATTR_NAME, "exported", 8, 0);
 }
 
 static _Bool


thanks,
Kinglong Mee

Re: some problems about permission of subdirectory

[houlinfei]

aGkgS2luZ2xvbmc6ClRoYW5rIHlvdSBmb3IgeW91ciBhbnN3ZXIgYWJvdXQgdGhpcyBwaGVub21l
bm9uLCBpIGp1c3QgdHJ5IHRoZSBtZXRob2QgeW91IHByb3ZpZGUsIGJ1dCBwcm9ibGVtIGFsc28g
ZXhpc3QuIElmIGkgd2FudCB0byBjaGFuZ2UgdGhlIGNvZGUgYWJvdXQgbmZzZCBhbmQgcnBjLm1v
dW50ZCwgd2hlcmUgc2hvdWxkIGkgY2hhbmdlPyBJJ20gbm90IGZhbWlsaWFyIHdpdGggdGhpcyBw
YXJ0IG9mIHRoZSBjb2RlLiAKVGhhbmtzIHZlcnkgbXVjaH4KaG91bGluZmVp

Re: some problems about permission of subdirectory

[Kinglong Mee]

On 2/22/2017 15:16, houlinfei wrote:
> hi Kinglong:
> Thank you for your answer about this phenomenon, i just try the method you provide, but problem also exist. If i want to change the code about nfsd and rpc.mountd, where should i change? I'm not familiar with this part of the code. 

Sorry, I test it with the following sets (with "no_root_squash"),
/root/hh/hh1    *(rw,no_subtree_check,no_root_squash,insecure)

It's okay for your problem, but all client's process will be treated as root.
But, it's just a hacker method with many limits,
I think you'd better change your exports as,

/root/hh/hh1  *(ro,sync,insecure,no_subtree_check)
/root/hh/hh2  *(rw,sync,insecure,no_subtree_check)

thanks,
Kinglong Mee

some problems about permission of subdirectory

[houlinfei]

CmhpIGV2ZXJ5b25lOgpJIG1ldCBhIHByb2JsZW0gYWJvdXQgc3ViZGlyZWN0b3J5IHBlcm1pc3Np
b24gd2hlbiBjbGllbnQgbW91bnQgdGhpcyBzdWJkaXJlY3RvcnkgdXNpbmcgbmZzNC4gRm9yIGV4
YW1wbGU6CnRoZSBjb250ZW50cyBvZiB0aGUgL2V0Yy9leHBvcnRzIGZpbGUgaXMKL3Jvb3QvaGgg
KihybyxzeW5jLGluc2VjdXJlLG5vX3N1YnRyZWVfY2hlY2spCi9yb290L2hoL2hoMSAqKHJ3LHN5
bmMsaW5zZWN1cmUsbm9fc3VidHJlZV9jaGVjaykKYW5kIHRoZSB0d28gZGlyZWN0b3J5IHBlcm1p
c3Npb24gaXMgNzc3LiBBbmQgdGhlIHBhcmVudCBkaXJlY3RvcnkncyBleHBvcnQgcGVybWlzc2lv
biBpcyByZWFkLW9ubHksIHRoZSBzdWJkaXJlY3RvcnkncyBleHBvcnQgcGVybWlzc2lvbiBpcyBy
ZWFkLXdyaXRlLgpUaGVuIGNsaWVudCBtb3VudCAvcm9vdC9oaC9oaDEgb24gL21udC95eSB1c2lu
ZyBuZnM0LiBCdXQgdGhlIC9tbnQveXkgZGlyZWN0b3J5IG9ubHkgY2FuIHJlYWQuSWYgY2xpZW50
IG1vdW50IC9yb290L2hoL2hoMSBvbiAvbW50L3l5IHVzaW5nIG5mczMsIHRoZSAvbW50L3l5IGNh
biB3cml0ZS4KV2hvIGtub3cgaG93IHRvIHNvbHZlIHRoaXMgcHJvYmxlbSBhYm91dCBuZnM0PyBU
aGFua3MgdmVyeSBtdWNofgo=

Re: some problems about permission of subdirectory

[Kinglong Mee]

On 2/21/2017 16:52, houlinfei wrote:
> 
> hi everyone:
> I met a problem about subdirectory permission when client mount this subdirectory using nfs4. For example:
> the contents of the /etc/exports file is
> /root/hh *(ro,sync,insecure,no_subtree_check)
> /root/hh/hh1 *(rw,sync,insecure,no_subtree_check)
> and the two directory permission is 777. And the parent directory's export permission is read-only, the subdirectory's export permission is read-write.
> Then client mount /root/hh/hh1 on /mnt/yy using nfs4. But the /mnt/yy directory only can read.If client mount /root/hh/hh1 on /mnt/yy using nfs3, the /mnt/yy can write.

nfs3 gets the filehandle of /root/hh/hh1 from rpc.mountd before really mounting, 
so that, nfs3 do the later process with the filehandle of /root/hh/hh1,
with the second exports entry.
 
But, nfs4 get the filehandle by LOOKUP through nfsd step by step, 
at first, LOOKUP "/" as the pseudo filesystem with an pseudo exports entry,
second, LOOKUP "/root/" also use the pseudo export entry, 
next, LOOKUP "/root/hh/" will get a new export entry
for "/root/" use a pseudo export entry, but at last LOOKUP "/root/hh/hh1",
nfsd uses the export entry for "/root/hh/" that isn't a pseudo entry entry.

So that, nfsv3 client can write the directory, but nfsv4 client can't.

> Who know how to solve this problem about nfs4? Thanks very much~

Without change any codes of rpc.mountd and nfsd, there is a hacker method for it.
# chmod -x /root/hh/hh1
# chmod +t /root/hh/hh1
# setfattr -n "trusted.junction.nfs" -v "anything" /root/hh/hh1

Umount the nfs and remount as nfsv4.

Cc Bruce, Neil, Steve,

Is it needed adding an xattr as "junction.nfs" for fixing this problem?

thanks,
Kinglong Mee

Re: some problems about permission of subdirectory

[J. Bruce Fields]

On Tue, Feb 21, 2017 at 09:50:01PM +0800, Kinglong Mee wrote:
> On 2/21/2017 16:52, houlinfei wrote:
> > 
> > hi everyone:
> > I met a problem about subdirectory permission when client mount this subdirectory using nfs4. For example:
> > the contents of the /etc/exports file is
> > /root/hh *(ro,sync,insecure,no_subtree_check)
> > /root/hh/hh1 *(rw,sync,insecure,no_subtree_check)
> > and the two directory permission is 777. And the parent directory's export permission is read-only, the subdirectory's export permission is read-write.
> > Then client mount /root/hh/hh1 on /mnt/yy using nfs4. But the /mnt/yy directory only can read.If client mount /root/hh/hh1 on /mnt/yy using nfs3, the /mnt/yy can write.
> 
> nfs3 gets the filehandle of /root/hh/hh1 from rpc.mountd before really mounting, 
> so that, nfs3 do the later process with the filehandle of /root/hh/hh1,
> with the second exports entry.
>  
> But, nfs4 get the filehandle by LOOKUP through nfsd step by step, 
> at first, LOOKUP "/" as the pseudo filesystem with an pseudo exports entry,
> second, LOOKUP "/root/" also use the pseudo export entry, 
> next, LOOKUP "/root/hh/" will get a new export entry
> for "/root/" use a pseudo export entry, but at last LOOKUP "/root/hh/hh1",
> nfsd uses the export entry for "/root/hh/" that isn't a pseudo entry entry.
> 
> So that, nfsv3 client can write the directory, but nfsv4 client can't.
> 
> > Who know how to solve this problem about nfs4? Thanks very much~
> 
> Without change any codes of rpc.mountd and nfsd, there is a hacker method for it.
> # chmod -x /root/hh/hh1
> # chmod +t /root/hh/hh1
> # setfattr -n "trusted.junction.nfs" -v "anything" /root/hh/hh1
> 
> Umount the nfs and remount as nfsv4.
> 
> Cc Bruce, Neil, Steve,
> 
> Is it needed adding an xattr as "junction.nfs" for fixing this problem?

Maybe.  Or another trick you can use right now is to create a mountpoint
there by mounting that directory on top of itself:

	mount --bind /root/hh/hh1 /root/hh/hh1

However, I strongly discourage this kind of setup.

The problem is that it's very easy for an attacker to fake up a
filehandle that points to a file under /root/hh while looking like it
points to a file under /root/hh/hh1, and therefore get rw access to
something outside /root/hh/hh1.  Turning on "subtree_check" will fix
that problem, but can cause other problems.

It's much better, whenever possible, to use entirely different
filesystems whenever you need to grant different access.

--b.

Re: some problems about permission of subdirectory

[Kinglong Mee]

On 2/25/2017 03:14, J. Bruce Fields wrote:
> On Tue, Feb 21, 2017 at 09:50:01PM +0800, Kinglong Mee wrote:
>> On 2/21/2017 16:52, houlinfei wrote:
>>>
>>> hi everyone:
>>> I met a problem about subdirectory permission when client mount this subdirectory using nfs4. For example:
>>> the contents of the /etc/exports file is
>>> /root/hh *(ro,sync,insecure,no_subtree_check)
>>> /root/hh/hh1 *(rw,sync,insecure,no_subtree_check)
>>> and the two directory permission is 777. And the parent directory's export permission is read-only, the subdirectory's export permission is read-write.
>>> Then client mount /root/hh/hh1 on /mnt/yy using nfs4. But the /mnt/yy directory only can read.If client mount /root/hh/hh1 on /mnt/yy using nfs3, the /mnt/yy can write.
>>
>> nfs3 gets the filehandle of /root/hh/hh1 from rpc.mountd before really mounting, 
>> so that, nfs3 do the later process with the filehandle of /root/hh/hh1,
>> with the second exports entry.
>>  
>> But, nfs4 get the filehandle by LOOKUP through nfsd step by step, 
>> at first, LOOKUP "/" as the pseudo filesystem with an pseudo exports entry,
>> second, LOOKUP "/root/" also use the pseudo export entry, 
>> next, LOOKUP "/root/hh/" will get a new export entry
>> for "/root/" use a pseudo export entry, but at last LOOKUP "/root/hh/hh1",
>> nfsd uses the export entry for "/root/hh/" that isn't a pseudo entry entry.
>>
>> So that, nfsv3 client can write the directory, but nfsv4 client can't.
>>
>>> Who know how to solve this problem about nfs4? Thanks very much~
>>
>> Without change any codes of rpc.mountd and nfsd, there is a hacker method for it.
>> # chmod -x /root/hh/hh1
>> # chmod +t /root/hh/hh1
>> # setfattr -n "trusted.junction.nfs" -v "anything" /root/hh/hh1
>>
>> Umount the nfs and remount as nfsv4.
>>
>> Cc Bruce, Neil, Steve,
>>
>> Is it needed adding an xattr as "junction.nfs" for fixing this problem?
> 
> Maybe.  Or another trick you can use right now is to create a mountpoint
> there by mounting that directory on top of itself:
> 
> 	mount --bind /root/hh/hh1 /root/hh/hh1
> 
> However, I strongly discourage this kind of setup.
> 
> The problem is that it's very easy for an attacker to fake up a
> filehandle that points to a file under /root/hh while looking like it
> points to a file under /root/hh/hh1, and therefore get rw access to
> something outside /root/hh/hh1.  Turning on "subtree_check" will fix
> that problem, but can cause other problems.
> 
> It's much better, whenever possible, to use entirely different
> filesystems whenever you need to grant different access.

Hi linfei,

Can the two suggestions resolve your requirements?
I don't think the hacker method that change the code is sensible.

thanks,
Kinglong Mee