Question about NFS4 facls in combination with a GIT shared bare repo on NFSv4 share

Question about NFS4 facls in combination with a GIT shared bare repo on NFSv4 share

[Joschi Brauchle]

[-- Attachment #1: Type: text/plain, Size: 2306 bytes --]

Hi everyone,

I have three short questions about NFSv4 ACLs. Both originating from 
problems with a shared bare GIT repository located on a NFSv4 share.

Question 1:
==================
After creating a shared bare GIT repo on an NFSv4 share with
---------------
/nfsv4/share # git init --bare --shared=group repodir
/nfsv4/share # ls -l repodir
drwxrwsr-x  7 myuser mygroup   4096 Feb 16 14:12 repodir/
---------------
users on NFSv4 clients cannot push to this repo, but instead get the 
following error message:
---------------
/local/repo # git push
...
remote: fatal: error when closing sha1 file: Permission denied
error: unpack failed: unpack-objects abnormal exit
...
---------------

An strace on the 'git push' command shows:
---------------
# strace -fF git push
[pid  3620] open("objects/0f/tmp_obj_lc5ecp", O_RDWR|O_CREAT|O_EXCL, 
0444) = 3
[pid  3620] brk(0x27ff000)              = 0x27ff000
[pid  3620] write(3, "x\1\235\316A\n\303 
\20\0\300\236}\205\367BPw\325\4J\351Wt]\223\34\214\305l\350\367"..., 
153) = 153
[pid  3620] brk(0x27ef000)              = 0x27ef000
[pid  3620] brk(0x27df000)              = 0x27df000
[pid  3620] brk(0x27de000)              = 0x27de000
[pid  3620] close(3)                    = -1 EACCES (Permission denied)
---------------
where the file "objects/0f/tmp_obj_lc5ecp" is located in 
"/nfsv4/share/repodir/".

What is the problem here really?


Question 2:
==================
The problem of Q1 is solved/worked around by adding the NFSv4 
"write-owner - change ownership of the file/directory" (o flag) for the 
group "mygroup", such that:
---------------
/nfsv4/share # nfs4_getfacl repodir
A:fdg:mygroup@mydomain:rwaDxtTnNcCoy
....
---------------
Why is this needed in addition to the already existing setgid bit?


Question 3:
==================
Note that the problem is NOT solved with this ACL:
---------------
/nfsv4/share # nfs4_getfacl repodir
A:fdg:GROUP@:rwaDxtTnNcCoy
....
/nfsv4/share # ls -l repodir
drwxrwsr-x  7 myuser mygroup   4096 Feb 16 14:12 repodir/
---------------

Why are the same access rights not working for @GROUP (and posix group 
"mygroup" but only when specifying mygroup@mydomain explicitly?



Thanks for any help!

Best regards,
Joschi Brauchle



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4917 bytes --]

Re: Question about NFS4 facls in combination with a GIT shared bare repo on NFSv4 share

[mdw]

On Mon, Feb 16, 2015 at 02:48:34PM +0100, Joschi Brauchle wrote:
...
> Question 3:
> ==================
> Note that the problem is NOT solved with this ACL:
> ---------------
> /nfsv4/share # nfs4_getfacl repodir
> A:fdg:GROUP@:rwaDxtTnNcCoy
> ....
> /nfsv4/share # ls -l repodir
> drwxrwsr-x  7 myuser mygroup   4096 Feb 16 14:12 repodir/
> ---------------
...

I fancy I know something about nfsv4 acls, and I have to confess I can't
figure out what should be happening in your case, because you aren't
including enough detail for me to decide.  Which means I also can't
figure out where or what problem you're having, although I do agree
getting an error on "close" suggests something made a pretty wonky
decision.  I can't guarantee to be helpful in any case, but if you don't
get a better answer from somebody who knows more about your setup,
the following might help:
	
In order to figure out what "should" happen - complete acls, group membership, etc.
Your "...." could hide an awful lot of surprises.
You speak of "domains" - what sort are these and why do you think
they be considered the same as "@GROUP"?

Client & server identities and groups might map differently - is this the case here?

what is the fileserver running?   netapp?  solaris?  linux nfsd??  something else?

Presumably your client is linux - what version?  If from a distribution
which distro & package version?

Can you demonstrate your problem happen with dd or cp?  Or does it only
happen with git?

Are you using gssapi?  (better for tracing purposes if you can do this without.)
Can you record a network trace of the bad thing happening?  (this will
point whether it's a client or server side problem.)

Possibly helpful to know,:
	unix mode permissions work differently than an acl,
	with unix modes,
		owner permissions eclipse group permissions.
	with acls,
		group permissions add to owner permissions.
To accurately map unix mode bits into nfsv4 acls, you have to also
include 'deny' entries to keep groups from adding permissions the user
doen't have.

					-Marcus Watts

Re: Question about NFS4 facls in combination with a GIT shared bare repo on NFSv4 share

[Joschi Brauchle]

[-- Attachment #1: Type: text/plain, Size: 3329 bytes --]

Hi Marcus,

thanks for your reply. I'll provide some more details belog

Question 3:
==================
This is the complete *working* ACL:
---------------
myuser@myhost:/nfsv4/share/> nfs4_getfacl repodir
A:fdg:MYGROUP@mydomain.fqdn:rwaDxtTnNcCoy
A:fd:OWNER@:rwaDdxtTnNcCoy
A::EVERYONE@:rxtncy
D::OWNER@:
D::EVERYONE@:waDTC
---------------

This is the complete non-working ACL:
---------------
myuser@myhost:/nfsv4/share/> nfs4_getfacl repodir
A:fdg:GROUP@:rwaDxtTnNcCoy
A:fd:OWNER@:rwaDdxtTnNcCoy
A::EVERYONE@:rxtncy
D::OWNER@:
D::EVERYONE@:waDTC
---------------

 > what is the fileserver running?   netapp?  solaris?  linux nfsd?? 
something else?
Server:	NetApp FAS 3140 mit ONTAP 8.1.4P1 7-mode

 > Presumably your client is linux - what version?  If from a 
distribution which distro & package version?
Client: openSUSE 13.2, 3.16.7-7-desktop, nfs-client = 1.3.0

 > Are you using gssapi?  (better for tracing purposes if you can do 
this without.)
We are using GSSAPI/Kerberos protected NFSv4 shares mounted like to:
nfsv4_server.fqdn:/vol/myshare on /nfsv4/share type nfs4 
(rw,relatime,vers=4.0,rsize=65536,wsize=65536,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=krb5,local_lock=none)

We have the user "myuser" a member of "mygroup" but not his primary 
group, i.e.
# id myuser
uid=xxx(myuser) ... groups=xxx(mygroup) ...

 > You speak of "domains" - what sort are these and why do you think
 > they be considered the same as "@GROUP"?
 >
We have the following ID mapping on the Linux clients:
# cat /etc/idmapd.conf
---------------
[General]

Verbosity = 0
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
Domain = mydomain.fqdn

[Mapping]

Nobody-User = nobody
Nobody-Group = nobody
---------------
Note that in general, all POSIX ACLS based on user groups are working 
perfectly fine. So ID-mapping seems to be OK in general.

 > Client & server identities and groups might map differently - is this 
the case here?
I have no idea about the server side...

> Can you demonstrate your problem happen with dd or cp?  Or does it only
> happen with git?
Nope, only git so far.

 > Can you record a network trace of the bad thing happening?  (this will
 > point whether it's a client or server side problem.)
Please find a wireshark trace of the failed "git push" command here:
http://wikisend.com/download/940986/wireshark_git.pcapng
(7 days availability) where the problem seems to be
---------------
148	0.070260000	192.168.109.118	10.162.229.2	NFS	438	V4 Call (Reply In 
149) WRITE StateID: 0xcded Offset: 0 Len: 156
149	0.071190000	10.162.229.2	192.168.109.118	NFS	162	V4 Reply (Call In 
148) WRITE Status: NFS4ERR_ACCESS
---------------
me = 192.168.109.118
nfs4_server = 10.162.229.2
Not much more that I can see here unfortunately...

> Possibly helpful to know,:
> 	unix mode permissions work differently than an acl,
> 	with unix modes,
> 		owner permissions eclipse group permissions.
> 	with acls,
> 		group permissions add to owner permissions.
> To accurately map unix mode bits into nfsv4 acls, you have to also
> include 'deny' entries to keep groups from adding permissions the user
> doen't have.

Thanks for the hints and for your help!
I'll do some more thinking about the problem :)

JB


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4917 bytes --]