[patch] cifs: accidentally creating empty files

[patch] cifs: accidentally creating empty files

[Dan Carpenter]

This solves a problem for where a user without write permissions
was creating empty files.  This function was supposed to do a lookup
only, the create happens later.

Signed-off-by: Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>

diff --git a/fs/cifs/dir.c b/fs/cifs/dir.c
index 72d448b..8515afe 100644
--- a/fs/cifs/dir.c
+++ b/fs/cifs/dir.c
@@ -566,11 +566,13 @@ cifs_lookup(struct inode *parent_dir_inode, struct dentry *direntry,
 		if (nd && !(nd->flags & LOOKUP_DIRECTORY) &&
 		     (nd->flags & LOOKUP_OPEN) && !pTcon->broken_posix_open &&
 		     (nd->intent.open.file->f_flags & O_CREAT)) {
+			unsigned int f_flags;
+
+			f_flags = (nd->intent.open.file->f_flags & ~O_CREAT);
 			rc = cifs_posix_open(full_path, &newInode,
 					parent_dir_inode->i_sb,
 					nd->intent.open.create_mode,
-					nd->intent.open.file->f_flags, &oplock,
-					&fileHandle, xid);
+					f_flags, &oplock, &fileHandle, xid);
 			/*
 			 * The check below works around a bug in POSIX
 			 * open in samba versions 3.3.1 and earlier where

Re: [patch] cifs: accidentally creating empty files

[Dan Carpenter]

On Wed, Sep 28, 2011 at 12:28:10AM +0300, Dan Carpenter wrote:
> This solves a problem for where a user without write permissions
> was creating empty files.  This function was supposed to do a lookup
> only, the create happens later.
> 

I never heard back on this.

regards,
dan carpenter

Re: [patch] cifs: accidentally creating empty files

[Steve French]

Looking at it now - thanks for reminding me.

Anyone else review/ack the patch?

On Thu, Oct 13, 2011 at 9:00 AM, Dan Carpenter <dan.carpenter@oracle.com>wrote:

> On Wed, Sep 28, 2011 at 12:28:10AM +0300, Dan Carpenter wrote:
> > This solves a problem for where a user without write permissions
> > was creating empty files.  This function was supposed to do a lookup
> > only, the create happens later.
> >
>
> I never heard back on this.
>
> regards,
> dan carpenter
>
>


-- 
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Jeff Layton]

On Wed, 28 Sep 2011 00:28:10 +0300
Dan Carpenter <dan.carpenter@oracle.com> wrote:

> This solves a problem for where a user without write permissions
> was creating empty files.  This function was supposed to do a lookup
> only, the create happens later.
> 

Not quite. This uses the open-intent goop in the VFS layer that Al Viro
is trying to get rid of. The idea here is that doing a lookup just to
do an open is a waste, when you could just attempt the open. There's no
real reason to exempt creates from that if cifs used a sane permissions
model...

> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
> 
> diff --git a/fs/cifs/dir.c b/fs/cifs/dir.c
> index 72d448b..8515afe 100644
> --- a/fs/cifs/dir.c
> +++ b/fs/cifs/dir.c
> @@ -566,11 +566,13 @@ cifs_lookup(struct inode *parent_dir_inode, struct dentry *direntry,
>  		if (nd && !(nd->flags & LOOKUP_DIRECTORY) &&
>  		     (nd->flags & LOOKUP_OPEN) && !pTcon->broken_posix_open &&
>  		     (nd->intent.open.file->f_flags & O_CREAT)) {
> +			unsigned int f_flags;
> +
> +			f_flags = (nd->intent.open.file->f_flags & ~O_CREAT);
>  			rc = cifs_posix_open(full_path, &newInode,
>  					parent_dir_inode->i_sb,
>  					nd->intent.open.create_mode,
> -					nd->intent.open.file->f_flags, &oplock,
> -					&fileHandle, xid);
> +					f_flags, &oplock, &fileHandle, xid);

	...so this makes it only do the open at lookup time if the
	file already exists.

	I suspect the real problem here is that cifs is trying to
	enforce permissions on the client, which happens after the
	lookup.

	If the client simply allowed the server to handle the
	permissions (and used the right credentials for each user),
	then this would probably work fine. Another nail in the coffin
	for the whole model of client side permissions enforcement,
	IMO...

	In any case, this seems fine to me, so...

>  			/*
>  			 * The check below works around a bug in POSIX
>  			 * open in samba versions 3.3.1 and earlier where

Acked-by: Jeff Layton <jlayton@samba.org>

Re: [patch] cifs: accidentally creating empty files

[Steve French]

On Thu, Oct 13, 2011 at 12:27 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
>
> On Wed, 28 Sep 2011 00:28:10 +0300
> Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>
> > This solves a problem for where a user without write permissions
> > was creating empty files.  This function was supposed to do a lookup
> > only, the create happens later.
> >
>
> Not quite. This uses the open-intent goop in the VFS layer that Al Viro
> is trying to get rid of. The idea here is that doing a lookup just to
> do an open is a waste, when you could just attempt the open. There's no
> real reason to exempt creates from that if cifs used a sane permissions
> model...
>
> > Signed-off-by: Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> >
> > diff --git a/fs/cifs/dir.c b/fs/cifs/dir.c
> > index 72d448b..8515afe 100644
> > --- a/fs/cifs/dir.c
> > +++ b/fs/cifs/dir.c
> > @@ -566,11 +566,13 @@ cifs_lookup(struct inode *parent_dir_inode, struct dentry *direntry,
> >               if (nd && !(nd->flags & LOOKUP_DIRECTORY) &&
> >                    (nd->flags & LOOKUP_OPEN) && !pTcon->broken_posix_open &&
> >                    (nd->intent.open.file->f_flags & O_CREAT)) {
> > +                     unsigned int f_flags;
> > +
> > +                     f_flags = (nd->intent.open.file->f_flags & ~O_CREAT);
> >                       rc = cifs_posix_open(full_path, &newInode,
> >                                       parent_dir_inode->i_sb,
> >                                       nd->intent.open.create_mode,
> > -                                     nd->intent.open.file->f_flags, &oplock,
> > -                                     &fileHandle, xid);
> > +                                     f_flags, &oplock, &fileHandle, xid);
>
>        ...so this makes it only do the open at lookup time if the
>        file already exists.
>
>        I suspect the real problem here is that cifs is trying to
>        enforce permissions on the client, which happens after the
>        lookup.
>
>        If the client simply allowed the server to handle the
>        permissions (and used the right credentials for each user),
>        then this would probably work fine. Another nail in the coffin
>        for the whole model of client side permissions enforcement,
>        IMO...

so it works with "noperm" on.  Presumably we should be recommending
turning noperm and multiuser on (and probably cifsacl if the server is
not samba).

I still would much prefer being able to hook in to the create and open
"closer" to the syscall itself rather than after it is broken down
into (unneeded) lower level fs entry points (for getattr/setattr and
create vs. open).

--
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Steve French]

On Thu, Oct 13, 2011 at 12:27 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> On Wed, 28 Sep 2011 00:28:10 +0300
> Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
>
>> This solves a problem for where a user without write permissions
>> was creating empty files.  This function was supposed to do a lookup
>> only, the create happens later.
>>
>
> Not quite. This uses the open-intent goop in the VFS layer that Al Viro
> is trying to get rid of. The idea here is that doing a lookup just to
> do an open is a waste, when you could just attempt the open. There's no
> real reason to exempt creates from that if cifs used a sane permissions
> model...
>
>> Signed-off-by: Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
>>
>> diff --git a/fs/cifs/dir.c b/fs/cifs/dir.c
>> index 72d448b..8515afe 100644
>> --- a/fs/cifs/dir.c
>> +++ b/fs/cifs/dir.c
>> @@ -566,11 +566,13 @@ cifs_lookup(struct inode *parent_dir_inode, struct dentry *direntry,
>>               if (nd && !(nd->flags & LOOKUP_DIRECTORY) &&
>>                    (nd->flags & LOOKUP_OPEN) && !pTcon->broken_posix_open &&
>>                    (nd->intent.open.file->f_flags & O_CREAT)) {
>> +                     unsigned int f_flags;
>> +
>> +                     f_flags = (nd->intent.open.file->f_flags & ~O_CREAT);
>>                       rc = cifs_posix_open(full_path, &newInode,
>>                                       parent_dir_inode->i_sb,
>>                                       nd->intent.open.create_mode,
>> -                                     nd->intent.open.file->f_flags, &oplock,
>> -                                     &fileHandle, xid);
>> +                                     f_flags, &oplock, &fileHandle, xid);
>
>        ...so this makes it only do the open at lookup time if the
>        file already exists.
>
>        I suspect the real problem here is that cifs is trying to
>        enforce permissions on the client, which happens after the
>        lookup.
>
>        If the client simply allowed the server to handle the
>        permissions (and used the right credentials for each user),

Doesn't this force the create to happen later - rather than
at lookup time where it belongs?

if the issue is just noperm ... we should let this through if the user
has local permissions.  Doubling the cost of a file create to Samba
seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
doubles the roundrip, doubles the load on the server etc.) - the whole
point of this is to let us do an "atomic" open or create operation
and not have to split it into multiple requests.  In any case why would
we do the open and then follow it with a create?

Can we fix this to (at least) narrow the performance penalty.


-- 
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Dan Carpenter]

On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
> Doesn't this force the create to happen later - rather than
> at lookup time where it belongs?
> 
> if the issue is just noperm ... we should let this through if the user
> has local permissions.  Doubling the cost of a file create to Samba
> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
> doubles the roundrip, doubles the load on the server etc.) - the whole
> point of this is to let us do an "atomic" open or create operation
> and not have to split it into multiple requests.  In any case why would
> we do the open and then follow it with a create?
> 
> Can we fix this to (at least) narrow the performance penalty.
> 

Yes, it does add another back and forth to file create...  It's hard/
impossible to check the permissions first and then decide whether to
pass the O_CREAT flag.  Maybe we could add an if statement to check
whether noperm was used and the noperm version could use the create
on lookup.  It's obviously not pretty but it might be worth it for
speedup.

regards,
dan carpenter

Re: [patch] cifs: accidentally creating empty files

[Steve French]

On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter-QHcLZuEGTsthl2p70BpVqQ@public.gmane.orgm> wrote:
> On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
>> Doesn't this force the create to happen later - rather than
>> at lookup time where it belongs?
>>
>> if the issue is just noperm ... we should let this through if the user
>> has local permissions.  Doubling the cost of a file create to Samba
>> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
>> doubles the roundrip, doubles the load on the server etc.) - the whole
>> point of this is to let us do an "atomic" open or create operation
>> and not have to split it into multiple requests.  In any case why would
>> we do the open and then follow it with a create?
>>
>> Can we fix this to (at least) narrow the performance penalty.
>>
>
> Yes, it does add another back and forth to file create...  It's hard/
> impossible to check the permissions first and then decide whether to
> pass the O_CREAT flag.  Maybe we could add an if statement to check
> whether noperm was used and the noperm version could use the create
> on lookup.

Why is it hard to check local permissions? We have the local mode
for the parent.

Also note that the "intent" flags and the atomic create is not just
about performance, but also making it atomic reduces some of the weird
failure cases.



-- 
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Jeff Layton]

On Fri, 28 Oct 2011 10:27:04 -0500
Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
> > On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
> >> Doesn't this force the create to happen later - rather than
> >> at lookup time where it belongs?
> >>
> >> if the issue is just noperm ... we should let this through if the user
> >> has local permissions.  Doubling the cost of a file create to Samba
> >> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
> >> doubles the roundrip, doubles the load on the server etc.) - the whole
> >> point of this is to let us do an "atomic" open or create operation
> >> and not have to split it into multiple requests.  In any case why would
> >> we do the open and then follow it with a create?
> >>
> >> Can we fix this to (at least) narrow the performance penalty.
> >>
> >
> > Yes, it does add another back and forth to file create...  It's hard/
> > impossible to check the permissions first and then decide whether to
> > pass the O_CREAT flag.  Maybe we could add an if statement to check
> > whether noperm was used and the noperm version could use the create
> > on lookup.
> 
> Why is it hard to check local permissions? We have the local mode
> for the parent.
> 
> Also note that the "intent" flags and the atomic create is not just
> about performance, but also making it atomic reduces some of the weird
> failure cases.
> 

We have the local mode for the parent, but we do not have the ownership
and mode for the file that has not yet been created. Because of the
special (ahem) way that cifs handles permissions, it's easily possible
for the ownership of the file not to match the user doing that create.
At that point, later operations on the file can easily fail.

This was the primary reason for the multiuser patch series, and why I
still say that doing permissions checking on the client is a broken
model.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Re: [patch] cifs: accidentally creating empty files

[Steve French]

On Fri, Oct 28, 2011 at 10:44 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> On Fri, 28 Oct 2011 10:27:04 -0500
> Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
>> On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
>> > On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
>> >> Doesn't this force the create to happen later - rather than
>> >> at lookup time where it belongs?
>> >>
>> >> if the issue is just noperm ... we should let this through if the user
>> >> has local permissions.  Doubling the cost of a file create to Samba
>> >> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
>> >> doubles the roundrip, doubles the load on the server etc.) - the whole
>> >> point of this is to let us do an "atomic" open or create operation
>> >> and not have to split it into multiple requests.  In any case why would
>> >> we do the open and then follow it with a create?
>> >>
>> >> Can we fix this to (at least) narrow the performance penalty.
>> >>
>> >
>> > Yes, it does add another back and forth to file create...  It's hard/
>> > impossible to check the permissions first and then decide whether to
>> > pass the O_CREAT flag.  Maybe we could add an if statement to check
>> > whether noperm was used and the noperm version could use the create
>> > on lookup.
>>
>> Why is it hard to check local permissions? We have the local mode
>> for the parent.
>>
>> Also note that the "intent" flags and the atomic create is not just
>> about performance, but also making it atomic reduces some of the weird
>> failure cases.
>>
>
> We have the local mode for the parent, but we do not have the ownership
> and mode for the file that has not yet been created. Because of the
> special (ahem) way that cifs handles permissions, it's easily possible
> for the ownership of the file not to match the user doing that create.
> At that point, later operations on the file can easily fail.
>
> This was the primary reason for the multiuser patch series, and why I
> still say that doing permissions checking on the client is a broken
> model.

We don't have much choice - we have to allow permission
checking on the client when client security context
can't match security context on server - but I would like
to make multiuser the default (especially if we
can figure out a way to integrate winbind-like
upcall for ntlmv2 auth) at least for the future (smb2 etc.)
even if we can't change the default for cifs mounts.



-- 
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Jeff Layton]

On Sat, 29 Oct 2011 00:07:12 -0500
Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> On Fri, Oct 28, 2011 at 10:44 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> > On Fri, 28 Oct 2011 10:27:04 -0500
> > Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> >> On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
> >> > On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
> >> >> Doesn't this force the create to happen later - rather than
> >> >> at lookup time where it belongs?
> >> >>
> >> >> if the issue is just noperm ... we should let this through if the user
> >> >> has local permissions.  Doubling the cost of a file create to Samba
> >> >> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
> >> >> doubles the roundrip, doubles the load on the server etc.) - the whole
> >> >> point of this is to let us do an "atomic" open or create operation
> >> >> and not have to split it into multiple requests.  In any case why would
> >> >> we do the open and then follow it with a create?
> >> >>
> >> >> Can we fix this to (at least) narrow the performance penalty.
> >> >>
> >> >
> >> > Yes, it does add another back and forth to file create...  It's hard/
> >> > impossible to check the permissions first and then decide whether to
> >> > pass the O_CREAT flag.  Maybe we could add an if statement to check
> >> > whether noperm was used and the noperm version could use the create
> >> > on lookup.
> >>
> >> Why is it hard to check local permissions? We have the local mode
> >> for the parent.
> >>
> >> Also note that the "intent" flags and the atomic create is not just
> >> about performance, but also making it atomic reduces some of the weird
> >> failure cases.
> >>
> >
> > We have the local mode for the parent, but we do not have the ownership
> > and mode for the file that has not yet been created. Because of the
> > special (ahem) way that cifs handles permissions, it's easily possible
> > for the ownership of the file not to match the user doing that create.
> > At that point, later operations on the file can easily fail.
> >
> > This was the primary reason for the multiuser patch series, and why I
> > still say that doing permissions checking on the client is a broken
> > model.
> 
> We don't have much choice - we have to allow permission
> checking on the client when client security context
> can't match security context on server - but I would like
> to make multiuser the default (especially if we
> can figure out a way to integrate winbind-like
> upcall for ntlmv2 auth) at least for the future (smb2 etc.)
> even if we can't change the default for cifs mounts.
> 

That's fine, but that doesn't solve Dan's immediate problem. We need to
either take his patch or something like it for 3.2 (and probably for
stable as well) or just rip this crap out altogether.

I'd favor the latter, for the following reason:

----------------------[snip]-------------------
        /*
         * O_EXCL: optimize away the lookup, but don't hash the dentry. Let
         * the VFS handle the create.
         */
        if (nd && (nd->flags & LOOKUP_EXCL)) {
                d_instantiate(direntry, NULL);
                rc = 0;
                goto lookup_out;
        }

	[...]

        if (pTcon->unix_ext) {
                if (nd && !(nd->flags & LOOKUP_DIRECTORY) &&
                     (nd->flags & LOOKUP_OPEN) && !pTcon->broken_posix_open &&
                     (nd->intent.open.file->f_flags & O_CREAT)) {
                        rc = cifs_posix_open(full_path, &newInode,
                                        parent_dir_inode->i_sb,
                                        nd->intent.open.create_mode,
                                        nd->intent.open.file->f_flags, &oplock,
                                        &fileHandle, xid);
----------------------[snip]-------------------

Ok, so we optimize away the lookup in the O_EXCL case (which is fine).

Then, we only attempt a posix open if O_CREAT was set (note that
regular SMB CREATE_ANDX is prohibited here -- why?). Now, with Dan's
patch, we'll be doing this only when O_CREAT is set, and only when the
file actually exists on the server (since we'll be clearing 

This really just makes no sense at all. The times when we can use this
are so vanishingly small that it makes more sense to just remove this
stuff. If you really want to do this, it needs a redesign from the
ground up, with a consideration of each possible case and determination
of whether it's better (and safe) to use an open instead of lookup in
that case.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Re: [patch] cifs: accidentally creating empty files

[Steve French]

On Sat, Oct 29, 2011 at 12:48 AM, Jeff Layton <jlayton@samba.org> wrote:
> On Sat, 29 Oct 2011 00:07:12 -0500
> Steve French <smfrench@gmail.com> wrote:
>
>> On Fri, Oct 28, 2011 at 10:44 PM, Jeff Layton <jlayton@samba.org> wrote:
>> > On Fri, 28 Oct 2011 10:27:04 -0500
>> > Steve French <smfrench@gmail.com> wrote:
>> >
>> >> On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
>> >> > On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
>> >> >> Doesn't this force the create to happen later - rather than
>> >> >> at lookup time where it belongs?
>> >> >>
>> >> >> if the issue is just noperm ... we should let this through if the user
>> >> >> has local permissions.  Doubling the cost of a file create to Samba
>> >> >> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
>> >> >> doubles the roundrip, doubles the load on the server etc.) - the whole
>> >> >> point of this is to let us do an "atomic" open or create operation
>> >> >> and not have to split it into multiple requests.  In any case why would
>> >> >> we do the open and then follow it with a create?
>> >> >>
>> >> >> Can we fix this to (at least) narrow the performance penalty.
>> >> >>
>> >> >
>> >> > Yes, it does add another back and forth to file create...  It's hard/
>> >> > impossible to check the permissions first and then decide whether to
>> >> > pass the O_CREAT flag.  Maybe we could add an if statement to check
>> >> > whether noperm was used and the noperm version could use the create
>> >> > on lookup.
>> >>
>> >> Why is it hard to check local permissions? We have the local mode
>> >> for the parent.
>> >>
>> >> Also note that the "intent" flags and the atomic create is not just
>> >> about performance, but also making it atomic reduces some of the weird
>> >> failure cases.
>> >>
>> >
>> > We have the local mode for the parent, but we do not have the ownership
>> > and mode for the file that has not yet been created. Because of the
>> > special (ahem) way that cifs handles permissions, it's easily possible
>> > for the ownership of the file not to match the user doing that create.
>> > At that point, later operations on the file can easily fail.
>> >
>> > This was the primary reason for the multiuser patch series, and why I
>> > still say that doing permissions checking on the client is a broken
>> > model.
>>
>> We don't have much choice - we have to allow permission
>> checking on the client when client security context
>> can't match security context on server - but I would like
>> to make multiuser the default (especially if we
>> can figure out a way to integrate winbind-like
>> upcall for ntlmv2 auth) at least for the future (smb2 etc.)
>> even if we can't change the default for cifs mounts.
>>
>
> That's fine, but that doesn't solve Dan's immediate problem. We need to
> either take his patch or something like it for 3.2 (and probably for
> stable as well) or just rip this crap out altogether.

The benchmark gains are huge for a couple test cases
(e.g. creating lots of small files) - doesn't make sense to throw it out.
At worst (I think we can do better) - we only give up on atomic
open/create for when noperm is not set (we want our users
to be running noperm or equivalently multiuser).   It is especially
important since the servers we do best at with create turn out
to be Linux ( and other Unix which Samba runs on (due to posix extensions)).

If we break create apart as you suggest - we run into
lookup/getattr/create/open/setattr
races which are also annoying - we are supposed to send an atomic
create/open operation
to minimize strange consequences if the server file changes in between
(the alternative)
where getattr/create/open/setattr is done.




-- 
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Jeff Layton]

On Sat, 29 Oct 2011 17:03:46 -0500
Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> On Sat, Oct 29, 2011 at 12:48 AM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> > On Sat, 29 Oct 2011 00:07:12 -0500
> > Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> >> On Fri, Oct 28, 2011 at 10:44 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> >> > On Fri, 28 Oct 2011 10:27:04 -0500
> >> > Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >> >
> >> >> On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
> >> >> > On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
> >> >> >> Doesn't this force the create to happen later - rather than
> >> >> >> at lookup time where it belongs?
> >> >> >>
> >> >> >> if the issue is just noperm ... we should let this through if the user
> >> >> >> has local permissions.  Doubling the cost of a file create to Samba
> >> >> >> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
> >> >> >> doubles the roundrip, doubles the load on the server etc.) - the whole
> >> >> >> point of this is to let us do an "atomic" open or create operation
> >> >> >> and not have to split it into multiple requests.  In any case why would
> >> >> >> we do the open and then follow it with a create?
> >> >> >>
> >> >> >> Can we fix this to (at least) narrow the performance penalty.
> >> >> >>
> >> >> >
> >> >> > Yes, it does add another back and forth to file create...  It's hard/
> >> >> > impossible to check the permissions first and then decide whether to
> >> >> > pass the O_CREAT flag.  Maybe we could add an if statement to check
> >> >> > whether noperm was used and the noperm version could use the create
> >> >> > on lookup.
> >> >>
> >> >> Why is it hard to check local permissions? We have the local mode
> >> >> for the parent.
> >> >>
> >> >> Also note that the "intent" flags and the atomic create is not just
> >> >> about performance, but also making it atomic reduces some of the weird
> >> >> failure cases.
> >> >>
> >> >
> >> > We have the local mode for the parent, but we do not have the ownership
> >> > and mode for the file that has not yet been created. Because of the
> >> > special (ahem) way that cifs handles permissions, it's easily possible
> >> > for the ownership of the file not to match the user doing that create.
> >> > At that point, later operations on the file can easily fail.
> >> >
> >> > This was the primary reason for the multiuser patch series, and why I
> >> > still say that doing permissions checking on the client is a broken
> >> > model.
> >>
> >> We don't have much choice - we have to allow permission
> >> checking on the client when client security context
> >> can't match security context on server - but I would like
> >> to make multiuser the default (especially if we
> >> can figure out a way to integrate winbind-like
> >> upcall for ntlmv2 auth) at least for the future (smb2 etc.)
> >> even if we can't change the default for cifs mounts.
> >>
> >
> > That's fine, but that doesn't solve Dan's immediate problem. We need to
> > either take his patch or something like it for 3.2 (and probably for
> > stable as well) or just rip this crap out altogether.
> 
> The benchmark gains are huge for a couple test cases
> (e.g. creating lots of small files) - doesn't make sense to throw it out.
> At worst (I think we can do better) - we only give up on atomic
> open/create for when noperm is not set (we want our users
> to be running noperm or equivalently multiuser).   It is especially
> important since the servers we do best at with create turn out
> to be Linux ( and other Unix which Samba runs on (due to posix extensions)).
> 

I don't care at all about performance if the behavior is wrong. We need
to worry about correctness first and only then optimize for
performance. The create on lookup code has been the source of many
regressions and I think it's time to pull the plug on it until someone
wants to take the time to do it right.

I think it's obvious that the logic in cifs_lookup makes no sense. I'd
prefer to see that ripped out until someone presents a design for this
that does make sense, but if you want to present code that fixes this
more logically then I'd be happy to look at it.

> If we break create apart as you suggest - we run into
> lookup/getattr/create/open/setattr
> races which are also annoying - we are supposed to send an atomic
> create/open operation
> to minimize strange consequences if the server file changes in between
> (the alternative)
> where getattr/create/open/setattr is done.
> 

We only have to worry about atomicity in the O_EXCL case, and that is
already covered in the earlier check in cifs_lookup.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Re: [patch] cifs: accidentally creating empty files

[Steve French]

I thought we agreed that (at worst) the logic works for the "noperm"
(no client side enforcement, ie multiuser case).

On Sat, Oct 29, 2011 at 9:48 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> On Sat, 29 Oct 2011 17:03:46 -0500
> Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
>> On Sat, Oct 29, 2011 at 12:48 AM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
>> > On Sat, 29 Oct 2011 00:07:12 -0500
>> > Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >
>> >> On Fri, Oct 28, 2011 at 10:44 PM, Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
>> >> > On Fri, 28 Oct 2011 10:27:04 -0500
>> >> > Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >> >
>> >> >> On Fri, Oct 28, 2011 at 2:20 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
>> >> >> > On Wed, Oct 19, 2011 at 09:14:37PM -0500, Steve French wrote:
>> >> >> >> Doesn't this force the create to happen later - rather than
>> >> >> >> at lookup time where it belongs?
>> >> >> >>
>> >> >> >> if the issue is just noperm ... we should let this through if the user
>> >> >> >> has local permissions.  Doubling the cost of a file create to Samba
>> >> >> >> seems like a bad idea (ie doing aQueryPathInfo AND an NTCreateX
>> >> >> >> doubles the roundrip, doubles the load on the server etc.) - the whole
>> >> >> >> point of this is to let us do an "atomic" open or create operation
>> >> >> >> and not have to split it into multiple requests.  In any case why would
>> >> >> >> we do the open and then follow it with a create?
>> >> >> >>
>> >> >> >> Can we fix this to (at least) narrow the performance penalty.
>> >> >> >>
>> >> >> >
>> >> >> > Yes, it does add another back and forth to file create...  It's hard/
>> >> >> > impossible to check the permissions first and then decide whether to
>> >> >> > pass the O_CREAT flag.  Maybe we could add an if statement to check
>> >> >> > whether noperm was used and the noperm version could use the create
>> >> >> > on lookup.
>> >> >>
>> >> >> Why is it hard to check local permissions? We have the local mode
>> >> >> for the parent.
>> >> >>
>> >> >> Also note that the "intent" flags and the atomic create is not just
>> >> >> about performance, but also making it atomic reduces some of the weird
>> >> >> failure cases.
>> >> >>
>> >> >
>> >> > We have the local mode for the parent, but we do not have the ownership
>> >> > and mode for the file that has not yet been created. Because of the
>> >> > special (ahem) way that cifs handles permissions, it's easily possible
>> >> > for the ownership of the file not to match the user doing that create.
>> >> > At that point, later operations on the file can easily fail.
>> >> >
>> >> > This was the primary reason for the multiuser patch series, and why I
>> >> > still say that doing permissions checking on the client is a broken
>> >> > model.
>> >>
>> >> We don't have much choice - we have to allow permission
>> >> checking on the client when client security context
>> >> can't match security context on server - but I would like
>> >> to make multiuser the default (especially if we
>> >> can figure out a way to integrate winbind-like
>> >> upcall for ntlmv2 auth) at least for the future (smb2 etc.)
>> >> even if we can't change the default for cifs mounts.
>> >>
>> >
>> > That's fine, but that doesn't solve Dan's immediate problem. We need to
>> > either take his patch or something like it for 3.2 (and probably for
>> > stable as well) or just rip this crap out altogether.
>>
>> The benchmark gains are huge for a couple test cases
>> (e.g. creating lots of small files) - doesn't make sense to throw it out.
>> At worst (I think we can do better) - we only give up on atomic
>> open/create for when noperm is not set (we want our users
>> to be running noperm or equivalently multiuser).   It is especially
>> important since the servers we do best at with create turn out
>> to be Linux ( and other Unix which Samba runs on (due to posix extensions)).
>>
>
> I don't care at all about performance if the behavior is wrong. We need
> to worry about correctness first and only then optimize for
> performance. The create on lookup code has been the source of many
> regressions and I think it's time to pull the plug on it until someone
> wants to take the time to do it right.
>
> I think it's obvious that the logic in cifs_lookup makes no sense. I'd
> prefer to see that ripped out until someone presents a design for this
> that does make sense, but if you want to present code that fixes this
> more logically then I'd be happy to look at it.
>
>> If we break create apart as you suggest - we run into
>> lookup/getattr/create/open/setattr
>> races which are also annoying - we are supposed to send an atomic
>> create/open operation
>> to minimize strange consequences if the server file changes in between
>> (the alternative)
>> where getattr/create/open/setattr is done.
>>
>
> We only have to worry about atomicity in the O_EXCL case, and that is
> already covered in the earlier check in cifs_lookup.
>
> --
> Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>
>



-- 
Thanks,

Steve

Re: [patch] cifs: accidentally creating empty files

[Jeff Layton]

On Sat, 29 Oct 2011 21:53:13 -0500
Steve French <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> I thought we agreed that (at worst) the logic works for the "noperm"
> (no client side enforcement, ie multiuser case).
> 

It may work in that case, but that's really just an accident. It still
doesn't make any sense for the reasons I've already described.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>