CIFS VFS

CIFS VFS

[Steven French]

Although we are still working through a few bugs found during recent runs
of the fsx file system test suite against 2.5.38 and 2.5.39 (and working
on a few new features at the same time), suggestions for improving the
2.5 version of the CIFS VFS would be welcome.   So far most of the feedback
(e.g. on the recent addition of NTLMSSP security provider compatability and
some of the MS DFS work in progress) has been from the Samba community
(rather than Linux kernel side) since I have been doing much of my testing
against Samba servers and the project info is hosted there.    The current
2.5 source for the CIFS VFS is at
      http://cifs.bkbits.net/linux-2.5
and the project web site is at
      http://us1.samba.org/samba/Linux_CIFS_client.html

      Steve French
      Senior Software Engineer
      Linux Technology Center - IBM Austin
      phone: 512-838-2294
      email: sfrench@us.ibm.com

Re: CIFS VFS

[Christoph Hellwig]

On Mon, Sep 30, 2002 at 11:25:40PM -0500, Steven French wrote:
> Although we are still working through a few bugs found during recent runs
> of the fsx file system test suite against 2.5.38 and 2.5.39 (and working
> on a few new features at the same time), suggestions for improving the
> 2.5 version of the CIFS VFS would be welcome.   So far most of the feedback
> (e.g. on the recent addition of NTLMSSP security provider compatability and
> some of the MS DFS work in progress) has been from the Samba community
> (rather than Linux kernel side) since I have been doing much of my testing
> against Samba servers and the project info is hosted there.    The current
> 2.5 source for the CIFS VFS is at

Would you mind giving an short outline what the idea behind this filesystem is?
It looks like it's an replacement for smbfs, but what are the advantages and
why didn't you start with smbfs?

Re: CIFS VFS

[Steven French]

Sure.   This had been discussed among the Samba community starting at last
year's CIFS conference and again at the plugfests and interoperability
events this year.
I do not view the CIFS VFS as necessarily replacing smbfs, since smbfs
provides
interoperability with older servers (prior to NT 4) which is still of value
to some
users.   There were basically three reasons that we did not start from
smbfs and
felt that an additional filesystem was necessary to support the newer
version of the protocol.

1) Limited reuse potential - smbfs was missing quite a bit of the newer
function needed
which after looking at for a while and seemed easier to implement from
scratch rather
than try to force fit into smbfs, and customers could then choose whether
they wanted
to use the newer more experimental CIFS vfs or the older smbfs.

2) The recommendation of one of the previous smbfs maintainers (and some of
the Samba team)
last year who put a complete rewrite of smbfs fairly high on the list of
the "samba-related"
items that needed to get done.  Comments included perceived design problems
in the original
design of smbfs,  lack of structure definitions for most key protocol
elements and no support
for a few key Samba features.

3)  smbfs seemed to have different design goals including broad
compatability with old
servers (much like nfs v2 provides broad compatability vs. nfs v4 which
provides useful new
function) while we wanted to take advantage of features only available in
current servers
and also to provide a more useful reference implementation for the client
end of new
extensions to CIFS such as the CIFS Unix Extensions (Samba already provides
this
for the server side) as our work in the CIFS Technical Workgroup in the
Storage
Networking Industry Association continues.   In addition there were other
design differences
that were fairly fundamental - an important chunk of smbfs is implemented
in user space,
which makes implementing ms dfs (heirarchical name space) difficult and
would make it
impossible to boot (diskless systems) from and also complicates
reconnection.   Finally
another big difference in design goals is that smbfs preferentially uses
Netbios naming and
port 139 (as it should as an smb filesystem) while the CIFS VFS
preferentially
uses TCP naming and port 445 (as CIFS systems are supposed to) although it
can fall
back to 139.

The design goals that we started with were the following.   Most of these
were not
implemented in smbfs when we started, although a few have been added to the
kernel as patches (e.g. on Urban's web site) and others.

Features we have implemented in the CIFS VFS so far:
1) "Pure TCP/IP" (port 445 in addition to RFC1001 port 139) support  (NB
the
mount.cifs mount helper, needed for calling gethostbyname, is not quite
complete)
2) Full POSIX support including use of CIFS Unix Extensions and NT
hardlinks etc.
3) Support for "raw NTLMSSP" (security negotiation) and more secure session
establishment
4) Large file handling (larger than 2GB up to 2**63 in size)
5) Internationalization improvements (Unicode on the wire)
6) Use of SetPathinfo instead of SetFileInfo where appropriate, use of
current levels for
CIFS transact2 calls
7) Use of 32 bit errors (cifs network status codes) vs old dos error codes.
8) Byte Range and File locks
9) Improved support for Symbolic links  (NB Samba bug blocks some
POSIX test cases for some types of symlink names)
10) Improved debugging, procfs support
11) Case sensitive searches (done) and Unix behavioral defaults

Features that are being worked (in progress):
1) Transparent reconnection  after tcp or cifs session failure
2) Globally rooted, AFS-like heirarchical name space
3) Kerberos/SPNego authentication
4) NTLMv2 encryption
5) Distributed Lock/Token Management (e.g. Level I and II oplocks) -
safe client data caching
6) optional Multiple security contexts and transparent local to network
UUID mapping
7) Performance improvements (e.g. command chaining, overlapping multiplexed
PDUs to same server, reduced copy overhead, gather send using multiple
iovecs,
optimized dialect usage)

And features that we will be starting on next:
1) Per SMB security signatures (needed in many cases for .Net server
compat)
2) Replicated volumes
3) Mounts to multi-component UNC names (mounts to
4) Improved PAM/NSSwitch module integration (if a network authentication
module such as Winbind or PAM_Kerberos or PAM_LDAP are configured, for
example)
5) Network alias support (not started)
6) Directory and File change notification for optional safe caching of
inode metadata (not started)
7) Support for CIFS ACLs and xattr support (not started)
8) Improved Support for Disk Quotas (not started)
9) CIFS Sparse files and some related network IOCTLs (not started)
10) Offline files and CIFS heirarchical storage support


The CIFS VFS has been going through lots of interoperability testing,
starting at Connectathon
and including this years CIFS Conference and plugfest (smbfs was not tested
at these).   IBM
developed the initial smb filesystem years ago and it is still the "native"
remote filesystem
for OS/400 (and of course IBM PC-DOS and OS/2) so it also seemed
appropriate that we
should help out Linux interoperability in this area especially given the
explosion of storage
appliances that implement CIFS and also the large number of Windows and
Samba servers.


Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench@us.ibm.com

Re: CIFS VFS

[Matthew Wilcox]

[can you switch to a unix mail system?  it is positively painful to
read the shit that notes claims to be email]

On Tue, Oct 01, 2002 at 12:30:28PM -0500, Steven French wrote:
> 8) Byte Range and File locks

there's some changes in progress around file locking in 2.5... let
me know your requirements.

> 5) Distributed Lock/Token Management (e.g. Level I and II oplocks) -
> safe client data caching

this is also going to need some changes; again, talk to me about what
you want.

-- 
Revolutions do not require corporate support.

Re: CIFS VFS

[Petr Vandrovec]

On  1 Oct 02 at 19:42, Matthew Wilcox wrote:
> 
> On Tue, Oct 01, 2002 at 12:30:28PM -0500, Steven French wrote:
> > 8) Byte Range and File locks
> 
> there's some changes in progress around file locking in 2.5... let
> me know your requirements.

If I can place ncpfs's requirements here... 

For ncpfs easiest interface would be one which:

(1) does not merge locks; Netware does not do that, you must release
    exact records you locked, otherwise unlock will fail.
    
(2) first resolve locks locally, and then, if you succesfully obtained 
    lock locally, consult filesystem's lock method; ncpfs must do it 
    that way because of there is only one server filehandle for each 
    opened file, and so from server's view there is only one entity on 
    connection.

I would be also satisfied with interface F_GETLK uses: call default
posix_test_lock only if filesystem returned LOCK_USE_CLNT: in such
case filesystem can return < 0 on failure, LOCK_USE_CLNT when
lock succeeded, but kernel should do local bookkeeping itself,
or 0 when lock succeeded, and inode's i_flock was already updated.

If current method (first call filesystem, then obtain lock locally) is 
going to stay, then I believe that we should call F_UNLOCK when 
filesystem's lock succeeded, but local posix_lock_file failed (either 
due to EAGAIN or EINTR or ...). Otherwise server is never notified
that lock was not acquired by workstation, and will be never released.

                                                    Petr Vandrovec
                                                    vandrove@vc.cvut.cz
                                                    

Re: CIFS VFS

[Steven French]

For CIFS some of the same considerations apply - lock
merging is not what the servers expect and can cause
problems similar to ncpfs, so allowing a filesystem to
indicate that lock merging is not supported (perhaps
via a flag in the superblock) would be useful.  In addition
there is no way to specify advisory locks so mandatory locks
might be better indicated at the superblock level or via
the file_system_type struct rather than doing the
setguid/groupexecute mode trick.   There was a very strange set
of lock tests that the Samba team had fun working through about
a year ago - including what zero length locks mean in various
scenarios.   I will have to think through whether any of those
issues would apply coming from a Linux client and going to Samba
or Windows servers and I also need to look through the current
2.5 lock changes in more detail.

>> there's some changes in progress around file locking in 2.5... let
>> me know your requirements.


>For ncpfs easiest interface would be one which:

>(1) does not merge locks; Netware does not do that, you must release
    exact records you locked, otherwise unlock will fail.

>(2) first resolve locks locally, and then, if you succesfully obtained
    lock locally, consult filesystem's lock method; ncpfs must do it
    that way because of there is only one server filehandle for each
    opened file, and so from server's view there is only one entity on
    connection.



    Steve French
    Senior Software Engineer
    Linux Technology Center - IBM Austin
    phone: 512-838-2294
    email: sfrench@us.ibm.com

Re: CIFS VFS

[Matthew Wilcox]

On Tue, Oct 01, 2002 at 10:00:39PM +0200, Petr Vandrovec wrote:
> If I can place ncpfs's requirements here... 

Certainly!  Happy to hear them.

> (1) does not merge locks; Netware does not do that, you must release
>     exact records you locked, otherwise unlock will fail.

Yeah, NFS v4 has the same problem (potentially).

> (2) first resolve locks locally, and then, if you succesfully obtained 
>     lock locally, consult filesystem's lock method; ncpfs must do it 
>     that way because of there is only one server filehandle for each 
>     opened file, and so from server's view there is only one entity on 
>     connection.

How about this: if filesystem provides its own ->lock method, locks.c
promises to leave i_flock member alone?  That way your ->lock method
can do all this by itself.

The major problem with this is removing locks on file close.  Either we
need a magic value for PID that means PID_ANY or we need a new f_op method.
I favour the magic PID_ANY flag myself.

> I would be also satisfied with interface F_GETLK uses: call default
> posix_test_lock only if filesystem returned LOCK_USE_CLNT: in such
> case filesystem can return < 0 on failure, LOCK_USE_CLNT when
> lock succeeded, but kernel should do local bookkeeping itself,
> or 0 when lock succeeded, and inode's i_flock was already updated.
> 
> If current method (first call filesystem, then obtain lock locally) is 
> going to stay, then I believe that we should call F_UNLOCK when 
> filesystem's lock succeeded, but local posix_lock_file failed (either 
> due to EAGAIN or EINTR or ...). Otherwise server is never notified
> that lock was not acquired by workstation, and will be never released.

I think the scheme I propose will work fine for you, since you'll still
be able to call the local file locking code so you have a record of what
locks you've set on the server (necessary for crash recovery, i guess?)

I'm not averse to having some kind of split/merge lock library functions
in fs/locks.c either, even if they're not directly used by the local
file locking code.  I think several network filesystems will need this
behaviour.

-- 
Revolutions do not require corporate support.

Re: CIFS VFS

[Petr Vandrovec]

On Wed, Oct 02, 2002 at 03:17:43AM +0100, Matthew Wilcox wrote:
> On Tue, Oct 01, 2002 at 10:00:39PM +0200, Petr Vandrovec wrote:
> 
> > (2) first resolve locks locally, and then, if you succesfully obtained 
> >     lock locally, consult filesystem's lock method; ncpfs must do it 
> >     that way because of there is only one server filehandle for each 
> >     opened file, and so from server's view there is only one entity on 
> >     connection.
> 
> How about this: if filesystem provides its own ->lock method, locks.c
> promises to leave i_flock member alone?  That way your ->lock method
> can do all this by itself.

Yes, it would work. Although same as for CIFS applies - all locks are
mandatory, not advisory - but it should not matter for locking, as
properly written apps work same way on both system, and if someone
tries to read locked file, he'll get what he deserves...

> The major problem with this is removing locks on file close.  Either we
> need a magic value for PID that means PID_ANY or we need a new f_op method.
> I favour the magic PID_ANY flag myself.

ncpfs can be happy with "standardized" contents of i_flock, so VFS can
peek at list of locks placed on file by local workstation.

> > If current method (first call filesystem, then obtain lock locally) is 
> > going to stay, then I believe that we should call F_UNLOCK when 
> > filesystem's lock succeeded, but local posix_lock_file failed (either 
> > due to EAGAIN or EINTR or ...). Otherwise server is never notified
> > that lock was not acquired by workstation, and will be never released.
> 
> I think the scheme I propose will work fine for you, since you'll still
> be able to call the local file locking code so you have a record of what
> locks you've set on the server (necessary for crash recovery, i guess?)

Just make sure that if you call filesystem's lock, you do not call
posix_lock_file. As there is currently only one lock's user (nfs), it 
should cause no problems to move posix_lock_file at the end of nfs_lock 
instead of doing it in caller.

> I'm not averse to having some kind of split/merge lock library functions
> in fs/locks.c either, even if they're not directly used by the local
> file locking code.  I think several network filesystems will need this
> behaviour.

					Petr Vandrovec
					vandrove@vc.cvut.cz

Re: CIFS VFS

[Urban Widmark]

On Tue, 1 Oct 2002, Steven French wrote:

> Sure.   This had been discussed among the Samba community starting at last
> year's CIFS conference and again at the plugfests and interoperability
> events this year.
> I do not view the CIFS VFS as necessarily replacing smbfs, since smbfs
> provides
> interoperability with older servers (prior to NT 4) which is still of value

A vast majority of smbfs users are using NT4+. win9x is noticable but the
others are not making a lot of noise. Under those circumstances I think
having two filesystems for the same thing is a waste of resources (time).

I would suggest a replace or a merge. Not really sure how such things are
decided or if it is possible to get some idea in advance what the higher
authorities would prefer (Al?).

If smbfs should be replaced, or be placed in maintenance-only mode, then
there is no point in trying to introduce smbconnect before 2.6/3.0.


> 3)  smbfs seemed to have different design goals including broad
> compatability with old
> servers (much like nfs v2 provides broad compatability vs. nfs v4 which
> provides useful new
> function) while we wanted to take advantage of features only available in
> current servers
> and also to provide a more useful reference implementation for the client
> end of new
> extensions to CIFS such as the CIFS Unix Extensions (Samba already provides
> this
> for the server side) as our work in the CIFS Technical Workgroup in the
> Storage
> Networking Industry Association continues.   In addition there were other
> design differences
> that were fairly fundamental - an important chunk of smbfs is implemented
> in user space,
> which makes implementing ms dfs (heirarchical name space) difficult and
> would make it
> impossible to boot (diskless systems) from and also complicates
> reconnection.   Finally
> another big difference in design goals is that smbfs preferentially uses
> Netbios naming and
> port 139 (as it should as an smb filesystem) while the CIFS VFS
> preferentially
> uses TCP naming and port 445 (as CIFS systems are supposed to) although it
> can fall
> back to 139.

I personally feel that none of the issues in this point is very strong:

+ Compatibility with old systems can be provided with function pointer
  "classes" (see struct smb_ops in fs/smbfs/proc.c). So eventually the
  open call for os2 is different from the one used with win3000.

  This is more or less how the unix extensions was added.

  Old stuff can be moved to a smbfs_oldies.o module and doesn't even have
  to take up any space. There will still be a lot of things that can be
  shared. And if it breaks and you can't test it, wait for some poor soul
  that uses os2 and get them to send tcpdump's - that's what I do now.

  I believe nfs v2 vs v3 is supported by the same kernel module in this
  manner. Unlike nfs, smb/cifs doesn't have such strict versions.

+ smbmount is completely replaceable, nothing of the "internals" of smbfs
  is really connected to it. It is even possible to have both in-kernel
  connection code and a userspace tool at the same time.

  The big advantage of smbmount is that it reuses samba code, so in samba
  3.0 adding kerberos support to smbmount is (AFAIK) a matter of doing:

	"c->use_kerberos = 1;"

  CIFS VFS will have to bring the kerberos code into the kernel, or?
  If you don't you will depend on the userspace helper in exactly the same
  way smbfs depends on it.

+ The dfs stuff is possible to solve by changing smbmount to be a tool
  only for making connections (/bin/mount would mount smbfs directly).
  This "smbconnect" can be called the same way modprobe is run. I have
  rough but functioning patches for this vs 2.4.14.

+ Can't diskless systems use initrd?
  People that have asked about these things seem more troubled by the size
  of smbmount (the drawback of being a part of samba).

+ Isn't the difference between port 139 vs 445 only in the setup part?
  Well, if it is then that's handled by libsmb and smbfs really doesn't
  care. If not it's not a huge change.

The other two reasons aren't really described in enough detail (and this
letter is too long anyway). But that doesn't matter. I don't see the code
in the two being so different that you can't copy how things are done
between them.


> The CIFS VFS has been going through lots of interoperability testing,
> starting at Connectathon
> and including this years CIFS Conference and plugfest (smbfs was not tested
> at these).   IBM

No, the pricetags for both are significantly over my testing budget.
I'm not terribly fond of signing NDA's either ...

/Urban