SMB2 file system - should it be a distinct module

SMB2 file system - should it be a distinct module

[Steve French]

During SambaXP conference last week, one of the Samba 4 developers
talked about the work  doing a prototype server for SMB2 protocol
which got me thinking about the client side of this again.

As some background Windows Vista includes support for a new network
file system protocol, SMB2 (which is the default used on mount ie NET
USE or equivalent).  It is somewhat related to CIFS, although the
header format is bigger, better aligned and more of the operations are
handle based (rather than path based).  SMB2 is somewhat better than
CIFS in a few ways (although we would have to add Unix Extensions as
we have with CIFS to make it ideal for Linux).

Now that we (Samba team) understand enough about it to implement
prototypes (there is a prototype server in Samba 4, and a userspace
client library for testing), we need to decide whether the kernel
implementation of SMB2 client should be a distinct module or just part
of the cifs.ko module.

SMB2 (the protocol) is smaller than cifs, and obviously no backlevel
dialects to support (only slightly more than 12 different SMB2 network
commands would have to be used, although two have a couple of
different information levels each that the client would need to send).
 My guess is that less than 1/3 of the cifs module would overlap - but
that overlap is enough that it would be easier to do smb2 as part of
cifs (although it would make the cifs module somewhat larger).
During protocol negotiation (SMB Negotiate Protocol) SMB2 is listed as
a different dialect - if the server accepts SMB2 dialect then
subsequent commands begin with 0xFE instead of 0xFF and the header
fields are bigger and better aligned.

Any idea which would be preferred (smb2 support as part of cifs, or as
a distinct smb2.ko module)?

-- 
Thanks,

Steve

Re: SMB2 file system - should it be a distinct module

[Chris Friesen]

Steve French wrote:

> ...we need to decide whether the kernel
> implementation of SMB2 client should be a distinct module or just part
> of the cifs.ko module.

<snip>

> My guess is that less than 1/3 of the cifs module would overlap - but
> that overlap is enough that it would be easier to do smb2 as part of
> cifs (although it would make the cifs module somewhat larger).

What about pulling out the common code into a helper module, which can 
then be used by both cifs and smb2?

Chris

Re: SMB2 file system - should it be a distinct module

[Christoph Hellwig]

On Mon, Apr 30, 2007 at 05:52:43PM -0500, Steve French wrote:
> Any idea which would be preferred (smb2 support as part of cifs, or as
> a distinct smb2.ko module)?

Separate module please.  If we grow enough common code as some point
we can move it into a smb_common.ko helper library, but given your
above description I doubt that would useful.

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[simo]

On Tue, 2007-05-01 at 10:06 +0100, Christoph Hellwig wrote:
> On Mon, Apr 30, 2007 at 05:52:43PM -0500, Steve French wrote:
> > Any idea which would be preferred (smb2 support as part of cifs, or as
> > a distinct smb2.ko module)?
> 
> Separate module please.  If we grow enough common code as some point
> we can move it into a smb_common.ko helper library, but given your
> above description I doubt that would useful.

Separate modules would mean the user have to know which protocol to
choose each time. And this make little sense.
You really want to auto-negotiate which protocol to use, because you
could have at the same time a connection to a Vista/Longhorn (SMB2)
machine and one to a Windows 2000 server (plain SMB) in the same domain
using the same credentials.

To me it make no sense to separate them out, unless one can load the
other on demand when needed and be able to pass all relevant data and
the network connection to the other.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer
email: idra@samba.org
http://samba.org

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Christoph Hellwig]

On Thu, May 03, 2007 at 09:21:29AM -0400, simo wrote:
> Separate modules would mean the user have to know which protocol to
> choose each time. And this make little sense.

Of course it makes a lot of sense.  We don't have anyfs.ko either
because some ubuntu users are too braindead to know what's on their
disk.

> You really want to auto-negotiate which protocol to use, because you
> could have at the same time a connection to a Vista/Longhorn (SMB2)
> machine and one to a Windows 2000 server (plain SMB) in the same domain
> using the same credentials.

So what?

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[simo]

On Thu, 2007-05-03 at 15:17 +0100, Christoph Hellwig wrote:
> On Thu, May 03, 2007 at 09:21:29AM -0400, simo wrote:
> > Separate modules would mean the user have to know which protocol to
> > choose each time. And this make little sense.
> 
> Of course it makes a lot of sense.  We don't have anyfs.ko either
> because some ubuntu users are too braindead to know what's on their
> disk.
> 
> > You really want to auto-negotiate which protocol to use, because you
> > could have at the same time a connection to a Vista/Longhorn (SMB2)
> > machine and one to a Windows 2000 server (plain SMB) in the same domain
> > using the same credentials.
> 
> So what?

I guess DFS referrals can work cross protocol, so if you are redirected
from a longhorn server to a windoes 2000 or a samba server you want to
be able to follow the DFS referral and not return an error.
To do that you need to have either 1 module that support both protocols
or a way from one module to call the other. Just separating the 2
without any glue will not work (or you will have to add some userspace
upcall hack to make it work).

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer
email: idra@samba.org
http://samba.org

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Christoph Hellwig]

On Thu, May 03, 2007 at 10:36:53AM -0400, simo wrote:
> I guess DFS referrals can work cross protocol, so if you are redirected
> from a longhorn server to a windoes 2000 or a samba server you want to
> be able to follow the DFS referral and not return an error.
> To do that you need to have either 1 module that support both protocols
> or a way from one module to call the other. Just separating the 2
> without any glue will not work (or you will have to add some userspace
> upcall hack to make it work).

You can easily redirect from one filesystem to another through
->follow_link, and you can mount new filesystems instances from there.

Can we please stop the bullshitting now?  And can you please get a realname
before playing smart ass on filesystem developers lists?

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[simo]

On Thu, 2007-05-03 at 15:43 +0100, Christoph Hellwig wrote:
> On Thu, May 03, 2007 at 10:36:53AM -0400, simo wrote:
> > I guess DFS referrals can work cross protocol, so if you are redirected
> > from a longhorn server to a windoes 2000 or a samba server you want to
> > be able to follow the DFS referral and not return an error.
> > To do that you need to have either 1 module that support both protocols
> > or a way from one module to call the other. Just separating the 2
> > without any glue will not work (or you will have to add some userspace
> > upcall hack to make it work).
> 
> You can easily redirect from one filesystem to another through
> ->follow_link, and you can mount new filesystems instances from there.
> 
> Can we please stop the bullshitting now?  And can you please get a realname
> before playing smart ass on filesystem developers lists?

I guess you just had a bad day and are in a bad mood, I am have no
interest in arguing with you if this is the tone of the discussion.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer
email: idra@samba.org
http://samba.org

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Chuck Ebbert]

Christoph Hellwig wrote:
> 
> Can we please stop the bullshitting now?  And can you please get a realname
> before playing smart ass on filesystem developers lists?

Simo's full name is in his .sig

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Gerald Carter]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Simo,

> I guess DFS referrals can work cross protocol, so if you are redirected
> from a longhorn server to a windoes 2000 or a samba server you want to
> be able to follow the DFS referral and not return an error.
> To do that you need to have either 1 module that support both protocols
> or a way from one module to call the other. Just separating the 2
> without any glue will not work (or you will have to add some userspace
> upcall hack to make it work).

Long term I agree that CIFS and SMB2 should be in the same .ko
But NTLM 0.12 still works for Vista and DFS referrals.
Breaking out SMB2 initially means that it will not clutter
the working cifs.ko code.  Remember that an SMB2 client fs is
mostly research at this point, and not engineering.



cheers, jerry
=====================================================================
Samba                                    ------- http://www.samba.org
Centeris                         -----------  http://www.centeris.com
"What man is a man who does not make the world better?"      --Balian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGOfWsIR7qMdg1EfYRAk35AJ9bCG/s2rIn2HVB4XehxBMY8XH1AQCgkQUj
Jy522rF0YIdICgd70IWKj4s=
=EWTL
-----END PGP SIGNATURE-----

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[simo]

On Thu, 2007-05-03 at 09:46 -0500, Gerald Carter wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Simo,
> 
> > I guess DFS referrals can work cross protocol, so if you are redirected
> > from a longhorn server to a windoes 2000 or a samba server you want to
> > be able to follow the DFS referral and not return an error.
> > To do that you need to have either 1 module that support both protocols
> > or a way from one module to call the other. Just separating the 2
> > without any glue will not work (or you will have to add some userspace
> > upcall hack to make it work).
> 
> Long term I agree that CIFS and SMB2 should be in the same .ko
> But NTLM 0.12 still works for Vista and DFS referrals.
> Breaking out SMB2 initially means that it will not clutter
> the working cifs.ko code.  Remember that an SMB2 client fs is
> mostly research at this point, and not engineering.

Well, development can happen in any way Steve or any other like to do
it, but it seemed to me that the proposal was to split them long term.

I think this would be bad wrt supporting DFS referrals.

That said, it I'll shut up as kindly requested from someone that seem to
know better than Samba/CIFS developers.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer
email: idra@samba.org
http://samba.org

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Steve French]

To clarify:

1) SMB2 is negotiated like a new dialect (of SMB/CIFS).  The new
dialects is sent in the same SMB negotiate protocol request that has
been used for years.  The protocol header format changes to that of
SMB2 (if accepted by the server) on all subsequent requests.
2) If the server does not support SMB2, the client should fall back to
using CIFS (as existing SMB2 clients do).
3) So far, all servers which support SMB2 also support CIFS.

If SMB2 protocol support were coded as a distinct module, the smb2.ko
would need to load cifs.ko to complete session setup in many cases
(presumably when mounting to NetApp, EMC, most older Windows servers
and some Samba servers - at least for a few years).

Because we (Samba, Linux client and even MacOS, HP/UX) have added
optional extensions to CIFS which are not present in SMB2 (although we
should add them), the difficulty is simply which to try first:
1) If we knew the server did not support the CIFS Unix Exensions (CIFS
POSIX Protocol Extensions) but the server was new enough to support
SMB2 then we should request SMB2 (as soon as a stable SMB2 Linux
kernel client were written/tested).   SMB2 has increased limits on
file handles, and should perform marginally better (if the client were
well implemented, unlike some non-Linux clients ...)

But we don't know until the response from the SMB Negotiate Protocol
what the server capabilities are.

Eventually this problem goes away, when we specify and code Unix
Protocol Extensions to SMB2 for Samba etc. as we have done for CIFS,
but in the meantime, I am leaning toward:

1) trying SMB/CIFS only (not including SMB2 dialect) on DFS referrals
by default (unless the user configures the client differently at run
time)
2) On mount trying smb2 only if the user mounted explicitly with "-t
smb2" (but fallback to CIFS if the server does not support SMB2) -
otherwise (for "-t cifs") mount is unchanged.

In any case SMB2 would be considered experimental (for a Linux client)
for quite some time, so there is no hurry to decide until there is
feedback from users.  I am pleased that SMB2 does clean up the
protocol header format - and it should be easier to code in some ways
since there are so few operations to support/optimize (and almost all
are handlebased), but it is way to early to tell which is better (as
far as I know, no one has even proved whether SMB2 is faster than CIFS
or Vista to Vista mounts).

On 5/3/07, simo <idra@samba.org> wrote:
> On Thu, 2007-05-03 at 09:46 -0500, Gerald Carter wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Simo,
> >
> > > I guess DFS referrals can work cross protocol, so if you are redirected
> > > from a longhorn server to a windoes 2000 or a samba server you want to
> > > be able to follow the DFS referral and not return an error.
> > > To do that you need to have either 1 module that support both protocols
> > > or a way from one module to call the other. Just separating the 2
> > > without any glue will not work (or you will have to add some userspace
> > > upcall hack to make it work).
> >
> > Long term I agree that CIFS and SMB2 should be in the same .ko
> > But NTLM 0.12 still works for Vista and DFS referrals.
> > Breaking out SMB2 initially means that it will not clutter
> > the working cifs.ko code.  Remember that an SMB2 client fs is
> > mostly research at this point, and not engineering.
>
> Well, development can happen in any way Steve or any other like to do
> it, but it seemed to me that the proposal was to split them long term.
>
> I think this would be bad wrt supporting DFS referrals.
>
> That said, it I'll shut up as kindly requested from someone that seem to
> know better than Samba/CIFS developers.
>
> Simo.
>
> --
> Simo Sorce
> Samba Team GPL Compliance Officer
> email: idra@samba.org
> http://samba.org
>
>


-- 
Thanks,

Steve

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Christoph Hellwig]

On Thu, May 03, 2007 at 10:35:41AM -0500, Steve French wrote:
> If SMB2 protocol support were coded as a distinct module, the smb2.ko
> would need to load cifs.ko to complete session setup in many cases
> (presumably when mounting to NetApp, EMC, most older Windows servers
> and some Samba servers - at least for a few years).

Umm, no.  If the user tries mount -t smb2 it would try only that and
if it can't support the server it would fail.  Similar for mount -t cifs
if the server only supports modern dialects, which AFAIK none does.
If you feel fancy add autoprobing to mount, similar to how we do
automatic testing of superblocks for disk filesystems.

> In any case SMB2 would be considered experimental (for a Linux client)
> for quite some time, so there is no hurry to decide until there is
> feedback from users.  I am pleased that SMB2 does clean up the
> protocol header format - and it should be easier to code in some ways
> since there are so few operations to support/optimize (and almost all
> are handlebased), but it is way to early to tell which is better (as
> far as I know, no one has even proved whether SMB2 is faster than CIFS
> or Vista to Vista mounts).

Beeing handle based means it actually starts to get useable for Unix
based client, so even if the protocol is not inherently faster the
client side implementation will be a lot faster at least on big
systems because the expensive path generation will go away.

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Jeremy Allison]

On Thu, May 03, 2007 at 09:46:05AM -0500, Gerald Carter wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Simo,
> 
> > I guess DFS referrals can work cross protocol, so if you are redirected
> > from a longhorn server to a windoes 2000 or a samba server you want to
> > be able to follow the DFS referral and not return an error.
> > To do that you need to have either 1 module that support both protocols
> > or a way from one module to call the other. Just separating the 2
> > without any glue will not work (or you will have to add some userspace
> > upcall hack to make it work).
> 
> Long term I agree that CIFS and SMB2 should be in the same .ko

Actually I disagree. I think Christoph is correct. These
are two independent protocols and should be in two different
modules.

> But NTLM 0.12 still works for Vista and DFS referrals.
> Breaking out SMB2 initially means that it will not clutter
> the working cifs.ko code.  Remember that an SMB2 client fs is
> mostly research at this point, and not engineering.

Long term the common functions should be factored out
and put into a lower-level module that both cifs and
SMB2 are dependent upon.

That's the cleaner solution IMHO.

Jeremy.

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[simo]

On Fri, 2007-05-04 at 10:12 -0700, Jeremy Allison wrote:

> Actually I disagree. I think Christoph is correct. These
> are two independent protocols and should be in two different
> modules.

They are independent the same way NFS v4 is independent from NFS v3 and
v2. Independent but related, and most importantly, one is the fallback
of the other.

> > But NTLM 0.12 still works for Vista and DFS referrals.
> > Breaking out SMB2 initially means that it will not clutter
> > the working cifs.ko code.  Remember that an SMB2 client fs is
> > mostly research at this point, and not engineering.
> 
> Long term the common functions should be factored out
> and put into a lower-level module that both cifs and
> SMB2 are dependent upon.
> 
> That's the cleaner solution IMHO.

If the result is that the fallback work without user space intervention,
then I agree with you.
I was just pointing out that the 2 protocols are not in fact completely
independent and this fact need to be properly considered and factored in
into this decision, nothing more, nothing less.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer
email: idra@samba.org
http://samba.org

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Christoph Hellwig]

On Fri, May 04, 2007 at 05:43:13PM +0000, simo wrote:
> > Actually I disagree. I think Christoph is correct. These
> > are two independent protocols and should be in two different
> > modules.
> 
> They are independent the same way NFS v4 is independent from NFS v3 and
> v2. Independent but related, and most importantly, one is the fallback
> of the other.

Just FYI: although nfs2/3 and nfs4 are in the same kernel module they
actually are different file_system_types, and there is no automatic
fallback in the kernel.  Given how little is actually shared between
nfs v2/3 and 4 it might have been a better idea to make it a totally
separate module.

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinct module

[Steve French]

On 5/4/07, Jeremy Allison <jra@samba.org> wrote:
> On Thu, May 03, 2007 at 09:46:05AM -0500, Gerald Carter wrote:
> > Long term I agree that CIFS and SMB2 should be in the same .ko
>
> Actually I disagree. I think Christoph is correct. These
> are two independent protocols and should be in two different
> modules.
>
> > But NTLM 0.12 still works for Vista and DFS referrals.
> > Breaking out SMB2 initially means that it will not clutter
> > the working cifs.ko code.  Remember that an SMB2 client fs is
> > mostly research at this point, and not engineering.
>
> Long term the common functions should be factored out
> and put into a lower-level module that both cifs and
> SMB2 are dependent upon.
>
> That's the cleaner solution IMHO.
>
> Jeremy.

There is also the obvious tradeoff of  "easier to update frequently"
vs. "easier to write" which is a primary factor.

1) as distinct .ko files smb2 and cifs can be updated independently
(the former marked broken/experimental).  Updating smb2 won't
risk breaking cifs

2)  but implemented in the same module, there is somewhat less code to write.




-- 
Thanks,

Steve

Re: SMB2 file system - should it be a distinct module

[Jan Engelhardt]

On Apr 30 2007 17:52, Steve French wrote:
>
> Now that we (Samba team) understand enough about it to implement
> prototypes (there is a prototype server in Samba 4, and a userspace
> client library for testing), we need to decide whether the kernel
> implementation of SMB2 client should be a distinct module or just part
> of the cifs.ko module.

Do it like CONFIG_FAT_FS/CONFIG_MSDOS_FS/CONFIG_VFAT_FS...

config SMB_COMMON
    tristate

config CIFS
    select SMB_COMMON

config SMB2
    select SMB_COMMON

> SMB2 (the protocol) is smaller than cifs,

Could not they have named it CIFS2... :p

And, what also puzzles me... almost every filesystem that's not at revision 1
anymore (ext2/3/4, reiser4, smb2) does not have the usually omnipresent "fs"
suffix anymore (cf. reiserfs, smbfs). Maybe it's time to drop all the "fs"
suffixes? :)


Jan
-- 

Re: [linux-cifs-client] Re: SMB2 file system - should it be a distinctmodule

[Wagner, Chris (GEAE, CBTS)]

Then we can just rename the project CI. :)

Jan Engelhardt wrote:
> And, what also puzzles me... almost every filesystem that's not at revision 1
> anymore (ext2/3/4, reiser4, smb2) does not have the usually omnipresent "fs"
> suffix anymore (cf. reiserfs, smbfs). Maybe it's time to drop all the "fs"
> suffixes? :)


-- 
Chris Wagner
CBTS
GE Aircraft Engines
Chris.Wagner@ae.ge.com