[PATCH 0/7] OMFS filesystem version 3

[PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

(Andrew, perhaps this could bake in -mm for a cycle or two if 
there are no objections?)

These patches 1-7 add the Optimized MPEG Filesystem, a proprietary
filesystem used by the embedded devices Rio Karma and ReplayTV 
which are no longer manufactured.

I've been maintaining this reverse-engineered filesystem
out-of-tree since around 2.6.12.  It has stabilized to the point
that, while not quite feature-complete, it is close enough that
I'm pretty much just applying VFS API updates to it these days.
So, in the spirit of stable_api_nonsense, I request that it be
considered for inclusion for .26 or later.  

There is a small user community (<20?) using the driver today.
According to google, it has been packaged by others for Debian,
Ubuntu, and Mandriva.

I have also written an implementation in FUSE.  However, in
its current unoptimized state, the FUSE version is about half the 
speed of the kernel driver.  Either way, the kernel version could 
use a review given some distros have apparently picked it up.

version 3:
 - fixed sparse warnings
 - added more sanity checking in list traversals

version 2:
 - removed private inode cache 
 - included GPL v2 notice
 - rewrote omfs_count_free to use hamming weight library functions 
 - removed duplicated crc-itu-t

Version 1: initial post

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Sat, 12 Apr 2008 18:58:34 -0400 Bob Copeland <me@bobcopeland.com> wrote:

> (Andrew, perhaps this could bake in -mm for a cycle or two if 
> there are no objections?)

Sure, if it compiles I can stick in in there for some airtime.

> These patches 1-7 add the Optimized MPEG Filesystem, a proprietary
> filesystem used by the embedded devices Rio Karma and ReplayTV 
> which are no longer manufactured.

Adding a new FS to Linux is a pretty major thing.

- A key question (which you don't seem to have addressed at all!) is: why
  is this a useful addition to Linux?  What are the filesystem's strengths?
  What is its application?  How does it improves Linux and by how much?

  I'll go aehad and assume that its sweet-spot is streaming media files
  (perhaps more than one at a time?).  If so, then comparative performance
  measurements would be the key piece of information which we'll need when
  making a merge decision.  Probably against ext3, ext4 and XFS.

- What are the filesystem's features?  Does it journal changes, or is it
  inherently crash-safe?  Or is fsck-style repair needed?

- We'd like to see some documentation:

  - Mount options

  - Location of userspace support tools such as mkfs and fsck

  - Documentation for those tools

- A MAINTAINERS record, please?


If I include this in -mm, some kind people will try it out.  It would be
most helpful for the changelog (or even the Kconfig help) to include
sufficient information for people to be able to create, mount and check an
OMFS partition.

> I've been maintaining this reverse-engineered filesystem
> out-of-tree since around 2.6.12.

Who did the reverse-engineering, and how was it done?

Please make us confident that we won't get our butts sued off or something.

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Sat, Apr 12, 2008 at 05:03:04PM -0700, Andrew Morton wrote:
> Adding a new FS to Linux is a pretty major thing.
> 
> - A key question (which you don't seem to have addressed at all!) is: why
>   is this a useful addition to Linux?  What are the filesystem's strengths?
>   What is its application?  How does it improves Linux and by how much?

Honestly, I'm not sure if this FS is mainline material or not.  If it is, 
it is in the sense that the Amiga FS, befs, and so forth are useful:
it's a dead filesystem that a very few people still have a reason to
use.  If FUSE is where this should live, then I'll just simply focus my
time on that instead (since I already have it in FUSE).

Primarily the users are Rio Karma owners such as myself.  For some reason
the designers stuck this filesystem on the device, so unless someone hacks 
the firmware, using OMFS is the only way to get MP3 files on the thing 
from Linux.  Just so happens the same company made ReplayTV.

>   I'll go aehad and assume that its sweet-spot is streaming media files
>   (perhaps more than one at a time?).  If so, then comparative performance
>   measurements would be the key piece of information which we'll need when
>   making a merge decision.  Probably against ext3, ext4 and XFS.

Okay.  I can do benchmarks but I'm pretty sure it will be worse than all 
of them by a long shot.  

Note my Kconfig text:
[...]
+	  player and ReplayTV DVR.  Despite the name, this filesystem is not
+	  more efficient than a standard FS for MPEG files, in fact likely
+	  the opposite is true.  

Uncached lookups in particular are very slow because directories are 
essentially a huge hash table.  You might have to seek through several 
siblings in a bucket to get to the target file.

They seem to have designed for online recovery with checksumming and
mirroring of every FS object.  This module updates these, but it does
not do any online checking/repair.  I have written an mkfs/fsck, though.

> - We'd like to see some documentation:

Gotcha.

> Who did the reverse-engineering, and how was it done?
> Please make us confident that we won't get our butts sued off or something.

I'm not sure there's a company to sue anymore.

Most of it was done by members of the ReplayTV community; I just used
their documentation.  I'll see if I can get in touch with one of them to 
see how they came up with that.  I only figured out a few things, for 
which I'd use the Karma as a black box for copying/deleting files etc. 
and then poke at the resulting disk image with a hex editor.

-- 
Bob Copeland %% www.bobcopeland.com 

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Sat, 12 Apr 2008 23:33:44 -0400 Bob Copeland <me@bobcopeland.com> wrote:

> On Sat, Apr 12, 2008 at 05:03:04PM -0700, Andrew Morton wrote:
> > Adding a new FS to Linux is a pretty major thing.
> > 
> > - A key question (which you don't seem to have addressed at all!) is: why
> >   is this a useful addition to Linux?  What are the filesystem's strengths?
> >   What is its application?  How does it improves Linux and by how much?
> 
> Honestly, I'm not sure if this FS is mainline material or not.

OIC.

>  If it is, 
> it is in the sense that the Amiga FS, befs, and so forth are useful:
> it's a dead filesystem that a very few people still have a reason to
> use.  If FUSE is where this should live, then I'll just simply focus my
> time on that instead (since I already have it in FUSE).

Yes, pursuing the FUSE implementation sounds a better approach - it avoids
burdening the kernel with a filesysstem which few will be interested in and
is more practical for use by those who _are_ interested in it.

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Sat, Apr 12, 2008 at 08:55:44PM -0700, Andrew Morton wrote:
> Yes, pursuing the FUSE implementation sounds a better approach - it avoids
> burdening the kernel with a filesysstem which few will be interested in and
> is more practical for use by those who _are_ interested in it.

Roger that, thanks.

-- 
Bob Copeland %% www.bobcopeland.com 

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Sat, Apr 12, 2008 at 08:55:44PM -0700, Andrew Morton wrote:
> > it's a dead filesystem that a very few people still have a reason to
> > use.  If FUSE is where this should live, then I'll just simply focus my
> > time on that instead (since I already have it in FUSE).
> 
> Yes, pursuing the FUSE implementation sounds a better approach - it avoids
> burdening the kernel with a filesysstem which few will be interested in and
> is more practical for use by those who _are_ interested in it.

No way.  For a normal foreign block filesystem a proper kernel
implementation is much better.  And this one is particularly
well-written.  Lately I really start wondering why we keep adding crap
all over the core, but if we have a modular new filesystem that's quite
nice people start complaining.

> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
---end quoted text---

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Sun, 13 Apr 2008 04:01:30 -0400 Christoph Hellwig <hch@infradead.org> wrote:

> On Sat, Apr 12, 2008 at 08:55:44PM -0700, Andrew Morton wrote:
> > > it's a dead filesystem that a very few people still have a reason to
> > > use.  If FUSE is where this should live, then I'll just simply focus my
> > > time on that instead (since I already have it in FUSE).
> > 
> > Yes, pursuing the FUSE implementation sounds a better approach - it avoids
> > burdening the kernel with a filesysstem which few will be interested in and
> > is more practical for use by those who _are_ interested in it.
> 
> No way.  For a normal foreign block filesystem a proper kernel
> implementation is much better.  And this one is particularly
> well-written.  Lately I really start wondering why we keep adding crap
> all over the core, but if we have a modular new filesystem that's quite
> nice people start complaining.
> 

I'm not complaining about anything.  Who has?

As the filesystem is for occasional, non-performance-sensitive use
by a very small number of people, doing it via FUSE sounds like an
all-round more practical approach.  This has nothing to do with quality of
implementation at all.

I don't have particularly strong opinions either way.

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

> I'm not complaining about anything.  Who has?
> 
> As the filesystem is for occasional, non-performance-sensitive use
> by a very small number of people, doing it via FUSE sounds like an
> all-round more practical approach.  This has nothing to do with quality of
> implementation at all.

It's a stupid idea.  Moving a simple block based filesystem means it's
more complicated, less efficient because of the additional context
switches and harder to use because you need additional userspace
packages and need to setup fuse.

We made writing block based filesystems trivial in the kernel to grow
more support for filesystems like this one.

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

> Honestly, I'm not sure if this FS is mainline material or not.  If it is, 

It looks mainline to me. It is far more relevant than most of the other
file systems we support and it doesn't mess up core code at all. It's a
loadable kernel module so the impact is basically nil, as well as being
useful to other folk writing file systems.

> Most of it was done by members of the ReplayTV community; I just used
> their documentation.  I'll see if I can get in touch with one of them to 
> see how they came up with that.  I only figured out a few things, for 
> which I'd use the Karma as a black box for copying/deleting files etc. 
> and then poke at the resulting disk image with a hex editor.

If you used their documentation and didn't do the reverse engineering or
read any ReplayTV code there shouldn't be any problems. The big danger is
if you reverse engineer their code and you write new code you may copy
their implementation even accidentally (or be held to have done so).

Alan

Re: [PATCH 0/7] OMFS filesystem version 3

[Pavel Machek]

Hi!

> Uncached lookups in particular are very slow because directories are 
> essentially a huge hash table.  You might have to seek through several 
> siblings in a bucket to get to the target file.
> 
> They seem to have designed for online recovery with checksumming and
> mirroring of every FS object.  This module updates these, but it does
> not do any online checking/repair.  I have written an mkfs/fsck, though.

This is actually quite interesting. Checksuming filesystem sounds
pretty useful.

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > I'm not complaining about anything.  Who has?
> > 
> > As the filesystem is for occasional, non-performance-sensitive use
> > by a very small number of people, doing it via FUSE sounds like an
> > all-round more practical approach.  This has nothing to do with quality of
> > implementation at all.
> 
> It's a stupid idea.  Moving a simple block based filesystem means it's
> more complicated, less efficient because of the additional context
> switches and harder to use because you need additional userspace
> packages and need to setup fuse.
> 
> We made writing block based filesystems trivial in the kernel to grow
> more support for filesystems like this one.

I don't feel strongly either way, and Christoph's arguments against
fuse are mostly valid (although neither of them are serious).

There's one thing which makes fuse a slightly better candidate for
applications where the number of users is low: stability.  Unless you
or your users test the hell out of your filesystem, there always a
chance that some bugs will remain.  These rarely bring down the whole
system, but it usually requires a reboot to let you continue using the
Oopsing fs.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[David Woodhouse]

On Sun, 2008-04-13 at 12:37 +0200, Miklos Szeredi wrote:
> There's one thing which makes fuse a slightly better candidate for
> applications where the number of users is low: stability.  Unless you
> or your users test the hell out of your filesystem, there always a
> chance that some bugs will remain.  These rarely bring down the whole
> system, but it usually requires a reboot to let you continue using the
> Oopsing fs.

I think it's a slippery slope from that to rewriting Linux as a
microkernel.

Simple block filesystems like this belong in the kernel.

-- 
dwmw2

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Sun, 13 Apr 2008 22:15:58 +0100 David Woodhouse <dwmw2@infradead.org> wrote:

> On Sun, 2008-04-13 at 12:37 +0200, Miklos Szeredi wrote:
> >
> > [technical stuff]
> 
> [technical stuff]
>

I guess I can keep making this point in various ways until someone
actually notices it:

	This filesystem has only 20 users.

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

> I guess I can keep making this point in various ways until someone
> actually notices it:
> 
> 	This filesystem has only 20 users.

At the moment. And that probably exceeds Amiga users, 386 users, some of
the serial port users, several network card users ...

In the past we've merged drivers for network cards where only two
existed in the world. Linus has repeatedly stated he wants to see stuff
people are using getting in. Good clean code that doesn't affect the core
is good reference material.

I think you are (unusually) the one out of step here ?

Alan

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Sun, 13 Apr 2008 23:49:20 +0100 Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > I guess I can keep making this point in various ways until someone
> > actually notices it:
> > 
> > 	This filesystem has only 20 users.
> 
> At the moment. And that probably exceeds Amiga users, 386 users, some of
> the serial port users, several network card users ...
> 
> In the past we've merged drivers for network cards where only two
> existed in the world. Linus has repeatedly stated he wants to see stuff
> people are using getting in.

None of that means that merging this filesystem is the best decision.

> Good clean code that doesn't affect the core
> is good reference material.

The reference block filesystem is ext2 (used to be minixfs) - there is no
need for another.

> I think you are (unusually) the one out of step here ?

I appear to be the only one who is looking at the whole picture.

Merging a new filesystem has costs - I don't need to enumerate them.  Do
the benefits of OMFS exceed them?

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Sun, Apr 13, 2008 at 12:37:31PM +0200, Miklos Szeredi wrote:
> I don't feel strongly either way, and Christoph's arguments against
> fuse are mostly valid (although neither of them are serious).

I don't have hard numbers, but anecdotally my FUSE version is quite 
a bit less performant.  That's no criticism of FUSE - I just haven't
put the time into optimizing and adding various caches.

> There's one thing which makes fuse a slightly better candidate for
> applications where the number of users is low: stability.  Unless you
> or your users test the hell out of your filesystem, there always a
> chance that some bugs will remain.  

Sure, though this FS won't see the same kind of use as ext2.  Most users
would just mount it, copy a bunch of files, then unmount it, and if that
works then great.

It has at least seen some testing with fsx, though I had to turn off most 
of the checks since growing truncate is still unimplemented.  

-- 
Bob Copeland %% www.bobcopeland.com 

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Sun, Apr 13, 2008 at 04:10:14PM -0700, Andrew Morton wrote:
> On Sun, 13 Apr 2008 23:49:20 +0100 Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > > 	This filesystem has only 20 users.
> 
> None of that means that merging this filesystem is the best decision.

Well, 20 may have been aiming just a tad low.  I'd never make it in sales.
Here are the stats from linux-karma.sf.net:

 - most recent release, for 2.6.25: 36 downloads since 3/16/08
 - # subscribers on mailing list: 39 [1]
 - most # of downloads of a given release: 252 (May-Oct 2007) [2]

[1] Some subscribers may not use the FS at all since there is also
software for using the ethernet interface.  Both are discussed on 
the list.

[2] I doubt that 252 d/ls translates to that many _current_ users; a
year is a long time in consumer electronics.  I don't have stats on 
unique IPs.

I'm unaware of ReplayTV users - one guy contacted me once and then
disappeared.

> Merging a new filesystem has costs - I don't need to enumerate them.  Do
> the benefits of OMFS exceed them?

You guys would know best.  I can see both arguments...

-- 
Bob Copeland %% www.bobcopeland.com 

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Sun, 13 Apr 2008 21:32:48 -0400 Bob Copeland <me@bobcopeland.com> wrote:

> On Sun, Apr 13, 2008 at 04:10:14PM -0700, Andrew Morton wrote:
> > On Sun, 13 Apr 2008 23:49:20 +0100 Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > > > 	This filesystem has only 20 users.
> > 
> > None of that means that merging this filesystem is the best decision.
> 
> Well, 20 may have been aiming just a tad low.  I'd never make it in sales.
> Here are the stats from linux-karma.sf.net:
> 
>  - most recent release, for 2.6.25: 36 downloads since 3/16/08
>  - # subscribers on mailing list: 39 [1]
>  - most # of downloads of a given release: 252 (May-Oct 2007) [2]
> 
> [1] Some subscribers may not use the FS at all since there is also
> software for using the ethernet interface.  Both are discussed on 
> the list.
> 
> [2] I doubt that 252 d/ls translates to that many _current_ users; a
> year is a long time in consumer electronics.  I don't have stats on 
> unique IPs.
> 
> I'm unaware of ReplayTV users - one guy contacted me once and then
> disappeared.

OK.

> > Merging a new filesystem has costs - I don't need to enumerate them.  Do
> > the benefits of OMFS exceed them?
> 
> You guys would know best.  I can see both arguments...

Well as I say - no strong opinions either way here.  It just seems a bit
odd to burden the kernel with an additional fs for such a small user base
when either options exist.

OTOH when we merge a new fs that also has the side-effect of increasing the
active developer base - the maintainers of the fs fix other stuff.  And
I'll do anything to get some acpi bugs fixed ;)

So.. whatever.  I'll keep a look out for v4.

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> On Sun, 2008-04-13 at 12:37 +0200, Miklos Szeredi wrote:
> > There's one thing which makes fuse a slightly better candidate for
> > applications where the number of users is low: stability.  Unless you
> > or your users test the hell out of your filesystem, there always a
> > chance that some bugs will remain.  These rarely bring down the whole
> > system, but it usually requires a reboot to let you continue using the
> > Oopsing fs.
> 
> I think it's a slippery slope from that to rewriting Linux as a
> microkernel.

You say that as if a microkernel had _no_ advantages.  Which isn't
true: it's just a trade between performance and encapsulation.  What I
was saying, that if there are few users, and so the tester base is
limited, then they _might_ just be better off with a slower, but more
stable solution.

I'm not advocating moving ext3 to fuse.  And I didn't advocate moving
ntfs to fuse, still that was done and the resulting filesystem at the
moment happens to outperform the kernel one in every respect ;)

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > I don't feel strongly either way, and Christoph's arguments against
> > fuse are mostly valid (although neither of them are serious).
> 
> I don't have hard numbers, but anecdotally my FUSE version is quite 
> a bit less performant.  That's no criticism of FUSE - I just haven't
> put the time into optimizing and adding various caches.

The worst I/O performance problems should be gone by 2.6.26.
Otherwise there shouldn't be a need to add optimizations to the
userspace code.  The kernel caches take care of that, just like for a
kernel filesystem.

> > There's one thing which makes fuse a slightly better candidate for
> > applications where the number of users is low: stability.  Unless you
> > or your users test the hell out of your filesystem, there always a
> > chance that some bugs will remain.  
> 
> Sure, though this FS won't see the same kind of use as ext2.  Most users
> would just mount it, copy a bunch of files, then unmount it, and if that
> works then great.

Exactly.  Which means, that bugs which happen only in special
circumstances don't surface early and cause more headaches later.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 09:25:46 +0200 Miklos Szeredi <miklos@szeredi.hu> wrote:

> And I didn't advocate moving
> ntfs to fuse, still that was done and the resulting filesystem at the
> moment happens to outperform the kernel one in every respect ;)

Gad.  Why?

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > And I didn't advocate moving
> > ntfs to fuse, still that was done and the resulting filesystem at the
> > moment happens to outperform the kernel one in every respect ;)
> 
> Gad.  Why?

I think mostly because of dedication of the maintainer (Szabolcs
Szakacsits).  I don't claim fuse has a great part in it, except the
obvious advantages of developing in userspace: no kernel crashes,
easier debugging, etc...

And of course it could be ported back to the kernel (ntfs-3g is GPL
too), and that would probably result in even more speedups.  But
that's not a trivial task either.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Anton Altaparmakov]

Hi,

On 14 Apr 2008, at 08:49, Andrew Morton wrote:
> On Mon, 14 Apr 2008 09:25:46 +0200 Miklos Szeredi  
> <miklos@szeredi.hu> wrote:
>
>> And I didn't advocate moving
>> ntfs to fuse, still that was done and the resulting filesystem at the
>> moment happens to outperform the kernel one in every respect ;)
>
> Gad.  Why?

Miklos has the wrong end of the stick.  No-one has "moved" ntfs to  
fuse.  And the fuse implementation doesn't outperform the kernel  
implementation in anything at all.  However the kernel one as  
available in the kernel source tree doesn't have many write-features,  
it can only overwrite files, it cannot create/delete files, etc.  So I  
guess if you define "performance" to mean "features" then sure  
ntfsmount/ntfs-3g have more features than the public kernel driver.   
If you define "performance" to mean "speed" then no ntfsmount/ntfs-3g  
can't compare to the kernel except in very limited and meaningless  
benchmarks...

btw. even comparing features, the fuse solutions lag behind in some  
respects, e.g. no-one can "kill -9" the kernel driver leaving a  
corrupt file system on the volume (and under no-one I include the OOM  
killer for example!) and another example is that the fuse solutions  
require large amounts of ram whereas the kernel driver can happily  
function in 1MiB ram and less even as everything is in the page cache  
so it will just cause heavy paging whilst the fuse solutions just blow  
up / OOM the machine when they find a large directory and the user has  
only 32MiB ram for example...  At least I have seen reports of this on  
the mailing lists, not that I have ever cared to try.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

> I appear to be the only one who is looking at the whole picture.
> 
> Merging a new filesystem has costs - I don't need to enumerate them.  Do
> the benefits of OMFS exceed them?

I think the exceed them quite easily. The costs are almost nil, while
merging this provides another nice example fs (and one much easier to
follow than ext*) for hardware that does have a few users and will no
doubt get many more

I wasn't aware Linus had introduced a new rule required 500 people sign
up to use a feature before it gets added ?

Alan

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> >> And I didn't advocate moving
> >> ntfs to fuse, still that was done and the resulting filesystem at the
> >> moment happens to outperform the kernel one in every respect ;)
> >
> > Gad.  Why?
> 
> Miklos has the wrong end of the stick.  No-one has "moved" ntfs to  
> fuse.  And the fuse implementation doesn't outperform the kernel  
> implementation in anything at all.  However the kernel one as  
> available in the kernel source tree doesn't have many write-features,  
> it can only overwrite files, it cannot create/delete files, etc.  So I  
> guess if you define "performance" to mean "features" then sure  
> ntfsmount/ntfs-3g have more features than the public kernel driver.   
> If you define "performance" to mean "speed" then no ntfsmount/ntfs-3g  
> can't compare to the kernel except in very limited and meaningless  
> benchmarks...

OK, I was exaggerating (notice the smiley).  But I do have a feeling
(and just a feeling, no hard data), that ntfs-3g is making the
in-kernel ntfs filesystem increasingly irrelevant.  And yes, that's
mostly because of the features, but also because the performance is
not at all as bad, as some people would think a userspace filesystem
has to be.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Xavier Bestel]

On Sun, 2008-04-13 at 16:10 -0700, Andrew Morton wrote:
> I appear to be the only one who is looking at the whole picture.
> 
> Merging a new filesystem has costs - I don't need to enumerate them.  Do
> the benefits of OMFS exceed them?

Eh you, corporate linux developer, please don't discriminate what's
going in the kernel just because of extra cost. Let linux continue to be
"by the people, for the people" if you see what I mean.

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 09:16:39 +0100 Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > I appear to be the only one who is looking at the whole picture.
> > 
> > Merging a new filesystem has costs - I don't need to enumerate them.  Do
> > the benefits of OMFS exceed them?
> 
> I think the exceed them quite easily. The costs are almost nil,

They're modest, but they're there.

> while
> merging this provides another nice example fs (and one much easier to
> follow than ext*) for hardware that does have a few users and will no
> doubt get many more
> 
> I wasn't aware Linus had introduced a new rule required 500 people sign
> up to use a feature before it gets added ?

He doesn't need to.

We'll frequently add features in the expectation that they will be useful
to a significant part of our user base.  It's a part of the decision-making
process.

Many features will have a small number of users initialy, but that grows
over time.  otoh, OMFS's user base is small and it appears that it will
dwindle.

(And in the past Linus has set the bar *much* higher for new filesystems
than I have..)

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Mon, Apr 14, 2008 at 09:16:39AM +0100, Alan Cox wrote:
> I think the exceed them quite easily. The costs are almost nil, while
> merging this provides another nice example fs (and one much easier to
> follow than ext*) for hardware that does have a few users and will no
> doubt get many more
> 
> I wasn't aware Linus had introduced a new rule required 500 people sign
> up to use a feature before it gets added ?

I'm also very surprised by this, especially as it seems to be applied
very selectively.  This filesystems is an almost 0 maintainance burden
unlike a lot of really crappy driver we're shoving in constantly.

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Mon, Apr 14, 2008 at 10:30:59AM +0200, Xavier Bestel wrote:
> Eh you, corporate linux developer, please don't discriminate what's
> going in the kernel just because of extra cost. Let linux continue to be
> "by the people, for the people" if you see what I mean.

Seconded (as a 95% non-coporate kernel hacker :))

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 10:30:59 +0200 Xavier Bestel <xavier.bestel@free.fr> wrote:

> On Sun, 2008-04-13 at 16:10 -0700, Andrew Morton wrote:
> > I appear to be the only one who is looking at the whole picture.
> > 
> > Merging a new filesystem has costs - I don't need to enumerate them.  Do
> > the benefits of OMFS exceed them?
> 
> Eh you, corporate linux developer, please don't discriminate what's
> going in the kernel just because of extra cost.

By "cost" I refer to extra developer time spent on maintaining the
filesystem.  The most recent example is the write_begin/write_end changes
which took a lot of Nick's time and rather a lot of mine also.

I get to see these costs.  I seek to minimise them.

> Let linux continue to be
> "by the people, for the people" if you see what I mean.

"the people" here are those who work on the kernel.  We want our time to be
spent as effectively as possible.  Sorry if that sounds corporate.

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Mon, Apr 14, 2008 at 01:44:45AM -0700, Andrew Morton wrote:
> By "cost" I refer to extra developer time spent on maintaining the
> filesystem.  The most recent example is the write_begin/write_end changes
> which took a lot of Nick's time and rather a lot of mine also.

Which are only an extra cost for complex filesystem.  Trivial block
filesystems like omfs just use the generic callbacks and can be
converted in a simple sweep.

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Mon, Apr 14, 2008 at 01:44:45AM -0700, Andrew Morton wrote:
> > Let linux continue to be
> > "by the people, for the people" if you see what I mean.
> 
> "the people" here are those who work on the kernel.  We want our time to be
> spent as effectively as possible.  Sorry if that sounds corporate.

My time is definitively spent better on reviewig and if needed fixing a
fun filesystem like omfs that allows linux to deal with fun hardware
instead of fighting the lastet crap code from the corporate drones.

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 05:09:45 -0400 Christoph Hellwig <hch@infradead.org> wrote:

> the lastet crap code from the corporate drones.

That's about the fourth time you've made this assertion.  It is pointless,
it is of no use to anyone and it is unjustly insulting to a large number of
kernel developers.  Many of whom do a hell of a lot more kernel work than
you, I might add.

If you have issues with certain parts of the kernel code then tell us, with
a usable level of detail what they are.  Preferably as a reply-to-all prior
to that code being merged, but after merging is fine too.

Otherwise...

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Miklos Szeredi wrote:
> And I didn't advocate moving
> ntfs to fuse, still that was done

Yes, that's quite annoying as FUSE doesn't work with my architecture,
so I can't use the only good NTFS driver.... grr! :-)

FUSE is great for user-mounted filesystems, though.  Having to use
"su" in shell scripts which modify loopback filesystems was always
very ugly.

I don't see why we need such a dichotomy, though.  The VFS API is
sensible enough that it should be possible to compile the same
filesystem code to run in kernel or in FUSE.

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > And I didn't advocate moving
> > ntfs to fuse, still that was done
> 
> Yes, that's quite annoying as FUSE doesn't work with my architecture,
> so I can't use the only good NTFS driver.... grr! :-)

I'm not quite getting this joke :)

> FUSE is great for user-mounted filesystems, though.  Having to use
> "su" in shell scripts which modify loopback filesystems was always
> very ugly.
> 
> I don't see why we need such a dichotomy, though.  The VFS API is
> sensible enough that it should be possible to compile the same
> filesystem code to run in kernel or in FUSE.

Well, yeah:

  http://lkml.org/lkml/2006/2/27/148

Oh, you mean without having to run the whole kernel around the
filesystem?  Well, that's a bit more tricky: filesystems use not just
the VFS API, but the block, VM, SLAB, etc. APIs as well.  Porting all
of these to userspace and keeping them in sync with the kernel changes
doesn't sound much fun.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[David Woodhouse]

On Mon, 2008-04-14 at 02:21 -0700, Andrew Morton wrote:
> On Mon, 14 Apr 2008 05:09:45 -0400 Christoph Hellwig <hch@infradead.org> wrote:
> 
> > the lastet crap code from the corporate drones.
> 
> That's about the fourth time you've made this assertion.  It is pointless,
> it is of no use to anyone and it is unjustly insulting to a large number of
> kernel developers.  

No, it isn't. Christoph can be insulting sometimes, but that wasn't it.

Anyone who thinks that Christoph was referring to them obviously
identifies _themselves_ as 'corporate drone who posts crap code'.
Why is it insulting for Christoph to opine that his time is better spent
on reviewing something fun?

I find it extremely annoying when people claim offence on behalf of
unspecified third parties, when no real adult actually _is_ so wet as to
have actually been offended.

> If you have issues with certain parts of the kernel code then tell us, with
> a usable level of detail what they are.  Preferably as a reply-to-all prior
> to that code being merged, but after merging is fine too.

As if Christoph needs encouragement to do just that... :)

-- 
dwmw2

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 11:09:46 +0100 David Woodhouse <dwmw2@infradead.org> wrote:

> On Mon, 2008-04-14 at 02:21 -0700, Andrew Morton wrote:
> > On Mon, 14 Apr 2008 05:09:45 -0400 Christoph Hellwig <hch@infradead.org> wrote:
> > 
> > > the lastet crap code from the corporate drones.
> > 
> > That's about the fourth time you've made this assertion.  It is pointless,
> > it is of no use to anyone and it is unjustly insulting to a large number of
> > kernel developers.  
> 
> No, it isn't. Christoph can be insulting sometimes, but that wasn't it.
> 
> Anyone who thinks that Christoph was referring to them obviously
> identifies _themselves_ as 'corporate drone who posts crap code'.
> Why is it insulting for Christoph to opine that his time is better spent
> on reviewing something fun?
> 
> I find it extremely annoying when people claim offence on behalf of
> unspecified third parties, when no real adult actually _is_ so wet as to
> have actually been offended.

Not so.  There is a habit of treating corporate developers as somehow
inferior to the purer old-timers.  I know - people have told me.  Quite
often.  And for various reasons, those people feel limited in their options
for standing up for themselves.

Re: [PATCH 0/7] OMFS filesystem version 3

[David Woodhouse]

On Mon, 2008-04-14 at 03:22 -0700, Andrew Morton wrote:
> Not so.  There is a habit of treating corporate developers as somehow
> inferior to the purer old-timers.  I know - people have told me.  Quite
> often. 

I'm not sure I agree. There is a habit of treating people who post crap
code as somehow inferior to the purer old-timers, certainly. And that's
probably not such a good thing, but at least it helps to keep the
signal:noise ratio down a little, overall.

But except to the extent that there is a correlation between 'corporate
developers' and 'people who post crap code', I wouldn't agree with your
statement above. 

We have a lot of people working for large corporations who don't post
crap code, and to whom Christoph almost never promotes an attitude of
violence. Looking at Jon's 'who wrote 2.6.23' analysis, I see quite a
lot of successful 'corporate' involvement.

> And for various reasons, those people feel limited in their options
> for standing up for themselves.

Anyone trying to defend crap code _should_ be limited in their options,
surely? And sometimes, 'corporate developers' do try to defend crap
code, because they've made traditional corporate mistakes like
developing it all in private and presenting it as a fait accomplis
without proper a priori review, and/or haven't budgeted for the
necessary time to fix it.

-- 
dwmw2

Re: [PATCH 0/7] OMFS filesystem version 3

[David Woodhouse]

On Mon, 2008-04-14 at 11:58 +0200, Miklos Szeredi wrote:
> Oh, you mean without having to run the whole kernel around the
> filesystem?  Well, that's a bit more tricky: filesystems use not just
> the VFS API, but the block, VM, SLAB, etc. APIs as well.  Porting all
> of these to userspace and keeping them in sync with the kernel changes
> doesn't sound much fun.

That would be UML :)

But seriously -- wouldn't our time be better spent on improving error
handling in the kernel, rather than taking advantage of the benefits of
microkernel design... but only for those parts which don't matter?

-- 
dwmw2

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Mon, Apr 14, 2008 at 03:22:32AM -0700, Andrew Morton wrote:
> Not so.  There is a habit of treating corporate developers as somehow
> inferior to the purer old-timers.  I know - people have told me.  Quite
> often.  And for various reasons, those people feel limited in their options
> for standing up for themselves.

Corporate drones obviously doesn't refer to all of them.  It referes to
those who are primarily corporate and secondarily Linux people of
course.  There's quite a lot of developers who came to Linux with a
corporate background and are fully integrated into the "scene".
Surprising they tend to write mostly good code and are involved with
review and other important activities.

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Miklos Szeredi wrote:
> > Yes, that's quite annoying as FUSE doesn't work with my architecture,
> > so I can't use the only good NTFS driver.... grr! :-)
> 
> I'm not quite getting this joke :)

It's not really a joke.  It really is a problem: I'm supposed to
provide NTFS support for a project, but the kernel I'm using doesn't
work with FUSE (too old for the FUSE version needed by ntfs-3g), and
the in-kernel NTFS isn't good enough - all the active development work
is done on ntfs-3g.

The :-) is because it's my problem, nobody else's.

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Mon, Apr 14, 2008 at 12:55:40PM +0100, Jamie Lokier wrote:
> It's not really a joke.  It really is a problem: I'm supposed to
> provide NTFS support for a project, but the kernel I'm using doesn't
> work with FUSE (too old for the FUSE version needed by ntfs-3g), and
> the in-kernel NTFS isn't good enough - all the active development work
> is done on ntfs-3g.

It's getting a little offtopic, but what keeps you from backporting
the required fuse version?

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > > Yes, that's quite annoying as FUSE doesn't work with my architecture,
> > > so I can't use the only good NTFS driver.... grr! :-)
> > 
> > I'm not quite getting this joke :)
> 
> It's not really a joke.  It really is a problem: I'm supposed to
> provide NTFS support for a project, but the kernel I'm using doesn't
> work with FUSE (too old for the FUSE version needed by ntfs-3g),

How old?  The out of tree fuse module (hoping to get rid of it soon)
works down to 2.6.9.  Beyond that it does probably require a fair
amount of porting work.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Mon, 14 Apr 2008 Anton Altaparmakov wrote:
> On 14 Apr 2008, at 08:49, Andrew Morton wrote:
> > On Mon, 14 Apr 2008 09:25:46 +0200 Miklos Szeredi  <miklos@szeredi.hu>
> > wrote:
> > 
> > > And I didn't advocate moving
> > > ntfs to fuse, still that was done and the resulting filesystem at the
> > > moment happens to outperform the kernel one in every respect ;)
> > 
> > Gad.  Why?

Well, not only the kernel ntfs (in metadata operations) but also XFS, JFS, 
ext2 in metadata operations, ext3 in large bulk writes on commodity 
hardware.

It's also important to note that ntfs-3g isn't optimized yet at all. There 
are several major performance bootlenecks still in the kernel, FUSE and 
ntfs-3g itself.

Answering the why, the issue is almost never the context switch but the 
file system design and the quality of the implementation. While the latter 
still needs a lot of work on several areas (the size of the Microsoft NTFS 
driver is almost like all the in-kernel file system altogether) but the 
design can very nicely compensate the latter. E.g. ntfs (and ntfs-3g) 
already supports almost all the things what ext4 will include.

> No-one has "moved" ntfs to fuse.  

Yes, historically the user space ntfs code was moved to the kernel. The 
kernel always lagged somehow behind when this work was stopped a few years 
ago.

> However the kernel one as available in the kernel source tree doesn't 
> have many write-features, it can only overwrite files, it cannot 
> create/delete files, etc.  So I guess if you define "performance" to mean 
> "features" then sure ntfsmount/ntfs-3g have more features than the public 
> kernel driver.  If you define "performance" to mean "speed" then no 
> ntfsmount/ntfs-3g can't compare to the kernel except in very limited and 
> meaningless benchmarks...

First comes reliability for ntfs-3g. There are just too many million users. 
This is non-negotiable. Because of this, ntfs-3g is indeed not optimized 
yet. 

However the bootlenecks are almost always storage related (seeks, 
bandwidth, buses) on commodity desktop systems. Embedded devices and 
servers come a bit later with the CPU usage optimizations.

> btw. even comparing features, the fuse solutions lag behind in some  respects,
> e.g. no-one can "kill -9" the kernel driver 

I consider this as a benefit for FUSE file systems. An unloadable kernel 
module results reboot which is much more intrusive.

> leaving a corrupt file system on the volume 

That would be a file system driver implementation bug. What I never denied 
for ntfs-3g. Surely it's solvable but this was never a priority based on 
user wishes.

> (and under no-one I include the OOM killer for example!) 

The OOM killer can be configured and if the fs still uses too much memory 
then probably it's better to be killed/restarted with journaling support.
The important here would be the kernel finally fixing the non-sync behavior 
when it clams to do so (see recent kernel threads).

On the other hand, when in-kernel drivers leak then you get a system 
panic. 

> and another example is that the fuse solutions require large amounts of 
> ram whereas the kernel driver can happily function in 1MiB ram 

A FUSE driver can also happily function in much less than 1 MB RAM.

> when they find a large directory and the user has only 32MiB ram for 
> example...  

This is a FUSE fs implementation, not FUSE problem.

	Szaka

--
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> On Mon, 2008-04-14 at 11:58 +0200, Miklos Szeredi wrote:
> > Oh, you mean without having to run the whole kernel around the
> > filesystem?  Well, that's a bit more tricky: filesystems use not just
> > the VFS API, but the block, VM, SLAB, etc. APIs as well.  Porting all
> > of these to userspace and keeping them in sync with the kernel changes
> > doesn't sound much fun.
> 
> That would be UML :)
> 
> But seriously -- wouldn't our time be better spent on improving error
> handling in the kernel,

Sure, but that needs reviewer time, tester time and developer time.
All of those is limited, and for a piece of code with <100 users and
>1kloc it may not be worth it.

But if people do it for fun, or for having a simple example code to
look at, I have no problem with that :)

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Anton Altaparmakov]

Hi,

On 14 Apr 2008, at 13:46, Szabolcs Szakacsits wrote:
>
> On Mon, 14 Apr 2008 Anton Altaparmakov wrote:
>> On 14 Apr 2008, at 08:49, Andrew Morton wrote:
>>> On Mon, 14 Apr 2008 09:25:46 +0200 Miklos Szeredi  <miklos@szeredi.hu 
>>> >
>>> wrote:
>>>
>>>> And I didn't advocate moving
>>>> ntfs to fuse, still that was done and the resulting filesystem at  
>>>> the
>>>> moment happens to outperform the kernel one in every respect ;)
>>>
>>> Gad.  Why?
>
> Well, not only the kernel ntfs (in metadata operations) but also  
> XFS, JFS,
> ext2 in metadata operations, ext3 in large bulk writes on commodity
> hardware.
>
> It's also important to note that ntfs-3g isn't optimized yet at all.  
> There
> are several major performance bootlenecks still in the kernel, FUSE  
> and
> ntfs-3g itself.
>
> Answering the why, the issue is almost never the context switch but  
> the
> file system design and the quality of the implementation. While the  
> latter
> still needs a lot of work on several areas (the size of the  
> Microsoft NTFS
> driver is almost like all the in-kernel file system altogether) but  
> the
> design can very nicely compensate the latter. E.g. ntfs (and ntfs-3g)
> already supports almost all the things what ext4 will include.
>
>> No-one has "moved" ntfs to fuse.
>
> Yes, historically the user space ntfs code was moved to the kernel.  
> The
> kernel always lagged somehow behind when this work was stopped a few  
> years
> ago.

Yes.  It was much easier for me to write the ntfs code for user space  
in ntfsprogs/libntfs and then once I had the ntfs side of it working  
reliably port it to the kernel so I then only had to deal with kernel  
interface/threading/etc issues but at least the NTFS code would be  
there.  Over time this changed for me and I started working primarily  
on the kernel and pushing things back into libntfs and as these things  
go I eventually was so busy that I no longer had the time to do any  
back porting to libntfs and since then I have only been working on the  
kernel driver.  Unfortunately I haven't been allowed to release my  
work yet as it is used in a closed source production environment but  
hopefully one day I will be allowed to release it and update the  
kernel driver with my code which will move it up to being a fully  
function read-write driver.  Or if I never end up being able to do  
that then things are not too bad either as my MacOSX NTFS driver  
(Leopard already has the read-only driver in it) will eventually be  
released and that is BSD/GPL dual licensed so anyone will be able to  
take the source and port it to Linux thus cutting me out of the loop...

>> However the kernel one as available in the kernel source tree doesn't
>> have many write-features, it can only overwrite files, it cannot
>> create/delete files, etc.  So I guess if you define "performance"  
>> to mean
>> "features" then sure ntfsmount/ntfs-3g have more features than the  
>> public
>> kernel driver.  If you define "performance" to mean "speed" then no
>> ntfsmount/ntfs-3g can't compare to the kernel except in very  
>> limited and
>> meaningless benchmarks...
>
> First comes reliability for ntfs-3g. There are just too many million  
> users.
> This is non-negotiable. Because of this, ntfs-3g is indeed not  
> optimized
> yet.
>
> However the bootlenecks are almost always storage related (seeks,
> bandwidth, buses) on commodity desktop systems. Embedded devices and
> servers come a bit later with the CPU usage optimizations.
>
>> btw. even comparing features, the fuse solutions lag behind in  
>> some  respects,
>> e.g. no-one can "kill -9" the kernel driver
>
> I consider this as a benefit for FUSE file systems. An unloadable  
> kernel
> module results reboot which is much more intrusive.

Kernel modules don't become "unloadable" unless there is a bug.  The  
"kill -9" can happen inadvertently even without any bugs in the FUSE  
or the FUSE-fs.  And that IMO is unacceptable.

>> leaving a corrupt file system on the volume
>
> That would be a file system driver implementation bug. What I never  
> denied
> for ntfs-3g. Surely it's solvable but this was never a priority  
> based on
> user wishes.
>
>> (and under no-one I include the OOM killer for example!)
>
> The OOM killer can be configured and if the fs still uses too much  
> memory
> then probably it's better to be killed/restarted with journaling  
> support.
> The important here would be the kernel finally fixing the non-sync  
> behavior
> when it clams to do so (see recent kernel threads).

You don't get the point.  Any process in the system can be using too  
much memory and trigger the OOM killer even when the FS is behaving  
just fine...

> On the other hand, when in-kernel drivers leak then you get a system
> panic.

So don't leak then!  (-;

>> and another example is that the fuse solutions require large  
>> amounts of
>> ram whereas the kernel driver can happily function in 1MiB ram
>
> A FUSE driver can also happily function in much less than 1 MB RAM.

Obviously it can!  We use a FUSE based file system I wrote here at the  
University on about 850 machines and it doesn't use any RAM at all  
(well, ok a few kiB but that is hardly worth talking about!).

>> when they find a large directory and the user has only 32MiB ram for
>> example...
>
> This is a FUSE fs implementation, not FUSE problem.

I never said it was a FUSE problem!  It is a ntfsmount/ntfs-3g  
problem.  At least a few years ago someone was trying to use ntfsmount  
(or ntfs-3g I can't remember if you had already forked it then) on a  
32MiB RAM embedded ARM box and he was running OOM when trying to list  
directories due to the ntfs/fuse implementation.  In the kernel ntfs  
driver that does not happen.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [PATCH 0/7] OMFS filesystem version 3

[Chris Mason]

On Monday 14 April 2008, Christoph Hellwig wrote:
> On Mon, Apr 14, 2008 at 09:16:39AM +0100, Alan Cox wrote:
> > I think the exceed them quite easily. The costs are almost nil, while
> > merging this provides another nice example fs (and one much easier to
> > follow than ext*) for hardware that does have a few users and will no
> > doubt get many more
> >
> > I wasn't aware Linus had introduced a new rule required 500 people sign
> > up to use a feature before it gets added ?
>
> I'm also very surprised by this, especially as it seems to be applied
> very selectively.  This filesystems is an almost 0 maintainance burden
> unlike a lot of really crappy driver we're shoving in constantly.

Thanks to Bob Copeland for taking the time to submit this for mainline.  
Please don't mistake the resulting debate as a sign we don't appreciate the 
effort of making it available and getting it reviewed.

This zero maintenance burden idea is a little suprising to me.  The kernel 
interfaces do change and we either end up changing every filesystem to 
support the new interface or we need to carry around the old ones for the old 
filesystems.

Even though OMFS seems to be using the generic interfaces well, there is still 
a testing burden for every change.  Someone needs to try it, report any 
problems and get them fixed.  Since none of the people making the changes is 
likely to have an OMFS test bed, all of that burden will fall on Bob, his 
users, and anyone who tries to compile the module (Andrew).

Unlike all the device drivers we don't want floating around out of the tree, 
filesystem authors do have a choice between FUSE and being in-kernel.  Since 
OMFS has been maintained out of tree since 2.6.12 or so, Bob probably has a 
very good idea of how much time he has needed to spend updating things for 
each release.

So, there are two big questions I'd ask:

* Which setup (kernel or fuse) would Bob find easier to maintain things?
* Is this a first step toward more development and an increased user base?

Tossing it in -mm isn't a bad step toward getting attention to see if more 
people want to use it.

-chris

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Mon, 14 Apr 2008, Anton Altaparmakov wrote:

> > I consider this as a benefit for FUSE file systems. An unloadable kernel
> > module results reboot which is much more intrusive.
> 
> Kernel modules don't become "unloadable" unless there is a bug.  

That's exactly what I meant. The majority of the system crashes are due to 
kernel drivers.

> The "kill -9" can happen inadvertently even without any bugs in the FUSE 
> or the FUSE-fs. 

Not really. And if so then distros solve it, as some of them already did 
(e.g. during system shutdown). 

> > The OOM killer can be configured and if the fs still uses too much memory
> > then probably it's better to be killed/restarted with journaling support.
> > The important here would be the kernel finally fixing the non-sync behavior
> > when it clams to do so (see recent kernel threads).
> 
> You don't get the point.  Any process in the system can be using too much
> memory and trigger the OOM killer even when the FS is behaving just fine...

Actually you missed when I wrote "the OOM killer can be configured".

FUSE is a new thing which sometimes requires non-conventional thinking and 
minor adjustments here and there. These works are ongoing for some years 
now.
 
> I never said it was a FUSE problem!  It is a ntfsmount/ntfs-3g problem.  At
> least a few years ago someone was trying to use ntfsmount (or ntfs-3g I can't
> remember if you had already forked it then) on a 32MiB RAM embedded ARM box
> and he was running OOM when trying to list directories due to the ntfs/fuse
> implementation.  In the kernel ntfs driver that does not happen.

Listing a directory with over 100k files can be still an ENOMEM problem 
using 32 MB RAM but of course it's solvable. Nobody was interested so far.

	Szaka

--
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Mon, Apr 14, 2008 at 10:20 AM, Chris Mason <chris.mason@oracle.com> wrote:
>  Thanks to Bob Copeland for taking the time to submit this for mainline.
>  Please don't mistake the resulting debate as a sign we don't appreciate the
>  effort of making it available and getting it reviewed.

Sure, no worries...

>  Unlike all the device drivers we don't want floating around out of the tree,
>  filesystem authors do have a choice between FUSE and being in-kernel.  Since
>  OMFS has been maintained out of tree since 2.6.12 or so, Bob probably has a
>  very good idea of how much time he has needed to spend updating things for
>  each release.

The recent deprecation of iget was probably the most involved of any.  It
wasn't that bad, but the #ifdef jungle going back to at least .17 is getting
somewhat hairy (I have 19 of them).  I would say over the past year or so
most of the changes have been <10 liners with kmem_cache_* interface changes.
Luckily through review here, that got zapped anyway.

In the normal case, I'll grab an -rc3 kernel or so, try to compile it, if
it fails, look at minix to see the minimum set of changes I need to make.
The challenge for me maintaining it out-of-tree is that sometimes the git
logs do not carry enough context to know why XYZ changed and if there's a
better, new way to do things.  Then it's time to scour the mailing lists.

>  So, there are two big questions I'd ask:
>
>  * Which setup (kernel or fuse) would Bob find easier to maintain things?
>  * Is this a first step toward more development and an increased user base?

Well, FUSE is easier from the standpoint of having somewhat more control.
On the other hand, that also results in an inferior product if it isn't given
the many-eyes treatment.  For the code itself, it's really not that much
different since FUSE is so close to the VFS API anyway.

For your second question:

More users are always welcome.  At present, the FUSE version implements a few
things that the kernel module does not - better/different block allocation
algorithm, growing truncate, automatic byte-swapping for the ReplayTV model
that had a busted block layer.  However, the FUSE version is known to be buggy
with respect to every day use (I had one MD5 fail among 20G of files in one
copy test).

I could see adding 1 and 2 to the kernel version, but probably not the
byte-swapping bit.  I'm not sure what to do from there on.

As a user, I use the kernel module because it works, it's faster, and the
other bells and whistles aren't that important to me.  It's hard to tell
what the user community prefers, because the kernel module was first and
so far no one has had a reason to try the FUSE implementation (if they
have, I don't know about it).

-- 
Bob Copeland %% www.bobcopeland.com

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

What does the replaytv byteswap - the entire blocks or some fields ?

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Mon, Apr 14, 2008 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>
>  What does the replaytv byteswap - the entire blocks or some fields ?

Entire blocks (both meta- and normal data).  E.g. the filename 'Filename'
would look like 'eliFeman' on disk.  I have a dm target that can do this
gathering dust somewhere.

-- 
Bob Copeland %% www.bobcopeland.com

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

On Mon, 14 Apr 2008 13:18:53 -0400
"Bob Copeland" <me@bobcopeland.com> wrote:

> On Mon, Apr 14, 2008 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> >
> >  What does the replaytv byteswap - the entire blocks or some fields ?
> 
> Entire blocks (both meta- and normal data).  E.g. the filename 'Filename'
> would look like 'eliFeman' on disk.  I have a dm target that can do this
> gathering dust somewhere.

Thats what I was wondering. The dm target would be useful and is also
needed for certain old 68K boxes with libata

Re: [PATCH 0/7] OMFS filesystem version 3

[david]

On Mon, 14 Apr 2008, Chris Mason wrote:

> On Monday 14 April 2008, Christoph Hellwig wrote:
>> On Mon, Apr 14, 2008 at 09:16:39AM +0100, Alan Cox wrote:
>>> I think the exceed them quite easily. The costs are almost nil, while
>>> merging this provides another nice example fs (and one much easier to
>>> follow than ext*) for hardware that does have a few users and will no
>>> doubt get many more
>>>
>>> I wasn't aware Linus had introduced a new rule required 500 people sign
>>> up to use a feature before it gets added ?
>>
>> I'm also very surprised by this, especially as it seems to be applied
>> very selectively.  This filesystems is an almost 0 maintainance burden
>> unlike a lot of really crappy driver we're shoving in constantly.
>
> Thanks to Bob Copeland for taking the time to submit this for mainline.
> Please don't mistake the resulting debate as a sign we don't appreciate the
> effort of making it available and getting it reviewed.

seconded.

> Unlike all the device drivers we don't want floating around out of the tree,
> filesystem authors do have a choice between FUSE and being in-kernel.  Since
> OMFS has been maintained out of tree since 2.6.12 or so, Bob probably has a
> very good idea of how much time he has needed to spend updating things for
> each release.

switching to FUSE also has a cost for users, namely that they need to have 
FUSE setup (and the various interactions and deadlocks that can happen 
with a userspace filesystem, such as swapping to it)

as a user I would prefer to see filesystems (even ones I don't expect to 
uer) be all treated the same way, not have to figure out that to use this 
list of filesystems I configure them in the kernel, and to use that list 
of filesystem I have to run FUSE.

for testing, or for things that aren't really filesystems (views into 
version control systems, tarballs, etc) FUSE is a good match.

but for real filesystems it's a poor second.

David Lang

Re: [PATCH 0/7] OMFS filesystem version 3

[Jeff Garzik]

Christoph Hellwig wrote:
> On Mon, Apr 14, 2008 at 09:16:39AM +0100, Alan Cox wrote:
>> I think the exceed them quite easily. The costs are almost nil, while
>> merging this provides another nice example fs (and one much easier to
>> follow than ext*) for hardware that does have a few users and will no
>> doubt get many more
>>
>> I wasn't aware Linus had introduced a new rule required 500 people sign
>> up to use a feature before it gets added ?
> 
> I'm also very surprised by this, especially as it seems to be applied
> very selectively.  This filesystems is an almost 0 maintainance burden
> unlike a lot of really crappy driver we're shoving in constantly.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Agreed.  And adding to the pile-on, I think that we should be 
_encouraging_ rather than discouraging innovative new filesystems.

I rather dislike limiting filesystem maintenance to the "old guard".

See comments like those on Evgeniy's blog: 
http://tservice.net.ru/~s0mbre/blog/devel/fs/2008_04_14.html

Sure we do not need to merge every filesystem under the sun, or things 
that much better belong in userspace (sshfs), but AFAICS there is not a 
huge increase in maintenance cost to add a new kernel filesystem.

	Jeff

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 16:55:04 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> See comments like those on Evgeniy's blog: 
> http://tservice.net.ru/~s0mbre/blog/devel/fs/2008_04_14.html

Sigh.  I've already addressed that.

A new filesystem such as the one he's working on would, one hopes, be used
by an increasing number of people.  Whereas I expect that the already-tiny
number of people who use OMFS will *fall* over time.


Guys, a few years ago Linus's rule of thumb for merging an FS was,
approximately, "it is already being shipped by distros".  The bar is _much_
lower than that now.  Largely because I personally think that it should be.

Re: [PATCH 0/7] OMFS filesystem version 3

[Evgeniy Polyakov]

Hi.

On Mon, Apr 14, 2008 at 02:11:39PM -0700, Andrew Morton (akpm@linux-foundation.org) wrote:
> A new filesystem such as the one he's working on would, one hopes, be used
> by an increasing number of people.  Whereas I expect that the already-tiny
> number of people who use OMFS will *fall* over time.

It is not about pohmelfs, but any other. When did you look at fs/kconfig
last time? There are lots of filesystems which were obsoleted way before
I first time started using computer. Hope you saw with pdp-11 in action
(I only on pictures), since sysv filesystem is very unlikely to get new
users today. Or befs - beos is very widely distributed os. Others?
 
> Guys, a few years ago Linus's rule of thumb for merging an FS was,
> approximately, "it is already being shipped by distros".  The bar is _much_
> lower than that now.  Largely because I personally think that it should be.

And Linus was wrong. This can not happen? He can not change his opinion?
Btw, which distro shipped ext4 before it was included (splitted) into
mainline?

Just say no to double standards!

P.S. Added Linus to copy list, hope it will be interesting...

-- 
	Evgeniy Polyakov

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Miklos Szeredi wrote:
> > > > Yes, that's quite annoying as FUSE doesn't work with my architecture,
> > > > so I can't use the only good NTFS driver.... grr! :-)
> > > 
> > > I'm not quite getting this joke :)
> > 
> > It's not really a joke.  It really is a problem: I'm supposed to
> > provide NTFS support for a project, but the kernel I'm using doesn't
> > work with FUSE (too old for the FUSE version needed by ntfs-3g),
> 
> How old?  The out of tree fuse module (hoping to get rid of it soon)
> works down to 2.6.9.  Beyond that it does probably require a fair
> amount of porting work.

I imagine that backporting FUSE to 2.4 no-mmu will be more work than
backporting a filesystem, but I could be mistaken.  Moving my project
to a 2.6 kernel (*any* 2.6 kernel) would be much more work than
either.

All other filesystems I care about are in-kernel - NTFS is quite
exceptional in needing FUSE just to get ordinary, stable file access.
(The in-kernel NTFS is regarded as unstable and missing essential
basic features.)

None of this should be taken as any kind of criticism.  I love what
you guys have done.

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[SL Baur]

On 4/14/08, Andrew Morton <akpm@linux-foundation.org> wrote:

>  Guys, a few years ago Linus's rule of thumb for merging an FS was,
>  approximately, "it is already being shipped by distros".  The bar is _much_
>  lower than that now.  Largely because I personally think that it should be.

I thought he wrote in the lead message that it had been picked up
by at least three different distros.  Does that not count?

-sb

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Tue, 15 Apr 2008 02:32:31 +0400
Evgeniy Polyakov <johnpol@2ka.mipt.ru> wrote:

> Hi.
> 
> On Mon, Apr 14, 2008 at 02:11:39PM -0700, Andrew Morton (akpm@linux-foundation.org) wrote:
> > A new filesystem such as the one he's working on would, one hopes, be used
> > by an increasing number of people.  Whereas I expect that the already-tiny
> > number of people who use OMFS will *fall* over time.
> 
> It is not about pohmelfs, but any other. When did you look at fs/kconfig
> last time? There are lots of filesystems which were obsoleted way before
> I first time started using computer. Hope you saw with pdp-11 in action
> (I only on pictures), since sysv filesystem is very unlikely to get new
> users today. Or befs - beos is very widely distributed os. Others?

This has already been addressed.

> > Guys, a few years ago Linus's rule of thumb for merging an FS was,
> > approximately, "it is already being shipped by distros".  The bar is _much_
> > lower than that now.  Largely because I personally think that it should be.
> 
> And Linus was wrong. This can not happen? He can not change his opinion?
> Btw, which distro shipped ext4 before it was included (splitted) into
> mainline?
> 
> Just say no to double standards!

I've already told you that we no longer apply such a strict test.

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Mon, 14 Apr 2008 16:09:41 -0700
"SL Baur" <steve@xemacs.org> wrote:

> On 4/14/08, Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> >  Guys, a few years ago Linus's rule of thumb for merging an FS was,
> >  approximately, "it is already being shipped by distros".  The bar is _much_
> >  lower than that now.  Largely because I personally think that it should be.
> 
> I thought he wrote in the lead message that it had been picked up
> by at least three different distros.

I missed that.

>  Does that not count?

Not really - I already agreed to merge v4.

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Mon, 14 Apr 2008 Miklos Szeredi wrote:

> I don't claim fuse has a great part in it, 

Yep, actually it's me who claims that without FUSE there wouldn't be today 
open source read/write NTFS driver for Linux, FreeBSD, NetBSD, OS X, 
Solaris, etc in a pretty short time developed as a spare time hobby (the 
estimated development cost of the Microsoft NTFS driver is $20-30 million). 
Quite many people contributed significantly in the last twelve years but 
the breakthrough came via FUSE.

Probably FUSE is the most significant innovation Linux ever contributed to 
computer science in the file system area. I think the hundreds of FUSE file 
systems and the happy __users__ on all the FUSE platforms justify this, who 
absolutely don't care if it's a kernel, user, hibrid, or mikrospace driver.

> except the obvious advantages of developing in userspace: no kernel 
> crashes, easier debugging, etc...

In my experience the huge difference is for users getting better quality 
and faster support. It takes from several months to many years to get new 
features and fixes to millions of user via kernel drivers but it takes a 
matter of hours/days via user space binaries. This results a much faster, 
quality and feature efficient release cycle.

The kernel part is indeed always the most problematic because fixes and new 
things take very long time to arrive to end users. Some longstanding major 
issues:

  - asynchronous unmount. It was solved within days in August of 2006,
    still a problem for users.

  - USB powering down/up problems with some popular external USB drives. 
    People are loosing data because of this since a year. 

  - sync(2) is not syncing

  - recent kernels ignore file permissions
    
  - large block support

  - non-aligned async writes are converted to sync reads

  - ARM 50x context switch speed up patch not used 


	Szaka

--
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Erez Zadok]

In message <20080413033344.GA27494@hash.localnet>, Bob Copeland writes:
> On Sat, Apr 12, 2008 at 05:03:04PM -0700, Andrew Morton wrote:
[...]

> > Who did the reverse-engineering, and how was it done?
> > Please make us confident that we won't get our butts sued off or something.
> 
> I'm not sure there's a company to sue anymore.

<IANAL>

Bob, you have to be careful there; an argument like "there may not be a
company to sue" may not hold.  IP rights, patents and their filings, and
more, can be sold, traded, transferred, inherited, etc. -- even if the
original company is no longer in business.  I don't know the history of
ReplayTV or how it went out of business, but it might be worth a check to
see if there's anyone who can still claim any rights over any part of OMFS
(e.g., chapter 11 creditors who may feel they didn't get fully compensated).
The irony is that as long as omfs gets little attention, no one is likely to
sue; but once and if it gets into mainline and lots of Big Name Linux
distros start carrying it, *then* someone may come out from the shadows to
sue.

</IANAL>

Erez.

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > > > > Yes, that's quite annoying as FUSE doesn't work with my architecture,
> > > > > so I can't use the only good NTFS driver.... grr! :-)
> > > > 
> > > > I'm not quite getting this joke :)
> > > 
> > > It's not really a joke.  It really is a problem: I'm supposed to
> > > provide NTFS support for a project, but the kernel I'm using doesn't
> > > work with FUSE (too old for the FUSE version needed by ntfs-3g),
> > 
> > How old?  The out of tree fuse module (hoping to get rid of it soon)
> > works down to 2.6.9.  Beyond that it does probably require a fair
> > amount of porting work.
> 
> I imagine that backporting FUSE to 2.4 no-mmu will be more work than
> backporting a filesystem, but I could be mistaken.  Moving my project
> to a 2.6 kernel (*any* 2.6 kernel) would be much more work than
> either.

Fuse-2.5 still supported linux-2.4.  Although I have no idea if it
works with no-mmu, maybe it's worth a try.

Backporting ntfs-3g to an older version of fuse shouldn't be a big
problem, as the fuse interface didn't change very much since then.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Adrian Bunk]

On Tue, Apr 15, 2008 at 03:11:27AM +0300, Szabolcs Szakacsits wrote:
>...
> In my experience the huge difference is for users getting better quality 
> and faster support. It takes from several months to many years to get new 
> features and fixes to millions of user via kernel drivers but it takes a 
> matter of hours/days via user space binaries. This results a much faster, 
> quality and feature efficient release cycle.
> 
> The kernel part is indeed always the most problematic because fixes and new 
> things take very long time to arrive to end users.
>...

I don't think that's true:

Why does it have to take longer to distribute e.g. a new version of the 
ALSA kernel drivers to end users than to distribute a new version of 
userspace binaries to them?

> 	Szaka

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH 0/7] OMFS filesystem version 3

[Adrian Bunk]

On Sun, Apr 13, 2008 at 04:10:14PM -0700, Andrew Morton wrote:
> On Sun, 13 Apr 2008 23:49:20 +0100 Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> 
> > > I guess I can keep making this point in various ways until someone
> > > actually notices it:
> > > 
> > > 	This filesystem has only 20 users.
> > 
> > At the moment. And that probably exceeds Amiga users, 386 users, some of
> > the serial port users, several network card users ...
> > 
> > In the past we've merged drivers for network cards where only two
> > existed in the world. Linus has repeatedly stated he wants to see stuff
> > people are using getting in.
> 
> None of that means that merging this filesystem is the best decision.
> 
> > Good clean code that doesn't affect the core
> > is good reference material.
> 
> The reference block filesystem is ext2 (used to be minixfs) - there is no
> need for another.
> 
> > I think you are (unusually) the one out of step here ?
> 
> I appear to be the only one who is looking at the whole picture.
> 
> Merging a new filesystem has costs - I don't need to enumerate them.  Do
> the benefits of OMFS exceed them?

We do not have a stable API for external modules, and part of the deal 
is that external modules have the chance of entering the kernel where 
they will get API changes automatically.

We are talking about a filesystem even Christoph considers OK.

And who asked about the costs of merging crap like 
drivers/infiniband/hw/nes/ ?

Speaking about the latter, with Linus' logic one might argue that OMFS 
must not be rejected since it adds support for some hardware...

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Miklos Szeredi wrote:
> > > How old?  The out of tree fuse module (hoping to get rid of it soon)
> > > works down to 2.6.9.  Beyond that it does probably require a fair
> > > amount of porting work.
> > 
> > I imagine that backporting FUSE to 2.4 no-mmu will be more work than
> > backporting a filesystem, but I could be mistaken.  Moving my project
> > to a 2.6 kernel (*any* 2.6 kernel) would be much more work than
> > either.
> 
> Fuse-2.5 still supported linux-2.4.  Although I have no idea if it
> works with no-mmu, maybe it's worth a try.
> 
> Backporting ntfs-3g to an older version of fuse shouldn't be a big
> problem, as the fuse interface didn't change very much since then.

Thanks for your advice.  I will look into all these possibilities.
Fuse working would be quite useful in many other ways too.

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Christoph Hellwig]

On Tue, Apr 15, 2008 at 06:16:43PM +0300, Adrian Bunk wrote:
> We are talking about a filesystem even Christoph considers OK.
> 
> And who asked about the costs of merging crap like 
> drivers/infiniband/hw/nes/ ?
> 
> Speaking about the latter, with Linus' logic one might argue that OMFS 
> must not be rejected since it adds support for some hardware...

Excatly.  I find it very strange to even consider rejecting a rather
small and very well written driver for let's say "political" reasons.

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Tue, 15 Apr 2008 12:57:27 -0400 Christoph Hellwig <hch@infradead.org> wrote:

> On Tue, Apr 15, 2008 at 06:16:43PM +0300, Adrian Bunk wrote:
> > We are talking about a filesystem even Christoph considers OK.
> > 
> > And who asked about the costs of merging crap like 
> > drivers/infiniband/hw/nes/ ?
> > 
> > Speaking about the latter, with Linus' logic one might argue that OMFS 
> > must not be rejected since it adds support for some hardware...
> 
> Excatly.  I find it very strange to even consider rejecting a rather
> small and very well written driver for let's say "political" reasons.

"economic" would be a far more accurate term.

Look, I have repeatedly described the reason why it is probable a poor
tradeoff to merge code such as this.  The only response has been "well
we've done it before", which is largely a non-reason.

You can continue to ignore my logic, but that won't go unnoticed.

Just as a thought exercise: should we merge a small and well-written
driver which has zero users?

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

> You can continue to ignore my logic, but that won't go unnoticed.
> 
> Just as a thought exercise: should we merge a small and well-written
> driver which has zero users?

We've previously merged ones that fit this category and have one user. If
the driver doesn't disrupt core code and is useful to help other people
write drivers then I think we should. It may also get more users as a
result.

This is a free software project not a business enterprise.

Alan

Re: [PATCH 0/7] OMFS filesystem version 3

[Adrian Bunk]

On Tue, Apr 15, 2008 at 11:34:52AM -0700, Andrew Morton wrote:
> On Tue, 15 Apr 2008 12:57:27 -0400 Christoph Hellwig <hch@infradead.org> wrote:
> 
> > On Tue, Apr 15, 2008 at 06:16:43PM +0300, Adrian Bunk wrote:
> > > We are talking about a filesystem even Christoph considers OK.
> > > 
> > > And who asked about the costs of merging crap like 
> > > drivers/infiniband/hw/nes/ ?
> > > 
> > > Speaking about the latter, with Linus' logic one might argue that OMFS 
> > > must not be rejected since it adds support for some hardware...
> > 
> > Excatly.  I find it very strange to even consider rejecting a rather
> > small and very well written driver for let's say "political" reasons.
> 
> "economic" would be a far more accurate term.
> 
> Look, I have repeatedly described the reason why it is probable a poor
> tradeoff to merge code such as this.  The only response has been "well
> we've done it before", which is largely a non-reason.
>...

It seems you missed the first point in my email:

We do not have a stable API for external modules, and part of the deal 
is that external modules have the chance of entering the kernel where 
they will get API changes automatically.


Plus my other point that one might argue that OMFS adds support for some 
hardware in which case a recent commandment by Linus would require it 
has to be merged...


cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH 0/7] OMFS filesystem version 3

[Alan Cox]

> > We've previously merged ones that fit this category and have one user.
> 
> Probably incorrectly.

I would disagree as drivers we've merged like that go on to have many
users or give us new developers who stay with the project (and in some
cases make projects practical to be done on Linux)

> > This is a free software project not a business enterprise.
> 
> And the same principles apply, don't they?  We seek to minimise overhead
> and to use our kernel development hours as effectively as we can.

One word if we are getting into the economics of this: Externalities.
There are lots of benefits from merging the code beyond simple value of
code merge including more developers, more reference code, more users.
The ratio of new contributions to maintenance is a different problem and
one I do not think should be conflated with it.

But yes I do think they are different - free software is largely done for
fun, by people who want to contribute. What was it John Betjeman said of
another large volunteer project:

"the result of the independent spirit which still survives in this
country and refuses to be crushed by the money-worshippers, centralizers
and the unimaginative theorists who are doing their best to kill it"

Alan

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Tue, 15 Apr 2008 19:53:30 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > You can continue to ignore my logic, but that won't go unnoticed.
> > 
> > Just as a thought exercise: should we merge a small and well-written
> > driver which has zero users?
> 
> We've previously merged ones that fit this category and have one user.

Probably incorrectly.

> If
> the driver doesn't disrupt core code and is useful to help other people
> write drivers then I think we should. It may also get more users as a
> result.

Maybe.

> This is a free software project not a business enterprise.

And the same principles apply, don't they?  We seek to minimise overhead
and to use our kernel development hours as effectively as we can.

This is particularly important when person A is thrusting additional work
onto persons B, C, and D.

Re: [PATCH 0/7] OMFS filesystem version 3

[Andrew Morton]

On Tue, 15 Apr 2008 22:24:32 +0300
Adrian Bunk <bunk@kernel.org> wrote:

> > Look, I have repeatedly described the reason why it is probable a poor
> > tradeoff to merge code such as this.  The only response has been "well
> > we've done it before", which is largely a non-reason.
> >...
> 
> It seems you missed the first point in my email:
> 
> We do not have a stable API for external modules, and part of the deal 
> is that external modules have the chance of entering the kernel where 
> they will get API changes automatically.
> 
> 
> Plus my other point that one might argue that OMFS adds support for some 
> hardware in which case a recent commandment by Linus would require it 
> has to be merged...

That's lawyerly trickery, sorry.  Take some set of guidelines and then say
"you are thereby committed to doing X".

We're not committed to doing anything and it would be bad if we were. 
Let's apply common sense and judgement to each case on its own.

Re: [PATCH 0/7] OMFS filesystem version 3

[Adrian Bunk]

On Tue, Apr 15, 2008 at 01:11:20PM -0700, Andrew Morton wrote:
> On Tue, 15 Apr 2008 22:24:32 +0300
> Adrian Bunk <bunk@kernel.org> wrote:
> 
> > > Look, I have repeatedly described the reason why it is probable a poor
> > > tradeoff to merge code such as this.  The only response has been "well
> > > we've done it before", which is largely a non-reason.
> > >...
> > 
> > It seems you missed the first point in my email:
> > 
> > We do not have a stable API for external modules, and part of the deal 
> > is that external modules have the chance of entering the kernel where 
> > they will get API changes automatically.
> > 
> > 
> > Plus my other point that one might argue that OMFS adds support for some 
> > hardware in which case a recent commandment by Linus would require it 
> > has to be merged...
> 
> That's lawyerly trickery, sorry.  Take some set of guidelines and then say
> "you are thereby committed to doing X".
> 
> We're not committed to doing anything and it would be bad if we were. 
> Let's apply common sense and judgement to each case on its own.

My favorite gems from the stuff even checkpatch finds in the 
INFINIBAND_NES driver, for which Linus has stated explicitely that 
merging it in this state in 2.6.25 was correct, can be seen with

  grep -C4 volatile drivers/infiniband/hw/nes/nes_nic.c


When we have the resources to maintain this kind of code, how could a 
small filesystem be a problem?


cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH 0/7] OMFS filesystem version 3

[david m. richter]

On Tue, 15 Apr 2008, Alan Cox wrote:
<snip>

> > > This is a free software project not a business enterprise.
> > 
> > And the same principles apply, don't they?  We seek to minimise overhead
> > and to use our kernel development hours as effectively as we can.
> 
> One word if we are getting into the economics of this: Externalities.
> There are lots of benefits from merging the code beyond simple value of
> code merge including more developers, more reference code, more users.
> The ratio of new contributions to maintenance is a different problem and
> one I do not think should be conflated with it.
> 
> But yes I do think they are different - free software is largely done for
> fun, by people who want to contribute. What was it John Betjeman said of
> another large volunteer project:
> 
> "the result of the independent spirit which still survives in this
> country and refuses to be crushed by the money-worshippers, centralizers
> and the unimaginative theorists who are doing their best to kill it"
> 
> Alan

	to interject, and in response to a variety of posts in this thread 
and in no way drawing solely on yours, alan: despite that it may sound 
like some interesting viewpoints are getting aired here, i think the 
discussion may've unproductively slewed to a point where andrew's 
unfortunately been painted into a corner in which he has little practical 
interest.

	it seems like some folks have misinterpreted andrew's remarks as 
being variously inflexible, elitist, "political", pro-"corporate-drone", 
working at cross-purposes to the spirit of open source, and otherwise.  
in rereading the thread, though, he's pretty much been the most 
even-handed, moderate, practical, and open to discussion and compromise of 
anyone involved.

	he gave an opinion about the merits of OMFS in-kernel vs FUSE, 
clarified his stance, remained open to other's comments and criticism, 
didn't rise to any bait when people called his ideas "stupid" and whatnot 
(despite that he specifically said he had "no strong opinions either 
way"), and he compromised: he offered to pull the code into -mm; he said 
he'd merge v4.

	known: OMFS has a few bugs (thanks, reviewers), needs testing, and 
has outstanding legal questions to resolve.  andrew offered to merge the 
next version.

	to ground the topic in terms of practicality: what else would all 
of you have andrew do, at this moment, that he hasn't already done?  can 
we let him get back to work?


	thanks,

	d
	.

Re: [PATCH 0/7] OMFS filesystem version 3

[Bodo Eggert]

Anton Altaparmakov <aia21@cam.ac.uk> wrote:
> On 14 Apr 2008, at 13:46, Szabolcs Szakacsits wrote:
>> On Mon, 14 Apr 2008 Anton Altaparmakov wrote:

>>> (and under no-one I include the OOM killer for example!)
>>
>> The OOM killer can be configured and if the fs still uses too much
>> memory
>> then probably it's better to be killed/restarted with journaling
>> support.
>> The important here would be the kernel finally fixing the non-sync
>> behavior
>> when it clams to do so (see recent kernel threads).
> 
> You don't get the point.  Any process in the system can be using too
> much memory and trigger the OOM killer even when the FS is behaving
> just fine...

Just set /proc/$pid/oom_adj to -17.

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Tue, 15 Apr 2008, Jamie Lokier wrote:
> Miklos Szeredi wrote:
> > > > How old?  The out of tree fuse module (hoping to get rid of it soon)
> > > > works down to 2.6.9.  Beyond that it does probably require a fair
> > > > amount of porting work.
> > > 
> > > I imagine that backporting FUSE to 2.4 no-mmu will be more work than
> > > backporting a filesystem, but I could be mistaken.  Moving my project
> > > to a 2.6 kernel (*any* 2.6 kernel) would be much more work than
> > > either.
> > 
> > Fuse-2.5 still supported linux-2.4.  Although I have no idea if it
> > works with no-mmu, maybe it's worth a try.
> > 
> > Backporting ntfs-3g to an older version of fuse shouldn't be a big
> > problem, as the fuse interface didn't change very much since then.
> 
> Thanks for your advice.  I will look into all these possibilities.
> Fuse working would be quite useful in many other ways too.

ntfs-3g was reported to work on 2.4 no-mmu kernels six months ago (though 
it had no our QA). Please see http://ntfs-3g.org/support.html#kernel24 and 
search for nommu on fuse-devel for the details.

	Szaka

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Szabolcs Szakacsits wrote:
> > > > > How old?  The out of tree fuse module (hoping to get rid of it soon)
> > > > > works down to 2.6.9.  Beyond that it does probably require a fair
> > > > > amount of porting work.
> > > > 
> > > > I imagine that backporting FUSE to 2.4 no-mmu will be more work than
> > > > backporting a filesystem, but I could be mistaken.  Moving my project
> > > > to a 2.6 kernel (*any* 2.6 kernel) would be much more work than
> > > > either.
> > > 
> > > Fuse-2.5 still supported linux-2.4.  Although I have no idea if it
> > > works with no-mmu, maybe it's worth a try.
> > > 
> > > Backporting ntfs-3g to an older version of fuse shouldn't be a big
> > > problem, as the fuse interface didn't change very much since then.
> > 
> > Thanks for your advice.  I will look into all these possibilities.
> > Fuse working would be quite useful in many other ways too.
> 
> ntfs-3g was reported to work on 2.4 no-mmu kernels six months ago (though 
> it had no our QA). Please see http://ntfs-3g.org/support.html#kernel24 and 
> search for nommu on fuse-devel for the details.

Ah, wonderful!  2.4 kernel support has been _added_ to NTFS-3g's
"FUSE-lite" since I last looked!

Nice to see 2.4 kernel support is still an active priority ;-)

Thanks for the info, it's really useful.  I would never have thought
to check for 2.4 support being added back, since an earlier
announcement said it was no longer supported.

(Btw, The NTFS-3g release notes, on it's decision to replace the FUSE
userspace libs (which is what makes it work with 2.4 kernels too),
suggest there may be serious problems in the standard FUSE libs,
especially on small devices.  It doesn't bother me, as I only need
NTFS-3g, not other FUSE capabilities, but perhaps FUSE developers
would like to take a look at those problems.)

Thanks again,
-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> (Btw, The NTFS-3g release notes, on it's decision to replace the FUSE
> userspace libs (which is what makes it work with 2.4 kernels too),
> suggest there may be serious problems in the standard FUSE libs,
> especially on small devices.

There were compatibility problems with uClibc, but those have now been
addressed in the standard fuse libs as well.

I actually think fuse-lite is a good thing (as long as someone is
maintining it): it allows dynamically changing projects like ntfs-3g
to distribute and link their own version of libfuse, which contains
the latest features they need.  Not having to wait for these features
to slowly make it into distributions can be a big advantage.

At the same time fuse-lite doesn't need to carry all the source and
binary compatibility cruft that the standard fuse lib needs, so
statically linking it has a relatively small overhead.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Sun, 13 Apr 2008, Bob Copeland wrote:

> I don't have hard numbers, but anecdotally my FUSE version is quite 
> a bit less performant.  That's no criticism of FUSE - I just haven't
> put the time into optimizing and adding various caches.

Thankfully you need none, it's already there by FUSE and the kernel. The 
trick is exactly that you can have the kernel performance and the left is 
moved to user space with typically negligible performance overhead which is 
usually well compensated with faster delivered new features and bug fixes.

The completely unoptimized ntfs-3g read/write saturates my USB disks with 
25-30 MB/sec using 8% (read), 35% (write) CPU time on a 2.5 GHz Core 2 Duo 
with cold caches.

If you have the free hot caches then it performs the same as in-kernel 
file systems, user space isn't involved at all.

I noticed that the OMFS kernel driver supports only the USB interface and 
the FUSE one both the network and the USB one. Isn't it possible that you 
compared the performance using the USB with the kernel vs the much slower 
and lower latency network with FUSE? You should compare performances using 
only the USB interface in both cases.

If you did use the USB interface with FUSE then what exactly do you mean by 
"quite a bit less performance" in numbers and workloads? What you did, how 
long it took using what CPU?

Thanks,
	    Szaka

-- 
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Szabolcs Szakacsits wrote:
> If you have the free hot caches then it performs the same as in-kernel 
> file systems, user space isn't involved at all.

Presumably if "user space isn't involved at all", it must require that
user space has granted caching rights to the kernel over a FUSE
cache coherency protocol?

Otherwise I don't see how the kernel could coherently cache file pages
for some kinds of FUSE filesystems.  (E.g. sshfs, for example: every
operation must surely invoke a user space request or involve granting
a caching right to the kernel, to keep accesses coherent with other
users of the same remote files).

Ergo, either its not coherent, or there is some coherency protocol,
which does require _some_ work in the user space implementation.

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> Szabolcs Szakacsits wrote:
> > If you have the free hot caches then it performs the same as in-kernel 
> > file systems, user space isn't involved at all.
> 
> Presumably if "user space isn't involved at all", it must require that
> user space has granted caching rights to the kernel over a FUSE
> cache coherency protocol?
> 

Most filesystems (ntfs-3g included) cannot be modified from the
outside, so coherency isn't an issue.

Otherwise currently fuse provides rather crude settings for caching:

  - timeout for attributes (per-inode, default 1s)
  - timeout for names (per-dentry, default 1s)
  - page cache is bypassed completely (per-file, default off)
  - invalidation of page cache on open (per-open, default on)

Writes are synchronous even in the caching case, for various reasons.
That is one of the reasons why write performance can be worse than
read performance.  For some filesystems (ntfs-3g), this limitation
could be lifted, but that requires that the mtime handling be moved to
the kernel in these cases, which is not yet possible.

There are also plans to add some sort of cache coherency protocol,
where the filesystem can asynchronously call back to fuse to
invalidate data or metadata.

> Otherwise I don't see how the kernel could coherently cache file pages
> for some kinds of FUSE filesystems.  (E.g. sshfs, for example: every
> operation must surely invoke a user space request or involve granting
> a caching right to the kernel, to keep accesses coherent with other
> users of the same remote files).
> 
> Ergo, either its not coherent, or there is some coherency protocol,
> which does require _some_ work in the user space implementation.

Sshfs is not coherent (but neither is NFS), it just has timeouts for
caches and invalidation based on modification time.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Sun, 13 Apr 2008, Christoph Hellwig wrote:
> > I'm not complaining about anything.  Who has?
> > 
> > As the filesystem is for occasional, non-performance-sensitive use
> > by a very small number of people, doing it via FUSE sounds like an
> > all-round more practical approach.  This has nothing to do with quality of
> > implementation at all.
> 
> It's a stupid idea.  

FUSE is more than just providing a framework for block based filesystems. 
Sometimes FUSE is criticised for the extra useful features it provides. 
Sometimes the criticism comes from misunderstandings.

> less efficient because of the additional context switches 

This is about as true as claiming FUSE doesn't have any context switch 
overhead. Sometimes it has none, sometimes small, sometimes it can be 
significant (e.g. when not using the 50 fold context switch speed up 
patch). The question is, how releavant it is? Just some short notes,

 1. Using the in-kernel cached data involves no context switch for FUSE.

 2. On commodity hardware Linux can do a million context switches. 
    File system workloads barely need or can do more than a few tens 
    of thousands file operations per second (fsops) due to storage 
    bottlenecks. Which means maximum about only extra 5% CPU use for 
    block based FUSE file systems. 

    If they do more fsops then typically it's served from the kernel caches
    and no user space and context switches are involved at all.

 3. ext3 with the highly optimized dir_index is far the fastest 
    traditional block based file system in file creation. Once I wrote
    a FUSE file system which apparently would have been faster if the 
    VFS wouldn't do needlessly a lookup() before create(). AFAIK some 
    network file systems have the some performance problem because of 
    this.

 4. The dominant factors for performance is design, quality of the
    implementation and lead time to optimize for the (latest) hardware. 
    What's the best way to realize this depends on many factors.

 5. Some workloads can indeed trigger very high context switches. This 
    could be improved/solved but probably it would be more beneficial to 
    improve context switch performance in software and hardware.

> and harder to use because you need additional userspace packages and need 
> to setup fuse.

The fuse install should solve all setup issues and a fuse fs can be written 
where the traditional commands work:

	mount -t fstype device mountpoint
	umount mountpoint

Ntfs-3g doesn't even need fuse user space being installed, only a 
modprobeable fuse kernel module.

	Szaka

-- 
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Miklos Szeredi wrote:
> Most filesystems (ntfs-3g included) cannot be modified from the
> outside, so coherency isn't an issue.
> 
> Otherwise currently fuse provides rather crude settings for caching:
> 
>   - timeout for attributes (per-inode, default 1s)
>   - timeout for names (per-dentry, default 1s)
>   - page cache is bypassed completely (per-file, default off)
>   - invalidation of page cache on open (per-open, default on)

Seems sensible.  As someone who is deep in coherency protocols at the
moment (I'm writing a robust distributed database/filesystem) I don't
like the crudeness, but for FUSE's real life use it seems a fine choice :-)

> There are also plans to add some sort of cache coherency protocol,
> where the filesystem can asynchronously call back to fuse to
> invalidate data or metadata.

Great!

I suggest adding another option (as well) where the filesystem can ask
fuse to send it synchronous validation requests - some things require
that.  (In my own work, the choice of A->B async invalidation and B->A
synchronous validation is heuristic: some usage patterns benefit from
one, some from the other.)

> > Otherwise I don't see how the kernel could coherently cache file pages
> > for some kinds of FUSE filesystems.  (E.g. sshfs, for example: every
> > operation must surely invoke a user space request or involve granting
> > a caching right to the kernel, to keep accesses coherent with other
> > users of the same remote files).
> > 
> > Ergo, either its not coherent, or there is some coherency protocol,
> > which does require _some_ work in the user space implementation.
> 
> Sshfs is not coherent (but neither is NFS), it just has timeouts for
> caches and invalidation based on modification time.

Fwiw, I think NFS version 4 is coherent (it uses leases), and older
NFS should be coherent when you use fcntl file locks (it's not very
efficient though).

I have been bitten a few times by timeout based caches in the past
(NFS and SMB (pre-oplock)).  Simple things like editing a file, then
running "ssh compiler-box make" from the editor quietly building
incorrect code - and even subsequent make commands don't fix it.  Or
when I edit a file, then tell someone I've changed the file - and then
they edit the file, and my edits are lost.  Very annoying.  Nobody
should build those kind of caches into new software.  :-)

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Mon, 14 Apr 2008 david@lang.hm wrote:

> and the various interactions and deadlocks that can happen with a
> userspace filesystem, such as swapping to it)

Swap on loopback could deadlock with whatever file system, not only FUSE.

The correct and safe way to use swap with FUSE is using the fuseblk fstype 
and a swap file on the FUSE file system (so it's exactly the same as one 
would do with an in-kernel file system). This completely avoids the usage 
of user space during swapping. This is in use for over a year with ntfs-3g, 
e.g. by Knoppix and Mr Klaus Knopper wrote only positive things to me so 
far.

	Szaka

-- 
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Fri, Apr 18, 2008 at 6:30 AM, Szabolcs Szakacsits <szaka@ntfs-3g.org> wrote:
>  On Sun, 13 Apr 2008, Bob Copeland wrote:
>
>  > I don't have hard numbers, but anecdotally my FUSE version is quite
>  > a bit less performant.  That's no criticism of FUSE - I just haven't
>  > put the time into optimizing and adding various caches.
>
>  Thankfully you need none, it's already there by FUSE and the kernel. The
>  trick is exactly that you can have the kernel performance and the left is
>  moved to user space with typically negligible performance overhead which is
>  usually well compensated with faster delivered new features and bug fixes.

Correct me if I'm wrong, but one place where caches seem necessary is for
lookup.  My file system already has an inode number; my understanding
is that the kernel inode cache and dcache are caching the FUSE inode by
filename and its hashed inode number.

In FUSE, on open, I'm passed a filename which I then have to resolve into an
inode # via my own lookup.  The VFS does the path_lookup as part of sys_open,
and since I get to put private data into the struct inode, I'll generally
already have the block # and various other info in the dcache by the time
open is called.

Also, if you stuff inode data into the private fh field in fuse_file_info,
you need to be sure that any subsequent lookups always return the same
inode structure, otherwise a thread doing ftruncate vs one doing truncate
will cause issues.  So I created an internal dcache to solve those two
problems.

>  I noticed that the OMFS kernel driver supports only the USB interface and
>  the FUSE one both the network and the USB one. Isn't it possible that you
>  compared the performance using the USB with the kernel vs the much slower
>  and lower latency network with FUSE?

Nope, that's not possible, sorry.  Both require use of USB.  lkarmafs and
omfs_fuse aren't the same thing.

>  If you did use the USB interface with FUSE then what exactly do you mean by
>  "quite a bit less performance" in numbers and workloads? What you did, how
>  long it took using what CPU?

Like I said it was anecdotal (copy 20 gigs of X) in both.  I'm sure a
good portion of it is my fault, such as doing unnecessary malloc & copies
in omfs_fuse.  I have put exactly zero effort into making it fast so far.

BTW, I hardly intended to start a huge VFS vs FUSE debate.  I think FUSE
is great.  I'm not sure it's the right fit for this, is all.

-- 
Bob Copeland %% www.bobcopeland.com

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> >  > I don't have hard numbers, but anecdotally my FUSE version is quite
> >  > a bit less performant.  That's no criticism of FUSE - I just haven't
> >  > put the time into optimizing and adding various caches.
> >
> >  Thankfully you need none, it's already there by FUSE and the kernel. The
> >  trick is exactly that you can have the kernel performance and the left is
> >  moved to user space with typically negligible performance overhead which is
> >  usually well compensated with faster delivered new features and bug fixes.
> 
> Correct me if I'm wrong, but one place where caches seem necessary is for
> lookup.  My file system already has an inode number; my understanding
> is that the kernel inode cache and dcache are caching the FUSE inode by
> filename and its hashed inode number.
> 
> In FUSE, on open, I'm passed a filename which I then have to resolve into an
> inode # via my own lookup.  The VFS does the path_lookup as part of sys_open,
> and since I get to put private data into the struct inode, I'll generally
> already have the block # and various other info in the dcache by the time
> open is called.

Fuse has two different APIs.  For the "high level", path based one,
this is true.  The "low level" one is very similar to the one provided
by the VFS.

On the low level interface, when the ->lookup() method is called, it
will provide a "nodeid", which is just a per-inode unique cookie (not
the same as st_ino).  It can be a pointer to the private inode data
for example, and in the ->forget() method this private data can be
freed.

> >  If you did use the USB interface with FUSE then what exactly do you mean by
> >  "quite a bit less performance" in numbers and workloads? What you did, how
> >  long it took using what CPU?
> 
> Like I said it was anecdotal (copy 20 gigs of X) in both.  I'm sure a
> good portion of it is my fault, such as doing unnecessary malloc & copies
> in omfs_fuse.  I have put exactly zero effort into making it fast so far.
> 
> BTW, I hardly intended to start a huge VFS vs FUSE debate.  I think FUSE
> is great.  I'm not sure it's the right fit for this, is all.

And I think the VFS is great.  Undoubtedly kernel programming has it's
own charm, and I definitely don't want to scare you away from that.
Merging into mainline is a great reward, which must be erned the hard
way.  The debate is just part of that ;)

I also want to dispel any myths surrounding fuse, because those help
nobody.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Fri, Apr 18, 2008 at 10:23 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>  Fuse has two different APIs.  For the "high level", path based one,
>  this is true.  The "low level" one is very similar to the one provided
>  by the VFS.

Ah nice, I do not know how I missed that.  Thanks for pointing that out.

>  And I think the VFS is great.  Undoubtedly kernel programming has it's
>  own charm, and I definitely don't want to scare you away from that.
>  Merging into mainline is a great reward, which must be erned the hard
>  way.  The debate is just part of that ;)
>
>  I also want to dispel any myths surrounding fuse, because those help
>  nobody.

Sure.  I'll go on record saying that omfs_fuse was written over a weekend and
can not be considered a good example of anything.  I already had a
proto-libomfs that I had made for omfsck and mkomfs, and wanted to see how
easy it would be to wire that stuff up into a FUSE fs.  Pretty easily, it
turned out.

-- 
Bob Copeland %% www.bobcopeland.com

Re: [PATCH 0/7] OMFS filesystem version 3

[Miklos Szeredi]

> > There are also plans to add some sort of cache coherency protocol,
> > where the filesystem can asynchronously call back to fuse to
> > invalidate data or metadata.
> 
> Great!
> 
> I suggest adding another option (as well) where the filesystem can ask
> fuse to send it synchronous validation requests - some things require
> that.  (In my own work, the choice of A->B async invalidation and B->A
> synchronous validation is heuristic: some usage patterns benefit from
> one, some from the other.)

Yes, that makes sense.  I expect this could be done by extending the
existing requests with a flag saying the result is already cached.
And then the filesystem can either reply with a special "cached data
is valid" code, or it can reply normally with the fresh data.

> > > Otherwise I don't see how the kernel could coherently cache file pages
> > > for some kinds of FUSE filesystems.  (E.g. sshfs, for example: every
> > > operation must surely invoke a user space request or involve granting
> > > a caching right to the kernel, to keep accesses coherent with other
> > > users of the same remote files).
> > > 
> > > Ergo, either its not coherent, or there is some coherency protocol,
> > > which does require _some_ work in the user space implementation.
> > 
> > Sshfs is not coherent (but neither is NFS), it just has timeouts for
> > caches and invalidation based on modification time.
> 
> Fwiw, I think NFS version 4 is coherent (it uses leases), and older
> NFS should be coherent when you use fcntl file locks (it's not very
> efficient though).
> 
> I have been bitten a few times by timeout based caches in the past
> (NFS and SMB (pre-oplock)).  Simple things like editing a file, then
> running "ssh compiler-box make" from the editor quietly building
> incorrect code - and even subsequent make commands don't fix it.  Or
> when I edit a file, then tell someone I've changed the file - and then
> they edit the file, and my edits are lost.  Very annoying.  Nobody
> should build those kind of caches into new software.  :-)

Oh well, you can turn off caching if it bothers you :)  OTOH it would
be rather hard (and probably against the point) to try to extend the
sftp protocol to handle cache coherency.  Sshfs is not meant to be a
normal filesystem (although some people are trying to use it for home
directories and such), just a simple way to access remote files.

Miklos

Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier]

Miklos Szeredi wrote:
> > I have been bitten a few times by timeout based caches in the past
> > (NFS and SMB (pre-oplock)).  Simple things like editing a file, then
> > running "ssh compiler-box make" from the editor quietly building
> > incorrect code - and even subsequent make commands don't fix it.  Or
> > when I edit a file, then tell someone I've changed the file - and then
> > they edit the file, and my edits are lost.  Very annoying.  Nobody
> > should build those kind of caches into new software.  :-)
> 
> Oh well, you can turn off caching if it bothers you :)  OTOH it would
> be rather hard (and probably against the point) to try to extend the
> sftp protocol to handle cache coherency.  Sshfs is not meant to be a
> normal filesystem (although some people are trying to use it for home
> directories and such), just a simple way to access remote files.

Oh, I agree.  Violent agreement, they call it :-)

-- Jamie

Re: [PATCH 0/7] OMFS filesystem version 3

[Szabolcs Szakacsits]

On Fri, 18 Apr 2008, Bob Copeland wrote:
> 
> Like I said it was anecdotal (copy 20 gigs of X) in both.  I'm sure a
> good portion of it is my fault, such as doing unnecessary malloc & copies
> in omfs_fuse.  I have put exactly zero effort into making it fast so far.

I also don't advocate any solution, only interested in the FUSE myth 
busting and making it easier to use and develop for, i.e. having the best 
possible performance with zero effort.

I checked both solutions quickly, I think they are nicely written.

The kernel driver had 30 MB/s with 10% CPU usage, the FUSE version had 
6 MB/s with high I/O waiting.

The major reason seems to be that the FUSE version reads heavily from the 
block device during pure write operations, while the kernel driver never. 
There can be several non-exclusive explanations.

One of them, as you wrote, the functionality is not exactly the same, the 
FUSE one does more. If it's relevant or not, you should know. If anything 
is involved with reading from multiply places regularly then it's relevant.

Moreover when you're writing to a block device from user space then the 
size and position of the block should be page aligned, otherwise you end up 
doing unwanted synchronous reads instead of the believed asynchronous 
writes. Solving this issue for the most common cases resulted a sometimes 
over 10 fold write speedup in ntfs-3g. But of course it would be nice if 
the kernel just provided this for everybody.

Btw, mkomfs.c is missing a '#define _FILE_OFFSET_BITS 64' which is needed 
to open LFS files.

        Szaka

--
NTFS-3G:  http://ntfs-3g.org

Re: [PATCH 0/7] OMFS filesystem version 3

[Bob Copeland]

On Fri, Apr 18, 2008 at 1:35 PM, Szabolcs Szakacsits <szaka@ntfs-3g.org> wrote:
>  The major reason seems to be that the FUSE version reads heavily from the
>  block device during pure write operations, while the kernel driver never.
>  There can be several non-exclusive explanations.

Yep, I suspect switching to the lower-level API plus switching to mmap()
and getting rid of various memcpy's in omfs.c would make a huge difference.
And also drop the line count without the hash table stuff.

>  Moreover when you're writing to a block device from user space then the
>  size and position of the block should be page aligned, otherwise you end up
>  doing unwanted synchronous reads instead of the believed asynchronous
>  writes.

Cool, thanks for the tips!

-- 
Bob Copeland %% www.bobcopeland.com