From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bedros Hanounik <2bedros@gmail.com> Subject: Re: reiser4 plugins Date: Tue, 21 Jun 2005 22:11:13 -0700 Message-ID: References: <20050620235458.5b437274.akpm@osdl.org> <42B831B4.9020603@pobox.com> <42B87318.80607@namesys.com> <20050621202448.GB30182@infradead.org> <42B8B9EE.7020002@namesys.com> <42B8BB5E.8090008@pobox.com> <42B8E834.5030809@slaphack.com> Reply-To: Bedros Hanounik <2bedros@gmail.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_1313_19082218.1119417073125" Return-path: list-help: list-unsubscribe: list-post: Errors-To: flx@namesys.com In-Reply-To: <42B8E834.5030809@slaphack.com> List-Id: To: David Masover Cc: Jeff Garzik , Hans Reiser , Christoph Hellwig , Andrew Morton , linux-kernel@vger.kernel.org, ReiserFS List ------=_Part_1313_19082218.1119417073125 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline First of all, I'm HW engineer, and don't know much about implementation=20 details of FS. I know some coding, I've coded for different levels from=20 linux device drivers to GUI's, but I know a lot about integration between= =20 levels of hierarchy. I generally agree with the idea that high level functionality such as=20 compressing, encrypting, ...etc should not be implemented in the FS. But,= =20 the term "high-level" is a relative term and three years down the road, wha= t=20 we consider high-level now becomes low-level. my suggestion is to keep everything as is. if Hans and the reiser4 guys wan= t=20 to implement high level stuff as plug-ins that's fine. If the VFS guys want= =20 to implement the same functionality in VFS on top of the system FS; they ca= n=20 do it too. BUT PLEASE MAKE THE API COMPATIBLE!!!! programmers on both sides should discuss the features and make sure the API= =20 is compatible. in such a scenario, if I want to compress a directory called /mystuff/data= =20 on a reiser4 FS, I could call the VFS API to do so; The VFS finds that the= =20 system is reiser4 and compression is supported, so it passes the call to=20 reiser4. On another hand, if the system is ext3 and it doens't support=20 compression, VFS performs the task and passes the file to ext3. My point is that the effect of gravity is upon us, high level stuff will=20 become low-level in a short time. Moreover, at the low level in FS and=20 especially when done by FS guys, functions such as compression can be done= =20 more efficiently than in VFS space. my worries are that the waste of effort by implementing the same=20 functionality in two levels in the hierarchy, and incompatible API can caus= e=20 serious headache. just my two cents. -Bedros On 6/21/05, David Masover wrote: >=20 > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > Jeff Garzik wrote: > > Hans Reiser wrote: > > > >> Christoph, > >> > >> Reiser4 users love the plugin concept, and all audiences which have > >> listened to a presentation on plugins have been quite positive about > >> it. Many users think it is the best thing about reiser4. Can you > >> articulate why you are opposed to plugins in more detail? Perhaps you > >> are simply not as familiar with it as the audiences I have presented > >> to. Perhaps persons on our mailing list can comment..... > >> > >> In particular, what is wrong with having a plugin id associated with > >> every file, storing the pluginid on disk in permanent storage in the > >> stat data, and having that plugin id define the set of methods that > >> implement the vfs operations associated with a particular file, rather > >> than defining VFS methods only at filesystem granularity? > > > > > > You're basically implementing another VFS layer inside of reiser4, whic= h > > is a big layering violation. >=20 > There's been sloppy code in the kernel before. I remember one bit in > particular which was commented "Fuck me gently with a chainsaw." If I > remember correctly, this had all of the PCI ids and the names and > manufacturers of the corresponding devices -- in a data structure -- in > C source code. It was something like one massive array definition, or > maybe it was a structure. I don't remember exactly, but... >=20 > The point is, this was in the kernel for quite awhile, and it was so > ugly that someone would rather be fucked with a chainsaw. If something > that bad can make it in the kernel and stay for awhile because it > worked, and no one wanted to replace it -- nowdays, that database is > kept in userland as some nice text format -- then I vote for putting > Reiser4 in the kernel now, and ironing the sloppiness ("violation") out > later. It runs now. >=20 > > This sort of feature should -not- be done at the low-level filesystem > > level. >=20 > I agree there, too. In fact, some people have suggested that all > "legacy" (read: non-reiser) filesystems should be implemented as Reiser4 > plugins, effectively killing VFS.* >=20 > So, Reiser4 may eventually take over VFS and be the only Linux > filesystem, but if so, it will have to do it much more slowly. Take the > good ideas -- things like plugins -- and make them at least look like > incremental updates to the current VFS, and make them available to all > filesystems. >=20 > Eventually, this would result in a full merge of Reiser and Linux, such > that the only thing left of "Reiser4" are a few plugins -- things like > storage plugins and maybe some more exotic things like fibration that I > don't really understand. >=20 > > What happens if people decide plugins are a good idea, and they want > > them in ext3? We need massive surgery to extract the guts from reiser4. >=20 > And here is the crucial point. Reiser4 is usable and useful NOW, not > after it has undergone massive surgery, and Namesys is bankrupt, and > users have given up and moved on to XFS. But the massive surgery should > happen eventually, partly to make all filesystems better (see below), > and partly to make the transition easier and more palatable for those > who don't work for Namesys. >=20 >=20 >=20 >=20 >=20 > * Imagine ext3 as a storage-level plugin for reiser4. You get one > benefit immediately -- lazy allocation. Lazy allocation is nice both > for fragmentation and for busy systems writing and nuking a lot of > temporary files. Imagine a file which is written and then deleted > before it hits disk -- it should never touch the disk, regardless of the > underlying storage layout. >=20 > Other benefits are subtler but inevitable. Right now, it would be an > understatement to say that there's duplication of effort between ext3, > xfs, and reiser4. And yet, I don't think there are many core design > decisions that influence my decision as to which I pick up. I just want > it to be fast, stable, and have a stable on-disk format so I don't have > to reformat too often. I honestly don't care about how XFS is > segmenting the disk, or Reiser4 packs really well, or ext3 can be read > as ext2 and flushes to disk every five seconds. These are the kinds of > things which should be set to sane defaults, tunable for enterprise > users, but are not really enough to warrant completely separate code=20 > bases. >=20 > I would imagine that it wouldn't be too long after this before an > uber-fs rose, something which combined enough of the strong points of > all the existing Linux filesystems that no one would use anything else. > But, Linux still needs support for FAT32, ISO9660, UDF, and all the > other filesystems which won't go away as easily as XFS and ext3, and it > would be nice if these could all share as much code as possible. >=20 >=20 > I don't know if storage plugins really work that way, but they should. >=20 > I think. I don't work here. >=20 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.1 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org >=20 > iQIVAwUBQrjoNHgHNmZLgCUhAQIYYw/7BWZ0gVvy0ln5tRo405yUoRHJ/jVFBGyP > 5pQ610ECMZORVWRO1qP/NXbvGZwDjEggM5iIeahsGqnBWzNGDsB6RslMUZAxzCYy > iAovi0881zoARf3dmhKrDgbGkvNLPTx+/ypa20oRcHLnyI+NAjerUxNuvGc79PJn > Ybm9JhX6tQsqGIKjZye9uNDHCifLqzA1gdxucPwWo9sz4ymzM9FgsMGvb+IxrU50 > 2a2G2K6AHcH+nkomEHw3xgY3PmUZUy0s2hQDSkJx6dCido7fGZwwykaSXm4IZs9s > xZqBxKx32G/LDnDV3W8gpj8ZisUE+58kefRbIjo4Ml6IzgXvQqpRjaQOuz8JoKDX > 9KUV43tcZkPpK+neIYPQYpXCrdMQqB5+ISpbNKuz/Z/abkcVw1042sy0EKM+/VnD > n3Jr7PpSyk0lfCyADr+33qnqPQRFq02gQTohg9FheoMthhV01aaeGW5ExmTM/Wwa > 6Dmv/qnn2oNImi+Ebz5u3Lbnz7lL3MVpRL4aoMmEOVUB3xhehnxesf//yacBmwj9 > M/3KVae9epwX4K6yi8qQXzH4160IBFHpWUxBLc9LnOZlHQuZ+Fz3BIrbKQBvmaHT > 7lrwi0Os6TmiBGMSFd6OUWHcYs4p97aMw30NG33fCL6e8X6oNVFwwnJXZpBPN1Mr > +lrsVAywKEI=3D > =3DrHdK > -----END PGP SIGNATURE----- > ------=_Part_1313_19082218.1119417073125 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline First of all, I'm HW engineer, and don't know much about implementation details of FS. I know some coding, I've coded for different levels from linux device drivers to GUI's, but I know a lot about integration between levels of hierarchy.

I generally agree with the idea that high level functionality such as compressing, encrypting, ...etc should not be implemented in the FS. But, the term "high-level" is a relative term and three years dow= n the road, what we consider high-level now becomes low-level.

my suggestion is to keep everything as is. if Hans and the reiser4 guys want to implement high level stuff as plug-ins that's fine. If the VFS guys want to implement the same functionality in VFS on top of the system FS; they can do it too.

BUT PLEASE MAKE THE API COMPATIBLE!!!!

programmers on both sides should discuss the features and make sure the API= is compatible.

in such a scenario, if I want to compress a directory called /mystuff/data on a reiser4 FS, I could call the VFS API to do so; The VFS finds that the system is reiser4 and compression is supported, so it passes the call to reiser4. On another hand, if the system is ext3 and it doens't support compression, VFS performs the task and passes the file to ext3.

My point is that  the effect of gravity is upon us, high level stuff will become low-level in a short time. Moreover, at the low level in FS and especially when done by FS guys, functions such as compression can be done more efficiently than in VFS space.

my  worries are that the waste of effort by implementing the same functionality in two levels in the hierarchy, and incompatible API can cause serious headache.

just my two cents.

-Bedros



On 6/21/05, David Masover <ninja@s= laphack.com> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jeff Garzik wrote:<= br>> Hans Reiser wrote:
>
>> Christoph,
>>
&g= t;> Reiser4 users love the plugin concept, and all audiences which have
>> listened to a presentation on plugins have been quite positive= about
>> it.  Many users think it is the best thing abo= ut reiser4.  Can you
>> articulate why you are opposed t= o plugins in more detail?  Perhaps you
>> are simply not as familiar with it as the audiences I have pre= sented
>> to.  Perhaps persons on our mailing list can c= omment.....
>>
>> In particular, what is wrong with havin= g a plugin id associated with
>> every file, storing the pluginid on disk in permanent storage = in the
>> stat data, and having that plugin id define the set of m= ethods that
>> implement the vfs operations associated with a part= icular file, rather
>> than defining VFS methods only at filesystem granularity?
&= gt;
>
> You're basically implementing another VFS layer inside = of reiser4, which
> is a big layering violation.

There's been = sloppy code in the kernel before.  I remember one bit in
particular which was commented "Fuck me gently with a chainsaw.&qu= ot;  If I
remember correctly, this had all of the PCI ids and = the names and
manufacturers of the corresponding devices -- in a data st= ructure -- in
C source code.  It was something like one massive array defin= ition, or
maybe it was a structure.  I don't remember exactly,= but...

The point is, this was in the kernel for quite awhile, and i= t was so
ugly that someone would rather be fucked with a chainsaw. =  If something
that bad can make it in the kernel and stay for awhile because it
wo= rked, and no one wanted to replace it -- nowdays, that database is
kept = in userland as some nice text format -- then I vote for putting
Reiser4 = in the kernel now, and ironing the sloppiness ("violation") out
later.  It runs now.

> This sort of feature should = -not- be done at the low-level filesystem
> level.

I agree the= re, too.  In fact, some people have suggested that all
"l= egacy" (read: non-reiser) filesystems should be implemented as Reiser4
plugins, effectively killing VFS.*

So, Reiser4 may eventually ta= ke over VFS and be the only Linux
filesystem, but if so, it will have to= do it much more slowly.  Take the
good ideas -- things like p= lugins -- and make them at least look like
incremental updates to the current VFS, and make them available to all<= br>filesystems.

Eventually, this would result in a full merge of Rei= ser and Linux, such
that the only thing left of "Reiser4" are = a few plugins -- things like
storage plugins and maybe some more exotic things like fibration that I=
don't really understand.

> What happens if people decide plug= ins are a good idea, and they want
> them in ext3?  We need= massive surgery to extract the guts from reiser4.

And here is the crucial point.  Reiser4 is usable and use= ful NOW, not
after it has undergone massive surgery, and Namesys is bank= rupt, and
users have given up and moved on to XFS.  But the ma= ssive surgery should
happen eventually, partly to make all filesystems better (see below),and partly to make the transition easier and more palatable for those
= who don't work for Namesys.





* Imagine ext3 as a stor= age-level plugin for reiser4.  You get one
benefit immediately -- lazy allocation.  Lazy allocation is n= ice both
for fragmentation and for busy systems writing and nuking a lot= of
temporary files.  Imagine a file which is written and then= deleted
before it hits disk -- it should never touch the disk, regardle= ss of the
underlying storage layout.

Other benefits are subtler but inevit= able.  Right now, it would be an
understatement to say that th= ere's duplication of effort between ext3,
xfs, and reiser4.  A= nd yet, I don't think there are many core design
decisions that influence my decision as to which I pick up.  = I just want
it to be fast, stable, and have a stable on-disk format so I= don't have
to reformat too often.  I honestly don't care abou= t how XFS is
segmenting the disk, or Reiser4 packs really well, or ext3 can be read
a= s ext2 and flushes to disk every five seconds.  These are the kin= ds of
things which should be set to sane defaults, tunable for enterpris= e
users, but are not really enough to warrant completely separate code b= ases.

I would imagine that it wouldn't be too long after this before anuber-fs rose, something which combined enough of the strong points of
= all the existing Linux filesystems that no one would use anything else.
But, Linux still needs support for FAT32, ISO9660, UDF, and all theother filesystems which won't go away as easily as XFS and ext3, and itwould be nice if these could all share as much code as possible.


I don't know if storage plugins really work that way, but they should.<= br>
I think.  I don't work here.

-----BEGIN PGP SIGNATU= RE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with T= hunderbird -=20 http://enigmail.mozdev.org
iQIVAwUBQrjoNHgHNmZLgCUhAQIYYw/7BWZ0gVvy0ln5tRo405yUoRHJ/jVFBGyP
5pQ61= 0ECMZORVWRO1qP/NXbvGZwDjEggM5iIeahsGqnBWzNGDsB6RslMUZAxzCYy
iAovi0881zoA= Rf3dmhKrDgbGkvNLPTx+/ypa20oRcHLnyI+NAjerUxNuvGc79PJn
Ybm9JhX6tQsqGIKjZye9uNDHCifLqzA1gdxucPwWo9sz4ymzM9FgsMGvb+IxrU50
2a2= G2K6AHcH+nkomEHw3xgY3PmUZUy0s2hQDSkJx6dCido7fGZwwykaSXm4IZs9s
xZqBxKx32G= /LDnDV3W8gpj8ZisUE+58kefRbIjo4Ml6IzgXvQqpRjaQOuz8JoKDX
9KUV43tcZkPpK+neI= YPQYpXCrdMQqB5+ISpbNKuz/Z/abkcVw1042sy0EKM+/VnD
n3Jr7PpSyk0lfCyADr+33qnqPQRFq02gQTohg9FheoMthhV01aaeGW5ExmTM/Wwa
6Dm= v/qnn2oNImi+Ebz5u3Lbnz7lL3MVpRL4aoMmEOVUB3xhehnxesf//yacBmwj9
M/3KVae9ep= wX4K6yi8qQXzH4160IBFHpWUxBLc9LnOZlHQuZ+Fz3BIrbKQBvmaHT
7lrwi0Os6TmiBGMSF= d6OUWHcYs4p97aMw30NG33fCL6e8X6oNVFwwnJXZpBPN1Mr
+lrsVAywKEI=3D
=3DrHdK
-----END PGP SIGNATURE-----

------=_Part_1313_19082218.1119417073125--