How to start on new db-based FS?

How to start on new db-based FS?

[Johnny Q. Hacker]

Hello.

I'm looking to start work in the next half-year to year on a filesystem
  with LDAP and then maybe later someSQL backing.  Where's a good place
  to start?  I'm going to have a look at NFS, because I think I might
  be able to use its interface into the kernel (I'm planning on making
  it a userspace daemon).  Maybe autofs, although I think that's a
  different interface.
Any recommendations on what I should read before delving in?  Books,
  articles, web sites, code bases are all fair game.
Thanks!

-Joseph
-- 
"We're moving toward a world where all the capabilities of the Internet
 are reprocessed through a single filter, with Microsoft's business plan
 behind it."  Mozilla's Mitchell Baker, 
http://news.com.com/2100-1023-941926.html

Re: How to start on new db-based FS?

[Jose Luis Domingo Lopez]

On Friday, 26 July 2002, at 11:07:42 -0500,
Johnny Q. Hacker wrote:

> I'm looking to start work in the next half-year to year on a filesystem
>   with LDAP and then maybe later someSQL backing.  Where's a good place
>   to start?  I'm going to have a look at NFS, because I think I might
>   be able to use its interface into the kernel (I'm planning on making
>   it a userspace daemon).  Maybe autofs, although I think that's a
>   different interface.
>
Just a pointer to a project on filesystems that seems "revolutionary" in
concepts and objectives: ReiserFS 4 (http://www.namesys.com).

Please don't ask me about details, because I don't know a thing about
filesystem design and such :-), but documentation at the above site can 
give you some ideas, and maybe a place to start contributing.

Regards, 

-- 
Jose Luis Domingo Lopez
Linux Registered User #189436     Debian Linux Woody (Linux 2.4.19-pre6aa1)

Re: How to start on new db-based FS?

[Hans Reiser]

Jose Luis Domingo Lopez wrote:

>On Friday, 26 July 2002, at 11:07:42 -0500,
>Johnny Q. Hacker wrote:
>
>  
>
>>I'm looking to start work in the next half-year to year on a filesystem
>>  with LDAP and then maybe later someSQL backing.  Where's a good place
>>  to start?  I'm going to have a look at NFS, because I think I might
>>  be able to use its interface into the kernel (I'm planning on making
>>  it a userspace daemon).  Maybe autofs, although I think that's a
>>  different interface.
>>
>>    
>>
>Just a pointer to a project on filesystems that seems "revolutionary" in
>concepts and objectives: ReiserFS 4 (http://www.namesys.com).
>
>Please don't ask me about details, because I don't know a thing about
>filesystem design and such :-), but documentation at the above site can 
>give you some ideas, and maybe a place to start contributing.
>
>Regards, 
>
>  
>
We would be happy to cooperate with persons interested in implementing 
LDAP optimizing plugins for reiser4.

Probably you can get decent results compared to the competition even 
without writing additional plugins.

-- 
Hans

Re: How to start on new db-based FS?

[Daniel Mose]

Hans Reiser wrote:
> We would be happy to cooperate with persons interested in implementing 
> LDAP optimizing plugins for reiser4.
 
I'm doing a scan on the web for disk storage layout documentation on
different file systems. I have I think, downloaded just about all 
there is to download on www.namesys.com, but I fail to find anything
that does describe the reiserfs storage layout in any detail.
Is there such documentation available? 
I would be very happy for directions to it in this case.

Reason? I want to know if the root file system that I my self is 
about to develop perhaps is already implemented to some extent in
any existing root FS:s ? No need to re-invent the wheel. =)

I now know for sure that neither the JFS or the XFS does work in the 
same ways as my drafts from reading their on disk storage scheemes.

kind regards
Daniel Mose.

Re: How to start on new db-based FS?

[Hans Reiser]

Daniel Mose wrote:

>Hans Reiser wrote:
>  
>
>>We would be happy to cooperate with persons interested in implementing 
>>LDAP optimizing plugins for reiser4.
>>    
>>
> 
>I'm doing a scan on the web for disk storage layout documentation on
>different file systems. I have I think, downloaded just about all 
>there is to download on www.namesys.com, but I fail to find anything
>that does describe the reiserfs storage layout in any detail.
>Is there such documentation available? 
>I would be very happy for directions to it in this case.
>
>Reason? I want to know if the root file system that I my self is 
>about to develop perhaps is already implemented to some extent in
>any existing root FS:s ? 
>

What is a root filesystem? (I am accustomed to the term as describing 
what the OS uses for storing the semantic layer's root directory).

>No need to re-invent the wheel. =)
>
>I now know for sure that neither the JFS or the XFS does work in the 
>same ways as my drafts from reading their on disk storage scheemes.
>
>kind regards
>Daniel Mose.
>
>
>  
>


-- 
Hans

Re: How to start on new db-based FS?

[Matthew D. Pitts]

Hans,

I think Daniel is referring to filesystems in general, when he uses "root"

Matthew
----- Original Message -----
From: "Hans Reiser" <reiser@namesys.com>
To: "Daniel Mose" <imcol@unicyclist.com>
Cc: "Jose Luis Domingo Lopez" <linux-kernel@24x7linux.org.gilby.com>;
<linux-kernel@vger.kernel.org>
Sent: Saturday, July 27, 2002 9:45 PM
Subject: Re: How to start on new db-based FS?


> Daniel Mose wrote:
>
> >Hans Reiser wrote:
> >
> >
> >>We would be happy to cooperate with persons interested in implementing
> >>LDAP optimizing plugins for reiser4.
> >>
> >>
> >
> >I'm doing a scan on the web for disk storage layout documentation on
> >different file systems. I have I think, downloaded just about all
> >there is to download on www.namesys.com, but I fail to find anything
> >that does describe the reiserfs storage layout in any detail.
> >Is there such documentation available?
> >I would be very happy for directions to it in this case.
> >
> >Reason? I want to know if the root file system that I my self is
> >about to develop perhaps is already implemented to some extent in
> >any existing root FS:s ?
> >
>
> What is a root filesystem? (I am accustomed to the term as describing
> what the OS uses for storing the semantic layer's root directory).
>
> >No need to re-invent the wheel. =)
> >
> >I now know for sure that neither the JFS or the XFS does work in the
> >same ways as my drafts from reading their on disk storage scheemes.
> >
> >kind regards
> >Daniel Mose.
> >
> >
> >
> >
>
>
> --
> Hans
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: How to start on new db-based FS?

[Daniel Mose]

Matthew D. Pitts wrote:
> Hans,
> 
> I think Daniel is referring to filesystems in general, when he uses "root"
> 
> Matthew

Well, I think that this would depend on what 
you refer to with "file systems" in general ;)

kind regards 
Daniel Mose

Re: How to start on new db-based FS?

[Daniel Mose]

Hans Reiser wrote:
> Daniel Mose wrote:
> >I'm doing a scan on the web for disk storage layout documentation on
> >different file systems. I have I think, downloaded just about all 
> >there is to download on www.namesys.com, but I fail to find anything
> >that does describe the reiserfs storage layout in any detail.
> >Is there such documentation available? 

I do believe that I've been somewhat unclear. What I would hope 
to find is :
Documentation of On-disk Per-partition storage layout 
of file lookup and maintainance structures for reiserfs. 

> >I would be very happy for directions to it in this case.
> >
> >Reason? I want to know if the root file system that I my self is 
                                 ^^^^^^^^^^^^^^^^
Correction, Should be: File system for root mountable partitions. 

> >about to develop perhaps is already implemented to some extent in
> >any existing root FS:s ? 
                ^^^^^^^^^ 
same correction here.
> 
> What is a root filesystem? (I am accustomed to the term as describing 
> what the OS uses for storing the semantic layer's root directory).

I do not have a CS - convention, or Unix background, so I am a bit
unfamiliar with terms like "semantic layer's root directory" 
I am a "learn by doing" type of guy, and up to now it has worked out
ok.

There seems to be a lot of discrepancies when it comes to 
defining what a "file system" really is. I often find that the 
term is used with widely different meanings, depending on who 
you are actually talking to, and also in what context it is used.  

I believe that the developers at www.oss.sgi, as well as the JFS 
developers hosted at IBM, use the term: "root file system" to make 
clear that 

A.	It is a Unix type of file system, that can be used as "/"
	at boot, i e a system "magically" mountable root partition.
	
B.	The file system occupies a Local mountable disk partition, 
	and is acessible via the  Unix mount command.

C.	The file system is used for file access, and maintainance of 
	On-disk	file-lookup hierarchies, such as one or more 
	Superblocks, File allocation scheemes ( f ex allocation group 
	descriptors,block and inode bitmap tables, as well as inodes )
	and Directory files ( ext2fs or FFS )

I hope that this makes a bit more sense to you.
Thank you for taking time with me, all the same

I'm happy for any further direction.

kind regards 
Daniel Mose

Re: How to start on new db-based FS?

[Hans Reiser]

Daniel Mose wrote:

>Hans Reiser wrote:
>  
>
>>Daniel Mose wrote:
>>    
>>
>>>I'm doing a scan on the web for disk storage layout documentation on
>>>different file systems. I have I think, downloaded just about all 
>>>there is to download on www.namesys.com, but I fail to find anything
>>>that does describe the reiserfs storage layout in any detail.
>>>Is there such documentation available? 
>>>      
>>>
>
>I do believe that I've been somewhat unclear. What I would hope 
>to find is :
>Documentation of On-disk Per-partition storage layout 
>of file lookup and maintainance structures for reiserfs. 
>
>  
>
>>>I would be very happy for directions to it in this case.
>>>
>>>Reason? I want to know if the root file system that I my self is 
>>>      
>>>
>                                 ^^^^^^^^^^^^^^^^
>Correction, Should be: File system for root mountable partitions. 
>
>  
>
>>>about to develop perhaps is already implemented to some extent in
>>>any existing root FS:s ? 
>>>      
>>>
>                ^^^^^^^^^ 
>same correction here.
>  
>
>>What is a root filesystem? (I am accustomed to the term as describing 
>>what the OS uses for storing the semantic layer's root directory).
>>    
>>
>
>I do not have a CS - convention, or Unix background, so I am a bit
>unfamiliar with terms like "semantic layer's root directory" 
>I am a "learn by doing" type of guy, and up to now it has worked out
>ok.
>
>There seems to be a lot of discrepancies when it comes to 
>defining what a "file system" really is. I often find that the 
>term is used with widely different meanings, depending on who 
>you are actually talking to, and also in what context it is used.  
>
>I believe that the developers at www.oss.sgi, as well as the JFS 
>developers hosted at IBM, use the term: "root file system" to make 
>clear that 
>
>A.	It is a Unix type of file system, that can be used as "/"
>	at boot, i e a system "magically" mountable root partition.
>	
>B.	The file system occupies a Local mountable disk partition, 
>	and is acessible via the  Unix mount command.
>
>C.	The file system is used for file access, and maintainance of 
>	On-disk	file-lookup hierarchies, such as one or more 
>	Superblocks, File allocation scheemes ( f ex allocation group 
>	descriptors,block and inode bitmap tables, as well as inodes )
>	and Directory files ( ext2fs or FFS )
>
>I hope that this makes a bit more sense to you.
>Thank you for taking time with me, all the same
>
>I'm happy for any further direction.
>
>kind regards 
>Daniel Mose
>
>
>  
>
We are gradually adding this to our web page, and what is not present 
there can be found by reading the source code which is quite readable 
for reiser4. Look in the node plugin, disk format plugin, and item 
plugin files/directories. These have some very long winded comments 
which are not yet on our web page.;-) green@namesys.com will send you a 
tarball of the source code if you ask for it.

-- 
Hans

Re: How to start on new db-based FS?

[Christian Jaeger]

There's always the possibility to build upon AVFS 
(http://www.inf.bme.hu/~mszeredi/avfs), either using the LD_PRELOAD 
version improved by Frederik Eaton, or the version using the (CODA or 
better the) FUSE kernel module (the latter is still in development 
from what I hear).

We have built a DB backed filesystem for use in a content management 
system (http://www.ethlife.ethz.ch/newcms), based on the AVFS 
LD_PRELOAD version, and embedding perl so we could write the actual 
filesystem implementation in perl. It has required quite a bit of 
labour, but works well for our purpose now. It's GPL'd, but not yet 
polished for publication, so contact me if you're interested.

Christian.
-- 
Christian Jaeger  Programmer & System Engineer
ETHLife CMS Project - www.ethlife.ethz.ch/newcms - www.ethlife.ethz.ch