From mboxrd@z Thu Jan 1 00:00:00 1970 From: Livio Baldini Soares Subject: Re: Ext2 detailed documentation Date: Fri, 4 Jul 2003 21:31:28 -0300 Sender: linux-fsdevel-owner@vger.kernel.org Message-ID: <20030705003128.GA14192@ime.usp.br> References: <000701c34283$743fe7c0$01000001@southernstar> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: linux-fsdevel@vger.kernel.org Return-path: Received: from birosca.ime.usp.br ([143.107.45.59]:1953 "HELO birosca.ime.usp.br") by vger.kernel.org with SMTP id S266237AbTGEARq (ORCPT ); Fri, 4 Jul 2003 20:17:46 -0400 To: James Buchanan Content-Disposition: inline In-Reply-To: <000701c34283$743fe7c0$01000001@southernstar> List-Id: linux-fsdevel.vger.kernel.org Hey James, James Buchanan writes: > Aside from reading the source code (and fully understanding it, which > I am not sure I am capable of yet), is there any detailed > documentation on how the Ext2 fs works? Specifically, I'd like to > know some fairly high-level details so I know what to look for when > I'm studying the sources. > > Here's what I'm looking for: where's the code that gets blocks of > data off the disk and sends it up through the kernel via the Ext2 fs > code? And vice versa. I see a lot of code dealing with pages, does > this mean the ext2 code sends the blocks into kernel pages where the > kernel retrieves it and makes it available to system calls? > > I am hoping that a detailed primer on how the ext2 works will give me > some better background for studying the source code and understanding > what's happening. I'd like to get to a stage where I can modify the > ext2fs code comfortably. By the types of questions you're asking, it seems to me that you need to learn first, how filesystems interact with VFS, virtual memory and the low-level block layer in Linux. After you have some feeling of how that works (which is common to most on-disk filesystems), you can start worrying about ext2 specifics. The best documentation I've found on this is the book "Understanding the Linux Kernel", by D. P. Bovet & M. Cesati. The newest edition is the second, which is suppose to cover kernel 2.4. I've read over the first edition, and it's really good. Topics like VM, the block layer, VFS and, in particular, the ext2 file system, are all covered in the book. Aside from that, the kernelnewbies site (www.kernelnewbies.org), specifically their "External Links" page (www.kernelnewbies.org/links/) has a valuable list of documentation sites, some even regarding filesystems in general, and the ext2fs. > I've read the design and implementation docs on the e2fsprogs sf site, > and the kernel source doc on ext2 which lists that as a reference. > There is a reference to a site which has some code to compress data on > ext2 filesystems, and that would be really ideal (see below) but > apparently it's no longer maintained or supported. > > What I want to do as a research project is implement compression. > Essentially the code has to compress and decompress on the fly. I > would imagine compression would be limited to each block individually, > otherwise data on other blocks would be needed to decompress a single > block. So I need to hook into the part of ext2 where the data flies > through on its journey back and forth from the kernel to the physical > disk. I'd also like to implement secure deletion by wiping blocks > when they're freed so recovery is impossible, and other experiments. Just as a note, there are some other compressed file systems for Linux floating around.... For example: E2compress project: http://www.alizt.com/index.html SquashFS: http://squashfs.sourceforge.net/ CramFS: http://sourceforge.net/projects/cramfs/ And perhaps others... Google around a bit. The advantage of looking at these other projects, is that you learn a bit of what they have done, and why, etc. > PS. How do fs developers debug their code? Do they use syslog, or > can they somehow run the fs code in debugger like gdb? Personally, printing stuff to syslog is not very efficient, in most cases. If you really intend to do kernel hacking, you should get acquainted with some virtual machines. The order of (my) preference is: User mode Linux: http://user-mode-linux.sf.net/ Bochs Emulator: http://bochs.sf.net/ There are, of course, some kernel debuggers, like kgdb (kgdb.sourceforge.net/) and kdb (oss.sgi.com/projects/kdb/), but I find that they are only useful when your work is mature and you are ready to do some testing on real machines. good luck, -- Livio B. Soares