Concise description of disk layout?

Concise description of disk layout?

[Frank Mayhar]

Does such a thing exist anywhere?  I.e. "inodes are here, data blocks
are here, etc., etc."  I need to come up with something like this but if
one already exists that would save a lot of time.
-- 
Frank Mayhar <fmayhar@google.com>
Google, Inc.

Re: Concise description of disk layout?

[Theodore Tso]

On Wed, Apr 15, 2009 at 04:33:12PM -0700, Frank Mayhar wrote:
> Does such a thing exist anywhere?  I.e. "inodes are here, data blocks
> are here, etc., etc."  I need to come up with something like this but if
> one already exists that would save a lot of time.

At what level?  You mean something like this:

   http://e2fsprogs.sourceforge.net/ext2intro.html

but updated for ext4?

Hmm, not really.  Fragments of what's there can be found in various
slide decks and papers, here:

      http://ext4.wiki.kernel.org/index.php/Publications

If you'd like to write up something comprehensive for the ext4 wiki,
that would be great.  :-)

					- Ted

Re: Concise description of disk layout?

[Eric Sandeen]

Theodore Tso wrote:
> On Wed, Apr 15, 2009 at 04:33:12PM -0700, Frank Mayhar wrote:
>> Does such a thing exist anywhere?  I.e. "inodes are here, data blocks
>> are here, etc., etc."  I need to come up with something like this but if
>> one already exists that would save a lot of time.
> 
> At what level?  You mean something like this:
> 
>    http://e2fsprogs.sourceforge.net/ext2intro.html
> 
> but updated for ext4?

Or do you mean like some XML DTD for a generic description?   That's the
first thing that came to mind when I read it.

or ... an e2image actually *captures* all of that information about a
filesystem... depends what you're looking for I suppose?

-Eric

Re: Concise description of disk layout?

[Frank Mayhar]

Sorry for the late reply.

On Wed, 15 Apr 2009 20:28:09 -0400, Theodore Tso wrote:
> On Wed, Apr 15, 2009 at 04:33:12PM -0700, Frank Mayhar wrote:
>> Does such a thing exist anywhere?  I.e. "inodes are here, data blocks
>> are here, etc., etc."  I need to come up with something like this but
>> if one already exists that would save a lot of time.
> 
> At what level?  You mean something like this:
> 
>    http://e2fsprogs.sourceforge.net/ext2intro.html
> 
> but updated for ext4?

Yeah, the "Physical Structure" section.

> Hmm, not really.  Fragments of what's there can be found in various
> slide decks and papers, here:
> 
>       http://ext4.wiki.kernel.org/index.php/Publications

Sigh.  Okay, I'll dig through those.

> If you'd like to write up something comprehensive for the ext4 wiki,
> that would be great.  :-)

I'm going to have to come up with something, although it probably won't 
be comprehensive.  With luck I'll have time to make it suitable for 
public consumption. :-)
-- 
Frank Mayhar
fmayhar@google.com

Re: Concise description of disk layout?

[Frank Mayhar]

On Fri, 2009-04-17 at 18:30 +0000, Frank Mayhar wrote:
> Sorry for the late reply.
> 
> On Wed, 15 Apr 2009 20:28:09 -0400, Theodore Tso wrote:
> >    http://e2fsprogs.sourceforge.net/ext2intro.html
> > 
> > but updated for ext4?
> 
> Yeah, the "Physical Structure" section.

I've read through some of the documents you pointed to and it sounds
like the actual on-disk format hasn't changed significantly, i.e. block
groups are laid out in basically the same way and the stuff within them
is laid out in more or less the same way as in ext2/ext3; the major
difference seems to be the allocation strategies embedded in the code
itself.  The new strategies simply mean that files and inodes are
allocated more efficiently but not really in new places.  Is this (more
or less) correct or am I totally off-base?
-- 
Frank Mayhar <fmayhar@google.com>
Google, Inc.

Re: Concise description of disk layout?

[Theodore Tso]

On Fri, Apr 17, 2009 at 11:49:26AM -0700, Frank Mayhar wrote:
> On Fri, 2009-04-17 at 18:30 +0000, Frank Mayhar wrote:
> > Sorry for the late reply.
> > 
> > On Wed, 15 Apr 2009 20:28:09 -0400, Theodore Tso wrote:
> > >    http://e2fsprogs.sourceforge.net/ext2intro.html
> > > 
> > > but updated for ext4?
> > 
> > Yeah, the "Physical Structure" section.
> 
> I've read through some of the documents you pointed to and it sounds
> like the actual on-disk format hasn't changed significantly, i.e. block
> groups are laid out in basically the same way and the stuff within them
> is laid out in more or less the same way as in ext2/ext3; the major
> difference seems to be the allocation strategies embedded in the code
> itself.  The new strategies simply mean that files and inodes are
> allocated more efficiently but not really in new places.  Is this (more
> or less) correct or am I totally off-base?

The biggest structural change would be the extent allocation trees,
and the flex_bg changes, which move the block and inode bitmaps and
the inode table out of their individual block groups, and group them
together into larger units.  Take a a look at the dumpe2fs output of a
freshly created ext4 filesystem, and that should become clear.  

On the whole, though, you're correct; the overall on-disk structure
really hasn't changed that much.  Inodes have gotten bigger, and for
64-bit block-sized filesystems, the block group descriptors will also
get bigger, and of course the ext2 description doesn't talk about
where extended attributes are stored, or the htree directory
structures.  But the changes in ext2/3/4 have always been
evolutionary.

Regards,

					- Ted