Upgrading datastructures between different filesystem versions

Upgrading datastructures between different filesystem versions

[kernel learner]

[-- Attachment #1: Type: text/plain, Size: 372 bytes --]

Hi,

ext3 filesystem has 32-bit block address and ext4 filesystem has 48-bit
block address. If a user installs ext4, how will the file system handle
already existing block with 32 bit values? Can anyone point me to the
correct pointer for this backward compatibility stuff? I searched for it but
cudn't find much info. Is the work still pending on this front?

Thanks,
KL

[-- Attachment #2: Type: text/html, Size: 397 bytes --]

Re: Upgrading datastructures between different filesystem versions

[Jim Cromie]

kernel learner wrote:
> Hi,
>
> ext3 filesystem has 32-bit block address and ext4 filesystem has 
> 48-bit block address. If a user installs ext4, how will the file 
> system handle already existing block with 32 bit values? 
Why should it ?
thats what ext3 is for.
your kernel can have both FS's supported, and would use ext3 drivers for 
ext3
filesystems its asked to mount.

Id expect ext4 drivers handling ext3 filesystems is a distant, secondary 
goal
to getting a fast, reliable, clean 48bit filesystem working.

> Can anyone point me to the correct pointer for this backward 
> compatibility stuff? I searched for it but cudn't find much info. Is 
> the work still pending on this front?
>
> Thanks,
> KL

Re: Upgrading datastructures between different filesystem versions

[Andreas Dilger]

On Sep 25, 2007  23:40 -0600, Jim Cromie wrote:
> kernel learner wrote:
> >ext3 filesystem has 32-bit block address and ext4 filesystem has 
> >48-bit block address. If a user installs ext4, how will the file 
> >system handle already existing block with 32 bit values? 
>
> Why should it ? thats what ext3 is for.

Bzzt. Wrong answer.  The ext4 code will be able to read existing ext3
(and ext2) filesystems just fine.  Otherwise there wouldn't be much
of an upgrade path.

> Id expect ext4 drivers handling ext3 filesystems is a distant, secondary 
> goal to getting a fast, reliable, clean 48bit filesystem working.

Far from the truth.  One of the main goals of ext4 is that it is a drop-in
replacement for ext3.  The code is mostly incremental improvements over
ext3, and that IS one of the reasons that it is reliable.  We didn't throw
away 10 years of bug fixes in the ext2/ext3 code when adding the ext4
features.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: Upgrading datastructures between different filesystem versions

[Sachin Gaikwad]

On 9/26/07, Andreas Dilger <adilger@clusterfs.com> wrote:
> On Sep 25, 2007  23:40 -0600, Jim Cromie wrote:
> > kernel learner wrote:
> > >ext3 filesystem has 32-bit block address and ext4 filesystem has
> > >48-bit block address. If a user installs ext4, how will the file
> > >system handle already existing block with 32 bit values?
> >
> > Why should it ? thats what ext3 is for.
>
> Bzzt. Wrong answer.  The ext4 code will be able to read existing ext3
> (and ext2) filesystems just fine.  Otherwise there wouldn't be much
> of an upgrade path.
>
> > Id expect ext4 drivers handling ext3 filesystems is a distant, secondary
> > goal to getting a fast, reliable, clean 48bit filesystem working.
>
> Far from the truth.  One of the main goals of ext4 is that it is a drop-in
> replacement for ext3.  The code is mostly incremental improvements over
> ext3, and that IS one of the reasons that it is reliable.  We didn't throw
> away 10 years of bug fixes in the ext2/ext3 code when adding the ext4
> features.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.
>
>
> --
> To unsubscribe from this list: send an email with
> "unsubscribe kernelnewbies" to ecartis@nl.linux.org
> Please read the FAQ at http://kernelnewbies.org/FAQ
>
>

Is it not the case that VFS takes care of all filesystems available ?
VFS will see if a particular file belongs to ext3 or ext4 and call
that FS's drivers to access information ??

Correct me if I am wrong. I am a newbie!

Sachin

Re: Upgrading datastructures between different filesystem versions

[Theodore Tso]

On Wed, Sep 26, 2007 at 06:29:19PM -0500, Sachin Gaikwad wrote:
> Is it not the case that VFS takes care of all filesystems available ?
> VFS will see if a particular file belongs to ext3 or ext4 and call
> that FS's drivers to access information ??

No, it doesn't quite work that way.  You have to mount a particular
partition using a specific filesystem (i.e., ntfs, vfat, ext2, ext3,
ext4, etc.).  A partition formatted using ext2 can be mounted using
the ext2, ext3, or ext4 filesystem driver.  You can explicitly specify
what filesystem should be used to mount a particuar partition using
the -t option to the mount program, or by specifying a particular
filesystem type in the /etc/fstab file.

						- Ted

Re: Upgrading datastructures between different filesystem versions

[Erik Mouw]

[-- Attachment #1: Type: text/plain, Size: 1324 bytes --]

On Wed, Sep 26, 2007 at 06:29:19PM -0500, Sachin Gaikwad wrote:
> Is it not the case that VFS takes care of all filesystems available ?
> VFS will see if a particular file belongs to ext3 or ext4 and call
> that FS's drivers to access information ??

No, the VFS won't do that. The mount(8) command has been made smart
enough enough to figure out what particular type of filesystem you are
trying to mount by inspecting superblocks.

There are however ways to confuse it: if you reformat an ext3
filesystem to reiserfs (version 3), mounting that filesystem without
"-t reiserfs" will trick mount(8) into mounting it as an ext3
filesystem (which will usually fail). This is because the ext3
superblocks lives at offset 0x400, and the reiserfs superblock at
0x8000. When you format a partition as reiserfs, it will not erase old
ext3 superblocks. Before looking for a reiserfs superblock, mount(8)
first looks for an ext3 superblock. The old ext3 superblock wasn't
erased, but usually most of the other ext3 structures are and so
mount(8) will fail to mount the filesystem. Don't know if this
particular "bug" is still there, but it has bitten me in the past.


Erik

-- 
They're all fools. Don't worry. Darwin may be slow, but he'll
eventually get them. -- Matthew Lammers in alt.sysadmin.recovery

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Upgrading datastructures between different filesystem versions

[Christoph Hellwig]

On Fri, Sep 28, 2007 at 03:11:00PM +0200, Erik Mouw wrote:
> There are however ways to confuse it: if you reformat an ext3
> filesystem to reiserfs (version 3), mounting that filesystem without
> "-t reiserfs" will trick mount(8) into mounting it as an ext3
> filesystem (which will usually fail). This is because the ext3
> superblocks lives at offset 0x400, and the reiserfs superblock at
> 0x8000. When you format a partition as reiserfs, it will not erase old
> ext3 superblocks. Before looking for a reiserfs superblock, mount(8)
> first looks for an ext3 superblock. The old ext3 superblock wasn't
> erased, but usually most of the other ext3 structures are and so
> mount(8) will fail to mount the filesystem. Don't know if this
> particular "bug" is still there, but it has bitten me in the past.

This is easy to fix, though.  Quoting mkfs.xfs:

	/*
         * Zero out the beginning of the device, to obliterate any old
	 * filesystem signatures out there.  This should take care of
	 * swap (somewhere around the page size), jfs (32k),
	 * ext[2,3] and reiserfs (64k) - and hopefully all else.
	 */
	buf = libxfs_getbuf(xi.ddev, 0, BTOBB(WHACK_SIZE));
	bzero(XFS_BUF_PTR(buf), WHACK_SIZE);
	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
	libxfs_purgebuf(buf);

Re: Upgrading datastructures between different filesystem versions

[Theodore Tso]

On Fri, Sep 28, 2007 at 02:31:46PM +0100, Christoph Hellwig wrote:
> On Fri, Sep 28, 2007 at 03:11:00PM +0200, Erik Mouw wrote:
> > There are however ways to confuse it: if you reformat an ext3
> > filesystem to reiserfs (version 3), mounting that filesystem without
> > "-t reiserfs" will trick mount(8) into mounting it as an ext3
> > filesystem (which will usually fail). This is because the ext3
> > superblocks lives at offset 0x400, and the reiserfs superblock at
> > 0x8000. When you format a partition as reiserfs, it will not erase old
> > ext3 superblocks. Before looking for a reiserfs superblock, mount(8)
> > first looks for an ext3 superblock. The old ext3 superblock wasn't
> > erased, but usually most of the other ext3 structures are and so
> > mount(8) will fail to mount the filesystem. Don't know if this
> > particular "bug" is still there, but it has bitten me in the past.
> 
> This is easy to fix, though.  Quoting mkfs.xfs:
> 
> 	/*
>          * Zero out the beginning of the device, to obliterate any old
> 	 * filesystem signatures out there.  This should take care of
> 	 * swap (somewhere around the page size), jfs (32k),
> 	 * ext[2,3] and reiserfs (64k) - and hopefully all else.
> 	 */
> 	buf = libxfs_getbuf(xi.ddev, 0, BTOBB(WHACK_SIZE));
> 	bzero(XFS_BUF_PTR(buf), WHACK_SIZE);
> 	libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
> 	libxfs_purgebuf(buf);

Ext3 does something similar, zapping space at the beginning AND the
end of the partition (because the MD superblocks are at the end).
It's just a misfeature of reiserfs's mkfs that it doesn't do this.

							- Ted

Re: Upgrading datastructures between different filesystem versions

[Christoph Hellwig]

On Fri, Sep 28, 2007 at 03:47:24PM -0400, Theodore Tso wrote:
> Ext3 does something similar, zapping space at the beginning AND the
> end of the partition (because the MD superblocks are at the end).
> It's just a misfeature of reiserfs's mkfs that it doesn't do this.

mkfs.xfs of course also whipes at the end.  I just wanted to show how
easy this is to fix.