[PATCH v1 00/31] e2fsprogs September 2013 patchbomb

[PATCH v1 00/31] e2fsprogs September 2013 patchbomb

[Darrick J. Wong]

Oh boy.  A larger pile of random e2fsprogs patches than last month! :D

The first seven patches fix miscellaneous errors.  The first patch fixes a bug
in one of the patches that was accepted from last month's patchbomb.  The
second fixes a bug where multiple directory entries could be created for a
single call to ext2fs_link.  The third and fourth patches fix some omissions in
the metadata checksumming error catalog.  The fifth patch fixes a bug wherein
ext2fs_link fails to notice to add checksum space when de-hashing a directory.
The sixth patch fixes a crash in getflags because the ioctl is defined to take
an unsigned long, whereas the program only allocates an int.  The seventh patch
causes the large_file feature to be set if someone uses the fileio APIs to
write out a file size bigger than 2GB.

The next four patches fix a number of off-by-one errors in the hole-punching
code.  Most of them stem from a misunderstanding of the 'start' and 'end
parameters to ext2fs_punch; assuming that the API means "punch out block
$start, block $end, and every block between the two", these patches fix that.
There's also a fix for a problem wherein we miss punching extents because
concurrent deleting-while-iterating has the effect of advancing the extent
pointer.

After that, there are eight patches to fix various 64bit value truncation
problems I observed first by trying to resize2fs a 20T filesystem.  I then
rebuilt with -Wconversion and fixed a few more places where we would rip off
the upper 32 bits of a 64-bit value, or use strtoul instead of strtoull.  I
also updated badblocks to deal with extents.

Then comes a patch to add sparse checking to the build process.

For the next six patches, I turned on bigalloc and found a number of bugs
relating to the fact that block_alloc_stats2() takes a block number but
operates on clusters.  I've fixed up all the allocation errors that I found.
I also decided to make the quota code use ext2fs_punch rather than try to
correct its behavior wrt bigalloc.  There was also a bug wherein the
requirement that 64-bit bitmaps be enabled (via EXT2_FLAG_64BITS) for bigalloc
filesystems.

The next three patches provide the ability to toggle the 64bit feature on any
ext4 filesystem.  Since extents are required to address blocks above 2^32, I
also added in warnings to prevent the user from having 64bit,^extents.

The final two patches add the ability to edit extended attributes and add a
fuse2fs driver for e2fsprogs.  I admit that the xattr editing functions
probably clash with the inline_data patches.  The fuse driver should work with
the latest versions of Linux fuse (2.9.2) and osxfuse (2.6.1).  I've been using
the fuse driver to test e2fsprogs functionality, which is how I came across
most of the bugs fixed above.

There are still bigalloc bugs in resize2fs -- it still thinks it's freeing
blocks, when in reality it's freeing clusters.  Also it's silly to have two
xattr implementations; I will probably just end up moving to Zheng Liu's
implementation ("libext2fs: add functions to operate on extended attribute")
since it's sitting in -pu.  For today I have not, because there were plenty of
other bugs to fix! :)  Long term, if fuse2fs actually goes anywhere, I think
it'd be useful to reintroduce journal replay too.  (Or cheat and use e2fsck -E
journal_only...)

I've tested these e2fsprogs changes against the -next branch as of a few days
ago.  After a longish while I think I've managed to reconstruct my old metadata
checksumming test program from the old days, so I've posted that version here:
https://gist.github.com/djwong/6372995 ... though now it has tests for
hole-punching and resizing. :) If you want to test fuse2fs with icsum.sh, use
this script: https://gist.github.com/djwong/6772237

These days, I use a 2GB ramdisk and a 20T "disk" I constructed out of
dm-snapshot to test in an x64 VM.

Comments and questions are, as always, welcome.

--D

[PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Darrick J. Wong]

When we're constructing the initial block bitmap as part of removing the
gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
cluster numbers because ext2fs_mark_block_bitmap2() does this for us.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 misc/tune2fs.c |   13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)


diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index ddf3259..52247e0 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -820,20 +820,17 @@ static void disable_uninit_bg(ext2_filsys fs, __u32 csum_feature_flag)
 	/* The bbitmap is zeroed; we must mark group metadata blocks in use */
 	for (i = 0; i < fs->group_desc_count; i++) {
 		b = ext2fs_block_bitmap_loc(fs, i);
-		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
+		ext2fs_mark_block_bitmap2(fs->block_map, b);
 		b = ext2fs_inode_bitmap_loc(fs, i);
-		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
+		ext2fs_mark_block_bitmap2(fs->block_map, b);
 
 		retval = ext2fs_super_and_bgd_loc2(fs, i, &b, &c, &d, NULL);
 		if (retval == 0 && b)
-			ext2fs_mark_block_bitmap2(fs->block_map,
-						  EXT2FS_B2C(fs, b));
+			ext2fs_mark_block_bitmap2(fs->block_map, b);
 		if (retval == 0 && c)
-			ext2fs_mark_block_bitmap2(fs->block_map,
-						  EXT2FS_B2C(fs, c));
+			ext2fs_mark_block_bitmap2(fs->block_map, c);
 		if (retval == 0 && d)
-			ext2fs_mark_block_bitmap2(fs->block_map,
-						  EXT2FS_B2C(fs, d));
+			ext2fs_mark_block_bitmap2(fs->block_map, d);
 		if (retval) {
 			com_err("disable_uninit_bg", retval,
 				"while initializing block bitmaps");

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:26:49 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when
>     clearing uninit_bg
> 
> When we're constructing the initial block bitmap as part of removing the
> gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
> cluster numbers because ext2fs_mark_block_bitmap2() does this for us.

I wonder if it's possible to use the old-style bitmap interface (the
one which does not understand 64-bit bitmaps). If so, then you also
need to fix ext2fs_mark_generic_bmap() (and others) so that we
convert blocks to clusters if needed.

Thanks!
-Lukas

> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  misc/tune2fs.c |   13 +++++--------
>  1 file changed, 5 insertions(+), 8 deletions(-)
> 
> 
> diff --git a/misc/tune2fs.c b/misc/tune2fs.c
> index ddf3259..52247e0 100644
> --- a/misc/tune2fs.c
> +++ b/misc/tune2fs.c
> @@ -820,20 +820,17 @@ static void disable_uninit_bg(ext2_filsys fs, __u32 csum_feature_flag)
>  	/* The bbitmap is zeroed; we must mark group metadata blocks in use */
>  	for (i = 0; i < fs->group_desc_count; i++) {
>  		b = ext2fs_block_bitmap_loc(fs, i);
> -		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
> +		ext2fs_mark_block_bitmap2(fs->block_map, b);
>  		b = ext2fs_inode_bitmap_loc(fs, i);
> -		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
> +		ext2fs_mark_block_bitmap2(fs->block_map, b);
>  
>  		retval = ext2fs_super_and_bgd_loc2(fs, i, &b, &c, &d, NULL);
>  		if (retval == 0 && b)
> -			ext2fs_mark_block_bitmap2(fs->block_map,
> -						  EXT2FS_B2C(fs, b));
> +			ext2fs_mark_block_bitmap2(fs->block_map, b);
>  		if (retval == 0 && c)
> -			ext2fs_mark_block_bitmap2(fs->block_map,
> -						  EXT2FS_B2C(fs, c));
> +			ext2fs_mark_block_bitmap2(fs->block_map, c);
>  		if (retval == 0 && d)
> -			ext2fs_mark_block_bitmap2(fs->block_map,
> -						  EXT2FS_B2C(fs, d));
> +			ext2fs_mark_block_bitmap2(fs->block_map, d);
>  		if (retval) {
>  			com_err("disable_uninit_bg", retval,
>  				"while initializing block bitmaps");
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Darrick J. Wong]

On Thu, Oct 03, 2013 at 06:53:38PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> 
> > Date: Mon, 30 Sep 2013 18:26:49 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when
> >     clearing uninit_bg
> > 
> > When we're constructing the initial block bitmap as part of removing the
> > gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
> > cluster numbers because ext2fs_mark_block_bitmap2() does this for us.
> 
> I wonder if it's possible to use the old-style bitmap interface (the
> one which does not understand 64-bit bitmaps). If so, then you also
> need to fix ext2fs_mark_generic_bmap() (and others) so that we
> convert blocks to clusters if needed.

It's possible to do this (bad thing) by not specifying EXT2_FLAG_64BITS when
calling ext2fs_openfs(); see patch 26 for a quick fix.

(I'm imagining that most bigalloc users probably want 64bit anyway?)

(No idea, bigalloc isn't (currently) my use case.)

--D
> 
> Thanks!
> -Lukas
> 
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  misc/tune2fs.c |   13 +++++--------
> >  1 file changed, 5 insertions(+), 8 deletions(-)
> > 
> > 
> > diff --git a/misc/tune2fs.c b/misc/tune2fs.c
> > index ddf3259..52247e0 100644
> > --- a/misc/tune2fs.c
> > +++ b/misc/tune2fs.c
> > @@ -820,20 +820,17 @@ static void disable_uninit_bg(ext2_filsys fs, __u32 csum_feature_flag)
> >  	/* The bbitmap is zeroed; we must mark group metadata blocks in use */
> >  	for (i = 0; i < fs->group_desc_count; i++) {
> >  		b = ext2fs_block_bitmap_loc(fs, i);
> > -		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
> > +		ext2fs_mark_block_bitmap2(fs->block_map, b);
> >  		b = ext2fs_inode_bitmap_loc(fs, i);
> > -		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
> > +		ext2fs_mark_block_bitmap2(fs->block_map, b);
> >  
> >  		retval = ext2fs_super_and_bgd_loc2(fs, i, &b, &c, &d, NULL);
> >  		if (retval == 0 && b)
> > -			ext2fs_mark_block_bitmap2(fs->block_map,
> > -						  EXT2FS_B2C(fs, b));
> > +			ext2fs_mark_block_bitmap2(fs->block_map, b);
> >  		if (retval == 0 && c)
> > -			ext2fs_mark_block_bitmap2(fs->block_map,
> > -						  EXT2FS_B2C(fs, c));
> > +			ext2fs_mark_block_bitmap2(fs->block_map, c);
> >  		if (retval == 0 && d)
> > -			ext2fs_mark_block_bitmap2(fs->block_map,
> > -						  EXT2FS_B2C(fs, d));
> > +			ext2fs_mark_block_bitmap2(fs->block_map, d);
> >  		if (retval) {
> >  			com_err("disable_uninit_bg", retval,
> >  				"while initializing block bitmaps");
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Lukáš Czerner]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3676 bytes --]

On Thu, 3 Oct 2013, Darrick J. Wong wrote:

> Date: Thu, 3 Oct 2013 12:04:44 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: Lukáš Czerner <lczerner@redhat.com>
> Cc: tytso@mit.edu, linux-ext4@vger.kernel.org
> Subject: Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when
>     clearing uninit_bg
> 
> On Thu, Oct 03, 2013 at 06:53:38PM +0200, Lukáš Czerner wrote:
> > On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> > 
> > > Date: Mon, 30 Sep 2013 18:26:49 -0700
> > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > To: tytso@mit.edu, darrick.wong@oracle.com
> > > Cc: linux-ext4@vger.kernel.org
> > > Subject: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when
> > >     clearing uninit_bg
> > > 
> > > When we're constructing the initial block bitmap as part of removing the
> > > gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
> > > cluster numbers because ext2fs_mark_block_bitmap2() does this for us.
> > 
> > I wonder if it's possible to use the old-style bitmap interface (the
> > one which does not understand 64-bit bitmaps). If so, then you also
> > need to fix ext2fs_mark_generic_bmap() (and others) so that we
> > convert blocks to clusters if needed.
> 
> It's possible to do this (bad thing) by not specifying EXT2_FLAG_64BITS when
> calling ext2fs_openfs(); see patch 26 for a quick fix.
> 
> (I'm imagining that most bigalloc users probably want 64bit anyway?)
> 
> (No idea, bigalloc isn't (currently) my use case.)

Fair enough. The patch no. 26 seems to do the trick.

Thanks!
-Lukas

> 
> --D
> > 
> > Thanks!
> > -Lukas
> > 
> > > 
> > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > > ---
> > >  misc/tune2fs.c |   13 +++++--------
> > >  1 file changed, 5 insertions(+), 8 deletions(-)
> > > 
> > > 
> > > diff --git a/misc/tune2fs.c b/misc/tune2fs.c
> > > index ddf3259..52247e0 100644
> > > --- a/misc/tune2fs.c
> > > +++ b/misc/tune2fs.c
> > > @@ -820,20 +820,17 @@ static void disable_uninit_bg(ext2_filsys fs, __u32 csum_feature_flag)
> > >  	/* The bbitmap is zeroed; we must mark group metadata blocks in use */
> > >  	for (i = 0; i < fs->group_desc_count; i++) {
> > >  		b = ext2fs_block_bitmap_loc(fs, i);
> > > -		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
> > > +		ext2fs_mark_block_bitmap2(fs->block_map, b);
> > >  		b = ext2fs_inode_bitmap_loc(fs, i);
> > > -		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, b));
> > > +		ext2fs_mark_block_bitmap2(fs->block_map, b);
> > >  
> > >  		retval = ext2fs_super_and_bgd_loc2(fs, i, &b, &c, &d, NULL);
> > >  		if (retval == 0 && b)
> > > -			ext2fs_mark_block_bitmap2(fs->block_map,
> > > -						  EXT2FS_B2C(fs, b));
> > > +			ext2fs_mark_block_bitmap2(fs->block_map, b);
> > >  		if (retval == 0 && c)
> > > -			ext2fs_mark_block_bitmap2(fs->block_map,
> > > -						  EXT2FS_B2C(fs, c));
> > > +			ext2fs_mark_block_bitmap2(fs->block_map, c);
> > >  		if (retval == 0 && d)
> > > -			ext2fs_mark_block_bitmap2(fs->block_map,
> > > -						  EXT2FS_B2C(fs, d));
> > > +			ext2fs_mark_block_bitmap2(fs->block_map, d);
> > >  		if (retval) {
> > >  			com_err("disable_uninit_bg", retval,
> > >  				"while initializing block bitmaps");
> > > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Theodore Ts'o]

On Thu, Oct 03, 2013 at 12:04:44PM -0700, Darrick J. Wong wrote:
> On Thu, Oct 03, 2013 at 06:53:38PM +0200, Lukáš Czerner wrote:
> > On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> > 
> > > Date: Mon, 30 Sep 2013 18:26:49 -0700
> > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > To: tytso@mit.edu, darrick.wong@oracle.com
> > > Cc: linux-ext4@vger.kernel.org
> > > Subject: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when
> > >     clearing uninit_bg
> > > 
> > > When we're constructing the initial block bitmap as part of removing the
> > > gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
> > > cluster numbers because ext2fs_mark_block_bitmap2() does this for us.
> > 
> > I wonder if it's possible to use the old-style bitmap interface (the
> > one which does not understand 64-bit bitmaps). If so, then you also
> > need to fix ext2fs_mark_generic_bmap() (and others) so that we
> > convert blocks to clusters if needed.
> 
> It's possible to do this (bad thing) by not specifying EXT2_FLAG_64BITS when
> calling ext2fs_openfs(); see patch 26 for a quick fix.

Bigalloc requires the new-style bitmaps, so patch 26 is the right fix
for this.

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 09:03:28AM -0400, Theodore Ts'o wrote:
> On Thu, Oct 03, 2013 at 12:04:44PM -0700, Darrick J. Wong wrote:
> > On Thu, Oct 03, 2013 at 06:53:38PM +0200, Lukáš Czerner wrote:
> > > On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> > > 
> > > > Date: Mon, 30 Sep 2013 18:26:49 -0700
> > > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > > To: tytso@mit.edu, darrick.wong@oracle.com
> > > > Cc: linux-ext4@vger.kernel.org
> > > > Subject: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when
> > > >     clearing uninit_bg
> > > > 
> > > > When we're constructing the initial block bitmap as part of removing the
> > > > gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
> > > > cluster numbers because ext2fs_mark_block_bitmap2() does this for us.
> > > 
> > > I wonder if it's possible to use the old-style bitmap interface (the
> > > one which does not understand 64-bit bitmaps). If so, then you also
> > > need to fix ext2fs_mark_generic_bmap() (and others) so that we
> > > convert blocks to clusters if needed.
> > 
> > It's possible to do this (bad thing) by not specifying EXT2_FLAG_64BITS when
> > calling ext2fs_openfs(); see patch 26 for a quick fix.
> 
> Bigalloc requires the new-style bitmaps, so patch 26 is the right fix
> for this.

Any thoughts on patch #1?  It fixes bugs in the recently-landed patch "tune2fs:
zero inode table when removing checksums".

--D
> 
> 						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Theodore Ts'o]

On Wed, Oct 09, 2013 at 03:10:28PM -0700, Darrick J. Wong wrote:
> 
> Any thoughts on patch #1?  It fixes bugs in the recently-landed patch "tune2fs:
> zero inode table when removing checksums".

I haven't gotten to it yet since I've been focusing on patches which
apply to the maint branch (i.e., for a 1.42.x release).

The metadata checksum patches are on the master/next branch, which
will be for an eventual 1.43 release.

What I plan to do is to get all or most of the changes which are
appropriate for the maint branch applied.  I'll then merge the maint
branch into the next branch, and then start applying the patches which
are specific for a future 1.43 release.

Cheers,

					- Ted

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Darrick J. Wong]

On Wed, Oct 09, 2013 at 08:26:02PM -0400, Theodore Ts'o wrote:
> On Wed, Oct 09, 2013 at 03:10:28PM -0700, Darrick J. Wong wrote:
> > 
> > Any thoughts on patch #1?  It fixes bugs in the recently-landed patch "tune2fs:
> > zero inode table when removing checksums".
> 
> I haven't gotten to it yet since I've been focusing on patches which
> apply to the maint branch (i.e., for a 1.42.x release).
> 
> The metadata checksum patches are on the master/next branch, which
> will be for an eventual 1.43 release.
> 
> What I plan to do is to get all or most of the changes which are
> appropriate for the maint branch applied.  I'll then merge the maint
> branch into the next branch, and then start applying the patches which
> are specific for a future 1.43 release.

Ok.  Do you want me to hold off on reissuing changed patches and sending out
new patches until then?  Or should I just send 'em and we can deal with the
blizzard later?

Patches 2, 14-16, and 22 have changed, and there are now 9 more patches, mostly
to fix resize2fs bugs.

Eh, I'll move the ones you took to the start of my quilt, and hope they don't
change too much when they get merged in.  That'll help me keep the in-review
patches in a consecutive pile.

--D
> 
> Cheers,
> 
> 					- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:26:49PM -0700, Darrick J. Wong wrote:
> When we're constructing the initial block bitmap as part of removing the
> gdt_csum (i.e. uninit_bg) feature, we mustn't convert the block numbers to
> cluster numbers because ext2fs_mark_block_bitmap2() does this for us.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied to the next branch.

I also added the following test which checks for the bug fixed by this
commit.

						- Ted

>From e7619b71c095637e2947d2efea4fdaec60bad1f1 Mon Sep 17 00:00:00 2001
From: Theodore Ts'o <tytso@mit.edu>
Date: Sat, 12 Oct 2013 23:08:02 -0400
Subject: [PATCH] tests: add new test t_uninit_bg_rm

This test checks tune2fs's support for removing the uninit_bg feature
flag.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
---
 tests/filter.sed            |  7 ++++++-
 tests/t_uninit_bg_rm/expect | 21 +++++++++++++++++++
 tests/t_uninit_bg_rm/script | 50 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 tests/t_uninit_bg_rm/expect
 create mode 100644 tests/t_uninit_bg_rm/script

diff --git a/tests/filter.sed b/tests/filter.sed
index 91b956b..59fad4e 100644
--- a/tests/filter.sed
+++ b/tests/filter.sed
@@ -1,4 +1,9 @@
-/^[dbgumpe2fsckrsiz]* [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
+/^debugfs [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
+/^dumpe2fs [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
+/^e2fsck [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
+/^mke2fs [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
+/^resize2fs [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
+/^tune2fs [1-9]\.[0-9]*[.-][^ ]* ([0-9]*-[A-Za-z]*-[0-9]*)/d
 s/\\015//g
 /automatically checked/d
 /^Directory Hash Seed:/d
diff --git a/tests/t_uninit_bg_rm/expect b/tests/t_uninit_bg_rm/expect
new file mode 100644
index 0000000..61e9eaa
--- /dev/null
+++ b/tests/t_uninit_bg_rm/expect
@@ -0,0 +1,21 @@
+mke2fs -q -t ext4 -F -o Linux -b 1024 test.img 1G
+tune2fs -f -O ^uninit_bg test.img
+ 
+fsck -yf -N test_filesys test.img
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/65536 files (0.0% non-contiguous), 52294/1048576 blocks
+ 
+mke2fs -q -t ext4 -O bigalloc -F -o Linux -b 1024 -C 8192 test.img 10G
+tune2fs -f -O ^uninit_bg test.img
+ 
+fsck -yf -N test_filesys test.img
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 11/655360 files (0.0% non-contiguous), 199864/10485760 blocks
diff --git a/tests/t_uninit_bg_rm/script b/tests/t_uninit_bg_rm/script
new file mode 100644
index 0000000..cd397c5
--- /dev/null
+++ b/tests/t_uninit_bg_rm/script
@@ -0,0 +1,50 @@
+test_description="remove uninit_bg"
+OUT=$test_name.log
+FSCK_OPT=-yf
+EXP=$test_dir/expect
+
+cp /dev/null $TMPFILE
+rm -f $OUT.new
+
+echo mke2fs -q -t ext4 -F -o Linux -b 1024 $TMPFILE 1G >> $OUT.new
+$MKE2FS -q -t ext4 -F -o Linux -b 1024 $TMPFILE 1G >> $OUT.new 2>&1
+
+echo "tune2fs -f -O ^uninit_bg $TMPFILE" >> $OUT.new
+$TUNE2FS -f -O ^uninit_bg $TMPFILE >> $OUT.new 2>&1
+
+echo " " >> $OUT.new
+echo fsck $FSCK_OPT -N test_filesys test.img >> $OUT.new
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT.new 2>&1
+
+echo " " >> $OUT.new
+cp /dev/null $TMPFILE
+echo mke2fs -q -t ext4 -O bigalloc -F -o Linux -b 1024 -C 8192 $TMPFILE 10G >> $OUT.new
+$MKE2FS -q -t ext4 -O bigalloc -F -o Linux -b 1024 -C 8192 $TMPFILE 10G >> $OUT.new 2>&1
+
+echo "tune2fs -f -O ^uninit_bg $TMPFILE" >> $OUT.new
+$TUNE2FS -f -O ^uninit_bg $TMPFILE >> $OUT.new 2>&1
+
+echo " " >> $OUT.new
+echo fsck $FSCK_OPT -N test_filesys test.img >> $OUT.new
+$FSCK $FSCK_OPT -N test_filesys $TMPFILE >> $OUT.new 2>&1
+
+sed -f $cmd_dir/filter.sed -e "s;$TMPFILE;test.img;" $OUT.new > $OUT
+
+rm -f $OUT.new $TMPFILE
+
+#
+# Do the verification
+#
+
+cmp -s $OUT $EXP
+status=$?
+
+if [ "$status" = 0 ] ; then
+	echo "$test_name: $test_description: ok"
+	touch $test_name.ok
+else
+	echo "$test_name: $test_description: failed"
+	diff $DIFF_OPTS $EXP $OUT > $test_name.failed
+fi
+
+unset IMAGE FSCK_OPT OUT EXP
-- 
1.7.12.rc0.22.gcdd159b

[PATCH 02/31] libext2fs: Only link an inode into a directory once

[Darrick J. Wong]

The ext2fs_link helper function link_proc does not check the value of ls->done,
which means that if the function finds multiple empty spaces that will fit the
new directory entry, it will create a directory entry in each of the spaces.
Instead of doing that, check the done value and don't do anything more if we've
already added the directory entry.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/link.c |    3 +++
 1 file changed, 3 insertions(+)


diff --git a/lib/ext2fs/link.c b/lib/ext2fs/link.c
index f715067..e3ff450 100644
--- a/lib/ext2fs/link.c
+++ b/lib/ext2fs/link.c
@@ -43,6 +43,9 @@ static int link_proc(struct ext2_dir_entry *dirent,
 	int ret = 0;
 	int csum_size = 0;
 
+	if (ls->done)
+		return 0;
+
 	rec_len = EXT2_DIR_REC_LEN(ls->namelen);
 
 	ls->err = ext2fs_get_rec_len(ls->fs, dirent, &curr_rec_len);

Re: [PATCH 02/31] libext2fs: Only link an inode into a directory once

[jon ernst]

On Mon, Sep 30, 2013 at 9:26 PM, Darrick J. Wong
<darrick.wong@oracle.com> wrote:
> The ext2fs_link helper function link_proc does not check the value of ls->done,
> which means that if the function finds multiple empty spaces that will fit the
> new directory entry, it will create a directory entry in each of the spaces.
> Instead of doing that, check the done value and don't do anything more if we've
> already added the directory entry.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/link.c |    3 +++
>  1 file changed, 3 insertions(+)
>
>
> diff --git a/lib/ext2fs/link.c b/lib/ext2fs/link.c
> index f715067..e3ff450 100644
> --- a/lib/ext2fs/link.c
> +++ b/lib/ext2fs/link.c
> @@ -43,6 +43,9 @@ static int link_proc(struct ext2_dir_entry *dirent,
>         int ret = 0;
>         int csum_size = 0;
>
> +       if (ls->done)
> +               return 0;
maybe "return ret;" here to keep consistency.
> +
>         rec_len = EXT2_DIR_REC_LEN(ls->namelen);
>
>         ls->err = ext2fs_get_rec_len(ls->fs, dirent, &curr_rec_len);
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 02/31] libext2fs: Only link an inode into a directory once

[Darrick J. Wong]

On Tue, Oct 01, 2013 at 11:37:50AM -0400, jon ernst wrote:
> On Mon, Sep 30, 2013 at 9:26 PM, Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> > The ext2fs_link helper function link_proc does not check the value of ls->done,
> > which means that if the function finds multiple empty spaces that will fit the
> > new directory entry, it will create a directory entry in each of the spaces.
> > Instead of doing that, check the done value and don't do anything more if we've
> > already added the directory entry.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  lib/ext2fs/link.c |    3 +++
> >  1 file changed, 3 insertions(+)
> >
> >
> > diff --git a/lib/ext2fs/link.c b/lib/ext2fs/link.c
> > index f715067..e3ff450 100644
> > --- a/lib/ext2fs/link.c
> > +++ b/lib/ext2fs/link.c
> > @@ -43,6 +43,9 @@ static int link_proc(struct ext2_dir_entry *dirent,
> >         int ret = 0;
> >         int csum_size = 0;
> >
> > +       if (ls->done)
> > +               return 0;
> maybe "return ret;" here to keep consistency.

Actually, DIRENT_ABORT is more appropriate here since the iteration stops.
I'll update the patch; thank you for pointing this out.

(This patch is years old now...)

--D
> > +
> >         rec_len = EXT2_DIR_REC_LEN(ls->namelen);
> >
> >         ls->err = ext2fs_get_rec_len(ls->fs, dirent, &curr_rec_len);
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 02/31] libext2fs: Only link an inode into a directory once

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:26:55PM -0700, Darrick J. Wong wrote:
> The ext2fs_link helper function link_proc does not check the value of ls->done,
> which means that if the function finds multiple empty spaces that will fit the
> new directory entry, it will create a directory entry in each of the spaces.
> Instead of doing that, check the done value and don't do anything more if we've
> already added the directory entry.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied.

One minor nit, which I'll generally fix myself when I have the chance:

I generally like to fill comments at column 70, and in general, to try
to keep the summary line under 76 columns if possible, so that "git
log" doesn't wrap on 80 column screens.  It's not always possible to
do the latter, but it's something I will try to do.  Also, personally
prefer to use a lowercase to start the description since these
constraints generally mean the summary line can't be a complete
sentence.

						- Ted

Re: [PATCH 02/31] libext2fs: Only link an inode into a directory once

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 09:17:27AM -0400, Theodore Ts'o wrote:
> On Mon, Sep 30, 2013 at 06:26:55PM -0700, Darrick J. Wong wrote:
> > The ext2fs_link helper function link_proc does not check the value of ls->done,
> > which means that if the function finds multiple empty spaces that will fit the
> > new directory entry, it will create a directory entry in each of the spaces.
> > Instead of doing that, check the done value and don't do anything more if we've
> > already added the directory entry.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Thanks, applied.
> 
> One minor nit, which I'll generally fix myself when I have the chance:
> 
> I generally like to fill comments at column 70, and in general, to try
> to keep the summary line under 76 columns if possible, so that "git
> log" doesn't wrap on 80 column screens.  It's not always possible to
> do the latter, but it's something I will try to do.  Also, personally
> prefer to use a lowercase to start the description since these
> constraints generally mean the summary line can't be a complete
> sentence.

Ok.  Assuming that you're reformatting the descriptions of the patches you're
taking into your tree, I'll reformat the descriptions on the ones that don't
make it in or require rework.

--D
> 
> 						- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 03/31] Define an error code for block bitmap checksum failures

[Darrick J. Wong]

Apparently libext2fs didn't have an error code defined for block bitmap
checksum errors, so add one.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/ext2_err.et.in |    3 +++
 lib/ext2fs/rw_bitmaps.c   |    2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)


diff --git a/lib/ext2fs/ext2_err.et.in b/lib/ext2fs/ext2_err.et.in
index de1b05b..d100acd 100644
--- a/lib/ext2fs/ext2_err.et.in
+++ b/lib/ext2fs/ext2_err.et.in
@@ -476,4 +476,7 @@ ec	EXT2_ET_MMP_CSUM_INVALID,
 ec	EXT2_ET_FILE_EXISTS,
 	"Ext2 file already exists"
 
+ec	EXT2_ET_BLOCK_BITMAP_CSUM_INVALID,
+	"Block bitmap checksum does not match bitmap"
+
 	end
diff --git a/lib/ext2fs/rw_bitmaps.c b/lib/ext2fs/rw_bitmaps.c
index cc14aaf..386cbeb 100644
--- a/lib/ext2fs/rw_bitmaps.c
+++ b/lib/ext2fs/rw_bitmaps.c
@@ -275,7 +275,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 				    !ext2fs_block_bitmap_csum_verify(fs, i,
 						block_bitmap, block_nbytes)) {
 					retval =
-					EXT2_ET_BLOCK_BITMAP_READ;
+					EXT2_ET_BLOCK_BITMAP_CSUM_INVALID;
 					goto cleanup;
 				}
 			} else

Re: [PATCH 03/31] Define an error code for block bitmap checksum failures

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:02PM -0700, Darrick J. Wong wrote:
> Apparently libext2fs didn't have an error code defined for block bitmap
> checksum errors, so add one.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

					- Ted

[PATCH 04/31] libext2fs: Fix a minor grammatical error in the error catalog

[Darrick J. Wong]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 6590 bytes --]

'an block' should be 'a block'.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/ext2_err.et.in |    2 +-
 po/ca.po                  |    2 +-
 po/cs.po                  |    2 +-
 po/de.po                  |    2 +-
 po/e2fsprogs.pot          |    2 +-
 po/es.po                  |    2 +-
 po/fr.po                  |    2 +-
 po/id.po                  |    2 +-
 po/it.po                  |    2 +-
 po/nl.po                  |    2 +-
 po/pl.po                  |    2 +-
 po/sv.po                  |    2 +-
 po/tr.po                  |    2 +-
 po/vi.po                  |    2 +-
 po/zh_CN.po               |    2 +-
 15 files changed, 15 insertions(+), 15 deletions(-)


diff --git a/lib/ext2fs/ext2_err.et.in b/lib/ext2fs/ext2_err.et.in
index d100acd..c547a2c 100644
--- a/lib/ext2fs/ext2_err.et.in
+++ b/lib/ext2fs/ext2_err.et.in
@@ -96,7 +96,7 @@ ec	EXT2_ET_INODE_BITMAP_READ,
 	"Can't read an inode bitmap"
 
 ec	EXT2_ET_BLOCK_BITMAP_WRITE,
-	"Can't write an block bitmap"
+	"Can't write a block bitmap"
 
 ec	EXT2_ET_BLOCK_BITMAP_READ,
 	"Can't read an block bitmap"
diff --git a/po/ca.po b/po/ca.po
index 44cd45a..0d8f36a 100644
--- a/po/ca.po
+++ b/po/ca.po
@@ -6010,7 +6010,7 @@ msgstr "escriptura dels mapes de bits de bloc i de node-i"
 
 #: lib/ext2fs/ext2_err.c:40
 #, fuzzy
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "en escriure el mapa de bits dels blocs"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/cs.po b/po/cs.po
index b110c43..fe34399 100644
--- a/po/cs.po
+++ b/po/cs.po
@@ -5972,7 +5972,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Bitmapu iuzlů nelze přečíst"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Bitmapu bloků nelze zapsat"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/de.po b/po/de.po
index 0e7b39e..b7f71c2 100644
--- a/po/de.po
+++ b/po/de.po
@@ -5976,7 +5976,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Die Inode-Bitmap kann nicht gelesen werden"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Die Block-Bitmap kann nicht geschrieben werden"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/e2fsprogs.pot b/po/e2fsprogs.pot
index e298bed..0e63d47 100644
--- a/po/e2fsprogs.pot
+++ b/po/e2fsprogs.pot
@@ -5640,7 +5640,7 @@ msgid "Can't read an inode bitmap"
 msgstr ""
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr ""
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/es.po b/po/es.po
index 95c99ff..f47480b 100644
--- a/po/es.po
+++ b/po/es.po
@@ -6181,7 +6181,7 @@ msgstr "escribiendo los mapas de bits de los nodos-i"
 
 #: lib/ext2fs/ext2_err.c:40
 #, fuzzy
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "leyendo los mapas de bits del nodo-i y del bloque"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/fr.po b/po/fr.po
index 0c59a39..602441e 100644
--- a/po/fr.po
+++ b/po/fr.po
@@ -5987,7 +5987,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Ne peut lire un bitmap d'i-noeuds"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Ne peut écrire un bitmap de blocs"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/id.po b/po/id.po
index dce13bd..fb4e8fb 100644
--- a/po/id.po
+++ b/po/id.po
@@ -6027,7 +6027,7 @@ msgstr "menulis inode bitmap"
 
 #: lib/ext2fs/ext2_err.c:40
 #, fuzzy
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "membaca inode dan blok bitmap"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/it.po b/po/it.po
index d26d63d..ba8ba61 100644
--- a/po/it.po
+++ b/po/it.po
@@ -6088,7 +6088,7 @@ msgstr "scrivendo le mappe di bit di inode"
 
 #: lib/ext2fs/ext2_err.c:40
 #, fuzzy
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "lettura delle mappe di bit inode e blocco"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/nl.po b/po/nl.po
index 3125bbc..b98cb32 100644
--- a/po/nl.po
+++ b/po/nl.po
@@ -5982,7 +5982,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Kan een inode-bitkaart niet lezen"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Kan een blok-bitkaart niet schrijven"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/pl.po b/po/pl.po
index 924fcba..62fac2e 100644
--- a/po/pl.po
+++ b/po/pl.po
@@ -5946,7 +5946,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Nie mo¿na odczytaæ bitmapy i-wêz³ów"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Nie mo¿na zapisaæ bitmapy bloków"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/sv.po b/po/sv.po
index b2c5df1..d89370e 100644
--- a/po/sv.po
+++ b/po/sv.po
@@ -5952,7 +5952,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Kan inte läsa en inodsbitkarta"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Kan inte skriva en blockbitkarta"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/tr.po b/po/tr.po
index 186cd79..c677675 100644
--- a/po/tr.po
+++ b/po/tr.po
@@ -6352,7 +6352,7 @@ msgstr "düğüm biteşlemleri yazılıyor"
 
 #: lib/ext2fs/ext2_err.c:40
 #, fuzzy
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "düğüm ve blok biteşlemleri okunuyor"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/vi.po b/po/vi.po
index 4f15af2..736542e 100644
--- a/po/vi.po
+++ b/po/vi.po
@@ -5929,7 +5929,7 @@ msgid "Can't read an inode bitmap"
 msgstr "Không thể đọc mảng ảnh nút"
 
 #: lib/ext2fs/ext2_err.c:40
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "Không thể ghi mảng ảnh khối"
 
 #: lib/ext2fs/ext2_err.c:41
diff --git a/po/zh_CN.po b/po/zh_CN.po
index edbcc27..77f4a22 100644
--- a/po/zh_CN.po
+++ b/po/zh_CN.po
@@ -5644,7 +5644,7 @@ msgstr "inode 位图"
 
 #: lib/ext2fs/ext2_err.c:40
 #, fuzzy
-msgid "Can't write an block bitmap"
+msgid "Can't write a block bitmap"
 msgstr "块位图"
 
 #: lib/ext2fs/ext2_err.c:41

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 04/31] libext2fs: Fix a minor grammatical error in the error catalog

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:08PM -0700, Darrick J. Wong wrote:
> 'an block' should be 'a block'.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied.

					- Ted

[PATCH 05/31] libext2fs: Add space for metadata checksum when unconverting a hashed directory block

[Darrick J. Wong]

The ext2fs_link function has the unfortunate habit of converting hashed
directories into unhashed directories.  It doesn't notice that it's slicing
and dicing directory entries from a former dx_{root,node} block, and therefore
doesn't write a protective dirent into the end of the block to store the
checksum.  Teach it to do this.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/link.c |   40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)


diff --git a/lib/ext2fs/link.c b/lib/ext2fs/link.c
index e3ff450..24fa083 100644
--- a/lib/ext2fs/link.c
+++ b/lib/ext2fs/link.c
@@ -42,6 +42,7 @@ static int link_proc(struct ext2_dir_entry *dirent,
 	unsigned int rec_len, min_rec_len, curr_rec_len;
 	int ret = 0;
 	int csum_size = 0;
+	struct ext2_dir_entry_tail *t;
 
 	if (ls->done)
 		return 0;
@@ -71,6 +72,40 @@ static int link_proc(struct ext2_dir_entry *dirent,
 	}
 
 	/*
+	 * Since ext2fs_link blows away htree data, we need to be careful --
+	 * if metadata_csum is enabled and we're passed in a dirent that
+	 * contains htree data, we need to create the fake entry at the end
+	 * of the block that hides the checksum.
+	 */
+
+	/* De-convert a dx_node block */
+	if (csum_size &&
+	    curr_rec_len == ls->fs->blocksize &&
+	    !dirent->inode) {
+		curr_rec_len -= csum_size;
+		ls->err = ext2fs_set_rec_len(ls->fs, curr_rec_len, dirent);
+		if (ls->err)
+			return DIRENT_ABORT;
+		t = EXT2_DIRENT_TAIL(buf, ls->fs->blocksize);
+		ext2fs_initialize_dirent_tail(ls->fs, t);
+		ret = DIRENT_CHANGED;
+	}
+
+	/* De-convert a dx_root block */
+	if (csum_size &&
+	    curr_rec_len == ls->fs->blocksize - EXT2_DIR_REC_LEN(1) &&
+	    offset == EXT2_DIR_REC_LEN(1) &&
+	    dirent->name[0] == '.' && dirent->name[1] == '.') {
+		curr_rec_len -= csum_size;
+		ls->err = ext2fs_set_rec_len(ls->fs, curr_rec_len, dirent);
+		if (ls->err)
+			return DIRENT_ABORT;
+		t = EXT2_DIRENT_TAIL(buf, ls->fs->blocksize);
+		ext2fs_initialize_dirent_tail(ls->fs, t);
+		ret = DIRENT_CHANGED;
+	}
+
+	/*
 	 * If the directory entry is used, see if we can split the
 	 * directory entry to make room for the new name.  If so,
 	 * truncate it and return.
@@ -152,6 +187,11 @@ errcode_t ext2fs_link(ext2_filsys fs, ext2_ino_t dir, const char *name,
 	if ((retval = ext2fs_read_inode(fs, dir, &inode)) != 0)
 		return retval;
 
+	/*
+	 * If this function changes to preserve the htree, remove the two
+	 * hunks in link_proc that shove checksum tails into the former
+	 * dx_root/dx_node blocks.
+	 */
 	if (inode.i_flags & EXT2_INDEX_FL) {
 		inode.i_flags &= ~EXT2_INDEX_FL;
 		if ((retval = ext2fs_write_inode(fs, dir, &inode)) != 0)

Re: [PATCH 05/31] libext2fs: Add space for metadata checksum when unconverting a hashed directory block

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:14PM -0700, Darrick J. Wong wrote:
> The ext2fs_link function has the unfortunate habit of converting hashed
> directories into unhashed directories.  It doesn't notice that it's slicing
> and dicing directory entries from a former dx_{root,node} block, and therefore
> doesn't write a protective dirent into the end of the block to store the
> checksum.  Teach it to do this.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied to the next branch.

					- Ted

[PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch

[Darrick J. Wong]

The EXT2_IOC_[GS]ETFLAGS ioctls take longs as arguments, however this code only
reserves enough storage for an int.  The kernel drivers (so far) don't transfer
more than an int but FUSE sees the long and assumes that it's ok to write the
full size of the long, which crashes if sizeof(long) > sizeof(int).

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/e2p/fgetflags.c |    3 ++-
 lib/e2p/fsetflags.c |    3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)


diff --git a/lib/e2p/fgetflags.c b/lib/e2p/fgetflags.c
index 2af8462..bfa87f2 100644
--- a/lib/e2p/fgetflags.c
+++ b/lib/e2p/fgetflags.c
@@ -66,7 +66,8 @@ int fgetflags (const char * name, unsigned long * flags)
 	return 0;
 #else /* !HAVE_STAT_FLAGS || (APPLE_DARWIN && HAVE_EXT2_IOCTLS) */
 #if HAVE_EXT2_IOCTLS
-	int fd, r, f, save_errno = 0;
+	int fd, r, save_errno = 0;
+	unsigned long f;
 
 	if (!lstat(name, &buf) &&
 	    !S_ISREG(buf.st_mode) && !S_ISDIR(buf.st_mode)) {
diff --git a/lib/e2p/fsetflags.c b/lib/e2p/fsetflags.c
index 167d16e..050cb4a 100644
--- a/lib/e2p/fsetflags.c
+++ b/lib/e2p/fsetflags.c
@@ -71,7 +71,8 @@ int fsetflags (const char * name, unsigned long flags)
 	return chflags (name, bsd_flags);
 #else /* !HAVE_CHFLAGS || (APPLE_DARWIN && HAVE_EXT2_IOCTLS) */
 #if HAVE_EXT2_IOCTLS
-	int fd, r, f, save_errno = 0;
+	int fd, r, save_errno = 0;
+	unsigned long f;
 	struct stat buf;
 
 	if (!lstat(name, &buf) &&

Re: [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:21PM -0700, Darrick J. Wong wrote:

> The EXT2_IOC_[GS]ETFLAGS ioctls take longs as arguments, however
> this code only reserves enough storage for an int.  The kernel
> drivers (so far) don't transfer more than an int but FUSE sees the
> long and assumes that it's ok to write the full size of the long,
> which crashes if sizeof(long) > sizeof(int).

All of the kernel code I was able to audit is treating the
EXT2_IOC_[SG]ETFLAGS ioctls as taking an int, not a long.  So the
defacto definition of [SG]ETFLAGS is that that they take ints, not
longs.  If we make this change which you are proposing, it will cause
problems on big-endian systems where sizeof(long) > sizeof(int) ---
for example, as would be the case on the ppc64 architecture.

I'm not sure what the FUSE problem is?  Can you say more?  Is there
some other way we can work around the problem you are trying to solve?

     	       	      	   	      	      - Ted

Re: [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 09:33:04AM -0400, Theodore Ts'o wrote:
> On Mon, Sep 30, 2013 at 06:27:21PM -0700, Darrick J. Wong wrote:
> 
> > The EXT2_IOC_[GS]ETFLAGS ioctls take longs as arguments, however
> > this code only reserves enough storage for an int.  The kernel
> > drivers (so far) don't transfer more than an int but FUSE sees the
> > long and assumes that it's ok to write the full size of the long,
> > which crashes if sizeof(long) > sizeof(int).
> 
> All of the kernel code I was able to audit is treating the
> EXT2_IOC_[SG]ETFLAGS ioctls as taking an int, not a long.  So the
> defacto definition of [SG]ETFLAGS is that that they take ints, not
> longs.  If we make this change which you are proposing, it will cause
> problems on big-endian systems where sizeof(long) > sizeof(int) ---
> for example, as would be the case on the ppc64 architecture.
> 
> I'm not sure what the FUSE problem is?  Can you say more?  Is there
> some other way we can work around the problem you are trying to solve?

If I mount a FUSE fs (specifically the fuse2fs thing in the last patch) and try
to run 'chattr +i /mnt/foo', chattr segfaults.  valgrind had this[1] to offer.
It seems that FUSE's ioctl implementation depends on the 'size' argument to
_IOR (in the EXT2_IOC_[SG]ETFLAGS definition) to figure out how much data to
transfer in or out of userspace.  Unfortunately for fgetflags, it passes in a
pointer to an int, and fuse seems to smash up the stack trying to write back a
long.  Valgrind and gdb show that the lower 32-bits of name get overwritten,
which causes the segfault.

Unfortunately, this is a bit of a heisenbug; if I build with -O0 (gcc 4.6.3,
Ubuntu 12.04, libfuse 2.9.2 backport) it goes away.  The stack corruption also
seems to go away if I print the address of f, though save_errno gets
overwritten with some suspicious looking 32695 value.

I'll keep poking at what's going on here, though I'll try to come up with a
clever solution for BE machines.

--D

[1] Valgrind messsage:
==3512== Memcheck, a memory error detector
==3512== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3512== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==3512== Command: chattr +i /mnt/foo
==3512== 
==3512== Syscall param lstat(file_name) points to unaddressable byte(s)
==3512==    at 0x53232B5: _lxstat (lxstat.c:38)
==3512==    by 0x4E34610: fsetflags (in /lib/x86_64-linux-gnu/libe2p.so.2.3)
==3512==    by 0x401350: ??? (in /usr/bin/chattr)
==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
==3512==    by 0x525E76C: (below main) (libc-start.c:226)
==3512==  Address 0x700007fb7 is not stack'd, malloc'd or (recently) free'd
==3512== 
==3512== Syscall param open(filename) points to unaddressable byte(s)
==3512==    at 0x53236C0: __open_nocancel (syscall-template.S:82)
==3512==    by 0x4E3463E: fsetflags (in /lib/x86_64-linux-gnu/libe2p.so.2.3)
==3512==    by 0x401350: ??? (in /usr/bin/chattr)
==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
==3512==    by 0x525E76C: (below main) (libc-start.c:226)
==3512==  Address 0x700007fb7 is not stack'd, malloc'd or (recently) free'd
==3512== 
chattr: Bad address ==3512== Invalid read of size 1
==3512==    at 0x52883B1: vfprintf (vfprintf.c:1630)
==3512==    by 0x528B1A3: buffered_vfprintf (vfprintf.c:2313)
==3512==    by 0x5285BDD: vfprintf (vfprintf.c:1316)
==3512==    by 0x53463D7: __vfprintf_chk (vfprintf_chk.c:35)
==3512==    by 0x503ABA8: ??? (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
==3512==    by 0x503AD1E: com_err (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
==3512==    by 0x401435: ??? (in /usr/bin/chattr)
==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
==3512==    by 0x525E76C: (below main) (libc-start.c:226)
==3512==  Address 0x700007fb7 is not stack'd, malloc'd or (recently) free'd
==3512== 
==3512== 
==3512== Process terminating with default action of signal 11 (SIGSEGV)
==3512==  Access not within mapped region at address 0x700007FB7
==3512==    at 0x52883B1: vfprintf (vfprintf.c:1630)
==3512==    by 0x528B1A3: buffered_vfprintf (vfprintf.c:2313)
==3512==    by 0x5285BDD: vfprintf (vfprintf.c:1316)
==3512==    by 0x53463D7: __vfprintf_chk (vfprintf_chk.c:35)
==3512==    by 0x503ABA8: ??? (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
==3512==    by 0x503AD1E: com_err (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
==3512==    by 0x401435: ??? (in /usr/bin/chattr)
==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
==3512==    by 0x525E76C: (below main) (libc-start.c:226)
==3512==  If you believe this happened as a result of a stack
==3512==  overflow in your program's main thread (unlikely but
==3512==  possible), you can try to increase the size of the
==3512==  main thread stack using the --main-stacksize= flag.
==3512==  The main thread stack size used in this run was 8388608.
==3512== 
==3512== HEAP SUMMARY:
==3512==     in use at exit: 0 bytes in 0 blocks
==3512==   total heap usage: 247 allocs, 247 frees, 22,481 bytes allocated
==3512== 
==3512== All heap blocks were freed -- no leaks are possible
==3512== 
==3512== For counts of detected and suppressed errors, rerun with: -v
==3512== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)
Segmentation fault

Re: [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch

[Darrick J. Wong]

Well, it's nastier than this.  For an ioctl that only writes to userspace,
libfuse allocates an internal buffer with whatever junk is in memory at the
time.  If the fuse server's ioctl handler doesn't zero the buffer before
returning, the remnants of this garbage are passed from the server back to the
kernel, which dutifully copies the garbage into the user program's memory.
Even if the result is some error code!

fuse2fs' getflags handler would write an int to the lower end of this buffer,
so garbage + flags would get sent back to userspace.  In my particular case,
gcc was applying some kind of optimization to the chattr binary that places the
flags variable next to the 'name' variable in change_attributes(), and so the
return from the ioctl (through fs/fuse/file.c) would write a 64-bit value over
flags and the lower 32 bits of name.  The next time anyone tried to use name,
it would get a garbage pointer and (probably) crash.

Yuck.  FUSE assumes an interface contract (the data size encoded in the ioctl
number) that neither userspace nor kernel actually abide.  This has gone on for
years with no problems, since both components made the same implicit assumption
about data size in the same way.  Unfortunately, userspace breaks only on FUSE,
so I don't know what to do.

It's possible to adapt both e2p and FUSE servers to allocate an unsigned long
and shift the values around accordingly, but that won't help avoid a crash with
existing e2p binaries.  Even if I comment out the .ioctl function pointer in
fuse2fs.c, running chattr will still crash--FUSE copies the buffer even for an
error result.

Long term I guess we could define a new pair of ioctls that work with pointers
to 64-bit values and deprecate the old ones.  Or perhaps there's a better
suggestion than "don't run chattr/lsattr on a FUSE"?

--D

On Mon, Oct 07, 2013 at 01:40:55PM -0700, Darrick J. Wong wrote:
> On Mon, Oct 07, 2013 at 09:33:04AM -0400, Theodore Ts'o wrote:
> > On Mon, Sep 30, 2013 at 06:27:21PM -0700, Darrick J. Wong wrote:
> > 
> > > The EXT2_IOC_[GS]ETFLAGS ioctls take longs as arguments, however
> > > this code only reserves enough storage for an int.  The kernel
> > > drivers (so far) don't transfer more than an int but FUSE sees the
> > > long and assumes that it's ok to write the full size of the long,
> > > which crashes if sizeof(long) > sizeof(int).
> > 
> > All of the kernel code I was able to audit is treating the
> > EXT2_IOC_[SG]ETFLAGS ioctls as taking an int, not a long.  So the
> > defacto definition of [SG]ETFLAGS is that that they take ints, not
> > longs.  If we make this change which you are proposing, it will cause
> > problems on big-endian systems where sizeof(long) > sizeof(int) ---
> > for example, as would be the case on the ppc64 architecture.
> > 
> > I'm not sure what the FUSE problem is?  Can you say more?  Is there
> > some other way we can work around the problem you are trying to solve?
> 
> If I mount a FUSE fs (specifically the fuse2fs thing in the last patch) and try
> to run 'chattr +i /mnt/foo', chattr segfaults.  valgrind had this[1] to offer.
> It seems that FUSE's ioctl implementation depends on the 'size' argument to
> _IOR (in the EXT2_IOC_[SG]ETFLAGS definition) to figure out how much data to
> transfer in or out of userspace.  Unfortunately for fgetflags, it passes in a
> pointer to an int, and fuse seems to smash up the stack trying to write back a
> long.  Valgrind and gdb show that the lower 32-bits of name get overwritten,
> which causes the segfault.
> 
> Unfortunately, this is a bit of a heisenbug; if I build with -O0 (gcc 4.6.3,
> Ubuntu 12.04, libfuse 2.9.2 backport) it goes away.  The stack corruption also
> seems to go away if I print the address of f, though save_errno gets
> overwritten with some suspicious looking 32695 value.
> 
> I'll keep poking at what's going on here, though I'll try to come up with a
> clever solution for BE machines.
> 
> --D
> 
> [1] Valgrind messsage:
> ==3512== Memcheck, a memory error detector
> ==3512== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
> ==3512== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
> ==3512== Command: chattr +i /mnt/foo
> ==3512== 
> ==3512== Syscall param lstat(file_name) points to unaddressable byte(s)
> ==3512==    at 0x53232B5: _lxstat (lxstat.c:38)
> ==3512==    by 0x4E34610: fsetflags (in /lib/x86_64-linux-gnu/libe2p.so.2.3)
> ==3512==    by 0x401350: ??? (in /usr/bin/chattr)
> ==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
> ==3512==    by 0x525E76C: (below main) (libc-start.c:226)
> ==3512==  Address 0x700007fb7 is not stack'd, malloc'd or (recently) free'd
> ==3512== 
> ==3512== Syscall param open(filename) points to unaddressable byte(s)
> ==3512==    at 0x53236C0: __open_nocancel (syscall-template.S:82)
> ==3512==    by 0x4E3463E: fsetflags (in /lib/x86_64-linux-gnu/libe2p.so.2.3)
> ==3512==    by 0x401350: ??? (in /usr/bin/chattr)
> ==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
> ==3512==    by 0x525E76C: (below main) (libc-start.c:226)
> ==3512==  Address 0x700007fb7 is not stack'd, malloc'd or (recently) free'd
> ==3512== 
> chattr: Bad address ==3512== Invalid read of size 1
> ==3512==    at 0x52883B1: vfprintf (vfprintf.c:1630)
> ==3512==    by 0x528B1A3: buffered_vfprintf (vfprintf.c:2313)
> ==3512==    by 0x5285BDD: vfprintf (vfprintf.c:1316)
> ==3512==    by 0x53463D7: __vfprintf_chk (vfprintf_chk.c:35)
> ==3512==    by 0x503ABA8: ??? (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
> ==3512==    by 0x503AD1E: com_err (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
> ==3512==    by 0x401435: ??? (in /usr/bin/chattr)
> ==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
> ==3512==    by 0x525E76C: (below main) (libc-start.c:226)
> ==3512==  Address 0x700007fb7 is not stack'd, malloc'd or (recently) free'd
> ==3512== 
> ==3512== 
> ==3512== Process terminating with default action of signal 11 (SIGSEGV)
> ==3512==  Access not within mapped region at address 0x700007FB7
> ==3512==    at 0x52883B1: vfprintf (vfprintf.c:1630)
> ==3512==    by 0x528B1A3: buffered_vfprintf (vfprintf.c:2313)
> ==3512==    by 0x5285BDD: vfprintf (vfprintf.c:1316)
> ==3512==    by 0x53463D7: __vfprintf_chk (vfprintf_chk.c:35)
> ==3512==    by 0x503ABA8: ??? (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
> ==3512==    by 0x503AD1E: com_err (in /lib/x86_64-linux-gnu/libcom_err.so.2.1)
> ==3512==    by 0x401435: ??? (in /usr/bin/chattr)
> ==3512==    by 0x4010A0: ??? (in /usr/bin/chattr)
> ==3512==    by 0x525E76C: (below main) (libc-start.c:226)
> ==3512==  If you believe this happened as a result of a stack
> ==3512==  overflow in your program's main thread (unlikely but
> ==3512==  possible), you can try to increase the size of the
> ==3512==  main thread stack using the --main-stacksize= flag.
> ==3512==  The main thread stack size used in this run was 8388608.
> ==3512== 
> ==3512== HEAP SUMMARY:
> ==3512==     in use at exit: 0 bytes in 0 blocks
> ==3512==   total heap usage: 247 allocs, 247 frees, 22,481 bytes allocated
> ==3512== 
> ==3512== All heap blocks were freed -- no leaks are possible
> ==3512== 
> ==3512== For counts of detected and suppressed errors, rerun with: -v
> ==3512== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)
> Segmentation fault
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch

[Theodore Ts'o]

On Mon, Oct 07, 2013 at 04:23:58PM -0700, Darrick J. Wong wrote:
> Yuck.  FUSE assumes an interface contract (the data size encoded in the ioctl
> number) that neither userspace nor kernel actually abide.  This has gone on for
> years with no problems, since both components made the same implicit assumption
> about data size in the same way.  Unfortunately, userspace breaks only on FUSE,
> so I don't know what to do.

I suspect we've never noticed because traditionally, FUSE has never
been used to front-end a file system that supports chattr/lsattr ---
most of thsoe file systems are available as native Linux file systems,
so it's probably not a common use case for FUSE.

Can we make the FUSE ioctl handler in fs/fuse/ioctl.c special case
handle the EXT2_IOC_[SG]ETFLAG ioctls.  That would it be consistent
with the other file systms.

> Long term I guess we could define a new pair of ioctls that work with pointers
> to 64-bit values and deprecate the old ones.  Or perhaps there's a better
> suggestion than "don't run chattr/lsattr on a FUSE"?

Well we can create a new pair of ioctls, and then have the userspace
code try the new ioctl, and if the kernel doesn't support it, try the
new ioctl.  But then we would have to fix up all of the file systems
in Linux, and it would take a while before users have a new kernel and
a new userspace which supports the new ioctl.

If we put the hack in fs/fuse/file.c's ioctl handler, then it only
requires a kernel upgrade....

						- Ted

Re: [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 08:06:02PM -0400, Theodore Ts'o wrote:
> On Mon, Oct 07, 2013 at 04:23:58PM -0700, Darrick J. Wong wrote:
> > Yuck.  FUSE assumes an interface contract (the data size encoded in the ioctl
> > number) that neither userspace nor kernel actually abide.  This has gone on for
> > years with no problems, since both components made the same implicit assumption
> > about data size in the same way.  Unfortunately, userspace breaks only on FUSE,
> > so I don't know what to do.
> 
> I suspect we've never noticed because traditionally, FUSE has never
> been used to front-end a file system that supports chattr/lsattr ---
> most of thsoe file systems are available as native Linux file systems,
> so it's probably not a common use case for FUSE.
> 
> Can we make the FUSE ioctl handler in fs/fuse/ioctl.c special case
> handle the EXT2_IOC_[SG]ETFLAG ioctls.  That would it be consistent
> with the other file systms.

I can try changing fuse_do_ioctl() to add in the special case.  There don't
seem to be any definitions of _IOR('f', 1, long) that aren't *_IOC_GETFLAGS
(same for _IOW_('f', 2, long) for *_IOC_SETFLAGS) so a hack to bring FUSE in
line with xfs/btrfs/ext*/hfs+/reiser3/nilfs/gfs2/logfs/f2fs/ubifs/cifs might be
doable.

If not I suppose new ioctls are a slow-moving plan B.

--D
> 
> > Long term I guess we could define a new pair of ioctls that work with pointers
> > to 64-bit values and deprecate the old ones.  Or perhaps there's a better
> > suggestion than "don't run chattr/lsattr on a FUSE"?
> 
> Well we can create a new pair of ioctls, and then have the userspace
> code try the new ioctl, and if the kernel doesn't support it, try the
> new ioctl.  But then we would have to fix up all of the file systems
> in Linux, and it would take a while before users have a new kernel and
> a new userspace which supports the new ioctl.
> 
> If we put the hack in fs/fuse/file.c's ioctl handler, then it only
> requires a kernel upgrade....
> 
> 						- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 07/31] libext2fs: When writing a file that has a i_size > 2GB, set the large_file feature flag and update the superblock.

[Darrick J. Wong]

If someone tries to write a file that is larger than 2GB, we need to set the
large_file feature flag to affirm that i_size_hi can hold meaningful contents.
(I don't see anything checking large_file except e2fsck...)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/fileio.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)


diff --git a/lib/ext2fs/fileio.c b/lib/ext2fs/fileio.c
index 1f7002c..ae5cbf1 100644
--- a/lib/ext2fs/fileio.c
+++ b/lib/ext2fs/fileio.c
@@ -389,6 +389,18 @@ errcode_t ext2fs_file_set_size2(ext2_file_t file, ext2_off64_t size)
 	old_truncate = ((old_size + file->fs->blocksize - 1) >>
 		      EXT2_BLOCK_SIZE_BITS(file->fs->super)) + 1;
 
+	/* If we're writing a large file, set the large_file flag */
+	if (LINUX_S_ISREG(file->inode.i_mode) &&
+	    EXT2_I_SIZE(&file->inode) > 0x7FFFFFFULL &&
+	    (!EXT2_HAS_RO_COMPAT_FEATURE(file->fs->super,
+					 EXT2_FEATURE_RO_COMPAT_LARGE_FILE) ||
+	     file->fs->super->s_rev_level == EXT2_GOOD_OLD_REV)) {
+		file->fs->super->s_feature_ro_compat |=
+				EXT2_FEATURE_RO_COMPAT_LARGE_FILE;
+		ext2fs_update_dynamic_rev(file->fs);
+		ext2fs_mark_super_dirty(file->fs);
+	}
+
 	file->inode.i_size = size & 0xffffffff;
 	file->inode.i_size_high = (size >> 32);
 	if (file->ino) {

Re: [PATCH 07/31] libext2fs: When writing a file that has a i_size > 2GB, set the large_file feature flag and update the superblock.

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:28PM -0700, Darrick J. Wong wrote:
> If someone tries to write a file that is larger than 2GB, we need to set the
> large_file feature flag to affirm that i_size_hi can hold meaningful contents.
> (I don't see anything checking large_file except e2fsck...)

The large_file feature is there to protect ancient (2.2?) kernels that
didn't understand large files.  It's largely not needed in modern
kernels, which all understand it.

Still this patch is good for correctness; thanks for pointing it out!
Thanks, applied.

						- Ted

[PATCH 08/31] libext2fs: Fix off-by-one error in file truncation

[Darrick J. Wong]

When told to truncate a file, ext2fs_file_set_size2 should start with the first
block past the end of the file.  The current calculation jumps one more block
ahead, with the result that it fails to hack off the last block.  Adding
blocksize-1 and dividing is sufficient to find the last block.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/fileio.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


diff --git a/lib/ext2fs/fileio.c b/lib/ext2fs/fileio.c
index ae5cbf1..d875630 100644
--- a/lib/ext2fs/fileio.c
+++ b/lib/ext2fs/fileio.c
@@ -384,10 +384,10 @@ errcode_t ext2fs_file_set_size2(ext2_file_t file, ext2_off64_t size)
 	EXT2_CHECK_MAGIC(file, EXT2_ET_MAGIC_EXT2_FILE);
 
 	truncate_block = ((size + file->fs->blocksize - 1) >>
-			  EXT2_BLOCK_SIZE_BITS(file->fs->super)) + 1;
+			  EXT2_BLOCK_SIZE_BITS(file->fs->super));
 	old_size = EXT2_I_SIZE(&file->inode);
 	old_truncate = ((old_size + file->fs->blocksize - 1) >>
-		      EXT2_BLOCK_SIZE_BITS(file->fs->super)) + 1;
+		      EXT2_BLOCK_SIZE_BITS(file->fs->super));
 
 	/* If we're writing a large file, set the large_file flag */
 	if (LINUX_S_ISREG(file->inode.i_mode) &&

Re: [PATCH 08/31] libext2fs: Fix off-by-one error in file truncation

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:27:34 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 08/31] libext2fs: Fix off-by-one error in file truncation
> 
> When told to truncate a file, ext2fs_file_set_size2 should start with the first
> block past the end of the file.  The current calculation jumps one more block
> ahead, with the result that it fails to hack off the last block.  Adding
> blocksize-1 and dividing is sufficient to find the last block.

Looks good.

Reviewed-by: Lukas Czerner <lczerner@redhat.com>

> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/fileio.c |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> 
> diff --git a/lib/ext2fs/fileio.c b/lib/ext2fs/fileio.c
> index ae5cbf1..d875630 100644
> --- a/lib/ext2fs/fileio.c
> +++ b/lib/ext2fs/fileio.c
> @@ -384,10 +384,10 @@ errcode_t ext2fs_file_set_size2(ext2_file_t file, ext2_off64_t size)
>  	EXT2_CHECK_MAGIC(file, EXT2_ET_MAGIC_EXT2_FILE);
>  
>  	truncate_block = ((size + file->fs->blocksize - 1) >>
> -			  EXT2_BLOCK_SIZE_BITS(file->fs->super)) + 1;
> +			  EXT2_BLOCK_SIZE_BITS(file->fs->super));
>  	old_size = EXT2_I_SIZE(&file->inode);
>  	old_truncate = ((old_size + file->fs->blocksize - 1) >>
> -		      EXT2_BLOCK_SIZE_BITS(file->fs->super)) + 1;
> +		      EXT2_BLOCK_SIZE_BITS(file->fs->super));
>  
>  	/* If we're writing a large file, set the large_file flag */
>  	if (LINUX_S_ISREG(file->inode.i_mode) &&
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 08/31] libext2fs: Fix off-by-one error in file truncation

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:34PM -0700, Darrick J. Wong wrote:
> When told to truncate a file, ext2fs_file_set_size2 should start with the first
> block past the end of the file.  The current calculation jumps one more block
> ahead, with the result that it fails to hack off the last block.  Adding
> blocksize-1 and dividing is sufficient to find the last block.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

					- Ted

[PATCH 09/31] libext2fs: Rewind extent pointer when totally deleting an extent

[Darrick J. Wong]

During a punch operation, if we decide to delete an extent out of the extent
tree, the subsequent extents are moved on top of the current extent (that is to
say, they're memmmove'd down one slot).  Therefore it is not correct to advance
to the next leaf because that means we miss half the extents in the range!
Rereading the current pointer should be fine.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/punch.c |   22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)


diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
index b53653a..11c7668 100644
--- a/lib/ext2fs/punch.c
+++ b/lib/ext2fs/punch.c
@@ -186,6 +186,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 	blk64_t			free_start, next;
 	__u32			free_count, newlen;
 	int			freed = 0;
+	int			op;
 
 	retval = ext2fs_extent_open2(fs, ino, inode, &handle);
 	if (retval)
@@ -195,6 +196,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 	if (retval)
 		goto errout;
 	while (1) {
+		op = EXT2_EXTENT_NEXT_LEAF;
 		dbg_print_extent("main loop", &extent);
 		next = extent.e_lblk + extent.e_len;
 		dbg_printf("start %llu, end %llu, next %llu\n",
@@ -256,8 +258,23 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 			dbg_print_extent("replacing", &extent);
 			retval = ext2fs_extent_replace(handle, 0, &extent);
 		} else {
+			struct ext2fs_extent	newex;
 			dbg_printf("deleting current extent%s\n", "");
 			retval = ext2fs_extent_delete(handle, 0);
+			if (retval)
+				goto errout;
+			/*
+			 * We just moved the next extent into the current
+			 * extent's position, so re-read the extent next time.
+			 */
+			retval = ext2fs_extent_get(handle,
+						   EXT2_EXTENT_PREV_LEAF,
+						   &newex);
+			/* Can't go back? Just reread current. */
+			if (retval == EXT2_ET_EXTENT_NO_PREV) {
+				retval = 0;
+				op = EXT2_EXTENT_CURRENT;
+			}
 		}
 		if (retval)
 			goto errout;
@@ -268,9 +285,10 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 			freed++;
 		}
 	next_extent:
-		retval = ext2fs_extent_get(handle, EXT2_EXTENT_NEXT_LEAF,
+		retval = ext2fs_extent_get(handle, op,
 					   &extent);
-		if (retval == EXT2_ET_EXTENT_NO_NEXT)
+		if (retval == EXT2_ET_EXTENT_NO_NEXT ||
+		    retval == EXT2_ET_NO_CURRENT_NODE)
 			break;
 		if (retval)
 			goto errout;

Re: [PATCH 09/31] libext2fs: Rewind extent pointer when totally deleting an extent

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:40PM -0700, Darrick J. Wong wrote:
> During a punch operation, if we decide to delete an extent out of the extent
> tree, the subsequent extents are moved on top of the current extent (that is to
> say, they're memmmove'd down one slot).  Therefore it is not correct to advance
> to the next leaf because that means we miss half the extents in the range!
> Rereading the current pointer should be fine.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied.

BTW, in the future, this is the sort of change where creating a
regression test would be highly appreciated --- especially since you
presumably had to create test cases while you were creating the patch,
so it's much less effort to encapsulate it into a test case while
developing the patch.

					- Ted

Re: [PATCH 09/31] libext2fs: Rewind extent pointer when totally deleting an extent

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 09:37:25AM -0400, Theodore Ts'o wrote:
> On Mon, Sep 30, 2013 at 06:27:40PM -0700, Darrick J. Wong wrote:
> > During a punch operation, if we decide to delete an extent out of the extent
> > tree, the subsequent extents are moved on top of the current extent (that is to
> > say, they're memmmove'd down one slot).  Therefore it is not correct to advance
> > to the next leaf because that means we miss half the extents in the range!
> > Rereading the current pointer should be fine.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Thanks, applied.
> 
> BTW, in the future, this is the sort of change where creating a
> regression test would be highly appreciated --- especially since you
> presumably had to create test cases while you were creating the patch,
> so it's much less effort to encapsulate it into a test case while
> developing the patch.

I do have one, but it's hung up (with a bunch of other tests) in that icsum.sh
script.  I'm working on turning that into make check tests, but there's nearly
70 of them.

(Alternately, fire up fuse2fs and try to truncate a fragmented extent file.)

--D
> 
> 					- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 10/31] libext2fs: Allow callers to punch a single block

[Darrick J. Wong]

The range of blocks to punch is treated as an inclusive range on both ends,
i.e. if start=1 and end=2, both blocks 1 and 2 are punched out.  Thus, start ==
end means that the caller wishes to punch a single block.  Remove the check
that prevents us from punching a single block.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/ext2fs.h |    4 ++++
 lib/ext2fs/punch.c  |    5 +----
 2 files changed, 5 insertions(+), 4 deletions(-)


diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 311ceda..c37e00b 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1385,6 +1385,10 @@ extern errcode_t ext2fs_check_mount_point(const char *device, int *mount_flags,
 					  char *mtpt, int mtlen);
 
 /* punch.c */
+/*
+ * NOTE: This function removes from an inode the blocks "start", "end", and
+ * every block in between.
+ */
 extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
 			      struct ext2_inode *inode,
 			      char *block_buf, blk64_t start,
diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
index 11c7668..aac1942 100644
--- a/lib/ext2fs/punch.c
+++ b/lib/ext2fs/punch.c
@@ -315,9 +315,6 @@ extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
 	if (start > end)
 		return EINVAL;
 
-	if (start == end)
-		return 0;

Re: [PATCH 10/31] libext2fs: Allow callers to punch a single block

[jon ernst]

On Mon, Sep 30, 2013 at 9:27 PM, Darrick J. Wong
<darrick.wong@oracle.com> wrote:
> The range of blocks to punch is treated as an inclusive range on both ends,
> i.e. if start=1 and end=2, both blocks 1 and 2 are punched out.  Thus, start ==
> end means that the caller wishes to punch a single block.  Remove the check
> that prevents us from punching a single block.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/ext2fs.h |    4 ++++
>  lib/ext2fs/punch.c  |    5 +----
>  2 files changed, 5 insertions(+), 4 deletions(-)
>
>
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index 311ceda..c37e00b 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -1385,6 +1385,10 @@ extern errcode_t ext2fs_check_mount_point(const char *device, int *mount_flags,
>                                           char *mtpt, int mtlen);
>
>  /* punch.c */
> +/*
> + * NOTE: This function removes from an inode the blocks "start", "end", and
> + * every block in between.
> + */
>  extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
>                               struct ext2_inode *inode,
>                               char *block_buf, blk64_t start,
> diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
> index 11c7668..aac1942 100644
> --- a/lib/ext2fs/punch.c
> +++ b/lib/ext2fs/punch.c
> @@ -315,9 +315,6 @@ extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
>         if (start > end)
>                 return EINVAL;
>
> -       if (start == end)
> -               return 0;
> -
>         /* Read inode structure if necessary */
>         if (!inode) {
>                 retval = ext2fs_read_inode(fs, ino, &inode_buf);
> @@ -332,7 +329,7 @@ extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
>
>                 if (start > ~0U)
>                         return 0;
> -               count = ((end - start) < ~0U) ? (end - start) : ~0U;
> +               count = ((end - start + 1) < ~0U) ? (end - start + 1) : ~0U;
can you simply put "count =  end - start + 1;" since you have checked
start > end case earlier.
>                 retval = ext2fs_punch_ind(fs, inode, block_buf,
>                                           (blk_t) start, count);
>         }
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/31] libext2fs: Allow callers to punch a single block

[Darrick J. Wong]

On Tue, Oct 01, 2013 at 03:09:44PM -0400, jon ernst wrote:
> On Mon, Sep 30, 2013 at 9:27 PM, Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> > The range of blocks to punch is treated as an inclusive range on both ends,
> > i.e. if start=1 and end=2, both blocks 1 and 2 are punched out.  Thus, start ==
> > end means that the caller wishes to punch a single block.  Remove the check
> > that prevents us from punching a single block.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  lib/ext2fs/ext2fs.h |    4 ++++
> >  lib/ext2fs/punch.c  |    5 +----
> >  2 files changed, 5 insertions(+), 4 deletions(-)
> >
> >
> > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > index 311ceda..c37e00b 100644
> > --- a/lib/ext2fs/ext2fs.h
> > +++ b/lib/ext2fs/ext2fs.h
> > @@ -1385,6 +1385,10 @@ extern errcode_t ext2fs_check_mount_point(const char *device, int *mount_flags,
> >                                           char *mtpt, int mtlen);
> >
> >  /* punch.c */
> > +/*
> > + * NOTE: This function removes from an inode the blocks "start", "end", and
> > + * every block in between.
> > + */
> >  extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
> >                               struct ext2_inode *inode,
> >                               char *block_buf, blk64_t start,
> > diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
> > index 11c7668..aac1942 100644
> > --- a/lib/ext2fs/punch.c
> > +++ b/lib/ext2fs/punch.c
> > @@ -315,9 +315,6 @@ extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
> >         if (start > end)
> >                 return EINVAL;
> >
> > -       if (start == end)
> > -               return 0;
> > -
> >         /* Read inode structure if necessary */
> >         if (!inode) {
> >                 retval = ext2fs_read_inode(fs, ino, &inode_buf);
> > @@ -332,7 +329,7 @@ extern errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,
> >
> >                 if (start > ~0U)
> >                         return 0;
> > -               count = ((end - start) < ~0U) ? (end - start) : ~0U;
> > +               count = ((end - start + 1) < ~0U) ? (end - start + 1) : ~0U;
> can you simply put "count =  end - start + 1;" since you have checked
> start > end case earlier.

No.  This statement clamps 'count' at 2^32-1 (since the highest block number in
a non-extent file is 4294967295), which is necessary if the 'start' and 'end'
arguments are > 32bit arguments.  That happens in the truncate case of
ext2fs_file_set_size2().

For example, if end=0x100000080 and start=0x80, end-start+1 = 1, which is not
the behavior that we want.

For the record, if end=~0ULL and start=0, this also ensures that count=~0U.

--D

> >                 retval = ext2fs_punch_ind(fs, inode, block_buf,
> >                                           (blk_t) start, count);
> >         }
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/31] libext2fs: Allow callers to punch a single block

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:46PM -0700, Darrick J. Wong wrote:
> The range of blocks to punch is treated as an inclusive range on both ends,
> i.e. if start=1 and end=2, both blocks 1 and 2 are punched out.  Thus, start ==
> end means that the caller wishes to punch a single block.  Remove the check
> that prevents us from punching a single block.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied to the maint branch, modulo commit description
reformat.

		 	       		   - Ted

Re: [PATCH 10/31] libext2fs: Allow callers to punch a single block

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:46PM -0700, Darrick J. Wong wrote:
> The range of blocks to punch is treated as an inclusive range on both ends,
> i.e. if start=1 and end=2, both blocks 1 and 2 are punched out.  Thus, start ==
> end means that the caller wishes to punch a single block.  Remove the check
> that prevents us from punching a single block.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

					- Ted

[PATCH 11/31] libext2fs: ind_punch() must not stop examining blocks prematurely

[Darrick J. Wong]

When we're iterating the main loop in ind_punch(), "offset" tracks how far
we've progressed into the block map, "start" tells us where to start punching,
and "count" tells us how many blocks we are to punch after "start".  Therefore,
we would like to break out of the loop once the "offset" that we're looking at
has progressed past the end of the punch range.  Unfortunately, if start !=0,
the if-break clause in the loop causes us to break out of the loop early.

Therefore, change the breakout test to terminate the loop at the correct time.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/punch.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
index aac1942..0929400 100644
--- a/lib/ext2fs/punch.c
+++ b/lib/ext2fs/punch.c
@@ -60,7 +60,7 @@ static errcode_t ind_punch(ext2_filsys fs, struct ext2_inode *inode,
 #endif
 	incr = 1 << ((EXT2_BLOCK_SIZE_BITS(fs->super)-2)*level);
 	for (i=0, offset=0; i < max; i++, p++, offset += incr) {
-		if (offset > count)
+		if (offset >= start + count)
 			break;
 		if (*p == 0 || (offset+incr) <= start)
 			continue;

Re: [PATCH 11/31] libext2fs: ind_punch() must not stop examining blocks prematurely

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:53PM -0700, Darrick J. Wong wrote:
> When we're iterating the main loop in ind_punch(), "offset" tracks how far
> we've progressed into the block map, "start" tells us where to start punching,
> and "count" tells us how many blocks we are to punch after "start".  Therefore,
> we would like to break out of the loop once the "offset" that we're looking at
> has progressed past the end of the punch range.  Unfortunately, if start !=0,
> the if-break clause in the loop causes us to break out of the loop early.
> 
> Therefore, change the breakout test to terminate the loop at the correct time.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.  Again, it's my bad for not including more test cases
when originally implementing the ext2fs_punch() funciton, but one of
these days, we should add better testing for this function.

      	       	      	  	 	 - Ted

[PATCH 12/31] e2fsprogs: Fix blk_t <- blk64_t assignment mismatches

[Darrick J. Wong]

Fix all the places where we should be using a blk64_t instead of a blk_t.
These fixes are more severe because 64bit values could be truncated silently.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 debugfs/logdump.c      |    8 ++++----
 debugfs/set_fields.c   |   10 ++++------
 e2fsck/e2fsck.h        |    6 +++---
 e2fsck/journal.c       |    6 +++++-
 e2fsck/message.c       |    2 +-
 e2fsck/pass1.c         |    2 +-
 e2fsck/pass3.c         |    5 ++++-
 e2fsck/rehash.c        |    2 +-
 e2fsck/unix.c          |    2 +-
 e2fsck/util.c          |    6 +++---
 lib/ext2fs/closefs.c   |    2 +-
 lib/ext2fs/ext2fsP.h   |    2 +-
 lib/ext2fs/fileio.c    |    2 +-
 lib/ext2fs/inode.c     |    6 +++---
 lib/ext2fs/punch.c     |    2 +-
 lib/ext2fs/tst_iscan.c |    2 +-
 misc/dumpe2fs.c        |    2 +-
 misc/tune2fs.c         |   14 +++++++-------
 18 files changed, 43 insertions(+), 38 deletions(-)


diff --git a/debugfs/logdump.c b/debugfs/logdump.c
index 4a09bdb..db085cb 100644
--- a/debugfs/logdump.c
+++ b/debugfs/logdump.c
@@ -37,10 +37,10 @@ extern char *optarg;
 
 enum journal_location {JOURNAL_IS_INTERNAL, JOURNAL_IS_EXTERNAL};
 
-#define ANY_BLOCK ((blk_t) -1)
+#define ANY_BLOCK ((blk64_t) -1)
 
 int		dump_all, dump_contents, dump_descriptors;
-blk_t		block_to_dump, bitmap_to_dump, inode_block_to_dump;
+blk64_t		block_to_dump, bitmap_to_dump, inode_block_to_dump;
 unsigned int	group_to_dump, inode_offset_to_dump;
 ext2_ino_t	inode_to_dump;
 
@@ -162,7 +162,7 @@ void do_logdump(int argc, char **argv)
 			(group_offset / inodes_per_block);
 		inode_offset_to_dump = ((group_offset % inodes_per_block)
 					* sizeof(struct ext2_inode));
-		printf("Inode %u is at group %u, block %u, offset %u\n",
+		printf("Inode %u is at group %u, block %llu, offset %u\n",
 		       inode_to_dump, inode_group,
 		       inode_block_to_dump, inode_offset_to_dump);
 	}
@@ -624,7 +624,7 @@ static void dump_metadata_block(FILE *out_file, struct journal_source *source,
 		offset = ((block_to_dump - super->s_first_data_block) %
 			  super->s_blocks_per_group);
 
-		fprintf(out_file, "    (block bitmap for block %u: "
+		fprintf(out_file, "    (block bitmap for block %llu: "
 			"block is %s)\n",
 			block_to_dump,
 			ext2fs_test_bit(offset, buf) ? "SET" : "CLEAR");
diff --git a/debugfs/set_fields.c b/debugfs/set_fields.c
index 3925f24..aad1cd8 100644
--- a/debugfs/set_fields.c
+++ b/debugfs/set_fields.c
@@ -531,22 +531,20 @@ static errcode_t parse_hashalg(struct field_set_info *info,
 static errcode_t parse_bmap(struct field_set_info *info,
 			    char *field EXT2FS_ATTR((unused)), char *arg)
 {
-	unsigned long	num;
-	blk_t		blk;
+	blk64_t		blk;
 	errcode_t	retval;
 	char		*tmp;
 
-	num = strtoul(arg, &tmp, 0);
+	blk = strtoull(arg, &tmp, 0);
 	if (*tmp) {
 		fprintf(stderr, "Couldn't parse '%s' for field %s.\n",
 			arg, info->name);
 		return EINVAL;
 	}
-	blk = num;
 
-	retval = ext2fs_bmap(current_fs, set_ino,
+	retval = ext2fs_bmap2(current_fs, set_ino,
 			     (struct ext2_inode *) &set_inode,
-			     0, BMAP_SET, array_idx, &blk);
+			     NULL, BMAP_SET, array_idx, NULL, &blk);
 	if (retval) {
 		com_err("set_inode", retval, "while setting block map");
 	}
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 09a9d08..13d70f1 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -119,9 +119,9 @@ struct dx_dir_info {
 
 struct dx_dirblock_info {
 	int		type;
-	blk_t		phys;
+	blk64_t		phys;
 	int		flags;
-	blk_t		parent;
+	blk64_t		parent;
 	ext2_dirhash_t	min_hash;
 	ext2_dirhash_t	max_hash;
 	ext2_dirhash_t	node_min_hash;
@@ -547,7 +547,7 @@ extern void e2fsck_write_inode_full(e2fsck_t ctx, unsigned long ino,
 #ifdef MTRACE
 extern void mtrace_print(char *mesg);
 #endif
-extern blk_t get_backup_sb(e2fsck_t ctx, ext2_filsys fs,
+extern blk64_t get_backup_sb(e2fsck_t ctx, ext2_filsys fs,
 			   const char *name, io_manager manager);
 extern int ext2_file_type(unsigned int mode);
 extern int write_all(int fd, char *buf, size_t count);
diff --git a/e2fsck/journal.c b/e2fsck/journal.c
index 45d9462..2509303 100644
--- a/e2fsck/journal.c
+++ b/e2fsck/journal.c
@@ -291,6 +291,7 @@ static errcode_t e2fsck_get_journal(e2fsck_t ctx, journal_t **ret_journal)
 	unsigned long long	start = 0;
 	int			ext_journal = 0;
 	int			tried_backup_jnl = 0;
+	blk64_t			maxlen;
 
 	clear_problem_context(&pctx);
 
@@ -474,7 +475,10 @@ static errcode_t e2fsck_get_journal(e2fsck_t ctx, journal_t **ret_journal)
 			goto errout;
 		}
 
-		journal->j_maxlen = ext2fs_blocks_count(&jsuper);
+		maxlen = ext2fs_blocks_count(&jsuper);
+		if (maxlen > 1ULL << 32)
+			maxlen = (1ULL << 32) - 1;
+		journal->j_maxlen = maxlen;
 		start++;
 	}
 
diff --git a/e2fsck/message.c b/e2fsck/message.c
index 72a56ca..8ddfd12 100644
--- a/e2fsck/message.c
+++ b/e2fsck/message.c
@@ -489,7 +489,7 @@ static _INLINE_ void expand_percent_expression(FILE *f, ext2_filsys fs,
 #endif
 		break;
 	case 'S':
-		fprintf(f, "%u", get_backup_sb(NULL, fs, NULL, NULL));
+		fprintf(f, "%llu", get_backup_sb(NULL, fs, NULL, NULL));
 		break;
 	case 's':
 		fprintf(f, "%*s", width, ctx->str ? ctx->str : "NULL");
diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c
index 24b2e16..ab23e42 100644
--- a/e2fsck/pass1.c
+++ b/e2fsck/pass1.c
@@ -2685,7 +2685,7 @@ static int process_bad_block(ext2_filsys fs,
 	return 0;
 }
 
-static void new_table_block(e2fsck_t ctx, blk_t first_block, int group,
+static void new_table_block(e2fsck_t ctx, blk64_t first_block, int group,
 			    const char *name, int num, blk64_t *new_block)
 {
 	ext2_filsys fs = ctx->fs;
diff --git a/e2fsck/pass3.c b/e2fsck/pass3.c
index 193ec1c..2dd414b 100644
--- a/e2fsck/pass3.c
+++ b/e2fsck/pass3.c
@@ -764,6 +764,7 @@ errcode_t e2fsck_expand_directory(e2fsck_t ctx, ext2_ino_t dir,
 	errcode_t	retval;
 	struct expand_dir_struct es;
 	struct ext2_inode	inode;
+	blk64_t		sz;
 
 	if (!(fs->flags & EXT2_FLAG_RW))
 		return EXT2_ET_RO_FILSYS;
@@ -799,7 +800,9 @@ errcode_t e2fsck_expand_directory(e2fsck_t ctx, ext2_ino_t dir,
 	if (retval)
 		return retval;
 
-	inode.i_size = (es.last_block + 1) * fs->blocksize;
+	sz = (es.last_block + 1) * fs->blocksize;
+	inode.i_size = sz;
+	inode.i_size_high = sz >> 32;
 	ext2fs_iblk_add_blocks(fs, &inode, es.newblocks);
 	quota_data_add(ctx->qctx, &inode, dir, es.newblocks * fs->blocksize);
 
diff --git a/e2fsck/rehash.c b/e2fsck/rehash.c
index c1c74c8..6ef3568 100644
--- a/e2fsck/rehash.c
+++ b/e2fsck/rehash.c
@@ -684,7 +684,7 @@ struct write_dir_struct {
 	struct out_dir *outdir;
 	errcode_t	err;
 	e2fsck_t	ctx;
-	int		cleared;
+	blk64_t		cleared;
 	ext2_ino_t	dir;
 };
 
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 5b705ed..0546653 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1168,7 +1168,7 @@ int main (int argc, char *argv[])
 	const char	*lib_ver_date;
 	int		my_ver, lib_ver;
 	e2fsck_t	ctx;
-	blk_t		orig_superblock;
+	blk64_t		orig_superblock;
 	struct problem_context pctx;
 	int flags, run_result;
 	int journal_size;
diff --git a/e2fsck/util.c b/e2fsck/util.c
index 18005f4..44d7ef6 100644
--- a/e2fsck/util.c
+++ b/e2fsck/util.c
@@ -503,14 +503,14 @@ void mtrace_print(char *mesg)
 }
 #endif
 
-blk_t get_backup_sb(e2fsck_t ctx, ext2_filsys fs, const char *name,
-		   io_manager manager)
+blk64_t get_backup_sb(e2fsck_t ctx, ext2_filsys fs, const char *name,
+		      io_manager manager)
 {
 	struct ext2_super_block *sb;
 	io_channel		io = NULL;
 	void			*buf = NULL;
 	int			blocksize;
-	blk_t			superblock, ret_sb = 8193;
+	blk64_t			superblock, ret_sb = 8193;
 
 	if (fs && fs->super) {
 		ret_sb = (fs->super->s_blocks_per_group +
diff --git a/lib/ext2fs/closefs.c b/lib/ext2fs/closefs.c
index 157cbbe..db05637 100644
--- a/lib/ext2fs/closefs.c
+++ b/lib/ext2fs/closefs.c
@@ -244,7 +244,7 @@ void ext2fs_update_dynamic_rev(ext2_filsys fs)
 }
 
 static errcode_t write_backup_super(ext2_filsys fs, dgrp_t group,
-				    blk_t group_block,
+				    blk64_t group_block,
 				    struct ext2_super_block *super_shadow)
 {
 	errcode_t retval;
diff --git a/lib/ext2fs/ext2fsP.h b/lib/ext2fs/ext2fsP.h
index 3de9278..80d2d0a 100644
--- a/lib/ext2fs/ext2fsP.h
+++ b/lib/ext2fs/ext2fsP.h
@@ -66,7 +66,7 @@ struct dir_context {
  */
 struct ext2_inode_cache {
 	void *				buffer;
-	blk_t				buffer_blk;
+	blk64_t				buffer_blk;
 	int				cache_last;
 	int				cache_size;
 	int				refcount;
diff --git a/lib/ext2fs/fileio.c b/lib/ext2fs/fileio.c
index d875630..3100ee9 100644
--- a/lib/ext2fs/fileio.c
+++ b/lib/ext2fs/fileio.c
@@ -158,7 +158,7 @@ errcode_t ext2fs_file_flush(ext2_file_t file)
  */
 static errcode_t sync_buffer_position(ext2_file_t file)
 {
-	blk_t	b;
+	blk64_t	b;
 	errcode_t	retval;
 
 	b = file->pos / file->fs->blocksize;
diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
index d154d7e..46c1c58 100644
--- a/lib/ext2fs/inode.c
+++ b/lib/ext2fs/inode.c
@@ -295,9 +295,9 @@ errcode_t ext2fs_inode_scan_goto_blockgroup(ext2_inode_scan scan,
  * increasing order.
  */
 static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
-					    blk_t *num_blocks)
+					    blk64_t *num_blocks)
 {
-	blk_t	blk = scan->current_block;
+	blk64_t	blk = scan->current_block;
 	badblocks_list	bb = scan->fs->badblocks;
 
 	/*
@@ -354,7 +354,7 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
  */
 static errcode_t get_next_blocks(ext2_inode_scan scan)
 {
-	blk_t		num_blocks;
+	blk64_t		num_blocks;
 	errcode_t	retval;
 
 	/*
diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
index 0929400..4471f46 100644
--- a/lib/ext2fs/punch.c
+++ b/lib/ext2fs/punch.c
@@ -281,7 +281,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 		dbg_printf("Free start %llu, free count = %u\n",
 		       free_start, free_count);
 		while (free_count-- > 0) {
-			ext2fs_block_alloc_stats(fs, free_start++, -1);
+			ext2fs_block_alloc_stats2(fs, free_start++, -1);
 			freed++;
 		}
 	next_extent:
diff --git a/lib/ext2fs/tst_iscan.c b/lib/ext2fs/tst_iscan.c
index 6f783c3..a95296c 100644
--- a/lib/ext2fs/tst_iscan.c
+++ b/lib/ext2fs/tst_iscan.c
@@ -26,7 +26,7 @@
 #include "ext2_fs.h"
 #include "ext2fs.h"
 
-blk_t test_vec[] = { 8, 12, 24, 34, 43, 44, 100, 0 };
+blk64_t test_vec[] = { 8, 12, 24, 34, 43, 44, 100, 0 };
 
 ext2_filsys	test_fs;
 ext2fs_block_bitmap bad_block_map, touched_map;
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index 5369ffb..ae70f70 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -144,7 +144,7 @@ static void print_bg_rel_offset(ext2_filsys fs, blk64_t block, int itable,
 		printf(" (+%u)", (unsigned)(block - first_block));
 	} else if (fs->super->s_feature_incompat &
 		   EXT4_FEATURE_INCOMPAT_FLEX_BG) {
-		dgrp_t flex_grp = ext2fs_group_of_blk(fs, block);
+		dgrp_t flex_grp = ext2fs_group_of_blk2(fs, block);
 		printf(" (bg #%u + %u)", flex_grp,
 		       (unsigned)(block-ext2fs_group_first_block(fs,flex_grp)));
 	}
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 52247e0..60d1378 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -972,7 +972,7 @@ static int update_feature_set(ext2_filsys fs, char *features)
 
 		/* We need to force out the group descriptors as well */
 		fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
-		ext2fs_block_alloc_stats(fs, sb->s_mmp_block, -1);
+		ext2fs_block_alloc_stats2(fs, sb->s_mmp_block, -1);
 mmp_error:
 		sb->s_mmp_block = 0;
 		sb->s_mmp_update_interval = 0;
@@ -1836,10 +1836,10 @@ static int get_move_bitmaps(ext2_filsys fs, int new_ino_blks_per_grp,
 	return 0;
 }
 
-static int ext2fs_is_meta_block(ext2_filsys fs, blk_t blk)
+static int ext2fs_is_meta_block(ext2_filsys fs, blk64_t blk)
 {
 	dgrp_t group;
-	group = ext2fs_group_of_blk(fs, blk);
+	group = ext2fs_group_of_blk2(fs, blk);
 	if (ext2fs_block_bitmap_loc(fs, group) == blk)
 		return 1;
 	if (ext2fs_inode_bitmap_loc(fs, group) == blk)
@@ -1847,9 +1847,9 @@ static int ext2fs_is_meta_block(ext2_filsys fs, blk_t blk)
 	return 0;
 }
 
-static int ext2fs_is_block_in_group(ext2_filsys fs, dgrp_t group, blk_t blk)
+static int ext2fs_is_block_in_group(ext2_filsys fs, dgrp_t group, blk64_t blk)
 {
-	blk_t start_blk, end_blk;
+	blk64_t start_blk, end_blk;
 	start_blk = fs->super->s_first_data_block +
 			EXT2_BLOCKS_PER_GROUP(fs->super) * group;
 	/*
@@ -1889,7 +1889,7 @@ static int move_block(ext2_filsys fs, ext2fs_block_bitmap bmap)
 			 * the respective fs metadata pointers. Otherwise
 			 * fail
 			 */
-			group = ext2fs_group_of_blk(fs, blk);
+			group = ext2fs_group_of_blk2(fs, blk);
 			goal = ext2fs_group_first_block2(fs, group);
 			meta_data = 1;
 
@@ -2050,7 +2050,7 @@ err_out:
 static int group_desc_scan_and_fix(ext2_filsys fs, ext2fs_block_bitmap bmap)
 {
 	dgrp_t i;
-	blk_t blk, new_blk;
+	blk64_t blk, new_blk;
 
 	for (i = 0; i < fs->group_desc_count; i++) {
 		blk = ext2fs_block_bitmap_loc(fs, i);

Re: [PATCH 12/31] e2fsprogs: Fix blk_t <- blk64_t assignment mismatches

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:27:59PM -0700, Darrick J. Wong wrote:
> Fix all the places where we should be using a blk64_t instead of a blk_t.
> These fixes are more severe because 64bit values could be truncated silently.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied to the maint branch, thanks.

					- Ted

[PATCH 13/31] e2fsprogs: Less critical fixes to use the appropriate blk*t types

[Darrick J. Wong]

More fixing of places where raw data types (int, long, etc.) stood in for blk_t
and blk64_t.  Also fix some sites where we should probably be using blk64_t.
These fixes aren't as critical as the previous patch.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/brel.h    |   18 +++++++++---------
 lib/ext2fs/brel_ma.c |   24 ++++++++++++------------
 lib/ext2fs/irel.h    |    2 +-
 3 files changed, 22 insertions(+), 22 deletions(-)


diff --git a/lib/ext2fs/brel.h b/lib/ext2fs/brel.h
index a0dd5b9..9fdddd4 100644
--- a/lib/ext2fs/brel.h
+++ b/lib/ext2fs/brel.h
@@ -10,11 +10,11 @@
  */
 
 struct ext2_block_relocate_entry {
-	blk_t	new;
+	blk64_t	new;
 	__s16	offset;
 	__u16	flags;
 	union {
-		blk_t		block_ref;
+		blk64_t		block_ref;
 		ext2_ino_t	inode_ref;
 	} owner;
 };
@@ -28,19 +28,19 @@ typedef struct ext2_block_relocation_table *ext2_brel;
 struct ext2_block_relocation_table {
 	__u32	magic;
 	char	*name;
-	blk_t	current;
+	blk64_t	current;
 	void	*priv_data;
 
 	/*
 	 * Add a block relocation entry.
 	 */
-	errcode_t (*put)(ext2_brel brel, blk_t old,
+	errcode_t (*put)(ext2_brel brel, blk64_t old,
 			      struct ext2_block_relocate_entry *ent);
 
 	/*
 	 * Get a block relocation entry.
 	 */
-	errcode_t (*get)(ext2_brel brel, blk_t old,
+	errcode_t (*get)(ext2_brel brel, blk64_t old,
 			      struct ext2_block_relocate_entry *ent);
 
 	/*
@@ -52,19 +52,19 @@ struct ext2_block_relocation_table {
 	 * The iterator function for the inode relocation entries.
 	 * Returns an inode number of 0 when out of entries.
 	 */
-	errcode_t (*next)(ext2_brel brel, blk_t *old,
+	errcode_t (*next)(ext2_brel brel, blk64_t *old,
 			  struct ext2_block_relocate_entry *ent);
 
 	/*
 	 * Move the inode relocation table from one block number to
 	 * another.
 	 */
-	errcode_t (*move)(ext2_brel brel, blk_t old, blk_t new);
+	errcode_t (*move)(ext2_brel brel, blk64_t old, blk_t new);
 
 	/*
 	 * Remove a block relocation entry.
 	 */
-	errcode_t (*delete)(ext2_brel brel, blk_t old);
+	errcode_t (*delete)(ext2_brel brel, blk64_t old);
 
 
 	/*
@@ -73,7 +73,7 @@ struct ext2_block_relocation_table {
 	errcode_t (*free)(ext2_brel brel);
 };
 
-errcode_t ext2fs_brel_memarray_create(char *name, blk_t max_block,
+errcode_t ext2fs_brel_memarray_create(char *name, blk64_t max_block,
 				    ext2_brel *brel);
 
 #define ext2fs_brel_put(brel, old, ent) ((brel)->put((brel), old, ent))
diff --git a/lib/ext2fs/brel_ma.c b/lib/ext2fs/brel_ma.c
index e8a8280..a12afae 100644
--- a/lib/ext2fs/brel_ma.c
+++ b/lib/ext2fs/brel_ma.c
@@ -27,24 +27,24 @@
 #include "ext2fs.h"
 #include "brel.h"
 
-static errcode_t bma_put(ext2_brel brel, blk_t old,
+static errcode_t bma_put(ext2_brel brel, blk64_t old,
 			struct ext2_block_relocate_entry *ent);
-static errcode_t bma_get(ext2_brel brel, blk_t old,
+static errcode_t bma_get(ext2_brel brel, blk64_t old,
 			struct ext2_block_relocate_entry *ent);
 static errcode_t bma_start_iter(ext2_brel brel);
-static errcode_t bma_next(ext2_brel brel, blk_t *old,
+static errcode_t bma_next(ext2_brel brel, blk64_t *old,
 			 struct ext2_block_relocate_entry *ent);
-static errcode_t bma_move(ext2_brel brel, blk_t old, blk_t new);
-static errcode_t bma_delete(ext2_brel brel, blk_t old);
+static errcode_t bma_move(ext2_brel brel, blk64_t old, blk64_t new);
+static errcode_t bma_delete(ext2_brel brel, blk64_t old);
 static errcode_t bma_free(ext2_brel brel);
 
 struct brel_ma {
 	__u32 magic;
-	blk_t max_block;
+	blk64_t max_block;
 	struct ext2_block_relocate_entry *entries;
 };
 
-errcode_t ext2fs_brel_memarray_create(char *name, blk_t max_block,
+errcode_t ext2fs_brel_memarray_create(char *name, blk64_t max_block,
 				      ext2_brel *new_brel)
 {
 	ext2_brel		brel = 0;
@@ -102,7 +102,7 @@ errout:
 	return retval;
 }
 
-static errcode_t bma_put(ext2_brel brel, blk_t old,
+static errcode_t bma_put(ext2_brel brel, blk64_t old,
 			struct ext2_block_relocate_entry *ent)
 {
 	struct brel_ma 	*ma;
@@ -114,7 +114,7 @@ static errcode_t bma_put(ext2_brel brel, blk_t old,
 	return 0;
 }
 
-static errcode_t bma_get(ext2_brel brel, blk_t old,
+static errcode_t bma_get(ext2_brel brel, blk64_t old,
 			struct ext2_block_relocate_entry *ent)
 {
 	struct brel_ma 	*ma;
@@ -134,7 +134,7 @@ static errcode_t bma_start_iter(ext2_brel brel)
 	return 0;
 }
 
-static errcode_t bma_next(ext2_brel brel, blk_t *old,
+static errcode_t bma_next(ext2_brel brel, blk64_t *old,
 			  struct ext2_block_relocate_entry *ent)
 {
 	struct brel_ma 	*ma;
@@ -151,7 +151,7 @@ static errcode_t bma_next(ext2_brel brel, blk_t *old,
 	return 0;
 }
 
-static errcode_t bma_move(ext2_brel brel, blk_t old, blk_t new)
+static errcode_t bma_move(ext2_brel brel, blk64_t old, blk64_t new)
 {
 	struct brel_ma 	*ma;
 
@@ -165,7 +165,7 @@ static errcode_t bma_move(ext2_brel brel, blk_t old, blk_t new)
 	return 0;
 }
 
-static errcode_t bma_delete(ext2_brel brel, blk_t old)
+static errcode_t bma_delete(ext2_brel brel, blk64_t old)
 {
 	struct brel_ma 	*ma;
 
diff --git a/lib/ext2fs/irel.h b/lib/ext2fs/irel.h
index 9a4958b..8aaa2d2 100644
--- a/lib/ext2fs/irel.h
+++ b/lib/ext2fs/irel.h
@@ -10,7 +10,7 @@
  */
 
 struct ext2_inode_reference {
-	blk_t	block;
+	blk64_t	block;
 	__u16 offset;
 };
 

Re: [PATCH 13/31] e2fsprogs: Less critical fixes to use the appropriate blk*t types

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:28:05PM -0700, Darrick J. Wong wrote:
> More fixing of places where raw data types (int, long, etc.) stood in for blk_t
> and blk64_t.  Also fix some sites where we should probably be using blk64_t.
> These fixes aren't as critical as the previous patch.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied with the final commit description change:

Author: Darrick J. Wong <darrick.wong@oracle.com>
Date:   Mon Oct 7 09:57:36 2013 -0400

    e2fsprogs: fix inode and block relocation functions to use blk64_t
    
    The inode and block relocation functions aren't currently compiled in
    (so we don't need to worry about breaking ABI compatibility).  They
    were originally intended for use by resize2fs, but we never ended up
    using them, so (wisely) they weren't ever included in libext2fs as an
    exported interface (they're not even compiled by the Makefile).
    
    Fix them so that in case we ever use them, so that in places where raw
    data types (int, long, etc.) stood in for blk_t and blk64_t.  Also fix
    some sites where we should probably be using blk64_t.
    
    Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
    Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

					- Ted

[PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter

[Darrick J. Wong]

Since it's possible for very large filesystems to store backup superblocks at
very large (> 2^32) block numbers, we need to be able to handle the case of a
caller directing us to read one of these high-numbered backups.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 debugfs/debugfs.c   |    4 ++--
 e2fsck/journal.c    |    6 +++---
 e2fsck/unix.c       |    8 ++++----
 lib/ext2fs/ext2fs.h |    4 ++++
 lib/ext2fs/openfs.c |   13 +++++++++++--
 misc/dumpe2fs.c     |    4 ++--
 6 files changed, 26 insertions(+), 13 deletions(-)


diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index 2660218..da44ce4 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -80,8 +80,8 @@ static void open_filesystem(char *device, int open_flags, blk64_t superblock,
 	if (catastrophic)
 		open_flags |= EXT2_FLAG_SKIP_MMP;
 
-	retval = ext2fs_open(device, open_flags, superblock, blocksize,
-			     unix_io_manager, &current_fs);
+	retval = ext2fs_open3(device, NULL, open_flags, superblock, blocksize,
+			      unix_io_manager, &current_fs);
 	if (retval) {
 		com_err(device, retval, "while opening filesystem");
 		current_fs = NULL;
diff --git a/e2fsck/journal.c b/e2fsck/journal.c
index 2509303..af35a38 100644
--- a/e2fsck/journal.c
+++ b/e2fsck/journal.c
@@ -967,9 +967,9 @@ int e2fsck_run_ext3_journal(e2fsck_t ctx)
 
 	ext2fs_mmp_stop(ctx->fs);
 	ext2fs_free(ctx->fs);
-	retval = ext2fs_open(ctx->filesystem_name, EXT2_FLAG_RW,
-			     ctx->superblock, blocksize, io_ptr,
-			     &ctx->fs);
+	retval = ext2fs_open3(ctx->filesystem_name, NULL, EXT2_FLAG_RW,
+			      ctx->superblock, blocksize, io_ptr,
+			      &ctx->fs);
 	if (retval) {
 		com_err(ctx->program_name, retval,
 			_("while trying to re-open %s"),
diff --git a/e2fsck/unix.c b/e2fsck/unix.c
index 0546653..fb41ca0 100644
--- a/e2fsck/unix.c
+++ b/e2fsck/unix.c
@@ -1040,7 +1040,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
 
 	*ret_fs = NULL;
 	if (ctx->superblock && ctx->blocksize) {
-		retval = ext2fs_open2(ctx->filesystem_name, ctx->io_options,
+		retval = ext2fs_open3(ctx->filesystem_name, ctx->io_options,
 				      flags, ctx->superblock, ctx->blocksize,
 				      io_ptr, ret_fs);
 	} else if (ctx->superblock) {
@@ -1051,7 +1051,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
 				ext2fs_free(*ret_fs);
 				*ret_fs = NULL;
 			}
-			retval = ext2fs_open2(ctx->filesystem_name,
+			retval = ext2fs_open3(ctx->filesystem_name,
 					      ctx->io_options, flags,
 					      ctx->superblock, blocksize,
 					      io_ptr, ret_fs);
@@ -1059,7 +1059,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
 				break;
 		}
 	} else
-		retval = ext2fs_open2(ctx->filesystem_name, ctx->io_options,
+		retval = ext2fs_open3(ctx->filesystem_name, ctx->io_options,
 				      flags, 0, 0, io_ptr, ret_fs);
 
 	if (retval == 0)
@@ -1375,7 +1375,7 @@ failure:
 	 * don't need to update the mount count and last checked
 	 * fields in the backup superblock (the kernel doesn't update
 	 * the backup superblocks anyway).  With newer versions of the
-	 * library this flag is set by ext2fs_open2(), but we set this
+	 * library this flag is set by ext2fs_open3(), but we set this
 	 * here just to be sure.  (No, we don't support e2fsck running
 	 * with some other libext2fs than the one that it was shipped
 	 * with, but just in case....)
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index c37e00b..d5d8d03 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1437,6 +1437,10 @@ extern errcode_t ext2fs_open2(const char *name, const char *io_options,
 			      int flags, int superblock,
 			      unsigned int block_size, io_manager manager,
 			      ext2_filsys *ret_fs);
+extern errcode_t ext2fs_open3(const char *name, const char *io_options,
+			      int flags, blk64_t superblock,
+			      unsigned int block_size, io_manager manager,
+			      ext2_filsys *ret_fs);
 extern blk64_t ext2fs_descriptor_block_loc2(ext2_filsys fs,
 					blk64_t group_block, dgrp_t i);
 extern blk_t ext2fs_descriptor_block_loc(ext2_filsys fs, blk_t group_block,
diff --git a/lib/ext2fs/openfs.c b/lib/ext2fs/openfs.c
index 89117f4..3c0bf14 100644
--- a/lib/ext2fs/openfs.c
+++ b/lib/ext2fs/openfs.c
@@ -76,6 +76,15 @@ errcode_t ext2fs_open(const char *name, int flags, int superblock,
 			    manager, ret_fs);
 }
 
+errcode_t ext2fs_open2(const char *name, const char *io_options,
+		       int flags, int superblock,
+		       unsigned int block_size, io_manager manager,
+		       ext2_filsys *ret_fs)
+{
+	return ext2fs_open3(name, io_options, flags, superblock, block_size,
+			    manager, ret_fs);
+}
+
 /*
  *  Note: if superblock is non-zero, block-size must also be non-zero.
  * 	Superblock and block_size can be zero to use the default size.
@@ -90,8 +99,8 @@ errcode_t ext2fs_open(const char *name, int flags, int superblock,
  *	EXT2_FLAG_64BITS - Allow 64-bit bitfields (needed for large
  *				filesystems)
  */
-errcode_t ext2fs_open2(const char *name, const char *io_options,
-		       int flags, int superblock,
+errcode_t ext2fs_open3(const char *name, const char *io_options,
+		       int flags, blk64_t superblock,
 		       unsigned int block_size, io_manager manager,
 		       ext2_filsys *ret_fs)
 {
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index ae70f70..b139977 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -611,7 +611,7 @@ int main (int argc, char ** argv)
 		for (use_blocksize = EXT2_MIN_BLOCK_SIZE;
 		     use_blocksize <= EXT2_MAX_BLOCK_SIZE;
 		     use_blocksize *= 2) {
-			retval = ext2fs_open (device_name, flags,
+			retval = ext2fs_open3(device_name, NULL, flags,
 					      use_superblock,
 					      use_blocksize, unix_io_manager,
 					      &fs);
@@ -619,7 +619,7 @@ int main (int argc, char ** argv)
 				break;
 		}
 	} else
-		retval = ext2fs_open (device_name, flags, use_superblock,
+		retval = ext2fs_open3(device_name, NULL, flags, use_superblock,
 				      use_blocksize, unix_io_manager, &fs);
 	if (retval) {
 		com_err (program_name, retval, _("while trying to open %s"),

Re: [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:28:12 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the
>     superblock parameter
> 
> Since it's possible for very large filesystems to store backup superblocks at
> very large (> 2^32) block numbers, we need to be able to handle the case of a
> caller directing us to read one of these high-numbered backups.

I do not think this is enough, because in ext2fs_open2() or rather
ext2fs_open3() as you rename it we read the superblock block with
io_channel_read_blk() which takes unsigned long as a block number
rather than io_channel_read_blk64().

Having a simple test for this particular case might be useful.

Thanks!
-Lukas



> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  debugfs/debugfs.c   |    4 ++--
>  e2fsck/journal.c    |    6 +++---
>  e2fsck/unix.c       |    8 ++++----
>  lib/ext2fs/ext2fs.h |    4 ++++
>  lib/ext2fs/openfs.c |   13 +++++++++++--
>  misc/dumpe2fs.c     |    4 ++--
>  6 files changed, 26 insertions(+), 13 deletions(-)
> 
> 
> diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
> index 2660218..da44ce4 100644
> --- a/debugfs/debugfs.c
> +++ b/debugfs/debugfs.c
> @@ -80,8 +80,8 @@ static void open_filesystem(char *device, int open_flags, blk64_t superblock,
>  	if (catastrophic)
>  		open_flags |= EXT2_FLAG_SKIP_MMP;
>  
> -	retval = ext2fs_open(device, open_flags, superblock, blocksize,
> -			     unix_io_manager, &current_fs);
> +	retval = ext2fs_open3(device, NULL, open_flags, superblock, blocksize,
> +			      unix_io_manager, &current_fs);
>  	if (retval) {
>  		com_err(device, retval, "while opening filesystem");
>  		current_fs = NULL;
> diff --git a/e2fsck/journal.c b/e2fsck/journal.c
> index 2509303..af35a38 100644
> --- a/e2fsck/journal.c
> +++ b/e2fsck/journal.c
> @@ -967,9 +967,9 @@ int e2fsck_run_ext3_journal(e2fsck_t ctx)
>  
>  	ext2fs_mmp_stop(ctx->fs);
>  	ext2fs_free(ctx->fs);
> -	retval = ext2fs_open(ctx->filesystem_name, EXT2_FLAG_RW,
> -			     ctx->superblock, blocksize, io_ptr,
> -			     &ctx->fs);
> +	retval = ext2fs_open3(ctx->filesystem_name, NULL, EXT2_FLAG_RW,
> +			      ctx->superblock, blocksize, io_ptr,
> +			      &ctx->fs);
>  	if (retval) {
>  		com_err(ctx->program_name, retval,
>  			_("while trying to re-open %s"),
> diff --git a/e2fsck/unix.c b/e2fsck/unix.c
> index 0546653..fb41ca0 100644
> --- a/e2fsck/unix.c
> +++ b/e2fsck/unix.c
> @@ -1040,7 +1040,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
>  
>  	*ret_fs = NULL;
>  	if (ctx->superblock && ctx->blocksize) {
> -		retval = ext2fs_open2(ctx->filesystem_name, ctx->io_options,
> +		retval = ext2fs_open3(ctx->filesystem_name, ctx->io_options,
>  				      flags, ctx->superblock, ctx->blocksize,
>  				      io_ptr, ret_fs);
>  	} else if (ctx->superblock) {
> @@ -1051,7 +1051,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
>  				ext2fs_free(*ret_fs);
>  				*ret_fs = NULL;
>  			}
> -			retval = ext2fs_open2(ctx->filesystem_name,
> +			retval = ext2fs_open3(ctx->filesystem_name,
>  					      ctx->io_options, flags,
>  					      ctx->superblock, blocksize,
>  					      io_ptr, ret_fs);
> @@ -1059,7 +1059,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
>  				break;
>  		}
>  	} else
> -		retval = ext2fs_open2(ctx->filesystem_name, ctx->io_options,
> +		retval = ext2fs_open3(ctx->filesystem_name, ctx->io_options,
>  				      flags, 0, 0, io_ptr, ret_fs);
>  
>  	if (retval == 0)
> @@ -1375,7 +1375,7 @@ failure:
>  	 * don't need to update the mount count and last checked
>  	 * fields in the backup superblock (the kernel doesn't update
>  	 * the backup superblocks anyway).  With newer versions of the
> -	 * library this flag is set by ext2fs_open2(), but we set this
> +	 * library this flag is set by ext2fs_open3(), but we set this
>  	 * here just to be sure.  (No, we don't support e2fsck running
>  	 * with some other libext2fs than the one that it was shipped
>  	 * with, but just in case....)
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index c37e00b..d5d8d03 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -1437,6 +1437,10 @@ extern errcode_t ext2fs_open2(const char *name, const char *io_options,
>  			      int flags, int superblock,
>  			      unsigned int block_size, io_manager manager,
>  			      ext2_filsys *ret_fs);
> +extern errcode_t ext2fs_open3(const char *name, const char *io_options,
> +			      int flags, blk64_t superblock,
> +			      unsigned int block_size, io_manager manager,
> +			      ext2_filsys *ret_fs);
>  extern blk64_t ext2fs_descriptor_block_loc2(ext2_filsys fs,
>  					blk64_t group_block, dgrp_t i);
>  extern blk_t ext2fs_descriptor_block_loc(ext2_filsys fs, blk_t group_block,
> diff --git a/lib/ext2fs/openfs.c b/lib/ext2fs/openfs.c
> index 89117f4..3c0bf14 100644
> --- a/lib/ext2fs/openfs.c
> +++ b/lib/ext2fs/openfs.c
> @@ -76,6 +76,15 @@ errcode_t ext2fs_open(const char *name, int flags, int superblock,
>  			    manager, ret_fs);
>  }
>  
> +errcode_t ext2fs_open2(const char *name, const char *io_options,
> +		       int flags, int superblock,
> +		       unsigned int block_size, io_manager manager,
> +		       ext2_filsys *ret_fs)
> +{
> +	return ext2fs_open3(name, io_options, flags, superblock, block_size,
> +			    manager, ret_fs);
> +}
> +
>  /*
>   *  Note: if superblock is non-zero, block-size must also be non-zero.
>   * 	Superblock and block_size can be zero to use the default size.
> @@ -90,8 +99,8 @@ errcode_t ext2fs_open(const char *name, int flags, int superblock,
>   *	EXT2_FLAG_64BITS - Allow 64-bit bitfields (needed for large
>   *				filesystems)
>   */
> -errcode_t ext2fs_open2(const char *name, const char *io_options,
> -		       int flags, int superblock,
> +errcode_t ext2fs_open3(const char *name, const char *io_options,
> +		       int flags, blk64_t superblock,
>  		       unsigned int block_size, io_manager manager,
>  		       ext2_filsys *ret_fs)
>  {
> diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
> index ae70f70..b139977 100644
> --- a/misc/dumpe2fs.c
> +++ b/misc/dumpe2fs.c
> @@ -611,7 +611,7 @@ int main (int argc, char ** argv)
>  		for (use_blocksize = EXT2_MIN_BLOCK_SIZE;
>  		     use_blocksize <= EXT2_MAX_BLOCK_SIZE;
>  		     use_blocksize *= 2) {
> -			retval = ext2fs_open (device_name, flags,
> +			retval = ext2fs_open3(device_name, NULL, flags,
>  					      use_superblock,
>  					      use_blocksize, unix_io_manager,
>  					      &fs);
> @@ -619,7 +619,7 @@ int main (int argc, char ** argv)
>  				break;
>  		}
>  	} else
> -		retval = ext2fs_open (device_name, flags, use_superblock,
> +		retval = ext2fs_open3(device_name, NULL, flags, use_superblock,
>  				      use_blocksize, unix_io_manager, &fs);
>  	if (retval) {
>  		com_err (program_name, retval, _("while trying to open %s"),
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 04:30:39PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> 
> > Date: Mon, 30 Sep 2013 18:28:12 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the
> >     superblock parameter
> > 
> > Since it's possible for very large filesystems to store backup superblocks at
> > very large (> 2^32) block numbers, we need to be able to handle the case of a
> > caller directing us to read one of these high-numbered backups.
> 
> I do not think this is enough, because in ext2fs_open2() or rather
> ext2fs_open3() as you rename it we read the superblock block with
> io_channel_read_blk() which takes unsigned long as a block number
> rather than io_channel_read_blk64().
> 
> Having a simple test for this particular case might be useful.

Good catch!  I've updated the patch.

--D
> 
> Thanks!
> -Lukas
> 
> 
> 
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  debugfs/debugfs.c   |    4 ++--
> >  e2fsck/journal.c    |    6 +++---
> >  e2fsck/unix.c       |    8 ++++----
> >  lib/ext2fs/ext2fs.h |    4 ++++
> >  lib/ext2fs/openfs.c |   13 +++++++++++--
> >  misc/dumpe2fs.c     |    4 ++--
> >  6 files changed, 26 insertions(+), 13 deletions(-)
> > 
> > 
> > diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
> > index 2660218..da44ce4 100644
> > --- a/debugfs/debugfs.c
> > +++ b/debugfs/debugfs.c
> > @@ -80,8 +80,8 @@ static void open_filesystem(char *device, int open_flags, blk64_t superblock,
> >  	if (catastrophic)
> >  		open_flags |= EXT2_FLAG_SKIP_MMP;
> >  
> > -	retval = ext2fs_open(device, open_flags, superblock, blocksize,
> > -			     unix_io_manager, &current_fs);
> > +	retval = ext2fs_open3(device, NULL, open_flags, superblock, blocksize,
> > +			      unix_io_manager, &current_fs);
> >  	if (retval) {
> >  		com_err(device, retval, "while opening filesystem");
> >  		current_fs = NULL;
> > diff --git a/e2fsck/journal.c b/e2fsck/journal.c
> > index 2509303..af35a38 100644
> > --- a/e2fsck/journal.c
> > +++ b/e2fsck/journal.c
> > @@ -967,9 +967,9 @@ int e2fsck_run_ext3_journal(e2fsck_t ctx)
> >  
> >  	ext2fs_mmp_stop(ctx->fs);
> >  	ext2fs_free(ctx->fs);
> > -	retval = ext2fs_open(ctx->filesystem_name, EXT2_FLAG_RW,
> > -			     ctx->superblock, blocksize, io_ptr,
> > -			     &ctx->fs);
> > +	retval = ext2fs_open3(ctx->filesystem_name, NULL, EXT2_FLAG_RW,
> > +			      ctx->superblock, blocksize, io_ptr,
> > +			      &ctx->fs);
> >  	if (retval) {
> >  		com_err(ctx->program_name, retval,
> >  			_("while trying to re-open %s"),
> > diff --git a/e2fsck/unix.c b/e2fsck/unix.c
> > index 0546653..fb41ca0 100644
> > --- a/e2fsck/unix.c
> > +++ b/e2fsck/unix.c
> > @@ -1040,7 +1040,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
> >  
> >  	*ret_fs = NULL;
> >  	if (ctx->superblock && ctx->blocksize) {
> > -		retval = ext2fs_open2(ctx->filesystem_name, ctx->io_options,
> > +		retval = ext2fs_open3(ctx->filesystem_name, ctx->io_options,
> >  				      flags, ctx->superblock, ctx->blocksize,
> >  				      io_ptr, ret_fs);
> >  	} else if (ctx->superblock) {
> > @@ -1051,7 +1051,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
> >  				ext2fs_free(*ret_fs);
> >  				*ret_fs = NULL;
> >  			}
> > -			retval = ext2fs_open2(ctx->filesystem_name,
> > +			retval = ext2fs_open3(ctx->filesystem_name,
> >  					      ctx->io_options, flags,
> >  					      ctx->superblock, blocksize,
> >  					      io_ptr, ret_fs);
> > @@ -1059,7 +1059,7 @@ static errcode_t try_open_fs(e2fsck_t ctx, int flags, io_manager io_ptr,
> >  				break;
> >  		}
> >  	} else
> > -		retval = ext2fs_open2(ctx->filesystem_name, ctx->io_options,
> > +		retval = ext2fs_open3(ctx->filesystem_name, ctx->io_options,
> >  				      flags, 0, 0, io_ptr, ret_fs);
> >  
> >  	if (retval == 0)
> > @@ -1375,7 +1375,7 @@ failure:
> >  	 * don't need to update the mount count and last checked
> >  	 * fields in the backup superblock (the kernel doesn't update
> >  	 * the backup superblocks anyway).  With newer versions of the
> > -	 * library this flag is set by ext2fs_open2(), but we set this
> > +	 * library this flag is set by ext2fs_open3(), but we set this
> >  	 * here just to be sure.  (No, we don't support e2fsck running
> >  	 * with some other libext2fs than the one that it was shipped
> >  	 * with, but just in case....)
> > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > index c37e00b..d5d8d03 100644
> > --- a/lib/ext2fs/ext2fs.h
> > +++ b/lib/ext2fs/ext2fs.h
> > @@ -1437,6 +1437,10 @@ extern errcode_t ext2fs_open2(const char *name, const char *io_options,
> >  			      int flags, int superblock,
> >  			      unsigned int block_size, io_manager manager,
> >  			      ext2_filsys *ret_fs);
> > +extern errcode_t ext2fs_open3(const char *name, const char *io_options,
> > +			      int flags, blk64_t superblock,
> > +			      unsigned int block_size, io_manager manager,
> > +			      ext2_filsys *ret_fs);
> >  extern blk64_t ext2fs_descriptor_block_loc2(ext2_filsys fs,
> >  					blk64_t group_block, dgrp_t i);
> >  extern blk_t ext2fs_descriptor_block_loc(ext2_filsys fs, blk_t group_block,
> > diff --git a/lib/ext2fs/openfs.c b/lib/ext2fs/openfs.c
> > index 89117f4..3c0bf14 100644
> > --- a/lib/ext2fs/openfs.c
> > +++ b/lib/ext2fs/openfs.c
> > @@ -76,6 +76,15 @@ errcode_t ext2fs_open(const char *name, int flags, int superblock,
> >  			    manager, ret_fs);
> >  }
> >  
> > +errcode_t ext2fs_open2(const char *name, const char *io_options,
> > +		       int flags, int superblock,
> > +		       unsigned int block_size, io_manager manager,
> > +		       ext2_filsys *ret_fs)
> > +{
> > +	return ext2fs_open3(name, io_options, flags, superblock, block_size,
> > +			    manager, ret_fs);
> > +}
> > +
> >  /*
> >   *  Note: if superblock is non-zero, block-size must also be non-zero.
> >   * 	Superblock and block_size can be zero to use the default size.
> > @@ -90,8 +99,8 @@ errcode_t ext2fs_open(const char *name, int flags, int superblock,
> >   *	EXT2_FLAG_64BITS - Allow 64-bit bitfields (needed for large
> >   *				filesystems)
> >   */
> > -errcode_t ext2fs_open2(const char *name, const char *io_options,
> > -		       int flags, int superblock,
> > +errcode_t ext2fs_open3(const char *name, const char *io_options,
> > +		       int flags, blk64_t superblock,
> >  		       unsigned int block_size, io_manager manager,
> >  		       ext2_filsys *ret_fs)
> >  {
> > diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
> > index ae70f70..b139977 100644
> > --- a/misc/dumpe2fs.c
> > +++ b/misc/dumpe2fs.c
> > @@ -611,7 +611,7 @@ int main (int argc, char ** argv)
> >  		for (use_blocksize = EXT2_MIN_BLOCK_SIZE;
> >  		     use_blocksize <= EXT2_MAX_BLOCK_SIZE;
> >  		     use_blocksize *= 2) {
> > -			retval = ext2fs_open (device_name, flags,
> > +			retval = ext2fs_open3(device_name, NULL, flags,
> >  					      use_superblock,
> >  					      use_blocksize, unix_io_manager,
> >  					      &fs);
> > @@ -619,7 +619,7 @@ int main (int argc, char ** argv)
> >  				break;
> >  		}
> >  	} else
> > -		retval = ext2fs_open (device_name, flags, use_superblock,
> > +		retval = ext2fs_open3(device_name, NULL, flags, use_superblock,
> >  				      use_blocksize, unix_io_manager, &fs);
> >  	if (retval) {
> >  		com_err (program_name, retval, _("while trying to open %s"),
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:28:12PM -0700, Darrick J. Wong wrote:
> Since it's possible for very large filesystems to store backup superblocks at
> very large (> 2^32) block numbers, we need to be able to handle the case of a
> caller directing us to read one of these high-numbered backups.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Hmm... This is technically true, but I'm wondering how much we should
care.  In practice, users almost always use the first couple of backup
superblocks.  I could imagine a situation with a RAID array where the
first disk(s) were trashed, so we needed to use a backup superblock
beyond 2**32, but it's a bit unlikely.

If there was some other reason why we needed to add a new ext2fs_open3
variant, it would certainly be a good thing to fix.  But I'm wondering
if it's worth adding a new interface just for this.

Is there perhaps any other extensions to ext2fs_open() that we might
want to make, either now or in the future?

Regards,

						- Ted

Re: [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter

[Darrick J. Wong]

On Tue, Oct 08, 2013 at 11:58:59AM -0400, Theodore Ts'o wrote:
> On Mon, Sep 30, 2013 at 06:28:12PM -0700, Darrick J. Wong wrote:
> > Since it's possible for very large filesystems to store backup superblocks at
> > very large (> 2^32) block numbers, we need to be able to handle the case of a
> > caller directing us to read one of these high-numbered backups.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Hmm... This is technically true, but I'm wondering how much we should
> care.  In practice, users almost always use the first couple of backup
> superblocks.  I could imagine a situation with a RAID array where the
> first disk(s) were trashed, so we needed to use a backup superblock
> beyond 2**32, but it's a bit unlikely.

True, it seems unlikely now that one would ever need to use the backup sbs
beyond 2^32, but since mke2fs is writing them there, we ought to be able to use
them.

For now, though, it's truncating the top 32 bits of a (usually) user-specified
value, so if we don't extend the API we ought to at least complain and exit.

> If there was some other reason why we needed to add a new ext2fs_open3
> variant, it would certainly be a good thing to fix.  But I'm wondering
> if it's worth adding a new interface just for this.
> 
> Is there perhaps any other extensions to ext2fs_open() that we might
> want to make, either now or in the future?

I don't have any off the top of my head, but I'll think about it.

--D
> 
> Regards,
> 
> 						- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses

[Darrick J. Wong]

The extended attribute refcounting code only accepts blk_t, which is dangerous
because EA blocks can exist at high addresses (> 2^32) as well.  Therefore,
widen the block fields to 64 bits.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 e2fsck/e2fsck.h      |   12 ++++++------
 e2fsck/ea_refcount.c |   36 ++++++++++++++++++------------------
 2 files changed, 24 insertions(+), 24 deletions(-)


diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 13d70f1..f1df525 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -432,17 +432,17 @@ extern struct dx_dir_info *e2fsck_dx_dir_info_iter(e2fsck_t ctx, int *control);
 /* ea_refcount.c */
 extern errcode_t ea_refcount_create(int size, ext2_refcount_t *ret);
 extern void ea_refcount_free(ext2_refcount_t refcount);
-extern errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
+extern errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk64_t blk,
 				   int *ret);
 extern errcode_t ea_refcount_increment(ext2_refcount_t refcount,
-				       blk_t blk, int *ret);
+				       blk64_t blk, int *ret);
 extern errcode_t ea_refcount_decrement(ext2_refcount_t refcount,
-				       blk_t blk, int *ret);
+				       blk64_t blk, int *ret);
 extern errcode_t ea_refcount_store(ext2_refcount_t refcount,
-				   blk_t blk, int count);
-extern blk_t ext2fs_get_refcount_size(ext2_refcount_t refcount);
+				   blk64_t blk, int count);
+extern blk64_t ext2fs_get_refcount_size(ext2_refcount_t refcount);
 extern void ea_refcount_intr_begin(ext2_refcount_t refcount);
-extern blk_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
+extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
 
 /* ehandler.c */
 extern const char *ehandler_operation(const char *op);
diff --git a/e2fsck/ea_refcount.c b/e2fsck/ea_refcount.c
index e66e636..bcce6a0 100644
--- a/e2fsck/ea_refcount.c
+++ b/e2fsck/ea_refcount.c
@@ -25,14 +25,14 @@
  * checked, its bit is set in the block_ea_map bitmap.
  */
 struct ea_refcount_el {
-	blk_t	ea_blk;
+	blk64_t	ea_blk;
 	int	ea_count;
 };
 
 struct ea_refcount {
-	blk_t		count;
-	blk_t		size;
-	blk_t		cursor;
+	blk64_t		count;
+	blk64_t		size;
+	blk64_t		cursor;
 	struct ea_refcount_el	*list;
 };
 
@@ -111,11 +111,11 @@ static void refcount_collapse(ext2_refcount_t refcount)
  * 	specified position.
  */
 static struct ea_refcount_el *insert_refcount_el(ext2_refcount_t refcount,
-						 blk_t blk, int pos)
+						 blk64_t blk, int pos)
 {
 	struct ea_refcount_el 	*el;
 	errcode_t		retval;
-	blk_t			new_size = 0;
+	blk64_t			new_size = 0;
 	int			num;
 
 	if (refcount->count >= refcount->size) {
@@ -153,7 +153,7 @@ static struct ea_refcount_el *insert_refcount_el(ext2_refcount_t refcount,
  * 	and we can't find an entry, create one in the sorted list.
  */
 static struct ea_refcount_el *get_refcount_el(ext2_refcount_t refcount,
-					      blk_t blk, int create)
+					      blk64_t blk, int create)
 {
 	int	low, high, mid;
 
@@ -206,7 +206,7 @@ retry:
 	return 0;
 }
 
-errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
+errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk64_t blk,
 				int *ret)
 {
 	struct ea_refcount_el	*el;
@@ -220,7 +220,7 @@ errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
 	return 0;
 }
 
-errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk_t blk, int *ret)
+errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk64_t blk, int *ret)
 {
 	struct ea_refcount_el	*el;
 
@@ -234,7 +234,7 @@ errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk_t blk, int *ret)
 	return 0;
 }
 
-errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk_t blk, int *ret)
+errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk64_t blk, int *ret)
 {
 	struct ea_refcount_el	*el;
 
@@ -249,7 +249,7 @@ errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk_t blk, int *ret)
 	return 0;
 }
 
-errcode_t ea_refcount_store(ext2_refcount_t refcount, blk_t blk, int count)
+errcode_t ea_refcount_store(ext2_refcount_t refcount, blk64_t blk, int count)
 {
 	struct ea_refcount_el	*el;
 
@@ -263,7 +263,7 @@ errcode_t ea_refcount_store(ext2_refcount_t refcount, blk_t blk, int count)
 	return 0;
 }
 
-blk_t ext2fs_get_refcount_size(ext2_refcount_t refcount)
+blk64_t ext2fs_get_refcount_size(ext2_refcount_t refcount)
 {
 	if (!refcount)
 		return 0;
@@ -277,7 +277,7 @@ void ea_refcount_intr_begin(ext2_refcount_t refcount)
 }
 
 
-blk_t ea_refcount_intr_next(ext2_refcount_t refcount,
+blk64_t ea_refcount_intr_next(ext2_refcount_t refcount,
 				int *ret)
 {
 	struct ea_refcount_el	*list;
@@ -370,7 +370,7 @@ int main(int argc, char **argv)
 	int	i = 0;
 	ext2_refcount_t refcount;
 	int		size, arg;
-	blk_t		blk;
+	blk64_t		blk;
 	errcode_t	retval;
 
 	while (1) {
@@ -394,7 +394,7 @@ int main(int argc, char **argv)
 			printf("Freeing refcount\n");
 			break;
 		case BCODE_STORE:
-			blk = (blk_t) bcode_program[i++];
+			blk = (blk64_t) bcode_program[i++];
 			arg = bcode_program[i++];
 			printf("Storing blk %u with value %d\n", blk, arg);
 			retval = ea_refcount_store(refcount, blk, arg);
@@ -403,7 +403,7 @@ int main(int argc, char **argv)
 					"while storing blk %u", blk);
 			break;
 		case BCODE_FETCH:
-			blk = (blk_t) bcode_program[i++];
+			blk = (blk64_t) bcode_program[i++];
 			retval = ea_refcount_fetch(refcount, blk, &arg);
 			if (retval)
 				com_err("ea_refcount_fetch", retval,
@@ -413,7 +413,7 @@ int main(int argc, char **argv)
 				       blk, arg);
 			break;
 		case BCODE_INCR:
-			blk = (blk_t) bcode_program[i++];
+			blk = (blk64_t) bcode_program[i++];
 			retval = ea_refcount_increment(refcount, blk, &arg);
 			if (retval)
 				com_err("ea_refcount_increment", retval,
@@ -423,7 +423,7 @@ int main(int argc, char **argv)
 				       blk, arg);
 			break;
 		case BCODE_DECR:
-			blk = (blk_t) bcode_program[i++];
+			blk = (blk64_t) bcode_program[i++];
 			retval = ea_refcount_decrement(refcount, blk, &arg);
 			if (retval)
 				com_err("ea_refcount_decrement", retval,

Re: [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:28:18 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block
>      addresses

Did you meant to say 64bit ?

> 
> The extended attribute refcounting code only accepts blk_t, which is dangerous
> because EA blocks can exist at high addresses (> 2^32) as well.  Therefore,
> widen the block fields to 64 bits.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  e2fsck/e2fsck.h      |   12 ++++++------
>  e2fsck/ea_refcount.c |   36 ++++++++++++++++++------------------
>  2 files changed, 24 insertions(+), 24 deletions(-)
> 
> 
> diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
> index 13d70f1..f1df525 100644
> --- a/e2fsck/e2fsck.h
> +++ b/e2fsck/e2fsck.h
> @@ -432,17 +432,17 @@ extern struct dx_dir_info *e2fsck_dx_dir_info_iter(e2fsck_t ctx, int *control);
>  /* ea_refcount.c */
>  extern errcode_t ea_refcount_create(int size, ext2_refcount_t *ret);
>  extern void ea_refcount_free(ext2_refcount_t refcount);
> -extern errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
> +extern errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk64_t blk,
>  				   int *ret);
>  extern errcode_t ea_refcount_increment(ext2_refcount_t refcount,
> -				       blk_t blk, int *ret);
> +				       blk64_t blk, int *ret);
>  extern errcode_t ea_refcount_decrement(ext2_refcount_t refcount,
> -				       blk_t blk, int *ret);
> +				       blk64_t blk, int *ret);
>  extern errcode_t ea_refcount_store(ext2_refcount_t refcount,
> -				   blk_t blk, int count);
> -extern blk_t ext2fs_get_refcount_size(ext2_refcount_t refcount);
> +				   blk64_t blk, int count);
> +extern blk64_t ext2fs_get_refcount_size(ext2_refcount_t refcount);
>  extern void ea_refcount_intr_begin(ext2_refcount_t refcount);
> -extern blk_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
> +extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
>  
>  /* ehandler.c */
>  extern const char *ehandler_operation(const char *op);
> diff --git a/e2fsck/ea_refcount.c b/e2fsck/ea_refcount.c
> index e66e636..bcce6a0 100644
> --- a/e2fsck/ea_refcount.c
> +++ b/e2fsck/ea_refcount.c
> @@ -25,14 +25,14 @@
>   * checked, its bit is set in the block_ea_map bitmap.
>   */
>  struct ea_refcount_el {
> -	blk_t	ea_blk;
> +	blk64_t	ea_blk;
>  	int	ea_count;
>  };
>  
>  struct ea_refcount {
> -	blk_t		count;
> -	blk_t		size;
> -	blk_t		cursor;
> +	blk64_t		count;
> +	blk64_t		size;
> +	blk64_t		cursor;

I am not sure if we really need count, size and cursor to be blk_t
let alone blk64_t. It's a bit misleading because AFAICT those
variable does not represent block numbers at all. Maybe it should be
changed to something less confusing, preferably matching the actual
xattr implementation ?

Thanks!
-Lukas

>  	struct ea_refcount_el	*list;
>  };
>  
> @@ -111,11 +111,11 @@ static void refcount_collapse(ext2_refcount_t refcount)
>   * 	specified position.
>   */
>  static struct ea_refcount_el *insert_refcount_el(ext2_refcount_t refcount,
> -						 blk_t blk, int pos)
> +						 blk64_t blk, int pos)
>  {
>  	struct ea_refcount_el 	*el;
>  	errcode_t		retval;
> -	blk_t			new_size = 0;
> +	blk64_t			new_size = 0;
>  	int			num;
>  
>  	if (refcount->count >= refcount->size) {
> @@ -153,7 +153,7 @@ static struct ea_refcount_el *insert_refcount_el(ext2_refcount_t refcount,
>   * 	and we can't find an entry, create one in the sorted list.
>   */
>  static struct ea_refcount_el *get_refcount_el(ext2_refcount_t refcount,
> -					      blk_t blk, int create)
> +					      blk64_t blk, int create)
>  {
>  	int	low, high, mid;
>  
> @@ -206,7 +206,7 @@ retry:
>  	return 0;
>  }
>  
> -errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
> +errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk64_t blk,
>  				int *ret)
>  {
>  	struct ea_refcount_el	*el;
> @@ -220,7 +220,7 @@ errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
>  	return 0;
>  }
>  
> -errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk_t blk, int *ret)
> +errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk64_t blk, int *ret)
>  {
>  	struct ea_refcount_el	*el;
>  
> @@ -234,7 +234,7 @@ errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk_t blk, int *ret)
>  	return 0;
>  }
>  
> -errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk_t blk, int *ret)
> +errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk64_t blk, int *ret)
>  {
>  	struct ea_refcount_el	*el;
>  
> @@ -249,7 +249,7 @@ errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk_t blk, int *ret)
>  	return 0;
>  }
>  
> -errcode_t ea_refcount_store(ext2_refcount_t refcount, blk_t blk, int count)
> +errcode_t ea_refcount_store(ext2_refcount_t refcount, blk64_t blk, int count)
>  {
>  	struct ea_refcount_el	*el;
>  
> @@ -263,7 +263,7 @@ errcode_t ea_refcount_store(ext2_refcount_t refcount, blk_t blk, int count)
>  	return 0;
>  }
>  
> -blk_t ext2fs_get_refcount_size(ext2_refcount_t refcount)
> +blk64_t ext2fs_get_refcount_size(ext2_refcount_t refcount)
>  {
>  	if (!refcount)
>  		return 0;
> @@ -277,7 +277,7 @@ void ea_refcount_intr_begin(ext2_refcount_t refcount)
>  }
>  
>  
> -blk_t ea_refcount_intr_next(ext2_refcount_t refcount,
> +blk64_t ea_refcount_intr_next(ext2_refcount_t refcount,
>  				int *ret)
>  {
>  	struct ea_refcount_el	*list;
> @@ -370,7 +370,7 @@ int main(int argc, char **argv)
>  	int	i = 0;
>  	ext2_refcount_t refcount;
>  	int		size, arg;
> -	blk_t		blk;
> +	blk64_t		blk;
>  	errcode_t	retval;
>  
>  	while (1) {
> @@ -394,7 +394,7 @@ int main(int argc, char **argv)
>  			printf("Freeing refcount\n");
>  			break;
>  		case BCODE_STORE:
> -			blk = (blk_t) bcode_program[i++];
> +			blk = (blk64_t) bcode_program[i++];
>  			arg = bcode_program[i++];
>  			printf("Storing blk %u with value %d\n", blk, arg);
>  			retval = ea_refcount_store(refcount, blk, arg);
> @@ -403,7 +403,7 @@ int main(int argc, char **argv)
>  					"while storing blk %u", blk);
>  			break;
>  		case BCODE_FETCH:
> -			blk = (blk_t) bcode_program[i++];
> +			blk = (blk64_t) bcode_program[i++];
>  			retval = ea_refcount_fetch(refcount, blk, &arg);
>  			if (retval)
>  				com_err("ea_refcount_fetch", retval,
> @@ -413,7 +413,7 @@ int main(int argc, char **argv)
>  				       blk, arg);
>  			break;
>  		case BCODE_INCR:
> -			blk = (blk_t) bcode_program[i++];
> +			blk = (blk64_t) bcode_program[i++];
>  			retval = ea_refcount_increment(refcount, blk, &arg);
>  			if (retval)
>  				com_err("ea_refcount_increment", retval,
> @@ -423,7 +423,7 @@ int main(int argc, char **argv)
>  				       blk, arg);
>  			break;
>  		case BCODE_DECR:
> -			blk = (blk_t) bcode_program[i++];
> +			blk = (blk64_t) bcode_program[i++];
>  			retval = ea_refcount_decrement(refcount, blk, &arg);
>  			if (retval)
>  				com_err("ea_refcount_decrement", retval,
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 05:30:17PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> 
> > Date: Mon, 30 Sep 2013 18:28:18 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block
> >      addresses
> 
> Did you meant to say 64bit ?

Yep.

> 
> > 
> > The extended attribute refcounting code only accepts blk_t, which is dangerous
> > because EA blocks can exist at high addresses (> 2^32) as well.  Therefore,
> > widen the block fields to 64 bits.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  e2fsck/e2fsck.h      |   12 ++++++------
> >  e2fsck/ea_refcount.c |   36 ++++++++++++++++++------------------
> >  2 files changed, 24 insertions(+), 24 deletions(-)
> > 
> > 
> > diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
> > index 13d70f1..f1df525 100644
> > --- a/e2fsck/e2fsck.h
> > +++ b/e2fsck/e2fsck.h
> > @@ -432,17 +432,17 @@ extern struct dx_dir_info *e2fsck_dx_dir_info_iter(e2fsck_t ctx, int *control);
> >  /* ea_refcount.c */
> >  extern errcode_t ea_refcount_create(int size, ext2_refcount_t *ret);
> >  extern void ea_refcount_free(ext2_refcount_t refcount);
> > -extern errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
> > +extern errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk64_t blk,
> >  				   int *ret);
> >  extern errcode_t ea_refcount_increment(ext2_refcount_t refcount,
> > -				       blk_t blk, int *ret);
> > +				       blk64_t blk, int *ret);
> >  extern errcode_t ea_refcount_decrement(ext2_refcount_t refcount,
> > -				       blk_t blk, int *ret);
> > +				       blk64_t blk, int *ret);
> >  extern errcode_t ea_refcount_store(ext2_refcount_t refcount,
> > -				   blk_t blk, int count);
> > -extern blk_t ext2fs_get_refcount_size(ext2_refcount_t refcount);
> > +				   blk64_t blk, int count);
> > +extern blk64_t ext2fs_get_refcount_size(ext2_refcount_t refcount);
> >  extern void ea_refcount_intr_begin(ext2_refcount_t refcount);
> > -extern blk_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
> > +extern blk64_t ea_refcount_intr_next(ext2_refcount_t refcount, int *ret);
> >  
> >  /* ehandler.c */
> >  extern const char *ehandler_operation(const char *op);
> > diff --git a/e2fsck/ea_refcount.c b/e2fsck/ea_refcount.c
> > index e66e636..bcce6a0 100644
> > --- a/e2fsck/ea_refcount.c
> > +++ b/e2fsck/ea_refcount.c
> > @@ -25,14 +25,14 @@
> >   * checked, its bit is set in the block_ea_map bitmap.
> >   */
> >  struct ea_refcount_el {
> > -	blk_t	ea_blk;
> > +	blk64_t	ea_blk;
> >  	int	ea_count;
> >  };
> >  
> >  struct ea_refcount {
> > -	blk_t		count;
> > -	blk_t		size;
> > -	blk_t		cursor;
> > +	blk64_t		count;
> > +	blk64_t		size;
> > +	blk64_t		cursor;
> 
> I am not sure if we really need count, size and cursor to be blk_t
> let alone blk64_t. It's a bit misleading because AFAICT those
> variable does not represent block numbers at all. Maybe it should be
> changed to something less confusing, preferably matching the actual
> xattr implementation ?

Oops, I got a little too s/blk_t/blk64_t/ happy there.  Those could be __u32, I
think.  Or unsigned long.

--D
> 
> Thanks!
> -Lukas
> 
> >  	struct ea_refcount_el	*list;
> >  };
> >  
> > @@ -111,11 +111,11 @@ static void refcount_collapse(ext2_refcount_t refcount)
> >   * 	specified position.
> >   */
> >  static struct ea_refcount_el *insert_refcount_el(ext2_refcount_t refcount,
> > -						 blk_t blk, int pos)
> > +						 blk64_t blk, int pos)
> >  {
> >  	struct ea_refcount_el 	*el;
> >  	errcode_t		retval;
> > -	blk_t			new_size = 0;
> > +	blk64_t			new_size = 0;
> >  	int			num;
> >  
> >  	if (refcount->count >= refcount->size) {
> > @@ -153,7 +153,7 @@ static struct ea_refcount_el *insert_refcount_el(ext2_refcount_t refcount,
> >   * 	and we can't find an entry, create one in the sorted list.
> >   */
> >  static struct ea_refcount_el *get_refcount_el(ext2_refcount_t refcount,
> > -					      blk_t blk, int create)
> > +					      blk64_t blk, int create)
> >  {
> >  	int	low, high, mid;
> >  
> > @@ -206,7 +206,7 @@ retry:
> >  	return 0;
> >  }
> >  
> > -errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
> > +errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk64_t blk,
> >  				int *ret)
> >  {
> >  	struct ea_refcount_el	*el;
> > @@ -220,7 +220,7 @@ errcode_t ea_refcount_fetch(ext2_refcount_t refcount, blk_t blk,
> >  	return 0;
> >  }
> >  
> > -errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk_t blk, int *ret)
> > +errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk64_t blk, int *ret)
> >  {
> >  	struct ea_refcount_el	*el;
> >  
> > @@ -234,7 +234,7 @@ errcode_t ea_refcount_increment(ext2_refcount_t refcount, blk_t blk, int *ret)
> >  	return 0;
> >  }
> >  
> > -errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk_t blk, int *ret)
> > +errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk64_t blk, int *ret)
> >  {
> >  	struct ea_refcount_el	*el;
> >  
> > @@ -249,7 +249,7 @@ errcode_t ea_refcount_decrement(ext2_refcount_t refcount, blk_t blk, int *ret)
> >  	return 0;
> >  }
> >  
> > -errcode_t ea_refcount_store(ext2_refcount_t refcount, blk_t blk, int count)
> > +errcode_t ea_refcount_store(ext2_refcount_t refcount, blk64_t blk, int count)
> >  {
> >  	struct ea_refcount_el	*el;
> >  
> > @@ -263,7 +263,7 @@ errcode_t ea_refcount_store(ext2_refcount_t refcount, blk_t blk, int count)
> >  	return 0;
> >  }
> >  
> > -blk_t ext2fs_get_refcount_size(ext2_refcount_t refcount)
> > +blk64_t ext2fs_get_refcount_size(ext2_refcount_t refcount)
> >  {
> >  	if (!refcount)
> >  		return 0;
> > @@ -277,7 +277,7 @@ void ea_refcount_intr_begin(ext2_refcount_t refcount)
> >  }
> >  
> >  
> > -blk_t ea_refcount_intr_next(ext2_refcount_t refcount,
> > +blk64_t ea_refcount_intr_next(ext2_refcount_t refcount,
> >  				int *ret)
> >  {
> >  	struct ea_refcount_el	*list;
> > @@ -370,7 +370,7 @@ int main(int argc, char **argv)
> >  	int	i = 0;
> >  	ext2_refcount_t refcount;
> >  	int		size, arg;
> > -	blk_t		blk;
> > +	blk64_t		blk;
> >  	errcode_t	retval;
> >  
> >  	while (1) {
> > @@ -394,7 +394,7 @@ int main(int argc, char **argv)
> >  			printf("Freeing refcount\n");
> >  			break;
> >  		case BCODE_STORE:
> > -			blk = (blk_t) bcode_program[i++];
> > +			blk = (blk64_t) bcode_program[i++];
> >  			arg = bcode_program[i++];
> >  			printf("Storing blk %u with value %d\n", blk, arg);
> >  			retval = ea_refcount_store(refcount, blk, arg);
> > @@ -403,7 +403,7 @@ int main(int argc, char **argv)
> >  					"while storing blk %u", blk);
> >  			break;
> >  		case BCODE_FETCH:
> > -			blk = (blk_t) bcode_program[i++];
> > +			blk = (blk64_t) bcode_program[i++];
> >  			retval = ea_refcount_fetch(refcount, blk, &arg);
> >  			if (retval)
> >  				com_err("ea_refcount_fetch", retval,
> > @@ -413,7 +413,7 @@ int main(int argc, char **argv)
> >  				       blk, arg);
> >  			break;
> >  		case BCODE_INCR:
> > -			blk = (blk_t) bcode_program[i++];
> > +			blk = (blk64_t) bcode_program[i++];
> >  			retval = ea_refcount_increment(refcount, blk, &arg);
> >  			if (retval)
> >  				com_err("ea_refcount_increment", retval,
> > @@ -423,7 +423,7 @@ int main(int argc, char **argv)
> >  				       blk, arg);
> >  			break;
> >  		case BCODE_DECR:
> > -			blk = (blk_t) bcode_program[i++];
> > +			blk = (blk64_t) bcode_program[i++];
> >  			retval = ea_refcount_decrement(refcount, blk, &arg);
> >  			if (retval)
> >  				com_err("ea_refcount_decrement", retval,
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses

[Theodore Ts'o]

On Mon, Oct 07, 2013 at 11:37:43AM -0700, Darrick J. Wong wrote:
> > I am not sure if we really need count, size and cursor to be blk_t
> > let alone blk64_t. It's a bit misleading because AFAICT those
> > variable does not represent block numbers at all. Maybe it should be
> > changed to something less confusing, preferably matching the actual
> > xattr implementation ?
> 
> Oops, I got a little too s/blk_t/blk64_t/ happy there.  Those could be __u32, I
> think.  Or unsigned long.

__u32 should be fine, yes.  Could you send me an updated patch?

Thanks,

						- Ted

Re: [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses

[Darrick J. Wong]

On Tue, Oct 08, 2013 at 12:01:05PM -0400, Theodore Ts'o wrote:
> On Mon, Oct 07, 2013 at 11:37:43AM -0700, Darrick J. Wong wrote:
> > > I am not sure if we really need count, size and cursor to be blk_t
> > > let alone blk64_t. It's a bit misleading because AFAICT those
> > > variable does not represent block numbers at all. Maybe it should be
> > > changed to something less confusing, preferably matching the actual
> > > xattr implementation ?
> > 
> > Oops, I got a little too s/blk_t/blk64_t/ happy there.  Those could be __u32, I
> > think.  Or unsigned long.
> 
> __u32 should be fine, yes.  Could you send me an updated patch?

Yes, I will.  Are you planning to push e2fsprogs git to kernel.org/github soon?

I also now have eight more patches to fix various bigalloc and metadata_csum
related bugs in resize2fs.

--D

> 
> Thanks,
> 
> 						- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 16/31] debugfs: Handle 64bit block numbers

[Darrick J. Wong]

debugfs should use strtoull (and equivalent wrappers) for reading block numbers
from the command line.  "unsigned long" isn't wide enough on 32bit platforms.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 debugfs/debugfs.c      |   19 ++++++++++---------
 debugfs/extent_inode.c |   24 +++++++++++++-----------
 debugfs/util.c         |    2 +-
 3 files changed, 24 insertions(+), 21 deletions(-)


diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index da44ce4..60452c9 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -167,8 +167,9 @@ void do_open_filesys(int argc, char **argv)
 				return;
 			break;
 		case 's':
-			superblock = parse_ulong(optarg, argv[0],
-						 "superblock number", &err);
+			superblock = parse_ulonglong(optarg, argv[0],
+						     "superblock number",
+						     &err);
 			if (err)
 				return;
 			break;
@@ -269,8 +270,8 @@ void do_init_filesys(int argc, char **argv)
 		return;
 
 	memset(&param, 0, sizeof(struct ext2_super_block));
-	ext2fs_blocks_count_set(&param, parse_ulong(argv[2], argv[0],
-						    "blocks count", &err));
+	ext2fs_blocks_count_set(&param, parse_ulonglong(argv[2], argv[0],
+							"blocks count", &err));
 	if (err)
 		return;
 	retval = ext2fs_initialize(argv[1], 0, &param,
@@ -2050,7 +2051,7 @@ void do_bmap(int argc, char *argv[])
 	ino = string_to_inode(argv[1]);
 	if (!ino)
 		return;
-	blk = parse_ulong(argv[2], argv[0], "logical_block", &err);
+	blk = parse_ulonglong(argv[2], argv[0], "logical_block", &err);
 
 	errcode = ext2fs_bmap2(current_fs, ino, 0, 0, 0, blk, 0, &pblk);
 	if (errcode) {
@@ -2195,9 +2196,9 @@ void do_punch(int argc, char *argv[])
 	ino = string_to_inode(argv[1]);
 	if (!ino)
 		return;
-	start = parse_ulong(argv[2], argv[0], "logical_block", &err);
+	start = parse_ulonglong(argv[2], argv[0], "logical_block", &err);
 	if (argc == 4)
-		end = parse_ulong(argv[3], argv[0], "logical_block", &err);
+		end = parse_ulonglong(argv[3], argv[0], "logical_block", &err);
 	else
 		end = ~0;
 
@@ -2415,8 +2416,8 @@ int main(int argc, char **argv)
 						"block size", 0);
 			break;
 		case 's':
-			superblock = parse_ulong(optarg, argv[0],
-						 "superblock number", 0);
+			superblock = parse_ulonglong(optarg, argv[0],
+						     "superblock number", 0);
 			break;
 		case 'c':
 			catastrophic = 1;
diff --git a/debugfs/extent_inode.c b/debugfs/extent_inode.c
index 0bbc4c5..d5c4721 100644
--- a/debugfs/extent_inode.c
+++ b/debugfs/extent_inode.c
@@ -264,7 +264,8 @@ void do_replace_node(int argc, char *argv[])
 		return;
 	}
 
-	extent.e_lblk = parse_ulong(argv[1], argv[0], "logical block", &err);
+	extent.e_lblk = parse_ulonglong(argv[1], argv[0], "logical block",
+					&err);
 	if (err)
 		return;
 
@@ -272,7 +273,8 @@ void do_replace_node(int argc, char *argv[])
 	if (err)
 		return;
 
-	extent.e_pblk = parse_ulong(argv[3], argv[0], "logical block", &err);
+	extent.e_pblk = parse_ulonglong(argv[3], argv[0], "logical block",
+					&err);
 	if (err)
 		return;
 
@@ -338,8 +340,8 @@ void do_insert_node(int argc, char *argv[])
 		return;
 	}
 
-	extent.e_lblk = parse_ulong(argv[1], cmd,
-				    "logical block", &err);
+	extent.e_lblk = parse_ulonglong(argv[1], cmd,
+					"logical block", &err);
 	if (err)
 		return;
 
@@ -348,8 +350,8 @@ void do_insert_node(int argc, char *argv[])
 	if (err)
 		return;
 
-	extent.e_pblk = parse_ulong(argv[3], cmd,
-				    "pysical block", &err);
+	extent.e_pblk = parse_ulonglong(argv[3], cmd,
+					"physical block", &err);
 	if (err)
 		return;
 
@@ -366,8 +368,8 @@ void do_set_bmap(int argc, char **argv)
 	const char	*usage = "[--uninit] <lblk> <pblk>";
 	struct ext2fs_extent extent;
 	errcode_t	retval;
-	blk_t		logical;
-	blk_t		physical;
+	blk64_t		logical;
+	blk64_t		physical;
 	char		*cmd = argv[0];
 	int		flags = 0;
 	int		err;
@@ -387,18 +389,18 @@ void do_set_bmap(int argc, char **argv)
 		return;
 	}
 
-	logical = parse_ulong(argv[1], cmd,
+	logical = parse_ulonglong(argv[1], cmd,
 				    "logical block", &err);
 	if (err)
 		return;
 
-	physical = parse_ulong(argv[2], cmd,
+	physical = parse_ulonglong(argv[2], cmd,
 				    "physical block", &err);
 	if (err)
 		return;
 
 	retval = ext2fs_extent_set_bmap(current_handle, logical,
-					(blk64_t) physical, flags);
+					physical, flags);
 	if (retval) {
 		com_err(cmd, retval, 0);
 		return;
diff --git a/debugfs/util.c b/debugfs/util.c
index cf3a6c6..245d556 100644
--- a/debugfs/util.c
+++ b/debugfs/util.c
@@ -377,7 +377,7 @@ int common_block_args_process(int argc, char *argv[],
 	}
 
 	if (argc > 2) {
-		*count = parse_ulong(argv[2], argv[0], "count", &err);
+		*count = parse_ulonglong(argv[2], argv[0], "count", &err);
 		if (err)
 			return 1;
 	}

Re: [PATCH 16/31] debugfs: Handle 64bit block numbers

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:28:24 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 16/31] debugfs: Handle 64bit block numbers
> 
> debugfs should use strtoull (and equivalent wrappers) for reading block numbers
> from the command line.  "unsigned long" isn't wide enough on 32bit platforms.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  debugfs/debugfs.c      |   19 ++++++++++---------
>  debugfs/extent_inode.c |   24 +++++++++++++-----------
>  debugfs/util.c         |    2 +-
>  3 files changed, 24 insertions(+), 21 deletions(-)

Look good, but some nitpicks bellow.

> 
> 
> diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
> index da44ce4..60452c9 100644
> --- a/debugfs/debugfs.c
> +++ b/debugfs/debugfs.c
> @@ -167,8 +167,9 @@ void do_open_filesys(int argc, char **argv)
>  				return;
>  			break;
>  		case 's':
> -			superblock = parse_ulong(optarg, argv[0],
> -						 "superblock number", &err);
> +			superblock = parse_ulonglong(optarg, argv[0],
> +						     "superblock number",

this is not really a superlock number, but rather block number.

> +						     &err);
>  			if (err)
>  				return;
>  			break;
> @@ -269,8 +270,8 @@ void do_init_filesys(int argc, char **argv)
>  		return;
>  
>  	memset(&param, 0, sizeof(struct ext2_super_block));
> -	ext2fs_blocks_count_set(&param, parse_ulong(argv[2], argv[0],
> -						    "blocks count", &err));
> +	ext2fs_blocks_count_set(&param, parse_ulonglong(argv[2], argv[0],
> +							"blocks count", &err));
>  	if (err)
>  		return;
>  	retval = ext2fs_initialize(argv[1], 0, &param,
> @@ -2050,7 +2051,7 @@ void do_bmap(int argc, char *argv[])
>  	ino = string_to_inode(argv[1]);
>  	if (!ino)
>  		return;
> -	blk = parse_ulong(argv[2], argv[0], "logical_block", &err);
> +	blk = parse_ulonglong(argv[2], argv[0], "logical_block", &err);

Also I wonder whether we could use strtoblk() in cases like this ?
This would fix the "logical_block" string as well since it should be
either "logical block" or "block number".

>  
>  	errcode = ext2fs_bmap2(current_fs, ino, 0, 0, 0, blk, 0, &pblk);
>  	if (errcode) {
> @@ -2195,9 +2196,9 @@ void do_punch(int argc, char *argv[])
>  	ino = string_to_inode(argv[1]);
>  	if (!ino)
>  		return;
> -	start = parse_ulong(argv[2], argv[0], "logical_block", &err);
> +	start = parse_ulonglong(argv[2], argv[0], "logical_block", &err);
>  	if (argc == 4)
> -		end = parse_ulong(argv[3], argv[0], "logical_block", &err);
> +		end = parse_ulonglong(argv[3], argv[0], "logical_block", &err);
>  	else
>  		end = ~0;
>  
> @@ -2415,8 +2416,8 @@ int main(int argc, char **argv)
>  						"block size", 0);
>  			break;
>  		case 's':
> -			superblock = parse_ulong(optarg, argv[0],
> -						 "superblock number", 0);
> +			superblock = parse_ulonglong(optarg, argv[0],
> +						     "superblock number", 0);

same here

Thanks!
-Lukas

>  			break;
>  		case 'c':
>  			catastrophic = 1;
> diff --git a/debugfs/extent_inode.c b/debugfs/extent_inode.c
> index 0bbc4c5..d5c4721 100644
> --- a/debugfs/extent_inode.c
> +++ b/debugfs/extent_inode.c
> @@ -264,7 +264,8 @@ void do_replace_node(int argc, char *argv[])
>  		return;
>  	}
>  
> -	extent.e_lblk = parse_ulong(argv[1], argv[0], "logical block", &err);
> +	extent.e_lblk = parse_ulonglong(argv[1], argv[0], "logical block",
> +					&err);
>  	if (err)
>  		return;
>  
> @@ -272,7 +273,8 @@ void do_replace_node(int argc, char *argv[])
>  	if (err)
>  		return;
>  
> -	extent.e_pblk = parse_ulong(argv[3], argv[0], "logical block", &err);
> +	extent.e_pblk = parse_ulonglong(argv[3], argv[0], "logical block",
> +					&err);
>  	if (err)
>  		return;
>  
> @@ -338,8 +340,8 @@ void do_insert_node(int argc, char *argv[])
>  		return;
>  	}
>  
> -	extent.e_lblk = parse_ulong(argv[1], cmd,
> -				    "logical block", &err);
> +	extent.e_lblk = parse_ulonglong(argv[1], cmd,
> +					"logical block", &err);
>  	if (err)
>  		return;
>  
> @@ -348,8 +350,8 @@ void do_insert_node(int argc, char *argv[])
>  	if (err)
>  		return;
>  
> -	extent.e_pblk = parse_ulong(argv[3], cmd,
> -				    "pysical block", &err);
> +	extent.e_pblk = parse_ulonglong(argv[3], cmd,
> +					"physical block", &err);
>  	if (err)
>  		return;
>  
> @@ -366,8 +368,8 @@ void do_set_bmap(int argc, char **argv)
>  	const char	*usage = "[--uninit] <lblk> <pblk>";
>  	struct ext2fs_extent extent;
>  	errcode_t	retval;
> -	blk_t		logical;
> -	blk_t		physical;
> +	blk64_t		logical;
> +	blk64_t		physical;
>  	char		*cmd = argv[0];
>  	int		flags = 0;
>  	int		err;
> @@ -387,18 +389,18 @@ void do_set_bmap(int argc, char **argv)
>  		return;
>  	}
>  
> -	logical = parse_ulong(argv[1], cmd,
> +	logical = parse_ulonglong(argv[1], cmd,
>  				    "logical block", &err);
>  	if (err)
>  		return;
>  
> -	physical = parse_ulong(argv[2], cmd,
> +	physical = parse_ulonglong(argv[2], cmd,
>  				    "physical block", &err);
>  	if (err)
>  		return;
>  
>  	retval = ext2fs_extent_set_bmap(current_handle, logical,
> -					(blk64_t) physical, flags);
> +					physical, flags);
>  	if (retval) {
>  		com_err(cmd, retval, 0);
>  		return;
> diff --git a/debugfs/util.c b/debugfs/util.c
> index cf3a6c6..245d556 100644
> --- a/debugfs/util.c
> +++ b/debugfs/util.c
> @@ -377,7 +377,7 @@ int common_block_args_process(int argc, char *argv[],
>  	}
>  
>  	if (argc > 2) {
> -		*count = parse_ulong(argv[2], argv[0], "count", &err);
> +		*count = parse_ulonglong(argv[2], argv[0], "count", &err);
>  		if (err)
>  			return 1;
>  	}
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 16/31] debugfs: Handle 64bit block numbers

[Darrick J. Wong]

On Mon, Oct 07, 2013 at 05:49:45PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> 
> > Date: Mon, 30 Sep 2013 18:28:24 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 16/31] debugfs: Handle 64bit block numbers
> > 
> > debugfs should use strtoull (and equivalent wrappers) for reading block numbers
> > from the command line.  "unsigned long" isn't wide enough on 32bit platforms.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  debugfs/debugfs.c      |   19 ++++++++++---------
> >  debugfs/extent_inode.c |   24 +++++++++++++-----------
> >  debugfs/util.c         |    2 +-
> >  3 files changed, 24 insertions(+), 21 deletions(-)
> 
> Look good, but some nitpicks bellow.
> 
> > 
> > 
> > diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
> > index da44ce4..60452c9 100644
> > --- a/debugfs/debugfs.c
> > +++ b/debugfs/debugfs.c
> > @@ -167,8 +167,9 @@ void do_open_filesys(int argc, char **argv)
> >  				return;
> >  			break;
> >  		case 's':
> > -			superblock = parse_ulong(optarg, argv[0],
> > -						 "superblock number", &err);
> > +			superblock = parse_ulonglong(optarg, argv[0],
> > +						     "superblock number",
> 
> this is not really a superlock number, but rather block number.
> 
> > +						     &err);
> >  			if (err)
> >  				return;
> >  			break;
> > @@ -269,8 +270,8 @@ void do_init_filesys(int argc, char **argv)
> >  		return;
> >  
> >  	memset(&param, 0, sizeof(struct ext2_super_block));
> > -	ext2fs_blocks_count_set(&param, parse_ulong(argv[2], argv[0],
> > -						    "blocks count", &err));
> > +	ext2fs_blocks_count_set(&param, parse_ulonglong(argv[2], argv[0],
> > +							"blocks count", &err));
> >  	if (err)
> >  		return;
> >  	retval = ext2fs_initialize(argv[1], 0, &param,
> > @@ -2050,7 +2051,7 @@ void do_bmap(int argc, char *argv[])
> >  	ino = string_to_inode(argv[1]);
> >  	if (!ino)
> >  		return;
> > -	blk = parse_ulong(argv[2], argv[0], "logical_block", &err);
> > +	blk = parse_ulonglong(argv[2], argv[0], "logical_block", &err);
> 
> Also I wonder whether we could use strtoblk() in cases like this ?
> This would fix the "logical_block" string as well since it should be
> either "logical block" or "block number".

Sounds reasonable for all of these cases.  I'll update the patch.

--D
> 
> >  
> >  	errcode = ext2fs_bmap2(current_fs, ino, 0, 0, 0, blk, 0, &pblk);
> >  	if (errcode) {
> > @@ -2195,9 +2196,9 @@ void do_punch(int argc, char *argv[])
> >  	ino = string_to_inode(argv[1]);
> >  	if (!ino)
> >  		return;
> > -	start = parse_ulong(argv[2], argv[0], "logical_block", &err);
> > +	start = parse_ulonglong(argv[2], argv[0], "logical_block", &err);
> >  	if (argc == 4)
> > -		end = parse_ulong(argv[3], argv[0], "logical_block", &err);
> > +		end = parse_ulonglong(argv[3], argv[0], "logical_block", &err);
> >  	else
> >  		end = ~0;
> >  
> > @@ -2415,8 +2416,8 @@ int main(int argc, char **argv)
> >  						"block size", 0);
> >  			break;
> >  		case 's':
> > -			superblock = parse_ulong(optarg, argv[0],
> > -						 "superblock number", 0);
> > +			superblock = parse_ulonglong(optarg, argv[0],
> > +						     "superblock number", 0);
> 
> same here
> 
> Thanks!
> -Lukas
> 
> >  			break;
> >  		case 'c':
> >  			catastrophic = 1;
> > diff --git a/debugfs/extent_inode.c b/debugfs/extent_inode.c
> > index 0bbc4c5..d5c4721 100644
> > --- a/debugfs/extent_inode.c
> > +++ b/debugfs/extent_inode.c
> > @@ -264,7 +264,8 @@ void do_replace_node(int argc, char *argv[])
> >  		return;
> >  	}
> >  
> > -	extent.e_lblk = parse_ulong(argv[1], argv[0], "logical block", &err);
> > +	extent.e_lblk = parse_ulonglong(argv[1], argv[0], "logical block",
> > +					&err);
> >  	if (err)
> >  		return;
> >  
> > @@ -272,7 +273,8 @@ void do_replace_node(int argc, char *argv[])
> >  	if (err)
> >  		return;
> >  
> > -	extent.e_pblk = parse_ulong(argv[3], argv[0], "logical block", &err);
> > +	extent.e_pblk = parse_ulonglong(argv[3], argv[0], "logical block",
> > +					&err);
> >  	if (err)
> >  		return;
> >  
> > @@ -338,8 +340,8 @@ void do_insert_node(int argc, char *argv[])
> >  		return;
> >  	}
> >  
> > -	extent.e_lblk = parse_ulong(argv[1], cmd,
> > -				    "logical block", &err);
> > +	extent.e_lblk = parse_ulonglong(argv[1], cmd,
> > +					"logical block", &err);
> >  	if (err)
> >  		return;
> >  
> > @@ -348,8 +350,8 @@ void do_insert_node(int argc, char *argv[])
> >  	if (err)
> >  		return;
> >  
> > -	extent.e_pblk = parse_ulong(argv[3], cmd,
> > -				    "pysical block", &err);
> > +	extent.e_pblk = parse_ulonglong(argv[3], cmd,
> > +					"physical block", &err);
> >  	if (err)
> >  		return;
> >  
> > @@ -366,8 +368,8 @@ void do_set_bmap(int argc, char **argv)
> >  	const char	*usage = "[--uninit] <lblk> <pblk>";
> >  	struct ext2fs_extent extent;
> >  	errcode_t	retval;
> > -	blk_t		logical;
> > -	blk_t		physical;
> > +	blk64_t		logical;
> > +	blk64_t		physical;
> >  	char		*cmd = argv[0];
> >  	int		flags = 0;
> >  	int		err;
> > @@ -387,18 +389,18 @@ void do_set_bmap(int argc, char **argv)
> >  		return;
> >  	}
> >  
> > -	logical = parse_ulong(argv[1], cmd,
> > +	logical = parse_ulonglong(argv[1], cmd,
> >  				    "logical block", &err);
> >  	if (err)
> >  		return;
> >  
> > -	physical = parse_ulong(argv[2], cmd,
> > +	physical = parse_ulonglong(argv[2], cmd,
> >  				    "physical block", &err);
> >  	if (err)
> >  		return;
> >  
> >  	retval = ext2fs_extent_set_bmap(current_handle, logical,
> > -					(blk64_t) physical, flags);
> > +					physical, flags);
> >  	if (retval) {
> >  		com_err(cmd, retval, 0);
> >  		return;
> > diff --git a/debugfs/util.c b/debugfs/util.c
> > index cf3a6c6..245d556 100644
> > --- a/debugfs/util.c
> > +++ b/debugfs/util.c
> > @@ -377,7 +377,7 @@ int common_block_args_process(int argc, char *argv[],
> >  	}
> >  
> >  	if (argc > 2) {
> > -		*count = parse_ulong(argv[2], argv[0], "count", &err);
> > +		*count = parse_ulonglong(argv[2], argv[0], "count", &err);
> >  		if (err)
> >  			return 1;
> >  	}
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit data types

[Darrick J. Wong]

The curernt ext2_u32_list implementation manages a sorted sparse list of 32-bit
numbers.  This is currently used to collect directory inodes for rehashing in
e2fsck, and creating a list of bad blocks for mkfs.  However, block numbers are
now 64-bit, so we need to refactor the sparse list to be able to handle both 32
and 64-bit numbers.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/badblocks.c |  271 +++++++++++++++++++++++++++++++++++++-----------
 lib/ext2fs/ext2fs.h    |   23 +++-
 lib/ext2fs/ext2fsP.h   |   13 +-
 lib/ext2fs/inode.c     |   12 +-
 4 files changed, 237 insertions(+), 82 deletions(-)


diff --git a/lib/ext2fs/badblocks.c b/lib/ext2fs/badblocks.c
index 0f23983..4512243 100644
--- a/lib/ext2fs/badblocks.c
+++ b/lib/ext2fs/badblocks.c
@@ -27,42 +27,58 @@
 #include "ext2_fs.h"
 #include "ext2fsP.h"
 
+static void *LIST_ITEM(struct ext2_sparse_list *bb, unsigned int i)
+{
+	return bb->list + (i * bb->item_size);
+}
+
 /*
  * Helper function for making a badblocks list
  */
-static errcode_t make_u32_list(int size, int num, __u32 *list,
-			       ext2_u32_list *ret)
+static errcode_t sparse_list_make(unsigned int size, unsigned int num,
+				  unsigned int item_size,
+				  struct ext2_sparse_list *list,
+				  struct ext2_sparse_list **ret)
 {
-	ext2_u32_list	bb;
+	struct ext2_sparse_list *bb;
 	errcode_t	retval;
 
-	retval = ext2fs_get_mem(sizeof(struct ext2_struct_u32_list), &bb);
+	if (list && (list->magic != EXT2_ET_MAGIC_BADBLOCKS_LIST ||
+		     list->item_size != item_size)) {
+		return EXT2_ET_INVALID_ARGUMENT;
+	}
+
+	/* For now we only support 32 and 64bit unsigned */
+	if (item_size != sizeof(__u32) && item_size != sizeof(__u64))
+		return EXT2_ET_INVALID_ARGUMENT;
+
+	retval = ext2fs_get_mem(sizeof(struct ext2_sparse_list), &bb);
 	if (retval)
 		return retval;
-	memset(bb, 0, sizeof(struct ext2_struct_u32_list));
+	memset(bb, 0, sizeof(struct ext2_sparse_list));
 	bb->magic = EXT2_ET_MAGIC_BADBLOCKS_LIST;
 	bb->size = size ? size : 10;
 	bb->num = num;
-	retval = ext2fs_get_array(bb->size, sizeof(blk_t), &bb->list);
+	bb->item_size = item_size;
+	retval = ext2fs_get_array(bb->size, bb->item_size, &bb->list);
 	if (retval) {
 		ext2fs_free_mem(&bb);
 		return retval;
 	}
 	if (list)
-		memcpy(bb->list, list, bb->size * sizeof(blk_t));
+		memcpy(bb->list, list->list, bb->size * bb->item_size);
 	else
-		memset(bb->list, 0, bb->size * sizeof(blk_t));
+		memset(bb->list, 0, bb->size * bb->item_size);
 	*ret = bb;
 	return 0;
 }
 
-
 /*
  * This procedure creates an empty u32 list.
  */
 errcode_t ext2fs_u32_list_create(ext2_u32_list *ret, int size)
 {
-	return make_u32_list(size, 0, 0, ret);
+	return sparse_list_make(size, 0, sizeof(__u32), NULL, ret);
 }
 
 /*
@@ -70,28 +86,36 @@ errcode_t ext2fs_u32_list_create(ext2_u32_list *ret, int size)
  */
 errcode_t ext2fs_badblocks_list_create(ext2_badblocks_list *ret, int size)
 {
-	return make_u32_list(size, 0, 0, (ext2_badblocks_list *) ret);
+	return sparse_list_make(size, 0, sizeof(blk64_t), NULL,
+				(ext2_badblocks_list *) ret);
 }
 
 
 /*
  * This procedure copies a badblocks list
  */
-errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest)
+static errcode_t sparse_list_copy(struct ext2_sparse_list *src,
+			   struct ext2_sparse_list **dest)
 {
 	errcode_t	retval;
 
-	retval = make_u32_list(src->size, src->num, src->list, dest);
+	retval = sparse_list_make(src->size, src->num, src->item_size, src,
+				  dest);
 	if (retval)
 		return retval;
 	(*dest)->badblocks_flags = src->badblocks_flags;
 	return 0;
 }
 
+errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest)
+{
+	return sparse_list_copy(src, dest);
+}
+
 errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
 				ext2_badblocks_list *dest)
 {
-	return ext2fs_u32_copy((ext2_u32_list) src,
+	return sparse_list_copy((ext2_u32_list) src,
 			       (ext2_u32_list *) dest);
 }
 
@@ -105,64 +129,107 @@ errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
 /*
  * This procedure adds a block to a badblocks list.
  */
-errcode_t ext2fs_u32_list_add(ext2_u32_list bb, __u32 blk)
+static int compare32(const void *a, const void *b)
+{
+	__u32 i, j;
+	i = *(__u32 *)a;
+	j = *(__u32 *)b;
+
+	return i - j;
+}
+
+static int compare64(const void *a, const void *b)
+{
+	__u64 i, j;
+	i = *(__u64 *)a;
+	j = *(__u64 *)b;
+
+	return i - j;
+}
+
+static errcode_t sparse_list_add(struct ext2_sparse_list *bb, void *item)
 {
 	errcode_t	retval;
-	int		i, j;
+	int		i, j, k;
 	unsigned long	old_size;
+	int (*cmp)	(const void *a, const void *b);
 
 	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
 
 	if (bb->num >= bb->size) {
-		old_size = bb->size * sizeof(__u32);
+		old_size = bb->size * bb->item_size;
 		bb->size += 100;
-		retval = ext2fs_resize_mem(old_size, bb->size * sizeof(__u32),
+		retval = ext2fs_resize_mem(old_size, bb->size * bb->item_size,
 					   &bb->list);
 		if (retval) {
 			bb->size -= 100;
 			return retval;
 		}
 	}
-
+	if (bb->item_size == sizeof(__u32))
+		cmp = compare32;
+	else if (bb->item_size == sizeof(__u64))
+		cmp = compare64;
+	else
+		return EXT2_ET_INVALID_ARGUMENT;
 	/*
 	 * Add special case code for appending to the end of the list
 	 */
 	i = bb->num-1;
-	if ((bb->num != 0) && (bb->list[i] == blk))
+	if ((bb->num != 0) && cmp(LIST_ITEM(bb, i), item) == 0)
 		return 0;
-	if ((bb->num == 0) || (bb->list[i] < blk)) {
-		bb->list[bb->num++] = blk;
+	if ((bb->num == 0) || cmp(LIST_ITEM(bb, i), item) < 0) {
+		memcpy(LIST_ITEM(bb, bb->num++), item, bb->item_size);
 		return 0;
 	}
 
 	j = bb->num;
 	for (i=0; i < bb->num; i++) {
-		if (bb->list[i] == blk)
+		k = cmp(LIST_ITEM(bb, i), item);
+		if (k == 0)
 			return 0;
-		if (bb->list[i] > blk) {
+		if (k > 0) {
 			j = i;
 			break;
 		}
 	}
-	for (i=bb->num; i > j; i--)
-		bb->list[i] = bb->list[i-1];
-	bb->list[j] = blk;
+	memmove(LIST_ITEM(bb, i + 1), LIST_ITEM(bb, i),
+		(bb->num - j) * bb->item_size);
+	memcpy(LIST_ITEM(bb, j), item, bb->item_size);
 	bb->num++;
 	return 0;
 }
 
+errcode_t ext2fs_u32_list_add(ext2_u32_list bb, __u32 blk)
+{
+	return sparse_list_add(bb, &blk);
+}
+
+errcode_t ext2fs_badblocks_list_add2(ext2_badblocks_list bb, blk64_t blk)
+{
+	return sparse_list_add(bb, &blk);
+}
+
 errcode_t ext2fs_badblocks_list_add(ext2_badblocks_list bb, blk_t blk)
 {
-	return ext2fs_u32_list_add((ext2_u32_list) bb, (__u32) blk);
+	return ext2fs_badblocks_list_add2(bb, blk);
 }
 
 /*
  * This procedure finds a particular block is on a badblocks
  * list.
  */
-int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
+static int sparse_list_find(struct ext2_sparse_list *bb, void *item)
 {
-	int	low, high, mid;
+	int	low, high, mid, k;
+	int (*cmp)	(const void *a, const void *b);
+
+	if (bb->item_size == sizeof(__u32))
+		cmp = compare32;
+	else if (bb->item_size == sizeof(__u64))
+		cmp = compare64;
+	else
+		return EXT2_ET_INVALID_ARGUMENT;
 
 	if (bb->magic != EXT2_ET_MAGIC_BADBLOCKS_LIST)
 		return -1;
@@ -172,18 +239,19 @@ int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
 
 	low = 0;
 	high = bb->num-1;
-	if (blk == bb->list[low])
+	if (cmp(LIST_ITEM(bb, low), item) == 0)
 		return low;
-	if (blk == bb->list[high])
+	if (cmp(LIST_ITEM(bb, high), item) == 0)
 		return high;
 
 	while (low < high) {
 		mid = ((unsigned)low + (unsigned)high)/2;
 		if (mid == low || mid == high)
 			break;
-		if (blk == bb->list[mid])
+		k = cmp(item, LIST_ITEM(bb, mid));
+		if (k == 0)
 			return mid;
-		if (blk < bb->list[mid])
+		if (k < 0)
 			high = mid;
 		else
 			low = mid;
@@ -191,13 +259,26 @@ int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
 	return -1;
 }
 
+int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
+{
+	return sparse_list_find(bb, &blk);
+}
+
 /*
  * This procedure tests to see if a particular block is on a badblocks
  * list.
  */
 int ext2fs_u32_list_test(ext2_u32_list bb, __u32 blk)
 {
-	if (ext2fs_u32_list_find(bb, blk) < 0)
+	if (sparse_list_find(bb, &blk) < 0)
+		return 0;
+	else
+		return 1;
+}
+
+int ext2fs_badblocks_list_test2(ext2_badblocks_list bb, blk64_t blk)
+{
+	if (sparse_list_find(bb, &blk) < 0)
 		return 0;
 	else
 		return 1;
@@ -205,65 +286,83 @@ int ext2fs_u32_list_test(ext2_u32_list bb, __u32 blk)
 
 int ext2fs_badblocks_list_test(ext2_badblocks_list bb, blk_t blk)
 {
-	return ext2fs_u32_list_test((ext2_u32_list) bb, (__u32) blk);
+	return ext2fs_badblocks_list_test2(bb, blk);
 }
 
 
 /*
  * Remove a block from the badblock list
  */
-int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk)
+static int sparse_list_del(struct ext2_sparse_list *bb, void *item)
 {
-	int	remloc, i;
+	int	remloc;
 
 	if (bb->num == 0)
 		return -1;
 
-	remloc = ext2fs_u32_list_find(bb, blk);
+	remloc = sparse_list_find(bb, item);
 	if (remloc < 0)
 		return -1;
 
-	for (i = remloc ; i < bb->num-1; i++)
-		bb->list[i] = bb->list[i+1];
+	memmove(LIST_ITEM(bb, remloc), LIST_ITEM(bb, remloc + 1),
+		(bb->num - remloc - 1) * bb->item_size);
 	bb->num--;
 	return 0;
 }
 
+int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk)
+{
+	return sparse_list_del(bb, &blk);
+}
+
+void ext2fs_badblocks_list_del2(ext2_u32_list bb, blk64_t blk)
+{
+	sparse_list_del(bb, &blk);
+}
+
 void ext2fs_badblocks_list_del(ext2_u32_list bb, __u32 blk)
 {
-	ext2fs_u32_list_del(bb, blk);
+	ext2fs_badblocks_list_del(bb, blk);
 }
 
-errcode_t ext2fs_u32_list_iterate_begin(ext2_u32_list bb,
-					ext2_u32_iterate *ret)
+static errcode_t sparse_list_iterate_begin(struct ext2_sparse_list *bb,
+					   struct ext2_sparse_list_iterate **r)
 {
-	ext2_u32_iterate iter;
+	struct ext2_sparse_list_iterate *iter;
 	errcode_t		retval;
 
 	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
 
-	retval = ext2fs_get_mem(sizeof(struct ext2_struct_u32_iterate), &iter);
+	retval = ext2fs_get_mem(sizeof(struct ext2_sparse_list_iterate),
+				&iter);
 	if (retval)
 		return retval;
 
 	iter->magic = EXT2_ET_MAGIC_BADBLOCKS_ITERATE;
 	iter->bb = bb;
 	iter->ptr = 0;
-	*ret = iter;
+	*r = iter;
 	return 0;
 }
 
+errcode_t ext2fs_u32_list_iterate_begin(ext2_u32_list bb,
+					ext2_u32_iterate *ret)
+{
+	return sparse_list_iterate_begin(bb, ret);
+}
+
 errcode_t ext2fs_badblocks_list_iterate_begin(ext2_badblocks_list bb,
 					      ext2_badblocks_iterate *ret)
 {
-	return ext2fs_u32_list_iterate_begin((ext2_u32_list) bb,
-					      (ext2_u32_iterate *) ret);
+	return sparse_list_iterate_begin((ext2_u32_list) bb,
+					 (ext2_u32_iterate *) ret);
 }
 
 
-int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
+static int sparse_list_iterate(struct ext2_sparse_list_iterate *iter,
+			       void *item)
 {
-	ext2_u32_list	bb;
+	struct ext2_sparse_list	*bb;
 
 	if (iter->magic != EXT2_ET_MAGIC_BADBLOCKS_ITERATE)
 		return 0;
@@ -274,21 +373,35 @@ int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
 		return 0;
 
 	if (iter->ptr < bb->num) {
-		*blk = bb->list[iter->ptr++];
+		memcpy(item, LIST_ITEM(bb, iter->ptr++), bb->item_size);
 		return 1;
 	}
-	*blk = 0;
+	memset(item, 0, bb->item_size);
 	return 0;
 }
 
+int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
+{
+	return sparse_list_iterate(iter, blk);
+}
+
+int ext2fs_badblocks_list_iterate2(ext2_badblocks_iterate iter, blk64_t *blk)
+{
+	return sparse_list_iterate(iter, blk);
+}
+
 int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter, blk_t *blk)
 {
-	return ext2fs_u32_list_iterate((ext2_u32_iterate) iter,
-				       (__u32 *) blk);
+	blk64_t x;
+	int ret;
+
+	ret = ext2fs_badblocks_list_iterate2(iter, &x);
+	*blk = x;
+	return ret;
 }
 
 
-void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
+static void sparse_list_iterate_end(struct ext2_sparse_list_iterate *iter)
 {
 	if (!iter || (iter->magic != EXT2_ET_MAGIC_BADBLOCKS_ITERATE))
 		return;
@@ -297,13 +410,19 @@ void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
 	ext2fs_free_mem(&iter);
 }
 
+void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
+{
+	sparse_list_iterate_end(iter);
+}
+
 void ext2fs_badblocks_list_iterate_end(ext2_badblocks_iterate iter)
 {
-	ext2fs_u32_list_iterate_end((ext2_u32_iterate) iter);
+	sparse_list_iterate_end(iter);
 }
 
 
-int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
+static int sparse_list_equal(struct ext2_sparse_list *bb1,
+			     struct ext2_sparse_list *bb2)
 {
 	EXT2_CHECK_MAGIC(bb1, EXT2_ET_MAGIC_BADBLOCKS_LIST);
 	EXT2_CHECK_MAGIC(bb2, EXT2_ET_MAGIC_BADBLOCKS_LIST);
@@ -311,18 +430,46 @@ int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
 	if (bb1->num != bb2->num)
 		return 0;
 
-	if (memcmp(bb1->list, bb2->list, bb1->num * sizeof(blk_t)) != 0)
+	if (bb1->item_size != bb2->item_size)
+		return 0;
+
+	if (memcmp(bb1->list, bb2->list, bb1->num * bb1->item_size) != 0)
 		return 0;
 	return 1;
 }
 
+int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
+{
+	return sparse_list_equal(bb1, bb2);
+}
+
 int ext2fs_badblocks_equal(ext2_badblocks_list bb1, ext2_badblocks_list bb2)
 {
-	return ext2fs_u32_list_equal((ext2_u32_list) bb1,
-				     (ext2_u32_list) bb2);
+	return sparse_list_equal(bb1, bb2);
 }
 
-int ext2fs_u32_list_count(ext2_u32_list bb)
+static unsigned int sparse_list_count(struct ext2_sparse_list *bb)
 {
+	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
 	return bb->num;
 }
+
+int ext2fs_u32_list_count(ext2_u32_list bb)
+{
+	return sparse_list_count(bb);
+}
+
+unsigned int ext2fs_badblocks_count(ext2_badblocks_list bb)
+{
+	return sparse_list_count(bb);
+}
+
+blk64_t ext2fs_badblocks_get(ext2_badblocks_list bb, unsigned int i)
+{
+	blk64_t x;
+
+	if (i < 0 || i >= bb->num)
+		return 0;
+	memcpy(&x, LIST_ITEM(bb, i), bb->item_size);
+	return x;
+}
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index d5d8d03..b4ba421 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -111,15 +111,15 @@ typedef struct ext2fs_struct_generic_bitmap *ext2fs_block_bitmap;
  * Badblocks list definitions
  */
 
-typedef struct ext2_struct_u32_list *ext2_badblocks_list;
-typedef struct ext2_struct_u32_iterate *ext2_badblocks_iterate;
+typedef struct ext2_sparse_list *ext2_badblocks_list;
+typedef struct ext2_sparse_list_iterate *ext2_badblocks_iterate;
 
-typedef struct ext2_struct_u32_list *ext2_u32_list;
-typedef struct ext2_struct_u32_iterate *ext2_u32_iterate;
+typedef struct ext2_sparse_list *ext2_u32_list;
+typedef struct ext2_sparse_list_iterate *ext2_u32_iterate;
 
 /* old */
-typedef struct ext2_struct_u32_list *badblocks_list;
-typedef struct ext2_struct_u32_iterate *badblocks_iterate;
+typedef struct ext2_sparse_list *badblocks_list;
+typedef struct ext2_sparse_list_iterate *badblocks_iterate;
 
 #define BADBLOCKS_FLAG_DIRTY	1
 
@@ -702,6 +702,8 @@ extern int ext2fs_u32_list_iterate(ext2_u32_iterate iter, blk_t *blk);
 extern void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter);
 extern errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest);
 extern int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2);
+extern int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk);
+extern int ext2fs_u32_list_count(ext2_u32_list bb);
 
 extern errcode_t ext2fs_badblocks_list_create(ext2_badblocks_list *ret,
 					    int size);
@@ -709,7 +711,6 @@ extern errcode_t ext2fs_badblocks_list_add(ext2_badblocks_list bb,
 					   blk_t blk);
 extern int ext2fs_badblocks_list_test(ext2_badblocks_list bb,
 				    blk_t blk);
-extern int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk);
 extern void ext2fs_badblocks_list_del(ext2_u32_list bb, __u32 blk);
 extern errcode_t
 	ext2fs_badblocks_list_iterate_begin(ext2_badblocks_list bb,
@@ -721,7 +722,13 @@ extern errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
 				       ext2_badblocks_list *dest);
 extern int ext2fs_badblocks_equal(ext2_badblocks_list bb1,
 				  ext2_badblocks_list bb2);
-extern int ext2fs_u32_list_count(ext2_u32_list bb);
+extern errcode_t ext2fs_badblocks_list_add2(ext2_badblocks_list bb,
+					    blk64_t blk);
+extern int ext2fs_badblocks_list_test2(ext2_badblocks_list bb, blk64_t blk);
+extern void ext2fs_badblocks_list_del2(ext2_u32_list bb, blk64_t blk);
+extern int ext2fs_badblocks_list_iterate2(ext2_badblocks_iterate iter,
+					  blk64_t *blk);
+extern blk64_t ext2fs_badblocks_get(ext2_badblocks_list bb, unsigned int i);
 
 /* bb_compat */
 extern errcode_t badblocks_list_create(badblocks_list *ret, int size);
diff --git a/lib/ext2fs/ext2fsP.h b/lib/ext2fs/ext2fsP.h
index 80d2d0a..592527c 100644
--- a/lib/ext2fs/ext2fsP.h
+++ b/lib/ext2fs/ext2fsP.h
@@ -16,17 +16,18 @@
 /*
  * Badblocks list
  */
-struct ext2_struct_u32_list {
+struct ext2_sparse_list {
 	int	magic;
-	int	num;
-	int	size;
-	__u32	*list;
+	unsigned int	num;
+	unsigned int	size;
+	unsigned int	item_size;
+	void	*list;
 	int	badblocks_flags;
 };
 
-struct ext2_struct_u32_iterate {
+struct ext2_sparse_list_iterate {
 	int			magic;
-	ext2_u32_list		bb;
+	struct ext2_sparse_list	*bb;
 	int			ptr;
 };
 
diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
index 46c1c58..6579512 100644
--- a/lib/ext2fs/inode.c
+++ b/lib/ext2fs/inode.c
@@ -313,8 +313,8 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
 	 * is no longer the case.  If we run out of bad blocks, then
 	 * we don't need to do any more checking!
 	 */
-	while (blk > bb->list[scan->bad_block_ptr]) {
-		if (++scan->bad_block_ptr >= bb->num) {
+	while (blk > ext2fs_badblocks_get(bb, scan->bad_block_ptr)) {
+		if (++scan->bad_block_ptr >= ext2fs_badblocks_count(bb)) {
 			scan->scan_flags &= ~EXT2_SF_CHK_BADBLOCKS;
 			return 0;
 		}
@@ -328,10 +328,10 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
 	 * expense of a huge expense of code complexity, and for an
 	 * uncommon case at that.)
 	 */
-	if (blk == bb->list[scan->bad_block_ptr]) {
+	if (blk == ext2fs_badblocks_get(bb, scan->bad_block_ptr)) {
 		scan->scan_flags |= EXT2_SF_BAD_INODE_BLK;
 		*num_blocks = 1;
-		if (++scan->bad_block_ptr >= bb->num)
+		if (++scan->bad_block_ptr >= ext2fs_badblocks_count(bb))
 			scan->scan_flags &= ~EXT2_SF_CHK_BADBLOCKS;
 		return 0;
 	}
@@ -342,8 +342,8 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
 	 * don't read in the bad block.  (Then the next block to read
 	 * will be the bad block, which is handled in the above case.)
 	 */
-	if ((blk + *num_blocks) > bb->list[scan->bad_block_ptr])
-		*num_blocks = (int) (bb->list[scan->bad_block_ptr] - blk);
+	if ((blk + *num_blocks) > ext2fs_badblocks_get(bb, scan->bad_block_ptr))
+		*num_blocks = (int)(ext2fs_badblocks_get(bb, scan->bad_block_ptr) - blk);
 
 	return 0;
 }

Re: [PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit data types

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:28:31 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit
>     data types
> 
> The curernt ext2_u32_list implementation manages a sorted sparse list of 32-bit
> numbers.  This is currently used to collect directory inodes for rehashing in
> e2fsck, and creating a list of bad blocks for mkfs.  However, block numbers are
> now 64-bit, so we need to refactor the sparse list to be able to handle both 32
> and 64-bit numbers.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/badblocks.c |  271 +++++++++++++++++++++++++++++++++++++-----------
>  lib/ext2fs/ext2fs.h    |   23 +++-
>  lib/ext2fs/ext2fsP.h   |   13 +-
>  lib/ext2fs/inode.c     |   12 +-
>  4 files changed, 237 insertions(+), 82 deletions(-)
> 

-- snip --

>  
> @@ -105,64 +129,107 @@ errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
>  /*
>   * This procedure adds a block to a badblocks list.
>   */
> -errcode_t ext2fs_u32_list_add(ext2_u32_list bb, __u32 blk)
> +static int compare32(const void *a, const void *b)
> +{
> +	__u32 i, j;
> +	i = *(__u32 *)a;
> +	j = *(__u32 *)b;
> +
> +	return i - j;
> +}
> +
> +static int compare64(const void *a, const void *b)
> +{
> +	__u64 i, j;
> +	i = *(__u64 *)a;
> +	j = *(__u64 *)b;
> +
> +	return i - j;

Hmm this does not seem right. What if:

i = 4294967296
j = 8589934592

then you would return 0 even though the two block numbers differs.
The problem here is that even though you compare __u64 you return
int.

Thanks!
-Lukas

> +}
> +
> +static errcode_t sparse_list_add(struct ext2_sparse_list *bb, void *item)
>  {
>  	errcode_t	retval;
> -	int		i, j;
> +	int		i, j, k;
>  	unsigned long	old_size;
> +	int (*cmp)	(const void *a, const void *b);
>  
>  	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
>  
>  	if (bb->num >= bb->size) {
> -		old_size = bb->size * sizeof(__u32);
> +		old_size = bb->size * bb->item_size;
>  		bb->size += 100;
> -		retval = ext2fs_resize_mem(old_size, bb->size * sizeof(__u32),
> +		retval = ext2fs_resize_mem(old_size, bb->size * bb->item_size,
>  					   &bb->list);
>  		if (retval) {
>  			bb->size -= 100;
>  			return retval;
>  		}
>  	}
> -
> +	if (bb->item_size == sizeof(__u32))
> +		cmp = compare32;
> +	else if (bb->item_size == sizeof(__u64))
> +		cmp = compare64;
> +	else
> +		return EXT2_ET_INVALID_ARGUMENT;
>  	/*
>  	 * Add special case code for appending to the end of the list
>  	 */
>  	i = bb->num-1;
> -	if ((bb->num != 0) && (bb->list[i] == blk))
> +	if ((bb->num != 0) && cmp(LIST_ITEM(bb, i), item) == 0)
>  		return 0;
> -	if ((bb->num == 0) || (bb->list[i] < blk)) {
> -		bb->list[bb->num++] = blk;
> +	if ((bb->num == 0) || cmp(LIST_ITEM(bb, i), item) < 0) {
> +		memcpy(LIST_ITEM(bb, bb->num++), item, bb->item_size);
>  		return 0;
>  	}
>  
>  	j = bb->num;
>  	for (i=0; i < bb->num; i++) {
> -		if (bb->list[i] == blk)
> +		k = cmp(LIST_ITEM(bb, i), item);
> +		if (k == 0)
>  			return 0;
> -		if (bb->list[i] > blk) {
> +		if (k > 0) {

Since you're possibly comparing 64bit numbers and return int this
would not work very well at all.

>  			j = i;
>  			break;
>  		}
>  	}
> -	for (i=bb->num; i > j; i--)
> -		bb->list[i] = bb->list[i-1];
> -	bb->list[j] = blk;
> +	memmove(LIST_ITEM(bb, i + 1), LIST_ITEM(bb, i),
> +		(bb->num - j) * bb->item_size);
> +	memcpy(LIST_ITEM(bb, j), item, bb->item_size);

What is the difference between j and i here ? I've got the feeling
that both should be the same ?

Thanks!
-Lukas

>  	bb->num++;
>  	return 0;
>  }
>  
> +errcode_t ext2fs_u32_list_add(ext2_u32_list bb, __u32 blk)
> +{
> +	return sparse_list_add(bb, &blk);
> +}
> +
> +errcode_t ext2fs_badblocks_list_add2(ext2_badblocks_list bb, blk64_t blk)
> +{
> +	return sparse_list_add(bb, &blk);
> +}
> +
>  errcode_t ext2fs_badblocks_list_add(ext2_badblocks_list bb, blk_t blk)
>  {
> -	return ext2fs_u32_list_add((ext2_u32_list) bb, (__u32) blk);
> +	return ext2fs_badblocks_list_add2(bb, blk);
>  }
>  
>  /*
>   * This procedure finds a particular block is on a badblocks
>   * list.
>   */
> -int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
> +static int sparse_list_find(struct ext2_sparse_list *bb, void *item)
>  {
> -	int	low, high, mid;
> +	int	low, high, mid, k;
> +	int (*cmp)	(const void *a, const void *b);
> +
> +	if (bb->item_size == sizeof(__u32))
> +		cmp = compare32;
> +	else if (bb->item_size == sizeof(__u64))
> +		cmp = compare64;
> +	else
> +		return EXT2_ET_INVALID_ARGUMENT;
>  
>  	if (bb->magic != EXT2_ET_MAGIC_BADBLOCKS_LIST)
>  		return -1;
> @@ -172,18 +239,19 @@ int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
>  
>  	low = 0;
>  	high = bb->num-1;
> -	if (blk == bb->list[low])
> +	if (cmp(LIST_ITEM(bb, low), item) == 0)
>  		return low;
> -	if (blk == bb->list[high])
> +	if (cmp(LIST_ITEM(bb, high), item) == 0)
>  		return high;
>  
>  	while (low < high) {
>  		mid = ((unsigned)low + (unsigned)high)/2;
>  		if (mid == low || mid == high)
>  			break;
> -		if (blk == bb->list[mid])
> +		k = cmp(item, LIST_ITEM(bb, mid));
> +		if (k == 0)
>  			return mid;
> -		if (blk < bb->list[mid])
> +		if (k < 0)
>  			high = mid;
>  		else
>  			low = mid;
> @@ -191,13 +259,26 @@ int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
>  	return -1;
>  }
>  
> +int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
> +{
> +	return sparse_list_find(bb, &blk);
> +}
> +
>  /*
>   * This procedure tests to see if a particular block is on a badblocks
>   * list.
>   */
>  int ext2fs_u32_list_test(ext2_u32_list bb, __u32 blk)
>  {
> -	if (ext2fs_u32_list_find(bb, blk) < 0)
> +	if (sparse_list_find(bb, &blk) < 0)
> +		return 0;
> +	else
> +		return 1;
> +}
> +
> +int ext2fs_badblocks_list_test2(ext2_badblocks_list bb, blk64_t blk)
> +{
> +	if (sparse_list_find(bb, &blk) < 0)
>  		return 0;
>  	else
>  		return 1;
> @@ -205,65 +286,83 @@ int ext2fs_u32_list_test(ext2_u32_list bb, __u32 blk)
>  
>  int ext2fs_badblocks_list_test(ext2_badblocks_list bb, blk_t blk)
>  {
> -	return ext2fs_u32_list_test((ext2_u32_list) bb, (__u32) blk);
> +	return ext2fs_badblocks_list_test2(bb, blk);
>  }
>  
>  
>  /*
>   * Remove a block from the badblock list
>   */
> -int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk)
> +static int sparse_list_del(struct ext2_sparse_list *bb, void *item)
>  {
> -	int	remloc, i;
> +	int	remloc;
>  
>  	if (bb->num == 0)
>  		return -1;
>  
> -	remloc = ext2fs_u32_list_find(bb, blk);
> +	remloc = sparse_list_find(bb, item);
>  	if (remloc < 0)
>  		return -1;
>  
> -	for (i = remloc ; i < bb->num-1; i++)
> -		bb->list[i] = bb->list[i+1];
> +	memmove(LIST_ITEM(bb, remloc), LIST_ITEM(bb, remloc + 1),
> +		(bb->num - remloc - 1) * bb->item_size);
>  	bb->num--;
>  	return 0;
>  }
>  
> +int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk)
> +{
> +	return sparse_list_del(bb, &blk);
> +}
> +
> +void ext2fs_badblocks_list_del2(ext2_u32_list bb, blk64_t blk)
> +{
> +	sparse_list_del(bb, &blk);
> +}
> +
>  void ext2fs_badblocks_list_del(ext2_u32_list bb, __u32 blk)
>  {
> -	ext2fs_u32_list_del(bb, blk);
> +	ext2fs_badblocks_list_del(bb, blk);
>  }
>  
> -errcode_t ext2fs_u32_list_iterate_begin(ext2_u32_list bb,
> -					ext2_u32_iterate *ret)
> +static errcode_t sparse_list_iterate_begin(struct ext2_sparse_list *bb,
> +					   struct ext2_sparse_list_iterate **r)
>  {
> -	ext2_u32_iterate iter;
> +	struct ext2_sparse_list_iterate *iter;
>  	errcode_t		retval;
>  
>  	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
>  
> -	retval = ext2fs_get_mem(sizeof(struct ext2_struct_u32_iterate), &iter);
> +	retval = ext2fs_get_mem(sizeof(struct ext2_sparse_list_iterate),
> +				&iter);
>  	if (retval)
>  		return retval;
>  
>  	iter->magic = EXT2_ET_MAGIC_BADBLOCKS_ITERATE;
>  	iter->bb = bb;
>  	iter->ptr = 0;
> -	*ret = iter;
> +	*r = iter;
>  	return 0;
>  }
>  
> +errcode_t ext2fs_u32_list_iterate_begin(ext2_u32_list bb,
> +					ext2_u32_iterate *ret)
> +{
> +	return sparse_list_iterate_begin(bb, ret);
> +}
> +
>  errcode_t ext2fs_badblocks_list_iterate_begin(ext2_badblocks_list bb,
>  					      ext2_badblocks_iterate *ret)
>  {
> -	return ext2fs_u32_list_iterate_begin((ext2_u32_list) bb,
> -					      (ext2_u32_iterate *) ret);
> +	return sparse_list_iterate_begin((ext2_u32_list) bb,
> +					 (ext2_u32_iterate *) ret);
>  }
>  
>  
> -int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
> +static int sparse_list_iterate(struct ext2_sparse_list_iterate *iter,
> +			       void *item)
>  {
> -	ext2_u32_list	bb;
> +	struct ext2_sparse_list	*bb;
>  
>  	if (iter->magic != EXT2_ET_MAGIC_BADBLOCKS_ITERATE)
>  		return 0;
> @@ -274,21 +373,35 @@ int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
>  		return 0;
>  
>  	if (iter->ptr < bb->num) {
> -		*blk = bb->list[iter->ptr++];
> +		memcpy(item, LIST_ITEM(bb, iter->ptr++), bb->item_size);
>  		return 1;
>  	}
> -	*blk = 0;
> +	memset(item, 0, bb->item_size);
>  	return 0;
>  }
>  
> +int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
> +{
> +	return sparse_list_iterate(iter, blk);
> +}
> +
> +int ext2fs_badblocks_list_iterate2(ext2_badblocks_iterate iter, blk64_t *blk)
> +{
> +	return sparse_list_iterate(iter, blk);
> +}
> +
>  int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter, blk_t *blk)
>  {
> -	return ext2fs_u32_list_iterate((ext2_u32_iterate) iter,
> -				       (__u32 *) blk);
> +	blk64_t x;
> +	int ret;
> +
> +	ret = ext2fs_badblocks_list_iterate2(iter, &x);
> +	*blk = x;
> +	return ret;
>  }
>  
>  
> -void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
> +static void sparse_list_iterate_end(struct ext2_sparse_list_iterate *iter)
>  {
>  	if (!iter || (iter->magic != EXT2_ET_MAGIC_BADBLOCKS_ITERATE))
>  		return;
> @@ -297,13 +410,19 @@ void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
>  	ext2fs_free_mem(&iter);
>  }
>  
> +void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
> +{
> +	sparse_list_iterate_end(iter);
> +}
> +
>  void ext2fs_badblocks_list_iterate_end(ext2_badblocks_iterate iter)
>  {
> -	ext2fs_u32_list_iterate_end((ext2_u32_iterate) iter);
> +	sparse_list_iterate_end(iter);
>  }
>  
>  
> -int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
> +static int sparse_list_equal(struct ext2_sparse_list *bb1,
> +			     struct ext2_sparse_list *bb2)
>  {
>  	EXT2_CHECK_MAGIC(bb1, EXT2_ET_MAGIC_BADBLOCKS_LIST);
>  	EXT2_CHECK_MAGIC(bb2, EXT2_ET_MAGIC_BADBLOCKS_LIST);
> @@ -311,18 +430,46 @@ int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
>  	if (bb1->num != bb2->num)
>  		return 0;
>  
> -	if (memcmp(bb1->list, bb2->list, bb1->num * sizeof(blk_t)) != 0)
> +	if (bb1->item_size != bb2->item_size)
> +		return 0;
> +
> +	if (memcmp(bb1->list, bb2->list, bb1->num * bb1->item_size) != 0)
>  		return 0;
>  	return 1;
>  }
>  
> +int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
> +{
> +	return sparse_list_equal(bb1, bb2);
> +}
> +
>  int ext2fs_badblocks_equal(ext2_badblocks_list bb1, ext2_badblocks_list bb2)
>  {
> -	return ext2fs_u32_list_equal((ext2_u32_list) bb1,
> -				     (ext2_u32_list) bb2);
> +	return sparse_list_equal(bb1, bb2);
>  }
>  
> -int ext2fs_u32_list_count(ext2_u32_list bb)
> +static unsigned int sparse_list_count(struct ext2_sparse_list *bb)
>  {
> +	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
>  	return bb->num;
>  }
> +
> +int ext2fs_u32_list_count(ext2_u32_list bb)
> +{
> +	return sparse_list_count(bb);
> +}
> +
> +unsigned int ext2fs_badblocks_count(ext2_badblocks_list bb)
> +{
> +	return sparse_list_count(bb);
> +}
> +
> +blk64_t ext2fs_badblocks_get(ext2_badblocks_list bb, unsigned int i)
> +{
> +	blk64_t x;
> +
> +	if (i < 0 || i >= bb->num)
> +		return 0;
> +	memcpy(&x, LIST_ITEM(bb, i), bb->item_size);
> +	return x;
> +}
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index d5d8d03..b4ba421 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -111,15 +111,15 @@ typedef struct ext2fs_struct_generic_bitmap *ext2fs_block_bitmap;
>   * Badblocks list definitions
>   */
>  
> -typedef struct ext2_struct_u32_list *ext2_badblocks_list;
> -typedef struct ext2_struct_u32_iterate *ext2_badblocks_iterate;
> +typedef struct ext2_sparse_list *ext2_badblocks_list;
> +typedef struct ext2_sparse_list_iterate *ext2_badblocks_iterate;
>  
> -typedef struct ext2_struct_u32_list *ext2_u32_list;
> -typedef struct ext2_struct_u32_iterate *ext2_u32_iterate;
> +typedef struct ext2_sparse_list *ext2_u32_list;
> +typedef struct ext2_sparse_list_iterate *ext2_u32_iterate;
>  
>  /* old */
> -typedef struct ext2_struct_u32_list *badblocks_list;
> -typedef struct ext2_struct_u32_iterate *badblocks_iterate;
> +typedef struct ext2_sparse_list *badblocks_list;
> +typedef struct ext2_sparse_list_iterate *badblocks_iterate;
>  
>  #define BADBLOCKS_FLAG_DIRTY	1
>  
> @@ -702,6 +702,8 @@ extern int ext2fs_u32_list_iterate(ext2_u32_iterate iter, blk_t *blk);
>  extern void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter);
>  extern errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest);
>  extern int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2);
> +extern int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk);
> +extern int ext2fs_u32_list_count(ext2_u32_list bb);
>  
>  extern errcode_t ext2fs_badblocks_list_create(ext2_badblocks_list *ret,
>  					    int size);
> @@ -709,7 +711,6 @@ extern errcode_t ext2fs_badblocks_list_add(ext2_badblocks_list bb,
>  					   blk_t blk);
>  extern int ext2fs_badblocks_list_test(ext2_badblocks_list bb,
>  				    blk_t blk);
> -extern int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk);
>  extern void ext2fs_badblocks_list_del(ext2_u32_list bb, __u32 blk);
>  extern errcode_t
>  	ext2fs_badblocks_list_iterate_begin(ext2_badblocks_list bb,
> @@ -721,7 +722,13 @@ extern errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
>  				       ext2_badblocks_list *dest);
>  extern int ext2fs_badblocks_equal(ext2_badblocks_list bb1,
>  				  ext2_badblocks_list bb2);
> -extern int ext2fs_u32_list_count(ext2_u32_list bb);
> +extern errcode_t ext2fs_badblocks_list_add2(ext2_badblocks_list bb,
> +					    blk64_t blk);
> +extern int ext2fs_badblocks_list_test2(ext2_badblocks_list bb, blk64_t blk);
> +extern void ext2fs_badblocks_list_del2(ext2_u32_list bb, blk64_t blk);
> +extern int ext2fs_badblocks_list_iterate2(ext2_badblocks_iterate iter,
> +					  blk64_t *blk);
> +extern blk64_t ext2fs_badblocks_get(ext2_badblocks_list bb, unsigned int i);
>  
>  /* bb_compat */
>  extern errcode_t badblocks_list_create(badblocks_list *ret, int size);
> diff --git a/lib/ext2fs/ext2fsP.h b/lib/ext2fs/ext2fsP.h
> index 80d2d0a..592527c 100644
> --- a/lib/ext2fs/ext2fsP.h
> +++ b/lib/ext2fs/ext2fsP.h
> @@ -16,17 +16,18 @@
>  /*
>   * Badblocks list
>   */
> -struct ext2_struct_u32_list {
> +struct ext2_sparse_list {
>  	int	magic;
> -	int	num;
> -	int	size;
> -	__u32	*list;
> +	unsigned int	num;
> +	unsigned int	size;
> +	unsigned int	item_size;
> +	void	*list;
>  	int	badblocks_flags;
>  };
>  
> -struct ext2_struct_u32_iterate {
> +struct ext2_sparse_list_iterate {
>  	int			magic;
> -	ext2_u32_list		bb;
> +	struct ext2_sparse_list	*bb;
>  	int			ptr;
>  };
>  
> diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
> index 46c1c58..6579512 100644
> --- a/lib/ext2fs/inode.c
> +++ b/lib/ext2fs/inode.c
> @@ -313,8 +313,8 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
>  	 * is no longer the case.  If we run out of bad blocks, then
>  	 * we don't need to do any more checking!
>  	 */
> -	while (blk > bb->list[scan->bad_block_ptr]) {
> -		if (++scan->bad_block_ptr >= bb->num) {
> +	while (blk > ext2fs_badblocks_get(bb, scan->bad_block_ptr)) {
> +		if (++scan->bad_block_ptr >= ext2fs_badblocks_count(bb)) {
>  			scan->scan_flags &= ~EXT2_SF_CHK_BADBLOCKS;
>  			return 0;
>  		}
> @@ -328,10 +328,10 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
>  	 * expense of a huge expense of code complexity, and for an
>  	 * uncommon case at that.)
>  	 */
> -	if (blk == bb->list[scan->bad_block_ptr]) {
> +	if (blk == ext2fs_badblocks_get(bb, scan->bad_block_ptr)) {
>  		scan->scan_flags |= EXT2_SF_BAD_INODE_BLK;
>  		*num_blocks = 1;
> -		if (++scan->bad_block_ptr >= bb->num)
> +		if (++scan->bad_block_ptr >= ext2fs_badblocks_count(bb))
>  			scan->scan_flags &= ~EXT2_SF_CHK_BADBLOCKS;
>  		return 0;
>  	}
> @@ -342,8 +342,8 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
>  	 * don't read in the bad block.  (Then the next block to read
>  	 * will be the bad block, which is handled in the above case.)
>  	 */
> -	if ((blk + *num_blocks) > bb->list[scan->bad_block_ptr])
> -		*num_blocks = (int) (bb->list[scan->bad_block_ptr] - blk);
> +	if ((blk + *num_blocks) > ext2fs_badblocks_get(bb, scan->bad_block_ptr))
> +		*num_blocks = (int)(ext2fs_badblocks_get(bb, scan->bad_block_ptr) - blk);
>  
>  	return 0;
>  }
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit data types

[Darrick J. Wong]

On Thu, Oct 10, 2013 at 04:46:23PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> 
> > Date: Mon, 30 Sep 2013 18:28:31 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit
> >     data types
> > 
> > The curernt ext2_u32_list implementation manages a sorted sparse list of 32-bit
> > numbers.  This is currently used to collect directory inodes for rehashing in
> > e2fsck, and creating a list of bad blocks for mkfs.  However, block numbers are
> > now 64-bit, so we need to refactor the sparse list to be able to handle both 32
> > and 64-bit numbers.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  lib/ext2fs/badblocks.c |  271 +++++++++++++++++++++++++++++++++++++-----------
> >  lib/ext2fs/ext2fs.h    |   23 +++-
> >  lib/ext2fs/ext2fsP.h   |   13 +-
> >  lib/ext2fs/inode.c     |   12 +-
> >  4 files changed, 237 insertions(+), 82 deletions(-)
> > 
> 
> -- snip --
> 
> >  
> > @@ -105,64 +129,107 @@ errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
> >  /*
> >   * This procedure adds a block to a badblocks list.
> >   */
> > -errcode_t ext2fs_u32_list_add(ext2_u32_list bb, __u32 blk)
> > +static int compare32(const void *a, const void *b)
> > +{
> > +	__u32 i, j;
> > +	i = *(__u32 *)a;
> > +	j = *(__u32 *)b;
> > +
> > +	return i - j;
> > +}
> > +
> > +static int compare64(const void *a, const void *b)
> > +{
> > +	__u64 i, j;
> > +	i = *(__u64 *)a;
> > +	j = *(__u64 *)b;
> > +
> > +	return i - j;
> 
> Hmm this does not seem right. What if:
> 
> i = 4294967296
> j = 8589934592
> 
> then you would return 0 even though the two block numbers differs.
> The problem here is that even though you compare __u64 you return
> int.

Oops. :/

I think Ted would rather not deal with 64bit badblocks, so I think I'm
withdrawing patches 17-19 anyway.

--D
> 
> Thanks!
> -Lukas
> 
> > +}
> > +
> > +static errcode_t sparse_list_add(struct ext2_sparse_list *bb, void *item)
> >  {
> >  	errcode_t	retval;
> > -	int		i, j;
> > +	int		i, j, k;
> >  	unsigned long	old_size;
> > +	int (*cmp)	(const void *a, const void *b);
> >  
> >  	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
> >  
> >  	if (bb->num >= bb->size) {
> > -		old_size = bb->size * sizeof(__u32);
> > +		old_size = bb->size * bb->item_size;
> >  		bb->size += 100;
> > -		retval = ext2fs_resize_mem(old_size, bb->size * sizeof(__u32),
> > +		retval = ext2fs_resize_mem(old_size, bb->size * bb->item_size,
> >  					   &bb->list);
> >  		if (retval) {
> >  			bb->size -= 100;
> >  			return retval;
> >  		}
> >  	}
> > -
> > +	if (bb->item_size == sizeof(__u32))
> > +		cmp = compare32;
> > +	else if (bb->item_size == sizeof(__u64))
> > +		cmp = compare64;
> > +	else
> > +		return EXT2_ET_INVALID_ARGUMENT;
> >  	/*
> >  	 * Add special case code for appending to the end of the list
> >  	 */
> >  	i = bb->num-1;
> > -	if ((bb->num != 0) && (bb->list[i] == blk))
> > +	if ((bb->num != 0) && cmp(LIST_ITEM(bb, i), item) == 0)
> >  		return 0;
> > -	if ((bb->num == 0) || (bb->list[i] < blk)) {
> > -		bb->list[bb->num++] = blk;
> > +	if ((bb->num == 0) || cmp(LIST_ITEM(bb, i), item) < 0) {
> > +		memcpy(LIST_ITEM(bb, bb->num++), item, bb->item_size);
> >  		return 0;
> >  	}
> >  
> >  	j = bb->num;
> >  	for (i=0; i < bb->num; i++) {
> > -		if (bb->list[i] == blk)
> > +		k = cmp(LIST_ITEM(bb, i), item);
> > +		if (k == 0)
> >  			return 0;
> > -		if (bb->list[i] > blk) {
> > +		if (k > 0) {
> 
> Since you're possibly comparing 64bit numbers and return int this
> would not work very well at all.
> 
> >  			j = i;
> >  			break;
> >  		}
> >  	}
> > -	for (i=bb->num; i > j; i--)
> > -		bb->list[i] = bb->list[i-1];
> > -	bb->list[j] = blk;
> > +	memmove(LIST_ITEM(bb, i + 1), LIST_ITEM(bb, i),
> > +		(bb->num - j) * bb->item_size);
> > +	memcpy(LIST_ITEM(bb, j), item, bb->item_size);
> 
> What is the difference between j and i here ? I've got the feeling
> that both should be the same ?
> 
> Thanks!
> -Lukas
> 
> >  	bb->num++;
> >  	return 0;
> >  }
> >  
> > +errcode_t ext2fs_u32_list_add(ext2_u32_list bb, __u32 blk)
> > +{
> > +	return sparse_list_add(bb, &blk);
> > +}
> > +
> > +errcode_t ext2fs_badblocks_list_add2(ext2_badblocks_list bb, blk64_t blk)
> > +{
> > +	return sparse_list_add(bb, &blk);
> > +}
> > +
> >  errcode_t ext2fs_badblocks_list_add(ext2_badblocks_list bb, blk_t blk)
> >  {
> > -	return ext2fs_u32_list_add((ext2_u32_list) bb, (__u32) blk);
> > +	return ext2fs_badblocks_list_add2(bb, blk);
> >  }
> >  
> >  /*
> >   * This procedure finds a particular block is on a badblocks
> >   * list.
> >   */
> > -int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
> > +static int sparse_list_find(struct ext2_sparse_list *bb, void *item)
> >  {
> > -	int	low, high, mid;
> > +	int	low, high, mid, k;
> > +	int (*cmp)	(const void *a, const void *b);
> > +
> > +	if (bb->item_size == sizeof(__u32))
> > +		cmp = compare32;
> > +	else if (bb->item_size == sizeof(__u64))
> > +		cmp = compare64;
> > +	else
> > +		return EXT2_ET_INVALID_ARGUMENT;
> >  
> >  	if (bb->magic != EXT2_ET_MAGIC_BADBLOCKS_LIST)
> >  		return -1;
> > @@ -172,18 +239,19 @@ int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
> >  
> >  	low = 0;
> >  	high = bb->num-1;
> > -	if (blk == bb->list[low])
> > +	if (cmp(LIST_ITEM(bb, low), item) == 0)
> >  		return low;
> > -	if (blk == bb->list[high])
> > +	if (cmp(LIST_ITEM(bb, high), item) == 0)
> >  		return high;
> >  
> >  	while (low < high) {
> >  		mid = ((unsigned)low + (unsigned)high)/2;
> >  		if (mid == low || mid == high)
> >  			break;
> > -		if (blk == bb->list[mid])
> > +		k = cmp(item, LIST_ITEM(bb, mid));
> > +		if (k == 0)
> >  			return mid;
> > -		if (blk < bb->list[mid])
> > +		if (k < 0)
> >  			high = mid;
> >  		else
> >  			low = mid;
> > @@ -191,13 +259,26 @@ int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
> >  	return -1;
> >  }
> >  
> > +int ext2fs_u32_list_find(ext2_u32_list bb, __u32 blk)
> > +{
> > +	return sparse_list_find(bb, &blk);
> > +}
> > +
> >  /*
> >   * This procedure tests to see if a particular block is on a badblocks
> >   * list.
> >   */
> >  int ext2fs_u32_list_test(ext2_u32_list bb, __u32 blk)
> >  {
> > -	if (ext2fs_u32_list_find(bb, blk) < 0)
> > +	if (sparse_list_find(bb, &blk) < 0)
> > +		return 0;
> > +	else
> > +		return 1;
> > +}
> > +
> > +int ext2fs_badblocks_list_test2(ext2_badblocks_list bb, blk64_t blk)
> > +{
> > +	if (sparse_list_find(bb, &blk) < 0)
> >  		return 0;
> >  	else
> >  		return 1;
> > @@ -205,65 +286,83 @@ int ext2fs_u32_list_test(ext2_u32_list bb, __u32 blk)
> >  
> >  int ext2fs_badblocks_list_test(ext2_badblocks_list bb, blk_t blk)
> >  {
> > -	return ext2fs_u32_list_test((ext2_u32_list) bb, (__u32) blk);
> > +	return ext2fs_badblocks_list_test2(bb, blk);
> >  }
> >  
> >  
> >  /*
> >   * Remove a block from the badblock list
> >   */
> > -int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk)
> > +static int sparse_list_del(struct ext2_sparse_list *bb, void *item)
> >  {
> > -	int	remloc, i;
> > +	int	remloc;
> >  
> >  	if (bb->num == 0)
> >  		return -1;
> >  
> > -	remloc = ext2fs_u32_list_find(bb, blk);
> > +	remloc = sparse_list_find(bb, item);
> >  	if (remloc < 0)
> >  		return -1;
> >  
> > -	for (i = remloc ; i < bb->num-1; i++)
> > -		bb->list[i] = bb->list[i+1];
> > +	memmove(LIST_ITEM(bb, remloc), LIST_ITEM(bb, remloc + 1),
> > +		(bb->num - remloc - 1) * bb->item_size);
> >  	bb->num--;
> >  	return 0;
> >  }
> >  
> > +int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk)
> > +{
> > +	return sparse_list_del(bb, &blk);
> > +}
> > +
> > +void ext2fs_badblocks_list_del2(ext2_u32_list bb, blk64_t blk)
> > +{
> > +	sparse_list_del(bb, &blk);
> > +}
> > +
> >  void ext2fs_badblocks_list_del(ext2_u32_list bb, __u32 blk)
> >  {
> > -	ext2fs_u32_list_del(bb, blk);
> > +	ext2fs_badblocks_list_del(bb, blk);
> >  }
> >  
> > -errcode_t ext2fs_u32_list_iterate_begin(ext2_u32_list bb,
> > -					ext2_u32_iterate *ret)
> > +static errcode_t sparse_list_iterate_begin(struct ext2_sparse_list *bb,
> > +					   struct ext2_sparse_list_iterate **r)
> >  {
> > -	ext2_u32_iterate iter;
> > +	struct ext2_sparse_list_iterate *iter;
> >  	errcode_t		retval;
> >  
> >  	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
> >  
> > -	retval = ext2fs_get_mem(sizeof(struct ext2_struct_u32_iterate), &iter);
> > +	retval = ext2fs_get_mem(sizeof(struct ext2_sparse_list_iterate),
> > +				&iter);
> >  	if (retval)
> >  		return retval;
> >  
> >  	iter->magic = EXT2_ET_MAGIC_BADBLOCKS_ITERATE;
> >  	iter->bb = bb;
> >  	iter->ptr = 0;
> > -	*ret = iter;
> > +	*r = iter;
> >  	return 0;
> >  }
> >  
> > +errcode_t ext2fs_u32_list_iterate_begin(ext2_u32_list bb,
> > +					ext2_u32_iterate *ret)
> > +{
> > +	return sparse_list_iterate_begin(bb, ret);
> > +}
> > +
> >  errcode_t ext2fs_badblocks_list_iterate_begin(ext2_badblocks_list bb,
> >  					      ext2_badblocks_iterate *ret)
> >  {
> > -	return ext2fs_u32_list_iterate_begin((ext2_u32_list) bb,
> > -					      (ext2_u32_iterate *) ret);
> > +	return sparse_list_iterate_begin((ext2_u32_list) bb,
> > +					 (ext2_u32_iterate *) ret);
> >  }
> >  
> >  
> > -int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
> > +static int sparse_list_iterate(struct ext2_sparse_list_iterate *iter,
> > +			       void *item)
> >  {
> > -	ext2_u32_list	bb;
> > +	struct ext2_sparse_list	*bb;
> >  
> >  	if (iter->magic != EXT2_ET_MAGIC_BADBLOCKS_ITERATE)
> >  		return 0;
> > @@ -274,21 +373,35 @@ int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
> >  		return 0;
> >  
> >  	if (iter->ptr < bb->num) {
> > -		*blk = bb->list[iter->ptr++];
> > +		memcpy(item, LIST_ITEM(bb, iter->ptr++), bb->item_size);
> >  		return 1;
> >  	}
> > -	*blk = 0;
> > +	memset(item, 0, bb->item_size);
> >  	return 0;
> >  }
> >  
> > +int ext2fs_u32_list_iterate(ext2_u32_iterate iter, __u32 *blk)
> > +{
> > +	return sparse_list_iterate(iter, blk);
> > +}
> > +
> > +int ext2fs_badblocks_list_iterate2(ext2_badblocks_iterate iter, blk64_t *blk)
> > +{
> > +	return sparse_list_iterate(iter, blk);
> > +}
> > +
> >  int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter, blk_t *blk)
> >  {
> > -	return ext2fs_u32_list_iterate((ext2_u32_iterate) iter,
> > -				       (__u32 *) blk);
> > +	blk64_t x;
> > +	int ret;
> > +
> > +	ret = ext2fs_badblocks_list_iterate2(iter, &x);
> > +	*blk = x;
> > +	return ret;
> >  }
> >  
> >  
> > -void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
> > +static void sparse_list_iterate_end(struct ext2_sparse_list_iterate *iter)
> >  {
> >  	if (!iter || (iter->magic != EXT2_ET_MAGIC_BADBLOCKS_ITERATE))
> >  		return;
> > @@ -297,13 +410,19 @@ void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
> >  	ext2fs_free_mem(&iter);
> >  }
> >  
> > +void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter)
> > +{
> > +	sparse_list_iterate_end(iter);
> > +}
> > +
> >  void ext2fs_badblocks_list_iterate_end(ext2_badblocks_iterate iter)
> >  {
> > -	ext2fs_u32_list_iterate_end((ext2_u32_iterate) iter);
> > +	sparse_list_iterate_end(iter);
> >  }
> >  
> >  
> > -int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
> > +static int sparse_list_equal(struct ext2_sparse_list *bb1,
> > +			     struct ext2_sparse_list *bb2)
> >  {
> >  	EXT2_CHECK_MAGIC(bb1, EXT2_ET_MAGIC_BADBLOCKS_LIST);
> >  	EXT2_CHECK_MAGIC(bb2, EXT2_ET_MAGIC_BADBLOCKS_LIST);
> > @@ -311,18 +430,46 @@ int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
> >  	if (bb1->num != bb2->num)
> >  		return 0;
> >  
> > -	if (memcmp(bb1->list, bb2->list, bb1->num * sizeof(blk_t)) != 0)
> > +	if (bb1->item_size != bb2->item_size)
> > +		return 0;
> > +
> > +	if (memcmp(bb1->list, bb2->list, bb1->num * bb1->item_size) != 0)
> >  		return 0;
> >  	return 1;
> >  }
> >  
> > +int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2)
> > +{
> > +	return sparse_list_equal(bb1, bb2);
> > +}
> > +
> >  int ext2fs_badblocks_equal(ext2_badblocks_list bb1, ext2_badblocks_list bb2)
> >  {
> > -	return ext2fs_u32_list_equal((ext2_u32_list) bb1,
> > -				     (ext2_u32_list) bb2);
> > +	return sparse_list_equal(bb1, bb2);
> >  }
> >  
> > -int ext2fs_u32_list_count(ext2_u32_list bb)
> > +static unsigned int sparse_list_count(struct ext2_sparse_list *bb)
> >  {
> > +	EXT2_CHECK_MAGIC(bb, EXT2_ET_MAGIC_BADBLOCKS_LIST);
> >  	return bb->num;
> >  }
> > +
> > +int ext2fs_u32_list_count(ext2_u32_list bb)
> > +{
> > +	return sparse_list_count(bb);
> > +}
> > +
> > +unsigned int ext2fs_badblocks_count(ext2_badblocks_list bb)
> > +{
> > +	return sparse_list_count(bb);
> > +}
> > +
> > +blk64_t ext2fs_badblocks_get(ext2_badblocks_list bb, unsigned int i)
> > +{
> > +	blk64_t x;
> > +
> > +	if (i < 0 || i >= bb->num)
> > +		return 0;
> > +	memcpy(&x, LIST_ITEM(bb, i), bb->item_size);
> > +	return x;
> > +}
> > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > index d5d8d03..b4ba421 100644
> > --- a/lib/ext2fs/ext2fs.h
> > +++ b/lib/ext2fs/ext2fs.h
> > @@ -111,15 +111,15 @@ typedef struct ext2fs_struct_generic_bitmap *ext2fs_block_bitmap;
> >   * Badblocks list definitions
> >   */
> >  
> > -typedef struct ext2_struct_u32_list *ext2_badblocks_list;
> > -typedef struct ext2_struct_u32_iterate *ext2_badblocks_iterate;
> > +typedef struct ext2_sparse_list *ext2_badblocks_list;
> > +typedef struct ext2_sparse_list_iterate *ext2_badblocks_iterate;
> >  
> > -typedef struct ext2_struct_u32_list *ext2_u32_list;
> > -typedef struct ext2_struct_u32_iterate *ext2_u32_iterate;
> > +typedef struct ext2_sparse_list *ext2_u32_list;
> > +typedef struct ext2_sparse_list_iterate *ext2_u32_iterate;
> >  
> >  /* old */
> > -typedef struct ext2_struct_u32_list *badblocks_list;
> > -typedef struct ext2_struct_u32_iterate *badblocks_iterate;
> > +typedef struct ext2_sparse_list *badblocks_list;
> > +typedef struct ext2_sparse_list_iterate *badblocks_iterate;
> >  
> >  #define BADBLOCKS_FLAG_DIRTY	1
> >  
> > @@ -702,6 +702,8 @@ extern int ext2fs_u32_list_iterate(ext2_u32_iterate iter, blk_t *blk);
> >  extern void ext2fs_u32_list_iterate_end(ext2_u32_iterate iter);
> >  extern errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest);
> >  extern int ext2fs_u32_list_equal(ext2_u32_list bb1, ext2_u32_list bb2);
> > +extern int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk);
> > +extern int ext2fs_u32_list_count(ext2_u32_list bb);
> >  
> >  extern errcode_t ext2fs_badblocks_list_create(ext2_badblocks_list *ret,
> >  					    int size);
> > @@ -709,7 +711,6 @@ extern errcode_t ext2fs_badblocks_list_add(ext2_badblocks_list bb,
> >  					   blk_t blk);
> >  extern int ext2fs_badblocks_list_test(ext2_badblocks_list bb,
> >  				    blk_t blk);
> > -extern int ext2fs_u32_list_del(ext2_u32_list bb, __u32 blk);
> >  extern void ext2fs_badblocks_list_del(ext2_u32_list bb, __u32 blk);
> >  extern errcode_t
> >  	ext2fs_badblocks_list_iterate_begin(ext2_badblocks_list bb,
> > @@ -721,7 +722,13 @@ extern errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
> >  				       ext2_badblocks_list *dest);
> >  extern int ext2fs_badblocks_equal(ext2_badblocks_list bb1,
> >  				  ext2_badblocks_list bb2);
> > -extern int ext2fs_u32_list_count(ext2_u32_list bb);
> > +extern errcode_t ext2fs_badblocks_list_add2(ext2_badblocks_list bb,
> > +					    blk64_t blk);
> > +extern int ext2fs_badblocks_list_test2(ext2_badblocks_list bb, blk64_t blk);
> > +extern void ext2fs_badblocks_list_del2(ext2_u32_list bb, blk64_t blk);
> > +extern int ext2fs_badblocks_list_iterate2(ext2_badblocks_iterate iter,
> > +					  blk64_t *blk);
> > +extern blk64_t ext2fs_badblocks_get(ext2_badblocks_list bb, unsigned int i);
> >  
> >  /* bb_compat */
> >  extern errcode_t badblocks_list_create(badblocks_list *ret, int size);
> > diff --git a/lib/ext2fs/ext2fsP.h b/lib/ext2fs/ext2fsP.h
> > index 80d2d0a..592527c 100644
> > --- a/lib/ext2fs/ext2fsP.h
> > +++ b/lib/ext2fs/ext2fsP.h
> > @@ -16,17 +16,18 @@
> >  /*
> >   * Badblocks list
> >   */
> > -struct ext2_struct_u32_list {
> > +struct ext2_sparse_list {
> >  	int	magic;
> > -	int	num;
> > -	int	size;
> > -	__u32	*list;
> > +	unsigned int	num;
> > +	unsigned int	size;
> > +	unsigned int	item_size;
> > +	void	*list;
> >  	int	badblocks_flags;
> >  };
> >  
> > -struct ext2_struct_u32_iterate {
> > +struct ext2_sparse_list_iterate {
> >  	int			magic;
> > -	ext2_u32_list		bb;
> > +	struct ext2_sparse_list	*bb;
> >  	int			ptr;
> >  };
> >  
> > diff --git a/lib/ext2fs/inode.c b/lib/ext2fs/inode.c
> > index 46c1c58..6579512 100644
> > --- a/lib/ext2fs/inode.c
> > +++ b/lib/ext2fs/inode.c
> > @@ -313,8 +313,8 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
> >  	 * is no longer the case.  If we run out of bad blocks, then
> >  	 * we don't need to do any more checking!
> >  	 */
> > -	while (blk > bb->list[scan->bad_block_ptr]) {
> > -		if (++scan->bad_block_ptr >= bb->num) {
> > +	while (blk > ext2fs_badblocks_get(bb, scan->bad_block_ptr)) {
> > +		if (++scan->bad_block_ptr >= ext2fs_badblocks_count(bb)) {
> >  			scan->scan_flags &= ~EXT2_SF_CHK_BADBLOCKS;
> >  			return 0;
> >  		}
> > @@ -328,10 +328,10 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
> >  	 * expense of a huge expense of code complexity, and for an
> >  	 * uncommon case at that.)
> >  	 */
> > -	if (blk == bb->list[scan->bad_block_ptr]) {
> > +	if (blk == ext2fs_badblocks_get(bb, scan->bad_block_ptr)) {
> >  		scan->scan_flags |= EXT2_SF_BAD_INODE_BLK;
> >  		*num_blocks = 1;
> > -		if (++scan->bad_block_ptr >= bb->num)
> > +		if (++scan->bad_block_ptr >= ext2fs_badblocks_count(bb))
> >  			scan->scan_flags &= ~EXT2_SF_CHK_BADBLOCKS;
> >  		return 0;
> >  	}
> > @@ -342,8 +342,8 @@ static errcode_t check_for_inode_bad_blocks(ext2_inode_scan scan,
> >  	 * don't read in the bad block.  (Then the next block to read
> >  	 * will be the bad block, which is handled in the above case.)
> >  	 */
> > -	if ((blk + *num_blocks) > bb->list[scan->bad_block_ptr])
> > -		*num_blocks = (int) (bb->list[scan->bad_block_ptr] - blk);
> > +	if ((blk + *num_blocks) > ext2fs_badblocks_get(bb, scan->bad_block_ptr))
> > +		*num_blocks = (int)(ext2fs_badblocks_get(bb, scan->bad_block_ptr) - blk);
> >  
> >  	return 0;
> >  }
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 18/31] libext2fs: Badblocks should handle 48-bit block numbers correctly

[Darrick J. Wong]

Currently, the badblocks code assumes 32-bit block numbers.  This leads to
unfortunate results, because feeding a badblocks file to mke2fs with 64-bit
block numbers causes libext2fs to rip off the upper 32 bits of the block number
and then assign a truncated block number to the badblocks file.

This is just as well, since the code that writes to the bb inode doesn't know
about extents anyway.  Rather than continuing to open-code block map
manipulation, simply use existing library functions to truncate the old bb
inode, mark all badblocks in use, and then assign them to the badblocks file.
We can even use extents now.

(It's arguable that badblocks is a vestigial organ now, but perhaps someone is
using it?  I use it to stress-test disk block allocation, but I might just be
nutty.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 e2fsck/badblocks.c         |   28 ++---
 lib/ext2fs/bb_inode.c      |  251 +++++++++-----------------------------------
 lib/ext2fs/ext2fs.h        |    7 +
 lib/ext2fs/read_bb.c       |   10 +-
 lib/ext2fs/read_bb_file.c  |   39 ++++++-
 lib/ext2fs/tst_badblocks.c |   42 ++++---
 lib/ext2fs/tst_iscan.c     |    3 -
 lib/ext2fs/write_bb_file.c |    7 +
 misc/dumpe2fs.c            |    4 -
 misc/mke2fs.c              |   37 +++---
 misc/tune2fs.c             |    2 
 resize/resize2fs.c         |    4 -
 12 files changed, 164 insertions(+), 270 deletions(-)


diff --git a/e2fsck/badblocks.c b/e2fsck/badblocks.c
index 71b1c56..9a456d7 100644
--- a/e2fsck/badblocks.c
+++ b/e2fsck/badblocks.c
@@ -14,13 +14,14 @@
 #include <et/com_err.h>
 #include "e2fsck.h"
 
-static int check_bb_inode_blocks(ext2_filsys fs, blk_t *block_nr, int blockcnt,
-				 void *priv_data);
+static int check_bb_inode_blocks(ext2_filsys fs, blk64_t *block_nr,
+				 e2_blkcnt_t blockcnt, blk64_t ref_blk,
+				 int ref_offset, void *priv_data);
 
-
-static void invalid_block(ext2_filsys fs EXT2FS_ATTR((unused)), blk_t blk)
+static void invalid_block(ext2_filsys fs EXT2FS_ATTR((unused)), blk64_t blk,
+			  char *badmsg, void *privdata)
 {
-	printf(_("Bad block %u out of range; ignored.\n"), blk);
+	printf(_("Bad block %llu out of range; ignored.\n"), blk);
 	return;
 }
 
@@ -39,8 +40,8 @@ void read_bad_blocks_file(e2fsck_t ctx, const char *bad_blocks_file,
 	 * Make sure the bad block inode is sane.  If there are any
 	 * illegal blocks, clear them.
 	 */
-	retval = ext2fs_block_iterate(fs, EXT2_BAD_INO, 0, 0,
-				      check_bb_inode_blocks, 0);
+	retval = ext2fs_block_iterate3(fs, EXT2_BAD_INO, 0, 0,
+				       check_bb_inode_blocks, 0);
 	if (retval) {
 		com_err("ext2fs_block_iterate", retval,
 			_("while sanity checking the bad blocks inode"));
@@ -84,7 +85,7 @@ void read_bad_blocks_file(e2fsck_t ctx, const char *bad_blocks_file,
 			goto fatal;
 		}
 	}
-	retval = ext2fs_read_bb_FILE(fs, f, &bb_list, invalid_block);
+	retval = ext2fs_read_bb_FILE3(fs, f, &bb_list, NULL, invalid_block);
 	if (bad_blocks_file)
 		fclose(f);
 	else
@@ -115,10 +116,9 @@ fatal:
 
 }
 
-static int check_bb_inode_blocks(ext2_filsys fs,
-				 blk_t *block_nr,
-				 int blockcnt EXT2FS_ATTR((unused)),
-				 void *priv_data EXT2FS_ATTR((unused)))
+static int check_bb_inode_blocks(ext2_filsys fs, blk64_t *block_nr,
+				 e2_blkcnt_t blockcnt, blk64_t ref_blk,
+				 int ref_offset, void *priv_data)
 {
 	if (!*block_nr)
 		return 0;
@@ -128,8 +128,8 @@ static int check_bb_inode_blocks(ext2_filsys fs,
 	 */
 	if (*block_nr >= ext2fs_blocks_count(fs->super) ||
 	    *block_nr < fs->super->s_first_data_block) {
-		printf(_("Warning: illegal block %u found in bad block inode.  "
-			 "Cleared.\n"), *block_nr);
+		printf(_("Warning: illegal block %llu found in bad block "
+			 "inode.  Cleared.\n"), *block_nr);
 		*block_nr = 0;
 		return BLOCK_CHANGED;
 	}
diff --git a/lib/ext2fs/bb_inode.c b/lib/ext2fs/bb_inode.c
index 268eecf..1b168c4 100644
--- a/lib/ext2fs/bb_inode.c
+++ b/lib/ext2fs/bb_inode.c
@@ -31,26 +31,6 @@
 #include "ext2_fs.h"
 #include "ext2fs.h"
 
-struct set_badblock_record {
-	ext2_badblocks_iterate	bb_iter;
-	int		bad_block_count;
-	blk_t		*ind_blocks;
-	int		max_ind_blocks;
-	int		ind_blocks_size;
-	int		ind_blocks_ptr;
-	char		*block_buf;
-	errcode_t	err;
-};
-
-static int set_bad_block_proc(ext2_filsys fs, blk_t *block_nr,
-			      e2_blkcnt_t blockcnt,
-			      blk_t ref_block, int ref_offset,
-			      void *priv_data);
-static int clear_bad_block_proc(ext2_filsys fs, blk_t *block_nr,
-				e2_blkcnt_t blockcnt,
-				blk_t ref_block, int ref_offset,
-				void *priv_data);
-
 /*
  * Given a bad blocks bitmap, update the bad blocks inode to reflect
  * the map.
@@ -58,210 +38,87 @@ static int clear_bad_block_proc(ext2_filsys fs, blk_t *block_nr,
 errcode_t ext2fs_update_bb_inode(ext2_filsys fs, ext2_badblocks_list bb_list)
 {
 	errcode_t			retval;
-	struct set_badblock_record 	rec;
+	ext2_badblocks_iterate		bb_iter;
 	struct ext2_inode		inode;
+	blk64_t				blk, bad_blocks = 0;
+	int				list_sz = 0;
 
 	EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
 
 	if (!fs->block_map)
 		return EXT2_ET_NO_BLOCK_BITMAP;
 
-	memset(&rec, 0, sizeof(rec));
-	rec.max_ind_blocks = 10;
-	retval = ext2fs_get_array(rec.max_ind_blocks, sizeof(blk_t),
-				&rec.ind_blocks);
-	if (retval)
-		return retval;
-	memset(rec.ind_blocks, 0, rec.max_ind_blocks * sizeof(blk_t));
-	retval = ext2fs_get_mem(fs->blocksize, &rec.block_buf);
-	if (retval)
+	if (bb_list)
+		list_sz = ext2fs_badblocks_count(bb_list);
+
+	if (list_sz > 0 &&
+	    ext2fs_badblocks_get(bb_list, list_sz - 1) > ~0U &&
+	    !EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+				       EXT3_FEATURE_INCOMPAT_EXTENTS)) {
+		retval = EXT2_ET_INODE_NOT_EXTENT;
 		goto cleanup;
-	memset(rec.block_buf, 0, fs->blocksize);
-	rec.err = 0;
+	}
 
-	/*
-	 * First clear the old bad blocks (while saving the indirect blocks)
-	 */
-	retval = ext2fs_block_iterate2(fs, EXT2_BAD_INO,
-				       BLOCK_FLAG_DEPTH_TRAVERSE, 0,
-				       clear_bad_block_proc, &rec);
+	retval = ext2fs_read_inode_full(fs, EXT2_BAD_INO, &inode,
+					sizeof(inode));
 	if (retval)
 		goto cleanup;
-	if (rec.err) {
-		retval = rec.err;
+
+	/* Truncate the file */
+	retval = ext2fs_punch(fs, EXT2_BAD_INO, &inode, 0, 0, ~0ULL);
+	if (retval)
 		goto cleanup;
-	}
+	inode.i_size = 0;
+	inode.i_size_high = 0;
 
-	/*
-	 * Now set the bad blocks!
-	 *
-	 * First, mark the bad blocks as used.  This prevents a bad
-	 * block from being used as an indirecto block for the bad
-	 * block inode (!).
-	 */
-	if (bb_list) {
+	if (bb_list && list_sz > 0) {
+		/* Mark all bad blocks in use */
 		retval = ext2fs_badblocks_list_iterate_begin(bb_list,
-							     &rec.bb_iter);
+							     &bb_iter);
 		if (retval)
 			goto cleanup;
-		retval = ext2fs_block_iterate2(fs, EXT2_BAD_INO,
-					       BLOCK_FLAG_APPEND, 0,
-					       set_bad_block_proc, &rec);
-		ext2fs_badblocks_list_iterate_end(rec.bb_iter);
+
+		while (ext2fs_badblocks_list_iterate2(bb_iter, &blk))
+			ext2fs_block_alloc_stats2(fs, blk, +1);
+		ext2fs_badblocks_list_iterate_end(bb_iter);
+
+		/* Flip the extents flag if necessary */
+		memset(&inode.i_blocks, 0, EXT2_N_BLOCKS * sizeof(__u32));
+		if (ext2fs_badblocks_get(bb_list, list_sz - 1) > ~0U)
+			inode.i_flags |= EXT4_EXTENTS_FL;
+		else
+			inode.i_flags &= ~EXT4_EXTENTS_FL;
+
+		/* Set the bad blocks */
+		retval = ext2fs_badblocks_list_iterate_begin(bb_list,
+							     &bb_iter);
 		if (retval)
 			goto cleanup;
-		if (rec.err) {
-			retval = rec.err;
-			goto cleanup;
+
+		while (ext2fs_badblocks_list_iterate2(bb_iter, &blk)) {
+			retval = ext2fs_bmap2(fs, EXT2_BAD_INO, &inode, NULL,
+					      BMAP_SET | BMAP_ALLOC,
+					      bad_blocks++, NULL, &blk);
+			if (retval)
+				break;
 		}
+		ext2fs_badblocks_list_iterate_end(bb_iter);
+		if (retval)
+			goto cleanup;
 	}
 
-	/*
-	 * Update the bad block inode's mod time and block count
-	 * field.
-	 */
-	retval = ext2fs_read_inode(fs, EXT2_BAD_INO, &inode);
-	if (retval)
-		goto cleanup;
-
+	/* Update inode mtime */
 	inode.i_atime = inode.i_mtime = fs->now ? fs->now : time(0);
 	if (!inode.i_ctime)
 		inode.i_ctime = fs->now ? fs->now : time(0);
-	ext2fs_iblk_set(fs, &inode, rec.bad_block_count);
-	inode.i_size = rec.bad_block_count * fs->blocksize;
-
-	retval = ext2fs_write_inode(fs, EXT2_BAD_INO, &inode);
+	bad_blocks *= fs->blocksize;
+	inode.i_size = bad_blocks & 0xFFFFFFFF;
+	inode.i_size_high = bad_blocks >> 32;
+	retval = ext2fs_write_inode_full(fs, EXT2_BAD_INO, &inode,
+					 sizeof(inode));
 	if (retval)
 		goto cleanup;
 
 cleanup:
-	ext2fs_free_mem(&rec.ind_blocks);
-	ext2fs_free_mem(&rec.block_buf);
 	return retval;
 }
-
-/*
- * Helper function for update_bb_inode()
- *
- * Clear the bad blocks in the bad block inode, while saving the
- * indirect blocks.
- */
-#ifdef __TURBOC__
- #pragma argsused
-#endif
-static int clear_bad_block_proc(ext2_filsys fs, blk_t *block_nr,
-				e2_blkcnt_t blockcnt,
-				blk_t ref_block EXT2FS_ATTR((unused)),
-				int ref_offset EXT2FS_ATTR((unused)),
-				void *priv_data)
-{
-	struct set_badblock_record *rec = (struct set_badblock_record *)
-		priv_data;
-	errcode_t	retval;
-	unsigned long 	old_size;
-
-	if (!*block_nr)
-		return 0;
-
-	/*
-	 * If the block number is outrageous, clear it and ignore it.
-	 */
-	if (*block_nr >= ext2fs_blocks_count(fs->super) ||
-	    *block_nr < fs->super->s_first_data_block) {
-		*block_nr = 0;
-		return BLOCK_CHANGED;
-	}
-
-	if (blockcnt < 0) {
-		if (rec->ind_blocks_size >= rec->max_ind_blocks) {
-			old_size = rec->max_ind_blocks * sizeof(blk_t);
-			rec->max_ind_blocks += 10;
-			retval = ext2fs_resize_mem(old_size,
-				   rec->max_ind_blocks * sizeof(blk_t),
-				   &rec->ind_blocks);
-			if (retval) {
-				rec->max_ind_blocks -= 10;
-				rec->err = retval;
-				return BLOCK_ABORT;
-			}
-		}
-		rec->ind_blocks[rec->ind_blocks_size++] = *block_nr;
-	}
-
-	/*
-	 * Mark the block as unused, and update accounting information
-	 */
-	ext2fs_block_alloc_stats2(fs, *block_nr, -1);
-
-	*block_nr = 0;
-	return BLOCK_CHANGED;
-}
-
-
-/*
- * Helper function for update_bb_inode()
- *
- * Set the block list in the bad block inode, using the supplied bitmap.
- */
-#ifdef __TURBOC__
- #pragma argsused
-#endif
-static int set_bad_block_proc(ext2_filsys fs, blk_t *block_nr,
-			      e2_blkcnt_t blockcnt,
-			      blk_t ref_block EXT2FS_ATTR((unused)),
-			      int ref_offset EXT2FS_ATTR((unused)),
-			      void *priv_data)
-{
-	struct set_badblock_record *rec = (struct set_badblock_record *)
-		priv_data;
-	errcode_t	retval;
-	blk_t		blk;
-
-	if (blockcnt >= 0) {
-		/*
-		 * Get the next bad block.
-		 */
-		if (!ext2fs_badblocks_list_iterate(rec->bb_iter, &blk))
-			return BLOCK_ABORT;
-		rec->bad_block_count++;
-	} else {
-		/*
-		 * An indirect block; fetch a block from the
-		 * previously used indirect block list.  The block
-		 * most be not marked as used; if so, get another one.
-		 * If we run out of reserved indirect blocks, allocate
-		 * a new one.
-		 */
-	retry:
-		if (rec->ind_blocks_ptr < rec->ind_blocks_size) {
-			blk = rec->ind_blocks[rec->ind_blocks_ptr++];
-			if (ext2fs_test_block_bitmap2(fs->block_map, blk))
-				goto retry;
-		} else {
-			retval = ext2fs_new_block(fs, 0, 0, &blk);
-			if (retval) {
-				rec->err = retval;
-				return BLOCK_ABORT;
-			}
-		}
-		retval = io_channel_write_blk64(fs->io, blk, 1, rec->block_buf);
-		if (retval) {
-			rec->err = retval;
-			return BLOCK_ABORT;
-		}
-	}
-
-	/*
-	 * Update block counts
-	 */
-	ext2fs_block_alloc_stats2(fs, blk, +1);
-
-	*block_nr = blk;
-	return BLOCK_CHANGED;
-}
-
-
-
-
-
-
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index b4ba421..847b33c 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1486,6 +1486,13 @@ extern errcode_t ext2fs_read_bb_inode(ext2_filsys fs,
 				      ext2_badblocks_list *bb_list);
 
 /* read_bb_file.c */
+extern errcode_t ext2fs_read_bb_FILE3(ext2_filsys fs, FILE *f,
+				      ext2_badblocks_list *bb_list,
+				      void *priv_data,
+				      void (*invalid)(ext2_filsys fs,
+						      blk64_t blk,
+						      char *badstr,
+						      void *priv_data));
 extern errcode_t ext2fs_read_bb_FILE2(ext2_filsys fs, FILE *f,
 				      ext2_badblocks_list *bb_list,
 				      void *priv_data,
diff --git a/lib/ext2fs/read_bb.c b/lib/ext2fs/read_bb.c
index b5a0d7b..348d5a3 100644
--- a/lib/ext2fs/read_bb.c
+++ b/lib/ext2fs/read_bb.c
@@ -38,9 +38,9 @@ struct read_bb_record {
 #ifdef __TURBOC__
  #pragma argsused
 #endif
-static int mark_bad_block(ext2_filsys fs, blk_t *block_nr,
+static int mark_bad_block(ext2_filsys fs, blk64_t *block_nr,
 			  e2_blkcnt_t blockcnt EXT2FS_ATTR((unused)),
-			  blk_t ref_block EXT2FS_ATTR((unused)),
+			  blk64_t ref_block EXT2FS_ATTR((unused)),
 			  int ref_offset EXT2FS_ATTR((unused)),
 			  void *priv_data)
 {
@@ -53,7 +53,7 @@ static int mark_bad_block(ext2_filsys fs, blk_t *block_nr,
 	    (*block_nr >= ext2fs_blocks_count(fs->super)))
 		return 0;	/* Ignore illegal blocks */
 
-	rb->err = ext2fs_badblocks_list_add(rb->bb_list, *block_nr);
+	rb->err = ext2fs_badblocks_list_add2(rb->bb_list, *block_nr);
 	if (rb->err)
 		return BLOCK_ABORT;
 	return 0;
@@ -67,7 +67,7 @@ errcode_t ext2fs_read_bb_inode(ext2_filsys fs, ext2_badblocks_list *bb_list)
 	errcode_t	retval;
 	struct read_bb_record rb;
 	struct ext2_inode inode;
-	blk_t	numblocks;
+	blk64_t	numblocks;
 
 	EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
 
@@ -92,7 +92,7 @@ errcode_t ext2fs_read_bb_inode(ext2_filsys fs, ext2_badblocks_list *bb_list)
 
 	rb.bb_list = *bb_list;
 	rb.err = 0;
-	retval = ext2fs_block_iterate2(fs, EXT2_BAD_INO, BLOCK_FLAG_READ_ONLY,
+	retval = ext2fs_block_iterate3(fs, EXT2_BAD_INO, BLOCK_FLAG_READ_ONLY,
 				       0, mark_bad_block, &rb);
 	if (retval)
 		return retval;
diff --git a/lib/ext2fs/read_bb_file.c b/lib/ext2fs/read_bb_file.c
index 7d7bb7a..44f1cde 100644
--- a/lib/ext2fs/read_bb_file.c
+++ b/lib/ext2fs/read_bb_file.c
@@ -30,16 +30,16 @@
 /*
  * Reads a list of bad blocks from  a FILE *
  */
-errcode_t ext2fs_read_bb_FILE2(ext2_filsys fs, FILE *f,
+errcode_t ext2fs_read_bb_FILE3(ext2_filsys fs, FILE *f,
 			       ext2_badblocks_list *bb_list,
 			       void *priv_data,
 			       void (*invalid)(ext2_filsys fs,
-					       blk_t blk,
+					       blk64_t blk,
 					       char *badstr,
 					       void *priv_data))
 {
 	errcode_t	retval;
-	blk_t		blockno;
+	blk64_t		blockno;
 	int		count;
 	char		buf[128];
 
@@ -55,7 +55,7 @@ errcode_t ext2fs_read_bb_FILE2(ext2_filsys fs, FILE *f,
 	while (!feof (f)) {
 		if (fgets(buf, sizeof(buf), f) == NULL)
 			break;
-		count = sscanf(buf, "%u", &blockno);
+		count = sscanf(buf, "%llu", &blockno);
 		if (count <= 0)
 			continue;
 		if (fs &&
@@ -65,13 +65,42 @@ errcode_t ext2fs_read_bb_FILE2(ext2_filsys fs, FILE *f,
 				(invalid)(fs, blockno, buf, priv_data);
 			continue;
 		}
-		retval = ext2fs_badblocks_list_add(*bb_list, blockno);
+		retval = ext2fs_badblocks_list_add2(*bb_list, blockno);
 		if (retval)
 			return retval;
 	}
 	return 0;
 }
 
+struct read_helper2_data {
+	void (*invalid)(ext2_filsys fs, blk_t blk, char *badstr,
+			void *priv_data);
+	void *priv_data;
+};
+
+static void read_helper2(ext2_filsys fs, blk64_t blk, char *badstr,
+			 void *priv_data)
+{
+	struct read_helper2_data *d = priv_data;
+	if (d->invalid)
+		d->invalid(fs, blk, badstr, d->priv_data);
+}
+
+errcode_t ext2fs_read_bb_FILE2(ext2_filsys fs, FILE *f,
+			       ext2_badblocks_list *bb_list,
+			       void *priv_data,
+			       void (*invalid)(ext2_filsys fs,
+					       blk_t blk,
+					       char *badstr,
+					       void *priv_data))
+{
+	struct read_helper2_data d;
+
+	d.invalid = invalid;
+	d.priv_data = priv_data;
+	return ext2fs_read_bb_FILE3(fs, f, bb_list, &d, read_helper2);
+}
+
 struct compat_struct {
 	void (*invalid)(ext2_filsys, blk_t);
 };
diff --git a/lib/ext2fs/tst_badblocks.c b/lib/ext2fs/tst_badblocks.c
index 3b39ef1..40168b9 100644
--- a/lib/ext2fs/tst_badblocks.c
+++ b/lib/ext2fs/tst_badblocks.c
@@ -29,11 +29,11 @@
 #define ADD_BLK	0x0001
 #define DEL_BLK	0x0002
 
-blk_t test1[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0 };
-blk_t test2[] = { 11, 10, 9, 8, 7, 6, 5, 4, 3, 3, 2, 1 };
-blk_t test3[] = { 3, 1, 4, 5, 9, 2, 7, 10, 5, 6, 10, 8, 0 };
-blk_t test4[] = { 20, 50, 12, 17, 13, 2, 66, 23, 56, 0 };
-blk_t test4a[] = {
+blk64_t test1[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0 };
+blk64_t test2[] = { 11, 10, 9, 8, 7, 6, 5, 4, 3, 3, 2, 1 };
+blk64_t test3[] = { 3, 1, 4, 5, 9, 2, 7, 10, 5, 6, 10, 8, 0 };
+blk64_t test4[] = { 20, 50, 12, 17, 13, 2, 66, 23, 56, 0 };
+blk64_t test4a[] = {
  	20, 1,
 	50, 1,
 	3, 0,
@@ -48,8 +48,8 @@ blk_t test4a[] = {
 	45, 0,
 	66, 1,
 	0 };
-blk_t test5[] = { 31, 20, 17, 51, 23, 1, 56, 57, 0 };
-blk_t test5a[] = {
+blk64_t test5[] = { 31, 20, 17, 51, 23, 1, 56, 57, 0 };
+blk64_t test5a[] = {
 	50, ADD_BLK,
 	51, DEL_BLK,
 	57, DEL_BLK,
@@ -66,7 +66,7 @@ blk_t test5a[] = {
 static int test_fail = 0;
 static int test_expected_fail = 0;
 
-static errcode_t create_test_list(blk_t *vec, badblocks_list *ret)
+static errcode_t create_test_list(blk64_t *vec, badblocks_list *ret)
 {
 	errcode_t	retval;
 	badblocks_list	bb;
@@ -78,7 +78,7 @@ static errcode_t create_test_list(blk_t *vec, badblocks_list *ret)
 		return retval;
 	}
 	for (i=0; vec[i]; i++) {
-		retval = ext2fs_badblocks_list_add(bb, vec[i]);
+		retval = ext2fs_badblocks_list_add2(bb, vec[i]);
 		if (retval) {
 			com_err("create_test_list", retval,
 				"while adding test vector %d", i);
@@ -94,7 +94,7 @@ static void print_list(badblocks_list bb, int verify)
 {
 	errcode_t	retval;
 	badblocks_iterate	iter;
-	blk_t			blk;
+	blk64_t			blk;
 	int			i, ok;
 
 	retval = ext2fs_badblocks_list_iterate_begin(bb, &iter);
@@ -103,7 +103,7 @@ static void print_list(badblocks_list bb, int verify)
 		return;
 	}
 	ok = i = 1;
-	while (ext2fs_badblocks_list_iterate(iter, &blk)) {
+	while (ext2fs_badblocks_list_iterate2(iter, &blk)) {
 		printf("%u ", blk);
 		if (i++ != blk)
 			ok = 0;
@@ -119,12 +119,12 @@ static void print_list(badblocks_list bb, int verify)
 	}
 }
 
-static void validate_test_seq(badblocks_list bb, blk_t *vec)
+static void validate_test_seq(badblocks_list bb, blk64_t *vec)
 {
 	int	i, match, ok;
 
 	for (i = 0; vec[i]; i += 2) {
-		match = ext2fs_badblocks_list_test(bb, vec[i]);
+		match = ext2fs_badblocks_list_test2(bb, vec[i]);
 		if (match == vec[i+1])
 			ok = 1;
 		else {
@@ -137,15 +137,15 @@ static void validate_test_seq(badblocks_list bb, blk_t *vec)
 	}
 }
 
-static void do_test_seq(badblocks_list bb, blk_t *vec)
+static void do_test_seq(badblocks_list bb, blk64_t *vec)
 {
 	int	i, match;
 
 	for (i = 0; vec[i]; i += 2) {
 		switch (vec[i+1]) {
 		case ADD_BLK:
-			ext2fs_badblocks_list_add(bb, vec[i]);
-			match = ext2fs_badblocks_list_test(bb, vec[i]);
+			ext2fs_badblocks_list_add2(bb, vec[i]);
+			match = ext2fs_badblocks_list_test2(bb, vec[i]);
 			printf("Adding block %u --- now %s\n", vec[i],
 			       match ? "present" : "absent");
 			if (!match) {
@@ -154,10 +154,10 @@ static void do_test_seq(badblocks_list bb, blk_t *vec)
 			}
 			break;
 		case DEL_BLK:
-			ext2fs_badblocks_list_del(bb, vec[i]);
-			match = ext2fs_badblocks_list_test(bb, vec[i]);
+			ext2fs_badblocks_list_del2(bb, vec[i]);
+			match = ext2fs_badblocks_list_test2(bb, vec[i]);
 			printf("Removing block %u --- now %s\n", vec[i],
-			       ext2fs_badblocks_list_test(bb, vec[i]) ?
+			       ext2fs_badblocks_list_test2(bb, vec[i]) ?
 			       "present" : "absent");
 			if (match) {
 				printf("FAILURE!\n");
@@ -188,7 +188,7 @@ int file_test(badblocks_list bb)
 	}
 
 	rewind(f);
-	retval = ext2fs_read_bb_FILE2(0, f, &new_bb, 0, 0);
+	retval = ext2fs_read_bb_FILE3(0, f, &new_bb, 0, 0);
 	if (retval) {
 		com_err("file_test", retval, "while reading bad blocks");
 		return 1;
@@ -204,7 +204,7 @@ int file_test(badblocks_list bb)
 	return 0;
 }
 
-static void invalid_proc(ext2_filsys fs, blk_t blk)
+static void invalid_proc(ext2_filsys fs, blk64_t blk)
 {
 	if (blk == 34500) {
 		printf("Expected invalid block\n");
diff --git a/lib/ext2fs/tst_iscan.c b/lib/ext2fs/tst_iscan.c
index a95296c..3013d41 100644
--- a/lib/ext2fs/tst_iscan.c
+++ b/lib/ext2fs/tst_iscan.c
@@ -124,7 +124,8 @@ static void setup(void)
 		exit(1);
 	}
 	for (i=0; test_vec[i]; i++) {
-		retval = ext2fs_badblocks_list_add(test_badblocks, test_vec[i]);
+		retval = ext2fs_badblocks_list_add2(test_badblocks,
+						    test_vec[i]);
 		if (retval) {
 			com_err("setup", retval,
 				"while adding test vector %d", i);
diff --git a/lib/ext2fs/write_bb_file.c b/lib/ext2fs/write_bb_file.c
index 5834340..aeeee82 100644
--- a/lib/ext2fs/write_bb_file.c
+++ b/lib/ext2fs/write_bb_file.c
@@ -20,16 +20,15 @@ errcode_t ext2fs_write_bb_FILE(ext2_badblocks_list bb_list,
 			       FILE *f)
 {
 	badblocks_iterate	bb_iter;
-	blk_t			blk;
+	blk64_t			blk;
 	errcode_t		retval;
 
 	retval = ext2fs_badblocks_list_iterate_begin(bb_list, &bb_iter);
 	if (retval)
 		return retval;
 
-	while (ext2fs_badblocks_list_iterate(bb_iter, &blk)) {
-		fprintf(f, "%u\n", blk);
-	}
+	while (ext2fs_badblocks_list_iterate2(bb_iter, &blk))
+		fprintf(f, "%llu\n", blk);
 	ext2fs_badblocks_list_iterate_end(bb_iter);
 	return 0;
 }
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index b139977..3035b62 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -293,7 +293,7 @@ static void list_bad_blocks(ext2_filsys fs, int dump)
 {
 	badblocks_list		bb_list = 0;
 	badblocks_iterate	bb_iter;
-	blk_t			blk;
+	blk64_t			blk;
 	errcode_t		retval;
 	const char		*header, *fmt;
 
@@ -314,7 +314,7 @@ static void list_bad_blocks(ext2_filsys fs, int dump)
 		header =  _("Bad blocks: %u");
 		fmt = ", %u";
 	}
-	while (ext2fs_badblocks_list_iterate(bb_iter, &blk)) {
+	while (ext2fs_badblocks_list_iterate2(bb_iter, &blk)) {
 		printf(header ? header : fmt, blk);
 		header = 0;
 	}
diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index d96f156..d8bd5ed 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -175,9 +175,10 @@ static int parse_version_number(const char *s)
 /*
  * Helper function for read_bb_file and test_disk
  */
-static void invalid_block(ext2_filsys fs EXT2FS_ATTR((unused)), blk_t blk)
+static void invalid_block(ext2_filsys fs EXT2FS_ATTR((unused)), blk64_t blk,
+			  char *badstr, void *private_data)
 {
-	fprintf(stderr, _("Bad block %u out of range; ignored.\n"), blk);
+	fprintf(stderr, _("Bad block %llu out of range; ignored.\n"), blk);
 	return;
 }
 
@@ -196,7 +197,7 @@ static void read_bb_file(ext2_filsys fs, badblocks_list *bb_list,
 			_("while trying to open %s"), bad_blocks_file);
 		exit(1);
 	}
-	retval = ext2fs_read_bb_FILE(fs, f, bb_list, invalid_block);
+	retval = ext2fs_read_bb_FILE3(fs, f, bb_list, NULL, invalid_block);
 	fclose (f);
 	if (retval) {
 		com_err("ext2fs_read_bb_FILE", retval,
@@ -225,7 +226,7 @@ static void test_disk(ext2_filsys fs, badblocks_list *bb_list)
 			_("while trying to run '%s'"), buf);
 		exit(1);
 	}
-	retval = ext2fs_read_bb_FILE(fs, f, bb_list, invalid_block);
+	retval = ext2fs_read_bb_FILE3(fs, f, bb_list, NULL, invalid_block);
 	pclose(f);
 	if (retval) {
 		com_err("ext2fs_read_bb_FILE", retval,
@@ -237,13 +238,13 @@ static void test_disk(ext2_filsys fs, badblocks_list *bb_list)
 static void handle_bad_blocks(ext2_filsys fs, badblocks_list bb_list)
 {
 	dgrp_t			i;
-	blk_t			j;
-	unsigned 		must_be_good;
-	blk_t			blk;
+	blk64_t			j;
+	blk64_t			must_be_good;
+	blk64_t			blk;
 	badblocks_iterate	bb_iter;
 	errcode_t		retval;
-	blk_t			group_block;
-	int			group;
+	blk64_t			group_block;
+	dgrp_t			group;
 	int			group_bad;
 
 	if (!bb_list)
@@ -254,12 +255,12 @@ static void handle_bad_blocks(ext2_filsys fs, badblocks_list bb_list)
 	 * good; if not, abort.
 	 */
 	must_be_good = fs->super->s_first_data_block + 1 + fs->desc_blocks;
-	for (i = fs->super->s_first_data_block; i <= must_be_good; i++) {
-		if (ext2fs_badblocks_list_test(bb_list, i)) {
-			fprintf(stderr, _("Block %d in primary "
-				"superblock/group descriptor area bad.\n"), i);
-			fprintf(stderr, _("Blocks %u through %u must be good "
-				"in order to build a filesystem.\n"),
+	for (j = fs->super->s_first_data_block; j <= must_be_good; j++) {
+		if (ext2fs_badblocks_list_test2(bb_list, j)) {
+			fprintf(stderr, _("Block %llu in primary "
+				"superblock/group descriptor area bad.\n"), j);
+			fprintf(stderr, _("Blocks %u through %llu must be "
+				"good in order to build a filesystem.\n"),
 				fs->super->s_first_data_block, must_be_good);
 			fputs(_("Aborting....\n"), stderr);
 			exit(1);
@@ -277,11 +278,11 @@ static void handle_bad_blocks(ext2_filsys fs, badblocks_list bb_list)
 	for (i = 1; i < fs->group_desc_count; i++) {
 		group_bad = 0;
 		for (j=0; j < fs->desc_blocks+1; j++) {
-			if (ext2fs_badblocks_list_test(bb_list,
+			if (ext2fs_badblocks_list_test2(bb_list,
 						       group_block + j)) {
 				if (!group_bad)
 					fprintf(stderr,
-_("Warning: the backup superblock/group descriptors at block %u contain\n"
+_("Warning: the backup superblock/group descriptors at block %llu contain\n"
 "	bad blocks.\n\n"),
 						group_block);
 				group_bad++;
@@ -303,7 +304,7 @@ _("Warning: the backup superblock/group descriptors at block %u contain\n"
 			_("while marking bad blocks as used"));
 		exit(1);
 	}
-	while (ext2fs_badblocks_list_iterate(bb_iter, &blk))
+	while (ext2fs_badblocks_list_iterate2(bb_iter, &blk))
 		ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, blk));
 	ext2fs_badblocks_list_iterate_end(bb_iter);
 }
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 60d1378..835f364 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -1812,7 +1812,7 @@ static int get_move_bitmaps(ext2_filsys fs, int new_ino_blks_per_grp,
 				/*
 				 * IF the block is a bad block we fail
 				 */
-				if (ext2fs_badblocks_list_test(bb_list, j)) {
+				if (ext2fs_badblocks_list_test2(bb_list, j)) {
 					ext2fs_badblocks_list_free(bb_list);
 					return ENOSPC;
 				}
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 76d7aa6..e9c207b 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -1240,8 +1240,8 @@ static errcode_t block_mover(ext2_resize_t rfs)
 			continue;
 		if (!ext2fs_test_block_bitmap2(rfs->move_blocks, blk))
 			continue;
-		if (ext2fs_badblocks_list_test(badblock_list, blk)) {
-			ext2fs_badblocks_list_del(badblock_list, blk);
+		if (ext2fs_badblocks_list_test2(badblock_list, blk)) {
+			ext2fs_badblocks_list_del2(badblock_list, blk);
 			bb_modified++;
 			continue;
 		}

Re: [PATCH 18/31] libext2fs: Badblocks should handle 48-bit block numbers correctly

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:28:37PM -0700, Darrick J. Wong wrote:
> Currently, the badblocks code assumes 32-bit block numbers.  This leads to
> unfortunate results, because feeding a badblocks file to mke2fs with 64-bit
> block numbers causes libext2fs to rip off the upper 32 bits of the block number
> and then assign a truncated block number to the badblocks file.
> 
> This is just as well, since the code that writes to the bb inode doesn't know
> about extents anyway.  Rather than continuing to open-code block map
> manipulation, simply use existing library functions to truncate the old bb
> inode, mark all badblocks in use, and then assign them to the badblocks file.
> We can even use extents now.
> 
> (It's arguable that badblocks is a vestigial organ now, but perhaps someone is
> using it?  I use it to stress-test disk block allocation, but I might just be
> nutty.)
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Yeah, I think badblocks is vestigal at this point, and for huge disk
arrays, almost certainly block replacement will be handed at the LVM,
storage array, or HDD level.  So it might be better simply to have
mke2fs throw an error if there is an attempt to hand it a 64-bit block
number.

							- Ted

Re: [PATCH 18/31] libext2fs: Badblocks should handle 48-bit block numbers correctly

[Darrick J. Wong]

On Tue, Oct 08, 2013 at 12:03:31PM -0400, Theodore Ts'o wrote:
> On Mon, Sep 30, 2013 at 06:28:37PM -0700, Darrick J. Wong wrote:
> > Currently, the badblocks code assumes 32-bit block numbers.  This leads to
> > unfortunate results, because feeding a badblocks file to mke2fs with 64-bit
> > block numbers causes libext2fs to rip off the upper 32 bits of the block number
> > and then assign a truncated block number to the badblocks file.
> > 
> > This is just as well, since the code that writes to the bb inode doesn't know
> > about extents anyway.  Rather than continuing to open-code block map
> > manipulation, simply use existing library functions to truncate the old bb
> > inode, mark all badblocks in use, and then assign them to the badblocks file.
> > We can even use extents now.
> > 
> > (It's arguable that badblocks is a vestigial organ now, but perhaps someone is
> > using it?  I use it to stress-test disk block allocation, but I might just be
> > nutty.)
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Yeah, I think badblocks is vestigal at this point, and for huge disk
> arrays, almost certainly block replacement will be handed at the LVM,
> storage array, or HDD level.  So it might be better simply to have
> mke2fs throw an error if there is an attempt to hand it a 64-bit block
> number.

Ok.  I'll audit all callers of ext2fs_badblocks_list_add() to make sure they
reject > 2^32 numbers.

--D
> 
> 							- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 19/31] badblocks: Use the new badblocks APIs for 64-bit block numbers

[Darrick J. Wong]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 misc/badblocks.c |   76 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 40 insertions(+), 36 deletions(-)


diff --git a/misc/badblocks.c b/misc/badblocks.c
index c9e47c7..660248e 100644
--- a/misc/badblocks.c
+++ b/misc/badblocks.c
@@ -105,14 +105,14 @@ static void exclusive_usage(void)
 	exit(1);
 }
 
-static blk_t currently_testing = 0;
-static blk_t num_blocks = 0;
-static blk_t num_read_errors = 0;
-static blk_t num_write_errors = 0;
-static blk_t num_corruption_errors = 0;
+static blk64_t currently_testing = 0;
+static blk64_t num_blocks = 0;
+static blk64_t num_read_errors = 0;
+static blk64_t num_write_errors = 0;
+static blk64_t num_corruption_errors = 0;
 static ext2_badblocks_list bb_list = NULL;
 static FILE *out;
-static blk_t next_bad = 0;
+static blk64_t next_bad = 0;
 static ext2_badblocks_iterate bb_iter = NULL;
 
 enum error_types { READ_ERROR, WRITE_ERROR, CORRUPTION_ERROR };
@@ -144,17 +144,17 @@ static void *allocate_buffer(size_t size)
  * This routine reports a new bad block.  If the bad block has already
  * been seen before, then it returns 0; otherwise it returns 1.
  */
-static int bb_output (blk_t bad, enum error_types error_type)
+static int bb_output(blk64_t bad, enum error_types error_type)
 {
 	errcode_t errcode;
 
-	if (ext2fs_badblocks_list_test(bb_list, bad))
+	if (ext2fs_badblocks_list_test2(bb_list, bad))
 		return 0;
 
 	fprintf(out, "%lu\n", (unsigned long) bad);
 	fflush(out);
 
-	errcode = ext2fs_badblocks_list_add (bb_list, bad);
+	errcode = ext2fs_badblocks_list_add2(bb_list, bad);
 	if (errcode) {
 		com_err (program_name, errcode, "adding to in-memory bad block list");
 		exit (1);
@@ -165,7 +165,7 @@ static int bb_output (blk_t bad, enum error_types error_type)
 	   an element was just added before the current iteration
 	   position.  This should not cause next_bad to change. */
 	if (bb_iter && bad < next_bad)
-		ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
+		ext2fs_badblocks_list_iterate2(bb_iter, &next_bad);
 
 	if (error_type == READ_ERROR) {
 	  num_read_errors++;
@@ -216,7 +216,7 @@ static void print_status(void)
 	gettimeofday(&time_end, 0);
 	len = snprintf(line_buf, sizeof(line_buf), 
 		       _("%6.2f%% done, %s elapsed. "
-		         "(%d/%d/%d errors)"),
+			 "(%llu/%llu/%llu errors)"),
 		       calc_percent((unsigned long) currently_testing,
 				    (unsigned long) num_blocks), 
 		       time_diff_format(&time_end, &time_start, diff_buf),
@@ -350,7 +350,7 @@ static void pattern_fill(unsigned char *buffer, unsigned int pattern,
  *    successfully sequentially read.
  */
 static int do_read (int dev, unsigned char * buffer, int try, int block_size,
-		    blk_t current_block)
+		    blk64_t current_block)
 {
 	long got;
 	struct timeval tv1, tv2;
@@ -469,16 +469,16 @@ static void flush_bufs(void)
 		com_err(program_name, retval, _("during ext2fs_sync_device"));
 }
 
-static unsigned int test_ro (int dev, blk_t last_block,
-			     int block_size, blk_t first_block,
-			     unsigned int blocks_at_once)
+static unsigned int test_ro(int dev, blk64_t last_block,
+			    int block_size, blk64_t first_block,
+			    unsigned int blocks_at_once)
 {
 	unsigned char * blkbuf;
 	int try;
 	int got;
 	unsigned int bb_count = 0;
 	errcode_t errcode;
-	blk_t recover_block = ~0;
+	blk64_t recover_block = ~0;
 
 	/* set up abend handler */
 	capture_terminate(NULL);
@@ -490,7 +490,7 @@ static unsigned int test_ro (int dev, blk_t last_block,
 		exit (1);
 	}
 	do {
-		ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
+		ext2fs_badblocks_list_iterate2(bb_iter, &next_bad);
 	} while (next_bad && next_bad < first_block);
 
 	if (t_flag) {
@@ -532,7 +532,8 @@ static unsigned int test_ro (int dev, blk_t last_block,
 		if (next_bad) {
 			if (currently_testing == next_bad) {
 				/* fprintf (out, "%lu\n", nextbad); */
-				ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
+				ext2fs_badblocks_list_iterate2(bb_iter,
+						&next_bad);
 				currently_testing++;
 				continue;
 			}
@@ -581,16 +582,16 @@ static unsigned int test_ro (int dev, blk_t last_block,
 	return bb_count;
 }
 
-static unsigned int test_rw (int dev, blk_t last_block,
-			     int block_size, blk_t first_block,
-			     unsigned int blocks_at_once)
+static unsigned int test_rw(int dev, blk64_t last_block,
+			    int block_size, blk64_t first_block,
+			    unsigned int blocks_at_once)
 {
 	unsigned char *buffer, *read_buffer;
 	const unsigned int patterns[] = {0xaa, 0x55, 0xff, 0x00};
 	const unsigned int *pattern;
 	int i, try, got, nr_pattern, pat_idx;
 	unsigned int bb_count = 0;
-	blk_t recover_block = ~0;
+	blk64_t recover_block = ~0;
 
 	/* set up abend handler */
 	capture_terminate(NULL);
@@ -716,13 +717,13 @@ static unsigned int test_rw (int dev, blk_t last_block,
 }
 
 struct saved_blk_record {
-	blk_t	block;
+	blk64_t	block;
 	int	num;
 };
 
-static unsigned int test_nd (int dev, blk_t last_block,
-			     int block_size, blk_t first_block,
-			     unsigned int blocks_at_once)
+static unsigned int test_nd(int dev, blk64_t last_block,
+			    int block_size, blk64_t first_block,
+			    unsigned int blocks_at_once)
 {
 	unsigned char *blkbuf, *save_ptr, *test_ptr, *read_ptr;
 	unsigned char *test_base, *save_base, *read_base;
@@ -731,7 +732,7 @@ static unsigned int test_nd (int dev, blk_t last_block,
 	const unsigned int *pattern;
 	int nr_pattern, pat_idx;
 	int got, used2, written;
-	blk_t save_currently_testing;
+	blk64_t save_currently_testing;
 	struct saved_blk_record *test_record;
 	/* This is static to prevent being clobbered by the longjmp */
 	static int num_saved;
@@ -740,7 +741,7 @@ static unsigned int test_nd (int dev, blk_t last_block,
 	unsigned long buf_used;
 	static unsigned int bb_count;
 	unsigned int granularity = blocks_at_once;
-	blk_t recover_block = ~0U;
+	blk64_t recover_block = ~0ULL;
 
 	bb_count = 0;
 	errcode = ext2fs_badblocks_list_iterate_begin(bb_list,&bb_iter);
@@ -750,7 +751,7 @@ static unsigned int test_nd (int dev, blk_t last_block,
 		exit (1);
 	}
 	do {
-		ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
+		ext2fs_badblocks_list_iterate2(bb_iter, &next_bad);
 	} while (next_bad && next_bad < first_block);
 
 	blkbuf = allocate_buffer(3 * blocks_at_once * block_size);
@@ -827,7 +828,8 @@ static unsigned int test_nd (int dev, blk_t last_block,
 			if (next_bad) {
 				if (currently_testing == next_bad) {
 					/* fprintf (out, "%lu\n", nextbad); */
-					ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
+					ext2fs_badblocks_list_iterate2(bb_iter,
+							&next_bad);
 					currently_testing++;
 					goto check_for_more;
 				}
@@ -1042,8 +1044,8 @@ int main (int argc, char ** argv)
 	int dev;
 	errcode_t errcode;
 	unsigned int pattern;
-	unsigned int (*test_func)(int, blk_t,
-				  int, blk_t,
+	unsigned int (*test_func)(int, blk64_t,
+				  int, blk64_t,
 				  unsigned int);
 	int open_flag;
 	long sysval;
@@ -1262,14 +1264,14 @@ int main (int argc, char ** argv)
 
 	if (in) {
 		for(;;) {
-			switch(fscanf (in, "%u\n", &next_bad)) {
+			switch (fscanf(in, "%lluu\n", &next_bad)) {
 				case 0:
 					com_err (program_name, 0, "input file - bad format");
 					exit (1);
 				case EOF:
 					break;
 				default:
-					errcode = ext2fs_badblocks_list_add(bb_list,next_bad);
+					errcode = ext2fs_badblocks_list_add2(bb_list, next_bad);
 					if (errcode) {
 						com_err (program_name, errcode, _("while adding to in-memory bad block list"));
 						exit (1);
@@ -1295,8 +1297,10 @@ int main (int argc, char ** argv)
 
 		if (v_flag)
 			fprintf(stderr,
-				_("Pass completed, %u bad blocks found. (%d/%d/%d errors)\n"),
-				bb_count, num_read_errors, num_write_errors, num_corruption_errors);
+				_("Pass completed, %u bad blocks found. "
+				  "(%llu/%llu/%llu errors)\n"),
+				bb_count, num_read_errors, num_write_errors,
+				num_corruption_errors);
 
 	} while (passes_clean < num_passes);
 

Re: [PATCH 19/31] badblocks: Use the new badblocks APIs for 64-bit block numbers

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:28:44 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 19/31] badblocks: Use the new badblocks APIs for 64-bit block
>     numbers
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  misc/badblocks.c |   76 ++++++++++++++++++++++++++++--------------------------
>  1 file changed, 40 insertions(+), 36 deletions(-)
> 
> 
> diff --git a/misc/badblocks.c b/misc/badblocks.c
> index c9e47c7..660248e 100644
> --- a/misc/badblocks.c
> +++ b/misc/badblocks.c
> @@ -105,14 +105,14 @@ static void exclusive_usage(void)
>  	exit(1);
>  }
>  
> -static blk_t currently_testing = 0;
> -static blk_t num_blocks = 0;
> -static blk_t num_read_errors = 0;
> -static blk_t num_write_errors = 0;
> -static blk_t num_corruption_errors = 0;
> +static blk64_t currently_testing = 0;
> +static blk64_t num_blocks = 0;
> +static blk64_t num_read_errors = 0;
> +static blk64_t num_write_errors = 0;
> +static blk64_t num_corruption_errors = 0;
>  static ext2_badblocks_list bb_list = NULL;
>  static FILE *out;
> -static blk_t next_bad = 0;
> +static blk64_t next_bad = 0;
>  static ext2_badblocks_iterate bb_iter = NULL;
>  
>  enum error_types { READ_ERROR, WRITE_ERROR, CORRUPTION_ERROR };
> @@ -144,17 +144,17 @@ static void *allocate_buffer(size_t size)
>   * This routine reports a new bad block.  If the bad block has already
>   * been seen before, then it returns 0; otherwise it returns 1.
>   */
> -static int bb_output (blk_t bad, enum error_types error_type)
> +static int bb_output(blk64_t bad, enum error_types error_type)
>  {
>  	errcode_t errcode;
>  
> -	if (ext2fs_badblocks_list_test(bb_list, bad))
> +	if (ext2fs_badblocks_list_test2(bb_list, bad))
>  		return 0;
>  
>  	fprintf(out, "%lu\n", (unsigned long) bad);
>  	fflush(out);
>  
> -	errcode = ext2fs_badblocks_list_add (bb_list, bad);
> +	errcode = ext2fs_badblocks_list_add2(bb_list, bad);
>  	if (errcode) {
>  		com_err (program_name, errcode, "adding to in-memory bad block list");
>  		exit (1);
> @@ -165,7 +165,7 @@ static int bb_output (blk_t bad, enum error_types error_type)
>  	   an element was just added before the current iteration
>  	   position.  This should not cause next_bad to change. */
>  	if (bb_iter && bad < next_bad)
> -		ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
> +		ext2fs_badblocks_list_iterate2(bb_iter, &next_bad);
>  
>  	if (error_type == READ_ERROR) {
>  	  num_read_errors++;
> @@ -216,7 +216,7 @@ static void print_status(void)
>  	gettimeofday(&time_end, 0);
>  	len = snprintf(line_buf, sizeof(line_buf), 
>  		       _("%6.2f%% done, %s elapsed. "
> -		         "(%d/%d/%d errors)"),
> +			 "(%llu/%llu/%llu errors)"),
>  		       calc_percent((unsigned long) currently_testing,
>  				    (unsigned long) num_blocks), 
>  		       time_diff_format(&time_end, &time_start, diff_buf),
> @@ -350,7 +350,7 @@ static void pattern_fill(unsigned char *buffer, unsigned int pattern,
>   *    successfully sequentially read.
>   */
>  static int do_read (int dev, unsigned char * buffer, int try, int block_size,
> -		    blk_t current_block)
> +		    blk64_t current_block)
>  {
>  	long got;
>  	struct timeval tv1, tv2;
> @@ -469,16 +469,16 @@ static void flush_bufs(void)
>  		com_err(program_name, retval, _("during ext2fs_sync_device"));
>  }
>  
> -static unsigned int test_ro (int dev, blk_t last_block,
> -			     int block_size, blk_t first_block,
> -			     unsigned int blocks_at_once)
> +static unsigned int test_ro(int dev, blk64_t last_block,
> +			    int block_size, blk64_t first_block,
> +			    unsigned int blocks_at_once)
>  {
>  	unsigned char * blkbuf;
>  	int try;

So blocks_at_once can be unsigned int, but try is int. Later on we
store blocks_at_once in try which can cause problems.

>  	int got;

same here, even though do_read() returns int. However the actual
variable 'got' is long (and try is yet again int) it is really mess
;)

Thanks!
-Lukas

>  	unsigned int bb_count = 0;
>  	errcode_t errcode;
> -	blk_t recover_block = ~0;
> +	blk64_t recover_block = ~0;
>  
>  	/* set up abend handler */
>  	capture_terminate(NULL);
> @@ -490,7 +490,7 @@ static unsigned int test_ro (int dev, blk_t last_block,
>  		exit (1);
>  	}
>  	do {
> -		ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
> +		ext2fs_badblocks_list_iterate2(bb_iter, &next_bad);
>  	} while (next_bad && next_bad < first_block);
>  
>  	if (t_flag) {
> @@ -532,7 +532,8 @@ static unsigned int test_ro (int dev, blk_t last_block,
>  		if (next_bad) {
>  			if (currently_testing == next_bad) {
>  				/* fprintf (out, "%lu\n", nextbad); */
> -				ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
> +				ext2fs_badblocks_list_iterate2(bb_iter,
> +						&next_bad);
>  				currently_testing++;
>  				continue;
>  			}
> @@ -581,16 +582,16 @@ static unsigned int test_ro (int dev, blk_t last_block,
>  	return bb_count;
>  }
>  
> -static unsigned int test_rw (int dev, blk_t last_block,
> -			     int block_size, blk_t first_block,
> -			     unsigned int blocks_at_once)
> +static unsigned int test_rw(int dev, blk64_t last_block,
> +			    int block_size, blk64_t first_block,
> +			    unsigned int blocks_at_once)
>  {
>  	unsigned char *buffer, *read_buffer;
>  	const unsigned int patterns[] = {0xaa, 0x55, 0xff, 0x00};
>  	const unsigned int *pattern;
>  	int i, try, got, nr_pattern, pat_idx;
>  	unsigned int bb_count = 0;
> -	blk_t recover_block = ~0;
> +	blk64_t recover_block = ~0;
>  
>  	/* set up abend handler */
>  	capture_terminate(NULL);
> @@ -716,13 +717,13 @@ static unsigned int test_rw (int dev, blk_t last_block,
>  }
>  
>  struct saved_blk_record {
> -	blk_t	block;
> +	blk64_t	block;
>  	int	num;
>  };
>  
> -static unsigned int test_nd (int dev, blk_t last_block,
> -			     int block_size, blk_t first_block,
> -			     unsigned int blocks_at_once)
> +static unsigned int test_nd(int dev, blk64_t last_block,
> +			    int block_size, blk64_t first_block,
> +			    unsigned int blocks_at_once)
>  {
>  	unsigned char *blkbuf, *save_ptr, *test_ptr, *read_ptr;
>  	unsigned char *test_base, *save_base, *read_base;
> @@ -731,7 +732,7 @@ static unsigned int test_nd (int dev, blk_t last_block,
>  	const unsigned int *pattern;
>  	int nr_pattern, pat_idx;
>  	int got, used2, written;
> -	blk_t save_currently_testing;
> +	blk64_t save_currently_testing;
>  	struct saved_blk_record *test_record;
>  	/* This is static to prevent being clobbered by the longjmp */
>  	static int num_saved;
> @@ -740,7 +741,7 @@ static unsigned int test_nd (int dev, blk_t last_block,
>  	unsigned long buf_used;
>  	static unsigned int bb_count;
>  	unsigned int granularity = blocks_at_once;
> -	blk_t recover_block = ~0U;
> +	blk64_t recover_block = ~0ULL;
>  
>  	bb_count = 0;
>  	errcode = ext2fs_badblocks_list_iterate_begin(bb_list,&bb_iter);
> @@ -750,7 +751,7 @@ static unsigned int test_nd (int dev, blk_t last_block,
>  		exit (1);
>  	}
>  	do {
> -		ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
> +		ext2fs_badblocks_list_iterate2(bb_iter, &next_bad);
>  	} while (next_bad && next_bad < first_block);
>  
>  	blkbuf = allocate_buffer(3 * blocks_at_once * block_size);
> @@ -827,7 +828,8 @@ static unsigned int test_nd (int dev, blk_t last_block,
>  			if (next_bad) {
>  				if (currently_testing == next_bad) {
>  					/* fprintf (out, "%lu\n", nextbad); */
> -					ext2fs_badblocks_list_iterate (bb_iter, &next_bad);
> +					ext2fs_badblocks_list_iterate2(bb_iter,
> +							&next_bad);
>  					currently_testing++;
>  					goto check_for_more;
>  				}
> @@ -1042,8 +1044,8 @@ int main (int argc, char ** argv)
>  	int dev;
>  	errcode_t errcode;
>  	unsigned int pattern;
> -	unsigned int (*test_func)(int, blk_t,
> -				  int, blk_t,
> +	unsigned int (*test_func)(int, blk64_t,
> +				  int, blk64_t,
>  				  unsigned int);
>  	int open_flag;
>  	long sysval;
> @@ -1262,14 +1264,14 @@ int main (int argc, char ** argv)
>  
>  	if (in) {
>  		for(;;) {
> -			switch(fscanf (in, "%u\n", &next_bad)) {
> +			switch (fscanf(in, "%lluu\n", &next_bad)) {
>  				case 0:
>  					com_err (program_name, 0, "input file - bad format");
>  					exit (1);
>  				case EOF:
>  					break;
>  				default:
> -					errcode = ext2fs_badblocks_list_add(bb_list,next_bad);
> +					errcode = ext2fs_badblocks_list_add2(bb_list, next_bad);
>  					if (errcode) {
>  						com_err (program_name, errcode, _("while adding to in-memory bad block list"));
>  						exit (1);
> @@ -1295,8 +1297,10 @@ int main (int argc, char ** argv)
>  
>  		if (v_flag)
>  			fprintf(stderr,
> -				_("Pass completed, %u bad blocks found. (%d/%d/%d errors)\n"),
> -				bb_count, num_read_errors, num_write_errors, num_corruption_errors);
> +				_("Pass completed, %u bad blocks found. "
> +				  "(%llu/%llu/%llu errors)\n"),
> +				bb_count, num_read_errors, num_write_errors,
> +				num_corruption_errors);
>  
>  	} while (passes_clean < num_passes);
>  
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH 20/31] e2fsprogs: Add (optional) sparse checking to the build

[Darrick J. Wong]

Run sparse against source files when building e2fsprogs with 'make C=1'.  If
instead C=2, it configures basic ext2 types for bitwise checking with sparse,
which can help find the (many many) spots where conversion errors are
(possibly) happening.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 Makefile.in             |   13 +++++++++++++
 debugfs/Makefile.in     |    1 +
 e2fsck/Makefile.in      |    1 +
 ext2ed/Makefile.in      |    1 +
 intl/Makefile.in        |    1 +
 lib/blkid/Makefile.in   |    1 +
 lib/e2p/Makefile.in     |    1 +
 lib/et/Makefile.in      |    1 +
 lib/ext2fs/Makefile.in  |    1 +
 lib/ext2fs/ext2fs.h     |   22 ++++++++++++++--------
 lib/quota/Makefile.in   |    1 +
 lib/ss/Makefile.in      |    1 +
 lib/uuid/Makefile.in    |    1 +
 misc/Makefile.in        |    1 +
 resize/Makefile.in      |    1 +
 tests/progs/Makefile.in |    1 +
 util/Makefile.in        |    1 +
 17 files changed, 42 insertions(+), 8 deletions(-)


diff --git a/Makefile.in b/Makefile.in
index 544ed02..95cbf1b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -5,6 +5,19 @@ top_builddir = .
 my_dir = .
 INSTALL = @INSTALL@
 
+CHECK=sparse
+CHECK_OPTS=-Wsparse-all -Wno-transparent-union -Wno-return-void -Wno-undef -Wno-non-pointer-null
+ifeq ("$(C)", "2")
+  CHECK_CMD=$(CHECK) $(CHECK_OPTS) -Wbitwise -D__CHECK_ENDIAN__
+else
+  ifeq ("$(C)", "1")
+    CHECK_CMD=$(CHECK) $(CHECK_OPTS)
+   else
+    CHECK_CMD=@true
+  endif
+endif
+export CHECK_CMD
+
 @MCONFIG@
 
 % : %.sh
diff --git a/debugfs/Makefile.in b/debugfs/Makefile.in
index 9a86dc6..bf037e0 100644
--- a/debugfs/Makefile.in
+++ b/debugfs/Makefile.in
@@ -44,6 +44,7 @@ STATIC_DEPLIBS= $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBSS) \
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 all:: $(PROGS) $(MANPAGES)
 
diff --git a/e2fsck/Makefile.in b/e2fsck/Makefile.in
index 9c27571..6ecfcd9 100644
--- a/e2fsck/Makefile.in
+++ b/e2fsck/Makefile.in
@@ -38,6 +38,7 @@ COMPILE_ET=$(top_builddir)/lib/et/compile_et --build-tree
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 
 #
diff --git a/ext2ed/Makefile.in b/ext2ed/Makefile.in
index b3b606a..290f06e 100644
--- a/ext2ed/Makefile.in
+++ b/ext2ed/Makefile.in
@@ -33,6 +33,7 @@ DOCS=   doc/ext2ed-design.pdf doc/user-guide.pdf doc/ext2fs-overview.pdf \
 
 .c.o:
 	$(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(CHECK_CMD) $(ALL_CFLAGS) $<
 
 .SUFFIXES: .sgml .ps .pdf .html
 
diff --git a/intl/Makefile.in b/intl/Makefile.in
index 9dbc84e..bf24f63 100644
--- a/intl/Makefile.in
+++ b/intl/Makefile.in
@@ -191,6 +191,7 @@ LTV_AGE=4
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(COMPILE) $<
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 .y.c:
 	$(YACC) $(YFLAGS) --output $@ $<
diff --git a/lib/blkid/Makefile.in b/lib/blkid/Makefile.in
index 0ec8564..3f15c67 100644
--- a/lib/blkid/Makefile.in
+++ b/lib/blkid/Makefile.in
@@ -55,6 +55,7 @@ DEPLIBS_BLKID=	$(DEPSTATIC_LIBBLKID) $(DEPSTATIC_LIBUUID)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/e2p/Makefile.in b/lib/e2p/Makefile.in
index e2d0940..d6992fc 100644
--- a/lib/e2p/Makefile.in
+++ b/lib/e2p/Makefile.in
@@ -55,6 +55,7 @@ BSDLIB_INSTALL_DIR = $(root_libdir)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/et/Makefile.in b/lib/et/Makefile.in
index 8d1ea26..fd4e6e2 100644
--- a/lib/et/Makefile.in
+++ b/lib/et/Makefile.in
@@ -43,6 +43,7 @@ BSDLIB_INSTALL_DIR = $(root_libdir)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index dc7b0d1..e1e6216 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -202,6 +202,7 @@ all:: ext2fs.pc
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 847b33c..9fef6d3 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -64,14 +64,20 @@ extern "C" {
 #include <ext2fs/ext3_extents.h>
 #endif /* EXT2_FLAT_INCLUDES */
 
-typedef __u32		ext2_ino_t;
-typedef __u32		blk_t;
-typedef __u64		blk64_t;
-typedef __u32		dgrp_t;
-typedef __u32		ext2_off_t;
-typedef __u64		ext2_off64_t;
-typedef __s64		e2_blkcnt_t;
-typedef __u32		ext2_dirhash_t;
+#ifdef __CHECK_ENDIAN__
+#define __bitwise __attribute__((bitwise))
+#else
+#define __bitwise
+#endif
+
+typedef __u32 __bitwise		ext2_ino_t;
+typedef __u32 __bitwise		blk_t;
+typedef __u64 __bitwise		blk64_t;
+typedef __u32 __bitwise		dgrp_t;
+typedef __u32 __bitwise		ext2_off_t;
+typedef __u64 __bitwise		ext2_off64_t;
+typedef __s64 __bitwise		e2_blkcnt_t;
+typedef __u32 __bitwise		ext2_dirhash_t;
 
 #if EXT2_FLAT_INCLUDES
 #include "com_err.h"
diff --git a/lib/quota/Makefile.in b/lib/quota/Makefile.in
index 720befd..ec4f6c4 100644
--- a/lib/quota/Makefile.in
+++ b/lib/quota/Makefile.in
@@ -47,6 +47,7 @@ LIBDIR= quota
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 #ELF_CMT#	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/ss/Makefile.in b/lib/ss/Makefile.in
index c396f2d..1b23b38 100644
--- a/lib/ss/Makefile.in
+++ b/lib/ss/Makefile.in
@@ -34,6 +34,7 @@ MK_CMDS=_SS_DIR_OVERRIDE=. ./mk_cmds
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $<
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -DSHARED_ELF_LIB -fPIC -o elfshared/$*.o -c $<
diff --git a/lib/uuid/Makefile.in b/lib/uuid/Makefile.in
index 7329467..14d08c1 100644
--- a/lib/uuid/Makefile.in
+++ b/lib/uuid/Makefile.in
@@ -62,6 +62,7 @@ BSDLIB_INSTALL_DIR = $(root_libdir)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) $(ALL_CFLAGS) -c $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 @CHECKER_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -checker -g -o checker/$*.o -c $<
 @ELF_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -fPIC -o elfshared/$*.o -c $<
diff --git a/misc/Makefile.in b/misc/Makefile.in
index 8a69855..a798f96 100644
--- a/misc/Makefile.in
+++ b/misc/Makefile.in
@@ -100,6 +100,7 @@ COMPILE_ET=$(top_builddir)/lib/et/compile_et --build-tree
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 @PROFILE_CMT@	$(Q) $(CC) $(ALL_CFLAGS) -g -pg -o profiled/$*.o -c $<
 
 all:: profiled $(SPROGS) $(UPROGS) $(USPROGS) $(SMANPAGES) $(UMANPAGES) \
diff --git a/resize/Makefile.in b/resize/Makefile.in
index a06b642..b8fb3ae 100644
--- a/resize/Makefile.in
+++ b/resize/Makefile.in
@@ -38,6 +38,7 @@ DEPSTATIC_LIBS= $(STATIC_LIBE2P) $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBCOM_ERR)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 all:: $(PROGS) $(TEST_PROGS) $(MANPAGES) 
 
diff --git a/tests/progs/Makefile.in b/tests/progs/Makefile.in
index e3c1ef4..37abf23 100644
--- a/tests/progs/Makefile.in
+++ b/tests/progs/Makefile.in
@@ -27,6 +27,7 @@ DEPLIBS= $(LIBEXT2FS) $(DEPLIBSS) $(DEPLIBCOM_ERR)
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(CC) -c $(ALL_CFLAGS) $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 all:: $(PROGS)
 
diff --git a/util/Makefile.in b/util/Makefile.in
index adf0b46..76c3f88 100644
--- a/util/Makefile.in
+++ b/util/Makefile.in
@@ -16,6 +16,7 @@ SRCS = $(srcdir)/subst.c
 .c.o:
 	$(E) "	CC $<"
 	$(Q) $(BUILD_CC) -c $(BUILD_CFLAGS) $< -o $@
+	$(Q) $(CHECK_CMD) $(ALL_CFLAGS) $<
 
 PROGS=		subst symlinks
 

Re: [PATCH 20/31] e2fsprogs: Add (optional) sparse checking to the build

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:28:50PM -0700, Darrick J. Wong wrote:
> Run sparse against source files when building e2fsprogs with 'make C=1'.  If
> instead C=2, it configures basic ext2 types for bitwise checking with sparse,
> which can help find the (many many) spots where conversion errors are
> (possibly) happening.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

						- Ted

[PATCH 21/31] libext2fs: Be more thorough in searching a range of blocks for a cluster

[Darrick J. Wong]

implied_cluster_alloc() is written such that if the the user passes in a
logical block that is the zeroth block in a logical cluster (lblk %
cluster_ratio == 0), then it will assume that there is no physical cluster
mapped to any other part of the logical cluster.

This is not true if we happen to be allocating logical blocks in reverse order.
Therefore, search the whole cluster, except for the lblk that we passed in.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/bmap.c |   19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)


diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
index aadd22e..5074587 100644
--- a/lib/ext2fs/bmap.c
+++ b/lib/ext2fs/bmap.c
@@ -140,7 +140,7 @@ static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
 static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
 				       struct ext2_inode *inode,
 				       ext2_extent_handle_t handle,
-				       blk64_t block, blk64_t *phys_blk)
+				       blk64_t lblk, blk64_t *phys_blk)
 {
 	blk64_t	base_block, pblock = 0;
 	int i;
@@ -149,10 +149,19 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
 					EXT4_FEATURE_RO_COMPAT_BIGALLOC))
 		return 0;
 
-	base_block = block & ~EXT2FS_CLUSTER_MASK(fs);
+	base_block = lblk & ~EXT2FS_CLUSTER_MASK(fs);
+	/*
+	 * Except for the logical block (lblk) that was passed in, search all
+	 * blocks in this logical cluster for a mapping to a physical cluster.
+	 * If any such map exists, calculate the physical block that maps to
+	 * the logical block and return that.
+	 *
+	 * The old code wouldn't even look if (block % cluster_ratio) == 0;
+	 * this is incorrect if we're allocating blocks in reverse order.
+	 */
 	for (i = 0; i < EXT2FS_CLUSTER_RATIO(fs); i++) {
-		if (block == base_block)
-			return 0;
+		if (base_block + i == lblk)
+			continue;
 		extent_bmap(fs, ino, inode, handle, 0, 0,
 			    base_block + i, 0, 0, &pblock);
 		if (pblock)
@@ -160,7 +169,7 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
 	}
 	if (pblock == 0)
 		return 0;
-	*phys_blk = pblock - i + (block - base_block);
+	*phys_blk = pblock - i + (lblk - base_block);
 	return 0;
 }
 

Re: [PATCH 21/31] libext2fs: Be more thorough in searching a range of blocks for a cluster

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:28:56PM -0700, Darrick J. Wong wrote:
> implied_cluster_alloc() is written such that if the the user passes in a
> logical block that is the zeroth block in a logical cluster (lblk %
> cluster_ratio == 0), then it will assume that there is no physical cluster
> mapped to any other part of the logical cluster.
> 
> This is not true if we happen to be allocating logical blocks in reverse order.
> Therefore, search the whole cluster, except for the lblk that we passed in.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Applied, thanks.

						- Ted

[PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched

[Darrick J. Wong]

When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free any block
in a cluster has the effect of freeing the entire cluster.  This is problematic
if a caller instructs us to punch, say, blocks 12-15 of a 16-block cluster,
because blocks 0-11 now point to a "free" cluster.

The naive way to solve this problem is to see if any of the other blocks in
this logical cluster map to a physical cluster.  If so, then we know that the
cluster is still in use and it mustn't be freed.  Otherwise, we are punching
the last mapped block in this cluster, so we can free the cluster.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/bmap.c   |   28 ++++++++++++++++++++++++++++
 lib/ext2fs/ext2fs.h |    3 +++
 lib/ext2fs/punch.c  |   30 +++++++++++++++++++++++++-----
 3 files changed, 56 insertions(+), 5 deletions(-)


diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
index 5074587..a6e35a9 100644
--- a/lib/ext2fs/bmap.c
+++ b/lib/ext2fs/bmap.c
@@ -173,6 +173,34 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
 	return 0;
 }
 
+errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
+				   struct ext2_inode *inode, blk64_t lblk,
+				   blk64_t *pblk)
+{
+	ext2_extent_handle_t handle;
+	errcode_t retval;
+
+	/* Need bigalloc and extents to be enabled */
+	*pblk = 0;
+	if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+					EXT4_FEATURE_RO_COMPAT_BIGALLOC) ||
+	    !(inode->i_flags & EXT4_EXTENTS_FL))
+		return 0;
+
+	retval = ext2fs_extent_open2(fs, ino, inode, &handle);
+	if (retval)
+		goto out;
+
+	retval = implied_cluster_alloc(fs, ino, inode, handle, lblk, pblk);
+	if (retval)
+		goto out2;
+
+out2:
+	ext2fs_extent_free(handle);
+out:
+	return retval;
+}
+
 static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
 			     struct ext2_inode *inode,
 			     ext2_extent_handle_t handle,
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 9fef6d3..88da8db 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -925,6 +925,9 @@ extern errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino,
 			      struct ext2_inode *inode,
 			      char *block_buf, int bmap_flags, blk64_t block,
 			      int *ret_flags, blk64_t *phys_blk);
+errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
+				   struct ext2_inode *inode, blk64_t lblk,
+				   blk64_t *pblk);
 
 #if 0
 /* bmove.c */
diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
index 4471f46..e0193b0 100644
--- a/lib/ext2fs/punch.c
+++ b/lib/ext2fs/punch.c
@@ -183,8 +183,8 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 	ext2_extent_handle_t	handle = 0;
 	struct ext2fs_extent	extent;
 	errcode_t		retval;
-	blk64_t			free_start, next;
-	__u32			free_count, newlen;
+	blk64_t			free_start, next, lfree_start, pblk;
+	__u32			free_count, newlen, cluster_freed;
 	int			freed = 0;
 	int			op;
 
@@ -210,6 +210,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 			/* Start of deleted region before extent; 
 			   adjust beginning of extent */
 			free_start = extent.e_pblk;
+			lfree_start = extent.e_lblk;
 			if (next > end)
 				free_count = end - extent.e_lblk + 1;
 			else
@@ -225,6 +226,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 			dbg_printf("Case #%d\n", 2);
 			newlen = start - extent.e_lblk;
 			free_start = extent.e_pblk + newlen;
+			lfree_start = extent.e_lblk + newlen;
 			free_count = extent.e_len - newlen;
 			extent.e_len = newlen;
 		} else {
@@ -240,6 +242,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 
 			extent.e_len = start - extent.e_lblk;
 			free_start = extent.e_pblk + extent.e_len;
+			lfree_start = extent.e_lblk + extent.e_len;
 			free_count = end - start + 1;
 
 			dbg_print_extent("inserting", &newex);
@@ -280,9 +283,26 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
 			goto errout;
 		dbg_printf("Free start %llu, free count = %u\n",
 		       free_start, free_count);
-		while (free_count-- > 0) {
-			ext2fs_block_alloc_stats2(fs, free_start++, -1);
-			freed++;
+		while (free_count > 0) {
+			retval = ext2fs_map_cluster_block(fs, ino, inode,
+							  lfree_start, &pblk);
+			if (retval)
+				goto errout;
+			if (!pblk) {
+				ext2fs_block_alloc_stats2(fs, free_start, -1);
+				freed++;
+				cluster_freed = EXT2FS_CLUSTER_RATIO(fs) -
+					(free_start & EXT2FS_CLUSTER_MASK(fs));
+				if (cluster_freed > free_count)
+					cluster_freed = free_count;
+				free_count -= cluster_freed;
+				free_start += cluster_freed;
+				lfree_start += cluster_freed;
+				continue;
+			}
+			free_count--;
+			free_start++;
+			lfree_start++;
 		}
 	next_extent:
 		retval = ext2fs_extent_get(handle, op,

Re: [PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:29:03 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 22/31] libext2fs: During punch,
>     only free a cluster if we're sure that all blocks in the cluster are being
>      punched
> 
> When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free any block
> in a cluster has the effect of freeing the entire cluster.  This is problematic
> if a caller instructs us to punch, say, blocks 12-15 of a 16-block cluster,
> because blocks 0-11 now point to a "free" cluster.
> 
> The naive way to solve this problem is to see if any of the other blocks in
> this logical cluster map to a physical cluster.  If so, then we know that the
> cluster is still in use and it mustn't be freed.  Otherwise, we are punching
> the last mapped block in this cluster, so we can free the cluster.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/bmap.c   |   28 ++++++++++++++++++++++++++++
>  lib/ext2fs/ext2fs.h |    3 +++
>  lib/ext2fs/punch.c  |   30 +++++++++++++++++++++++++-----
>  3 files changed, 56 insertions(+), 5 deletions(-)
> 
> 
> diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
> index 5074587..a6e35a9 100644
> --- a/lib/ext2fs/bmap.c
> +++ b/lib/ext2fs/bmap.c
> @@ -173,6 +173,34 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
>  	return 0;
>  }
>  
> +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> +				   struct ext2_inode *inode, blk64_t lblk,
> +				   blk64_t *pblk)
> +{
> +	ext2_extent_handle_t handle;
> +	errcode_t retval;
> +
> +	/* Need bigalloc and extents to be enabled */
> +	*pblk = 0;
> +	if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
> +					EXT4_FEATURE_RO_COMPAT_BIGALLOC) ||
> +	    !(inode->i_flags & EXT4_EXTENTS_FL))
> +		return 0;
> +
> +	retval = ext2fs_extent_open2(fs, ino, inode, &handle);
> +	if (retval)
> +		goto out;
> +
> +	retval = implied_cluster_alloc(fs, ino, inode, handle, lblk, pblk);
> +	if (retval)
> +		goto out2;
> +
> +out2:
> +	ext2fs_extent_free(handle);
> +out:
> +	return retval;
> +}
> +
>  static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
>  			     struct ext2_inode *inode,
>  			     ext2_extent_handle_t handle,
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index 9fef6d3..88da8db 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -925,6 +925,9 @@ extern errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino,
>  			      struct ext2_inode *inode,
>  			      char *block_buf, int bmap_flags, blk64_t block,
>  			      int *ret_flags, blk64_t *phys_blk);
> +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> +				   struct ext2_inode *inode, blk64_t lblk,
> +				   blk64_t *pblk);
>  
>  #if 0
>  /* bmove.c */
> diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
> index 4471f46..e0193b0 100644
> --- a/lib/ext2fs/punch.c
> +++ b/lib/ext2fs/punch.c
> @@ -183,8 +183,8 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
>  	ext2_extent_handle_t	handle = 0;
>  	struct ext2fs_extent	extent;
>  	errcode_t		retval;
> -	blk64_t			free_start, next;
> -	__u32			free_count, newlen;
> +	blk64_t			free_start, next, lfree_start, pblk;
> +	__u32			free_count, newlen, cluster_freed;
>  	int			freed = 0;
>  	int			op;
>  
> @@ -210,6 +210,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
>  			/* Start of deleted region before extent; 
>  			   adjust beginning of extent */
>  			free_start = extent.e_pblk;
> +			lfree_start = extent.e_lblk;
>  			if (next > end)
>  				free_count = end - extent.e_lblk + 1;
>  			else
> @@ -225,6 +226,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
>  			dbg_printf("Case #%d\n", 2);
>  			newlen = start - extent.e_lblk;
>  			free_start = extent.e_pblk + newlen;
> +			lfree_start = extent.e_lblk + newlen;
>  			free_count = extent.e_len - newlen;
>  			extent.e_len = newlen;
>  		} else {
> @@ -240,6 +242,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
>  
>  			extent.e_len = start - extent.e_lblk;
>  			free_start = extent.e_pblk + extent.e_len;
> +			lfree_start = extent.e_lblk + extent.e_len;
>  			free_count = end - start + 1;
>  
>  			dbg_print_extent("inserting", &newex);
> @@ -280,9 +283,26 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
>  			goto errout;
>  		dbg_printf("Free start %llu, free count = %u\n",
>  		       free_start, free_count);
> -		while (free_count-- > 0) {
> -			ext2fs_block_alloc_stats2(fs, free_start++, -1);
> -			freed++;
> +		while (free_count > 0) {
> +			retval = ext2fs_map_cluster_block(fs, ino, inode,
> +							  lfree_start, &pblk);
> +			if (retval)
> +				goto errout;
> +			if (!pblk) {
> +				ext2fs_block_alloc_stats2(fs, free_start, -1);
> +				freed++;
> +				cluster_freed = EXT2FS_CLUSTER_RATIO(fs) -
> +					(free_start & EXT2FS_CLUSTER_MASK(fs));
> +				if (cluster_freed > free_count)
> +					cluster_freed = free_count;
> +				free_count -= cluster_freed;
> +				free_start += cluster_freed;
> +				lfree_start += cluster_freed;
> +				continue;
> +			}
> +			free_count--;
> +			free_start++;
> +			lfree_start++;

I think that this is a little bit excessive. What I think we should
do here is to identify first and last partial cluster and possibly
call ext2fs_map_cluster_block() for those since we might or might
not want to free then depending on whether there are other blocks in
it in-use.

Then just iterate over the whole clusters in between and free them
all. Having to call ext2fs_map_cluster_block() for every single
block we're freeing from the extent tree is not really necessary I think
especially since we really need to get this information for those
possibly partial clusters at the start and end of the extent.

Thanks!
-Lukas


>  		}
>  	next_extent:
>  		retval = ext2fs_extent_get(handle, op,
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched

[Darrick J. Wong]

On Thu, Oct 10, 2013 at 05:53:37PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> 
> > Date: Mon, 30 Sep 2013 18:29:03 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 22/31] libext2fs: During punch,
> >     only free a cluster if we're sure that all blocks in the cluster are being
> >      punched
> > 
> > When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free any block
> > in a cluster has the effect of freeing the entire cluster.  This is problematic
> > if a caller instructs us to punch, say, blocks 12-15 of a 16-block cluster,
> > because blocks 0-11 now point to a "free" cluster.
> > 
> > The naive way to solve this problem is to see if any of the other blocks in
> > this logical cluster map to a physical cluster.  If so, then we know that the
> > cluster is still in use and it mustn't be freed.  Otherwise, we are punching
> > the last mapped block in this cluster, so we can free the cluster.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  lib/ext2fs/bmap.c   |   28 ++++++++++++++++++++++++++++
> >  lib/ext2fs/ext2fs.h |    3 +++
> >  lib/ext2fs/punch.c  |   30 +++++++++++++++++++++++++-----
> >  3 files changed, 56 insertions(+), 5 deletions(-)
> > 
> > 
> > diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
> > index 5074587..a6e35a9 100644
> > --- a/lib/ext2fs/bmap.c
> > +++ b/lib/ext2fs/bmap.c
> > @@ -173,6 +173,34 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
> >  	return 0;
> >  }
> >  
> > +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> > +				   struct ext2_inode *inode, blk64_t lblk,
> > +				   blk64_t *pblk)
> > +{
> > +	ext2_extent_handle_t handle;
> > +	errcode_t retval;
> > +
> > +	/* Need bigalloc and extents to be enabled */
> > +	*pblk = 0;
> > +	if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
> > +					EXT4_FEATURE_RO_COMPAT_BIGALLOC) ||
> > +	    !(inode->i_flags & EXT4_EXTENTS_FL))
> > +		return 0;
> > +
> > +	retval = ext2fs_extent_open2(fs, ino, inode, &handle);
> > +	if (retval)
> > +		goto out;
> > +
> > +	retval = implied_cluster_alloc(fs, ino, inode, handle, lblk, pblk);
> > +	if (retval)
> > +		goto out2;
> > +
> > +out2:
> > +	ext2fs_extent_free(handle);
> > +out:
> > +	return retval;
> > +}
> > +
> >  static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
> >  			     struct ext2_inode *inode,
> >  			     ext2_extent_handle_t handle,
> > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > index 9fef6d3..88da8db 100644
> > --- a/lib/ext2fs/ext2fs.h
> > +++ b/lib/ext2fs/ext2fs.h
> > @@ -925,6 +925,9 @@ extern errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino,
> >  			      struct ext2_inode *inode,
> >  			      char *block_buf, int bmap_flags, blk64_t block,
> >  			      int *ret_flags, blk64_t *phys_blk);
> > +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> > +				   struct ext2_inode *inode, blk64_t lblk,
> > +				   blk64_t *pblk);
> >  
> >  #if 0
> >  /* bmove.c */
> > diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
> > index 4471f46..e0193b0 100644
> > --- a/lib/ext2fs/punch.c
> > +++ b/lib/ext2fs/punch.c
> > @@ -183,8 +183,8 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> >  	ext2_extent_handle_t	handle = 0;
> >  	struct ext2fs_extent	extent;
> >  	errcode_t		retval;
> > -	blk64_t			free_start, next;
> > -	__u32			free_count, newlen;
> > +	blk64_t			free_start, next, lfree_start, pblk;
> > +	__u32			free_count, newlen, cluster_freed;
> >  	int			freed = 0;
> >  	int			op;
> >  
> > @@ -210,6 +210,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> >  			/* Start of deleted region before extent; 
> >  			   adjust beginning of extent */
> >  			free_start = extent.e_pblk;
> > +			lfree_start = extent.e_lblk;
> >  			if (next > end)
> >  				free_count = end - extent.e_lblk + 1;
> >  			else
> > @@ -225,6 +226,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> >  			dbg_printf("Case #%d\n", 2);
> >  			newlen = start - extent.e_lblk;
> >  			free_start = extent.e_pblk + newlen;
> > +			lfree_start = extent.e_lblk + newlen;
> >  			free_count = extent.e_len - newlen;
> >  			extent.e_len = newlen;
> >  		} else {
> > @@ -240,6 +242,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> >  
> >  			extent.e_len = start - extent.e_lblk;
> >  			free_start = extent.e_pblk + extent.e_len;
> > +			lfree_start = extent.e_lblk + extent.e_len;
> >  			free_count = end - start + 1;
> >  
> >  			dbg_print_extent("inserting", &newex);
> > @@ -280,9 +283,26 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> >  			goto errout;
> >  		dbg_printf("Free start %llu, free count = %u\n",
> >  		       free_start, free_count);
> > -		while (free_count-- > 0) {
> > -			ext2fs_block_alloc_stats2(fs, free_start++, -1);
> > -			freed++;
> > +		while (free_count > 0) {
> > +			retval = ext2fs_map_cluster_block(fs, ino, inode,
> > +							  lfree_start, &pblk);
> > +			if (retval)
> > +				goto errout;
> > +			if (!pblk) {
> > +				ext2fs_block_alloc_stats2(fs, free_start, -1);
> > +				freed++;
> > +				cluster_freed = EXT2FS_CLUSTER_RATIO(fs) -
> > +					(free_start & EXT2FS_CLUSTER_MASK(fs));
> > +				if (cluster_freed > free_count)
> > +					cluster_freed = free_count;
> > +				free_count -= cluster_freed;
> > +				free_start += cluster_freed;
> > +				lfree_start += cluster_freed;
> > +				continue;
> > +			}
> > +			free_count--;
> > +			free_start++;
> > +			lfree_start++;
> 
> I think that this is a little bit excessive. What I think we should
> do here is to identify first and last partial cluster and possibly
> call ext2fs_map_cluster_block() for those since we might or might
> not want to free then depending on whether there are other blocks in
> it in-use.
> 
> Then just iterate over the whole clusters in between and free them
> all. Having to call ext2fs_map_cluster_block() for every single
> block we're freeing from the extent tree is not really necessary I think
> especially since we really need to get this information for those
> possibly partial clusters at the start and end of the extent.

Hmm.  I think I could eliminate the middle ext2fs_map_cluster_block() calls by
only calling it if free_start is within cluster_ratio blocks of the pre-loop
value of free_start, or if free_count < cluster_ratio.  I can also split the
whole thing into three loops (pre-cluster, clusters, and post-cluster), though
for the non-bigalloc case I'd skip to the middle loop.

--D
> 
> Thanks!
> -Lukas
> 
> 
> >  		}
> >  	next_extent:
> >  		retval = ext2fs_extent_get(handle, op,
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 23/31] libext2fs: expanddir and mkjournal need not update the summary counts when performing an implied cluster allocation

[Darrick J. Wong]

When we're appending a block to a directory file or the journal file, and the
new block is part of a cluster that has already been allocated to the file
(implied cluster allocation), don't update the bitmap or the summary counts
because that was performed when the cluster was allocated.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/expanddir.c |    2 +-
 lib/ext2fs/mkjournal.c |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/lib/ext2fs/expanddir.c b/lib/ext2fs/expanddir.c
index 22558d6..09a15fa 100644
--- a/lib/ext2fs/expanddir.c
+++ b/lib/ext2fs/expanddir.c
@@ -55,6 +55,7 @@ static int expand_dir_proc(ext2_filsys	fs,
 			return BLOCK_ABORT;
 		}
 		es->newblocks++;
+		ext2fs_block_alloc_stats2(fs, new_blk, +1);
 	}
 	if (blockcnt > 0) {
 		retval = ext2fs_new_dir_block(fs, 0, 0, &block);
@@ -82,7 +83,6 @@ static int expand_dir_proc(ext2_filsys	fs,
 	}
 	ext2fs_free_mem(&block);
 	*blocknr = new_blk;
-	ext2fs_block_alloc_stats2(fs, new_blk, +1);
 
 	if (es->done)
 		return (BLOCK_CHANGED | BLOCK_ABORT);
diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
index c636a97..dd3c35b 100644
--- a/lib/ext2fs/mkjournal.c
+++ b/lib/ext2fs/mkjournal.c
@@ -250,6 +250,7 @@ static int mkjournal_proc(ext2_filsys	fs,
 			es->err = retval;
 			return BLOCK_ABORT;
 		}
+		ext2fs_block_alloc_stats2(fs, new_blk, +1);
 		es->newblocks++;
 	}
 	if (blockcnt >= 0)
@@ -285,7 +286,6 @@ static int mkjournal_proc(ext2_filsys	fs,
 		return BLOCK_ABORT;
 	}
 	*blocknr = es->goal = new_blk;
-	ext2fs_block_alloc_stats2(fs, new_blk, +1);
 
 	if (es->num_blocks == 0)
 		return (BLOCK_CHANGED | BLOCK_ABORT);

Re: [PATCH 23/31] libext2fs: expanddir and mkjournal need not update the summary counts when performing an implied cluster allocation

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:29:09 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 23/31] libext2fs: expanddir and mkjournal need not update the
>     summary counts when performing an implied cluster allocation
> 
> When we're appending a block to a directory file or the journal file, and the
> new block is part of a cluster that has already been allocated to the file
> (implied cluster allocation), don't update the bitmap or the summary counts
> because that was performed when the cluster was allocated.

Looks good.

Reviewed-by: Lukas Czerner <lczerner@redhat.com>

> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/expanddir.c |    2 +-
>  lib/ext2fs/mkjournal.c |    2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> 
> diff --git a/lib/ext2fs/expanddir.c b/lib/ext2fs/expanddir.c
> index 22558d6..09a15fa 100644
> --- a/lib/ext2fs/expanddir.c
> +++ b/lib/ext2fs/expanddir.c
> @@ -55,6 +55,7 @@ static int expand_dir_proc(ext2_filsys	fs,
>  			return BLOCK_ABORT;
>  		}
>  		es->newblocks++;
> +		ext2fs_block_alloc_stats2(fs, new_blk, +1);
>  	}
>  	if (blockcnt > 0) {
>  		retval = ext2fs_new_dir_block(fs, 0, 0, &block);
> @@ -82,7 +83,6 @@ static int expand_dir_proc(ext2_filsys	fs,
>  	}
>  	ext2fs_free_mem(&block);
>  	*blocknr = new_blk;
> -	ext2fs_block_alloc_stats2(fs, new_blk, +1);
>  
>  	if (es->done)
>  		return (BLOCK_CHANGED | BLOCK_ABORT);
> diff --git a/lib/ext2fs/mkjournal.c b/lib/ext2fs/mkjournal.c
> index c636a97..dd3c35b 100644
> --- a/lib/ext2fs/mkjournal.c
> +++ b/lib/ext2fs/mkjournal.c
> @@ -250,6 +250,7 @@ static int mkjournal_proc(ext2_filsys	fs,
>  			es->err = retval;
>  			return BLOCK_ABORT;
>  		}
> +		ext2fs_block_alloc_stats2(fs, new_blk, +1);
>  		es->newblocks++;
>  	}
>  	if (blockcnt >= 0)
> @@ -285,7 +286,6 @@ static int mkjournal_proc(ext2_filsys	fs,
>  		return BLOCK_ABORT;
>  	}
>  	*blocknr = es->goal = new_blk;
> -	ext2fs_block_alloc_stats2(fs, new_blk, +1);
>  
>  	if (es->num_blocks == 0)
>  		return (BLOCK_CHANGED | BLOCK_ABORT);
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH 24/31] libext2fs: Use ext2fs_punch() to truncate quota file

[Darrick J. Wong]

Use the new ext2fs_punch() call to truncate the quota file.  This also
eliminates the need to fix it to work with bigalloc.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/quota/quotaio.c |   19 +++----------------
 1 file changed, 3 insertions(+), 16 deletions(-)


diff --git a/lib/quota/quotaio.c b/lib/quota/quotaio.c
index 8ddb92a..807fabd 100644
--- a/lib/quota/quotaio.c
+++ b/lib/quota/quotaio.c
@@ -98,19 +98,6 @@ void update_grace_times(struct dquot *q)
 	}
 }
 
-static int release_blocks_proc(ext2_filsys fs, blk64_t *blocknr,
-			       e2_blkcnt_t blockcnt EXT2FS_ATTR((unused)),
-			       blk64_t ref_block EXT2FS_ATTR((unused)),
-			       int ref_offset EXT2FS_ATTR((unused)),
-			       void *private EXT2FS_ATTR((unused)))
-{
-	blk64_t	block;
-
-	block = *blocknr;
-	ext2fs_block_alloc_stats2(fs, block, -1);
-	return 0;
-}
-
 static int compute_num_blocks_proc(ext2_filsys fs, blk64_t *blocknr,
 			       e2_blkcnt_t blockcnt EXT2FS_ATTR((unused)),
 			       blk64_t ref_block EXT2FS_ATTR((unused)),
@@ -135,9 +122,9 @@ errcode_t quota_inode_truncate(ext2_filsys fs, ext2_ino_t ino)
 		inode.i_dtime = fs->now ? fs->now : time(0);
 		if (!ext2fs_inode_has_valid_blocks2(fs, &inode))
 			return 0;

Re: [PATCH 24/31] libext2fs: Use ext2fs_punch() to truncate quota file

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:29:15 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 24/31] libext2fs: Use ext2fs_punch() to truncate quota file
> 
> Use the new ext2fs_punch() call to truncate the quota file.  This also
> eliminates the need to fix it to work with bigalloc.

Looks good.

Reviewed-by: Lukas Czerner <lczerner@redhat.com>

> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/quota/quotaio.c |   19 +++----------------
>  1 file changed, 3 insertions(+), 16 deletions(-)
> 
> 
> diff --git a/lib/quota/quotaio.c b/lib/quota/quotaio.c
> index 8ddb92a..807fabd 100644
> --- a/lib/quota/quotaio.c
> +++ b/lib/quota/quotaio.c
> @@ -98,19 +98,6 @@ void update_grace_times(struct dquot *q)
>  	}
>  }
>  
> -static int release_blocks_proc(ext2_filsys fs, blk64_t *blocknr,
> -			       e2_blkcnt_t blockcnt EXT2FS_ATTR((unused)),
> -			       blk64_t ref_block EXT2FS_ATTR((unused)),
> -			       int ref_offset EXT2FS_ATTR((unused)),
> -			       void *private EXT2FS_ATTR((unused)))
> -{
> -	blk64_t	block;
> -
> -	block = *blocknr;
> -	ext2fs_block_alloc_stats2(fs, block, -1);
> -	return 0;
> -}
> -
>  static int compute_num_blocks_proc(ext2_filsys fs, blk64_t *blocknr,
>  			       e2_blkcnt_t blockcnt EXT2FS_ATTR((unused)),
>  			       blk64_t ref_block EXT2FS_ATTR((unused)),
> @@ -135,9 +122,9 @@ errcode_t quota_inode_truncate(ext2_filsys fs, ext2_ino_t ino)
>  		inode.i_dtime = fs->now ? fs->now : time(0);
>  		if (!ext2fs_inode_has_valid_blocks2(fs, &inode))
>  			return 0;
> -
> -		ext2fs_block_iterate3(fs, ino, BLOCK_FLAG_READ_ONLY, NULL,
> -				      release_blocks_proc, NULL);
> +		err = ext2fs_punch(fs, ino, NULL, NULL, 0, ~0ULL);
> +		if (err)
> +			return err;
>  		fs->flags &= ~EXT2_FLAG_SUPER_ONLY;
>  		memset(&inode, 0, sizeof(struct ext2_inode));
>  	} else {
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH 25/31] e2fsck: Only release clusters when shortening a directory during a rehash

[Darrick J. Wong]

When the rehash process is running on a bigalloc filesystem, it compresses all
the directory entries and hash structures into the beginning of the directory
file and then uses block_iterate3() to free the blocks off the end of the file.
It seems to call ext2fs_block_alloc_stats2() for every block in a cluster,
which is unfortunate because this function allocates and frees entire clusters
(and updates the summary counts accordingly).  In this case e2fsck writes out
incorrect summary counts.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 e2fsck/rehash.c |   14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)


diff --git a/e2fsck/rehash.c b/e2fsck/rehash.c
index 6ef3568..29da9a1 100644
--- a/e2fsck/rehash.c
+++ b/e2fsck/rehash.c
@@ -719,10 +719,18 @@ static int write_dir_block(ext2_filsys fs,
 		/* We don't need this block, so release it */
 		e2fsck_read_bitmaps(wd->ctx);
 		blk = *block_nr;
-		ext2fs_unmark_block_bitmap2(wd->ctx->block_found_map, blk);
-		ext2fs_block_alloc_stats2(fs, blk, -1);
+		/*
+		 * In theory, we only release blocks from the end of the
+		 * directory file, so it's fine to clobber a whole cluster at
+		 * once.
+		 */
+		if (blk % EXT2FS_CLUSTER_RATIO(fs) == 0) {
+			ext2fs_unmark_block_bitmap2(wd->ctx->block_found_map,
+						    blk);
+			ext2fs_block_alloc_stats2(fs, blk, -1);
+			wd->cleared++;
+		}
 		*block_nr = 0;
-		wd->cleared++;
 		return BLOCK_CHANGED;
 	}
 

Re: [PATCH 25/31] e2fsck: Only release clusters when shortening a directory during a rehash

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:29:21 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 25/31] e2fsck: Only release clusters when shortening a
>     directory during a rehash
> 
> When the rehash process is running on a bigalloc filesystem, it compresses all
> the directory entries and hash structures into the beginning of the directory
> file and then uses block_iterate3() to free the blocks off the end of the file.
> It seems to call ext2fs_block_alloc_stats2() for every block in a cluster,
> which is unfortunate because this function allocates and frees entire clusters
> (and updates the summary counts accordingly).  In this case e2fsck writes out
> incorrect summary counts.


Looks good.

Reviewed-by: Lukas Czerner <lczerner@redhat.com>

> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  e2fsck/rehash.c |   14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> 
> diff --git a/e2fsck/rehash.c b/e2fsck/rehash.c
> index 6ef3568..29da9a1 100644
> --- a/e2fsck/rehash.c
> +++ b/e2fsck/rehash.c
> @@ -719,10 +719,18 @@ static int write_dir_block(ext2_filsys fs,
>  		/* We don't need this block, so release it */
>  		e2fsck_read_bitmaps(wd->ctx);
>  		blk = *block_nr;
> -		ext2fs_unmark_block_bitmap2(wd->ctx->block_found_map, blk);
> -		ext2fs_block_alloc_stats2(fs, blk, -1);
> +		/*
> +		 * In theory, we only release blocks from the end of the
> +		 * directory file, so it's fine to clobber a whole cluster at
> +		 * once.
> +		 */
> +		if (blk % EXT2FS_CLUSTER_RATIO(fs) == 0) {
> +			ext2fs_unmark_block_bitmap2(wd->ctx->block_found_map,
> +						    blk);
> +			ext2fs_block_alloc_stats2(fs, blk, -1);
> +			wd->cleared++;
> +		}
>  		*block_nr = 0;
> -		wd->cleared++;
>  		return BLOCK_CHANGED;
>  	}
>  
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH 26/31] libext2fs: openfs() musn't allow bigalloc without EXT2_FLAGS_64BITS

[Darrick J. Wong]

Currently, only the 64-bit bitmap implementation supports the block<->cluster
conversions that bigalloc requires.  Therefore, if we have a bigalloc
filesystem, require EXT2_FLAGS_64BITS be passed in to ext2fs_open().

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/openfs.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)


diff --git a/lib/ext2fs/openfs.c b/lib/ext2fs/openfs.c
index 3c0bf14..f486a35 100644
--- a/lib/ext2fs/openfs.c
+++ b/lib/ext2fs/openfs.c
@@ -272,6 +272,18 @@ errcode_t ext2fs_open3(const char *name, const char *io_options,
 		retval = EXT2_ET_CORRUPT_SUPERBLOCK;
 		goto cleanup;
 	}
+
+	/*
+	 * bigalloc requires cluster-aware bitfield operations, which at the
+	 * moment means we need EXT2_FLAG_64BITS.
+	 */
+	if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+				       EXT4_FEATURE_RO_COMPAT_BIGALLOC) &&
+	    !(flags & EXT2_FLAG_64BITS)) {
+		retval = EXT2_ET_CANT_USE_LEGACY_BITMAPS;
+		goto cleanup;
+	}
+
 	if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
 					EXT4_FEATURE_RO_COMPAT_BIGALLOC) &&
 	    (fs->super->s_log_block_size != fs->super->s_log_cluster_size)) {

Re: [PATCH 26/31] libext2fs: openfs() musn't allow bigalloc without EXT2_FLAGS_64BITS

[Lukáš Czerner]

On Mon, 30 Sep 2013, Darrick J. Wong wrote:

> Date: Mon, 30 Sep 2013 18:29:28 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 26/31] libext2fs: openfs() musn't allow bigalloc without
>     EXT2_FLAGS_64BITS
> 
> Currently, only the 64-bit bitmap implementation supports the block<->cluster
> conversions that bigalloc requires.  Therefore, if we have a bigalloc
> filesystem, require EXT2_FLAGS_64BITS be passed in to ext2fs_open().
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  lib/ext2fs/openfs.c |   12 ++++++++++++
>  1 file changed, 12 insertions(+)

Looks good.

Reviewed-by: Lukas Czerner <lczerner@redhat.com>

> 
> 
> diff --git a/lib/ext2fs/openfs.c b/lib/ext2fs/openfs.c
> index 3c0bf14..f486a35 100644
> --- a/lib/ext2fs/openfs.c
> +++ b/lib/ext2fs/openfs.c
> @@ -272,6 +272,18 @@ errcode_t ext2fs_open3(const char *name, const char *io_options,
>  		retval = EXT2_ET_CORRUPT_SUPERBLOCK;
>  		goto cleanup;
>  	}
> +
> +	/*
> +	 * bigalloc requires cluster-aware bitfield operations, which at the
> +	 * moment means we need EXT2_FLAG_64BITS.
> +	 */
> +	if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
> +				       EXT4_FEATURE_RO_COMPAT_BIGALLOC) &&
> +	    !(flags & EXT2_FLAG_64BITS)) {
> +		retval = EXT2_ET_CANT_USE_LEGACY_BITMAPS;
> +		goto cleanup;
> +	}
> +
>  	if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
>  					EXT4_FEATURE_RO_COMPAT_BIGALLOC) &&
>  	    (fs->super->s_log_block_size != fs->super->s_log_cluster_size)) {
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 26/31] libext2fs: openfs() musn't allow bigalloc without EXT2_FLAGS_64BITS

[Theodore Ts'o]

On Mon, Oct 07, 2013 at 02:50:52PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> > Currently, only the 64-bit bitmap implementation supports the block<->cluster
> > conversions that bigalloc requires.  Therefore, if we have a bigalloc
> > filesystem, require EXT2_FLAGS_64BITS be passed in to ext2fs_open().
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> 
> Reviewed-by: Lukas Czerner <lczerner@redhat.com>

Applied, thanks.

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 27/31] resize2fs: Convert fs to and from 64bit mode

[Darrick J. Wong]

resize2fs does its magic by loading a filesystem, duplicating the in-memory
image of that fs, moving relevant blocks out of the way of whatever new
metadata get created, and finally writing everything back out to disk.
Enabling 64bit mode enlarges the group descriptors, which makes resize2fs a
reasonable vehicle for taking care of the rest of the bookkeeping requirements,
so add to resize2fs the ability to convert a filesystem to 64bit mode and back.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 resize/main.c         |   40 ++++++-
 resize/resize2fs.8.in |   18 +++
 resize/resize2fs.c    |  281 ++++++++++++++++++++++++++++++++++++++++++++++++-
 resize/resize2fs.h    |    3 +
 4 files changed, 335 insertions(+), 7 deletions(-)


diff --git a/resize/main.c b/resize/main.c
index b648a15..5179e95 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -41,7 +41,7 @@ char *program_name, *device_name, *io_options;
 static void usage (char *prog)
 {
 	fprintf (stderr, _("Usage: %s [-d debug_flags] [-f] [-F] [-M] [-P] "
-			   "[-p] device [new_size]\n\n"), prog);
+			   "[-p] device [-b|-s|new_size]\n\n"), prog);
 
 	exit (1);
 }
@@ -199,7 +199,7 @@ int main (int argc, char ** argv)
 	if (argc && *argv)
 		program_name = *argv;
 
-	while ((c = getopt (argc, argv, "d:fFhMPpS:")) != EOF) {
+	while ((c = getopt(argc, argv, "d:fFhMPpS:bs")) != EOF) {
 		switch (c) {
 		case 'h':
 			usage(program_name);
@@ -225,6 +225,12 @@ int main (int argc, char ** argv)
 		case 'S':
 			use_stride = atoi(optarg);
 			break;
+		case 'b':
+			flags |= RESIZE_ENABLE_64BIT;
+			break;
+		case 's':
+			flags |= RESIZE_DISABLE_64BIT;
+			break;
 		default:
 			usage(program_name);
 		}
@@ -383,6 +389,10 @@ int main (int argc, char ** argv)
 		if (sys_page_size > fs->blocksize)
 			new_size &= ~((sys_page_size / fs->blocksize)-1);
 	}
+	/* If changing 64bit, don't change the filesystem size. */
+	if (flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)) {
+		new_size = ext2fs_blocks_count(fs->super);
+	}
 	if (!EXT2_HAS_INCOMPAT_FEATURE(fs->super,
 				       EXT4_FEATURE_INCOMPAT_64BIT)) {
 		/* Take 16T down to 2^32-1 blocks */
@@ -434,7 +444,31 @@ int main (int argc, char ** argv)
 			fs->blocksize / 1024, new_size);
 		exit(1);
 	}
-	if (new_size == ext2fs_blocks_count(fs->super)) {
+	if (flags & RESIZE_DISABLE_64BIT && flags & RESIZE_ENABLE_64BIT) {
+		fprintf(stderr, _("Cannot set and unset 64bit feature.\n"));
+		exit(1);
+	} else if (flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)) {
+		new_size = ext2fs_blocks_count(fs->super);
+		if (new_size >= (1ULL << 32)) {
+			fprintf(stderr, _("Cannot change the 64bit feature "
+				"on a filesystem that is larger than "
+				"2^32 blocks.\n"));
+			exit(1);
+		}
+		if (mount_flags & EXT2_MF_MOUNTED) {
+			fprintf(stderr, _("Cannot change the 64bit feature "
+				"while the filesystem is mounted.\n"));
+			exit(1);
+		}
+		if (flags & RESIZE_ENABLE_64BIT &&
+		    !EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+				EXT3_FEATURE_INCOMPAT_EXTENTS)) {
+			fprintf(stderr, _("Please enable the extents feature "
+				"with tune2fs before enabling the 64bit "
+				"feature.\n"));
+			exit(1);
+		}
+	} else if (new_size == ext2fs_blocks_count(fs->super)) {
 		fprintf(stderr, _("The filesystem is already %llu blocks "
 			"long.  Nothing to do!\n\n"), new_size);
 		exit(0);
diff --git a/resize/resize2fs.8.in b/resize/resize2fs.8.in
index 735fc91..d6deee6 100644
--- a/resize/resize2fs.8.in
+++ b/resize/resize2fs.8.in
@@ -8,7 +8,7 @@ resize2fs \- ext2/ext3/ext4 file system resizer
 .SH SYNOPSIS
 .B resize2fs
 [
-.B \-fFpPM
+.B \-fFpPMbs
 ]
 [
 .B \-d
@@ -85,8 +85,21 @@ to shrink the size of filesystem.  Then you may use
 to shrink the size of the partition.  When shrinking the size of
 the partition, make sure you do not make it smaller than the new size
 of the ext2 filesystem!
+.PP
+The
+.B \-b
+and
+.B \-s
+options enable and disable the 64bit feature, respectively.  The resize2fs
+program will, of course, take care of resizing the block group descriptors
+and moving other data blocks out of the way, as needed.  It is not possible
+to resize the filesystem concurrent with changing the 64bit status.
 .SH OPTIONS
 .TP
+.B \-b
+Turns on the 64bit feature, resizes the group descriptors as necessary, and
+moves other metadata out of the way.
+.TP
 .B \-d \fIdebug-flags
 Turns on various resize2fs debugging features, if they have been compiled
 into the binary.
@@ -124,6 +137,9 @@ of what the program is doing.
 .B \-P
 Print the minimum size of the filesystem and exit.
 .TP
+.B \-s
+Turns off the 64bit feature and frees blocks that are no longer in use.
+.TP
 .B \-S \fIRAID-stride
 The
 .B resize2fs
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index e9c207b..7bea0ef 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -53,6 +53,9 @@ static errcode_t ext2fs_calculate_summary_stats(ext2_filsys fs);
 static errcode_t fix_sb_journal_backup(ext2_filsys fs);
 static errcode_t mark_table_blocks(ext2_filsys fs,
 				   ext2fs_block_bitmap bmap);
+static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size);
+static errcode_t move_bg_metadata(ext2_resize_t rfs);
+static errcode_t zero_high_bits_in_inodes(ext2_resize_t rfs);
 
 /*
  * Some helper CPP macros
@@ -119,13 +122,30 @@ errcode_t resize_fs(ext2_filsys fs, blk64_t *new_size, int flags,
 	if (retval)
 		goto errout;
 
+	init_resource_track(&rtrack, "resize_group_descriptors", fs->io);
+	retval = resize_group_descriptors(rfs, *new_size);
+	if (retval)
+		goto errout;
+	print_resource_track(rfs, &rtrack, fs->io);
+
+	init_resource_track(&rtrack, "move_bg_metadata", fs->io);
+	retval = move_bg_metadata(rfs);
+	if (retval)
+		goto errout;
+	print_resource_track(rfs, &rtrack, fs->io);
+
+	init_resource_track(&rtrack, "zero_high_bits_in_metadata", fs->io);
+	retval = zero_high_bits_in_inodes(rfs);
+	if (retval)
+		goto errout;
+	print_resource_track(rfs, &rtrack, fs->io);
+
 	init_resource_track(&rtrack, "adjust_superblock", fs->io);
 	retval = adjust_superblock(rfs, *new_size);
 	if (retval)
 		goto errout;
 	print_resource_track(rfs, &rtrack, fs->io);
 
-
 	init_resource_track(&rtrack, "fix_uninit_block_bitmaps 2", fs->io);
 	fix_uninit_block_bitmaps(rfs->new_fs);
 	print_resource_track(rfs, &rtrack, fs->io);
@@ -221,6 +241,258 @@ errout:
 	return retval;
 }
 
+/* Toggle 64bit mode */
+static errcode_t resize_group_descriptors(ext2_resize_t rfs, blk64_t new_size)
+{
+	void *o, *n, *new_group_desc;
+	dgrp_t i;
+	int copy_size;
+	errcode_t retval;
+
+	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+		return 0;
+
+	if (new_size != ext2fs_blocks_count(rfs->new_fs->super) ||
+	    ext2fs_blocks_count(rfs->new_fs->super) >= (1ULL << 32) ||
+	    (rfs->flags & RESIZE_DISABLE_64BIT &&
+	     rfs->flags & RESIZE_ENABLE_64BIT))
+		return EXT2_ET_INVALID_ARGUMENT;
+
+	if (rfs->flags & RESIZE_DISABLE_64BIT) {
+		rfs->new_fs->super->s_feature_incompat &=
+				~EXT4_FEATURE_INCOMPAT_64BIT;
+		rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE;
+	} else if (rfs->flags & RESIZE_ENABLE_64BIT) {
+		rfs->new_fs->super->s_feature_incompat |=
+				EXT4_FEATURE_INCOMPAT_64BIT;
+		rfs->new_fs->super->s_desc_size = EXT2_MIN_DESC_SIZE_64BIT;
+	}
+
+	if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
+	    EXT2_DESC_SIZE(rfs->new_fs->super))
+		return 0;
+
+	o = rfs->new_fs->group_desc;
+	rfs->new_fs->desc_blocks = ext2fs_div_ceil(
+			rfs->old_fs->group_desc_count,
+			EXT2_DESC_PER_BLOCK(rfs->new_fs->super));
+	retval = ext2fs_get_arrayzero(rfs->new_fs->desc_blocks,
+				      rfs->old_fs->blocksize, &new_group_desc);
+	if (retval)
+		return retval;
+
+	n = new_group_desc;
+
+	if (EXT2_DESC_SIZE(rfs->old_fs->super) <=
+	    EXT2_DESC_SIZE(rfs->new_fs->super))
+		copy_size = EXT2_DESC_SIZE(rfs->old_fs->super);
+	else
+		copy_size = EXT2_DESC_SIZE(rfs->new_fs->super);
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+		memcpy(n, o, copy_size);
+		n += EXT2_DESC_SIZE(rfs->new_fs->super);
+		o += EXT2_DESC_SIZE(rfs->old_fs->super);
+	}
+
+	ext2fs_free_mem(&rfs->new_fs->group_desc);
+	rfs->new_fs->group_desc = new_group_desc;
+
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++)
+		ext2fs_group_desc_csum_set(rfs->new_fs, i);
+
+	return 0;
+}
+
+/* Move bitmaps/inode tables out of the way. */
+static errcode_t move_bg_metadata(ext2_resize_t rfs)
+{
+	dgrp_t i;
+	blk64_t b, c, d;
+	ext2fs_block_bitmap old_map, new_map;
+	int old, new;
+	errcode_t retval;
+	int zero = 0, one = 1;
+
+	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+		return 0;
+
+	retval = ext2fs_allocate_block_bitmap(rfs->old_fs, "oldfs", &old_map);
+	if (retval)
+		return retval;
+
+	retval = ext2fs_allocate_block_bitmap(rfs->new_fs, "newfs", &new_map);
+	if (retval)
+		goto out;
+
+	/* Construct bitmaps of super/descriptor blocks in old and new fs */
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+		retval = ext2fs_super_and_bgd_loc2(rfs->old_fs, i, &b, &c, &d,
+						   NULL);
+		if (retval)
+			goto out;
+		ext2fs_mark_block_bitmap2(old_map, b);
+		ext2fs_mark_block_bitmap2(old_map, c);
+		ext2fs_mark_block_bitmap2(old_map, d);
+
+		retval = ext2fs_super_and_bgd_loc2(rfs->new_fs, i, &b, &c, &d,
+						   NULL);
+		if (retval)
+			goto out;
+		ext2fs_mark_block_bitmap2(new_map, b);
+		ext2fs_mark_block_bitmap2(new_map, c);
+		ext2fs_mark_block_bitmap2(new_map, d);
+	}
+
+	/* Find changes in block allocations for bg metadata */
+	for (b = 0;
+	     b < ext2fs_blocks_count(rfs->new_fs->super);
+	     b += EXT2FS_CLUSTER_RATIO(rfs->new_fs)) {
+		old = ext2fs_test_block_bitmap2(old_map, b);
+		new = ext2fs_test_block_bitmap2(new_map, b);
+
+		if (old && !new)
+			ext2fs_unmark_block_bitmap2(rfs->new_fs->block_map, b);
+		else if (!old && new)
+			; /* empty ext2fs_mark_block_bitmap2(new_map, b); */
+		else
+			ext2fs_unmark_block_bitmap2(new_map, b);
+	}
+	/* new_map now shows blocks that have been newly allocated. */
+
+	/* Move any conflicting bitmaps and inode tables */
+	for (i = 0; i < rfs->old_fs->group_desc_count; i++) {
+		b = ext2fs_block_bitmap_loc(rfs->new_fs, i);
+		if (ext2fs_test_block_bitmap2(new_map, b))
+			ext2fs_block_bitmap_loc_set(rfs->new_fs, i, 0);
+
+		b = ext2fs_inode_bitmap_loc(rfs->new_fs, i);
+		if (ext2fs_test_block_bitmap2(new_map, b))
+			ext2fs_inode_bitmap_loc_set(rfs->new_fs, i, 0);
+
+		c = ext2fs_inode_table_loc(rfs->new_fs, i);
+		for (b = 0; b < rfs->new_fs->inode_blocks_per_group; b++) {
+			if (ext2fs_test_block_bitmap2(new_map, b + c)) {
+				ext2fs_inode_table_loc_set(rfs->new_fs, i, 0);
+				break;
+			}
+		}
+	}
+
+out:
+	if (old_map)
+		ext2fs_free_block_bitmap(old_map);
+	if (new_map)
+		ext2fs_free_block_bitmap(new_map);
+	return retval;
+}
+
+/* Zero out the high bits of extent fields */
+static errcode_t zero_high_bits_in_extents(ext2_filsys fs, ext2_ino_t ino,
+				 struct ext2_inode *inode)
+{
+	ext2_extent_handle_t	handle;
+	struct ext2fs_extent	extent;
+	int			op = EXT2_EXTENT_ROOT;
+	errcode_t		errcode;
+
+	if (!(inode->i_flags & EXT4_EXTENTS_FL))
+		return 0;
+
+	errcode = ext2fs_extent_open(fs, ino, &handle);
+	if (errcode)
+		return errcode;
+
+	while (1) {
+		errcode = ext2fs_extent_get(handle, op, &extent);
+		if (errcode)
+			break;
+
+		op = EXT2_EXTENT_NEXT_SIB;
+
+		if (extent.e_pblk > (1ULL << 32)) {
+			extent.e_pblk &= (1ULL << 32) - 1;
+			errcode = ext2fs_extent_replace(handle, 0, &extent);
+			if (errcode)
+				break;
+		}
+	}
+
+	/* Ok if we run off the end */
+	if (errcode == EXT2_ET_EXTENT_NO_NEXT)
+		errcode = 0;
+	return errcode;
+}
+
+/* Zero out the high bits of inodes. */
+static errcode_t zero_high_bits_in_inodes(ext2_resize_t rfs)
+{
+	ext2_filsys	fs = rfs->new_fs;
+	int length = EXT2_INODE_SIZE(fs->super);
+	struct ext2_inode *inode = NULL;
+	ext2_inode_scan	scan = NULL;
+	errcode_t	retval;
+	ext2_ino_t	ino;
+	blk64_t		file_acl_block;
+	int		inode_dirty;
+
+	if (!(rfs->flags & (RESIZE_DISABLE_64BIT | RESIZE_ENABLE_64BIT)))
+		return 0;
+
+	if (fs->super->s_creator_os != EXT2_OS_LINUX)
+		return 0;
+
+	retval = ext2fs_open_inode_scan(fs, 0, &scan);
+	if (retval)
+		return retval;
+
+	retval = ext2fs_get_mem(length, &inode);
+	if (retval)
+		goto out;
+
+	do {
+		retval = ext2fs_get_next_inode_full(scan, &ino, inode, length);
+		if (retval)
+			goto out;
+		if (!ino)
+			break;
+		if (!ext2fs_test_inode_bitmap2(fs->inode_map, ino))
+			continue;
+
+		/*
+		 * Here's how we deal with high block number fields:
+		 *
+		 *  - i_size_high has been been written out with i_size_lo
+		 *    since the ext2 days, so no conversion is needed.
+		 *
+		 *  - i_blocks_hi is guarded by both the huge_file feature and
+		 *    inode flags and has always been written out with
+		 *    i_blocks_lo if the feature is set.  The field is only
+		 *    ever read if both feature and inode flag are set, so
+		 *    we don't need to zero it now.
+		 *
+		 *  - i_file_acl_high can be uninitialized, so zero it if
+		 *    it isn't already.
+		 */
+		if (inode->osd2.linux2.l_i_file_acl_high) {
+			inode->osd2.linux2.l_i_file_acl_high = 0;
+			retval = ext2fs_write_inode_full(fs, ino, inode,
+							 length);
+			if (retval)
+				goto out;
+		}
+
+		retval = zero_high_bits_in_extents(fs, ino, inode);
+		if (retval)
+			goto out;
+	} while (ino);
+
+out:
+	if (inode)
+		ext2fs_free_mem(&inode);
+	if (scan)
+		ext2fs_close_inode_scan(scan);
+}
+
 /*
  * Clean up the bitmaps for unitialized bitmaps
  */
@@ -424,7 +696,8 @@ retry:
 	/*
 	 * Reallocate the group descriptors as necessary.
 	 */
-	if (old_fs->desc_blocks != fs->desc_blocks) {
+	if (EXT2_DESC_SIZE(old_fs->super) == EXT2_DESC_SIZE(fs->super) &&
+	    old_fs->desc_blocks != fs->desc_blocks) {
 		retval = ext2fs_resize_mem(old_fs->desc_blocks *
 					   fs->blocksize,
 					   fs->desc_blocks * fs->blocksize,
@@ -939,7 +1212,9 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
 		new_blocks = fs->desc_blocks + fs->super->s_reserved_gdt_blocks;
 	}
 
-	if (old_blocks == new_blocks) {
+	if (EXT2_DESC_SIZE(rfs->old_fs->super) ==
+	    EXT2_DESC_SIZE(rfs->new_fs->super) &&
+	    old_blocks == new_blocks) {
 		retval = 0;
 		goto errout;
 	}
diff --git a/resize/resize2fs.h b/resize/resize2fs.h
index d425491..e24319d 100644
--- a/resize/resize2fs.h
+++ b/resize/resize2fs.h
@@ -81,6 +81,9 @@ typedef struct ext2_sim_progress *ext2_sim_progmeter;
 #define RESIZE_PERCENT_COMPLETE		0x0100
 #define RESIZE_VERBOSE			0x0200
 
+#define RESIZE_ENABLE_64BIT		0x0400
+#define RESIZE_DISABLE_64BIT		0x0800
+
 /*
  * This structure is used for keeping track of how much resources have
  * been used for a particular resize2fs pass.

[PATCH 28/31] mke2fs: Complain about creating 64bit filesystems without extents

[Darrick J. Wong]

A 64bit filesystem without extents is not terribly useful, because the old
block map format does not support pointing to high block numbers.  Warn the
user who tries to create such an animal.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 misc/mke2fs.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)


diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index d8bd5ed..1e44a84 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -2460,6 +2460,18 @@ int main (int argc, char *argv[])
 				 "checksumming.  Pass -O 64bit to rectify.\n"));
 	}
 
+	/* Check the user's mkfs options for 64bit */
+	if (!quiet &&
+	    EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+			EXT4_FEATURE_INCOMPAT_64BIT) &&
+	    !EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+			EXT3_FEATURE_INCOMPAT_EXTENTS)) {
+		printf(_("Extents MUST be enabled for a 64-bit filesystem to "
+			 "be able to access the entire disk space.  Pass "
+			 "-O extents to rectify.\n"));
+		exit(1);
+	}
+
 	/* Calculate journal blocks */
 	if (!journal_device && ((journal_size) ||
 		(fs_param.s_feature_compat &

Re: [PATCH 28/31] mke2fs: Complain about creating 64bit filesystems without extents

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:29:46PM -0700, Darrick J. Wong wrote:
> A 64bit filesystem without extents is not terribly useful, because the old
> block map format does not support pointing to high block numbers.  Warn the
> user who tries to create such an animal.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

I moved the check to the PRS() function, which is where the rest of
the feature flag sanity checks are done.  I also removed the !quiet
check, since we aren't doing this for any of the other sanity checks.
I also shortened the error message so that it fits on an 80 column
screen.

						- Ted

commit 51a72e0161c826670e6ee66cb51c2620402098f8
Author: Darrick J. Wong <darrick.wong@oracle.com>
Date:   Mon Sep 30 18:29:46 2013 -0700

    mke2fs: complain about creating 64bit filesystems without extents
    
    A 64bit filesystem without extents is not terribly useful, because the
    old block map format does not support pointing to high block numbers.
    Warn the user who tries to create such an animal.
    
    Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
    Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index 22c2815..cc06a97 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -1806,6 +1806,14 @@ profile_error:
 		exit(1);
 	}
 
+	/* Check the user's mkfs options for 64bit */
+	if ((fs_param.s_feature_incompat & EXT4_FEATURE_INCOMPAT_64BIT) &&
+	    !(fs_param.s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS)) {
+		printf(_("Extents MUST be enabled for a 64-bit filesystem.  "
+			 "Pass -O extents to rectify.\n"));
+		exit(1);
+	}
+
 	/* Set first meta blockgroup via an environment variable */
 	/* (this is mostly for debugging purposes) */
 	if ((fs_param.s_feature_incompat & EXT2_FEATURE_INCOMPAT_META_BG) &&

[PATCH 29/31] e2fsck: Enable extents on all 64bit filesystems

[Darrick J. Wong]

Since it's impossible to address all blocks of a 64bit filesystem without
extents, have e2fsck turn on the feature if it finds (64bit && !extents).

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 e2fsck/problem.c |    5 +++++
 e2fsck/problem.h |    2 ++
 e2fsck/super.c   |   11 +++++++++++
 3 files changed, 18 insertions(+)


diff --git a/e2fsck/problem.c b/e2fsck/problem.c
index b0a6e19..897693a 100644
--- a/e2fsck/problem.c
+++ b/e2fsck/problem.c
@@ -447,6 +447,11 @@ static struct e2fsck_problem problem_table[] = {
 	  N_("@S MMP block checksum does not match MMP block.  "),
 	  PROMPT_FIX, PR_PREEN_OK | PR_NO_OK},
 
+	/* 64bit is set but extents is unset. */
+	{ PR_0_64BIT_WITHOUT_EXTENTS,
+	  N_("@S 64bit filesystems needs extents to access the whole disk.  "),
+	  PROMPT_FIX, PR_PREEN_OK | PR_NO_OK},
+
 	/* Pass 1 errors */
 
 	/* Pass 1: Checking inodes, blocks, and sizes */
diff --git a/e2fsck/problem.h b/e2fsck/problem.h
index fcdc1a1..dd17e37 100644
--- a/e2fsck/problem.h
+++ b/e2fsck/problem.h
@@ -258,6 +258,8 @@ struct problem_context {
 /* Superblock has invalid MMP checksum. */
 #define PR_0_MMP_CSUM_INVALID			0x000047
 
+/* 64bit is set but extents are not set. */
+#define PR_0_64BIT_WITHOUT_EXTENTS		0x000048
 
 /*
  * Pass 1 errors
diff --git a/e2fsck/super.c b/e2fsck/super.c
index 16a1c23..0c879f5 100644
--- a/e2fsck/super.c
+++ b/e2fsck/super.c
@@ -595,6 +595,17 @@ void check_super_block(e2fsck_t ctx)
 			ext2fs_group_desc_csum_set(fs, i);
 	}
 
+	/* Is 64bit set and extents unset? */
+	if (EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+			EXT4_FEATURE_INCOMPAT_64BIT) &&
+	    !EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+			EXT3_FEATURE_INCOMPAT_EXTENTS) &&
+	    fix_problem(ctx, PR_0_64BIT_WITHOUT_EXTENTS, &pctx)) {
+		fs->super->s_feature_incompat |=
+			EXT3_FEATURE_INCOMPAT_EXTENTS;
+		ext2fs_mark_super_dirty(fs);
+	}
+
 	/*
 	 * Verify the group descriptors....
 	 */

Re: [PATCH 29/31] e2fsck: Enable extents on all 64bit filesystems

[Theodore Ts'o]

On Mon, Sep 30, 2013 at 06:29:52PM -0700, Darrick J. Wong wrote:
> Since it's impossible to address all blocks of a 64bit filesystem without
> extents, have e2fsck turn on the feature if it finds (64bit && !extents).
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>

Thanks, applied.

						- Ted

[PATCH 30/31] libext2fs: Support modifying arbitrary extended attributes

[Darrick J. Wong]

Add functions to allow clients to get, set, and remove extended attributes from
any file.  It also supports modifying EAs living in i_file_acl.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 lib/ext2fs/ext2_err.et.in |   18 +
 lib/ext2fs/ext2fs.h       |   23 ++
 lib/ext2fs/ext_attr.c     |  671 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 712 insertions(+)


diff --git a/lib/ext2fs/ext2_err.et.in b/lib/ext2fs/ext2_err.et.in
index c547a2c..262f4a4 100644
--- a/lib/ext2fs/ext2_err.et.in
+++ b/lib/ext2fs/ext2_err.et.in
@@ -479,4 +479,22 @@ ec	EXT2_ET_FILE_EXISTS,
 ec	EXT2_ET_BLOCK_BITMAP_CSUM_INVALID,
 	"Block bitmap checksum does not match bitmap"
 
+ec	EXT2_ET_EA_BAD_NAME_LEN,
+	"Extended attribute has an invalid name length"
+
+ec	EXT2_ET_EA_BAD_VALUE_SIZE,
+	"Extended attribute has an invalid value length"
+
+ec	EXT2_ET_BAD_EA_HASH,
+	"Extended attribute has an incorrect hash"
+
+ec	EXT2_ET_BAD_EA_HEADER,
+	"Extended attribute block has a bad header"
+
+ec	EXT2_ET_EA_KEY_NOT_FOUND,
+	"Extended attribute key not found"
+
+ec	EXT2_ET_EA_NO_SPACE,
+	"Insufficient space to store extended attribute data"
+
 	end
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 88da8db..80bfb7e 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -1180,6 +1180,29 @@ extern errcode_t ext2fs_extent_goto2(ext2_extent_handle_t handle,
 				     int leaf_level, blk64_t blk);
 extern errcode_t ext2fs_extent_fix_parents(ext2_extent_handle_t handle);
 
+struct ext2_xattr_handle;
+errcode_t ext2fs_xattrs_expand(struct ext2_xattr_handle *h,
+			       unsigned int expandby);
+errcode_t ext2fs_xattrs_write(struct ext2_xattr_handle *handle);
+errcode_t ext2fs_xattrs_read(struct ext2_xattr_handle *handle);
+#define XATTR_ABORT	1
+#define XATTR_CHANGED	2
+errcode_t ext2fs_xattrs_iterate(struct ext2_xattr_handle *h,
+				int (*func)(char *name, char *value,
+					    void *data),
+				void *data);
+errcode_t ext2fs_xattr_get(struct ext2_xattr_handle *h, const char *key,
+			   void **value, unsigned int *value_len);
+errcode_t ext2fs_xattr_set(struct ext2_xattr_handle *handle,
+			   const char *key,
+			   const void *value,
+			   unsigned int value_len);
+errcode_t ext2fs_xattr_remove(struct ext2_xattr_handle *handle,
+			      const char *key);
+errcode_t ext2fs_xattrs_open(ext2_filsys fs, ext2_ino_t ino,
+			     struct ext2_xattr_handle **handle);
+errcode_t ext2fs_xattrs_close(struct ext2_xattr_handle **handle);
+
 /* fileio.c */
 extern errcode_t ext2fs_file_open2(ext2_filsys fs, ext2_ino_t ino,
 				   struct ext2_inode *inode,
diff --git a/lib/ext2fs/ext_attr.c b/lib/ext2fs/ext_attr.c
index 9649a14..3456b97 100644
--- a/lib/ext2fs/ext_attr.c
+++ b/lib/ext2fs/ext_attr.c
@@ -186,3 +186,674 @@ errcode_t ext2fs_adjust_ea_refcount(ext2_filsys fs, blk_t blk,
 	return ext2fs_adjust_ea_refcount2(fs, blk, block_buf, adjust,
 					  newcount);
 }
+
+/* Manipulate the contents of extended attribute regions */
+struct ext2_xattr {
+	char *name;
+	void *value;
+	unsigned int value_len;
+};
+
+struct ext2_xattr_handle {
+	ext2_filsys fs;
+	struct ext2_xattr *attrs;
+	unsigned int length;
+	ext2_ino_t ino;
+	int dirty;
+};
+
+errcode_t ext2fs_xattrs_expand(struct ext2_xattr_handle *h,
+			       unsigned int expandby)
+{
+	struct ext2_xattr *new_attrs;
+	errcode_t err;
+
+	err = ext2fs_get_arrayzero(h->length + expandby,
+				   sizeof(struct ext2_xattr), &new_attrs);
+	if (err)
+		return err;
+
+	memcpy(new_attrs, h->attrs, h->length * sizeof(struct ext2_xattr));
+	ext2fs_free_mem(&h->attrs);
+	h->length += expandby;
+	h->attrs = new_attrs;
+
+	return 0;
+}
+
+struct ea_name_index {
+	int index;
+	const char *name;
+};
+
+static struct ea_name_index ea_names[] = {
+	{1, "user."},
+	{2, "system.posix_acl_access"},
+	{3, "system.posix_acl_default"},
+	{4, "trusted."},
+	{6, "security."},
+	{7, "system."},
+	{0, NULL},
+};
+
+static const char *find_ea_prefix(int index)
+{
+	struct ea_name_index *e;
+
+	for (e = ea_names; e->name; e++)
+		if (e->index == index)
+			return e->name;
+
+	return NULL;
+}
+
+static int find_ea_index(const char *fullname, char **name, int *index)
+{
+	struct ea_name_index *e;
+
+	for (e = ea_names; e->name; e++)
+		if (memcmp(fullname, e->name, strlen(e->name)) == 0) {
+			*name = (char *)fullname + strlen(e->name);
+			*index = e->index;
+			return 1;
+		}
+	return 0;
+}
+
+static errcode_t prep_ea_block_for_write(ext2_filsys fs, ext2_ino_t ino,
+					 struct ext2_inode_large *inode)
+{
+	struct ext2_ext_attr_header *header;
+	void *block_buf = NULL;
+	dgrp_t grp;
+	blk64_t blk, goal;
+	errcode_t err;
+
+	/* Do we already have an EA block? */
+	blk = ext2fs_file_acl_block(fs, (struct ext2_inode *)inode);
+	if (blk != 0) {
+		if ((blk < fs->super->s_first_data_block) ||
+		    (blk >= ext2fs_blocks_count(fs->super))) {
+			err = EXT2_ET_BAD_EA_BLOCK_NUM;
+			goto out;
+		}
+
+		err = ext2fs_get_mem(fs->blocksize, &block_buf);
+		if (err)
+			goto out;
+
+		err = ext2fs_read_ext_attr3(fs, blk, block_buf, ino);
+		if (err)
+			goto out2;
+
+		header = (struct ext2_ext_attr_header *) block_buf;
+		if (header->h_magic != EXT2_EXT_ATTR_MAGIC) {
+			err = EXT2_ET_BAD_EA_HEADER;
+			goto out2;
+		}
+
+		/* Single-user block.  We're done here. */
+		if (header->h_refcount == 1)
+			return 0;
+
+		/* We need to CoW the block. */
+		header->h_refcount--;
+		err = ext2fs_write_ext_attr3(fs, blk, block_buf, ino);
+		if (err)
+			goto out2;
+	} else {
+		/* No block, we must increment i_blocks */
+		err = ext2fs_iblk_add_blocks(fs, (struct ext2_inode *)inode,
+					     1);
+		if (err)
+			goto out;
+	}
+
+	/* Allocate a block */
+	grp = ext2fs_group_of_ino(fs, ino);
+	goal = ext2fs_inode_table_loc(fs, grp);
+	err = ext2fs_alloc_block2(fs, goal, NULL, &blk);
+	if (err)
+		return err;
+	ext2fs_file_acl_block_set(fs, (struct ext2_inode *)inode, blk);
+out2:
+	ext2fs_free_mem(&block_buf);
+out:
+	return err;
+}
+
+
+static errcode_t write_xattrs_to_buffer(struct ext2_xattr_handle *handle,
+					struct ext2_xattr **pos,
+					void *entries_start,
+					unsigned int storage_size,
+					unsigned int value_offset_correction)
+{
+	struct ext2_xattr *x = *pos;
+	struct ext2_ext_attr_entry *e = entries_start;
+	void *end = entries_start + storage_size;
+	char *shortname;
+	unsigned int entry_size, value_size;
+	int idx, ret;
+
+	/* For all remaining x...  */
+	for (; x < handle->attrs + handle->length; x++) {
+		if (!x->name)
+			continue;
+
+		/* Calculate index and shortname position */
+		shortname = x->name;
+		ret = find_ea_index(x->name, &shortname, &idx);
+
+		/* Calculate entry and value size */
+		entry_size = (sizeof(*e) + strlen(shortname) +
+			      EXT2_EXT_ATTR_PAD - 1) &
+			     ~(EXT2_EXT_ATTR_PAD - 1);
+		value_size = ((x->value_len + EXT2_EXT_ATTR_PAD - 1) /
+			      EXT2_EXT_ATTR_PAD) * EXT2_EXT_ATTR_PAD;
+
+		/*
+		 * Would entry collide with value?
+		 * Note that we must leave sufficient room for a (u32)0 to
+		 * mark the end of the entries.
+		 */
+		if ((void *)e + entry_size + sizeof(__u32) > end - value_size)
+			break;
+
+		/* Fill out e appropriately */
+		e->e_name_len = strlen(shortname);
+		e->e_name_index = (ret ? idx : 0);
+		e->e_value_offs = end - value_size - (void *)entries_start +
+				value_offset_correction;
+		e->e_value_block = 0;
+		e->e_value_size = x->value_len;
+
+		/* Store name and value */
+		end -= value_size;
+		memcpy((void *)e + sizeof(*e), shortname, e->e_name_len);
+		memcpy(end, x->value, e->e_value_size);
+
+		e->e_hash = ext2fs_ext_attr_hash_entry(e, end);
+
+		e = EXT2_EXT_ATTR_NEXT(e);
+		*(__u32 *)e = 0;
+	}
+	*pos = x;
+
+	return 0;
+}
+
+errcode_t ext2fs_xattrs_write(struct ext2_xattr_handle *handle)
+{
+	struct ext2_xattr *x;
+	struct ext2_inode_large *inode;
+	void *start, *block_buf = NULL;
+	struct ext2_ext_attr_header *header;
+	__u32 ea_inode_magic;
+	blk64_t blk;
+	unsigned int storage_size;
+	unsigned int i, written;
+	errcode_t err;
+
+	if (!EXT2_HAS_COMPAT_FEATURE(handle->fs->super,
+				     EXT2_FEATURE_COMPAT_EXT_ATTR))
+		return 0;
+
+	err = ext2fs_get_memzero(EXT2_INODE_SIZE(handle->fs->super), &inode);
+	if (err)
+		return err;
+
+	err = ext2fs_read_inode_full(handle->fs, handle->ino,
+				     (struct ext2_inode *)inode,
+				     EXT2_INODE_SIZE(handle->fs->super));
+	if (err)
+		goto out;
+
+	/* Write the inode EA */
+	ea_inode_magic = EXT2_EXT_ATTR_MAGIC;
+	memcpy(((char *) inode) + EXT2_GOOD_OLD_INODE_SIZE +
+	       inode->i_extra_isize, &ea_inode_magic, sizeof(__u32));
+	storage_size = EXT2_INODE_SIZE(handle->fs->super) -
+		EXT2_GOOD_OLD_INODE_SIZE - inode->i_extra_isize -
+		sizeof(__u32);
+	start = ((char *) inode) + EXT2_GOOD_OLD_INODE_SIZE +
+		inode->i_extra_isize + sizeof(__u32);
+	x = handle->attrs;
+
+	err = write_xattrs_to_buffer(handle, &x, start, storage_size, 0);
+	if (err)
+		goto out;
+
+	/* Are we done? */
+	if (x == handle->attrs + handle->length)
+		goto skip_ea_block;
+
+	/* Write the EA block */
+	err = ext2fs_get_mem(handle->fs->blocksize, &block_buf);
+	if (err)
+		goto out;
+
+	storage_size = handle->fs->blocksize -
+		sizeof(struct ext2_ext_attr_header);
+	start = block_buf + sizeof(struct ext2_ext_attr_header);
+
+	err = write_xattrs_to_buffer(handle, &x, start, storage_size,
+				     (void *)start - block_buf);
+	if (err)
+		goto out2;
+
+	if (x < handle->attrs + handle->length) {
+		err = EXT2_ET_EA_NO_SPACE;
+		goto out2;
+	}
+
+skip_ea_block:
+	if (block_buf) {
+		/* Write a header on the EA block */
+		header = block_buf;
+		header->h_magic = EXT2_EXT_ATTR_MAGIC;
+		header->h_refcount = 1;
+		header->h_blocks = 1;
+
+		/* Get a new block for writing */
+		err = prep_ea_block_for_write(handle->fs, handle->ino, inode);
+		if (err)
+			goto out2;
+
+		/* Finally, write the new EA block */
+		blk = ext2fs_file_acl_block(handle->fs,
+					    (struct ext2_inode *)inode);
+		err = ext2fs_write_ext_attr3(handle->fs, blk, block_buf,
+					     handle->ino);
+		if (err)
+			goto out2;
+	}
+
+	blk = ext2fs_file_acl_block(handle->fs, (struct ext2_inode *)inode);
+	if (!block_buf && blk) {
+		/* xattrs shrunk, free the block */
+		ext2fs_file_acl_block_set(handle->fs,
+					  (struct ext2_inode *)inode, 0);
+		err = ext2fs_iblk_sub_blocks(handle->fs,
+					     (struct ext2_inode *)inode, 1);
+		if (err)
+			goto out;
+		ext2fs_block_alloc_stats2(handle->fs, blk, -1);
+	}
+
+	/* Write the inode */
+	err = ext2fs_write_inode_full(handle->fs, handle->ino,
+				      (struct ext2_inode *)inode,
+				      EXT2_INODE_SIZE(handle->fs->super));
+	if (err)
+		goto out2;
+
+out2:
+	ext2fs_free_mem(&block_buf);
+out:
+	ext2fs_free_mem(&inode);
+	handle->dirty = 0;
+	return err;
+}
+
+static errcode_t read_xattrs_from_buffer(struct ext2_xattr_handle *handle,
+					 struct ext2_ext_attr_entry *entries,
+					 unsigned int storage_size,
+					 void *value_start)
+{
+	struct ext2_xattr *x;
+	struct ext2_ext_attr_entry *entry;
+	const char *prefix;
+	void *ptr;
+	unsigned int remain, prefix_len;
+	errcode_t err;
+
+	x = handle->attrs;
+	while (x->name)
+		x++;
+
+	entry = entries;
+	while (!EXT2_EXT_IS_LAST_ENTRY(entry)) {
+		__u32 hash;
+
+		/* header eats this space */
+		remain -= sizeof(struct ext2_ext_attr_entry);
+
+		/* is attribute name valid? */
+		if (EXT2_EXT_ATTR_SIZE(entry->e_name_len) > remain)
+			return EXT2_ET_EA_BAD_NAME_LEN;
+
+		/* attribute len eats this space */
+		remain -= EXT2_EXT_ATTR_SIZE(entry->e_name_len);
+
+		/* check value size */
+		if (entry->e_value_size > remain)
+			return EXT2_ET_EA_BAD_VALUE_SIZE;
+
+		/* e_value_block must be 0 in inode's ea */
+		if (entry->e_value_block != 0)
+			return EXT2_ET_BAD_EA_BLOCK_NUM;
+
+		hash = ext2fs_ext_attr_hash_entry(entry, value_start +
+							 entry->e_value_offs);
+
+		/* e_hash may be 0 in older inode's ea */
+		if (entry->e_hash != 0 && entry->e_hash != hash)
+			return EXT2_ET_BAD_EA_HASH;
+
+		remain -= entry->e_value_size;
+
+		/* Allocate space for more attrs? */
+		if (x == handle->attrs + handle->length) {
+			err = ext2fs_xattrs_expand(handle, 4);
+			if (err)
+				return err;
+			x = handle->attrs + handle->length - 4;
+		}
+
+		/* Extract name/value */
+		prefix = find_ea_prefix(entry->e_name_index);
+		prefix_len = (prefix ? strlen(prefix) : 0);
+		err = ext2fs_get_memzero(entry->e_name_len + prefix_len + 1,
+					 &x->name);
+		if (err)
+			return err;
+		if (prefix)
+			memcpy(x->name, prefix, prefix_len);
+		if (entry->e_name_len)
+			memcpy(x->name + prefix_len,
+			       (void *)entry + sizeof(*entry),
+			       entry->e_name_len);
+
+		err = ext2fs_get_mem(entry->e_value_size, &x->value);
+		if (err)
+			return err;
+		x->value_len = entry->e_value_size;
+		memcpy(x->value, value_start + entry->e_value_offs,
+		       entry->e_value_size);
+		x++;
+		entry = EXT2_EXT_ATTR_NEXT(entry);
+	}
+
+	return 0;
+}
+
+errcode_t ext2fs_xattrs_read(struct ext2_xattr_handle *handle)
+{
+	struct ext2_xattr *attrs = NULL, *x;
+	unsigned int attrs_len;
+	struct ext2_inode_large *inode;
+	struct ext2_ext_attr_header *header;
+	__u32 ea_inode_magic;
+	unsigned int storage_size;
+	void *start, *block_buf = NULL;
+	blk64_t blk;
+	errcode_t err;
+
+	if (!EXT2_HAS_COMPAT_FEATURE(handle->fs->super,
+				     EXT2_FEATURE_COMPAT_EXT_ATTR))
+		return 0;
+
+	err = ext2fs_get_memzero(EXT2_INODE_SIZE(handle->fs->super), &inode);
+	if (err)
+		return err;
+
+	err = ext2fs_read_inode_full(handle->fs, handle->ino,
+				     (struct ext2_inode *)inode,
+				     EXT2_INODE_SIZE(handle->fs->super));
+	if (err)
+		goto out;
+
+	/* Look for EA in the inode */
+	memcpy(&ea_inode_magic, ((char *) inode) + EXT2_GOOD_OLD_INODE_SIZE +
+	       inode->i_extra_isize, sizeof(__u32));
+	if (ea_inode_magic == EXT2_EXT_ATTR_MAGIC) {
+		storage_size = EXT2_INODE_SIZE(handle->fs->super) -
+			EXT2_GOOD_OLD_INODE_SIZE - inode->i_extra_isize -
+			sizeof(__u32);
+		start = ((char *) inode) + EXT2_GOOD_OLD_INODE_SIZE +
+			inode->i_extra_isize + sizeof(__u32);
+
+		err = read_xattrs_from_buffer(handle, start, storage_size,
+					      start);
+		if (err)
+			goto out;
+	}
+
+	/* Look for EA in a separate EA block */
+	blk = ext2fs_file_acl_block(handle->fs, (struct ext2_inode *)inode);
+	if (blk != 0) {
+		if ((blk < handle->fs->super->s_first_data_block) ||
+		    (blk >= ext2fs_blocks_count(handle->fs->super))) {
+			err = EXT2_ET_BAD_EA_BLOCK_NUM;
+			goto out;
+		}
+
+		err = ext2fs_get_mem(handle->fs->blocksize, &block_buf);
+		if (err)
+			goto out;
+
+		err = ext2fs_read_ext_attr3(handle->fs, blk, block_buf,
+					    handle->ino);
+		if (err)
+			goto out3;
+
+		header = (struct ext2_ext_attr_header *) block_buf;
+		if (header->h_magic != EXT2_EXT_ATTR_MAGIC) {
+			err = EXT2_ET_BAD_EA_HEADER;
+			goto out3;
+		}
+
+		if (header->h_blocks != 1) {
+			err = EXT2_ET_BAD_EA_HEADER;
+			goto out3;
+		}
+
+		/* Read EAs */
+		storage_size = handle->fs->blocksize -
+			sizeof(struct ext2_ext_attr_header);
+		start = block_buf + sizeof(struct ext2_ext_attr_header);
+		err = read_xattrs_from_buffer(handle, start, storage_size,
+					      block_buf);
+		if (err)
+			goto out3;
+
+		ext2fs_free_mem(&block_buf);
+	}
+
+	ext2fs_free_mem(&block_buf);
+	ext2fs_free_mem(&inode);
+	return 0;
+
+out3:
+	ext2fs_free_mem(&block_buf);
+out:
+	ext2fs_free_mem(&inode);
+	return err;
+}
+
+#define XATTR_ABORT	1
+#define XATTR_CHANGED	2
+errcode_t ext2fs_xattrs_iterate(struct ext2_xattr_handle *h,
+				int (*func)(char *name, char *value,
+					    void *data),
+				void *data)
+{
+	struct ext2_xattr *x;
+	errcode_t err;
+	int ret;
+
+	for (x = h->attrs; x < h->attrs + h->length; x++) {
+		if (!x->name)
+			continue;
+
+		ret = func(x->name, x->value, data);
+		if (ret & XATTR_CHANGED)
+			h->dirty = 1;
+		if (ret & XATTR_ABORT)
+			return 0;
+	}
+
+	return 0;
+}
+
+errcode_t ext2fs_xattr_get(struct ext2_xattr_handle *h, const char *key,
+			   void **value, unsigned int *value_len)
+{
+	struct ext2_xattr *x;
+	void *val;
+	errcode_t err;
+
+	for (x = h->attrs; x < h->attrs + h->length; x++) {
+		if (!x->name)
+			continue;
+
+		if (strcmp(x->name, key) == 0) {
+			err = ext2fs_get_mem(x->value_len, &val);
+			if (err)
+				return err;
+			memcpy(val, x->value, x->value_len);
+			*value = val;
+			*value_len = x->value_len;
+			return 0;
+		}
+	}
+
+	return EXT2_ET_EA_KEY_NOT_FOUND;
+}
+
+errcode_t ext2fs_xattr_set(struct ext2_xattr_handle *handle,
+			   const char *key,
+			   const void *value,
+			   unsigned int value_len)
+{
+	struct ext2_xattr *x, *last_empty;
+	char *new_value;
+	errcode_t err;
+
+	last_empty = NULL;
+	for (x = handle->attrs; x < handle->attrs + handle->length; x++) {
+		if (!x->name) {
+			last_empty = x;
+			continue;
+		}
+
+		/* Replace xattr */
+		if (strcmp(x->name, key) == 0) {
+			err = ext2fs_get_mem(value_len, &new_value);
+			if (err)
+				return err;
+			memcpy(new_value, value, value_len);
+			ext2fs_free_mem(&x->value);
+			x->value = new_value;
+			x->value_len = value_len;
+			handle->dirty = 1;
+			return 0;
+		}
+	}
+
+	/* Add attr to empty slot */
+	if (last_empty) {
+		err = ext2fs_get_mem(strlen(key) + 1, &last_empty->name);
+		if (err)
+			return err;
+		strcpy(last_empty->name, key);
+
+		err = ext2fs_get_mem(value_len, &last_empty->value);
+		if (err)
+			return err;
+		memcpy(last_empty->value, value, value_len);
+		last_empty->value_len = value_len;
+		handle->dirty = 1;
+		return 0;
+	}
+
+	/* Expand array, append slot */
+	err = ext2fs_xattrs_expand(handle, 4);
+	if (err)
+		return err;
+
+	x = handle->attrs + handle->length - 4;
+	err = ext2fs_get_mem(strlen(key) + 1, &x->name);
+	if (err)
+		return err;
+	strcpy(x->name, key);
+
+	err = ext2fs_get_mem(value_len, &x->value);
+	if (err)
+		return err;
+	memcpy(x->value, value, value_len);
+	x->value_len = value_len;
+	handle->dirty = 1;
+	return 0;
+}
+
+errcode_t ext2fs_xattr_remove(struct ext2_xattr_handle *handle,
+			      const char *key)
+{
+	struct ext2_xattr *x;
+	errcode_t err;
+
+	for (x = handle->attrs; x < handle->attrs + handle->length; x++) {
+		if (!x->name)
+			continue;
+
+		if (strcmp(x->name, key) == 0) {
+			ext2fs_free_mem(&x->name);
+			ext2fs_free_mem(&x->value);
+			x->value_len = 0;
+			handle->dirty = 1;
+			return 0;
+		}
+	}
+
+	return EXT2_ET_EA_KEY_NOT_FOUND;
+}
+
+errcode_t ext2fs_xattrs_open(ext2_filsys fs, ext2_ino_t ino,
+			     struct ext2_xattr_handle **handle)
+{
+	struct ext2_xattr_handle *h;
+	errcode_t err;
+
+	err = ext2fs_get_memzero(sizeof(*h), &h);
+	if (err)
+		return err;
+
+	h->length = 4;
+	err = ext2fs_get_arrayzero(h->length, sizeof(struct ext2_xattr),
+				   &h->attrs);
+	if (err) {
+		ext2fs_free_mem(&h);
+		return err;
+	}
+	h->ino = ino;
+	h->fs = fs;
+	*handle = h;
+	return 0;
+}
+
+errcode_t ext2fs_xattrs_close(struct ext2_xattr_handle **handle)
+{
+	unsigned int i;
+	struct ext2_xattr_handle *h = *handle;
+	struct ext2_xattr *a = h->attrs;
+	errcode_t err;
+
+	if (h->dirty) {
+		err = ext2fs_xattrs_write(h);
+		if (err)
+			return err;
+	}
+
+	for (i = 0; i < h->length; i++) {
+		if (a[i].name)
+			ext2fs_free_mem(&a[i].name);
+		if (a[i].value)
+			ext2fs_free_mem(&a[i].value);
+	}
+
+	ext2fs_free_mem(&h->attrs);
+	ext2fs_free_mem(handle);
+	return 0;
+}

[PATCH 31/31] misc: Add fuse2fs, a FUSE server for e2fsprogs

[Darrick J. Wong]

This is the initial implementation of a FUSE server based on e2fsprogs.  The
point of this program is to enable ext4 to run on any OS that FUSE supports
(and doesn't already have a native driver), such as MacOS X, BSDs, and Windows.
The code requires FUSE API v28, which is available in Linux fuse and osxfuse
releases that are available as of August 2013.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 MCONFIG.in       |    1 
 configure        |   89 ++
 configure.in     |    9 
 misc/Makefile.in |   15 
 misc/fuse2fs.c   | 2870 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 2982 insertions(+), 2 deletions(-)
 create mode 100644 misc/fuse2fs.c


diff --git a/MCONFIG.in b/MCONFIG.in
index fa2b03e..9f88b55 100644
--- a/MCONFIG.in
+++ b/MCONFIG.in
@@ -93,6 +93,7 @@ LIBCOM_ERR = $(LIB)/libcom_err@LIB_EXT@ @PRIVATE_LIBS_CMT@ @SEM_INIT_LIB@
 LIBE2P = $(LIB)/libe2p@LIB_EXT@
 LIBEXT2FS = $(LIB)/libext2fs@LIB_EXT@
 LIBUUID = @LIBUUID@ @SOCKET_LIB@
+LIBFUSE = @FUSE_LIB@
 LIBQUOTA = @STATIC_LIBQUOTA@
 LIBBLKID = @LIBBLKID@ @PRIVATE_LIBS_CMT@ $(LIBUUID)
 LIBINTL = @LIBINTL@
diff --git a/configure b/configure
index 757c08e..eda3c82 100755
--- a/configure
+++ b/configure
@@ -639,6 +639,8 @@ CYGWIN_CMT
 LINUX_CMT
 UNI_DIFF_OPTS
 SEM_INIT_LIB
+FUSE_CMT
+FUSE_LIB
 SOCKET_LIB
 SIZEOF_OFF_T
 SIZEOF_LONG_LONG
@@ -11163,6 +11165,93 @@ if test "x$ac_cv_lib_socket_socket" = xyes; then :
 fi
 
 
+FUSE_CMT=''
+FUSE_LIB=''
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for fuse_main in -losxfuse" >&5
+$as_echo_n "checking for fuse_main in -losxfuse... " >&6; }
+if test "${ac_cv_lib_osxfuse_fuse_main+set}" = set; then :
+  $as_echo_n "(cached) " >&6
+else
+  ac_check_lib_save_LIBS=$LIBS
+LIBS="-losxfuse  $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char fuse_main ();
+int
+main ()
+{
+return fuse_main ();
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  ac_cv_lib_osxfuse_fuse_main=yes
+else
+  ac_cv_lib_osxfuse_fuse_main=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_osxfuse_fuse_main" >&5
+$as_echo "$ac_cv_lib_osxfuse_fuse_main" >&6; }
+if test "x$ac_cv_lib_osxfuse_fuse_main" = x""yes; then :
+  FUSE_LIB=-losxfuse
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: checking for fuse_main in -lfuse" >&5
+$as_echo_n "checking for fuse_main in -lfuse... " >&6; }
+if test "${ac_cv_lib_fuse_fuse_main+set}" = set; then :
+  $as_echo_n "(cached) " >&6
+else
+  ac_check_lib_save_LIBS=$LIBS
+LIBS="-lfuse  $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char fuse_main ();
+int
+main ()
+{
+return fuse_main ();
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  ac_cv_lib_fuse_fuse_main=yes
+else
+  ac_cv_lib_fuse_fuse_main=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_fuse_fuse_main" >&5
+$as_echo "$ac_cv_lib_fuse_fuse_main" >&6; }
+if test "x$ac_cv_lib_fuse_fuse_main" = x""yes; then :
+  FUSE_LIB=-lfuse
+else
+  FUSE_CMT="#"
+fi
+
+fi
+
+
+
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for optreset" >&5
 $as_echo_n "checking for optreset... " >&6; }
 if ${ac_cv_have_optreset+:} false; then :
diff --git a/configure.in b/configure.in
index 8963d17..ac52190 100644
--- a/configure.in
+++ b/configure.in
@@ -1119,6 +1119,15 @@ SOCKET_LIB=''
 AC_CHECK_LIB(socket, socket, [SOCKET_LIB=-lsocket])
 AC_SUBST(SOCKET_LIB)
 dnl
+dnl Check to see if the FUSE library is -lfuse or -losxfuse
+dnl
+FUSE_CMT=''
+FUSE_LIB=''
+dnl osxfuse.dylib supersedes fuselib.dylib
+AC_CHECK_LIB(osxfuse, fuse_main, [FUSE_LIB=-losxfuse], [AC_CHECK_LIB(fuse, fuse_main, [FUSE_LIB=-lfuse], [FUSE_CMT="#"])])
+AC_SUBST(FUSE_LIB)
+AC_SUBST(FUSE_CMT)
+dnl
 dnl See if optreset exists
 dnl
 AC_MSG_CHECKING(for optreset)
diff --git a/misc/Makefile.in b/misc/Makefile.in
index a798f96..1838d03 100644
--- a/misc/Makefile.in
+++ b/misc/Makefile.in
@@ -26,9 +26,12 @@ INSTALL = @INSTALL@
 @BLKID_CMT@FINDFS_LINK= findfs
 @BLKID_CMT@FINDFS_MAN= findfs.8
 
+@FUSE_CMT@FUSE_PROG= fuse2fs
+
 SPROGS=		mke2fs badblocks tune2fs dumpe2fs $(BLKID_PROG) logsave \
 			$(E2IMAGE_PROG) @FSCK_PROG@ e2undo
-USPROGS=	mklost+found filefrag e2freefrag $(UUIDD_PROG) $(E4DEFRAG_PROG)
+USPROGS=	mklost+found filefrag e2freefrag $(UUIDD_PROG) $(E4DEFRAG_PROG) \
+			$(FUSE_PROG)
 SMANPAGES=	tune2fs.8 mklost+found.8 mke2fs.8 dumpe2fs.8 badblocks.8 \
 			e2label.8 $(FINDFS_MAN) $(BLKID_MAN) $(E2IMAGE_MAN) \
 			logsave.8 filefrag.8 e2freefrag.8 e2undo.8 \
@@ -56,6 +59,7 @@ FILEFRAG_OBJS=	filefrag.o
 E2UNDO_OBJS=  e2undo.o
 E4DEFRAG_OBJS=	e4defrag.o
 E2FREEFRAG_OBJS= e2freefrag.o
+FUSE2FS_OBJS=	fuse2fs.o
 
 PROFILED_TUNE2FS_OBJS=	profiled/tune2fs.o profiled/util.o
 PROFILED_MKLPF_OBJS=	profiled/mklost+found.o
@@ -75,6 +79,7 @@ PROFILED_FILEFRAG_OBJS=	profiled/filefrag.o
 PROFILED_E2FREEFRAG_OBJS= profiled/e2freefrag.o
 PROFILED_E2UNDO_OBJS=	profiled/e2undo.o
 PROFILED_E4DEFRAG_OBJS=	profiled/e4defrag.o
+PROFILED_FUSE2FS_OJBS=	profiled/fuse2fs.o
 
 SRCS=	$(srcdir)/tune2fs.c $(srcdir)/mklost+found.c $(srcdir)/mke2fs.c \
 		$(srcdir)/chattr.c $(srcdir)/lsattr.c $(srcdir)/dumpe2fs.c \
@@ -82,7 +87,7 @@ SRCS=	$(srcdir)/tune2fs.c $(srcdir)/mklost+found.c $(srcdir)/mke2fs.c \
 		$(srcdir)/uuidgen.c $(srcdir)/blkid.c $(srcdir)/logsave.c \
 		$(srcdir)/filefrag.c $(srcdir)/base_device.c \
 		$(srcdir)/ismounted.c $(srcdir)/../e2fsck/profile.c \
-		$(srcdir)/e2undo.c $(srcdir)/e2freefrag.c
+		$(srcdir)/e2undo.c $(srcdir)/e2freefrag.c $(srcdir)/fuse2fs.c
 
 LIBS= $(LIBEXT2FS) $(LIBCOM_ERR) 
 DEPLIBS= $(LIBEXT2FS) $(DEPLIBCOM_ERR)
@@ -335,6 +340,12 @@ filefrag.profiled: $(FILEFRAG_OBJS)
 	$(Q) $(CC) $(ALL_LDFLAGS) -g -pg -o filefrag.profiled \
 		$(PROFILED_FILEFRAG_OBJS) 
 
+fuse2fs: $(FUSE2FS_OBJS) $(DEPLIBS) $(DEPLIBBLKID) $(DEPLIBUUID) \
+		$(DEPLIBQUOTA) $(LIBEXT2FS)
+	$(E) "	LD $@"
+	$(Q) $(CC) $(ALL_LDFLAGS) -o fuse2fs $(FUSE2FS_OBJS) $(LIBS) \
+		$(LIBFUSE) $(LIBBLKID) $(LIBUUID) $(LIBEXT2FS)
+
 tst_ismounted: $(srcdir)/ismounted.c $(STATIC_LIBEXT2FS) $(DEPLIBCOM_ERR)
 	$(E) "	LD $@"
 	$(CC) -o tst_ismounted $(srcdir)/ismounted.c -DDEBUG $(ALL_CFLAGS) \
diff --git a/misc/fuse2fs.c b/misc/fuse2fs.c
new file mode 100644
index 0000000..2d3f04c
--- /dev/null
+++ b/misc/fuse2fs.c
@@ -0,0 +1,2870 @@
+/*
+ * fuse2fs.c - FUSE server for e2fsprogs.
+ *
+ * Copyright (C) 2013 Oracle.
+ *
+ * %Begin-Header%
+ * This file may be redistributed under the terms of the GNU Public
+ * License.
+ * %End-Header%
+ */
+#define _FILE_OFFSET_BITS 64
+#define FUSE_USE_VERSION 29
+#define _GNU_SOURCE
+#include <pthread.h>
+#ifdef __linux__
+# include <linux/fs.h>
+# include <linux/falloc.h>
+# include <linux/xattr.h>
+#endif
+#include <sys/ioctl.h>
+#include <unistd.h>
+#include <fuse.h>
+#include "ext2fs/ext2fs.h"
+#include "ext2fs/ext2_fs.h"
+
+#if FUSE_VERSION >= FUSE_MAKE_VERSION(2, 8)
+# ifdef _IOR
+#  ifdef _IOW
+#   define SUPPORT_I_FLAGS
+#  endif
+# endif
+#endif
+
+#ifdef FALLOC_FL_KEEP_SIZE
+# define FL_KEEP_SIZE_FLAG FALLOC_FL_KEEP_SIZE
+#else
+# define FL_KEEP_SIZE_FLAG (0)
+#endif
+
+#ifdef FALLOC_FL_PUNCH_HOLE
+# define FL_PUNCH_HOLE_FLAG FALLOC_FL_PUNCH_HOLE
+#else
+# define FL_PUNCH_HOLE_FLAG (0)
+#endif
+
+/*
+ * ext2_file_t contains a struct inode, so we can't leave files open.
+ * Use this as a proxy instead.
+ */
+struct fuse2fs_file_handle {
+	ext2_ino_t ino;
+	int open_flags;
+};
+
+/* Main program context */
+struct fuse2fs {
+	ext2_filsys fs;
+	pthread_mutex_t bfl;
+	int panic_on_error;
+	FILE *err_fp;
+	unsigned int next_generation;
+};
+
+static int __translate_error(ext2_filsys fs, errcode_t err, ext2_ino_t ino,
+			     const char *file, int line);
+#define translate_error(fs, ino, err) __translate_error((fs), (err), (ino), \
+			__FILE__, __LINE__)
+
+/* for macosx */
+#ifndef W_OK
+#  define W_OK 2
+#endif
+
+#ifndef R_OK
+#  define R_OK 4
+#endif
+
+#define EXT4_EPOCH_BITS 2
+#define EXT4_EPOCH_MASK ((1 << EXT4_EPOCH_BITS) - 1)
+#define EXT4_NSEC_MASK  (~0UL << EXT4_EPOCH_BITS)
+
+/*
+ * Extended fields will fit into an inode if the filesystem was formatted
+ * with large inodes (-I 256 or larger) and there are not currently any EAs
+ * consuming all of the available space. For new inodes we always reserve
+ * enough space for the kernel's known extended fields, but for inodes
+ * created with an old kernel this might not have been the case. None of
+ * the extended inode fields is critical for correct filesystem operation.
+ * This macro checks if a certain field fits in the inode. Note that
+ * inode-size = GOOD_OLD_INODE_SIZE + i_extra_isize
+ */
+#define EXT4_FITS_IN_INODE(ext4_inode, field)		\
+	((offsetof(typeof(*ext4_inode), field) +	\
+	  sizeof((ext4_inode)->field))			\
+	<= (EXT2_GOOD_OLD_INODE_SIZE +			\
+	    (ext4_inode)->i_extra_isize))		\
+
+static inline __u32 ext4_encode_extra_time(const struct timespec *time)
+{
+	return (sizeof(time->tv_sec) > 4 ?
+		(time->tv_sec >> 32) & EXT4_EPOCH_MASK : 0) |
+	       ((time->tv_nsec << EXT4_EPOCH_BITS) & EXT4_NSEC_MASK);
+}
+
+static inline void ext4_decode_extra_time(struct timespec *time, __u32 extra)
+{
+	if (sizeof(time->tv_sec) > 4)
+		time->tv_sec |= (__u64)((extra) & EXT4_EPOCH_MASK) << 32;
+	time->tv_nsec = ((extra) & EXT4_NSEC_MASK) >> EXT4_EPOCH_BITS;
+}
+
+#define EXT4_INODE_SET_XTIME(xtime, timespec, raw_inode)		       \
+do {									       \
+	(raw_inode)->xtime = (timespec)->tv_sec;			       \
+	if (EXT4_FITS_IN_INODE(raw_inode, xtime ## _extra))		       \
+		(raw_inode)->xtime ## _extra =				       \
+				ext4_encode_extra_time(timespec);	       \
+} while (0)
+
+#define EXT4_EINODE_SET_XTIME(xtime, timespec, raw_inode)		       \
+do {									       \
+	if (EXT4_FITS_IN_INODE(raw_inode, xtime))			       \
+		(raw_inode)->xtime = (timespec)->tv_sec;		       \
+	if (EXT4_FITS_IN_INODE(raw_inode, xtime ## _extra))		       \
+		(raw_inode)->xtime ## _extra =				       \
+				ext4_encode_extra_time(timespec);	       \
+} while (0)
+
+#define EXT4_INODE_GET_XTIME(xtime, timespec, raw_inode)		       \
+do {									       \
+	(timespec)->tv_sec = (signed)((raw_inode)->xtime);		       \
+	if (EXT4_FITS_IN_INODE(raw_inode, xtime ## _extra))		       \
+		ext4_decode_extra_time((timespec),			       \
+				       raw_inode->xtime ## _extra);	       \
+	else								       \
+		(timespec)->tv_nsec = 0;				       \
+} while (0)
+
+#define EXT4_EINODE_GET_XTIME(xtime, timespec, raw_inode)		       \
+do {									       \
+	if (EXT4_FITS_IN_INODE(raw_inode, xtime))			       \
+		(timespec)->tv_sec =					       \
+			(signed)((raw_inode)->xtime);			       \
+	if (EXT4_FITS_IN_INODE(raw_inode, xtime ## _extra))		       \
+		ext4_decode_extra_time((timespec),			       \
+				       raw_inode->xtime ## _extra);	       \
+	else								       \
+		(timespec)->tv_nsec = 0;				       \
+} while (0)
+
+static void get_now(struct timespec *now)
+{
+#ifdef CLOCK_REALTIME
+	if (!clock_gettime(CLOCK_REALTIME, now))
+		return;
+#endif
+
+	now->tv_sec = time(NULL);
+	now->tv_nsec = 0;
+}
+
+static void increment_version(struct ext2_inode_large *inode)
+{
+	__u64 ver;
+
+	ver = inode->osd1.linux1.l_i_version;
+	if (EXT4_FITS_IN_INODE(inode, i_version_hi))
+		ver |= (__u64)inode->i_version_hi << 32;
+	ver++;
+	inode->osd1.linux1.l_i_version = ver;
+	if (EXT4_FITS_IN_INODE(inode, i_version_hi))
+		inode->i_version_hi = ver >> 32;
+}
+
+static void init_times(struct ext2_inode_large *inode)
+{
+	struct timespec now;
+
+	get_now(&now);
+	EXT4_INODE_SET_XTIME(i_atime, &now, inode);
+	EXT4_INODE_SET_XTIME(i_ctime, &now, inode);
+	EXT4_INODE_SET_XTIME(i_mtime, &now, inode);
+	EXT4_EINODE_SET_XTIME(i_crtime, &now, inode);
+	increment_version(inode);
+}
+
+static int update_ctime(ext2_filsys fs, ext2_ino_t ino,
+			struct ext2_inode_large *pinode)
+{
+	errcode_t err;
+	struct timespec now;
+	struct ext2_inode_large inode;
+
+	get_now(&now);
+
+	/* If user already has a inode buffer, just update that */
+	if (pinode) {
+		increment_version(pinode);
+		EXT4_INODE_SET_XTIME(i_ctime, &now, pinode);
+		return 0;
+	}
+
+	/* Otherwise we have to read-modify-write the inode */
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	increment_version(&inode);
+	EXT4_INODE_SET_XTIME(i_ctime, &now, &inode);
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	return 0;
+}
+
+static int update_atime(ext2_filsys fs, ext2_ino_t ino)
+{
+	errcode_t err;
+	struct ext2_inode_large inode, *pinode;
+	struct timespec atime, mtime, now;
+
+	if (!(fs->flags & EXT2_FLAG_RW))
+		return 0;
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	pinode = &inode;
+	EXT4_INODE_GET_XTIME(i_atime, &atime, pinode);
+	EXT4_INODE_GET_XTIME(i_mtime, &mtime, pinode);
+	get_now(&now);
+	/*
+	 * If atime is newer than mtime and atime hasn't been updated in more
+	 * than a day, skip the atime update.  Same idea as Linux "relatime".
+	 */
+	if (atime.tv_sec >= mtime.tv_sec && atime.tv_sec >= now.tv_sec - 86400)
+		return 0;
+	EXT4_INODE_SET_XTIME(i_atime, &now, &inode);
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	return 0;
+}
+
+static int update_mtime(ext2_filsys fs, ext2_ino_t ino)
+{
+	errcode_t err;
+	struct ext2_inode_large inode;
+	struct timespec now;
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	get_now(&now);
+	EXT4_INODE_SET_XTIME(i_mtime, &now, &inode);
+	EXT4_INODE_SET_XTIME(i_ctime, &now, &inode);
+	increment_version(&inode);
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	return 0;
+}
+
+static int ext2_file_type(unsigned int mode)
+{
+	if (LINUX_S_ISREG(mode))
+		return EXT2_FT_REG_FILE;
+
+	if (LINUX_S_ISDIR(mode))
+		return EXT2_FT_DIR;
+
+	if (LINUX_S_ISCHR(mode))
+		return EXT2_FT_CHRDEV;
+
+	if (LINUX_S_ISBLK(mode))
+		return EXT2_FT_BLKDEV;
+
+	if (LINUX_S_ISLNK(mode))
+		return EXT2_FT_SYMLINK;
+
+	if (LINUX_S_ISFIFO(mode))
+		return EXT2_FT_FIFO;
+
+	if (LINUX_S_ISSOCK(mode))
+		return EXT2_FT_SOCK;
+
+	return 0;
+}
+
+static int fs_writeable(ext2_filsys fs)
+{
+	return (fs->flags & EXT2_FLAG_RW) && (fs->super->s_error_count == 0);
+}
+
+static int __check_access(struct fuse_context *ctxt, ext2_filsys fs,
+			  ext2_ino_t ino, int mask, int ignore_flags)
+{
+	errcode_t err;
+	struct ext2_inode inode;
+	mode_t perms;
+
+	/* no writing to read-only or broken fs */
+	if ((mask & W_OK) && !fs_writeable(fs))
+		return -EROFS;
+
+	err = ext2fs_read_inode(fs, ino, &inode);
+	if (err)
+		return translate_error(fs, ino, err);
+
+	/* existence check */
+	if (mask == 0)
+		return 0;
+
+	/* is immutable? */
+	if (!ignore_flags && (mask & W_OK) &&
+	    (inode.i_flags & EXT2_IMMUTABLE_FL))
+		return -EPERM;
+
+	perms = inode.i_mode & 0777;
+
+	/* always allow root */
+	if (ctxt->uid == 0)
+		return 0;
+
+	/* allow owner, if perms match */
+	if (inode.i_uid == ctxt->uid) {
+		if ((mask << 6) & perms)
+			return 0;
+		return -EPERM;
+	}
+
+	/* allow group, if perms match */
+	if (inode.i_gid == ctxt->gid) {
+		if ((mask << 3) & perms)
+			return 0;
+		return -EPERM;
+	}
+
+	/* otherwise check other */
+	if (mask & perms)
+		return 0;
+	return -EPERM;
+}
+
+static int check_inum_access(struct fuse_context *ctxt, ext2_filsys fs,
+			     ext2_ino_t ino, int mask)
+{
+	return __check_access(ctxt, fs, ino, mask, 0);
+}
+
+static int check_flags_access(struct fuse_context *ctxt, ext2_filsys fs,
+			      ext2_ino_t ino, int mask)
+{
+	return __check_access(ctxt, fs, ino, mask, 1);
+}
+
+static void op_destroy(void *p)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+
+	if (fs->flags & EXT2_FLAG_RW) {
+		fs->super->s_state |= EXT2_VALID_FS;
+		if (fs->super->s_error_count)
+			fs->super->s_state |= EXT2_ERROR_FS;
+		ext2fs_mark_super_dirty(fs);
+		err = ext2fs_set_gdt_csum(fs);
+		if (err)
+			translate_error(fs, 0, err);
+	}
+}
+
+static void *op_init(struct fuse_conn_info *conn)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+
+	if (fs->flags & EXT2_FLAG_RW) {
+		fs->super->s_mnt_count++;
+		fs->super->s_mtime = time(NULL);
+		fs->super->s_state &= ~EXT2_VALID_FS;
+		ext2fs_mark_super_dirty(fs);
+		err = ext2fs_flush2(fs, 0);
+		if (err)
+			translate_error(fs, 0, err);
+	}
+	return ff;
+}
+
+static int stat_inode(ext2_filsys fs, ext2_ino_t ino, struct stat *statbuf)
+{
+	struct ext2_inode_large inode;
+	dev_t fakedev = 0;
+	errcode_t err;
+	int ret = 0;
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err)
+		return translate_error(fs, ino, err);
+
+	memcpy(&fakedev, fs->super->s_uuid, sizeof(fakedev));
+	statbuf->st_dev = fakedev;
+	statbuf->st_ino = ino;
+	statbuf->st_mode = inode.i_mode;
+	statbuf->st_nlink = inode.i_links_count;
+	statbuf->st_uid = inode.i_uid;
+	statbuf->st_gid = inode.i_gid;
+	statbuf->st_size = inode.i_size;
+	statbuf->st_blksize = fs->blocksize;
+	statbuf->st_blocks = inode.i_blocks;
+	statbuf->st_atime = inode.i_atime;
+	statbuf->st_mtime = inode.i_mtime;
+	statbuf->st_ctime = inode.i_ctime;
+	if (LINUX_S_ISCHR(inode.i_mode) ||
+	    LINUX_S_ISBLK(inode.i_mode)) {
+		if (inode.i_block[0])
+			statbuf->st_rdev = inode.i_block[0];
+		else
+			statbuf->st_rdev = inode.i_block[1];
+	}
+
+	return ret;
+}
+
+static int op_getattr(const char *path, struct stat *statbuf)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+	ret = stat_inode(fs, ino, statbuf);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_readlink(const char *path, char *buf, size_t len)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	struct ext2_inode inode;
+	unsigned int got;
+	ext2_file_t file;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	err = ext2fs_read_inode(fs, ino, &inode);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	if (!LINUX_S_ISLNK(inode.i_mode)) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	len--;
+	if (inode.i_size < len)
+		len = inode.i_size;
+	if (ext2fs_inode_data_blocks2(fs, &inode)) {
+		/* big symlink */
+
+		err = ext2fs_file_open(fs, ino, 0, &file);
+		if (err) {
+			ret = translate_error(fs, ino, err);
+			goto out;
+		}
+
+		err = ext2fs_file_read(file, buf, len, &got);
+		if (err || got != len) {
+			ext2fs_file_close(file);
+			ret = translate_error(fs, ino, err);
+			goto out;
+		}
+
+		err = ext2fs_file_close(file);
+		if (err) {
+			ret = translate_error(fs, ino, err);
+			goto out;
+		}
+	} else
+		/* inline symlink */
+		memcpy(buf, (char *)inode.i_block, len);
+	buf[len] = 0;
+
+	if (fs_writeable(fs)) {
+		ret = update_atime(fs, ino);
+		if (ret)
+			goto out;
+	}
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t parent, child;
+	char *temp_path = strdup(path);
+	errcode_t err;
+	char *node_name, a;
+	int filetype;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	if (!temp_path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name = strrchr(temp_path, '/');
+	if (!node_name) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name++;
+	a = *node_name;
+	*node_name = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &parent);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	ret = check_inum_access(ctxt, fs, parent, W_OK);
+	if (ret)
+		goto out2;
+
+	*node_name = a;
+
+	if (LINUX_S_ISCHR(mode))
+		filetype = EXT2_FT_CHRDEV;
+	else if (LINUX_S_ISBLK(mode))
+		filetype = EXT2_FT_BLKDEV;
+	else if (LINUX_S_ISFIFO(mode))
+		filetype = EXT2_FT_FIFO;
+	else {
+		ret = -EINVAL;
+		goto out2;
+	}
+
+	err = ext2fs_new_inode(fs, parent, mode, 0, &child);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	err = ext2fs_link(fs, parent, node_name, child, filetype);
+	if (err == EXT2_ET_DIR_NO_SPACE) {
+		err = ext2fs_expand_dir(fs, parent);
+		if (err) {
+			ret = translate_error(fs, parent, err);
+			goto out2;
+		}
+
+		err = ext2fs_link(fs, parent, node_name, child,
+				     filetype);
+	}
+	if (err) {
+		ret = translate_error(fs, parent, err);
+		goto out2;
+	}
+
+	ret = update_mtime(fs, parent);
+	if (ret)
+		goto out2;
+
+	memset(&inode, 0, sizeof(inode));
+	inode.i_mode = mode;
+
+	if (dev & ~0xFFFF)
+		inode.i_block[1] = dev;
+	else
+		inode.i_block[0] = dev;
+	inode.i_links_count = 1;
+	inode.i_extra_isize = sizeof(struct ext2_inode_large) -
+		EXT2_GOOD_OLD_INODE_SIZE;
+
+	err = ext2fs_write_new_inode(fs, child, (struct ext2_inode *)&inode);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	inode.i_generation = ff->next_generation++;
+	init_times(&inode);
+	err = ext2fs_write_inode_full(fs, child, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	ext2fs_inode_alloc_stats2(fs, child, 1, 0);
+
+	err = ext2fs_flush2(fs, EXT2_FLAG_FLUSH_NO_SYNC);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+out2:
+	pthread_mutex_unlock(&ff->bfl);
+out:
+	free(temp_path);
+	return ret;
+}
+
+static int op_mkdir(const char *path, mode_t mode)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t parent, child;
+	char *temp_path = strdup(path);
+	errcode_t err;
+	char *node_name, a;
+	struct ext2_inode_large inode;
+	char *block;
+	blk64_t blk;
+	int ret = 0;
+
+	if (!temp_path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name = strrchr(temp_path, '/');
+	if (!node_name) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name++;
+	a = *node_name;
+	*node_name = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &parent);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	ret = check_inum_access(ctxt, fs, parent, W_OK);
+	if (ret)
+		goto out2;
+
+	*node_name = a;
+
+	err = ext2fs_mkdir(fs, parent, 0, node_name);
+	if (err == EXT2_ET_DIR_NO_SPACE) {
+		err = ext2fs_expand_dir(fs, parent);
+		if (err) {
+			ret = translate_error(fs, parent, err);
+			goto out2;
+		}
+
+		err = ext2fs_mkdir(fs, parent, 0, node_name);
+	}
+	if (err) {
+		ret = translate_error(fs, parent, err);
+		goto out2;
+	}
+
+	ret = update_mtime(fs, parent);
+	if (ret)
+		goto out2;
+
+	/* Still have to update the uid/gid of the dir */
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &child);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	err = ext2fs_read_inode_full(fs, child, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	inode.i_uid = ctxt->uid;
+	inode.i_gid = ctxt->gid;
+	inode.i_generation = ff->next_generation++;
+
+	err = ext2fs_write_inode_full(fs, child, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	/* Rewrite the directory block checksum, having set i_generation */
+	if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+					EXT4_FEATURE_RO_COMPAT_METADATA_CSUM))
+		goto out2;
+	err = ext2fs_new_dir_block(fs, child, parent, &block);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+	err = ext2fs_bmap2(fs, child, (struct ext2_inode *)&inode, NULL, 0, 0,
+			   NULL, &blk);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out3;
+	}
+	err = ext2fs_write_dir_block4(fs, blk, block, 0, child);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out3;
+	}
+
+out3:
+	ext2fs_free_mem(&block);
+out2:
+	pthread_mutex_unlock(&ff->bfl);
+out:
+	free(temp_path);
+	return ret;
+}
+
+static int unlink_file_by_name(struct fuse_context *ctxt, ext2_filsys fs,
+			       const char *path)
+{
+	errcode_t err;
+	ext2_ino_t dir;
+	char *filename = strdup(path);
+	char *base_name;
+	int ret;
+
+	base_name = strrchr(filename, '/');
+	if (base_name) {
+		*base_name++ = '\0';
+		err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, filename,
+				   &dir);
+		if (err) {
+			free(filename);
+			return translate_error(fs, 0, err);
+		}
+	} else {
+		dir = EXT2_ROOT_INO;
+		base_name = filename;
+	}
+
+	ret = check_inum_access(ctxt, fs, dir, W_OK);
+	if (ret) {
+		free(filename);
+		return ret;
+	}
+
+	err = ext2fs_unlink(fs, dir, base_name, 0, 0);
+	free(filename);
+	if (err)
+		return translate_error(fs, dir, err);
+
+	return update_mtime(fs, dir);
+}
+
+static int release_blocks_proc(ext2_filsys fs, blk64_t *blocknr,
+			       e2_blkcnt_t blockcnt EXT2FS_ATTR((unused)),
+			       blk64_t ref_block EXT2FS_ATTR((unused)),
+			       int ref_offset EXT2FS_ATTR((unused)),
+			       void *private EXT2FS_ATTR((unused)))
+{
+	blk64_t blk = *blocknr;
+
+	if (blk % EXT2FS_CLUSTER_RATIO(fs) == 0)
+		ext2fs_block_alloc_stats2(fs, *blocknr, -1);
+	return 0;
+}
+
+static int remove_inode(struct fuse2fs *ff, ext2_ino_t ino)
+{
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	switch (inode.i_links_count) {
+	case 0:
+		return 0; /* XXX: already done? */
+	case 1:
+		inode.i_links_count--;
+		inode.i_dtime = fs->now ? fs->now : time(0);
+		break;
+	default:
+		inode.i_links_count--;
+	}
+
+	ret = update_ctime(fs, ino, &inode);
+	if (ret)
+		goto out;
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	if (inode.i_links_count)
+		goto out;
+
+	if (ext2fs_inode_has_valid_blocks2(fs, (struct ext2_inode *)&inode))
+		ext2fs_block_iterate3(fs, ino, BLOCK_FLAG_READ_ONLY, NULL,
+				      release_blocks_proc, NULL);
+	ext2fs_inode_alloc_stats2(fs, ino, -1,
+				  LINUX_S_ISDIR(inode.i_mode));
+
+	err = ext2fs_flush2(fs, EXT2_FLAG_FLUSH_NO_SYNC);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+out:
+	return ret;
+}
+
+static int __op_unlink(const char *path)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	ret = unlink_file_by_name(ctxt, fs, path);
+	if (ret)
+		goto out;
+
+	ret = remove_inode(ff, ino);
+	if (ret)
+		goto out;
+out:
+	return ret;
+}
+
+static int op_unlink(const char *path)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	int ret;
+
+	pthread_mutex_lock(&ff->bfl);
+	ret = __op_unlink(path);
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+struct rd_struct {
+	ext2_ino_t	parent;
+	int		empty;
+};
+
+static int rmdir_proc(ext2_ino_t dir EXT2FS_ATTR((unused)),
+		      int	entry EXT2FS_ATTR((unused)),
+		      struct ext2_dir_entry *dirent,
+		      int	offset EXT2FS_ATTR((unused)),
+		      int	blocksize EXT2FS_ATTR((unused)),
+		      char	*buf EXT2FS_ATTR((unused)),
+		      void	*private)
+{
+	struct rd_struct *rds = (struct rd_struct *) private;
+
+	if (dirent->inode == 0)
+		return 0;
+	if (((dirent->name_len & 0xFF) == 1) && (dirent->name[0] == '.'))
+		return 0;
+	if (((dirent->name_len & 0xFF) == 2) && (dirent->name[0] == '.') &&
+	    (dirent->name[1] == '.')) {
+		rds->parent = dirent->inode;
+		return 0;
+	}
+	rds->empty = 0;
+	return 0;
+}
+
+static int op_rmdir(const char *path)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t child;
+	errcode_t err;
+	struct ext2_inode inode;
+	struct rd_struct rds;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &child);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, child, W_OK);
+	if (ret)
+		goto out;
+
+	rds.parent = 0;
+	rds.empty = 1;
+
+	err = ext2fs_dir_iterate2(fs, child, 0, 0, rmdir_proc, &rds);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out;
+	}
+
+	if (rds.empty == 0) {
+		ret = -ENOTEMPTY;
+		goto out;
+	}
+
+	ret = unlink_file_by_name(ctxt, fs, path);
+	if (ret)
+		goto out;
+	/* Directories have to be "removed" twice. */
+	ret = remove_inode(ff, child);
+	if (ret)
+		goto out;
+	ret = remove_inode(ff, child);
+	if (ret)
+		goto out;
+
+	if (rds.parent) {
+		err = ext2fs_read_inode(fs, rds.parent, &inode);
+		if (err) {
+			ret = translate_error(fs, rds.parent, err);
+			goto out;
+		}
+		if (inode.i_links_count > 1)
+			inode.i_links_count--;
+		ret = update_mtime(fs, rds.parent);
+		if (ret)
+			goto out;
+		err = ext2fs_write_inode(fs, rds.parent, &inode);
+		if (err) {
+			ret = translate_error(fs, rds.parent, err);
+			goto out;
+		}
+	}
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_symlink(const char *src, const char *dest)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t parent, child;
+	char *temp_path = strdup(dest);
+	errcode_t err;
+	char *node_name, a;
+	struct ext2_inode_large inode;
+	int len = strlen(src);
+	int ret = 0;
+
+	if (!temp_path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name = strrchr(temp_path, '/');
+	if (!node_name) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name++;
+	a = *node_name;
+	*node_name = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &parent);
+	*node_name = a;
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	ret = check_inum_access(ctxt, fs, parent, W_OK);
+	if (ret)
+		goto out2;
+
+
+	/* Create symlink */
+	err = ext2fs_symlink(fs, parent, 0, node_name, (char *)src);
+	if (err) {
+		ret = translate_error(fs, parent, err);
+		goto out2;
+	}
+
+	/* Update parent dir's mtime */
+	ret = update_mtime(fs, parent);
+	if (ret)
+		goto out2;
+
+	/* Still have to update the uid/gid of the symlink */
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &child);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	err = ext2fs_read_inode_full(fs, child, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	inode.i_uid = ctxt->uid;
+	inode.i_gid = ctxt->gid;
+	inode.i_generation = ff->next_generation++;
+
+	err = ext2fs_write_inode_full(fs, child, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+out2:
+	pthread_mutex_unlock(&ff->bfl);
+out:
+	free(temp_path);
+	return ret;
+}
+
+static int op_rename(const char *from, const char *to)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t from_ino, to_ino, to_dir_ino, from_dir_ino;
+	char *temp_to = NULL, *temp_from = NULL;
+	char *cp, a;
+	struct ext2_inode from_inode;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, from, &from_ino);
+	if (err || from_ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, to, &to_ino);
+	if (err && err != EXT2_ET_FILE_NOT_FOUND) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	if (err == EXT2_ET_FILE_NOT_FOUND)
+		to_ino = 0;
+
+	/* Already the same file? */
+	if (to_ino != 0 && to_ino == from_ino) {
+		ret = 0;
+		goto out;
+	}
+
+	temp_to = strdup(to);
+	if (!temp_to) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	temp_from = strdup(from);
+	if (!temp_from) {
+		ret = -ENOMEM;
+		goto out2;
+	}
+
+	/* Find parent dir of the source and check write access */
+	cp = strrchr(temp_from, '/');
+	if (!cp) {
+		ret = -EINVAL;
+		goto out2;
+	}
+
+	a = *(cp + 1);
+	*(cp + 1) = 0;
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_from,
+			   &from_dir_ino);
+	*(cp + 1) = a;
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+	if (from_dir_ino == 0) {
+		ret = -ENOENT;
+		goto out2;
+	}
+
+	ret = check_inum_access(ctxt, fs, from_dir_ino, W_OK);
+	if (ret)
+		goto out2;
+
+	/* Find parent dir of the destination and check write access */
+	cp = strrchr(temp_to, '/');
+	if (!cp) {
+		ret = -EINVAL;
+		goto out2;
+	}
+
+	a = *(cp + 1);
+	*(cp + 1) = 0;
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_to,
+			   &to_dir_ino);
+	*(cp + 1) = a;
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+	if (to_dir_ino == 0) {
+		ret = -ENOENT;
+		goto out2;
+	}
+
+	ret = check_inum_access(ctxt, fs, to_dir_ino, W_OK);
+	if (ret)
+		goto out2;
+
+	/* Get ready to do the move */
+	err = ext2fs_read_inode(fs, from_ino, &from_inode);
+	if (err) {
+		ret = translate_error(fs, from_ino, err);
+		goto out2;
+	}
+
+	/* If the target exists, unlink it first */
+	if (to_ino != 0) {
+		ret = __op_unlink(to);
+		if (ret)
+			goto out2;
+	}
+
+	/* Link in the new file */
+	err = ext2fs_link(fs, to_dir_ino, cp + 1, from_ino,
+			  ext2_file_type(from_inode.i_mode));
+	if (err == EXT2_ET_DIR_NO_SPACE) {
+		err = ext2fs_expand_dir(fs, to_dir_ino);
+		if (err) {
+			ret = translate_error(fs, to_dir_ino, err);
+			goto out2;
+		}
+
+		err = ext2fs_link(fs, to_dir_ino, cp + 1, from_ino,
+				     ext2_file_type(from_inode.i_mode));
+	}
+	if (err) {
+		ret = translate_error(fs, to_dir_ino, err);
+		goto out2;
+	}
+
+	ret = update_mtime(fs, to_dir_ino);
+	if (ret)
+		goto out2;
+
+	/* Remove the old file */
+	ret = unlink_file_by_name(ctxt, fs, from);
+	if (ret)
+		goto out2;
+
+	/* Flush the whole mess out */
+	err = ext2fs_flush2(fs, 0);
+	if (err)
+		ret = translate_error(fs, 0, err);
+
+out2:
+	free(temp_from);
+	free(temp_to);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_link(const char *src, const char *dest)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	char *temp_path = strdup(dest);
+	errcode_t err;
+	char *node_name, a;
+	ext2_ino_t parent, ino;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	if (!temp_path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name = strrchr(temp_path, '/');
+	if (!node_name) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name++;
+	a = *node_name;
+	*node_name = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &parent);
+	*node_name = a;
+	if (err) {
+		err = -ENOENT;
+		goto out2;
+	}
+
+	ret = check_inum_access(ctxt, fs, parent, W_OK);
+	if (ret)
+		goto out2;
+
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, src, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	inode.i_links_count++;
+	ret = update_ctime(fs, ino, &inode);
+	if (ret)
+		goto out2;
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	err = ext2fs_link(fs, parent, node_name, ino,
+			  ext2_file_type(inode.i_mode));
+	if (err == EXT2_ET_DIR_NO_SPACE) {
+		err = ext2fs_expand_dir(fs, parent);
+		if (err) {
+			ret = translate_error(fs, parent, err);
+			goto out2;
+		}
+
+		err = ext2fs_link(fs, parent, node_name, ino,
+				     ext2_file_type(inode.i_mode));
+	}
+	if (err) {
+		ret = translate_error(fs, parent, err);
+		goto out2;
+	}
+
+	ret = update_mtime(fs, parent);
+	if (ret)
+		goto out;
+
+out2:
+	pthread_mutex_unlock(&ff->bfl);
+out:
+	free(temp_path);
+	return ret;
+}
+
+static int op_chmod(const char *path, mode_t mode)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	/* XXX: Fails if uid matches but u-w */
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	inode.i_mode &= ~0xFFF;
+	inode.i_mode |= mode & 0xFFF;
+	ret = update_ctime(fs, ino, &inode);
+	if (ret)
+		goto out;
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_chown(const char *path, uid_t owner, gid_t group)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	inode.i_uid = owner;
+	inode.i_gid = group;
+	ret = update_ctime(fs, ino, &inode);
+	if (ret)
+		goto out;
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_truncate(const char *path, off_t len)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	ext2_file_t file;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_file_open(fs, ino, EXT2_FILE_WRITE, &file);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_set_size2(file, len);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+out2:
+	err = ext2fs_file_close(file);
+	if (err && !ret) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	if (!ret)
+		ret = update_mtime(fs, ino);
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return err;
+}
+
+static int __op_open(const char *path, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	struct fuse2fs_file_handle *file;
+	int check, ret = 0;
+
+	file = calloc(1, sizeof(*file));
+	if (!file)
+		return -ENOMEM;
+
+	file->open_flags = 0;
+	if (fp->flags & (O_RDWR | O_WRONLY))
+		file->open_flags |= EXT2_FILE_WRITE;
+	if (fp->flags & O_CREAT)
+		file->open_flags |= EXT2_FILE_CREATE;
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &file->ino);
+	if (err || file->ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	check = R_OK;
+	if (file->open_flags & EXT2_FILE_WRITE)
+		check |= W_OK;
+	ret = check_inum_access(ctxt, fs, file->ino, check);
+	if (ret)
+		goto out;
+	fp->fh = (uint64_t)file;
+
+out:
+	if (ret)
+		free(file);
+	return ret;
+}
+
+static int op_open(const char *path, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	int ret;
+
+	pthread_mutex_lock(&ff->bfl);
+	ret = __op_open(path, fp);
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_read(const char *path, char *buf, size_t len, off_t offset,
+		     struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	ext2_file_t efp;
+	errcode_t err;
+	unsigned int got;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_file_open(fs, fh->ino, fh->open_flags, &efp);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_llseek(efp, offset, SEEK_SET, NULL);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_read(efp, buf, len, &got);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_close(efp);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	if (fs_writeable(fs)) {
+		ret = update_atime(fs, fh->ino);
+		if (ret)
+			goto out;
+	}
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return got ? got : ret;
+}
+
+static int op_write(const char *path, const char *buf, size_t len, off_t offset,
+		      struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	ext2_file_t efp;
+	errcode_t err;
+	unsigned int got;
+	__u64 fsize;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!fs_writeable(fs)) {
+		ret = -EROFS;
+		goto out;
+	}
+
+	err = ext2fs_file_open(fs, fh->ino, fh->open_flags, &efp);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_llseek(efp, offset, SEEK_SET, NULL);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_write(efp, buf, len, &got);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_flush(efp);
+	if (err) {
+		got = 0;
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	/*
+	 * Apparently ext2fs_file_write will dirty the inode (to allocate
+	 * blocks) without bothering to write out the inode, so change the
+	 * file size *after* the write, because changing the size forces
+	 * the inode out to disk.
+	 */
+	err = ext2fs_file_get_lsize(efp, &fsize);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+	if (offset + len > fsize) {
+		fsize = offset + len;
+		err = ext2fs_file_set_size2(efp, fsize);
+		if (err) {
+			ret = translate_error(fs, fh->ino, err);
+			goto out;
+		}
+	}
+
+	err = ext2fs_file_close(efp);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	ret = update_mtime(fs, fh->ino);
+	if (ret)
+		goto out;
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return got ? got : ret;
+}
+
+static int op_flush(const char *path, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (fs_writeable(fs) && fh->open_flags & EXT2_FILE_WRITE) {
+		err = ext2fs_flush2(fs, EXT2_FLAG_FLUSH_NO_SYNC);
+		if (err)
+			ret = translate_error(fs, fh->ino, err);
+	}
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+static int op_release(const char *path, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (fs_writeable(fs) && fh->open_flags & EXT2_FILE_WRITE) {
+		err = ext2fs_flush2(fs, EXT2_FLAG_FLUSH_NO_SYNC);
+		if (err)
+			ret = translate_error(fs, fh->ino, err);
+	}
+	fp->fh = 0;
+	pthread_mutex_unlock(&ff->bfl);
+
+	free(fh);
+
+	return ret;
+}
+
+static int op_fsync(const char *path, int datasync, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	errcode_t err;
+	int ret = 0;
+
+	/* For now, flush everything, even if it's slow */
+	pthread_mutex_lock(&ff->bfl);
+	if (fs_writeable(fs) && fh->open_flags & EXT2_FILE_WRITE) {
+		err = ext2fs_flush2(fs, 0);
+		if (err)
+			ret = translate_error(fs, fh->ino, err);
+	}
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+static int op_statfs(const char *path, struct statvfs *buf)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	uint64_t fsid, *f;
+
+	buf->f_bsize = fs->blocksize;
+	buf->f_frsize = 0;
+	buf->f_blocks = fs->super->s_blocks_count;
+	buf->f_bfree = fs->super->s_free_blocks_count;
+	if (fs->super->s_free_blocks_count < fs->super->s_r_blocks_count)
+		buf->f_bavail = 0;
+	else
+		buf->f_bavail = fs->super->s_free_blocks_count -
+				fs->super->s_r_blocks_count;
+	buf->f_files = fs->super->s_inodes_count;
+	buf->f_ffree = fs->super->s_free_inodes_count;
+	buf->f_favail = fs->super->s_free_inodes_count;
+	f = (uint64_t *)fs->super->s_uuid;
+	fsid = *f;
+	f++;
+	fsid ^= *f;
+	buf->f_fsid = fsid;
+	buf->f_flag = 0;
+	if (fs->flags & EXT2_FLAG_RW)
+		buf->f_flag |= ST_RDONLY;
+	buf->f_namemax = EXT2_NAME_LEN;
+
+	return 0;
+}
+
+static int op_getxattr(const char *path, const char *key, char *value,
+		       size_t len)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct ext2_xattr_handle *h;
+	void *ptr;
+	unsigned int plen;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!EXT2_HAS_COMPAT_FEATURE(fs->super,
+				     EXT2_FEATURE_COMPAT_EXT_ATTR)) {
+		ret = -ENOTSUP;
+		goto out;
+	}
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, R_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_xattrs_open(fs, ino, &h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	err = ext2fs_xattrs_read(h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	err = ext2fs_xattr_get(h, key, &ptr, &plen);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	if (!len) {
+		ret = plen;
+	} else if (len < plen) {
+		ret = -ERANGE;
+	} else {
+		memcpy(value, ptr, plen);
+		ret = plen;
+	}
+
+	ext2fs_free_mem(&ptr);
+out2:
+	err = ext2fs_xattrs_close(&h);
+	if (err)
+		ret = translate_error(fs, ino, err);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+static int count_buffer_space(char *name, char *value, void *data)
+{
+	unsigned int *x = data;
+
+	*x = *x + strlen(name) + 1;
+	return 0;
+}
+
+static int copy_names(char *name, char *value, void *data)
+{
+	char **b = data;
+
+	strncpy(*b, name, strlen(name));
+	*b = *b + strlen(name) + 1;
+
+	return 0;
+}
+
+static int op_listxattr(const char *path, char *names, size_t len)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct ext2_xattr_handle *h;
+	unsigned int bufsz;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!EXT2_HAS_COMPAT_FEATURE(fs->super,
+				     EXT2_FEATURE_COMPAT_EXT_ATTR)) {
+		ret = -ENOTSUP;
+		goto out;
+	}
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, R_OK);
+	if (ret)
+		goto out2;
+
+	err = ext2fs_xattrs_open(fs, ino, &h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	err = ext2fs_xattrs_read(h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	/* Count buffer space needed for names */
+	bufsz = 0;
+	err = ext2fs_xattrs_iterate(h, count_buffer_space, &bufsz);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	if (len == 0) {
+		ret = bufsz;
+		goto out2;
+	} else if (len < bufsz) {
+		ret = -ERANGE;
+		goto out2;
+	}
+
+	/* Copy names out */
+	memset(names, 0, len);
+	err = ext2fs_xattrs_iterate(h, copy_names, &names);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+	ret = bufsz;
+out2:
+	err = ext2fs_xattrs_close(&h);
+	if (err)
+		ret = translate_error(fs, ino, err);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+static int op_setxattr(const char *path, const char *key, const char *value,
+		       size_t len, int flags)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct ext2_xattr_handle *h;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!EXT2_HAS_COMPAT_FEATURE(fs->super,
+				     EXT2_FEATURE_COMPAT_EXT_ATTR)) {
+		ret = -ENOTSUP;
+		goto out;
+	}
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_xattrs_open(fs, ino, &h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	err = ext2fs_xattrs_read(h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	err = ext2fs_xattr_set(h, key, value, len);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	err = ext2fs_xattrs_write(h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+out2:
+	err = ext2fs_xattrs_close(&h);
+	if (err)
+		ret = translate_error(fs, ino, err);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+static int op_removexattr(const char *path, const char *key)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct ext2_xattr_handle *h;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!EXT2_HAS_COMPAT_FEATURE(fs->super,
+				     EXT2_FEATURE_COMPAT_EXT_ATTR)) {
+		ret = -ENOTSUP;
+		goto out;
+	}
+
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_xattrs_open(fs, ino, &h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	err = ext2fs_xattrs_read(h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	err = ext2fs_xattr_remove(h, key);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+	err = ext2fs_xattrs_write(h);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out2;
+	}
+
+out2:
+	err = ext2fs_xattrs_close(&h);
+	if (err)
+		ret = translate_error(fs, ino, err);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+struct readdir_iter {
+	void *buf;
+	fuse_fill_dir_t func;
+};
+
+static int op_readdir_iter(ext2_ino_t dir, int entry,
+			   struct ext2_dir_entry *dirent, int offset,
+			   int blocksize, char *buf, void *data)
+{
+	struct readdir_iter *i = data;
+	struct stat statbuf;
+	char namebuf[EXT2_NAME_LEN + 1];
+	int ret;
+
+	memcpy(namebuf, dirent->name, dirent->name_len & 0xFF);
+	namebuf[dirent->name_len & 0xFF] = 0;
+	statbuf.st_ino = dirent->inode;
+	statbuf.st_mode = S_IFREG;
+	ret = i->func(i->buf, namebuf, NULL, 0);
+	if (ret)
+		return DIRENT_ABORT;
+
+	return 0;
+}
+
+static int op_readdir(const char *path, void *buf, fuse_fill_dir_t fill_func,
+		      off_t offset, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	errcode_t err;
+	ext2_ino_t ino;
+	struct readdir_iter i;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	i.buf = buf;
+	i.func = fill_func;
+	err = ext2fs_dir_iterate2(fs, fh->ino, 0, NULL, op_readdir_iter, &i);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	if (fs_writeable(fs)) {
+		ret = update_atime(fs, fh->ino);
+		if (ret)
+			goto out;
+	}
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_access(const char *path, int mask)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err || ino == 0) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, mask);
+	if (ret)
+		goto out;
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+static int op_create(const char *path, mode_t mode, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct ext3_extent_header *eh;
+	ext2_ino_t parent, child;
+	char *temp_path = strdup(path);
+	errcode_t err;
+	char *node_name, a;
+	int filetype, i;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	if (!temp_path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name = strrchr(temp_path, '/');
+	if (!node_name) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	node_name++;
+	a = *node_name;
+	*node_name = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, temp_path,
+			   &parent);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out2;
+	}
+
+	err = check_inum_access(ctxt, fs, parent, W_OK);
+	if (err)
+		goto out;
+
+	*node_name = a;
+
+	filetype = ext2_file_type(mode);
+
+	err = ext2fs_new_inode(fs, parent, mode, 0, &child);
+	if (err) {
+		ret = translate_error(fs, parent, err);
+		goto out2;
+	}
+
+	err = ext2fs_link(fs, parent, node_name, child, filetype);
+	if (err == EXT2_ET_DIR_NO_SPACE) {
+		err = ext2fs_expand_dir(fs, parent);
+		if (err) {
+			ret = translate_error(fs, parent, err);
+			goto out2;
+		}
+
+		err = ext2fs_link(fs, parent, node_name, child,
+				     filetype);
+	}
+	if (err) {
+		ret = translate_error(fs, parent, err);
+		goto out2;
+	}
+
+	ret = update_mtime(fs, parent);
+	if (ret)
+		goto out2;
+
+	memset(&inode, 0, sizeof(inode));
+	inode.i_mode = mode;
+	inode.i_links_count = 1;
+	inode.i_extra_isize = sizeof(struct ext2_inode_large) -
+		EXT2_GOOD_OLD_INODE_SIZE;
+	if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS) {
+		inode.i_flags = EXT4_EXTENTS_FL;
+
+		/* This must be initialized, even for a zero byte file. */
+		eh = (struct ext3_extent_header *) &inode.i_block[0];
+		eh->eh_magic = ext2fs_cpu_to_le16(EXT3_EXT_MAGIC);
+		eh->eh_depth = 0;
+		eh->eh_entries = 0;
+		i = (sizeof(inode.i_block) - sizeof(*eh)) /
+			sizeof(struct ext3_extent);
+		eh->eh_max = ext2fs_cpu_to_le16(i);
+	}
+
+	err = ext2fs_write_new_inode(fs, child, (struct ext2_inode *)&inode);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	inode.i_generation = ff->next_generation++;
+	init_times(&inode);
+	err = ext2fs_write_inode_full(fs, child, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	ext2fs_inode_alloc_stats2(fs, child, 1, 0);
+
+	err = ext2fs_flush2(fs, EXT2_FLAG_FLUSH_NO_SYNC);
+	if (err) {
+		ret = translate_error(fs, child, err);
+		goto out2;
+	}
+
+	ret = __op_open(path, fp);
+	if (ret)
+		goto out2;
+out2:
+	pthread_mutex_unlock(&ff->bfl);
+out:
+	free(temp_path);
+	return ret;
+}
+
+static int op_ftruncate(const char *path, off_t len, struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	ext2_file_t efp;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!fs_writeable(fs)) {
+		ret = -EROFS;
+		goto out;
+	}
+
+	err = ext2fs_file_open(fs, fh->ino, fh->open_flags, &efp);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_set_size2(efp, len);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	err = ext2fs_file_close(efp);
+	if (err) {
+		ret = translate_error(fs, fh->ino, err);
+		goto out;
+	}
+
+	ret = update_mtime(fs, fh->ino);
+	if (ret)
+		goto out;
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return 0;
+}
+
+static int op_fgetattr(const char *path, struct stat *statbuf,
+		       struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	ret = stat_inode(fs, fh->ino, statbuf);
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+
+static int op_utimens(const char *path, const struct timespec tv[2])
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	errcode_t err;
+	ext2_ino_t ino;
+	struct ext2_inode_large inode;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	ret = check_inum_access(ctxt, fs, ino, W_OK);
+	if (ret)
+		goto out;
+
+	err = ext2fs_read_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+	EXT4_INODE_SET_XTIME(i_atime, tv, &inode);
+	EXT4_INODE_SET_XTIME(i_mtime, tv + 1, &inode);
+	ret = update_ctime(fs, ino, &inode);
+	if (ret)
+		goto out;
+
+	err = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return 0;
+}
+
+#ifdef SUPPORT_I_FLAGS
+static int ioctl_getflags(ext2_filsys fs, struct fuse2fs_file_handle *fh,
+			  void *data)
+{
+	errcode_t err;
+	struct ext2_inode_large inode;
+
+	err = ext2fs_read_inode_full(fs, fh->ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err)
+		return -EIO;
+
+	*(__u32 *)data = inode.i_flags & EXT2_FL_USER_VISIBLE;
+	return 0;
+}
+
+#define FUSE2FS_MODIFIABLE_IFLAGS \
+	(EXT2_IMMUTABLE_FL | EXT2_APPEND_FL | EXT2_NODUMP_FL | \
+	 EXT2_NOATIME_FL | EXT3_JOURNAL_DATA_FL | EXT2_DIRSYNC_FL | \
+	 EXT2_TOPDIR_FL)
+
+int ioctl_setflags(ext2_filsys fs, struct fuse2fs_file_handle *fh, void *data)
+{
+	errcode_t err;
+	struct ext2_inode_large inode;
+	int ret;
+	__u32 flags = *(__u32 *)data;
+	struct fuse_context *ctxt = fuse_get_context();
+
+	ret = check_flags_access(ctxt, fs, fh->ino, W_OK);
+	if (ret)
+		return ret;
+
+	err = ext2fs_read_inode_full(fs, fh->ino, (struct ext2_inode *)&inode,
+				     sizeof(inode));
+	if (err)
+		return -EIO;
+
+	if ((inode.i_flags ^ flags) & ~FUSE2FS_MODIFIABLE_IFLAGS)
+		return -EINVAL;
+
+	inode.i_flags = inode.i_flags & ~FUSE2FS_MODIFIABLE_IFLAGS |
+			flags & FUSE2FS_MODIFIABLE_IFLAGS;
+
+	err = ext2fs_write_inode_full(fs, fh->ino, (struct ext2_inode *)&inode,
+				      sizeof(inode));
+	if (err)
+		return -EIO;
+
+	return 0;
+}
+#endif /* SUPPORT_I_FLAGS */
+
+#if FUSE_VERSION >= FUSE_MAKE_VERSION(2, 8)
+static int op_ioctl(const char *path, int cmd, void *arg,
+		      struct fuse_file_info *fp, unsigned int flags, void *data)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	switch (cmd) {
+#ifdef SUPPORT_I_FLAGS
+	case EXT2_IOC_GETFLAGS:
+		ret = ioctl_getflags(fs, fh, data);
+		break;
+	case EXT2_IOC_SETFLAGS:
+		ret = ioctl_setflags(fs, fh, data);
+		break;
+#endif
+	default:
+		ret = -ENOTTY;
+	}
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+#endif /* FUSE 28 */
+
+static int op_bmap(const char *path, size_t blocksize, uint64_t *idx)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	ext2_ino_t ino;
+	errcode_t err;
+	int ret = 0;
+
+	pthread_mutex_lock(&ff->bfl);
+	err = ext2fs_namei(fs, EXT2_ROOT_INO, EXT2_ROOT_INO, path, &ino);
+	if (err) {
+		ret = translate_error(fs, 0, err);
+		goto out;
+	}
+
+	err = ext2fs_bmap2(fs, ino, NULL, NULL, 0, *idx, 0, (blk64_t *)idx);
+	if (err) {
+		ret = translate_error(fs, ino, err);
+		goto out;
+	}
+
+out:
+	pthread_mutex_unlock(&ff->bfl);
+	return ret;
+}
+
+#if FUSE_VERSION >= FUSE_MAKE_VERSION(2, 9)
+static int fallocate_helper(struct fuse_file_info *fp, int mode, off_t offset,
+			    off_t len)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	blk64_t blk, end, x;
+	__u64 fsize;
+	ext2_file_t efp;
+	struct ext2_inode_large inode;
+	errcode_t err;
+	int ret = 0;
+
+	/* Allocate a bunch of blocks */
+	end = (offset + len - 1) / fs->blocksize;
+	for (blk = offset / fs->blocksize; blk <= end; blk++) {
+		err = ext2fs_bmap2(fs, fh->ino, NULL, NULL, BMAP_ALLOC, blk,
+				   0, &x);
+		if (err)
+			return translate_error(fs, fh->ino, err);
+	}
+
+	/* Update i_size */
+	if (!(mode & FL_KEEP_SIZE_FLAG)) {
+		err = ext2fs_file_open(fs, fh->ino, fh->open_flags, &efp);
+		if (err)
+			return translate_error(fs, fh->ino, err);
+
+		err = ext2fs_file_get_lsize(efp, &fsize);
+		if (err) {
+			ret = translate_error(fs, fh->ino, err);
+			goto out_isize;
+		}
+		if (offset + len > fsize) {
+			fsize = offset + len;
+			err = ext2fs_file_set_size2(efp, fsize);
+			if (err) {
+				ret = translate_error(fs, fh->ino, err);
+				goto out_isize;
+			}
+		}
+
+out_isize:
+		err = ext2fs_file_close(efp);
+		if (ret)
+			return ret;
+		if (err)
+			return translate_error(fs, fh->ino, err);
+	}
+
+	return update_mtime(fs, fh->ino);
+}
+
+static int punch_helper(struct fuse_file_info *fp, int mode, off_t offset,
+			off_t len)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	struct fuse2fs_file_handle *fh = (struct fuse2fs_file_handle *)fp->fh;
+	blk64_t blk, start, end, x;
+	__u64 fsize;
+	ext2_file_t efp;
+	struct ext2_inode_large inode;
+	errcode_t err;
+	int ret = 0;
+
+	/* kernel ext4 punch requires this flag to be set */
+	if (!(mode & FL_KEEP_SIZE_FLAG))
+		return -EINVAL;
+
+	if (len < fs->blocksize)
+		return 0;
+
+	/* Punch out a bunch of blocks */
+	start = (offset + fs->blocksize - 1) / fs->blocksize;
+	end = (offset + len - fs->blocksize) / fs->blocksize;
+
+	if (start > end)
+		return -EINVAL;
+
+	err = ext2fs_punch(fs, fh->ino, NULL, NULL, start, end);
+	if (err)
+		return translate_error(fs, fh->ino, err);
+
+	return update_mtime(fs, fh->ino);
+}
+
+static int op_fallocate(const char *path, int mode, off_t offset, off_t len,
+			struct fuse_file_info *fp)
+{
+	struct fuse_context *ctxt = fuse_get_context();
+	struct fuse2fs *ff = (struct fuse2fs *)ctxt->private_data;
+	ext2_filsys fs = ff->fs;
+	int ret;
+
+	/* Catch unknown flags */
+	if (mode & ~(FL_PUNCH_HOLE_FLAG | FL_KEEP_SIZE_FLAG))
+		return -EINVAL;
+
+	pthread_mutex_lock(&ff->bfl);
+	if (!fs_writeable(fs)) {
+		ret = -EROFS;
+		goto out;
+	}
+	if (mode & FL_PUNCH_HOLE_FLAG)
+		ret = punch_helper(fp, mode, offset, len);
+	else
+		ret = fallocate_helper(fp, mode, offset, len);
+out:
+	pthread_mutex_unlock(&ff->bfl);
+
+	return ret;
+}
+#endif /* FUSE 29 */
+
+static struct fuse_operations fs_ops = {
+	.init = op_init,
+	.destroy = op_destroy,
+	.getattr = op_getattr,
+	.readlink = op_readlink,
+	.mknod = op_mknod,
+	.mkdir = op_mkdir,
+	.unlink = op_unlink,
+	.rmdir = op_rmdir,
+	.symlink = op_symlink,
+	.rename = op_rename,
+	.link = op_link,
+	.chmod = op_chmod,
+	.chown = op_chown,
+	.truncate = op_truncate,
+	.open = op_open,
+	.read = op_read,
+	.write = op_write,
+	.statfs = op_statfs,
+	.flush = op_flush,
+	.release = op_release,
+	.fsync = op_fsync,
+	.setxattr = op_setxattr,
+	.getxattr = op_getxattr,
+	.listxattr = op_listxattr,
+	.removexattr = op_removexattr,
+	.opendir = op_open,
+	.readdir = op_readdir,
+	.releasedir = op_release,
+	.fsyncdir = op_fsync,
+	.access = op_access,
+	.create = op_create,
+	.ftruncate = op_ftruncate,
+	.fgetattr = op_fgetattr,
+	.utimens = op_utimens,
+	.bmap = op_bmap,
+#ifdef SUPERFLUOUS
+	.lock = op_lock,
+	.poll = op_poll,
+#endif
+#if FUSE_VERSION >= FUSE_MAKE_VERSION(2, 8)
+	.ioctl = op_ioctl,
+	.flag_nullpath_ok = 1,
+#endif
+#if FUSE_VERSION >= FUSE_MAKE_VERSION(2, 9)
+	.flag_nopath = 1,
+	.fallocate = op_fallocate,
+#endif
+};
+
+static int get_random_bytes(void *p, size_t sz)
+{
+	int fd;
+	ssize_t r;
+
+	fd = open("/dev/random", O_RDONLY);
+	if (fd < 0) {
+		perror("/dev/random");
+		return 0;
+	}
+
+	r = read(fd, p, sz);
+
+	close(fd);
+	return r == sz;
+}
+
+int main(int argc, char *argv[])
+{
+	errcode_t err;
+	ext2_filsys fs;
+	char *tok, *arg, *logfile;
+	int i;
+	int readwrite = 1, panic_on_error = 0;
+	struct fuse2fs *ff;
+	char extra_args[BUFSIZ];
+	int ret = 0, flags = EXT2_FLAG_64BITS | EXT2_FLAG_EXCLUSIVE;
+
+	if (argc < 2) {
+		printf("Usage: %s dev mntpt [-o options] [fuse_args]\n",
+		       argv[0]);
+		return 1;
+	}
+
+	for (i = 1; i < argc - 1; i++) {
+		if (strcmp(argv[i], "-o"))
+			continue;
+		arg = argv[i + 1];
+		while ((tok = strtok(arg, ","))) {
+			arg = NULL;
+			if (!strcmp(tok, "ro"))
+				readwrite = 0;
+			else if (!strcmp(tok, "errors=panic"))
+				panic_on_error = 1;
+		}
+	}
+
+	if (!readwrite)
+		printf("Mounting read-only.\n");
+
+#ifdef ENABLE_NLS
+	setlocale(LC_MESSAGES, "");
+	setlocale(LC_CTYPE, "");
+	bindtextdomain(NLS_CAT_NAME, LOCALEDIR);
+	textdomain(NLS_CAT_NAME);
+	set_com_err_gettext(gettext);
+#endif
+	add_error_table(&et_ext2_error_table);
+
+	ff = calloc(1, sizeof(*ff));
+	if (!ff) {
+		perror("init");
+		return 1;
+	}
+	ff->panic_on_error = panic_on_error;
+
+	/* Set up error logging */
+	logfile = getenv("FUSE2FS_LOGFILE");
+	if (logfile) {
+		ff->err_fp = fopen(logfile, "a");
+		if (!ff->err_fp) {
+			perror(logfile);
+			goto out_nofs;
+		}
+	} else
+		ff->err_fp = stderr;
+
+	/* Start up the fs (while we still can use stdout) */
+	ret = 2;
+	if (readwrite)
+		flags |= EXT2_FLAG_RW;
+	err = ext2fs_open3(argv[1], NULL, flags, 0, 0, unix_io_manager, &fs);
+	if (err) {
+		printf("%s: %s.\n", argv[1], error_message(err));
+		printf("Please run e2fsck -fy %s.\n", argv[1]);
+		goto out_nofs;
+	}
+	ff->fs = fs;
+	fs->priv_data = ff;
+
+	ret = 3;
+	if (EXT2_HAS_INCOMPAT_FEATURE(fs->super,
+				      EXT3_FEATURE_INCOMPAT_RECOVER)) {
+		printf("Journal needs recovery; running `e2fsck -E "
+		       "journal_only' is required.\n");
+		goto out;
+	}
+
+	if (readwrite) {
+		if (EXT2_HAS_COMPAT_FEATURE(fs->super,
+					    EXT3_FEATURE_COMPAT_HAS_JOURNAL))
+			printf("Journal mode will not be used.\n");
+		err = ext2fs_read_inode_bitmap(fs);
+		if (err) {
+			translate_error(fs, 0, err);
+			goto out;
+		}
+		err = ext2fs_read_block_bitmap(fs);
+		if (err) {
+			translate_error(fs, 0, err);
+			goto out;
+		}
+	}
+
+	if (!(fs->super->s_state & EXT2_VALID_FS))
+		printf("Warning: Mounting unchecked fs, running e2fsck "
+		       "is recommended.\n");
+	if (fs->super->s_max_mnt_count > 0 &&
+	    fs->super->s_mnt_count >= fs->super->s_max_mnt_count)
+		printf("Warning: Maximal mount count reached, running "
+		       "e2fsck is recommended.\n");
+	if (fs->super->s_checkinterval > 0 &&
+	    fs->super->s_lastcheck + fs->super->s_checkinterval <= time(0))
+		printf("Warning: Check time reached; running e2fsck "
+		       "is recommended.\n");
+	if (fs->super->s_last_orphan)
+		printf("Orphans detected; running e2fsck is recommended.\n");
+
+	if (fs->super->s_state & EXT2_ERROR_FS) {
+		printf("Errors detected; running e2fsck is required.\n");
+		goto out;
+	}
+
+	/* Initialize generation counter */
+	get_random_bytes(&ff->next_generation, sizeof(unsigned int));
+
+	/* Stuff in some fuse parameters of our own */
+	snprintf(extra_args, BUFSIZ, "-okernel_cache,subtype=ext4,use_ino,"
+		 "fsname=%s", argv[1]);
+	argv[0] = argv[1];
+	argv[1] = argv[2];
+	argv[2] = extra_args;
+
+	pthread_mutex_init(&ff->bfl, NULL);
+	fuse_main(argc, argv, &fs_ops, ff);
+	pthread_mutex_destroy(&ff->bfl);
+
+	ret = 0;
+out:
+	err = ext2fs_close(fs);
+	if (err)
+		ret = translate_error(fs, 0, err);
+out_nofs:
+	free(ff);
+
+	return ret;
+}
+
+static int __translate_error(ext2_filsys fs, errcode_t err, ext2_ino_t ino,
+			     const char *file, int line)
+{
+	struct timespec now;
+	int ret;
+	struct fuse2fs *ff = fs->priv_data;
+	int is_err = 0;
+
+	/* Translate ext2 error to unix error code */
+	switch (err) {
+	case EXT2_ET_NO_MEMORY:
+	case EXT2_ET_TDB_ERR_OOM:
+		ret = -ENOMEM;
+		break;
+	case EXT2_ET_INVALID_ARGUMENT:
+	case EXT2_ET_LLSEEK_FAILED:
+		ret = -EINVAL;
+		break;
+	case EXT2_ET_NO_DIRECTORY:
+		ret = -ENOTDIR;
+		break;
+	case EXT2_ET_FILE_NOT_FOUND:
+		ret = -ENOENT;
+		break;
+	case EXT2_ET_DIR_NO_SPACE:
+		is_err = 1;
+	case EXT2_ET_TOOSMALL:
+	case EXT2_ET_BLOCK_ALLOC_FAIL:
+	case EXT2_ET_INODE_ALLOC_FAIL:
+	case EXT2_ET_EA_NO_SPACE:
+		ret = -ENOSPC;
+		break;
+	case EXT2_ET_SYMLINK_LOOP:
+		ret = -EMLINK;
+		break;
+	case EXT2_ET_FILE_TOO_BIG:
+		ret = -EFBIG;
+		break;
+	case EXT2_ET_TDB_ERR_EXISTS:
+	case EXT2_ET_FILE_EXISTS:
+		ret = -EEXIST;
+		break;
+	case EXT2_ET_MMP_FAILED:
+	case EXT2_ET_MMP_FSCK_ON:
+		ret = -EBUSY;
+		break;
+	case EXT2_ET_EA_KEY_NOT_FOUND:
+		ret = -ENODATA;
+		break;
+	default:
+		is_err = 1;
+		ret = -EIO;
+		break;
+	}
+
+	if (!is_err)
+		return ret;
+
+	if (ino)
+		fprintf(ff->err_fp, "FUSE2FS (%s): %s (inode #%d) at %s:%d.\n",
+			fs && fs->device_name ? fs->device_name : "???",
+			error_message(err), ino, file, line);
+	else
+		fprintf(ff->err_fp, "FUSE2FS (%s): %s at %s:%d.\n",
+			fs && fs->device_name ? fs->device_name : "???",
+			error_message(err), file, line);
+	fflush(ff->err_fp);
+
+	/* Make a note in the error log */
+	get_now(&now);
+	fs->super->s_last_error_time = now.tv_sec;
+	fs->super->s_last_error_ino = ino;
+	fs->super->s_last_error_line = line;
+	fs->super->s_last_error_block = 0;
+	strncpy(fs->super->s_last_error_func, file,
+		sizeof(fs->super->s_last_error_func));
+	if (fs->super->s_first_error_time == 0) {
+		fs->super->s_first_error_time = now.tv_sec;
+		fs->super->s_first_error_ino = ino;
+		fs->super->s_first_error_line = line;
+		fs->super->s_first_error_block = 0;
+		strncpy(fs->super->s_first_error_func, file,
+			sizeof(fs->super->s_first_error_func));
+	}
+
+	fs->super->s_error_count++;
+	ext2fs_mark_super_dirty(fs);
+	ext2fs_flush(fs);
+	if (ff->panic_on_error)
+		abort();
+
+	return ret;
+}