[PATCH 1/1] ext4: Remove empty index blocks when punching holes

[PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Allison Henderson]

When punching holes, it is possible that all the entries of an
index block may be removed.  These empty index blocks need to
be removed once punch hole completes.  This patch adds some extra
logic to the punch hole code to remove empty index blocks.

Signed-off-by: Allison Henderson <achender@linux.vnet.ibm.com>
---
:100644 100644 ad39627... 6bb2f55... M	fs/ext4/extents.c
 fs/ext4/extents.c |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index ad39627..6bb2f55 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -3709,7 +3709,7 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
 	struct ext4_extent newex, *ex, *ex2;
 	struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
 	ext4_fsblk_t newblock = 0;
-	int free_on_err = 0, err = 0, depth, ret;
+	int free_on_err = 0, err = 0, depth, ret, i;
 	unsigned int allocated = 0, offset = 0;
 	unsigned int allocated_clusters = 0, reserved_clusters = 0;
 	unsigned int punched_out = 0;
@@ -3880,6 +3880,21 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
 			err = ext4_ext_rm_leaf(handle, inode, path,
 					       &partial_cluster, map->m_lblk,
 					       map->m_lblk + punched_out);
+			if (err)
+				goto out2;
+
+			/*
+			 * Walk back up the path and remove
+			 * empty index blocks
+			 */
+			for (i = depth - 1; i > 0; i--) {
+				if (path[i].p_hdr->eh_entries == 0) {
+					err = ext4_ext_rm_idx(handle, inode,
+								path + i);
+					if (err)
+						goto out2;
+				}
+			}
 
 			if (!err && path->p_hdr->eh_entries == 0) {
 				/*
-- 
1.7.1

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Lukas Czerner]

On Wed, 29 Feb 2012, Allison Henderson wrote:

> When punching holes, it is possible that all the entries of an
> index block may be removed.  These empty index blocks need to
> be removed once punch hole completes.  This patch adds some extra
> logic to the punch hole code to remove empty index blocks.

Hi Allison,

I have not looked at the patch, but just to let you know. I am working
on a patch to reorganize the punch hole support a bit in order to fix
a bug I found but more importantly to make it faster, better
understandable and to reuse more of an existing code paths.

I am cleaning up the patches and testing them right now, so hopefully by
the end of the week I'll send those out.

Thanks!
-Lukas

> 
> Signed-off-by: Allison Henderson <achender@linux.vnet.ibm.com>
> ---
> :100644 100644 ad39627... 6bb2f55... M	fs/ext4/extents.c
>  fs/ext4/extents.c |   17 ++++++++++++++++-
>  1 files changed, 16 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
> index ad39627..6bb2f55 100644
> --- a/fs/ext4/extents.c
> +++ b/fs/ext4/extents.c
> @@ -3709,7 +3709,7 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
>  	struct ext4_extent newex, *ex, *ex2;
>  	struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
>  	ext4_fsblk_t newblock = 0;
> -	int free_on_err = 0, err = 0, depth, ret;
> +	int free_on_err = 0, err = 0, depth, ret, i;
>  	unsigned int allocated = 0, offset = 0;
>  	unsigned int allocated_clusters = 0, reserved_clusters = 0;
>  	unsigned int punched_out = 0;
> @@ -3880,6 +3880,21 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
>  			err = ext4_ext_rm_leaf(handle, inode, path,
>  					       &partial_cluster, map->m_lblk,
>  					       map->m_lblk + punched_out);
> +			if (err)
> +				goto out2;
> +
> +			/*
> +			 * Walk back up the path and remove
> +			 * empty index blocks
> +			 */
> +			for (i = depth - 1; i > 0; i--) {
> +				if (path[i].p_hdr->eh_entries == 0) {
> +					err = ext4_ext_rm_idx(handle, inode,
> +								path + i);
> +					if (err)
> +						goto out2;
> +				}
> +			}
>  
>  			if (!err && path->p_hdr->eh_entries == 0) {
>  				/*
> 

-- 

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Allison Henderson]

On 02/29/2012 12:44 AM, Lukas Czerner wrote:
> On Wed, 29 Feb 2012, Allison Henderson wrote:
>
>> When punching holes, it is possible that all the entries of an
>> index block may be removed.  These empty index blocks need to
>> be removed once punch hole completes.  This patch adds some extra
>> logic to the punch hole code to remove empty index blocks.
>
> Hi Allison,
>
> I have not looked at the patch, but just to let you know. I am working
> on a patch to reorganize the punch hole support a bit in order to fix
> a bug I found but more importantly to make it faster, better
> understandable and to reuse more of an existing code paths.
>
> I am cleaning up the patches and testing them right now, so hopefully by
> the end of the week I'll send those out.
>
> Thanks!
> -Lukas
>

Hi Lukas,

Alrighty then, sounds good.  This patch seemed to correct the loop 
discard bug we saw earlier for me.  So please make sure you have some 
similar fix in your new patch.  Basically we had some extra index blocks 
hanging around when the depth was greater than one, so the patch just 
adds some code to free up those index blocks in the path if they are 
empty.  Hope this helps!  Thx!  :)

Allison Henderson

>>
>> Signed-off-by: Allison Henderson<achender@linux.vnet.ibm.com>
>> ---
>> :100644 100644 ad39627... 6bb2f55... M	fs/ext4/extents.c
>>   fs/ext4/extents.c |   17 ++++++++++++++++-
>>   1 files changed, 16 insertions(+), 1 deletions(-)
>>
>> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
>> index ad39627..6bb2f55 100644
>> --- a/fs/ext4/extents.c
>> +++ b/fs/ext4/extents.c
>> @@ -3709,7 +3709,7 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
>>   	struct ext4_extent newex, *ex, *ex2;
>>   	struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
>>   	ext4_fsblk_t newblock = 0;
>> -	int free_on_err = 0, err = 0, depth, ret;
>> +	int free_on_err = 0, err = 0, depth, ret, i;
>>   	unsigned int allocated = 0, offset = 0;
>>   	unsigned int allocated_clusters = 0, reserved_clusters = 0;
>>   	unsigned int punched_out = 0;
>> @@ -3880,6 +3880,21 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
>>   			err = ext4_ext_rm_leaf(handle, inode, path,
>>   					&partial_cluster, map->m_lblk,
>>   					       map->m_lblk + punched_out);
>> +			if (err)
>> +				goto out2;
>> +
>> +			/*
>> +			 * Walk back up the path and remove
>> +			 * empty index blocks
>> +			 */
>> +			for (i = depth - 1; i>  0; i--) {
>> +				if (path[i].p_hdr->eh_entries == 0) {
>> +					err = ext4_ext_rm_idx(handle, inode,
>> +								path + i);
>> +					if (err)
>> +						goto out2;
>> +				}
>> +			}
>>
>>   			if (!err&&  path->p_hdr->eh_entries == 0) {
>>   				/*
>>
>

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Lukas Czerner]

On Wed, 29 Feb 2012, Allison Henderson wrote:

> On 02/29/2012 12:44 AM, Lukas Czerner wrote:
> > On Wed, 29 Feb 2012, Allison Henderson wrote:
> > 
> > > When punching holes, it is possible that all the entries of an
> > > index block may be removed.  These empty index blocks need to
> > > be removed once punch hole completes.  This patch adds some extra
> > > logic to the punch hole code to remove empty index blocks.
> > 
> > Hi Allison,
> > 
> > I have not looked at the patch, but just to let you know. I am working
> > on a patch to reorganize the punch hole support a bit in order to fix
> > a bug I found but more importantly to make it faster, better
> > understandable and to reuse more of an existing code paths.
> > 
> > I am cleaning up the patches and testing them right now, so hopefully by
> > the end of the week I'll send those out.
> > 
> > Thanks!
> > -Lukas
> > 
> 
> Hi Lukas,
> 
> Alrighty then, sounds good.  This patch seemed to correct the loop discard bug
> we saw earlier for me.  So please make sure you have some similar fix in your
> new patch.  Basically we had some extra index blocks hanging around when the
> depth was greater than one, so the patch just adds some code to free up those
> index blocks in the path if they are empty.  Hope this helps!  Thx!  :)

Thanks Allison,

Actually I started to modifying the punch hole code in order to find
that bug and this problem does go away with my pathes. I'll CC
you on the patches when I send them out.

Thanks!
-Lukas

> 
> Allison Henderson
> 
> > > 
> > > Signed-off-by: Allison Henderson<achender@linux.vnet.ibm.com>
> > > ---
> > > :100644 100644 ad39627... 6bb2f55... M	fs/ext4/extents.c
> > >   fs/ext4/extents.c |   17 ++++++++++++++++-
> > >   1 files changed, 16 insertions(+), 1 deletions(-)
> > > 
> > > diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
> > > index ad39627..6bb2f55 100644
> > > --- a/fs/ext4/extents.c
> > > +++ b/fs/ext4/extents.c
> > > @@ -3709,7 +3709,7 @@ int ext4_ext_map_blocks(handle_t *handle, struct
> > > inode *inode,
> > >   	struct ext4_extent newex, *ex, *ex2;
> > >   	struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
> > >   	ext4_fsblk_t newblock = 0;
> > > -	int free_on_err = 0, err = 0, depth, ret;
> > > +	int free_on_err = 0, err = 0, depth, ret, i;
> > >   	unsigned int allocated = 0, offset = 0;
> > >   	unsigned int allocated_clusters = 0, reserved_clusters = 0;
> > >   	unsigned int punched_out = 0;
> > > @@ -3880,6 +3880,21 @@ int ext4_ext_map_blocks(handle_t *handle, struct
> > > inode *inode,
> > >   			err = ext4_ext_rm_leaf(handle, inode, path,
> > >   					&partial_cluster, map->m_lblk,
> > >   					       map->m_lblk + punched_out);
> > > +			if (err)
> > > +				goto out2;
> > > +
> > > +			/*
> > > +			 * Walk back up the path and remove
> > > +			 * empty index blocks
> > > +			 */
> > > +			for (i = depth - 1; i>  0; i--) {
> > > +				if (path[i].p_hdr->eh_entries == 0) {
> > > +					err = ext4_ext_rm_idx(handle, inode,
> > > +								path + i);
> > > +					if (err)
> > > +						goto out2;
> > > +				}
> > > +			}
> > > 
> > >   			if (!err&&  path->p_hdr->eh_entries == 0) {
> > >   				/*
> > > 
> > 
> 
> 

-- 

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Ted Ts'o]

On Wed, Feb 29, 2012 at 09:18:54AM +0100, Lukas Czerner wrote:
> > 
> > Alrighty then, sounds good.  This patch seemed to correct the loop discard bug
> > we saw earlier for me.  So please make sure you have some similar fix in your
> > new patch.  Basically we had some extra index blocks hanging around when the
> > depth was greater than one, so the patch just adds some code to free up those
> > index blocks in the path if they are empty.  Hope this helps!  Thx!  :)
> 
> Actually I started to modifying the punch hole code in order to find
> that bug and this problem does go away with my pathes. I'll CC
> you on the patches when I send them out.

OK, so my understanding is that Lukas's patches solve the same problem
as Allison's, and those are the patches I should review and integrate?

   	      	  	    		- Ted

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Lukas Czerner]

On Mon, 5 Mar 2012, Ted Ts'o wrote:

> On Wed, Feb 29, 2012 at 09:18:54AM +0100, Lukas Czerner wrote:
> > > 
> > > Alrighty then, sounds good.  This patch seemed to correct the loop discard bug
> > > we saw earlier for me.  So please make sure you have some similar fix in your
> > > new patch.  Basically we had some extra index blocks hanging around when the
> > > depth was greater than one, so the patch just adds some code to free up those
> > > index blocks in the path if they are empty.  Hope this helps!  Thx!  :)
> > 
> > Actually I started to modifying the punch hole code in order to find
> > that bug and this problem does go away with my pathes. I'll CC
> > you on the patches when I send them out.
> 
> OK, so my understanding is that Lukas's patches solve the same problem
> as Allison's, and those are the patches I should review and integrate?
> 
>    	      	  	    		- Ted
> 

Hi Ted,

you're right my patches solves this problem (as I wrote in the commit
description) just because we now use a different code paths, which do
not have this problem.

Thanks!
-Lukas

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Ted Ts'o]

On Tue, Mar 06, 2012 at 08:11:37AM +0100, Lukas Czerner wrote:
> 
> you're right my patches solves this problem (as I wrote in the commit
> description) just because we now use a different code paths, which do
> not have this problem.

Ok, thanks.  I'll look at them before the end of this week.

Just to be clear, have the problematic code paths been removed in your
patches, or are they not just being used in the problem scenario?
Basically, is there any other time where we might need the additional
logic which Allison added?

					- Ted

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Allison Henderson]

On 03/06/2012 11:37 AM, Allison Henderson wrote:
> On 03/06/2012 09:44 AM, Ted Ts'o wrote:
>> On Tue, Mar 06, 2012 at 08:11:37AM +0100, Lukas Czerner wrote:
>>>
>>> you're right my patches solves this problem (as I wrote in the commit
>>> description) just because we now use a different code paths, which do
>>> not have this problem.
>>
>> Ok, thanks. I'll look at them before the end of this week.
>>
>> Just to be clear, have the problematic code paths been removed in your
>> patches, or are they not just being used in the problem scenario?
>> Basically, is there any other time where we might need the additional
>> logic which Allison added?
>>
>> - Ted
>>
>
> Hi Ted,
>
> I think we will be ok with out this patch if we pick up Lukas's patches.
> Since the new implementation is seated inside ext4_ext_remove_space,
> Lukas can take advantage of the existing code there.
>
> In the current solution, we are seated inside map blocks, and then call
> ext4_ext_rm_leaf from there. The bug in the current solution was that we
> needed to free index blocks in the path to the extent we just removed,
> but ext4_ext_remove_space will do this as it walks over the tree.
>
> There are some things in the new implementation that Lukas and I are
> looking at, but once we get it straightened out, I think it will be ok
> to let this patch go. Thx!
>
> Allison Henderson
>

forgot to reply to all.  Resending to keep everyone posted :)

Re: [PATCH 1/1] ext4: Remove empty index blocks when punching holes

[Lukas Czerner]

On Tue, 6 Mar 2012, Allison Henderson wrote:

> On 03/06/2012 11:37 AM, Allison Henderson wrote:
> > On 03/06/2012 09:44 AM, Ted Ts'o wrote:
> > > On Tue, Mar 06, 2012 at 08:11:37AM +0100, Lukas Czerner wrote:
> > > > 
> > > > you're right my patches solves this problem (as I wrote in the commit
> > > > description) just because we now use a different code paths, which do
> > > > not have this problem.
> > > 
> > > Ok, thanks. I'll look at them before the end of this week.
> > > 
> > > Just to be clear, have the problematic code paths been removed in your
> > > patches, or are they not just being used in the problem scenario?
> > > Basically, is there any other time where we might need the additional
> > > logic which Allison added?
> > > 
> > > - Ted
> > > 
> > 
> > Hi Ted,
> > 
> > I think we will be ok with out this patch if we pick up Lukas's patches.
> > Since the new implementation is seated inside ext4_ext_remove_space,
> > Lukas can take advantage of the existing code there.
> > 
> > In the current solution, we are seated inside map blocks, and then call
> > ext4_ext_rm_leaf from there. The bug in the current solution was that we
> > needed to free index blocks in the path to the extent we just removed,
> > but ext4_ext_remove_space will do this as it walks over the tree.
> > 
> > There are some things in the new implementation that Lukas and I are
> > looking at, but once we get it straightened out, I think it will be ok
> > to let this patch go. Thx!
> > 
> > Allison Henderson
> > 
> 
> forgot to reply to all.  Resending to keep everyone posted :)
> 

I second that. The previous codepaths are removed from map blocks, so
this bug is not there anymore.

Thanks!
-Lukas