Re: [patch 3/4] jffs2: avoid soft-lockup in jffs2_reserve_space_gc()

Re: [patch 3/4] jffs2: avoid soft-lockup in jffs2_reserve_space_gc()

[Brian Norris]

+ linux-mtd

Hi Li,

On Wed, Feb 12, 2014 at 12:44:56PM -0800, Andrew Morton wrote:
> From: Li Zefan <lizefan@huawei.com>
> Subject: jffs2: avoid soft-lockup in jffs2_reserve_space_gc()
> 
> We triggered soft-lockup under stress test on 2.6.34 kernel.
> 
> BUG: soft lockup - CPU#1 stuck for 60009ms! [lockf2.test:14488]
> ...
> [<bf09a4d4>] (jffs2_do_reserve_space+0x420/0x440 [jffs2])
> [<bf09a528>] (jffs2_reserve_space_gc+0x34/0x78 [jffs2])
> [<bf0a1350>] (jffs2_garbage_collect_dnode.isra.3+0x264/0x478 [jffs2])
> [<bf0a2078>] (jffs2_garbage_collect_pass+0x9c0/0xe4c [jffs2])
> [<bf09a670>] (jffs2_reserve_space+0x104/0x2a8 [jffs2])
> [<bf09dc48>] (jffs2_write_inode_range+0x5c/0x4d4 [jffs2])
> [<bf097d8c>] (jffs2_write_end+0x198/0x2c0 [jffs2])
> [<c00e00a4>] (generic_file_buffered_write+0x158/0x200)
> [<c00e14f4>] (__generic_file_aio_write+0x3a4/0x414)
> [<c00e15c0>] (generic_file_aio_write+0x5c/0xbc)
> [<c012334c>] (do_sync_write+0x98/0xd4)
> [<c0123a84>] (vfs_write+0xa8/0x150)
> [<c0123d74>] (sys_write+0x3c/0xc0)]
> 
> Fix this by adding a cond_resched() in the while loop.

This patch looks good.

> [akpm@linux-foundation.org: don't initialize `ret']
> Signed-off-by: Li Zefan <lizefan@huawei.com>
> Cc: David Woodhouse <dwmw2@infradead.org>
> Cc: Brian Norris <computersforpeace@gmail.com>
> Cc: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  fs/jffs2/nodemgmt.c |   13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff -puN fs/jffs2/nodemgmt.c~jffs2-avoid-soft-lockup-in-jffs2_reserve_space_gc fs/jffs2/nodemgmt.c
> --- a/fs/jffs2/nodemgmt.c~jffs2-avoid-soft-lockup-in-jffs2_reserve_space_gc
> +++ a/fs/jffs2/nodemgmt.c
> @@ -211,20 +211,25 @@ out:
>  int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize,
>  			   uint32_t *len, uint32_t sumsize)
>  {
> -	int ret = -EAGAIN;
> +	int ret;
>  	minsize = PAD(minsize);
>  
>  	jffs2_dbg(1, "%s(): Requested 0x%x bytes\n", __func__, minsize);
>  
> -	spin_lock(&c->erase_completion_lock);
> -	while(ret == -EAGAIN) {
> +	while (true) {
> +		spin_lock(&c->erase_completion_lock);
>  		ret = jffs2_do_reserve_space(c, minsize, len, sumsize);
>  		if (ret) {
>  			jffs2_dbg(1, "%s(): looping, ret is %d\n",
>  				  __func__, ret);
>  		}
> +		spin_unlock(&c->erase_completion_lock);
> +
> +		if (ret == -EAGAIN)
> +			cond_resched();

Just curious: would this be a place to use cond_resched_lock(), and keep
the lock outside the loop?

> +		else
> +			break;
>  	}
> -	spin_unlock(&c->erase_completion_lock);
>  	if (!ret)
>  		ret = jffs2_prealloc_raw_node_refs(c, c->nextblock, 1);
>  

Anyway, pushed to l2-mtd.git.

Thanks,
Brian

Re: [patch 3/4] jffs2: avoid soft-lockup in jffs2_reserve_space_gc()

[Li Zefan]

>>  fs/jffs2/nodemgmt.c |   13 +++++++++----
>>  1 file changed, 9 insertions(+), 4 deletions(-)
>>
>> diff -puN fs/jffs2/nodemgmt.c~jffs2-avoid-soft-lockup-in-jffs2_reserve_space_gc fs/jffs2/nodemgmt.c
>> --- a/fs/jffs2/nodemgmt.c~jffs2-avoid-soft-lockup-in-jffs2_reserve_space_gc
>> +++ a/fs/jffs2/nodemgmt.c
>> @@ -211,20 +211,25 @@ out:
>>  int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize,
>>  			   uint32_t *len, uint32_t sumsize)
>>  {
>> -	int ret = -EAGAIN;
>> +	int ret;
>>  	minsize = PAD(minsize);
>>  
>>  	jffs2_dbg(1, "%s(): Requested 0x%x bytes\n", __func__, minsize);
>>  
>> -	spin_lock(&c->erase_completion_lock);
>> -	while(ret == -EAGAIN) {
>> +	while (true) {
>> +		spin_lock(&c->erase_completion_lock);
>>  		ret = jffs2_do_reserve_space(c, minsize, len, sumsize);
>>  		if (ret) {
>>  			jffs2_dbg(1, "%s(): looping, ret is %d\n",
>>  				  __func__, ret);
>>  		}
>> +		spin_unlock(&c->erase_completion_lock);
>> +
>> +		if (ret == -EAGAIN)
>> +			cond_resched();
> 
> Just curious: would this be a place to use cond_resched_lock(), and keep
> the lock outside the loop?
> 

Yeah, cond_resched_lock() can be used too, but doesn't make much difference.
This patch adds 5 LOC, and using cond_resched_lock() will also add 5 LOC.

As you've already merged it, I think it isn't worth borthing to make this change.

diff --git a/fs/jffs2/nodemgmt.c b/fs/jffs2/nodemgmt.c
index 0331072..36e9538 100644
--- a/fs/jffs2/nodemgmt.c
+++ b/fs/jffs2/nodemgmt.c
@@ -223,6 +223,11 @@ int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t mi
nsize,
                        jffs2_dbg(1, "%s(): looping, ret is %d\n",
                                  __func__, ret);
                }
+
+               if (ret == -EAGAIN)
+                       cond_resched_lock(&c->erease_completion_lock);
+               else
+                       break;
        }
        spin_unlock(&c->erase_completion_lock);


>> +		else
>> +			break;
>>  	}
>> -	spin_unlock(&c->erase_completion_lock);
>>  	if (!ret)
>>  		ret = jffs2_prealloc_raw_node_refs(c, c->nextblock, 1);
>>  
> 
> Anyway, pushed to l2-mtd.git.
> 

Thanks!