[PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

[PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

[Vyacheslav Dubeyko]

Hi Ryusuke,

I changed place of increment operation of segbuf->sb_nbio in
nilfs_segbuf_submit_bio() and adding decrement operation for
BIO_EOPNOTSUPP case in second version of the patch.

Could you share your opinion about this version of the patch?

With the best regards,
Vyacheslav Dubeyko.
---
From: Vyacheslav Dubeyko <slava@dubeyko.com>
Subject: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

This patch removes double call of bio_put() in nilfs_end_bio_write()
for the case of BIO_EOPNOTSUPP error detection. The issue was found
by Dan Carpenter and he suggests first version of the fix too.

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Vyacheslav Dubeyko <slava@dubeyko.com>
CC: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
---
 fs/nilfs2/segbuf.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c
index dc9a913..0b09ec9 100644
--- a/fs/nilfs2/segbuf.c
+++ b/fs/nilfs2/segbuf.c
@@ -345,8 +345,7 @@ static void nilfs_end_bio_write(struct bio *bio, int err)
 
 	if (err == -EOPNOTSUPP) {
 		set_bit(BIO_EOPNOTSUPP, &bio->bi_flags);
-		bio_put(bio);
-		/* to be detected by submit_seg_bio() */
+		/* to be detected by nilfs_segbuf_submit_bio() */
 	}
 
 	if (!uptodate)
@@ -377,12 +376,13 @@ static int nilfs_segbuf_submit_bio(struct nilfs_segment_buffer *segbuf,
 	bio->bi_private = segbuf;
 	bio_get(bio);
 	submit_bio(mode, bio);
+	segbuf->sb_nbio++;
 	if (bio_flagged(bio, BIO_EOPNOTSUPP)) {
+		segbuf->sb_nbio--;
 		bio_put(bio);
 		err = -EOPNOTSUPP;
 		goto failed;
 	}
-	segbuf->sb_nbio++;
 	bio_put(bio);
 
 	wi->bio = NULL;
-- 
1.7.9.5

Re: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

[Ryusuke Konishi]

Hi Vyacheslav,
On Fri, 26 Jul 2013 17:59:14 +0400, Vyacheslav Dubeyko wrote:
> Hi Ryusuke,
> 
> I changed place of increment operation of segbuf->sb_nbio in
> nilfs_segbuf_submit_bio() and adding decrement operation for
> BIO_EOPNOTSUPP case in second version of the patch.
> 
> Could you share your opinion about this version of the patch?
> 
> With the best regards,
> Vyacheslav Dubeyko.
> ---
> From: Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org>
> Subject: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error
> 
> This patch removes double call of bio_put() in nilfs_end_bio_write()
> for the case of BIO_EOPNOTSUPP error detection. The issue was found
> by Dan Carpenter and he suggests first version of the fix too.
> 
> Reported-by: Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org>
> CC: Ryusuke Konishi <konishi.ryusuke-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org>
> ---
>  fs/nilfs2/segbuf.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c
> index dc9a913..0b09ec9 100644
> --- a/fs/nilfs2/segbuf.c
> +++ b/fs/nilfs2/segbuf.c
> @@ -345,8 +345,7 @@ static void nilfs_end_bio_write(struct bio *bio, int err)
>  
>  	if (err == -EOPNOTSUPP) {
>  		set_bit(BIO_EOPNOTSUPP, &bio->bi_flags);
> -		bio_put(bio);
> -		/* to be detected by submit_seg_bio() */
> +		/* to be detected by nilfs_segbuf_submit_bio() */
>  	}
>  
>  	if (!uptodate)
> @@ -377,12 +376,13 @@ static int nilfs_segbuf_submit_bio(struct nilfs_segment_buffer *segbuf,
>  	bio->bi_private = segbuf;
>  	bio_get(bio);
>  	submit_bio(mode, bio);
> +	segbuf->sb_nbio++;
>  	if (bio_flagged(bio, BIO_EOPNOTSUPP)) {

> +		segbuf->sb_nbio--;

This decrement looks wrong.

Otherwise, your change of nilfs_segbuf_submit_bio() is just a
equivalent transformation and doesn't fix problem, that is, a mismatch
of the number of calls between complete() and wait_for_completion().

In your patch, nilfs_end_bio_write() function calls the complete()
routine as before even if it received an EOPNOTSUPP error.

In that case, segbuf->sb_nbio must be incremented to call
wait_for_completion() the right number of times.

Note that wait_for_completion() is called based on the count of
segbuf->sb_nbio even if nilfs_segbuf_submit_bio() returns an error.
This is performed through the following path:

            nilfs_segbuf_submit_bio
          nilfs_segbuf_submit_bh
        nilfs_segbuf_write
      nilfs_write_logs
    nilfs_segctor_write
  nilfs_segctor_do_construct
    nilfs_segctor_abort_construction
      nilfs_wait_on_logs
        nilfs_segbuf_wait
          wait_for_completion
            (repeated for the number of times of segbuf->sb_nbio)


If you think this is a separate problem, then it should be fixed in
another patch and nilfs_segbuf_submit_bio() should not be touched in
this patch.


Regards,
Ryusuke Konishi


>  		bio_put(bio);
>  		err = -EOPNOTSUPP;
>  		goto failed;
>  	}
> -	segbuf->sb_nbio++;
>  	bio_put(bio);
>  
>  	wi->bio = NULL;
> -- 
> 1.7.9.5
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

[Vyacheslav Dubeyko]

Hi Ryusuke,

On Sat, 2013-07-27 at 12:02 +0900, Ryusuke Konishi wrote:

[snip]
> 
> This decrement looks wrong.
> 
> Otherwise, your change of nilfs_segbuf_submit_bio() is just a
> equivalent transformation and doesn't fix problem, that is, a mismatch
> of the number of calls between complete() and wait_for_completion().
> 
> In your patch, nilfs_end_bio_write() function calls the complete()
> routine as before even if it received an EOPNOTSUPP error.
> 
> In that case, segbuf->sb_nbio must be incremented to call
> wait_for_completion() the right number of times.
> 
> Note that wait_for_completion() is called based on the count of
> segbuf->sb_nbio even if nilfs_segbuf_submit_bio() returns an error.
> This is performed through the following path:
> 
>             nilfs_segbuf_submit_bio
>           nilfs_segbuf_submit_bh
>         nilfs_segbuf_write
>       nilfs_write_logs
>     nilfs_segctor_write
>   nilfs_segctor_do_construct
>     nilfs_segctor_abort_construction
>       nilfs_wait_on_logs
>         nilfs_segbuf_wait
>           wait_for_completion
>             (repeated for the number of times of segbuf->sb_nbio)
> 
> 
> If you think this is a separate problem, then it should be fixed in
> another patch and nilfs_segbuf_submit_bio() should not be touched in
> this patch.
> 

Sorry for delay with working on this patch. Now I am investigating
the issue (Kernel Bug: unable to handle kernel paging request) that it
was reported by Jérôme Poulin <jeromepoulin@gmail.com>. I suspect that
the reported issue is related to this patch. So, I feel a necessity to
investigate the issue more deeply before continuation to work on this
patch.

With the best regards,
Vyacheslav Dubeyko.


--
To unsubscribe from this list: send the line "unsubscribe kernel-janitors" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

[Vyacheslav Dubeyko]

Hi Ryusuke,

On Sat, 2013-07-27 at 12:02 +0900, Ryusuke Konishi wrote:

> > From: Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org>
> > Subject: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error
> > 
> > This patch removes double call of bio_put() in nilfs_end_bio_write()
> > for the case of BIO_EOPNOTSUPP error detection. The issue was found
> > by Dan Carpenter and he suggests first version of the fix too.
> > 
> > Reported-by: Dan Carpenter <dan.carpenter-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> > Signed-off-by: Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org>
> > CC: Ryusuke Konishi <konishi.ryusuke-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org>
> > ---
> >  fs/nilfs2/segbuf.c |    6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c
> > index dc9a913..0b09ec9 100644
> > --- a/fs/nilfs2/segbuf.c
> > +++ b/fs/nilfs2/segbuf.c
> > @@ -345,8 +345,7 @@ static void nilfs_end_bio_write(struct bio *bio, int err)
> >  
> >  	if (err == -EOPNOTSUPP) {
> >  		set_bit(BIO_EOPNOTSUPP, &bio->bi_flags);
> > -		bio_put(bio);
> > -		/* to be detected by submit_seg_bio() */
> > +		/* to be detected by nilfs_segbuf_submit_bio() */
> >  	}
> >  
> >  	if (!uptodate)
> > @@ -377,12 +376,13 @@ static int nilfs_segbuf_submit_bio(struct nilfs_segment_buffer *segbuf,
> >  	bio->bi_private = segbuf;
> >  	bio_get(bio);
> >  	submit_bio(mode, bio);
> > +	segbuf->sb_nbio++;
> >  	if (bio_flagged(bio, BIO_EOPNOTSUPP)) {
> 
> > +		segbuf->sb_nbio--;
> 
> This decrement looks wrong.
> 

I tried to understand your vision but I am thinking that my patch is
correct. Maybe, I missed something in my considerations. I describe my
vision in more details.

> Otherwise, your change of nilfs_segbuf_submit_bio() is just a
> equivalent transformation and doesn't fix problem, that is, a mismatch
> of the number of calls between complete() and wait_for_completion().
> 
> In your patch, nilfs_end_bio_write() function calls the complete()
> routine as before even if it received an EOPNOTSUPP error.
> 
> In that case, segbuf->sb_nbio must be incremented to call
> wait_for_completion() the right number of times.
> 

I moved incrementing of sb_nbio nearly after submit_bio(mode, bio). So,
firstly, we call submit_bio(mode, bio) and segbuf->sb_nbio++. Then,
because of asynchronous nature of nilfs_end_bio_write() call, we have
two alternatives: (1) nilfs_end_bio_write() detects that BIO_EOPNOTSUPP
flag is set; (2) BIO_EOPNOTSUPP flag isn't set because bio is not
processed yet or we haven't error.

If nilfs_end_bio_write() method would detect -EOPNOTSUPP error then it
set BIO_EOPNOTSUPP flag, increment sb_err field and to call complete(). 

So, if we detect that BIO_EOPNOTSUPP flag is set in
nilfs_segbuf_submit_bio() then we decrement sb_nbio and to return from
nilfs_segbuf_submit_bio() with error. And if we had submitted
successfully some bios before then it will need to wait theirs
completion.

Otherwise, if BIO_EOPNOTSUPP flag is not set yet during check in
nilfs_segbuf_submit_bio() then sb_nbio is remained incremented. And, for
example, nilfs_segbuf_wait() can end correctly the cycle with
wait_for_completion() call. We will know about error during bio
processing because of sb_err was incremented in nilfs_end_bio_write().

So, I suppose that my logic is right. Please, correct me if I am wrong.

What do you think?

By the way, why sb_err has atomic_t type but sb_nbio is simple int type?
Maybe, sb_nbio should be atomic_t type too?

With the best regards,
Vyacheslav Dubeyko.


> Note that wait_for_completion() is called based on the count of
> segbuf->sb_nbio even if nilfs_segbuf_submit_bio() returns an error.
> This is performed through the following path:
> 
>             nilfs_segbuf_submit_bio
>           nilfs_segbuf_submit_bh
>         nilfs_segbuf_write
>       nilfs_write_logs
>     nilfs_segctor_write
>   nilfs_segctor_do_construct
>     nilfs_segctor_abort_construction
>       nilfs_wait_on_logs
>         nilfs_segbuf_wait
>           wait_for_completion
>             (repeated for the number of times of segbuf->sb_nbio)
> 
> 
> If you think this is a separate problem, then it should be fixed in
> another patch and nilfs_segbuf_submit_bio() should not be touched in
> this patch.


--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] nilfs2: remove double bio_put() in nilfs_end_bio_write() for BIO_EOPNOTSUPP error

[Ryusuke Konishi]

On Mon, 12 Aug 2013 17:16:28 +0400, Vyacheslav Dubeyko wrote:
>> Otherwise, your change of nilfs_segbuf_submit_bio() is just a
>> equivalent transformation and doesn't fix problem, that is, a mismatch
>> of the number of calls between complete() and wait_for_completion().
>> 
>> In your patch, nilfs_end_bio_write() function calls the complete()
>> routine as before even if it received an EOPNOTSUPP error.
>> 
>> In that case, segbuf->sb_nbio must be incremented to call
>> wait_for_completion() the right number of times.
>> 
> 
> I moved incrementing of sb_nbio nearly after submit_bio(mode, bio). So,
> firstly, we call submit_bio(mode, bio) and segbuf->sb_nbio++. Then,
> because of asynchronous nature of nilfs_end_bio_write() call, we have
> two alternatives: (1) nilfs_end_bio_write() detects that BIO_EOPNOTSUPP
> flag is set; (2) BIO_EOPNOTSUPP flag isn't set because bio is not
> processed yet or we haven't error.
> 
> If nilfs_end_bio_write() method would detect -EOPNOTSUPP error then it
> set BIO_EOPNOTSUPP flag, increment sb_err field and to call complete(). 
> 
> So, if we detect that BIO_EOPNOTSUPP flag is set in
> nilfs_segbuf_submit_bio() then we decrement sb_nbio and to return from
> nilfs_segbuf_submit_bio() with error. And if we had submitted
> successfully some bios before then it will need to wait theirs

sb_nbio must be incremented exactly the same number of times as
complete() function was called (or will be called) because
nilfs_segbuf_wait() will call wail_for_completion() for the number of
times set to sb_nbio:

        do {
	        wait_for_completion(&segbuf->sb_bio_event);
	} while (--segbuf->sb_nbio > 0);


Two functions complete() and wait_for_completion() must be called the
same number of times for the same sb_bio_event.  Otherwise,
wait_for_completion() will hang or leak.

In your patch, nilfs_end_bio_write() will call complete() even in the
case "(1) nilfs_end_bio_write() detects that BIO_EOPNOTSUPP flag is
set" (like the current implementation).  Therefore, sb_nbio must be
incremented for that count.


This looks another issue of the current implemention and the second
bug can be fixed as follows:

        submit_bio(mode, bio);
+       segbuf->sb_nbio++;
        if (bio_flagged(bio, BIO_EOPNOTSUPP)) {
                bio_put(bio);
                err = -EOPNOTSUPP;
                goto failed;
        }
-       segbuf->sb_nbio++;
        bio_put(bio);



> By the way, why sb_err has atomic_t type but sb_nbio is simple int type?
> Maybe, sb_nbio should be atomic_t type too?

No, the current implementation is correct.

sb_err is incremented asynchronously by nilfs_end_bio_write(), so it
is implemented with atomic_t type.  On the other hand, sb_nbio is
operated sequentially and therefore it doesn't need to use atomic_t
type.


Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html