[PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

[PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

[Baptiste Lepers]

Per comment, record_root_in_trans orders the writes of the root->state
and root->last_trans:
      set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
      smp_wmb();
      root->last_trans = trans->transid;

But the barrier that enforces the order on the read side is misplaced:
     smp_rmb(); <-- misplaced
     if (root->last_trans == trans->transid &&
    <-- missing barrier here -->
            !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))

This patches fixes the ordering and wraps the racy accesses with
READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.

Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
Signed-off-by: Baptiste Lepers <baptiste.lepers@gmail.com>
---
 fs/btrfs/transaction.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 14b9fdc8aaa9..a609222e6704 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
 				   (unsigned long)root->root_key.objectid,
 				   BTRFS_ROOT_TRANS_TAG);
 		spin_unlock(&fs_info->fs_roots_radix_lock);
-		root->last_trans = trans->transid;
+		WRITE_ONCE(root->last_trans, trans->transid);
 
 		/* this is pretty tricky.  We don't want to
 		 * take the relocation lock in btrfs_record_root_in_trans
@@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root)
 {
 	struct btrfs_fs_info *fs_info = root->fs_info;
-	int ret;
+	int ret, last_trans;
 
 	if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
 		return 0;
@@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
 	 * see record_root_in_trans for comments about IN_TRANS_SETUP usage
 	 * and barriers
 	 */
+	last_trans = READ_ONCE(root->last_trans);
 	smp_rmb();
-	if (root->last_trans == trans->transid &&
+	if (last_trans == trans->transid &&
 	    !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
 		return 0;
 
-- 
2.17.1

Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

[David Sterba]

On Mon, Sep 06, 2021 at 11:25:59AM +1000, Baptiste Lepers wrote:
> Per comment, record_root_in_trans orders the writes of the root->state
> and root->last_trans:
>       set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
>       smp_wmb();
>       root->last_trans = trans->transid;
> 
> But the barrier that enforces the order on the read side is misplaced:
>      smp_rmb(); <-- misplaced
>      if (root->last_trans == trans->transid &&
>     <-- missing barrier here -->
>             !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
> 
> This patches fixes the ordering and wraps the racy accesses with
> READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
> 
> Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
> Signed-off-by: Baptiste Lepers <baptiste.lepers@gmail.com>
> ---
>  fs/btrfs/transaction.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
> index 14b9fdc8aaa9..a609222e6704 100644
> --- a/fs/btrfs/transaction.c
> +++ b/fs/btrfs/transaction.c
> @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
>  				   (unsigned long)root->root_key.objectid,
>  				   BTRFS_ROOT_TRANS_TAG);
>  		spin_unlock(&fs_info->fs_roots_radix_lock);
> -		root->last_trans = trans->transid;
> +		WRITE_ONCE(root->last_trans, trans->transid);
>  
>  		/* this is pretty tricky.  We don't want to
>  		 * take the relocation lock in btrfs_record_root_in_trans
> @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>  			       struct btrfs_root *root)
>  {
>  	struct btrfs_fs_info *fs_info = root->fs_info;
> -	int ret;
> +	int ret, last_trans;
>  
>  	if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
>  		return 0;
> @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>  	 * see record_root_in_trans for comments about IN_TRANS_SETUP usage
>  	 * and barriers
>  	 */
> +	last_trans = READ_ONCE(root->last_trans);
>  	smp_rmb();
> -	if (root->last_trans == trans->transid &&
> +	if (last_trans == trans->transid &&
>  	    !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))

Aren't the smp_rmb barriers supposed to be used before the condition?

Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

[Baptiste Lepers]

No, they need to be between the reads to have an effect. See
https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP
BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You
will see that the barriers are always between the ordered reads and
not before.

I think that Paul, the barrier guru, can confirm that the barrier was
misplaced in the original code? :)


On Tue, Sep 7, 2021 at 10:43 AM Baptiste Lepers
<baptiste.lepers@gmail.com> wrote:
>
>
>
> On Mon, Sep 6, 2021 at 10:27 PM David Sterba <dsterba@suse.cz> wrote:
>>
>> On Mon, Sep 06, 2021 at 11:25:59AM +1000, Baptiste Lepers wrote:
>> > Per comment, record_root_in_trans orders the writes of the root->state
>> > and root->last_trans:
>> >       set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
>> >       smp_wmb();
>> >       root->last_trans = trans->transid;
>> >
>> > But the barrier that enforces the order on the read side is misplaced:
>> >      smp_rmb(); <-- misplaced
>> >      if (root->last_trans == trans->transid &&
>> >     <-- missing barrier here -->
>> >             !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>> >
>> > This patches fixes the ordering and wraps the racy accesses with
>> > READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
>> >
>> > Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
>> > Signed-off-by: Baptiste Lepers <baptiste.lepers@gmail.com>
>> > ---
>> >  fs/btrfs/transaction.c | 7 ++++---
>> >  1 file changed, 4 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
>> > index 14b9fdc8aaa9..a609222e6704 100644
>> > --- a/fs/btrfs/transaction.c
>> > +++ b/fs/btrfs/transaction.c
>> > @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
>> >                                  (unsigned long)root->root_key.objectid,
>> >                                  BTRFS_ROOT_TRANS_TAG);
>> >               spin_unlock(&fs_info->fs_roots_radix_lock);
>> > -             root->last_trans = trans->transid;
>> > +             WRITE_ONCE(root->last_trans, trans->transid);
>> >
>> >               /* this is pretty tricky.  We don't want to
>> >                * take the relocation lock in btrfs_record_root_in_trans
>> > @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>> >                              struct btrfs_root *root)
>> >  {
>> >       struct btrfs_fs_info *fs_info = root->fs_info;
>> > -     int ret;
>> > +     int ret, last_trans;
>> >
>> >       if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
>> >               return 0;
>> > @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>> >        * see record_root_in_trans for comments about IN_TRANS_SETUP usage
>> >        * and barriers
>> >        */
>> > +     last_trans = READ_ONCE(root->last_trans);
>> >       smp_rmb();
>> > -     if (root->last_trans == trans->transid &&
>> > +     if (last_trans == trans->transid &&
>> >           !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>>
>> Aren't the smp_rmb barriers supposed to be used before the condition?
>
>
> No, they need to be between the reads to have an effect. See  https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You will see that the barriers are always between the ordered reads and not before.
>
> I think that Paul, the barrier guru, can confirm that the barrier was misplaced in the original code? :)

Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

[Baptiste Lepers]

Just curious about the status of this patch. :) Let me know if you
need further information.

Thanks!

On Tue, Sep 7, 2021 at 10:44 AM Baptiste Lepers
<baptiste.lepers@gmail.com> wrote:
>
> No, they need to be between the reads to have an effect. See
> https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP
> BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You
> will see that the barriers are always between the ordered reads and
> not before.
>
> I think that Paul, the barrier guru, can confirm that the barrier was
> misplaced in the original code? :)
>
>
> On Tue, Sep 7, 2021 at 10:43 AM Baptiste Lepers
> <baptiste.lepers@gmail.com> wrote:
> >
> >
> >
> > On Mon, Sep 6, 2021 at 10:27 PM David Sterba <dsterba@suse.cz> wrote:
> >>
> >> On Mon, Sep 06, 2021 at 11:25:59AM +1000, Baptiste Lepers wrote:
> >> > Per comment, record_root_in_trans orders the writes of the root->state
> >> > and root->last_trans:
> >> >       set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
> >> >       smp_wmb();
> >> >       root->last_trans = trans->transid;
> >> >
> >> > But the barrier that enforces the order on the read side is misplaced:
> >> >      smp_rmb(); <-- misplaced
> >> >      if (root->last_trans == trans->transid &&
> >> >     <-- missing barrier here -->
> >> >             !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
> >> >
> >> > This patches fixes the ordering and wraps the racy accesses with
> >> > READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
> >> >
> >> > Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
> >> > Signed-off-by: Baptiste Lepers <baptiste.lepers@gmail.com>
> >> > ---
> >> >  fs/btrfs/transaction.c | 7 ++++---
> >> >  1 file changed, 4 insertions(+), 3 deletions(-)
> >> >
> >> > diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
> >> > index 14b9fdc8aaa9..a609222e6704 100644
> >> > --- a/fs/btrfs/transaction.c
> >> > +++ b/fs/btrfs/transaction.c
> >> > @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
> >> >                                  (unsigned long)root->root_key.objectid,
> >> >                                  BTRFS_ROOT_TRANS_TAG);
> >> >               spin_unlock(&fs_info->fs_roots_radix_lock);
> >> > -             root->last_trans = trans->transid;
> >> > +             WRITE_ONCE(root->last_trans, trans->transid);
> >> >
> >> >               /* this is pretty tricky.  We don't want to
> >> >                * take the relocation lock in btrfs_record_root_in_trans
> >> > @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> >> >                              struct btrfs_root *root)
> >> >  {
> >> >       struct btrfs_fs_info *fs_info = root->fs_info;
> >> > -     int ret;
> >> > +     int ret, last_trans;
> >> >
> >> >       if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
> >> >               return 0;
> >> > @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> >> >        * see record_root_in_trans for comments about IN_TRANS_SETUP usage
> >> >        * and barriers
> >> >        */
> >> > +     last_trans = READ_ONCE(root->last_trans);
> >> >       smp_rmb();
> >> > -     if (root->last_trans == trans->transid &&
> >> > +     if (last_trans == trans->transid &&
> >> >           !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
> >>
> >> Aren't the smp_rmb barriers supposed to be used before the condition?
> >
> >
> > No, they need to be between the reads to have an effect. See  https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You will see that the barriers are always between the ordered reads and not before.
> >
> > I think that Paul, the barrier guru, can confirm that the barrier was misplaced in the original code? :)

Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

[Filipe Manana]

On Mon, Sep 6, 2021 at 2:38 AM Baptiste Lepers
<baptiste.lepers@gmail.com> wrote:
>
> Per comment, record_root_in_trans orders the writes of the root->state
> and root->last_trans:
>       set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
>       smp_wmb();
>       root->last_trans = trans->transid;
>
> But the barrier that enforces the order on the read side is misplaced:
>      smp_rmb(); <-- misplaced
>      if (root->last_trans == trans->transid &&
>     <-- missing barrier here -->
>             !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>
> This patches fixes the ordering and wraps the racy accesses with
> READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
>
> Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
> Signed-off-by: Baptiste Lepers <baptiste.lepers@gmail.com>
> ---
>  fs/btrfs/transaction.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
> index 14b9fdc8aaa9..a609222e6704 100644
> --- a/fs/btrfs/transaction.c
> +++ b/fs/btrfs/transaction.c
> @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
>                                    (unsigned long)root->root_key.objectid,
>                                    BTRFS_ROOT_TRANS_TAG);
>                 spin_unlock(&fs_info->fs_roots_radix_lock);
> -               root->last_trans = trans->transid;
> +               WRITE_ONCE(root->last_trans, trans->transid);
>
>                 /* this is pretty tricky.  We don't want to
>                  * take the relocation lock in btrfs_record_root_in_trans
> @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>                                struct btrfs_root *root)
>  {
>         struct btrfs_fs_info *fs_info = root->fs_info;
> -       int ret;
> +       int ret, last_trans;

last_trans should be u64, as root->last_trans is a u64.

Other than that it looks good to me.
Thanks.

>
>         if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
>                 return 0;
> @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>          * see record_root_in_trans for comments about IN_TRANS_SETUP usage
>          * and barriers
>          */
> +       last_trans = READ_ONCE(root->last_trans);
>         smp_rmb();
> -       if (root->last_trans == trans->transid &&
> +       if (last_trans == trans->transid &&
>             !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>                 return 0;
>
> --
> 2.17.1
>


-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”