[PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Andrea Parri]

This barrier only applies to the read-modify-write operations; in
particular, it does not apply to the atomic_read() primitive.

Replace the barrier with an smp_mb().

Fixes: 856cc4c237add ("IB/hfi1: Add the capability for reserved operations")
Cc: stable@vger.kernel.org
Reported-by: "Paul E. McKenney" <paulmck@linux.ibm.com>
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrea Parri <andrea.parri@amarulasolutions.com>
Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
Cc: Mike Marciniszyn <mike.marciniszyn@intel.com>
Cc: Doug Ledford <dledford@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: linux-rdma@vger.kernel.org
---
 drivers/infiniband/sw/rdmavt/qp.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c
index a34b9a2a32b60..b64fd151d31fb 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -1863,11 +1863,11 @@ static inline int rvt_qp_is_avail(
 	u32 reserved_used;
 
 	/* see rvt_qp_wqe_unreserve() */
-	smp_mb__before_atomic();
+	smp_mb();
 	reserved_used = atomic_read(&qp->s_reserved_used);
 	if (unlikely(reserved_op)) {
 		/* see rvt_qp_wqe_unreserve() */
-		smp_mb__before_atomic();
+		smp_mb();
 		if (reserved_used >= rdi->dparms.reserved_operations)
 			return -ENOMEM;
 		return 0;
@@ -1882,7 +1882,7 @@ static inline int rvt_qp_is_avail(
 		avail = slast - qp->s_head;
 
 	/* see rvt_qp_wqe_unreserve() */
-	smp_mb__before_atomic();
+	smp_mb();
 	reserved_used = atomic_read(&qp->s_reserved_used);
 	avail =  avail - 1 -
 		(rdi->dparms.reserved_operations - reserved_used);
-- 
2.7.4

RE: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Ruhl, Michael J]

>-----Original Message-----
>From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-
>owner@vger.kernel.org] On Behalf Of Andrea Parri
>Sent: Monday, April 29, 2019 4:15 PM
>To: linux-kernel@vger.kernel.org
>Cc: Andrea Parri <andrea.parri@amarulasolutions.com>;
>stable@vger.kernel.org; Dalessandro, Dennis
><dennis.dalessandro@intel.com>; Marciniszyn, Mike
><mike.marciniszyn@intel.com>; Doug Ledford <dledford@redhat.com>;
>Jason Gunthorpe <jgg@ziepe.ca>; linux-rdma@vger.kernel.org
>Subject: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()
>
>This barrier only applies to the read-modify-write operations; in
>particular, it does not apply to the atomic_read() primitive.
>
>Replace the barrier with an smp_mb().

This is one of a couple of barrier issues that we are currently looking into.

See:

[PATCH for-next 6/9] IB/rdmavt: Add new completion inline

We will take a look at this one as well.

Thanks,

Mike

>Fixes: 856cc4c237add ("IB/hfi1: Add the capability for reserved operations")
>Cc: stable@vger.kernel.org
>Reported-by: "Paul E. McKenney" <paulmck@linux.ibm.com>
>Reported-by: Peter Zijlstra <peterz@infradead.org>
>Signed-off-by: Andrea Parri <andrea.parri@amarulasolutions.com>
>Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
>Cc: Mike Marciniszyn <mike.marciniszyn@intel.com>
>Cc: Doug Ledford <dledford@redhat.com>
>Cc: Jason Gunthorpe <jgg@ziepe.ca>
>Cc: linux-rdma@vger.kernel.org
>---
> drivers/infiniband/sw/rdmavt/qp.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
>diff --git a/drivers/infiniband/sw/rdmavt/qp.c
>b/drivers/infiniband/sw/rdmavt/qp.c
>index a34b9a2a32b60..b64fd151d31fb 100644
>--- a/drivers/infiniband/sw/rdmavt/qp.c
>+++ b/drivers/infiniband/sw/rdmavt/qp.c
>@@ -1863,11 +1863,11 @@ static inline int rvt_qp_is_avail(
> 	u32 reserved_used;
>
> 	/* see rvt_qp_wqe_unreserve() */
>-	smp_mb__before_atomic();
>+	smp_mb();
> 	reserved_used = atomic_read(&qp->s_reserved_used);
> 	if (unlikely(reserved_op)) {
> 		/* see rvt_qp_wqe_unreserve() */
>-		smp_mb__before_atomic();
>+		smp_mb();
> 		if (reserved_used >= rdi->dparms.reserved_operations)
> 			return -ENOMEM;
> 		return 0;
>@@ -1882,7 +1882,7 @@ static inline int rvt_qp_is_avail(
> 		avail = slast - qp->s_head;
>
> 	/* see rvt_qp_wqe_unreserve() */
>-	smp_mb__before_atomic();
>+	smp_mb();
> 	reserved_used = atomic_read(&qp->s_reserved_used);
> 	avail =  avail - 1 -
> 		(rdi->dparms.reserved_operations - reserved_used);
>--
>2.7.4

Re: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Andrea Parri]

Hi Mike,

> >This barrier only applies to the read-modify-write operations; in
> >particular, it does not apply to the atomic_read() primitive.
> >
> >Replace the barrier with an smp_mb().
> 
> This is one of a couple of barrier issues that we are currently looking into.
> 
> See:
> 
> [PATCH for-next 6/9] IB/rdmavt: Add new completion inline
> 
> We will take a look at this one as well.

Thank you for the reference and for looking into this,

  Andrea

Re: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Peter Zijlstra]

On Mon, Apr 29, 2019 at 10:15:01PM +0200, Andrea Parri wrote:
> This barrier only applies to the read-modify-write operations; in
> particular, it does not apply to the atomic_read() primitive.
> 
> Replace the barrier with an smp_mb().
> 
> Fixes: 856cc4c237add ("IB/hfi1: Add the capability for reserved operations")
> Cc: stable@vger.kernel.org
> Reported-by: "Paul E. McKenney" <paulmck@linux.ibm.com>
> Reported-by: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Andrea Parri <andrea.parri@amarulasolutions.com>
> Cc: Dennis Dalessandro <dennis.dalessandro@intel.com>
> Cc: Mike Marciniszyn <mike.marciniszyn@intel.com>
> Cc: Doug Ledford <dledford@redhat.com>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> Cc: linux-rdma@vger.kernel.org
> ---
>  drivers/infiniband/sw/rdmavt/qp.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c
> index a34b9a2a32b60..b64fd151d31fb 100644
> --- a/drivers/infiniband/sw/rdmavt/qp.c
> +++ b/drivers/infiniband/sw/rdmavt/qp.c
> @@ -1863,11 +1863,11 @@ static inline int rvt_qp_is_avail(
>  	u32 reserved_used;
>  
>  	/* see rvt_qp_wqe_unreserve() */

I see a completely bogus comment in rvf_op_wqe_unreserve(), referring to
bogus comments makes this barrier bogus too.

> -	smp_mb__before_atomic();
> +	smp_mb();
>  	reserved_used = atomic_read(&qp->s_reserved_used);
>  	if (unlikely(reserved_op)) {
>  		/* see rvt_qp_wqe_unreserve() */
> -		smp_mb__before_atomic();

This was before, but there is nothing _after_ this. Which means this
barrier was complete garbage anyway.

> +		smp_mb();
>  		if (reserved_used >= rdi->dparms.reserved_operations)
>  			return -ENOMEM;
>  		return 0;
> @@ -1882,7 +1882,7 @@ static inline int rvt_qp_is_avail(
>  		avail = slast - qp->s_head;
>  
>  	/* see rvt_qp_wqe_unreserve() */
> -	smp_mb__before_atomic();
> +	smp_mb();

Same as the first.

>  	reserved_used = atomic_read(&qp->s_reserved_used);
>  	avail =  avail - 1 -
>  		(rdi->dparms.reserved_operations - reserved_used);

Re: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Andrea Parri]

On Tue, Apr 30, 2019 at 01:16:57AM +0200, Andrea Parri wrote:
> Hi Mike,
> 
> > >This barrier only applies to the read-modify-write operations; in
> > >particular, it does not apply to the atomic_read() primitive.
> > >
> > >Replace the barrier with an smp_mb().
> > 
> > This is one of a couple of barrier issues that we are currently looking into.
> > 
> > See:
> > 
> > [PATCH for-next 6/9] IB/rdmavt: Add new completion inline
> > 
> > We will take a look at this one as well.
> 
> Thank you for the reference and for looking into this,

So, I'm planning to just drop this patch; or can I do something to help?

Please let me know.

Thanx,
  Andrea


> 
>   Andrea

Re: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Dennis Dalessandro]

On 5/9/2019 5:12 PM, Andrea Parri wrote:
> On Tue, Apr 30, 2019 at 01:16:57AM +0200, Andrea Parri wrote:
>> Hi Mike,
>>
>>>> This barrier only applies to the read-modify-write operations; in
>>>> particular, it does not apply to the atomic_read() primitive.
>>>>
>>>> Replace the barrier with an smp_mb().
>>>
>>> This is one of a couple of barrier issues that we are currently looking into.
>>>
>>> See:
>>>
>>> [PATCH for-next 6/9] IB/rdmavt: Add new completion inline
>>>
>>> We will take a look at this one as well.
>>
>> Thank you for the reference and for looking into this,
> 
> So, I'm planning to just drop this patch; or can I do something to help?
> 
> Please let me know.

Mike was looking into this, and I've got a handful of patches from him 
to review. He's unavailable for a while but if it's not included in the 
patches I've got we'll get something out shortly. So yes I think we can 
hold off on this patch for now. Thanks.

-Denny

Re: [PATCH 5/5] IB/hfi1: Fix improper uses of smp_mb__before_atomic()

[Andrea Parri]

On Tue, May 14, 2019 at 08:32:52AM -0400, Dennis Dalessandro wrote:
> On 5/9/2019 5:12 PM, Andrea Parri wrote:
> >On Tue, Apr 30, 2019 at 01:16:57AM +0200, Andrea Parri wrote:
> >>Hi Mike,
> >>
> >>>>This barrier only applies to the read-modify-write operations; in
> >>>>particular, it does not apply to the atomic_read() primitive.
> >>>>
> >>>>Replace the barrier with an smp_mb().
> >>>
> >>>This is one of a couple of barrier issues that we are currently looking into.
> >>>
> >>>See:
> >>>
> >>>[PATCH for-next 6/9] IB/rdmavt: Add new completion inline
> >>>
> >>>We will take a look at this one as well.
> >>
> >>Thank you for the reference and for looking into this,
> >
> >So, I'm planning to just drop this patch; or can I do something to help?
> >
> >Please let me know.
> 
> Mike was looking into this, and I've got a handful of patches from him to
> review. He's unavailable for a while but if it's not included in the patches
> I've got we'll get something out shortly. So yes I think we can hold off on
> this patch for now. Thanks.

Thank you for the confirmation, Dennis.  I'll hold off on this one.

  Andrea