From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: Re: [PATCH v4 3/6] RDMA/i40iw: Eliminate duplicate barriers on weakly-ordered archs Date: Wed, 21 Mar 2018 14:02:53 -0600 Message-ID: <20180321200253.GA21453@ziepe.ca> References: <1521514068-8856-1-git-send-email-okaya@codeaurora.org> <1521514068-8856-4-git-send-email-okaya@codeaurora.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <1521514068-8856-4-git-send-email-okaya@codeaurora.org> Sender: linux-kernel-owner@vger.kernel.org To: Sinan Kaya Cc: linux-rdma@vger.kernel.org, timur@codeaurora.org, sulrich@codeaurora.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Faisal Latif , Shiraz Saleem , Doug Ledford , linux-kernel@vger.kernel.org List-Id: linux-arm-msm@vger.kernel.org On Mon, Mar 19, 2018 at 10:47:45PM -0400, Sinan Kaya wrote: > diff --git a/drivers/infiniband/hw/i40iw/i40iw_uk.c b/drivers/infiniband/hw/i40iw/i40iw_uk.c > index 8afa5a6..7f0ebed 100644 > +++ b/drivers/infiniband/hw/i40iw/i40iw_uk.c > @@ -723,7 +723,7 @@ static void i40iw_cq_request_notification(struct i40iw_cq_uk *cq, > > wmb(); /* make sure WQE is populated before valid bit is set */ > > - writel(cq->cq_id, cq->cqe_alloc_reg); > + writel_relaxed(cq->cq_id, cq->cqe_alloc_reg); > } Ah, this one is probably not OK, i40iw_cq_request_notification is called here: spin_lock_irqsave(&iwcq->lock, flags); ukcq->ops.iw_cq_request_notification(ukcq, cq_notify); spin_unlock_irqrestore(&iwcq->lock, flags); So this needs to add mmmiomb(); to keep the same semantics. Generally I think you need to be very careful to ensure that any conversion to _relaxed isn't contained by a spinlock, or the mmiomb() is present. Maybe even do a first series with this obviously correct pattern: wmb(); writel() -> writel_relaxed() writel() -> writel_relaxed() [..] mmiowmb(); Jason