From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <benh@kernel.crashing.org>
Received: from gate.crashing.org (gate.crashing.org [63.228.1.57])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by lists.ozlabs.org (Postfix) with ESMTPS id 4097Rh5hVQzF265
 for <linuxppc-dev@lists.ozlabs.org>; Tue, 27 Mar 2018 09:02:40 +1100 (AEDT)
Message-ID: <1522101717.7364.14.camel@kernel.crashing.org>
Subject: Re: RFC on writel and writel_relaxed
From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
To: Sinan Kaya <okaya@codeaurora.org>, Arnd Bergmann <arnd@arndb.de>, Jason
 Gunthorpe <jgg@ziepe.ca>
Cc: David Laight <David.Laight@aculab.com>, Oliver <oohall@gmail.com>, "open
 list:LINUX FOR POWERPC (32-BIT AND 64-BIT)"
 <linuxppc-dev@lists.ozlabs.org>, "linux-rdma@vger.kernel.org"
 <linux-rdma@vger.kernel.org>, Alexander Duyck
 <alexander.h.duyck@redhat.com>, Will Deacon <will.deacon@arm.com>, "Paul
 E. McKenney" <paulmck@linux.vnet.ibm.com>
Date: Tue, 27 Mar 2018 09:01:57 +1100
In-Reply-To: <ebb36fd4-62e2-394a-219f-686c45a6dc75@codeaurora.org>
References: <1521692689.16434.293.camel@kernel.crashing.org>
 <CAOSf1CFv1HHCL3YOVhRn2U=grdNjaQ=A4m3xwxN2Rek1-_TySg@mail.gmail.com>
 <1521726722.16434.312.camel@kernel.crashing.org>
 <20180323163510.GC13033@ziepe.ca>
 <1521854626.16434.359.camel@kernel.crashing.org>
 <58ce5b83f40f4775bec1be8db66adb0d@AcuMS.aculab.com>
 <20180326165425.GA15554@ziepe.ca>
 <CAK8P3a1zeMyj+Z-y4ER4moY6Zip9EWNOinf+VnboGOrgiwbBZA@mail.gmail.com>
 <20180326202545.GB15554@ziepe.ca>
 <CAK8P3a3fc43ZcW626hmsd3DVcLw7hGkdUMxp7s4Rn3mdkziwMQ@mail.gmail.com>
 <20180326210951.GD15554@ziepe.ca>
 <CAK8P3a2UU1xAM0NLo7Q4-Xgo1SzY3De1uqpFudr+2ZW7nHEPmA@mail.gmail.com>
 <ebb36fd4-62e2-394a-219f-686c45a6dc75@codeaurora.org>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
List-Id: Linux on PowerPC Developers Mail List <linuxppc-dev.lists.ozlabs.org>
List-Unsubscribe: <https://lists.ozlabs.org/options/linuxppc-dev>,
 <mailto:linuxppc-dev-request@lists.ozlabs.org?subject=unsubscribe>
List-Archive: <http://lists.ozlabs.org/pipermail/linuxppc-dev/>
List-Post: <mailto:linuxppc-dev@lists.ozlabs.org>
List-Help: <mailto:linuxppc-dev-request@lists.ozlabs.org?subject=help>
List-Subscribe: <https://lists.ozlabs.org/listinfo/linuxppc-dev>,
 <mailto:linuxppc-dev-request@lists.ozlabs.org?subject=subscribe>

On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote:
> On 3/26/2018 5:30 PM, Arnd Bergmann wrote:
> > > But that was never a requirement of writel(),
> > > Documentation/memory-barriers.txt gives an explicit example demanding
> > > the wmb() before writel() for ordering system memory against writel.
> > 
> > Indeed, but it's in an example for when to use dma_wmb(), not wmb().
> > Adding Alexander Duyck to Cc, he added that section as part of
> > 1077fa36f23e ("arch: Add lightweight memory barriers dma_rmb() and
> > dma_wmb()"). Also adding the other people that were involved with that.
> > 
> 
> ARM developers can get away with not including wmb() in their code and use
> writel() to observe memory writes due to implicit barriers.
> 
> However, same code will not work on Intel.

Wrong. It will.

You do NOT need wmb between writes to memory and writel.

> writel() has a compiler barrier in it for x86.
> wmb() has a sync operation in it for x86. 
> 
> Unless wmb() is called, PCIe device won't observe memory updates from the CPU.

This is completely wrong. They will. Intel provides the necessary
ordering guarantees without an explicit wmb.

Otherwise almost all drivers out there are broken which I very much
doubt :-)

Cheers,
Ben.