From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Subject: Re: RFC on writel and writel_relaxed Date: Tue, 27 Mar 2018 09:28:01 +1100 Message-ID: <1522103281.7364.17.camel@kernel.crashing.org> References: <1521692689.16434.293.camel@kernel.crashing.org> <1521726722.16434.312.camel@kernel.crashing.org> <20180323163510.GC13033@ziepe.ca> <1521854626.16434.359.camel@kernel.crashing.org> <58ce5b83f40f4775bec1be8db66adb0d@AcuMS.aculab.com> <20180326165425.GA15554@ziepe.ca> <20180326202545.GB15554@ziepe.ca> <20180326210951.GD15554@ziepe.ca> <1522101717.7364.14.camel@kernel.crashing.org> <4d85c5e9-a16a-0761-4700-169d1c061812@codeaurora.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <4d85c5e9-a16a-0761-4700-169d1c061812@codeaurora.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppe-linuxppc-embedded-2=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" To: Sinan Kaya , Arnd Bergmann , Jason Gunthorpe Cc: "linux-rdma@vger.kernel.org" , Will Deacon , Alexander Duyck , David Laight , Oliver , "Paul E. McKenney" , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" List-Id: linux-rdma@vger.kernel.org On Mon, 2018-03-26 at 18:08 -0400, Sinan Kaya wrote: > On 3/26/2018 6:01 PM, Benjamin Herrenschmidt wrote: > > On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > On 3/26/2018 5:30 PM, Arnd Bergmann wrote: > > > > > But that was never a requirement of writel(), > > > > > Documentation/memory-barriers.txt gives an explicit example demanding > > > > > the wmb() before writel() for ordering system memory against writel. > > > > > > > > Indeed, but it's in an example for when to use dma_wmb(), not wmb(). > > > > Adding Alexander Duyck to Cc, he added that section as part of > > > > 1077fa36f23e ("arch: Add lightweight memory barriers dma_rmb() and > > > > dma_wmb()"). Also adding the other people that were involved with that. > > > > > > > > > > ARM developers can get away with not including wmb() in their code and use > > > writel() to observe memory writes due to implicit barriers. > > > > > > However, same code will not work on Intel. > > > > Wrong. It will. > > > > You do NOT need wmb between writes to memory and writel. > > If writel() provides such a guarantee, why do I see code sequences like > > wmb() > writel() > > all over the place. Because it was badly documented and people didn't know what to do, or maybe the underlying mapping is WC ? I don't know for sure but I can tell you Linus opinion on the matter back in the days was very clear and that's why we implemented writel the way we did on powerpc. > > > > > writel() has a compiler barrier in it for x86. > > > wmb() has a sync operation in it for x86. > > > > > > Unless wmb() is called, PCIe device won't observe memory updates from the CPU. > > > > This is completely wrong. They will. Intel provides the necessary > > ordering guarantees without an explicit wmb. > > > > I'm still reserving my doubts here. I was told about an explicit > wmb() requirement last week. By whome ? > > Otherwise almost all drivers out there are broken which I very much > > doubt :-) > > > > Cheers, > > Ben. > > > > > > > >