From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate.crashing.org (gate.crashing.org [63.228.1.57]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40981l4rL8zF273 for ; Tue, 27 Mar 2018 09:28:43 +1100 (AEDT) Message-ID: <1522103281.7364.17.camel@kernel.crashing.org> Subject: Re: RFC on writel and writel_relaxed From: Benjamin Herrenschmidt To: Sinan Kaya , Arnd Bergmann , Jason Gunthorpe Cc: David Laight , Oliver , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" , "linux-rdma@vger.kernel.org" , Alexander Duyck , Will Deacon , "Paul E. McKenney" Date: Tue, 27 Mar 2018 09:28:01 +1100 In-Reply-To: <4d85c5e9-a16a-0761-4700-169d1c061812@codeaurora.org> References: <1521692689.16434.293.camel@kernel.crashing.org> <1521726722.16434.312.camel@kernel.crashing.org> <20180323163510.GC13033@ziepe.ca> <1521854626.16434.359.camel@kernel.crashing.org> <58ce5b83f40f4775bec1be8db66adb0d@AcuMS.aculab.com> <20180326165425.GA15554@ziepe.ca> <20180326202545.GB15554@ziepe.ca> <20180326210951.GD15554@ziepe.ca> <1522101717.7364.14.camel@kernel.crashing.org> <4d85c5e9-a16a-0761-4700-169d1c061812@codeaurora.org> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Mon, 2018-03-26 at 18:08 -0400, Sinan Kaya wrote: > On 3/26/2018 6:01 PM, Benjamin Herrenschmidt wrote: > > On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > On 3/26/2018 5:30 PM, Arnd Bergmann wrote: > > > > > But that was never a requirement of writel(), > > > > > Documentation/memory-barriers.txt gives an explicit example demanding > > > > > the wmb() before writel() for ordering system memory against writel. > > > > > > > > Indeed, but it's in an example for when to use dma_wmb(), not wmb(). > > > > Adding Alexander Duyck to Cc, he added that section as part of > > > > 1077fa36f23e ("arch: Add lightweight memory barriers dma_rmb() and > > > > dma_wmb()"). Also adding the other people that were involved with that. > > > > > > > > > > ARM developers can get away with not including wmb() in their code and use > > > writel() to observe memory writes due to implicit barriers. > > > > > > However, same code will not work on Intel. > > > > Wrong. It will. > > > > You do NOT need wmb between writes to memory and writel. > > If writel() provides such a guarantee, why do I see code sequences like > > wmb() > writel() > > all over the place. Because it was badly documented and people didn't know what to do, or maybe the underlying mapping is WC ? I don't know for sure but I can tell you Linus opinion on the matter back in the days was very clear and that's why we implemented writel the way we did on powerpc. > > > > > writel() has a compiler barrier in it for x86. > > > wmb() has a sync operation in it for x86. > > > > > > Unless wmb() is called, PCIe device won't observe memory updates from the CPU. > > > > This is completely wrong. They will. Intel provides the necessary > > ordering guarantees without an explicit wmb. > > > > I'm still reserving my doubts here. I was told about an explicit > wmb() requirement last week. By whome ? > > Otherwise almost all drivers out there are broken which I very much > > doubt :-) > > > > Cheers, > > Ben. > > > > > > > >