From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate.crashing.org (gate.crashing.org [63.228.1.57]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 409yy84rDTzF2Cj for ; Wed, 28 Mar 2018 17:43:28 +1100 (AEDT) Message-ID: <1522219376.7364.109.camel@kernel.crashing.org> Subject: Re: RFC on writel and writel_relaxed From: Benjamin Herrenschmidt To: Linus Torvalds Cc: Alexander Duyck , Will Deacon , Sinan Kaya , Arnd Bergmann , Jason Gunthorpe , David Laight , Oliver , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" , "linux-rdma@vger.kernel.org" , "Paul E. McKenney" , "netdev@vger.kernel.org" Date: Wed, 28 Mar 2018 17:42:56 +1100 In-Reply-To: References: <1521854626.16434.359.camel@kernel.crashing.org> <58ce5b83f40f4775bec1be8db66adb0d@AcuMS.aculab.com> <20180326165425.GA15554@ziepe.ca> <20180326202545.GB15554@ziepe.ca> <20180326210951.GD15554@ziepe.ca> <1522101616.7364.13.camel@kernel.crashing.org> <1e077f6a-90b6-cce9-6f0f-a8c003fec850@codeaurora.org> <20180327151029.GB17494@arm.com> <1522186396.7364.61.camel@kernel.crashing.org> <1522198981.7364.81.camel@kernel.crashing.org> <1522211620.7364.94.camel@kernel.crashing.org> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Tue, 2018-03-27 at 20:26 -1000, Linus Torvalds wrote: > On Tue, Mar 27, 2018 at 6:33 PM, Benjamin Herrenschmidt > wrote: > > > > This is why, I want (with your agreement) to define clearly and once > > and for all, that the Linux semantics of writel are that it is ordered > > with previous writes to coherent memory (*) > > Honestly, I think those are the sane semantics. In fact, make it > "ordered with previous writes" full stop, since it's not only ordered > wrt previous writes to memory, but also previous writel's. Of course. It was somewhat a given that it's ordered vs. any previous MMIO actually, but it doesn't hurt to spell it out once more. > > Also, can I assume the above ordering with writel() equally applies to > > readl() or not ? > > > > IE: > > dma_buf->foo = 1; > > readl(STUPID_DEVICE_DMA_KICK_ON_READ); > > If that KICK_ON_READ is UC, then that's definitely the case. And > honestly, status registers like that really should always be UC. > > But if somebody sets the area WC (which is crazy), then I think it > might be at least debatable. x86 semantics does allow reads to be done > before previous writes (or, put another way, writes to be buffered - > the buffers are ordered so writes don't get re-ordered, but reads can > happen during the buffering). Right, for now I worry about UC semantics. Once we have nailed that, we can look at WC, which is a lot more tricky as archs differs more widely, but one thing at a time. > But UC accesses are always done entirely ordered, and honestly, any > status register that starts a DMA would not make sense any other way. > > Of course, you'd have to be pretty odd to want to start a DMA with a > read anyway - partly exactly because it's bad for performance since > reads will be synchronous and not buffered like a write). I have bad memories of old adaptec controllers ... That said, I think the above might not be right on ARM if we want to make it the rule, Will, what do you reckon ? Cheers, Ben.