From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D86D9C6FA89 for ; Thu, 15 Sep 2022 15:21:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231243AbiIOPVs (ORCPT ); Thu, 15 Sep 2022 11:21:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231181AbiIOPVK (ORCPT ); Thu, 15 Sep 2022 11:21:10 -0400 Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58A1A9F8F5; Thu, 15 Sep 2022 08:16:49 -0700 (PDT) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id A87445802F7; Thu, 15 Sep 2022 11:16:46 -0400 (EDT) Received: from imap51 ([10.202.2.101]) by compute3.internal (MEProxy); Thu, 15 Sep 2022 11:16:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arndb.de; h=cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1663255006; x=1663258606; bh=iyfcgLT49L WSRff5Kts3C38d2554ZyUqP/64q59S5es=; b=W4TR4q+Dd0PqyZ/30nmu36Fr7i TSksvzLjmVIjZ+CSNgQdnADbEKSPqivzfMvKx6Bod07yFXxuIJVr2oz7v02ed130 8znPURVzseoVO3T/aUfoRUcV6M6x8VXtVG84PvMfX25FxRDv94nw9VdLu1U3tJLp UGKIA0TIidS5y1QxDVjpTP2bzsMnz/ElL+N9QT0dWnvrZUoYw8u0aceIjKvuXVUF ZTzJeCuiv+BbF0j9uhkK0UcZeVRoCUP89Wl6xcqbfXq6sCcZNpvFoKUjEMkcUxW0 ebz+nEmLHkObmKDmG/cfXcdPA8Q8z5h/TFBEnwDQ3zUWyFDS4lgHqOC1YGOA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1663255006; x=1663258606; bh=iyfcgLT49LWSRff5Kts3C38d2554 ZyUqP/64q59S5es=; b=DXOEfnQrCvp+dWZ0aS1+kLO+aXvqp4oHRwqiR0UJJsbu WRyMDwQHtJrMvHpObfPrH6rWG2+mawyUyQiLS2PVYGZgB9VM8f51Zj89U96w6jFt IfK23yDXab185lAGsbSZc8yeAohlhNk9jhUqLkQNPeOVRnOBSjmQJtYspfiPsEmr DdDte64EyF8l98PljFHTKn/NcgM863PfO0VACcWybNQ4Vl+wHJfV1Qen8XOPQC6X XDZDKX5Qs2f4wg6QH1YUAZdvhYDkB6TSOjH23pYoevULaLPr9waNuSA9O34Vd2pR CEQH3L0uN6fNpir/k7x1r5vRKq3zjPRxwZAqsgaFJA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfedukedgkeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepfdetrhhn ugcuuegvrhhgmhgrnhhnfdcuoegrrhhnugesrghrnhgusgdruggvqeenucggtffrrghtth gvrhhnpeffgeffuddtvdehffefleethfejjeegvdelffejieegueetledvtedtudelgfdu gfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrrh hnugesrghrnhgusgdruggv X-ME-Proxy: Feedback-ID: i56a14606:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 1CE8CB60086; Thu, 15 Sep 2022 11:16:45 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-935-ge4ccd4c47b-fm-20220914.001-ge4ccd4c4 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <20220915050106.650813-1-parav@nvidia.com> <96457b14-e196-4f29-be9a-7fa25ac805d9@www.fastmail.com> Date: Thu, 15 Sep 2022 17:16:24 +0200 From: "Arnd Bergmann" To: "Parav Pandit" , "stern@rowland.harvard.edu" , "parri.andrea@gmail.com" , "Will Deacon" , "Peter Zijlstra" , "boqun.feng@gmail.com" , "Nicholas Piggin" , "dhowells@redhat.com" , "j.alglave@ucl.ac.uk" , "luc.maranget@inria.fr" , "Paul E. McKenney" , "akiyks@gmail.com" , "Dan Lustig" , "joel@joelfernandes.org" , "Jonathan Corbet" , "linux-kernel@vger.kernel.org" , Linux-Arch , "linux-doc@vger.kernel.org" Subject: Re: [PATCH] locking/memory-barriers.txt: Improve documentation for writel() usage Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-arch@vger.kernel.org On Thu, Sep 15, 2022, at 4:18 PM, Parav Pandit wrote: >> From: Arnd Bergmann >> Sent: Thursday, September 15, 2022 8:38 AM >> >> On Thu, Sep 15, 2022, at 7:01 AM, Parav Pandit wrote: >> > The cited commit [1] describes that when using writel(), explcit wmb() >> > is not needed. However, it should have said that dma_wmb() is not >> > needed. >> >> Are you sure? As I understand it, the dma_wmb() only serializes a set of >> memory accesses, but does not serialized against an MMIO access, which >> depending on the CPU architecture may require a different type of barrier. >> >> E.g. on arm, writel() uses __iowmb(), which like wmb() is defined as "dsb(x); >> arm_heavy_mb();", while dma_wmb() is a "dmb(oshst)". > > You are right, on arm heavy barrier dsb() is needed, while on arm64, > dmb(oshst) is sufficient. > > So more accurate documentation is to say that > 'when using writel() a prior IO barrier is not needed ...' > > How about that? That's probably fine, not sure if it's worth changing. > It started with my cleanup efforts to two drivers [1] and [2] that had > difficulty in using writel() on 32-bit system, and it ended up open > coding writel() as wmb() + mlx5_write64(). > > I am cleaning up the repetitive pattern of, > wmb(); > mlx5_write64() > > Before I fix drivers, I thought to improve the documentation that I can > follow. :) Right, that is definitely a good idea. I see that there is more going on with that function, at least the loop in post_send_nop() probably just wants to use __iowrite64_copy(), but that also has no barrier in it, while changing mlx5_write64() to use iowrite64be() or similar would of course add excessive barriers inside of the loop. Arnd