From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 745F4C2D0E3 for ; Tue, 15 Sep 2020 23:19:19 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E6D6206F4 for ; Tue, 15 Sep 2020 23:19:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="F0pcdLBl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E6D6206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.crashing.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Mime-Version:References:In-Reply-To:Date:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=4JoLOmNqR4kismC98da4fSJkX+dhE0Y53MGRFa4xxWo=; b=F0pcdLBl5XO4E8gn+nQqnBFLW 0Asm1fw/RCLUdnZ4Ee0oL7RoA9rknmM1oYRIeCt3YhLsxjx/meJBpQHrk+57g2/HBXK91Clt3BAAL AM7z2zucG240A4fEjR2Ja7rPZpSiEPatswhvqfk+myRUqz4HCuS4E+/4Bt/OWLpBzPs3wrJ14YO5N 9iuE3hm4gujc6BpgwGKPQM+adT6jmo8Ukp286lcXuR7AJfa3d/AJkPRcXk67WFSsg19FnAxI9Jp/Y uEfDsCB3x9YsddqDdhPTDVnGPuY2OWDTvlue15nVg0f5nincgmbAjdJACrg7BZPb4pADaSu6JloRv pn6o6TtkA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIKD3-0001YI-OP; Tue, 15 Sep 2020 23:18:01 +0000 Received: from kernel.crashing.org ([76.164.61.194]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kIKD1-0001Xr-4H for linux-arm-kernel@lists.infradead.org; Tue, 15 Sep 2020 23:17:59 +0000 Received: from localhost (gate.crashing.org [63.228.1.57]) (authenticated bits=0) by kernel.crashing.org (8.14.7/8.14.7) with ESMTP id 08FNHd9O012611 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 15 Sep 2020 18:17:42 -0500 Message-ID: Subject: Re: [PATCH] arm64: Enable PCI write-combine resources under sysfs From: Benjamin Herrenschmidt To: Jason Gunthorpe , Lorenzo Pieralisi Date: Wed, 16 Sep 2020 09:17:38 +1000 In-Reply-To: <20200915110511.GQ904879@nvidia.com> References: <3110e00a1f4df7b7359ba4f2b7f86a35aa47405e.camel@kernel.crashing.org> <20200911214225.hml2wbbq2rofn4re@amazon.com> <20200914141726.GA904879@nvidia.com> <20200914142406.k44zrnp2wdsandsp@amazon.com> <20200914143819.GC904879@nvidia.com> <375c478593945a416f3180c3773bcb5240d2e36c.camel@kernel.crashing.org> <1d6f2ceb8d3538c906a1fdb8cd3d4c74ccffa42e.camel@kernel.crashing.org> <20200914225740.GP904879@nvidia.com> <2b539df4c9ec703458e46da2fc879ee3b310b31c.camel@kernel.crashing.org> <20200915101831.GA2616@e121166-lin.cambridge.arm.com> <20200915110511.GQ904879@nvidia.com> X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.2 Mime-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200915_191759_290523_BED79F8D X-CRM114-Status: GOOD ( 20.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Leon Romanovsky , linux-pci@vger.kernel.org, Bjorn Helgaas , catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, Clint Sbisa Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 2020-09-15 at 08:05 -0300, Jason Gunthorpe wrote: > > To sum it up: > > > > (1) RDMA drivers need a new mapping function/attribute to define their > > message push model. Actually the message model is not necessarily related > > to write combining a la x86, so we should probably come up with a better > > and consistent naming. Enabling this patchset may trigger performance > > regressions on mellanox drivers on arm64 - this ought to be > > addressed. > > It is pretty clear now that the certain ARM chips that don't do write > combining with pgprot_writecombine will performance regress if they > are running a certain uncommon Mellanox configuration. I suspect these > deployments are all running the out of tree patch for DEVICE_GRE > though. I'm not sure I understand... Today those ARM chips will not use pgprot_writecombine (at least not using that code path, they might still use it as the result of the other path in the driver that can enable it). So they get MT_DEVICE_nGnRnE (unless I missed something here). So they will not combine. With the patch, those device will now use MT_DEVICE_NC. Why would that be a regression ? It will allow speculation, that doesn't necessarily mean that the CPU will cause spurrious accesses, it probably won't in most case... And it should allow combining, no ? BTW. Lorenzo, why don't we use MT_DEVICE_GRE for pgprot_writecombine ? Its not supported on some chips ? Not that this lead me to discover annother weird thing ... What on earth is pgprot_device() ? This is new ? On ARM it will be MT_DEVICE_nGnRE, so it allows posted write. It seems to match what ioremap does. Should then ioremap use it as well ? But it's only ever used for PCI mmap. Why is it different from pgprot_noncached() which disables posted writes (nE) ? Because a whole lot of drivers will use pgprot_noncached() explicitly in either mmap or vmap, with the expectation that it's somewhat the same as what ioremap does... Cheers, Ben. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel