From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD0D0C433DF for ; Tue, 18 Aug 2020 10:09:38 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8049D2065D for ; Tue, 18 Aug 2020 10:09:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LUU/3bg7"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="zEdB5WL+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8049D2065D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CU1rfgMQi9ojB7oeQ3qOa51cfwVi567oqypNpzFTD44=; b=LUU/3bg70jSNo1JyDxXJtzbkI qsH/e1REIxKLpI98SZ1RgysH2vPzKfVXcuYJLUywknGrk5L0/igao6xF+0lq+uVsl4o1r91d8WPrL 3dtRK1w89+2VlKKwNEvTuDUpv9LF7+rtRAbEsHV6B4lY8j5zXTf9RhuPth8sK2s3Cdvb7+MQxA9y1 ITGcnBik7P8j6IyjLyR5jxturoPafTMfedYV0JZL6Ilf4xLD8CDKyWPh/u2nQWpXR8MvwzXPLCXD7 q7KPxoYsfIgOnd8TW1yR2SrYy+L8a/w9FcUW4aN9oLtIO1WqzBHj4Qmi39sf5R15hoOtM6J5zydDy SaZdk64pA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k7yXH-0001aU-0t; Tue, 18 Aug 2020 10:08:07 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k7yXE-0001Zp-GO for linux-arm-kernel@lists.infradead.org; Tue, 18 Aug 2020 10:08:05 +0000 Received: from willie-the-truck (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A2C5B2065D; Tue, 18 Aug 2020 10:08:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1597745283; bh=b6X4T9udwsSd8aIqKl0IHJmnTP0SOrvoOTKnY0xll6c=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=zEdB5WL++ak9OSgezaNr3YiCb55aM1Q3nYsIX2i6FxaocZJOY5sqqKhAQ1+xM5XWU xlg98bPTKEXvlRXwveeRkvUzrR4e4FTaM4iwg89m00sEvKQIQT7f8rK2cOaGN59erm K1uKoju7tI+HYGu+bwAl8BloAFoVONlAeQqrCzXU= Date: Tue, 18 Aug 2020 11:07:57 +0100 From: Will Deacon To: Cho KyongHo Subject: Re: [PATCH 1/2] dma-mapping: introduce relaxed version of dma sync Message-ID: <20200818100756.GA15543@willie-the-truck> References: <1597736591-20457-1-git-send-email-pullip.cho@samsung.com> <20200818082852.GA15145@willie-the-truck> <20200818093739.GB191752@KEI> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200818093739.GB191752@KEI> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200818_060804_700450_8C360608 X-CRM114-Status: GOOD ( 30.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: janghyuck.kim@samsung.com, catalin.marinas@arm.com, joro@8bytes.org, linux-kernel@vger.kernel.org, hyesoo.yu@samsung.com, iommu@lists.linux-foundation.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org, m.szyprowski@samsung.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Aug 18, 2020 at 06:37:39PM +0900, Cho KyongHo wrote: > On Tue, Aug 18, 2020 at 09:28:53AM +0100, Will Deacon wrote: > > On Tue, Aug 18, 2020 at 04:43:10PM +0900, Cho KyongHo wrote: > > > Cache maintenance operations in the most of CPU architectures needs > > > memory barrier after the cache maintenance for the DMAs to view the > > > region of the memory correctly. The problem is that memory barrier is > > > very expensive and dma_[un]map_sg() and dma_sync_sg_for_{device|cpu}() > > > involves the memory barrier per every single cache sg entry. In some > > > CPU micro-architecture, a single memory barrier consumes more time than > > > cache clean on 4KiB. It becomes more serious if the number of CPU cores > > > are larger. > > > > Have you got higher-level performance data for this change? It's more likely > > that the DSB is what actually forces the prior cache maintenance to > > complete, > > This patch does not skip necessary DSB after cache maintenance. It just > remove repeated dsb per every single sg entry and call dsb just once > after cache maintenance on all sg entries is completed. Yes, I realise that, but what I'm saying is that a big part of your justification for this change is: | The problem is that memory barrier is very expensive and dma_[un]map_sg() | and dma_sync_sg_for_{device|cpu}() involves the memory barrier per every | single cache sg entry. In some CPU micro-architecture, a single memory | barrier consumes more time than cache clean on 4KiB. and my point is that the DSB is likely completing the cache maintenance, so as cache maintenance instructions retire faster in the micro-architecture, the DSB becomes absolutely slower. In other words, it doesn't make much sense to me to compare the cost of the DSB with the cost of the cache maintenance; what matters more is the code of the high-level unmap() operation for the sglist. > > so it's important to look at the bigger picture, not just the > > apparent relative cost of these instructions. > > > If you mean bigger picture is the performance impact of this patch to a > complete user scenario, we are evaluating it in some latency sensitve > scenario. But I wonder if a performance gain in a platform/SoC specific > scenario is also persuasive. Latency is fine too, but phrasing the numbers (and we really need those) in terms of things like "The interrupt response time for this in-tree driver is improved by xxx ns (yy %) after this change" or "Throughput for this in-tree driver goes from xxx mb/s to yyy mb/s" would be really helpful. > > Also, it's a miracle that non-coherent DMA even works, > > I am sorry, Will. I don't understand this. Can you let me know what do > you mena with the above sentence? Non-coherent DMA sucks for software. For the most part, Linux does a nice job of hiding this from device drivers, and I think _that_ is the primary concern, rather than performance. If performance is a problem, then the solution is cache coherence or a shared non-cacheable buffer (rather than the streaming API). > > so I'm not sure > > that we should be complicating the implementation like this to try to > > make it "fast". > > > I agree that this patch makes the implementation of dma API a bit more > but I don't think this does not impact its complication seriously. It's death by a thousand cuts; this patch further fragments the architecture backends and leads to arm64-specific behaviour which consequently won't get well tested by anybody else. Now, it might be worth it, but there's not enough information here to make that call. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel