From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED,USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 694D3C43444 for ; Mon, 14 Jan 2019 19:14:04 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 394FC206B7 for ; Mon, 14 Jan 2019 19:14:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="a4ZcXEUD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 394FC206B7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Gk+6Z8B/87Y5uvApUhoqQgfmO3+QPR7q6xqb012YFUg=; b=a4ZcXEUDnfkihm EcbcjXz6oHgnVSlcF6rrj/V3stdxOTzqBistdHztpJJj1u9BXCJcGHzDlvLHGNJLMghFGpUKsh1iy KUtG4uQuDPF5giz3FMoH4J1tlo1Rq5a/QO7Cz8EqB7/F6HB6/LS2GAmT9DfDlYftwOFtIzEZeLJlQ 2+piD4m+5y4LqDoZpf3O3pdoccMUMaunEuVZVrexiWXlTRlNZmkguehY2ZrgGAL3dzvkmP42pt5jQ rE50vedYbCR77Y1eVhRQUi/KZ7P/O5ILHg/k7v4POUw2zoEopUOLkeoiNialPAlUikmYjSBdfndi1 8P+KniMBJWG/axFg9DNQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gj7gQ-0006Po-0E; Mon, 14 Jan 2019 19:14:02 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gj7gM-0006PG-Ex for linux-arm-kernel@lists.infradead.org; Mon, 14 Jan 2019 19:13:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6E13E80D; Mon, 14 Jan 2019 11:13:57 -0800 (PST) Received: from fuggles.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7A4283F5BD; Mon, 14 Jan 2019 11:13:55 -0800 (PST) Date: Mon, 14 Jan 2019 19:13:50 +0000 From: Will Deacon To: "Koenig, Christian" Subject: Re: [RFC PATCH] drm/ttm: force cached mappings for system RAM on ARM Message-ID: <20190114191350.GA29600@fuggles.cambridge.arm.com> References: <20190110072841.3283-1-ard.biesheuvel@linaro.org> <5d8135de-80fe-9c0e-2206-ecb809f64cdb@daenzer.net> <55facfb9-92af-86b8-40e9-d63b887b5592@amd.com> <9f956898-7973-98ee-6bf1-e1d445e9d365@amd.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <9f956898-7973-98ee-6bf1-e1d445e9d365@amd.com> User-Agent: Mutt/1.11.1+86 (6f28e57d73f2) () X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190114_111358_514154_24A3FE9A X-CRM114-Status: GOOD ( 25.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Carsten Haitzler , Ard Biesheuvel , David Airlie , Michel =?iso-8859-1?Q?D=E4nzer?= , Linux Kernel Mailing List , dri-devel , "Huang, Ray" , "Zhang, Jerry" , linux-arm-kernel , Bernhard =?iso-8859-1?Q?Rosenkr=E4nzer?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Jan 14, 2019 at 07:07:54PM +0000, Koenig, Christian wrote: > Am 14.01.19 um 18:32 schrieb Ard Biesheuvel: > - The reason remapping the CPU side as cacheable does work (which I > did test) is because the GPU's uncacheable accesses (which I assume > are made using the NoSnoop PCIe transaction attribute) are actually > emitted as cacheable in some cases. > . On my AMD Seattle, with or without SMMU (which is stage 2 only), I > must use cacheable accesses from the CPU side or things are broken. > This might be a h/w flaw, though. > . On systems with stage 1+2 SMMUs, the driver uses stage 1 > translations which always override the memory attributes to cacheable > for DMA coherent devices. This is what is affecting the Cavium > ThunderX2 (although it appears the attributes emitted by the RC may be > incorrect as well.) > > The latter issue is a shortcoming in the SMMU driver that we have to > fix, i.e., it should take care not to modify the incoming attributes > of DMA coherent PCIe devices for NoSnoop to be able to work. > > So in summary, the mismatch appears to be between the CPU accessing > the vmap region with non-cacheable attributes and the GPU accessing > the same memory with cacheable attributes, resulting in a loss of > coherency and lots of visible corruption. > > Actually it is the other way around. The CPU thinks some data is in the > cache and the GPU only updates the system memory version because the > snoop flag is not set. > > > That doesn't seem to be what is happening. As far as we can tell from > our experiments, all inbound transactions are always cacheable, and so > the only way to make things work is to ensure that the CPU uses the > same attributes. > > > Ok that doesn't make any sense. If inbound transactions are cacheable or not is > irrelevant when the CPU always uses uncached accesses. > > See on the PCIe side you have the snoop bit in the read/write transactions > which tells the root hub if the device wants to snoop caches or not. > > When the CPU accesses some memory as cached then devices need to snoop the > cache for coherent accesses. > > When the CPU accesses some memory as uncached then devices can disable snooping > to improve performance, but when they don't do this it is mandated by the spec > that this still works. Which spec? The Arm architecture (and others including Power afaiu) doesn't guarantee coherency when memory is accessed using mismatched cacheability attributes. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel