From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA177C433FF for ; Wed, 7 Aug 2019 16:50:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A113E2229C for ; Wed, 7 Aug 2019 16:50:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="BoNYFbqV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A113E2229C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=T5fV8q1cJ1mrIcLUjuMSeIpbyRwUljVLflYgta0th0U=; b=BoNYFbqVCACNcO Ft7HNy9DvyxaI8EjfzieEyr2ADjm8qTXnZxel8kG8T1MDrFhoTkKsCpTa10Hg3CfOVIGWAxmVeXpu LczqsLvdBkybUwLIkv2FjO3KQWl/h4o0H8x8wvuAAXhAsbMzZXlaqA+79MIi1VmC58TO7zVQ2hcl0 ueSj9Z781q2iThHC+TOJdx/0cp4QgXlKNrIDN1BBxSqlH6xypQjF3HLjvP7rRkxoNSQB/AD3YJJMH vAShxbFdk6hIe9Oi3rX9wRGu1UcDChs+D3Z3UMEQYnqtXoo6zKZckwyIpOOur01EqsSLB0+Gwr0Cm nsO2JNjc4w0yU9SiHpJQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hvP8d-00026N-Qp; Wed, 07 Aug 2019 16:50:11 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hvP8Z-00023N-VA for linux-arm-kernel@lists.infradead.org; Wed, 07 Aug 2019 16:50:09 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7600C344; Wed, 7 Aug 2019 09:50:06 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 711623F694; Wed, 7 Aug 2019 09:50:04 -0700 (PDT) Date: Wed, 7 Aug 2019 17:49:59 +0100 From: Mark Rutland To: Rob Clark Subject: Re: [PATCH 1/2] drm: add cache support for arm64 Message-ID: <20190807164958.GA44765@lakrids.cambridge.arm.com> References: <20190805211451.20176-1-robdclark@gmail.com> <20190806084821.GA17129@lst.de> <20190806143457.GF475@lakrids.cambridge.arm.com> <20190807123807.GD54191@lakrids.cambridge.arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.1+11 (2f07cb52) (2018-12-01) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190807_095008_097364_00886EB6 X-CRM114-Status: GOOD ( 28.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sean Paul , Maxime Ripard , Catalin Marinas , Maarten Lankhorst , LKML , dri-devel , David Airlie , Rob Clark , linux-arm-kernel@lists.infradead.org, Daniel Vetter , Greg Kroah-Hartman , Thomas Gleixner , Will Deacon , Christoph Hellwig , Allison Randal Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Aug 07, 2019 at 09:15:54AM -0700, Rob Clark wrote: > On Wed, Aug 7, 2019 at 5:38 AM Mark Rutland wrote: > > > > On Tue, Aug 06, 2019 at 09:31:55AM -0700, Rob Clark wrote: > > > On Tue, Aug 6, 2019 at 7:35 AM Mark Rutland wrote: > > > > > > > > On Tue, Aug 06, 2019 at 07:11:41AM -0700, Rob Clark wrote: > > > > > On Tue, Aug 6, 2019 at 1:48 AM Christoph Hellwig wrote: > > > > > > > > > > > > This goes in the wrong direction. drm_cflush_* are a bad API we need to > > > > > > get rid of, not add use of it. The reason for that is two-fold: > > > > > > > > > > > > a) it doesn't address how cache maintaince actually works in most > > > > > > platforms. When talking about a cache we three fundamental operations: > > > > > > > > > > > > 1) write back - this writes the content of the cache back to the > > > > > > backing memory > > > > > > 2) invalidate - this remove the content of the cache > > > > > > 3) write back + invalidate - do both of the above > > > > > > > > > > Agreed that drm_cflush_* isn't a great API. In this particular case > > > > > (IIUC), I need wb+inv so that there aren't dirty cache lines that drop > > > > > out to memory later, and so that I don't get a cache hit on > > > > > uncached/wc mmap'ing. > > > > > > > > Is there a cacheable alias lying around (e.g. the linear map), or are > > > > these addresses only mapped uncached/wc? > > > > > > > > If there's a cacheable alias, performing an invalidate isn't sufficient, > > > > since a CPU can allocate a new (clean) entry at any point in time (e.g. > > > > as a result of prefetching or arbitrary speculation). > > > > > > I *believe* that there are not alias mappings (that I don't control > > > myself) for pages coming from > > > shmem_file_setup()/shmem_read_mapping_page().. > > > > AFAICT, that's regular anonymous memory, so there will be a cacheable > > alias in the linear/direct map. > > tbh, I'm not 100% sure whether there is a cacheable alias, or whether > any potential linear map is torn down. I'm fairly confident that the linear/direct map cacheable alias is not torn down when pages are allocated. The gneeric page allocation code doesn't do so, and I see nothing the shmem code to do so. For arm64, we can tear down portions of the linear map, but that has to be done explicitly, and this is only possible when using rodata_full. If not using rodata_full, it is not possible to dynamically tear down the cacheable alias. > My understanding is that a cacheable alias is "ok", with some > caveats.. ie. that the cacheable alias is not accessed. Unfortunately, that is not true. You'll often get away with it in practice, but that's a matter of probability rather than a guarantee. You cannot prevent a CPU from accessing a VA arbitrarily (e.g. as the result of wild speculation). The ARM ARM (ARM DDI 0487E.a) points this out explicitly: | Entries for addresses that are Normal Cacheable can be allocated to | the cache at any time, and so the cache invalidate instruction cannot | ensure that the address is not present in a cache. ... along with: | Caches introduce a number of potential problems, mainly because: | | * Memory accesses can occur at times other than when the programmer | would expect them. ;) > I'm not entirely sure about pre-fetch from access to adjacent pages. > We have been using shmem as a source for pages since the beginning, > and I haven't seen it cause any problems in the last 6 years. (This > is limited to armv7 and armv8, I'm not really sure what would happen > on armv6, but that is a combo I don't have to care about.) Over time, CPUs get more aggressive w.r.t. prefetching and speculation, so having not seen such issues in the past does not imply we won't in future. Anecdotally, we had an issue a few years ago that we were only able to reproduce under heavy stress testing. A CPU would make speculative instruction fetches from a read-destructive MMIO register, despite the PC never going near the corresponding VA, and despite that code having (apparently) caused no issue for years before that. See commit: b6ccb9803e90c16b ("ARM: 7954/1: mm: remove remaining domain support from ARMv6") ... which unfortunately lacks the full war story. Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel