From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E210C43214 for ; Wed, 1 Sep 2021 08:29:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 06CD561054 for ; Wed, 1 Sep 2021 08:29:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 06CD561054 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 9FB916B006C; Wed, 1 Sep 2021 04:29:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9AB3A6B0071; Wed, 1 Sep 2021 04:29:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C1EE8D0001; Wed, 1 Sep 2021 04:29:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0113.hostedemail.com [216.40.44.113]) by kanga.kvack.org (Postfix) with ESMTP id 7F3E26B006C for ; Wed, 1 Sep 2021 04:29:29 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 479CB27DCA for ; Wed, 1 Sep 2021 08:29:29 +0000 (UTC) X-FDA: 78538330458.05.E8184E3 Received: from verein.lst.de (verein.lst.de [213.95.11.211]) by imf26.hostedemail.com (Postfix) with ESMTP id 9063720019CD for ; Wed, 1 Sep 2021 08:29:28 +0000 (UTC) Received: by verein.lst.de (Postfix, from userid 2407) id 711DE68AFE; Wed, 1 Sep 2021 10:29:25 +0200 (CEST) Date: Wed, 1 Sep 2021 10:29:25 +0200 From: Christoph Hellwig To: Felix Kuehling Cc: Christoph Hellwig , "Sierra Guiza, Alejandro (Alex)" , akpm@linux-foundation.org, linux-mm@kvack.org, rcampbell@nvidia.com, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, jgg@nvidia.com, jglisse@redhat.com Subject: Re: [PATCH v1 03/14] mm: add iomem vma selection for memory migration Message-ID: <20210901082925.GA21961@lst.de> References: <20210825034828.12927-1-alex.sierra@amd.com> <20210825034828.12927-4-alex.sierra@amd.com> <20210825074602.GA29620@lst.de> <20210830082800.GA6836@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) X-Rspamd-Queue-Id: 9063720019CD Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=none; spf=none (imf26.hostedemail.com: domain of hch@lst.de has no SPF policy when checking 213.95.11.211) smtp.mailfrom=hch@lst.de X-Rspamd-Server: rspam01 X-Stat-Signature: 66wub7bj373pkrfteuhch8ohemkjcutw X-HE-Tag: 1630484968-143978 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Aug 30, 2021 at 01:04:43PM -0400, Felix Kuehling wrote: > >> driver code is not really involved in updating the CPU mappings. Maybe > >> it's something we need to do in the migration helpers. > > It looks like I'm totally misunderstanding what you are adding here > > then. Why do we need any special treatment at all for memory that > > has normal struct pages and is part of the direct kernel map? > > The pages are like normal memory for purposes of mapping them in CPU > page tables and for coherent access from the CPU. That's the user page tables. What about the kernel direct map? If there is a normal kernel struct page backing there really should be no need for the pgmap. > From an application > perspective, we want file-backed and anonymous mappings to be able to > use DEVICE_PUBLIC pages with coherent CPU access. The goal is to > optimize performance for GPU heavy workloads while minimizing the need > to migrate data back-and-forth between system memory and device memory. I don't really understand that part. file backed pages are always allocated by the file system using the pagecache helpers, that is using the page allocator. Anonymouns memory also always comes from the page allocator. > The pages are special in two ways: > > 1. The memory is managed not by the Linux buddy allocator, but by the > GPU driver's TTM memory manager Why? > 2. We want to migrate data in response to GPU page faults and > application hints using the migrate_vma helpers Why?