From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A4F3C2D0E4 for ; Mon, 23 Nov 2020 16:32:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B589D20665 for ; Mon, 23 Nov 2020 16:32:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="JcEmEqmZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B589D20665 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 005856B00B4; Mon, 23 Nov 2020 11:32:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EF5936B00B6; Mon, 23 Nov 2020 11:32:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBFDD6B00B7; Mon, 23 Nov 2020 11:32:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id AC0216B00B4 for ; Mon, 23 Nov 2020 11:32:37 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 52D2B181AEF21 for ; Mon, 23 Nov 2020 16:32:37 +0000 (UTC) X-FDA: 77516226354.25.joke84_551691927366 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 2702C1804E3B7 for ; Mon, 23 Nov 2020 16:32:37 +0000 (UTC) X-HE-Tag: joke84_551691927366 X-Filterd-Recvd-Size: 4536 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Nov 2020 16:32:36 +0000 (UTC) Received: by mail-ed1-f47.google.com with SMTP id cq7so17741795edb.4 for ; Mon, 23 Nov 2020 08:32:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=PEbsa5HntcIdBOrGyUXi03wOc7HPqT6OCL9AKHl623g=; b=JcEmEqmZ5YVK3pKO04CD9oQmQCLnELQLzua8f5fXbSIiJnbSxM1z/pJlUKmuvdrff6 stFtRiKKcVo+tzBS2B+a0SPwL0Vizlvm8sKbSv8tqJcb3iVYLg557b9ZVNGpZlh1+Pmr kRVtCqShkcbuNliqdPmADaMb1KQr2MEPncsHK5QB3G+qH110ElyaG10C7ZRvZm+KE8Nd TfjZyeq3+gmJis6cKNWZ/N6mqrWENXzdHRxmfvKI+4mnBdFP0mW78bFvsirFFTgH5o01 6K6PcEQWA6Tongd41DiZaj5w5lpGxcDvg1jYwfT8pIPM2HyUD1m1ewAnXWMgh+63NcsP l9Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PEbsa5HntcIdBOrGyUXi03wOc7HPqT6OCL9AKHl623g=; b=CTO4eShXlVdVz14LoYv5GgaF0ZpBt0phlmYFt11oelvvnA00qCdythc6EwGKzuNUEM QWvO34adu5sol/BejfkotE0EPuIFjm7IxBB16uTu/EYZfcyVQiV1coOT8GoumGCPlSK8 UFyg7xE3rvPMw53Tc9yP+HYjSa/yddWE5/qh8vAQd0086g0XrTUtjT2+qQgZGnRZ3MtQ BhDZFp9xPLdRaocsG2a/psslHOngVuyOQvO61QwM1yezOBlEzoE//BeHDqc12wG+6HGp sTbP8ARWgHOPHce/L4I4ifH/NEnaJMKeH3D6sO7LXsmCwBs8mmVXAG30FMvmk2x2rwMz G7Sg== X-Gm-Message-State: AOAM533Ljn03MTYmNTM3w6OUSGhk/RV1pV8EDgyCxEzFR6NuZ5+a8ZMx GuL6TkkXKawXLXwBDw2OxH/AtdJYVIAgNX6aG8Jgdw== X-Google-Smtp-Source: ABdhPJyOWUtbV4XCremShmwksPGxuhOH2vLBSCgXGmGwoVIAY4yigU44H7ddoM3DsgNMnPRHPbGaTG1z2Gf3RGlI1RA= X-Received: by 2002:a05:6402:1777:: with SMTP id da23mr38863edb.116.1606149155388; Mon, 23 Nov 2020 08:32:35 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Pavel Tatashin Date: Mon, 23 Nov 2020 11:31:59 -0500 Message-ID: Subject: Re: Pinning ZONE_MOVABLE pages To: Vlastimil Babka Cc: linux-mm , Andrew Morton , LKML , Michal Hocko , David Hildenbrand , Oscar Salvador , Dan Williams , Sasha Levin , Tyler Hicks , Joonsoo Kim , sthemmin@microsoft.com, John Hubbard Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > Makes sense, as this means no userspace change. > > > 2. Add an internal move_pages_zone() similar to move_pages() syscall > > but instead of migrating to a different NUMA node, migrate pages from > > ZONE_MOVABLE to another zone. > > Call move_pages_zone() on demand prior to pinning pages from > > vfio_pin_map_dma() for instance. > > As others already said, migrating away before the longterm pin should be > the solution. IIRC it was one of the goals of long term pinning api > proposed long time ago by Peter Ziljstra I think? The implementation > that was merged relatively recently doesn't do that (yet?) for all > movable pages, just CMA, but it could. >From what I can tell, CMA is not solving exactly this problem. It migrates pages from CMA before pinning, but it migrates them to ZONE_MOVABLE. Also, we still need to take care of the fault scenario. > > > 3. Perhaps, it also makes sense to add madvise() flag, to allocate > > pages from non-movable zone. When a user application knows that it > > will do DMA mapping, and pin pages for a long time, the memory that it > > allocates should never be migrated or hot-removed, so make sure that > > it comes from the appropriate place. > > The benefit of adding madvise() flag is that we won't have to deal > > with slow page migration during pin time, but the disadvantage is that > > we would need to change the user interface. > > It's best if we avoid involving userspace until it's shown that's it's > insufficient. Agree. Thank you, Pasha