From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F844C3DA7A for ; Thu, 5 Jan 2023 15:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7CC88E0002; Thu, 5 Jan 2023 10:29:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A04BC8E0001; Thu, 5 Jan 2023 10:29:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87E768E0002; Thu, 5 Jan 2023 10:29:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 75ADF8E0001 for ; Thu, 5 Jan 2023 10:29:50 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1294D80DBE for ; Thu, 5 Jan 2023 15:29:50 +0000 (UTC) X-FDA: 80321130540.11.1E0A6DA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 0385212000A for ; Thu, 5 Jan 2023 15:29:47 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NowIiVab; spf=pass (imf29.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672932588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RK648xItnrxXKGxe6QMGc/XWpXJ5JcGUn31yBFqg+Lg=; b=zN/lqypLrCSuAnghVoFVqMT/bkr0Lvi/YgH5GWwVjbj+GFFRncVo+72OL4z+qkAbQTA34Q jYfPDhOoswcQ+SsnnmWuL5bHSHW5WGJI6xtmTChvRpHyk64COaT6CifSqKBJKRx1B/pAF9 55K+e/iNGBk0hnqF0a0CT3yNbZTFxS8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NowIiVab; spf=pass (imf29.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672932588; a=rsa-sha256; cv=none; b=iCttEavXHIeZyrS84oZ62r8GZYFfefBkghRvAVBDRYrKb+z2LjvQ9an57RUvnzPQceAALC 49U35g5HC27FRK4AtIASuoF888LRP7fUEd3rtaw3jGGb1bFFPta16VMhy9FoSM0e8lLuSv cwkpKpvDdYf6MEHAq5tjwonlPUI6kDo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1672932587; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RK648xItnrxXKGxe6QMGc/XWpXJ5JcGUn31yBFqg+Lg=; b=NowIiVabGe/sZws1z3xMaWcjIKE7o8Bg7boCB9BP85lU6QZQnI/hvzESbAFciAevh+3ieU oRmZRkwtxSBhAiESEqqc3WQDS71da4/vsjSdfeOkXaLg7N9+AsOmOxNzo9vBGJBmWy7REP AAmu8fAqrOWrWXwOHa7i9fN4VGRkZY8= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-540-9ubUJGt2P62BbLoEQcAcTA-1; Thu, 05 Jan 2023 10:29:45 -0500 X-MC-Unique: 9ubUJGt2P62BbLoEQcAcTA-1 Received: by mail-wm1-f70.google.com with SMTP id f20-20020a7bc8d4000000b003d1cda5bd6fso413671wml.9 for ; Thu, 05 Jan 2023 07:29:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RK648xItnrxXKGxe6QMGc/XWpXJ5JcGUn31yBFqg+Lg=; b=jNRS6+cq0ar2zKkkXrLY5FLpeJ5j3X8Kk2fousOuJ6pW6SBEj3DLgRkoByDw6qb0uh rmnjIrqxygUfS9W+trMDFj6vWyTUPof4/r9yNA6uPilWwcjSfgnjVld52Plo+6CHidt3 7FYpj0yNL2bWacQkG8WSIEGeQwv+MxPOMLGc+nhMCeMh3EAnvRytaCT0ZV6S9McN68Us 1CQPomVL6/eVuKRb6O4ddfYENrUJECU7aV7D0ccUxknn6OV0zpc0Vuho/gg6ckpxNa5A UvEynq1+vPfSur0KOuStdAZc62PvDCZXzuOOo169aZrbazS/s/az8ERLTIUsXt9QkSdU Oz/Q== X-Gm-Message-State: AFqh2kqmLcHjrhlCGrvXirkLVK0dOSFGxFgq+WIapvhaDPIaosGF6jzp FmnOxg/r0o7NvSt6upnZUqnrvk34E5a68jTSsYZLol0e+f2pJoKCYYmNsOyLwlTFGXM9F2wfs/h BQYBUkc6IvRs= X-Received: by 2002:a05:600c:3b21:b0:3d9:bad8:4e9e with SMTP id m33-20020a05600c3b2100b003d9bad84e9emr8966760wms.40.1672932584649; Thu, 05 Jan 2023 07:29:44 -0800 (PST) X-Google-Smtp-Source: AMrXdXujEgAJNEc1+gF/eVyy/27BrhmaFM+C8eLfee8wAo6Xcs1n+GlEiEUnqu5t95tSfepWEncVLg== X-Received: by 2002:a05:600c:3b21:b0:3d9:bad8:4e9e with SMTP id m33-20020a05600c3b2100b003d9bad84e9emr8966737wms.40.1672932584395; Thu, 05 Jan 2023 07:29:44 -0800 (PST) Received: from ?IPV6:2003:cb:c707:6e00:ff02:ec7a:ded5:ec1e? (p200300cbc7076e00ff02ec7aded5ec1e.dip0.t-ipconnect.de. [2003:cb:c707:6e00:ff02:ec7a:ded5:ec1e]) by smtp.gmail.com with ESMTPSA id t64-20020a1c4643000000b003cf75213bb9sm2664963wma.8.2023.01.05.07.29.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 05 Jan 2023 07:29:43 -0800 (PST) Message-ID: <797b85c0-ec50-f340-30dd-5a63b51dc45a@redhat.com> Date: Thu, 5 Jan 2023 16:29:41 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: [PATCH 09/46] mm: add MADV_SPLIT to enable HugeTLB HGM To: James Houghton , Mike Kravetz , Muchun Song , Peter Xu Cc: David Rientjes , Axel Rasmussen , Mina Almasry , Zach O'Keefe , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20230105101844.1893104-1-jthoughton@google.com> <20230105101844.1893104-10-jthoughton@google.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: <20230105101844.1893104-10-jthoughton@google.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0385212000A X-Stat-Signature: x63thnuyfr4yj774spgnpninykkz5gmh X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1672932587-522926 X-HE-Meta: U2FsdGVkX18I89iUA5bitSFJeOxejiToTmZtgv91n6bploCcsh2t0/RUMrWYMs/P3aiXh0ED9GGEkQAW70U1J+DkuCU4uhS3WdiGm4WzSF4J0vQukCae7c0y/BLV4M4aTYE64UZwb71NLTDGvAWXZUox2OTIWYFmJI4CcjC//kwtpGxJoXCEueNIQXLEkSbipbxB/mhNY1z4X+Ug2m1akect6GBT7Wk0SZK7sT9JO7YzkZdbbHFUtKXVUxqRJBq/wZ0VzGhhUkWCWIMxMFp5XRM0qafw/7pe7jbmUrovHytcVwNF/fw3Nn0Tsay78ckaVTGLu7+kVXJBQDCyylVDCJuCNTzowCiqzRnwiadEJ5O5DiMFyH7yRkufilKG8XxZv8R7CQm6HUYdbXsw2SecBDmjc5SjZMVAOgZi01MsIapndC1pYeXPleCtkiDgzRqvylPjg3jUbQ+Xc5154wBqnFQftB9MgqN98/1cqpZ2Z4h+isaBfaI6gx7/kbAY+6N5Gs+6yaUoi5ze/KDh6is7HiW3YA7iteYX7bOvFuMhFlB0Ut8Wm2+M3mf3ohdBygCl4NJ5gCwpQ6+DvPmvhyTrqjjZmZk1/pI+KDftBxP58ObmnrpbNVO/+LxzHIPLimHspgzbH2kBiJtvL5lmfBsrJ26GhHVtiB7jSHIVZJsyV32NmtDkhVEQOBhh2C1h51xKBsww7rhsQAl6jbWWDd2iqEexNkPk4JgHbdQwpP5zRxkV0HWvIagirkVImK0i6pg0Hk9kb0MuyMVp14b3gTvsxzyzofxfhcM9Nv3glF7an5/3kBwofkRvenFPxxSEdmYfOvrk1+2+py03kUOVrq6ZemM46kQW3N4cYc3qypjJaxpFoDLKw0bzvsek3afyAsNuAcbaj2N/ZFK9QThxWC6X6qTiU5OnXzKQAQi4v/lBQ8CRAbThjP5/vBx/mi1MeczMOYF+UqvKsHMJxKEJjiY 8WVIgaf4 W8BsTeRZ/DpbngUxAjJLpNOIi85tLnaj7LWJg+L8q1p2Fw40yCIxGeltqIJv7jS3c9VUmaZiWqILX2YPHumv1D6uQnD8D4a/qQuJVrzYziPVoZ9b0fFdCYJGFDgrSAdWVD7ohoT+h5J1pCdxFFrQQJkCHwpDBw4f2tKTTV/PF/MOXAcW0S0K8Az9HRQpHliw3w5gu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 05.01.23 11:18, James Houghton wrote: > Issuing ioctl(MADV_SPLIT) on a HugeTLB address range will enable > HugeTLB HGM. MADV_SPLIT was chosen for the name so that this API can be > applied to non-HugeTLB memory in the future, if such an application is > to arise. > > MADV_SPLIT provides several API changes for some syscalls on HugeTLB > address ranges: > 1. UFFDIO_CONTINUE is allowed for MAP_SHARED VMAs at PAGE_SIZE > alignment. > 2. read()ing a page fault event from a userfaultfd will yield a > PAGE_SIZE-rounded address, instead of a huge-page-size-rounded > address (unless UFFD_FEATURE_EXACT_ADDRESS is used). > > There is no way to disable the API changes that come with issuing > MADV_SPLIT. MADV_COLLAPSE can be used to collapse high-granularity page > table mappings that come from the extended functionality that comes with > using MADV_SPLIT. > > For post-copy live migration, the expected use-case is: > 1. mmap(MAP_SHARED, some_fd) primary mapping > 2. mmap(MAP_SHARED, some_fd) alias mapping > 3. MADV_SPLIT the primary mapping > 4. UFFDIO_REGISTER/etc. the primary mapping > 5. Copy memory contents into alias mapping and UFFDIO_CONTINUE the > corresponding PAGE_SIZE sections in the primary mapping. > > More API changes may be added in the future. > > Signed-off-by: James Houghton > --- > arch/alpha/include/uapi/asm/mman.h | 2 ++ > arch/mips/include/uapi/asm/mman.h | 2 ++ > arch/parisc/include/uapi/asm/mman.h | 2 ++ > arch/xtensa/include/uapi/asm/mman.h | 2 ++ > include/linux/hugetlb.h | 2 ++ > include/uapi/asm-generic/mman-common.h | 2 ++ > mm/hugetlb.c | 3 +-- > mm/madvise.c | 26 ++++++++++++++++++++++++++ > 8 files changed, 39 insertions(+), 2 deletions(-) > > diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h > index 763929e814e9..7a26f3648b90 100644 > --- a/arch/alpha/include/uapi/asm/mman.h > +++ b/arch/alpha/include/uapi/asm/mman.h > @@ -78,6 +78,8 @@ > > #define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */ > > +#define MADV_SPLIT 26 /* Enable hugepage high-granularity APIs */ I think we should make a split more generic, such that it also splits (pte-maps) a THP. Has that been discussed? -- Thanks, David / dhildenb