From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 595B6C61D85 for ; Wed, 22 Nov 2023 01:29:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E70866B0544; Tue, 21 Nov 2023 20:29:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E1F986B0545; Tue, 21 Nov 2023 20:29:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE7CA6B0547; Tue, 21 Nov 2023 20:29:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C26B86B0544 for ; Tue, 21 Nov 2023 20:29:03 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 996BBC0150 for ; Wed, 22 Nov 2023 01:29:03 +0000 (UTC) X-FDA: 81483856566.09.14296D2 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf23.hostedemail.com (Postfix) with ESMTP id D1EC9140004 for ; Wed, 22 Nov 2023 01:29:01 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=tLGlBONm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf23.hostedemail.com: domain of pcc@google.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=pcc@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700616541; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PQTXNqNKjwBGV/F0nq07Eb76dHmbBkOBd/hKqiZRe4I=; b=hLak9NKYPgEjVkt5EL8WKEMG4mtEoMzFhfL09+RhzrgfqZ/vtQcZ2PiV2zLhItXN++Yyx0 DYVdBfknWhM0gIyDPEhhN40XEaoJemVMONxMWPDUXRUP9M/jfSTShNCQwU9ZQlFQSqwiN1 M+XuSk4OqTwEVCAXwBAALbFgGvYVoYQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=tLGlBONm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf23.hostedemail.com: domain of pcc@google.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=pcc@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700616541; a=rsa-sha256; cv=none; b=6nXtte+yqfy1MN4WE0wY1jOBikkaHB2H3kJmNlvMif3Y9rM6A8x04ji+enMoN1T9xSMNFJ D9AfekJV8D3qcc+N29SHOsuJ49g9LH9OHOuc0l3IcX7egbeT9XUNVy7fiQOlMHMvbY6TNj oghEitsnnOnYMfS8RdRJSiwPMogSEno= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1cc79f73e58so44795ad.1 for ; Tue, 21 Nov 2023 17:29:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1700616540; x=1701221340; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=PQTXNqNKjwBGV/F0nq07Eb76dHmbBkOBd/hKqiZRe4I=; b=tLGlBONmvXvEjAz9R4wDTPnXWWPbR3IGZFdw520732wa8+uBVTOGDRct+6CaM+TyoM QZQtz9g/dB+7OXwjnevJr0BauISI82Q+FqYCsWb432Ihih1igNSiZc/jdNG/9/VuLHbZ MPrFmyk1l1h3yEA0lj4kH+B4ZLEgEkp0enq95n1YivCZGY0RKtO09LHroma68Nhs8Gzs QzH9+4L/oEKUiEdSa+OMIkp9nNIKeDWkSQnT/vsaITCc+aIcFLWCb8yvjyr87gEXJKVF imPBfjeZMtcppjLN9snfy0CNE4uM4C/pSd1c+6FwXDLcqYP9wr356e2bZ4Y1H/g1eATM Y9nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700616540; x=1701221340; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PQTXNqNKjwBGV/F0nq07Eb76dHmbBkOBd/hKqiZRe4I=; b=fdEX6bYtaMvICu10qidJMqhaUkYKlVzgUoImU77ZBFCQ4Xsc+Z0blmAXOOMSZW8YEH k9TZli6P6vmS01emSnRrPmrvJoiRWec0KgNUj+fQX9i6Ymlb/YS9eYoHnemzAXz8tHLK Fo/VdSn6SbMvZ4GMc2loowaM3ymncxSdTHVXZYwCQ32o8AUjDvxo2PR23z3Mgr2uTN+6 SXBKI5vKhpKE6hOm9atfdxmi4wy39+8PirnZtJpGUhUjci8yc/NMEEMCtwtOj5vJhBLq 6khdouu6QXoj0Fpq8nG1bVHbcEj15xjUVULBa//q53F4vqvXDyzAHaan0GAuLJzx8mRL ypAQ== X-Gm-Message-State: AOJu0Yx2+bWy8xgswzlAq0VzVF5YuzeOOYA+DxbM4JlkNeDtjksQ9vnk 1A/ojlTtCnrTuOfcXYF3AnqX6/x5TUBzPC9ylhTLIA== X-Google-Smtp-Source: AGHT+IHTLY6O8nnNjugZgX1idDgXA1sk1KnuEcL73QkrrMzzt0J7bEBMJ67A4b4lOJtIs84dHwwHlsyEGiBHb8A0s/M= X-Received: by 2002:a17:903:643:b0:1cf:6542:b4c6 with SMTP id kh3-20020a170903064300b001cf6542b4c6mr63497plb.13.1700616540267; Tue, 21 Nov 2023 17:29:00 -0800 (PST) MIME-Version: 1.0 References: <20231119165721.9849-1-alexandru.elisei@arm.com> <20231119165721.9849-21-alexandru.elisei@arm.com> In-Reply-To: <20231119165721.9849-21-alexandru.elisei@arm.com> From: Peter Collingbourne Date: Tue, 21 Nov 2023 17:28:49 -0800 Message-ID: Subject: Re: [PATCH RFC v2 20/27] mm: hugepage: Handle huge page fault on access To: Alexandru Elisei Cc: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D1EC9140004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: r465fjxywzk1mcjhqwrwjgghqt9juhid X-HE-Tag: 1700616541-229811 X-HE-Meta: U2FsdGVkX19gO+G70+rSJYhqps5x3JsSLngQLTK/KAjh+dniF7pIJoPcU/3exhS4/fW95Mm2cDLYDAp5BsSlDnPvAdQK1n1vSdGEYWMbtZibKC3lXxcGZgzU0eDPYKB/VOcFCFrQszuJ2yf0UMkUMWK5Iz7/HqMqwj6K/jtHXmOHAtBqKL/f1MWe9kZegAmNW1aDPzok4NhuDadWD0cS1AP72c1nQyBL3zCt5aXI4jKZ5BYZ7vrCwGV0+PeKFodLqhkh3ayqa0SPaVh8zdnAclHu7GnnPuNRKpEmGyod9ZVm6h02YCpkP8y0x1xaqfsOLhvjMGzyzf0lUsX3BO+s0shh4O+YiWjpRCMpvspxUByrSrI3tQWemNtcix9iPUFsm2UImJ0xRyA18QtWTOt4FgDPcS6RtJUv7hvIYenQeTRFLK3YH9oBSdfFZjPJftlS5JD3Pl90G22V48J6uE2WRv7ZWKmzIrMwa6fLJx7/g298peCnGj5eiQqDQFC9kGK9RVZM6ZGEypRnMB5g9u9J1oocqpOlBQRunx1m3jGgyXyrDtYo5ZstcxQ98nLRjdgZtliHpNZBmgQ0KM/sd+B1ik5nwIs2qs1lswLdKcuzppD3zZg59xP88ZyxYtdCL3SKSvwMXrl6UW6xwUrA6K/iS1jETdq1dbI1I4mgwb5EchJ68tk1RNVfcamgPML6KPtTfRBM8aRAdpT98pY9xJdZkwOSV4GqtA9GAHr2UMBExDmSlTDxnTS8i+Eamm8dl5rq10fwnSY8rsk0Ppzcpha735WiepN8dzDtGBIN8bC/R4my1nNtyCwtOzKM9nWjnNErIqoAM3JS1YI+v8cIbuvuGoc9RPJ1dhwYdAf52DfpN5WZDUOXHWap6pfx7AoQab8IyzSEMLSkKKzgS24mBDDcHeBVgGeQPd/hY7guJ40cJ/UReQYpBPPF3mivtpXk4067bzC9OWgJO5N6y9kAYgI EpmMZiEX f6I9H2pafYSQvR9gaEeqtyUiPhFyRBkaoqDi3ZMqrl0Ovo8ZeBGepGHvwyRhwyBnu35KpFeAlQiEPM3weC4Z0ltiu4/akqyT333ALWFlZLNwTgANN9KtuIB/6XacBPTnoW3tfO+m+zz87TrWH/tSnTGosY+exazpA6kNBt+vNYS6wSRDsi+0pv7Fhjm5aM4iTBpiCiJDGwYSYVBVxNYE0m/WUtBrRo0dVLftCW5QiqbCABhBNT12mklobEoi0b1IPc1x2ErINLLTnoTjwtzsVujDGCy4uROX3a3i8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Nov 19, 2023 at 8:59=E2=80=AFAM Alexandru Elisei wrote: > > Handle PAGE_FAULT_ON_ACCESS faults for huge pages in a similar way to > regular pages. > > Signed-off-by: Alexandru Elisei > --- > arch/arm64/include/asm/mte_tag_storage.h | 1 + > arch/arm64/include/asm/pgtable.h | 7 ++ > arch/arm64/mm/fault.c | 81 ++++++++++++++++++++++++ > include/linux/huge_mm.h | 2 + > include/linux/pgtable.h | 5 ++ > mm/huge_memory.c | 4 +- > mm/memory.c | 3 + > 7 files changed, 101 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/mte_tag_storage.h b/arch/arm64/includ= e/asm/mte_tag_storage.h > index c70ced60a0cd..b97406d369ce 100644 > --- a/arch/arm64/include/asm/mte_tag_storage.h > +++ b/arch/arm64/include/asm/mte_tag_storage.h > @@ -35,6 +35,7 @@ void free_tag_storage(struct page *page, int order); > bool page_tag_storage_reserved(struct page *page); > > vm_fault_t handle_page_missing_tag_storage(struct vm_fault *vmf); > +vm_fault_t handle_huge_page_missing_tag_storage(struct vm_fault *vmf); > #else > static inline bool tag_storage_enabled(void) > { > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pg= table.h > index 8cc135f1c112..1704411c096d 100644 > --- a/arch/arm64/include/asm/pgtable.h > +++ b/arch/arm64/include/asm/pgtable.h > @@ -477,6 +477,13 @@ static inline vm_fault_t arch_do_page_fault_on_acces= s(struct vm_fault *vmf) > return handle_page_missing_tag_storage(vmf); > return VM_FAULT_SIGBUS; > } > + > +static inline vm_fault_t arch_do_huge_page_fault_on_access(struct vm_fau= lt *vmf) > +{ > + if (tag_storage_enabled()) > + return handle_huge_page_missing_tag_storage(vmf); > + return VM_FAULT_SIGBUS; > +} > #endif /* CONFIG_ARCH_HAS_FAULT_ON_ACCESS */ > > #define pmd_present_invalid(pmd) (!!(pmd_val(pmd) & PMD_PRESENT_INVA= LID)) > diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c > index f5fa583acf18..6730a0812a24 100644 > --- a/arch/arm64/mm/fault.c > +++ b/arch/arm64/mm/fault.c > @@ -1041,6 +1041,87 @@ vm_fault_t handle_page_missing_tag_storage(struct = vm_fault *vmf) > > return 0; > > +out_retry: > + put_page(page); > + if (vmf->flags & FAULT_FLAG_VMA_LOCK) > + vma_end_read(vma); > + if (fault_flag_allow_retry_first(vmf->flags)) { > + err =3D VM_FAULT_RETRY; > + } else { > + /* Replay the fault. */ > + err =3D 0; > + } > + return err; > +} > + > +vm_fault_t handle_huge_page_missing_tag_storage(struct vm_fault *vmf) > +{ > + unsigned long haddr =3D vmf->address & HPAGE_PMD_MASK; > + struct vm_area_struct *vma =3D vmf->vma; > + pmd_t old_pmd, new_pmd; > + bool writable =3D false; > + struct page *page; > + vm_fault_t err; > + int ret; > + > + vmf->ptl =3D pmd_lock(vma->vm_mm, vmf->pmd); > + if (unlikely(!pmd_same(vmf->orig_pmd, *vmf->pmd))) { > + spin_unlock(vmf->ptl); > + return 0; > + } > + > + old_pmd =3D vmf->orig_pmd; > + new_pmd =3D pmd_modify(old_pmd, vma->vm_page_prot); > + > + /* > + * Detect now whether the PMD could be writable; this information > + * is only valid while holding the PT lock. > + */ > + writable =3D pmd_write(new_pmd); > + if (!writable && vma_wants_manual_pte_write_upgrade(vma) && > + can_change_pmd_writable(vma, vmf->address, new_pmd)) > + writable =3D true; > + > + page =3D vm_normal_page_pmd(vma, haddr, new_pmd); > + if (!page) > + goto out_map; > + > + if (!(vma->vm_flags & VM_MTE)) > + goto out_map; > + > + get_page(page); > + vma_set_access_pid_bit(vma); > + > + spin_unlock(vmf->ptl); > + writable =3D false; > + > + if (unlikely(is_migrate_isolate_page(page))) > + goto out_retry; > + > + ret =3D reserve_tag_storage(page, HPAGE_PMD_ORDER, GFP_HIGHUSER_M= OVABLE); > + if (ret) > + goto out_retry; > + > + put_page(page); > + > + vmf->ptl =3D pmd_lock(vma->vm_mm, vmf->pmd); > + if (unlikely(!pmd_same(old_pmd, *vmf->pmd))) { > + spin_unlock(vmf->ptl); > + return 0; > + } > + > +out_map: > + /* Restore the PMD */ > + new_pmd =3D pmd_modify(old_pmd, vma->vm_page_prot); > + new_pmd =3D pmd_mkyoung(new_pmd); > + if (writable) > + new_pmd =3D pmd_mkwrite(new_pmd, vma); > + set_pmd_at(vma->vm_mm, haddr, vmf->pmd, new_pmd); > + update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); > + spin_unlock(vmf->ptl); > + > + return 0; > + > out_retry: > put_page(page); > if (vmf->flags & FAULT_FLAG_VMA_LOCK) > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > index fa0350b0812a..bb84291f9231 100644 > --- a/include/linux/huge_mm.h > +++ b/include/linux/huge_mm.h > @@ -36,6 +36,8 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned= long old_addr, > int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, > pmd_t *pmd, unsigned long addr, pgprot_t newprot, > unsigned long cp_flags); > +bool can_change_pmd_writable(struct vm_area_struct *vma, unsigned long a= ddr, > + pmd_t pmd); > > vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool writ= e); > vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, pfn_t pfn, bool writ= e); > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > index e2c761dd6c41..de45f475bf8d 100644 > --- a/include/linux/pgtable.h > +++ b/include/linux/pgtable.h > @@ -1473,6 +1473,11 @@ static inline vm_fault_t arch_do_page_fault_on_acc= ess(struct vm_fault *vmf) > { > return VM_FAULT_SIGBUS; > } > + > +static inline vm_fault_t arch_do_huge_page_fault_on_access(struct vm_fau= lt *vmf) > +{ > + return VM_FAULT_SIGBUS; > +} > #endif > > #endif /* CONFIG_MMU */ > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index 9beead961a65..d1402b43ea39 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -1406,8 +1406,8 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf= ) > return VM_FAULT_FALLBACK; > } > > -static inline bool can_change_pmd_writable(struct vm_area_struct *vma, > - unsigned long addr, pmd_t pmd) > +inline bool can_change_pmd_writable(struct vm_area_struct *vma, Remove inline keyword here. Peter > + unsigned long addr, pmd_t pmd) > { > struct page *page; > > diff --git a/mm/memory.c b/mm/memory.c > index a04a971200b9..46b926625503 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -5168,6 +5168,9 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, > return 0; > } > if (pmd_trans_huge(vmf.orig_pmd) || pmd_devmap(vmf.orig_p= md)) { > + if (fault_on_access_pmd(vmf.orig_pmd) && vma_is_a= ccessible(vma)) > + return arch_do_huge_page_fault_on_access(= &vmf); > + > if (pmd_protnone(vmf.orig_pmd) && vma_is_accessib= le(vma)) > return do_huge_pmd_numa_page(&vmf); > > -- > 2.42.1 >