From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A2F0C25B10 for ; Mon, 13 May 2024 13:06:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52E506B026B; Mon, 13 May 2024 09:06:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4DF246B0294; Mon, 13 May 2024 09:06:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37EAF6B0295; Mon, 13 May 2024 09:06:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1415D6B026B for ; Mon, 13 May 2024 09:06:54 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5BB8D140EB8 for ; Mon, 13 May 2024 13:06:53 +0000 (UTC) X-FDA: 82113397506.20.884BC87 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf11.hostedemail.com (Postfix) with ESMTP id 5AF6C40022 for ; Mon, 13 May 2024 13:06:50 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=Qrpwlhwl; spf=pass (imf11.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715605610; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VrsGvge7X1e91Ca7nLsARXz/B6rasvGNzsoO0V6EifM=; b=a02MGiPIzI6T5kuJQ7mCnlDpT2InsimbFmliXdSyEDCaNsrMALKhOptdEjkiJdTubAbE5V zg505ZJ891jFOkAwXZ1OgaMT/67K+kCabvtR0RSJU1iyMSYELy+gqEVdVc2uooMThdcQbh abHgaiLN0r6Bk7S7j0R2uE2HJK+hLCE= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=Qrpwlhwl; spf=pass (imf11.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715605610; a=rsa-sha256; cv=none; b=54N+usKgpLmOk1CSGYgwJy5pOQy+gtX8tlgX+vg664u0lavWHpVJAXOQPacYty+/2MN8Xg km6VyLJBpgvdoj8qPXoXBjNP5E7LgTsCH1jGVYTXUnKn2mnZoxpUR4ozm5yBPA10Od7wda J6Rr1+ZNchfG6RtAy64dw+tFRRqVeqE= Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-a5a2d0d8644so879225066b.1 for ; Mon, 13 May 2024 06:06:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715605608; x=1716210408; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=VrsGvge7X1e91Ca7nLsARXz/B6rasvGNzsoO0V6EifM=; b=QrpwlhwllBkw+lG/FCyZ0Xts2lBvpy1MAEObjSLggrCXWxViqxwDh/OIKFATMsZWeV jWYtbgPIqc/09Kcoc1v6MMBdQ6qxH7du3srfDJnWZmeM2KnapSb5z67kO4L0XAeQG/VB 3tqk2UScLmY98/STnj7ZkLdjRlJ7oSe9hXl0yC2c+572zhELzh5JuLwkV1KNpPZDhCxl kJi70pL5R15dPVoWqMjw0wCcPJV63hZNF/Tm/XCX+bXCPfZndF8bRJ2fcLVr0dvaOloO PI53M0wILFoITzJqYUapOLRL2U+yCw+uipq6iZuy2eg6+TqC6qvkQ8TyNJURl8kg9fVJ Llfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715605608; x=1716210408; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VrsGvge7X1e91Ca7nLsARXz/B6rasvGNzsoO0V6EifM=; b=cBXz0h4Z5onvjPHid7dO0y9cn2ne+s/FNcBNDUZS2RX9NiVdHJyyxeXVS27DkrrI/6 mEEWJyVWG/1HfCO4UY8ogYTIbQSOJX8E6UkKbfX/QmTLxZMsuMjvkKDKyXdXuJDzGpJc KxZM3X4G1ftYH5li/wnHU+QT/QZVHk4p+nXlJBa5Au20zq8JE/sJz6Zb0yMqVDrm4/Pn 4TA8bwh4OA8U5wfu0SNX7ty3S6Q2eLQ0hLlDzUPo1WuPpumC5dRAtBzXDzy3ucKnYHxY I1bK96P8NV8vau5kog4F5M6/GVQgxx1upG2fxqAbYCVyH/EVeqWLQe/LjtZ8/PeQYEtp aIyA== X-Forwarded-Encrypted: i=1; AJvYcCVBV6wHMSBfcxoH/wFCd3jh9QX55GUPKcLp20MUx3hlA7f6fsOM+P6AWv7dwske2IY8+oXcZ7IQ0kfeaqzr1YAu5cw= X-Gm-Message-State: AOJu0Yz4E4Mm2ZR722Z7+fgA0TzJs7JSNi6j7LQTHEytEn8EAZ3kNf+O S++Jm787nJ5+1fDcVjRDP+SUVigaz5aJT6JoDfNZ7FgZ3L3Tcqds74Im7PoNqbT6F3GH3uB0vYe GhbokKRKLzmIQY2sBT5xjC+pfmIfPwMyIz7BWNg== X-Google-Smtp-Source: AGHT+IGuxtHdquCf9OJVK2sboxMvXTHyJCiKMZ5sdO54V5h9DlxVbVMWK85LKsksksgojjn9eM9Fi/1qVdBzaK4KbX0= X-Received: by 2002:a17:906:3398:b0:a59:a8a4:a5a7 with SMTP id a640c23a62f3a-a5a2d5c8be3mr718581566b.30.1715605608379; Mon, 13 May 2024 06:06:48 -0700 (PDT) MIME-Version: 1.0 References: <20240508113419.18620-1-alexghiti@rivosinc.com> <20240508113419.18620-2-alexghiti@rivosinc.com> <8aa0a606-f47c-4826-96ad-121ac1f7eb96@arm.com> In-Reply-To: <8aa0a606-f47c-4826-96ad-121ac1f7eb96@arm.com> From: Alexandre Ghiti Date: Mon, 13 May 2024 15:06:37 +0200 Message-ID: Subject: Re: [PATCH RESEND v2 1/9] riscv: Restore the pfn in a NAPOT pte when manipulated by core mm code To: Ryan Roberts Cc: Catalin Marinas , Will Deacon , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: k6r3u1dgpyyyhhui9h4k4kju7ewj4doe X-Rspamd-Queue-Id: 5AF6C40022 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1715605610-718233 X-HE-Meta: U2FsdGVkX197agmBGZ2EFnW6Fo5I/WWtRvjHd0Qa+3UQmu+sW9AZAt3Zj0iuyaAObA45KmZt+5XelWvVXNkCEpsRI1H13cxGLtiIdyutvCYD6nt20fRTcIzTAZaZVbd53RZ878dgieN+qHFsbntyXNggfKCXu8Vm7zDrlmlHvugYz0k4Lwo9qOoXu2QFUiivyjUwfxKRGPVoTrkNEastOYsqLbUmT/C4QiP5oSmc+41c2yGyfFDbmP3uvEJkx6rLhRR+bBR2hn1s4HSaVg46t/s+u4tfR6qg46zGVY+oph5vKEPGgwuHx8zEzj6oW2M02AnTSVQwxykoJR1RX1647q5j/NqxTkjff6Upi52wR7DbDUzadUY3yZEDzrtojFVdaRmPLy7fn8PDentMmlh2pEOx69rS9b0SXswTZqySUVOrq2WM8xseIl7Q476ZdR+r6LiIEz3ofobbSKWs2wtDByuXXOicPFmVdKB5nBEb7ESwQTcG3jsQDf0T/AnYCsEuIgu4upCS4RzIpKO0pxEH0N4F5QKxfHcAVeU8aIuT8usM0ncxhJt7QvXXitbdsn5rNW2/PVAK3HXk49c8zeDBTYBp0lxW1hSBVdWpYcHo02geKuDCixnb057qR/HQEa75o6cVKpPaL73In48CHDdHYtyXBRbxVB9jrvoNCK8T88PQW0/yTQchCchglNoxeDx0wc66+iV7w7VPWP01r8rGXg+Ov/OhX+1wMvm+/QlX3eslhgWBDQo2TIifzU4BwUP+3TQkNgEF2pGEpwGA8N3MKwsgpLl33op0lpgvtjGdUVKNcx9mmThfJwFUBnkd4xywG1OXGqunLDCDD+UKpLnYZHX6jRK3/D4tNQI9fFqdrFUD6Un9zwBqyT9EfFE4pPG+EHjhbfAXxD7d5x5O/DW3v7Hgxq2GoqDxClDnAXvDd2P5oFxhLAmfOoHeBL+RqFzTzxRN2oEWNRCu15g332d ct+G30bC k8qRjDF6ukZqlXtKBw+lUABNRgC6Fo0Fu3OLbK2ze9niLK0GNxL1ptpY784o+2HNXTQ5FQftyXeUiEuSKBEsW4/UbYdjmfWkOTWuqINmJxW4ueneQ3bFpyXCnRiHMf+V3wRYxoNe3lg0GfMOx8vfA6xztYWMdLKT+ygNVAvbsk3h85PgDSt0wdCFGfDXg3qBbzmVktlv0L4ujxwjB3TQQR1rKQlxqYELrP48pXd4xZNajH1Fe21bMOS1K7QEjeVmOcKBLI0boOx+0J0mvpwY13Vwk82ZxcD9oCSrxmYhyC0MFrCvFbKqwtOkOB1KOqAoUVLMStRJZgYPfFnIkkXP/Tub8iilxWzu56kdnSVR7tNBgtEyiOMoiCgYrviLR0z+TiyBaG4qAYofivS8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Ryan, On Fri, May 10, 2024 at 2:20=E2=80=AFPM Ryan Roberts = wrote: > > On 08/05/2024 12:34, Alexandre Ghiti wrote: > > The core mm code expects to be able to extract the pfn from a pte. NAPO= T > > mappings work differently since its ptes actually point to the first pf= n > > of the mapping, the other bits being used to encode the size of the > > mapping. > > > > So modify ptep_get() so that it returns a pte value that contains the > > *real* pfn (which is then different from what the HW expects) and right > > before storing the ptes to the page table, reset the pfn LSBs to the > > size of the mapping. > > Did you consider leaving the pte as is and instead modifying your pte_pfn= () > implementation? > > For arm64 at least, it is beneficial to keep the pte marked as contiguous= when > passing it up to core-mm because there are other helpers which need to pa= rse the > contiguous bit (e.g. pte_leaf_size()). If we were to clear the cont bit i= n > ptep_get() that info would be lost and perf_get_pgtable_size() would alwa= ys > conclude the leaf size is 4K even when it is actually 64K. I don't clear the contpte bit here (ie the napot bit), I'm just setting the right pfn so that the core-mm code knows exactly which page is targeted by each pte of a contpte region (remember riscv napot extension uses the lsb of the pte pfn to encode the size the mapping, so all ptes of a contpte region will return the same pfn). And from pte_pfn(), we have no way of knowing from the pte value alone which page is targeted, we need to know its position in the page table to "guess" the right pfn. > > > > > And make sure that all NAPOT mappings are set using set_ptes(). > > > > Signed-off-by: Alexandre Ghiti > > --- > > arch/riscv/include/asm/pgtable-64.h | 11 +++ > > arch/riscv/include/asm/pgtable.h | 105 ++++++++++++++++++++++++++-- > > arch/riscv/mm/hugetlbpage.c | 38 +++++----- > > 3 files changed, 128 insertions(+), 26 deletions(-) > > > > diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/a= sm/pgtable-64.h > > index 221a5c1ee287..9fe076fc503e 100644 > > --- a/arch/riscv/include/asm/pgtable-64.h > > +++ b/arch/riscv/include/asm/pgtable-64.h > > @@ -106,6 +106,17 @@ enum napot_cont_order { > > #define napot_cont_mask(order) (~(napot_cont_size(order) - 1UL)) > > #define napot_pte_num(order) BIT(order) > > > > +static inline bool is_napot_order(unsigned int order) > > +{ > > + unsigned int napot_order; > > + > > + for_each_napot_order(napot_order) > > + if (order =3D=3D napot_order) > > + return true; > > + > > + return false; > > +} > > + > > #ifdef CONFIG_RISCV_ISA_SVNAPOT > > #define HUGE_MAX_HSTATE (2 + (NAPOT_ORDER_MAX - NAPOT_CON= T_ORDER_BASE)) > > #else > > diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/= pgtable.h > > index 9f8ea0e33eb1..268c828f5152 100644 > > --- a/arch/riscv/include/asm/pgtable.h > > +++ b/arch/riscv/include/asm/pgtable.h > > @@ -297,6 +297,8 @@ static inline unsigned long pte_napot(pte_t pte) > > return pte_val(pte) & _PAGE_NAPOT; > > } > > > > +#define pte_valid_napot(pte) (pte_present(pte) && pte_napot(pte)) > > + > > static inline pte_t pte_mknapot(pte_t pte, unsigned int order) > > { > > int pos =3D order - 1 + _PAGE_PFN_SHIFT; > > @@ -306,6 +308,12 @@ static inline pte_t pte_mknapot(pte_t pte, unsigne= d int order) > > return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPO= T); > > } > > > > +/* pte at entry must *not* encode the mapping size in the pfn LSBs. */ > > +static inline pte_t pte_clear_napot(pte_t pte) > > +{ > > + return __pte(pte_val(pte) & ~_PAGE_NAPOT); > > +} > > + > > #else > > > > static __always_inline bool has_svnapot(void) { return false; } > > @@ -315,17 +323,14 @@ static inline unsigned long pte_napot(pte_t pte) > > return 0; > > } > > > > +#define pte_valid_napot(pte) false > > + > > #endif /* CONFIG_RISCV_ISA_SVNAPOT */ > > > > /* Yields the page frame number (PFN) of a page table entry */ > > static inline unsigned long pte_pfn(pte_t pte) > > { > > - unsigned long res =3D __page_val_to_pfn(pte_val(pte)); > > - > > - if (has_svnapot() && pte_napot(pte)) > > - res =3D res & (res - 1UL); > > - > > - return res; > > + return __page_val_to_pfn(pte_val(pte)); > > } > > > > #define pte_page(x) pfn_to_page(pte_pfn(x)) > > @@ -525,9 +530,91 @@ static inline void __set_pte_at(struct mm_struct *= mm, pte_t *ptep, pte_t pteval) > > > > #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT > > > > +#ifdef CONFIG_RISCV_ISA_SVNAPOT > > +static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned lo= ng size, > > + size_t *pgsize) > > +{ > > + pte_t __pte; > > + > > + /* We must read the raw value of the pte to get the size of the m= apping */ > > + __pte =3D READ_ONCE(*ptep); > > + > > + if (pgsize) { > > + if (size >=3D PGDIR_SIZE) > > + *pgsize =3D PGDIR_SIZE; > > + else if (size >=3D P4D_SIZE) > > + *pgsize =3D P4D_SIZE; > > + else if (size >=3D PUD_SIZE) > > + *pgsize =3D PUD_SIZE; > > + else if (size >=3D PMD_SIZE) > > + *pgsize =3D PMD_SIZE; > > + else > > + *pgsize =3D PAGE_SIZE; > > + } > > + > > + /* Make sure __pte is not a swap entry */ > > + if (pte_valid_napot(__pte)) > > + return napot_pte_num(napot_cont_order(__pte)); > > + > > + return 1; > > +} > > +#endif > > + > > +static inline pte_t ptep_get(pte_t *ptep) > > +{ > > + pte_t pte =3D READ_ONCE(*ptep); > > + > > +#ifdef CONFIG_RISCV_ISA_SVNAPOT > > + /* > > + * The pte we load has the N bit set and the size of the mapping = in > > + * the pfn LSBs: keep the N bit and replace the mapping size with > > + * the *real* pfn since the core mm code expects to find it there= . > > + * The mapping size will be reset just before being written to th= e > > + * page table in set_ptes(). > > + */ > > + if (unlikely(pte_valid_napot(pte))) { > > + unsigned int order =3D napot_cont_order(pte); > > + int pos =3D order - 1 + _PAGE_PFN_SHIFT; > > + unsigned long napot_mask =3D ~GENMASK(pos, _PAGE_PFN_SHIF= T); > > + pte_t *orig_ptep =3D PTR_ALIGN_DOWN(ptep, sizeof(*ptep) *= napot_pte_num(order)); > > + > > + pte =3D __pte((pte_val(pte) & napot_mask) + ((ptep - orig= _ptep) << _PAGE_PFN_SHIFT)); > > + } > > +#endif > > + > > + return pte; > > +} > > +#define ptep_get ptep_get > > + > > static inline void set_ptes(struct mm_struct *mm, unsigned long addr, > > pte_t *ptep, pte_t pteval, unsigned int nr) > > { > > +#ifdef CONFIG_RISCV_ISA_SVNAPOT > > + if (unlikely(pte_valid_napot(pteval))) { > > + unsigned int order =3D ilog2(nr); > > + > > + if (!is_napot_order(order)) { > > + /* > > + * Something's weird, we are given a NAPOT pte bu= t the > > + * size of the mapping is not a known NAPOT mappi= ng > > + * size, so clear the NAPOT bit and map this with= out > > + * NAPOT support: core mm only manipulates pte wi= th the > > + * real pfn so we know the pte is valid without t= he N > > + * bit. > > + */ > > + pr_err("Incorrect NAPOT mapping, resetting.\n"); > > + pteval =3D pte_clear_napot(pteval); > > + } else { > > + /* > > + * NAPOT ptes that arrive here only have the N bi= t set > > + * and their pfn does not contain the mapping siz= e, so > > + * set that here. > > + */ > > + pteval =3D pte_mknapot(pteval, order); > > + } > > + } > > +#endif > > I think all this complexity comes along due to using this function both a= s a > public interface that the core-mm uses (which never sets napot) > and also using > it as an internal interface that riscv-hugetlb uses (which does set napot= )? It > might be more understandable if you layer it into a lower level/internal = API and > a higher level/public API (similar to arm64)? I think you're right here, I'll try to do that too. Thanks for your comments, Alex > > > + > > page_table_check_ptes_set(mm, ptep, pteval, nr); > > > > for (;;) { > > @@ -535,6 +622,12 @@ static inline void set_ptes(struct mm_struct *mm, = unsigned long addr, > > if (--nr =3D=3D 0) > > break; > > ptep++; > > + > > +#ifdef CONFIG_RISCV_ISA_SVNAPOT > > + if (unlikely(pte_valid_napot(pteval))) > > + continue; > > +#endif > > + > > pte_val(pteval) +=3D 1 << _PAGE_PFN_SHIFT; > > } > > } > > diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c > > index 5ef2a6891158..fe8067ee71b4 100644 > > --- a/arch/riscv/mm/hugetlbpage.c > > +++ b/arch/riscv/mm/hugetlbpage.c > > @@ -256,8 +256,7 @@ void set_huge_pte_at(struct mm_struct *mm, > > > > clear_flush(mm, addr, ptep, pgsize, pte_num); > > > > - for (i =3D 0; i < pte_num; i++, ptep++, addr +=3D pgsize) > > - set_pte_at(mm, addr, ptep, pte); > > + set_ptes(mm, addr, ptep, pte, pte_num); > > } > > > > int huge_ptep_set_access_flags(struct vm_area_struct *vma, > > @@ -267,16 +266,16 @@ int huge_ptep_set_access_flags(struct vm_area_str= uct *vma, > > int dirty) > > { > > struct mm_struct *mm =3D vma->vm_mm; > > - unsigned long order; > > + size_t pgsize; > > pte_t orig_pte; > > - int i, pte_num; > > + int pte_num; > > > > if (!pte_napot(pte)) > > return ptep_set_access_flags(vma, addr, ptep, pte, dirty)= ; > > > > - order =3D napot_cont_order(pte); > > - pte_num =3D napot_pte_num(order); > > - ptep =3D huge_pte_offset(mm, addr, napot_cont_size(order)); > > + pte_num =3D arch_contpte_get_num_contig(ptep, 0, &pgsize); > > + ptep =3D huge_pte_offset(mm, addr, pte_num * pgsize); > > + > > orig_pte =3D get_clear_contig_flush(mm, addr, ptep, pte_num); > > > > if (pte_dirty(orig_pte)) > > @@ -285,8 +284,7 @@ int huge_ptep_set_access_flags(struct vm_area_struc= t *vma, > > if (pte_young(orig_pte)) > > pte =3D pte_mkyoung(pte); > > > > - for (i =3D 0; i < pte_num; i++, addr +=3D PAGE_SIZE, ptep++) > > - set_pte_at(mm, addr, ptep, pte); > > + set_ptes(mm, addr, ptep, pte, pte_num); > > > > return true; > > } > > @@ -301,7 +299,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, > > if (!pte_napot(orig_pte)) > > return ptep_get_and_clear(mm, addr, ptep); > > > > - pte_num =3D napot_pte_num(napot_cont_order(orig_pte)); > > + pte_num =3D arch_contpte_get_num_contig(ptep, 0, NULL); > > > > return get_clear_contig(mm, addr, ptep, pte_num); > > } > > @@ -311,24 +309,23 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm= , > > pte_t *ptep) > > { > > pte_t pte =3D ptep_get(ptep); > > - unsigned long order; > > + size_t pgsize; > > pte_t orig_pte; > > - int i, pte_num; > > + int pte_num; > > > > if (!pte_napot(pte)) { > > ptep_set_wrprotect(mm, addr, ptep); > > return; > > } > > > > - order =3D napot_cont_order(pte); > > - pte_num =3D napot_pte_num(order); > > - ptep =3D huge_pte_offset(mm, addr, napot_cont_size(order)); > > + pte_num =3D arch_contpte_get_num_contig(ptep, 0, &pgsize); > > + ptep =3D huge_pte_offset(mm, addr, pte_num * pgsize); > > + > > orig_pte =3D get_clear_contig_flush(mm, addr, ptep, pte_num); > > > > orig_pte =3D pte_wrprotect(orig_pte); > > > > - for (i =3D 0; i < pte_num; i++, addr +=3D PAGE_SIZE, ptep++) > > - set_pte_at(mm, addr, ptep, orig_pte); > > + set_ptes(mm, addr, ptep, orig_pte, pte_num); > > } > > > > pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, > > @@ -341,7 +338,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *= vma, > > if (!pte_napot(pte)) > > return ptep_clear_flush(vma, addr, ptep); > > > > - pte_num =3D napot_pte_num(napot_cont_order(pte)); > > + pte_num =3D arch_contpte_get_num_contig(ptep, 0, NULL); > > > > return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num); > > } > > @@ -351,6 +348,7 @@ void huge_pte_clear(struct mm_struct *mm, > > pte_t *ptep, > > unsigned long sz) > > { > > + size_t pgsize; > > pte_t pte =3D ptep_get(ptep); > > int i, pte_num; > > > > @@ -359,8 +357,8 @@ void huge_pte_clear(struct mm_struct *mm, > > return; > > } > > > > - pte_num =3D napot_pte_num(napot_cont_order(pte)); > > - for (i =3D 0; i < pte_num; i++, addr +=3D PAGE_SIZE, ptep++) > > + pte_num =3D arch_contpte_get_num_contig(ptep, 0, &pgsize); > > + for (i =3D 0; i < pte_num; i++, addr +=3D pgsize, ptep++) > > pte_clear(mm, addr, ptep); > > } > > >