From: Xu Lu <luxu.kernel@bytedance.com>
To: paul.walmsley@sifive.com, palmer@dabbelt.com,
aou@eecs.berkeley.edu, ardb@kernel.org, anup@brainfault.org,
atishp@atishpatra.org
Cc: dengliang.1214@bytedance.com, xieyongji@bytedance.com,
lihangjing@bytedance.com, songmuchun@bytedance.com,
punit.agrawal@bytedance.com, linux-kernel@vger.kernel.org,
linux-riscv@lists.infradead.org,
Xu Lu <luxu.kernel@bytedance.com>
Subject: [RFC PATCH V1 08/11] riscv: Apply Svnapot for base page mapping
Date: Thu, 23 Nov 2023 14:57:05 +0800 [thread overview]
Message-ID: <20231123065708.91345-9-luxu.kernel@bytedance.com> (raw)
In-Reply-To: <20231123065708.91345-1-luxu.kernel@bytedance.com>
The Svnapot extension on RISC-V is like contiguous PTE on ARM64. It
allows ptes of a naturally aligned power-of 2 (NAPOT) memory range be
encoded in the same format to save the TLB space.
This commit applies Svnapot for each base page's mapping. This commit is
the key to achieving larger base page's performance optimization.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
arch/riscv/include/asm/pgtable.h | 34 +++++++++++++++++++++++++++-----
1 file changed, 29 insertions(+), 5 deletions(-)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 56366f07985d..803dc5fb6314 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -230,6 +230,16 @@ static __always_inline unsigned long __pte_napot(unsigned long pteval)
return pteval & _PAGE_NAPOT;
}
+static __always_inline unsigned long __pte_mknapot(unsigned long pteval,
+ unsigned int order)
+{
+ int pos = order - 1 + _PAGE_PFN_SHIFT;
+ unsigned long napot_bit = BIT(pos);
+ unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT);
+
+ return (pteval & napot_mask) | napot_bit | _PAGE_NAPOT;
+}
+
static inline pte_t __pte(unsigned long pteval)
{
pte_t pte;
@@ -348,13 +358,11 @@ static inline unsigned long pte_napot(pte_t pte)
return __pte_napot(pte_val(pte));
}
-static inline pte_t pte_mknapot(pte_t pte, unsigned int order)
+static inline pte_t pte_mknapot(pte_t pte, unsigned int page_order)
{
- int pos = order - 1 + _PAGE_PFN_SHIFT;
- unsigned long napot_bit = BIT(pos);
- unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT);
+ unsigned int hw_page_order = page_order + (PAGE_SHIFT - HW_PAGE_SHIFT);
- return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT);
+ return __pte(__pte_mknapot(pte_val(pte), hw_page_order));
}
#else
@@ -366,6 +374,11 @@ static inline unsigned long pte_napot(pte_t pte)
return 0;
}
+static inline pte_t pte_mknapot(pte_t pte, unsigned int page_order)
+{
+ return pte;
+}
+
#endif /* CONFIG_RISCV_ISA_SVNAPOT */
/* Yields the page frame number (PFN) of a page table entry */
@@ -585,6 +598,17 @@ static inline int pte_same(pte_t pte_a, pte_t pte_b)
*/
static inline void set_pte(pte_t *ptep, pte_t pteval)
{
+ unsigned long order;
+
+ /*
+ * has_svnapot() always return false before riscv_isa is initialized.
+ */
+ if (has_svnapot() && pte_present(pteval) && !pte_napot(pteval)) {
+ for_each_napot_order(order) {
+ if (napot_cont_shift(order) == PAGE_SHIFT)
+ pteval = pte_mknapot(pteval, order);
+ }
+ }
*ptep = pteval;
}
--
2.20.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2023-11-23 6:58 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-23 6:56 [RFC PATCH V1 00/11] riscv: Introduce 64K base page Xu Lu
2023-11-23 6:56 ` [RFC PATCH V1 01/11] mm: Fix misused APIs on huge pte Xu Lu
2023-12-31 16:39 ` Alexandre Ghiti
2024-01-04 7:59 ` [External] " Xu Lu
2023-11-23 6:56 ` [RFC PATCH V1 02/11] riscv: Introduce concept of hardware base page Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 03/11] riscv: Adapt pte struct to gap between hw page and sw page Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 04/11] riscv: Adapt pte operations " Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 05/11] riscv: Decouple pmd operations and pte operations Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 06/11] riscv: Distinguish pmd huge pte and napot huge pte Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 07/11] riscv: Adapt satp operations to gap between hw page and sw page Xu Lu
2023-11-23 6:57 ` Xu Lu [this message]
2023-11-23 6:57 ` [RFC PATCH V1 09/11] riscv: Adjust fix_btmap slots number to match variable page size Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 10/11] riscv: kvm: Adapt kvm to gap between hw page and sw page Xu Lu
2023-11-23 6:57 ` [RFC PATCH V1 11/11] riscv: Introduce 64K page size Xu Lu
2023-11-23 9:29 ` [RFC PATCH V1 00/11] riscv: Introduce 64K base page Arnd Bergmann
2023-11-27 8:14 ` [External] " Xu Lu
2023-12-07 6:07 ` Xu Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231123065708.91345-9-luxu.kernel@bytedance.com \
--to=luxu.kernel@bytedance.com \
--cc=anup@brainfault.org \
--cc=aou@eecs.berkeley.edu \
--cc=ardb@kernel.org \
--cc=atishp@atishpatra.org \
--cc=dengliang.1214@bytedance.com \
--cc=lihangjing@bytedance.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=punit.agrawal@bytedance.com \
--cc=songmuchun@bytedance.com \
--cc=xieyongji@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox