* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses
@ 2013-11-27 7:34 mohun106 at gmail.com
2013-11-27 7:34 ` [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size mohun106 at gmail.com
` (3 more replies)
0 siblings, 4 replies; 19+ messages in thread
From: mohun106 at gmail.com @ 2013-11-27 7:34 UTC (permalink / raw)
To: linux-arm-kernel
From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com>
This patch series provides an implementation of supporting 48-bit
Physical Addresses for ARMv8 platforms. It is the maximum width that
any ARMv8 based processor can support.
The implementation extends the existing support of 40-bit PA.The kernel
and user space will now be able to access 128TB each. With 4KB page size
the Linux now will be using 4 levels of page tables by making use of
'pud'. And with 64KB page size the Linux will be using 3 levels of page
tables.
The code has been tested with LTP.
Radha Mohan Chintakuntla (2):
arm64: Add support for 48-bit Physical Addresses
arm64: Add 48-bit PA support for 64KB page size
arch/arm64/include/asm/memory.h | 6 +--
arch/arm64/include/asm/page.h | 4 +-
arch/arm64/include/asm/pgalloc.h | 20 ++++++-
arch/arm64/include/asm/pgtable-3level-hwdef.h | 34 ++++++++++++
arch/arm64/include/asm/pgtable-4level-hwdef.h | 57 ++++++++++++++++++++
arch/arm64/include/asm/pgtable-4level-types.h | 71 +++++++++++++++++++++++++
arch/arm64/include/asm/pgtable-hwdef.h | 9 ++--
arch/arm64/include/asm/pgtable.h | 50 +++++++++++++++---
arch/arm64/include/asm/tlb.h | 2 -
arch/arm64/kernel/head.S | 55 +++++++++++++++++--
arch/arm64/kernel/traps.c | 7 +++
arch/arm64/mm/proc.S | 2 +-
12 files changed, 289 insertions(+), 28 deletions(-)
create mode 100644 arch/arm64/include/asm/pgtable-4level-hwdef.h
create mode 100644 arch/arm64/include/asm/pgtable-4level-types.h
^ permalink raw reply [flat|nested] 19+ messages in thread* [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size 2013-11-27 7:34 [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses mohun106 at gmail.com @ 2013-11-27 7:34 ` mohun106 at gmail.com 2013-11-27 11:14 ` Mark Rutland 2013-11-27 7:34 ` [RFC PATCH 2/2] arm64: Add 48-bit PA support for 64KB " mohun106 at gmail.com ` (2 subsequent siblings) 3 siblings, 1 reply; 19+ messages in thread From: mohun106 at gmail.com @ 2013-11-27 7:34 UTC (permalink / raw) To: linux-arm-kernel From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> This patch adds support for 48-bit physical addresses in the ARMv8 based SoCs. The VA_BITS is expanded to 48 enabling access to 128TB of kernel space. The Linux will now be using 4 levels of page tables for address translations for 4KB page size. Signed-off-by: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> --- arch/arm64/include/asm/memory.h | 6 +-- arch/arm64/include/asm/page.h | 2 +- arch/arm64/include/asm/pgalloc.h | 16 ++++++ arch/arm64/include/asm/pgtable-4level-hwdef.h | 57 ++++++++++++++++++++ arch/arm64/include/asm/pgtable-4level-types.h | 71 +++++++++++++++++++++++++ arch/arm64/include/asm/pgtable-hwdef.h | 7 ++- arch/arm64/include/asm/pgtable.h | 41 +++++++++++++-- arch/arm64/kernel/head.S | 33 +++++++++-- arch/arm64/kernel/traps.c | 5 ++ arch/arm64/mm/proc.S | 2 +- 10 files changed, 220 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 3776217..91e92b4 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -41,11 +41,7 @@ * The module space lives between the addresses given by TASK_SIZE * and PAGE_OFFSET - it must be within 128MB of the kernel text. */ -#ifdef CONFIG_ARM64_64K_PAGES -#define VA_BITS (42) -#else -#define VA_BITS (39) -#endif +#define VA_BITS (48) #define PAGE_OFFSET (UL(0xffffffffffffffff) << (VA_BITS - 1)) #define MODULES_END (PAGE_OFFSET) #define MODULES_VADDR (MODULES_END - SZ_64M) diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h index 46bf666..64faf71 100644 --- a/arch/arm64/include/asm/page.h +++ b/arch/arm64/include/asm/page.h @@ -36,7 +36,7 @@ #ifdef CONFIG_ARM64_64K_PAGES #include <asm/pgtable-2level-types.h> #else -#include <asm/pgtable-3level-types.h> +#include <asm/pgtable-4level-types.h> #endif extern void __cpu_clear_user_page(void *p, unsigned long user); diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 9bea6e7..482816c 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -44,6 +44,22 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) set_pud(pud, __pud(__pa(pmd) | PMD_TYPE_TABLE)); } +static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) +{ + return (pud_t *)get_zeroed_page(GFP_KERNEL | __GFP_REPEAT); +} + +static inline void pud_free(struct mm_struct *mm, pmd_t *pud) +{ + BUG_ON((unsigned long)pud & (PAGE_SIZE-1)); + free_page((unsigned long)pud); +} + +static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pmd_t *pud) +{ + set_pgd(pgd, __pgd(__pa(pud) | PMD_TYPE_TABLE)); +} + #endif /* CONFIG_ARM64_64K_PAGES */ extern pgd_t *pgd_alloc(struct mm_struct *mm); diff --git a/arch/arm64/include/asm/pgtable-4level-hwdef.h b/arch/arm64/include/asm/pgtable-4level-hwdef.h new file mode 100644 index 0000000..9d1e4d1 --- /dev/null +++ b/arch/arm64/include/asm/pgtable-4level-hwdef.h @@ -0,0 +1,57 @@ +/* + * Copyright (C) 2013 Cavium Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + */ +#ifndef __ASM_PGTABLE_3LEVEL_HWDEF_H +#define __ASM_PGTABLE_3LEVEL_HWDEF_H + +/* + * With 48-bit addressing and 4KB pages, there are 4 levels of page tables. + * Each level has 512 entries of 8 bytes each, occupying a 4K page. The user + * and kernel address spaces are limited to 128TB each. + */ +#define PTRS_PER_PTE 512 +#define PTRS_PER_PMD 512 +#define PTRS_PER_PUD 512 +#define PTRS_PER_PGD 512 + +/* + * PGDIR_SHIFT determines the size a top-level page table entry can map. + */ +#define PGDIR_SHIFT 39 +#define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) +#define PGDIR_MASK (~(PGDIR_SIZE-1)) + +/* + * PUD_SHIFT determines the size a upper-level page table entry can map. + */ +#define PUD_SHIFT 30 +#define PUD_SIZE (_AC(1, UL) << PUD_SHIFT) +#define PUD_MASK (~(PUD_SIZE-1)) + +/* + * PMD_SHIFT determines the size a middle-level page table entry can map. + */ +#define PMD_SHIFT 21 +#define PMD_SIZE (_AC(1, UL) << PMD_SHIFT) +#define PMD_MASK (~(PMD_SIZE-1)) + +/* + * section address mask and size definitions. + */ +#define SECTION_SHIFT 21 +#define SECTION_SIZE (_AC(1, UL) << SECTION_SHIFT) +#define SECTION_MASK (~(SECTION_SIZE-1)) + +#endif diff --git a/arch/arm64/include/asm/pgtable-4level-types.h b/arch/arm64/include/asm/pgtable-4level-types.h new file mode 100644 index 0000000..f57f285 --- /dev/null +++ b/arch/arm64/include/asm/pgtable-4level-types.h @@ -0,0 +1,71 @@ +/* + * Copyright (C) 2013 Cavium Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + */ +#ifndef __ASM_PGTABLE_4LEVEL_TYPES_H +#define __ASM_PGTABLE_4LEVEL_TYPES_H + +typedef u64 pteval_t; +typedef u64 pmdval_t; +typedef u64 pudval_t; +typedef u64 pgdval_t; + +#undef STRICT_MM_TYPECHECKS + +#ifdef STRICT_MM_TYPECHECKS + +/* + * These are used to make use of C type-checking.. + */ +typedef struct { pteval_t pte; } pte_t; +typedef struct { pmdval_t pmd; } pmd_t; +typedef struct { pudval_t pud; } pud_t; +typedef struct { pgdval_t pgd; } pgd_t; +typedef struct { pteval_t pgprot; } pgprot_t; + +#define pte_val(x) ((x).pte) +#define pmd_val(x) ((x).pmd) +#define pud_val(x) ((x).pud) +#define pgd_val(x) ((x).pgd) +#define pgprot_val(x) ((x).pgprot) + +#define __pte(x) ((pte_t) { (x) }) +#define __pmd(x) ((pmd_t) { (x) }) +#define __pud(x) ((pud_t) { (x) }) +#define __pgd(x) ((pgd_t) { (x) }) +#define __pgprot(x) ((pgprot_t) { (x) }) + +#else /* !STRICT_MM_TYPECHECKS */ + +typedef pteval_t pte_t; +typedef pmdval_t pmd_t; +typedef pudval_t pud_t; +typedef pgdval_t pgd_t; +typedef pteval_t pgprot_t; + +#define pte_val(x) (x) +#define pmd_val(x) (x) +#define pud_val(x) (x) +#define pgd_val(x) (x) +#define pgprot_val(x) (x) + +#define __pte(x) (x) +#define __pmd(x) (x) +#define __pud(x) (x) +#define __pgd(x) (x) +#define __pgprot(x) (x) + +#endif /* STRICT_MM_TYPECHECKS */ + +#endif /* __ASM_PGTABLE_4LEVEL_TYPES_H */ diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 755f861..05fadaf 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -19,7 +19,7 @@ #ifdef CONFIG_ARM64_64K_PAGES #include <asm/pgtable-2level-hwdef.h> #else -#include <asm/pgtable-3level-hwdef.h> +#include <asm/pgtable-4level-hwdef.h> #endif /* @@ -100,9 +100,9 @@ #define PTE_HYP PTE_USER /* - * 40-bit physical address supported. + * 48-bit physical address supported. */ -#define PHYS_MASK_SHIFT (40) +#define PHYS_MASK_SHIFT (48) #define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1) /* @@ -123,6 +123,7 @@ #define TCR_TG0_64K (UL(1) << 14) #define TCR_TG1_64K (UL(1) << 30) #define TCR_IPS_40BIT (UL(2) << 32) +#define TCR_IPS_48BIT (UL(5) << 32) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 17bd3af..57efd3d 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -43,12 +43,14 @@ #ifndef __ASSEMBLY__ extern void __pte_error(const char *file, int line, unsigned long val); extern void __pmd_error(const char *file, int line, unsigned long val); +extern void __pud_error(const char *file, int line, unsigned long val); extern void __pgd_error(const char *file, int line, unsigned long val); #define pte_ERROR(pte) __pte_error(__FILE__, __LINE__, pte_val(pte)) #ifndef CONFIG_ARM64_64K_PAGES #define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd)) #endif +#define pud_ERROR(pud) __pud_error(__FILE__, __LINE__, pud_val(pud)) #define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) /* @@ -299,6 +301,9 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd) #ifndef CONFIG_ARM64_64K_PAGES +/* Find an entry in the kernel page upper directory */ +#define pud_index(addr) (((addr) >> PUD_SHIFT) & (PTRS_PER_PUD - 1)) + #define pud_none(pud) (!pud_val(pud)) #define pud_bad(pud) (!(pud_val(pud) & 2)) #define pud_present(pud) (pud_val(pud)) @@ -329,13 +334,41 @@ static inline pmd_t *pud_page_vaddr(pud_t pud) /* to find an entry in a kernel page-table-directory */ #define pgd_offset_k(addr) pgd_offset(&init_mm, addr) +#define pgd_none(pgd) (!pgd_val(pgd)) +#define pgd_present(pgd) (pgd_val(pgd)) +#define pgd_bad(pgd) (!(pgd_val(pgd) & 2)) + /* Find an entry in the second-level page table.. */ #ifndef CONFIG_ARM64_64K_PAGES + #define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1)) -static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr) + +static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) +{ + *pgdp = pgd; + dsb(); +} + +static inline void pgd_clear(pgd_t *pgdp) +{ + set_pgd(pgdp, __pgd(0)); +} + +static inline pud_t *pgd_page_vaddr(pgd_t pgd) +{ + return __va(pgd_val(pgd) & PHYS_MASK & (s32)PAGE_MASK); +} + +static inline pmd_t *pmd_offset(pmd_t *pmd, unsigned long addr) { - return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(addr); + return (pmd_t *)pud_page_vaddr(*pmd) + pmd_index(addr); } + +static inline pud_t *pud_offset(pud_t *pud, unsigned long addr) +{ + return (pud_t *)pgd_page_vaddr(*pud) + pud_index(addr); +} + #endif /* Find an entry in the third-level page table.. */ @@ -352,8 +385,8 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; -#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE) -#define IDMAP_DIR_SIZE (2 * PAGE_SIZE) +#define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) +#define IDMAP_DIR_SIZE (3 * PAGE_SIZE) /* * Encode and decode a swap entry: diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 7009387..cc764e5 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -45,8 +45,10 @@ #error KERNEL_RAM_VADDR must start at 0xXXX80000 #endif -#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE) -#define IDMAP_DIR_SIZE (2 * PAGE_SIZE) +#define create_page_entry create_pud_entry + +#define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) +#define IDMAP_DIR_SIZE (3 * PAGE_SIZE) .globl swapper_pg_dir .equ swapper_pg_dir, KERNEL_RAM_VADDR - SWAPPER_DIR_SIZE @@ -376,6 +378,19 @@ ENDPROC(__calc_phys_offset) str \tmp2, [\pgd, \tmp1, lsl #3] .endm +/* Macro to populate the PUD for the corresponding block entry in the next + * level (tbl) for the given virtual address. + * + * Preserves: pud, tbl, virt + * Corrupts: tmp1, tmp2 + */ + .macro create_pud_entry, pud, tbl, virt, tmp1, tmp2 + lsr \tmp1, \virt, #PUD_SHIFT + and \tmp1, \tmp1, #PTRS_PER_PUD - 1 // PUD index + orr \tmp2, \tbl, #3 // PUD entry table type + str \tmp2, [\pud, \tmp1, lsl #3] +.endm + /* * Macro to populate block entries in the page table for the start..end * virtual range (inclusive). @@ -436,7 +451,9 @@ __create_page_tables: add x0, x25, #PAGE_SIZE // section table address adr x3, __turn_mmu_on // virtual/physical address create_pgd_entry x25, x0, x3, x5, x6 - create_block_map x0, x7, x3, x5, x5, idmap=1 + add x1, x0, #PAGE_SIZE + create_page_entry x0, x1, x3, x5, x6 + create_block_map x1, x7, x3, x5, x5, idmap=1 /* * Map the kernel image (starting with PHYS_OFFSET). @@ -444,9 +461,11 @@ __create_page_tables: add x0, x26, #PAGE_SIZE // section table address mov x5, #PAGE_OFFSET create_pgd_entry x26, x0, x5, x3, x6 + add x1, x0, #PAGE_SIZE + create_page_entry x0, x1, x3, x5, x6 ldr x6, =KERNEL_END - 1 mov x3, x24 // phys offset - create_block_map x0, x7, x3, x5, x6 + create_block_map x1, x7, x3, x5, x6 /* * Map the FDT blob (maximum 2MB; must be within 512MB of @@ -462,7 +481,7 @@ __create_page_tables: add x5, x5, x6 // __va(FDT blob) add x6, x5, #1 << 21 // 2MB for the FDT blob sub x6, x6, #1 // inclusive range - create_block_map x0, x7, x3, x5, x6 + create_block_map x1, x7, x3, x5, x6 1: #ifdef CONFIG_EARLY_PRINTK /* @@ -470,8 +489,10 @@ __create_page_tables: * later based earlyprintk kernel parameter. */ ldr x5, =EARLYCON_IOBASE // UART virtual address - add x0, x26, #2 * PAGE_SIZE // section table address + add x0, x26, #PAGE_SIZE // section table address create_pgd_entry x26, x0, x5, x6, x7 + add x1, x0, #2 * PAGE_SIZE + create_page_entry x0, x1, x5, x6, x7 #endif ret ENDPROC(__create_page_tables) diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 7ffaddd..4565aa0 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -336,6 +336,11 @@ void __pmd_error(const char *file, int line, unsigned long val) printk("%s:%d: bad pmd %016lx.\n", file, line, val); } +void __pud_error(const char *file, int line, unsigned long val) +{ + printk("%s:%d: bad pud %016lx.\n", file, line, val); +} + void __pgd_error(const char *file, int line, unsigned long val) { printk("%s:%d: bad pgd %016lx.\n", file, line, val); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 421b99f..2e0041e 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -146,7 +146,7 @@ ENTRY(__cpu_setup) * Set/prepare TCR and TTBR. We use 512GB (39-bit) address range for * both user and kernel. */ - ldr x10, =TCR_TxSZ(VA_BITS) | TCR_FLAGS | TCR_IPS_40BIT | \ + ldr x10, =TCR_TxSZ(VA_BITS) | TCR_FLAGS | TCR_IPS_48BIT | \ TCR_ASID16 | TCR_TBI0 | (1 << 31) #ifdef CONFIG_ARM64_64K_PAGES orr x10, x10, TCR_TG0_64K -- 1.7.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size 2013-11-27 7:34 ` [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size mohun106 at gmail.com @ 2013-11-27 11:14 ` Mark Rutland 2013-11-27 16:00 ` Radha Mohan 0 siblings, 1 reply; 19+ messages in thread From: Mark Rutland @ 2013-11-27 11:14 UTC (permalink / raw) To: linux-arm-kernel Hi, On Wed, Nov 27, 2013 at 07:34:24AM +0000, mohun106 at gmail.com wrote: > From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > > This patch adds support for 48-bit physical addresses in the ARMv8 based > SoCs. The VA_BITS is expanded to 48 enabling access to 128TB of kernel > space. The Linux will now be using 4 levels of page tables for address > translations for 4KB page size. Given that this requires an additional level of page table to be allocated, it would be nice if this were a compile-time configuration option. As you mentioned you'd run LTP tests, what was the additional memory and time cost over 3 levels with 40-bit addressing? Has this been tested in conjunction with hugepages? > > Signed-off-by: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > --- > arch/arm64/include/asm/memory.h | 6 +-- > arch/arm64/include/asm/page.h | 2 +- > arch/arm64/include/asm/pgalloc.h | 16 ++++++ > arch/arm64/include/asm/pgtable-4level-hwdef.h | 57 ++++++++++++++++++++ > arch/arm64/include/asm/pgtable-4level-types.h | 71 +++++++++++++++++++++++++ As they're unused after this patch, it feels very odd to leave pgtable-3level-*.h lying around... > arch/arm64/include/asm/pgtable-hwdef.h | 7 ++- > arch/arm64/include/asm/pgtable.h | 41 +++++++++++++-- > arch/arm64/kernel/head.S | 33 +++++++++-- > arch/arm64/kernel/traps.c | 5 ++ > arch/arm64/mm/proc.S | 2 +- > 10 files changed, 220 insertions(+), 20 deletions(-) > > diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h > index 3776217..91e92b4 100644 > --- a/arch/arm64/include/asm/memory.h > +++ b/arch/arm64/include/asm/memory.h > @@ -41,11 +41,7 @@ > * The module space lives between the addresses given by TASK_SIZE > * and PAGE_OFFSET - it must be within 128MB of the kernel text. > */ > -#ifdef CONFIG_ARM64_64K_PAGES > -#define VA_BITS (42) > -#else > -#define VA_BITS (39) > -#endif > +#define VA_BITS (48) Doesn't this break 64k page support until the next patch? > #define PAGE_OFFSET (UL(0xffffffffffffffff) << (VA_BITS - 1)) > #define MODULES_END (PAGE_OFFSET) > #define MODULES_VADDR (MODULES_END - SZ_64M) > diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h > index 46bf666..64faf71 100644 > --- a/arch/arm64/include/asm/page.h > +++ b/arch/arm64/include/asm/page.h > @@ -36,7 +36,7 @@ > #ifdef CONFIG_ARM64_64K_PAGES > #include <asm/pgtable-2level-types.h> > #else > -#include <asm/pgtable-3level-types.h> > +#include <asm/pgtable-4level-types.h> > #endif > > extern void __cpu_clear_user_page(void *p, unsigned long user); > diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h > index 9bea6e7..482816c 100644 > --- a/arch/arm64/include/asm/pgalloc.h > +++ b/arch/arm64/include/asm/pgalloc.h > @@ -44,6 +44,22 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) > set_pud(pud, __pud(__pa(pmd) | PMD_TYPE_TABLE)); > } > > +static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) > +{ > + return (pud_t *)get_zeroed_page(GFP_KERNEL | __GFP_REPEAT); > +} This is probably a stupid question, but why do we use __GFP_REPEAT in pmd_alloc_one (and here pud_alloc_one), but not pgd_alloc or pte_alloc_one? [...] > diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h > index 755f861..05fadaf 100644 > --- a/arch/arm64/include/asm/pgtable-hwdef.h > +++ b/arch/arm64/include/asm/pgtable-hwdef.h > @@ -19,7 +19,7 @@ > #ifdef CONFIG_ARM64_64K_PAGES > #include <asm/pgtable-2level-hwdef.h> > #else > -#include <asm/pgtable-3level-hwdef.h> > +#include <asm/pgtable-4level-hwdef.h> > #endif > > /* > @@ -100,9 +100,9 @@ > #define PTE_HYP PTE_USER > > /* > - * 40-bit physical address supported. > + * 48-bit physical address supported. > */ > -#define PHYS_MASK_SHIFT (40) > +#define PHYS_MASK_SHIFT (48) The 64k page needs to be updated to handle this or it will be broken, no? > #define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1) > > /* > @@ -123,6 +123,7 @@ > #define TCR_TG0_64K (UL(1) << 14) > #define TCR_TG1_64K (UL(1) << 30) > #define TCR_IPS_40BIT (UL(2) << 32) > +#define TCR_IPS_48BIT (UL(5) << 32) > #define TCR_ASID16 (UL(1) << 36) > #define TCR_TBI0 (UL(1) << 37) > > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h > index 17bd3af..57efd3d 100644 > --- a/arch/arm64/include/asm/pgtable.h > +++ b/arch/arm64/include/asm/pgtable.h > @@ -43,12 +43,14 @@ > #ifndef __ASSEMBLY__ > extern void __pte_error(const char *file, int line, unsigned long val); > extern void __pmd_error(const char *file, int line, unsigned long val); > +extern void __pud_error(const char *file, int line, unsigned long val); > extern void __pgd_error(const char *file, int line, unsigned long val); > > #define pte_ERROR(pte) __pte_error(__FILE__, __LINE__, pte_val(pte)) > #ifndef CONFIG_ARM64_64K_PAGES > #define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd)) > #endif > +#define pud_ERROR(pud) __pud_error(__FILE__, __LINE__, pud_val(pud)) > #define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) Given that these don't seem to be called from assembly, and currently are identical other than the "pte", "pmd, or "pgd" string, could we not have a single implementation and pass the requisite "pte", "pmd", "pud", or "pgd" in as a parameter here? [...] > @@ -352,8 +385,8 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) > extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; > extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; > > -#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE) > -#define IDMAP_DIR_SIZE (2 * PAGE_SIZE) > +#define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) > +#define IDMAP_DIR_SIZE (3 * PAGE_SIZE) > > /* > * Encode and decode a swap entry: > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index 7009387..cc764e5 100644 > --- a/arch/arm64/kernel/head.S > +++ b/arch/arm64/kernel/head.S > @@ -45,8 +45,10 @@ > #error KERNEL_RAM_VADDR must start at 0xXXX80000 > #endif > > -#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE) > -#define IDMAP_DIR_SIZE (2 * PAGE_SIZE) > +#define create_page_entry create_pud_entry > + > +#define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) > +#define IDMAP_DIR_SIZE (3 * PAGE_SIZE) I hadn't realised that SWAPPER_DIR_SIZE and IDMAP_DIR_SIZE were duplicated. Could we not moved the definition in pgtable.h before the #ifndef __ASSEMBLY__? head.S includes pgtable.h, and page.h defines PAGE_SIZE before its #ifndef __ASSEMBLY__. [...] > @@ -336,6 +336,11 @@ void __pmd_error(const char *file, int line, unsigned long val) > printk("%s:%d: bad pmd %016lx.\n", file, line, val); > } > > +void __pud_error(const char *file, int line, unsigned long val) > +{ > + printk("%s:%d: bad pud %016lx.\n", file, line, val); > +} As mentioned above, I think we can unify the __p*_error functions rather than introducing a new one. Thanks, Mark. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size 2013-11-27 11:14 ` Mark Rutland @ 2013-11-27 16:00 ` Radha Mohan 0 siblings, 0 replies; 19+ messages in thread From: Radha Mohan @ 2013-11-27 16:00 UTC (permalink / raw) To: linux-arm-kernel On Wed, Nov 27, 2013 at 4:44 PM, Mark Rutland <mark.rutland@arm.com> wrote: > Hi, > > On Wed, Nov 27, 2013 at 07:34:24AM +0000, mohun106 at gmail.com wrote: >> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >> >> This patch adds support for 48-bit physical addresses in the ARMv8 based >> SoCs. The VA_BITS is expanded to 48 enabling access to 128TB of kernel >> space. The Linux will now be using 4 levels of page tables for address >> translations for 4KB page size. > > Given that this requires an additional level of page table to be > allocated, it would be nice if this were a compile-time configuration > option. > Having a compile time option increases the number of #ifdefs. Is this acceptable? > As you mentioned you'd run LTP tests, what was the additional memory and > time cost over 3 levels with 40-bit addressing? > Additional memory cost will be another page table. Haven't checked the time cost though. I will get back on the time costs. > Has this been tested in conjunction with hugepages? > This is still under progress. >> >> Signed-off-by: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >> --- >> arch/arm64/include/asm/memory.h | 6 +-- >> arch/arm64/include/asm/page.h | 2 +- >> arch/arm64/include/asm/pgalloc.h | 16 ++++++ >> arch/arm64/include/asm/pgtable-4level-hwdef.h | 57 ++++++++++++++++++++ >> arch/arm64/include/asm/pgtable-4level-types.h | 71 +++++++++++++++++++++++++ > > As they're unused after this patch, it feels very odd to leave > pgtable-3level-*.h lying around... pgtable-3level-*.h is for 64KB page size. I can remove pgtable-2level*.h. > >> arch/arm64/include/asm/pgtable-hwdef.h | 7 ++- >> arch/arm64/include/asm/pgtable.h | 41 +++++++++++++-- >> arch/arm64/kernel/head.S | 33 +++++++++-- >> arch/arm64/kernel/traps.c | 5 ++ >> arch/arm64/mm/proc.S | 2 +- >> 10 files changed, 220 insertions(+), 20 deletions(-) >> >> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h >> index 3776217..91e92b4 100644 >> --- a/arch/arm64/include/asm/memory.h >> +++ b/arch/arm64/include/asm/memory.h >> @@ -41,11 +41,7 @@ >> * The module space lives between the addresses given by TASK_SIZE >> * and PAGE_OFFSET - it must be within 128MB of the kernel text. >> */ >> -#ifdef CONFIG_ARM64_64K_PAGES >> -#define VA_BITS (42) >> -#else >> -#define VA_BITS (39) >> -#endif >> +#define VA_BITS (48) > > Doesn't this break 64k page support until the next patch? No, for 64KB also the max width is 48. > >> #define PAGE_OFFSET (UL(0xffffffffffffffff) << (VA_BITS - 1)) >> #define MODULES_END (PAGE_OFFSET) >> #define MODULES_VADDR (MODULES_END - SZ_64M) >> diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h >> index 46bf666..64faf71 100644 >> --- a/arch/arm64/include/asm/page.h >> +++ b/arch/arm64/include/asm/page.h >> @@ -36,7 +36,7 @@ >> #ifdef CONFIG_ARM64_64K_PAGES >> #include <asm/pgtable-2level-types.h> >> #else >> -#include <asm/pgtable-3level-types.h> >> +#include <asm/pgtable-4level-types.h> >> #endif >> >> extern void __cpu_clear_user_page(void *p, unsigned long user); >> diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h >> index 9bea6e7..482816c 100644 >> --- a/arch/arm64/include/asm/pgalloc.h >> +++ b/arch/arm64/include/asm/pgalloc.h >> @@ -44,6 +44,22 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) >> set_pud(pud, __pud(__pa(pmd) | PMD_TYPE_TABLE)); >> } >> >> +static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) >> +{ >> + return (pud_t *)get_zeroed_page(GFP_KERNEL | __GFP_REPEAT); >> +} > > This is probably a stupid question, but why do we use __GFP_REPEAT in > pmd_alloc_one (and here pud_alloc_one), but not pgd_alloc or > pte_alloc_one? > > [...] > >> diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h >> index 755f861..05fadaf 100644 >> --- a/arch/arm64/include/asm/pgtable-hwdef.h >> +++ b/arch/arm64/include/asm/pgtable-hwdef.h >> @@ -19,7 +19,7 @@ >> #ifdef CONFIG_ARM64_64K_PAGES >> #include <asm/pgtable-2level-hwdef.h> >> #else >> -#include <asm/pgtable-3level-hwdef.h> >> +#include <asm/pgtable-4level-hwdef.h> >> #endif >> >> /* >> @@ -100,9 +100,9 @@ >> #define PTE_HYP PTE_USER >> >> /* >> - * 40-bit physical address supported. >> + * 48-bit physical address supported. >> */ >> -#define PHYS_MASK_SHIFT (40) >> +#define PHYS_MASK_SHIFT (48) > > The 64k page needs to be updated to handle this or it will be broken, > no? > >> #define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1) >> >> /* >> @@ -123,6 +123,7 @@ >> #define TCR_TG0_64K (UL(1) << 14) >> #define TCR_TG1_64K (UL(1) << 30) >> #define TCR_IPS_40BIT (UL(2) << 32) >> +#define TCR_IPS_48BIT (UL(5) << 32) >> #define TCR_ASID16 (UL(1) << 36) >> #define TCR_TBI0 (UL(1) << 37) >> >> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h >> index 17bd3af..57efd3d 100644 >> --- a/arch/arm64/include/asm/pgtable.h >> +++ b/arch/arm64/include/asm/pgtable.h >> @@ -43,12 +43,14 @@ >> #ifndef __ASSEMBLY__ >> extern void __pte_error(const char *file, int line, unsigned long val); >> extern void __pmd_error(const char *file, int line, unsigned long val); >> +extern void __pud_error(const char *file, int line, unsigned long val); >> extern void __pgd_error(const char *file, int line, unsigned long val); >> >> #define pte_ERROR(pte) __pte_error(__FILE__, __LINE__, pte_val(pte)) >> #ifndef CONFIG_ARM64_64K_PAGES >> #define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd)) >> #endif >> +#define pud_ERROR(pud) __pud_error(__FILE__, __LINE__, pud_val(pud)) >> #define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) > > Given that these don't seem to be called from assembly, and currently > are identical other than the "pte", "pmd, or "pgd" string, could we not > have a single implementation and pass the requisite "pte", "pmd", "pud", > or "pgd" in as a parameter here? > > [...] > >> @@ -352,8 +385,8 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) >> extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; >> extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; >> >> -#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE) >> -#define IDMAP_DIR_SIZE (2 * PAGE_SIZE) >> +#define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) >> +#define IDMAP_DIR_SIZE (3 * PAGE_SIZE) >> >> /* >> * Encode and decode a swap entry: >> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S >> index 7009387..cc764e5 100644 >> --- a/arch/arm64/kernel/head.S >> +++ b/arch/arm64/kernel/head.S >> @@ -45,8 +45,10 @@ >> #error KERNEL_RAM_VADDR must start at 0xXXX80000 >> #endif >> >> -#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE) >> -#define IDMAP_DIR_SIZE (2 * PAGE_SIZE) >> +#define create_page_entry create_pud_entry >> + >> +#define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) >> +#define IDMAP_DIR_SIZE (3 * PAGE_SIZE) > > I hadn't realised that SWAPPER_DIR_SIZE and IDMAP_DIR_SIZE were > duplicated. Could we not moved the definition in pgtable.h before the > #ifndef __ASSEMBLY__? head.S includes pgtable.h, and page.h defines > PAGE_SIZE before its #ifndef __ASSEMBLY__. > > [...] > >> @@ -336,6 +336,11 @@ void __pmd_error(const char *file, int line, unsigned long val) >> printk("%s:%d: bad pmd %016lx.\n", file, line, val); >> } >> >> +void __pud_error(const char *file, int line, unsigned long val) >> +{ >> + printk("%s:%d: bad pud %016lx.\n", file, line, val); >> +} > > As mentioned above, I think we can unify the __p*_error functions > rather than introducing a new one. > > Thanks, > Mark. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 2/2] arm64: Add 48-bit PA support for 64KB page size 2013-11-27 7:34 [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses mohun106 at gmail.com 2013-11-27 7:34 ` [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size mohun106 at gmail.com @ 2013-11-27 7:34 ` mohun106 at gmail.com 2013-11-27 11:30 ` Mark Rutland 2013-11-27 11:04 ` [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses Marc Zyngier 2013-11-28 9:29 ` Steve Capper 3 siblings, 1 reply; 19+ messages in thread From: mohun106 at gmail.com @ 2013-11-27 7:34 UTC (permalink / raw) To: linux-arm-kernel From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> Extend the 48-bit physical address support to 64KB page size. The VA_BITS will be 48 and 3 levels of page tables are used for address translations. Signed-off-by: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> --- arch/arm64/include/asm/page.h | 2 +- arch/arm64/include/asm/pgalloc.h | 4 +- arch/arm64/include/asm/pgtable-3level-hwdef.h | 34 +++++++++++++++++++++++++ arch/arm64/include/asm/pgtable-hwdef.h | 2 +- arch/arm64/include/asm/pgtable.h | 29 +++++++++++--------- arch/arm64/include/asm/tlb.h | 2 - arch/arm64/kernel/head.S | 24 +++++++++++++++++ arch/arm64/kernel/traps.c | 2 + 8 files changed, 80 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h index 64faf71..82fd5ab 100644 --- a/arch/arm64/include/asm/page.h +++ b/arch/arm64/include/asm/page.h @@ -34,7 +34,7 @@ #ifndef __ASSEMBLY__ #ifdef CONFIG_ARM64_64K_PAGES -#include <asm/pgtable-2level-types.h> +#include <asm/pgtable-3level-types.h> #else #include <asm/pgtable-4level-types.h> #endif diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 482816c..b287c32 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -26,8 +26,6 @@ #define check_pgt_cache() do { } while (0) -#ifndef CONFIG_ARM64_64K_PAGES - static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr) { return (pmd_t *)get_zeroed_page(GFP_KERNEL | __GFP_REPEAT); @@ -44,6 +42,8 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) set_pud(pud, __pud(__pa(pmd) | PMD_TYPE_TABLE)); } +#ifndef CONFIG_ARM64_64K_PAGES + static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr) { return (pud_t *)get_zeroed_page(GFP_KERNEL | __GFP_REPEAT); diff --git a/arch/arm64/include/asm/pgtable-3level-hwdef.h b/arch/arm64/include/asm/pgtable-3level-hwdef.h index 3dbf941..fb9c1da 100644 --- a/arch/arm64/include/asm/pgtable-3level-hwdef.h +++ b/arch/arm64/include/asm/pgtable-3level-hwdef.h @@ -16,6 +16,7 @@ #ifndef __ASM_PGTABLE_3LEVEL_HWDEF_H #define __ASM_PGTABLE_3LEVEL_HWDEF_H +#ifndef CONFIG_ARM64_64K_PAGES /* * With LPAE and 4KB pages, there are 3 levels of page tables. Each level has * 512 entries of 8 bytes each, occupying a 4K page. The first level table @@ -47,4 +48,37 @@ #define SECTION_SIZE (_AC(1, UL) << SECTION_SHIFT) #define SECTION_MASK (~(SECTION_SIZE-1)) +#else /* !CONFIG_ARM64_64K_PAGES */ + +/* + * With 64KB pages, there are 3 levels of page tables. Each level has + * entries of 8 bytes each, occupying a 64K page. The first level table + * has 64 entries and rest of them have 8192 entries. + */ +#define PTRS_PER_PTE 8192 +#define PTRS_PER_PMD 8192 +#define PTRS_PER_PGD 64 + +/* + * PGDIR_SHIFT determines the size a top-level page table entry can map. + */ +#define PGDIR_SHIFT 42 +#define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) +#define PGDIR_MASK (~(PGDIR_SIZE-1)) + +/* + * PMD_SHIFT determines the size a middle-level page table entry can map. + */ +#define PMD_SHIFT 29 +#define PMD_SIZE (_AC(1, UL) << PMD_SHIFT) +#define PMD_MASK (~(PMD_SIZE-1)) + +/* + * section address mask and size definitions. + */ +#define SECTION_SHIFT 29 +#define SECTION_SIZE (_AC(1, UL) << SECTION_SHIFT) +#define SECTION_MASK (~(SECTION_SIZE-1)) + +#endif /* CONFIG_ARM64_64K_PAGES */ #endif diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 05fadaf..d552814 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -17,7 +17,7 @@ #define __ASM_PGTABLE_HWDEF_H #ifdef CONFIG_ARM64_64K_PAGES -#include <asm/pgtable-2level-hwdef.h> +#include <asm/pgtable-3level-hwdef.h> #else #include <asm/pgtable-4level-hwdef.h> #endif diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 57efd3d..8feb6c7 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -47,10 +47,10 @@ extern void __pud_error(const char *file, int line, unsigned long val); extern void __pgd_error(const char *file, int line, unsigned long val); #define pte_ERROR(pte) __pte_error(__FILE__, __LINE__, pte_val(pte)) -#ifndef CONFIG_ARM64_64K_PAGES #define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd)) -#endif +#ifndef CONFIG_ARM64_64K_PAGES #define pud_ERROR(pud) __pud_error(__FILE__, __LINE__, pud_val(pud)) +#endif #define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) /* @@ -299,8 +299,6 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd) */ #define mk_pte(page,prot) pfn_pte(page_to_pfn(page),prot) -#ifndef CONFIG_ARM64_64K_PAGES - /* Find an entry in the kernel page upper directory */ #define pud_index(addr) (((addr) >> PUD_SHIFT) & (PTRS_PER_PUD - 1)) @@ -324,8 +322,6 @@ static inline pmd_t *pud_page_vaddr(pud_t pud) return __va(pud_val(pud) & PHYS_MASK & (s32)PAGE_MASK); } -#endif /* CONFIG_ARM64_64K_PAGES */ - /* to find an entry in a page-table-directory */ #define pgd_index(addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1)) @@ -338,11 +334,8 @@ static inline pmd_t *pud_page_vaddr(pud_t pud) #define pgd_present(pgd) (pgd_val(pgd)) #define pgd_bad(pgd) (!(pgd_val(pgd) & 2)) -/* Find an entry in the second-level page table.. */ #ifndef CONFIG_ARM64_64K_PAGES -#define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1)) - static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) { *pgdp = pgd; @@ -359,16 +352,26 @@ static inline pud_t *pgd_page_vaddr(pgd_t pgd) return __va(pgd_val(pgd) & PHYS_MASK & (s32)PAGE_MASK); } +static inline pud_t *pud_offset(pud_t *pud, unsigned long addr) +{ + return (pud_t *)pgd_page_vaddr(*pud) + pud_index(addr); +} + +#endif + +/* Find an entry in the second-level page table.. */ +#define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1)) + +#ifndef CONFIG_ARM64_64K_PAGES static inline pmd_t *pmd_offset(pmd_t *pmd, unsigned long addr) { return (pmd_t *)pud_page_vaddr(*pmd) + pmd_index(addr); } - -static inline pud_t *pud_offset(pud_t *pud, unsigned long addr) +#else +static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr) { - return (pud_t *)pgd_page_vaddr(*pud) + pud_index(addr); + return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(addr); } - #endif /* Find an entry in the third-level page table.. */ diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index 717031a..61a265f 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -175,14 +175,12 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, tlb_remove_page(tlb, pte); } -#ifndef CONFIG_ARM64_64K_PAGES static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp, unsigned long addr) { tlb_add_flush(tlb, addr); tlb_remove_page(tlb, virt_to_page(pmdp)); } -#endif #define pte_free_tlb(tlb, ptep, addr) __pte_free_tlb(tlb, ptep, addr) #define pmd_free_tlb(tlb, pmdp, addr) __pmd_free_tlb(tlb, pmdp, addr) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index cc764e5..2c67b79 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -45,7 +45,11 @@ #error KERNEL_RAM_VADDR must start at 0xXXX80000 #endif +#ifdef CONFIG_ARM64_64K_PAGES +#define create_page_entry create_pmd_entry +#else #define create_page_entry create_pud_entry +#endif #define SWAPPER_DIR_SIZE (4 * PAGE_SIZE) #define IDMAP_DIR_SIZE (3 * PAGE_SIZE) @@ -391,6 +395,22 @@ ENDPROC(__calc_phys_offset) str \tmp2, [\pud, \tmp1, lsl #3] .endm +#ifdef CONFIG_ARM64_64K_PAGES +/* + * Macro to populate the PMD for the corresponding block entry in the next + * level (tbl) for the given virtual address. + * + * Preserves: pmd, tbl, virt + * Corrupts: tmp1, tmp2 + */ + .macro create_pmd_entry, pud, tbl, virt, tmp1, tmp2 + lsr \tmp1, \virt, #PMD_SHIFT + and \tmp1, \tmp1, #PTRS_PER_PMD - 1 // PMD index + orr \tmp2, \tbl, #3 // PMD entry table type + str \tmp2, [\pud, \tmp1, lsl #3] + .endm +#endif + /* * Macro to populate block entries in the page table for the start..end * virtual range (inclusive). @@ -489,7 +509,11 @@ __create_page_tables: * later based earlyprintk kernel parameter. */ ldr x5, =EARLYCON_IOBASE // UART virtual address +#ifndef CONFIG_ARM64_64K_PAGES add x0, x26, #PAGE_SIZE // section table address +#else + add x0, x26, #2 * PAGE_SIZE // section table address +#endif create_pgd_entry x26, x0, x5, x6, x7 add x1, x0, #2 * PAGE_SIZE create_page_entry x0, x1, x5, x6, x7 diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 4565aa0..ea63941 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -336,10 +336,12 @@ void __pmd_error(const char *file, int line, unsigned long val) printk("%s:%d: bad pmd %016lx.\n", file, line, val); } +#ifndef CONFIG_ARM64_64K_PAGES void __pud_error(const char *file, int line, unsigned long val) { printk("%s:%d: bad pud %016lx.\n", file, line, val); } +#endif void __pgd_error(const char *file, int line, unsigned long val) { -- 1.7.1 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [RFC PATCH 2/2] arm64: Add 48-bit PA support for 64KB page size 2013-11-27 7:34 ` [RFC PATCH 2/2] arm64: Add 48-bit PA support for 64KB " mohun106 at gmail.com @ 2013-11-27 11:30 ` Mark Rutland 0 siblings, 0 replies; 19+ messages in thread From: Mark Rutland @ 2013-11-27 11:30 UTC (permalink / raw) To: linux-arm-kernel Hi, On Wed, Nov 27, 2013 at 07:34:25AM +0000, mohun106 at gmail.com wrote: > From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > > Extend the 48-bit physical address support to 64KB page size. The > VA_BITS will be 48 and 3 levels of page tables are used for address > translations. Similarly to the 4k patch, has this been tested with hugepages? What is the performance and memory impact over 40-bit addresses? > > Signed-off-by: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > --- > arch/arm64/include/asm/page.h | 2 +- > arch/arm64/include/asm/pgalloc.h | 4 +- > arch/arm64/include/asm/pgtable-3level-hwdef.h | 34 +++++++++++++++++++++++++ > arch/arm64/include/asm/pgtable-hwdef.h | 2 +- > arch/arm64/include/asm/pgtable.h | 29 +++++++++++--------- > arch/arm64/include/asm/tlb.h | 2 - > arch/arm64/kernel/head.S | 24 +++++++++++++++++ > arch/arm64/kernel/traps.c | 2 + > 8 files changed, 80 insertions(+), 19 deletions(-) This patches leaves arch/arm64/include/asm/pgtable-2level-*.h unused, yet they are not removed... > diff --git a/arch/arm64/include/asm/pgtable-3level-hwdef.h b/arch/arm64/include/asm/pgtable-3level-hwdef.h > index 3dbf941..fb9c1da 100644 > --- a/arch/arm64/include/asm/pgtable-3level-hwdef.h > +++ b/arch/arm64/include/asm/pgtable-3level-hwdef.h > @@ -16,6 +16,7 @@ > #ifndef __ASM_PGTABLE_3LEVEL_HWDEF_H > #define __ASM_PGTABLE_3LEVEL_HWDEF_H > > +#ifndef CONFIG_ARM64_64K_PAGES As far as I could tell the last patch made 4k pages always use 4 levels. Given that, why this #ifndef? > /* > * With LPAE and 4KB pages, there are 3 levels of page tables. Each level has > * 512 entries of 8 bytes each, occupying a 4K page. The first level table > @@ -47,4 +48,37 @@ > #define SECTION_SIZE (_AC(1, UL) << SECTION_SHIFT) > #define SECTION_MASK (~(SECTION_SIZE-1)) [...] > --- a/arch/arm64/include/asm/pgtable-hwdef.h > +++ b/arch/arm64/include/asm/pgtable-hwdef.h > @@ -17,7 +17,7 @@ > #define __ASM_PGTABLE_HWDEF_H > > #ifdef CONFIG_ARM64_64K_PAGES > -#include <asm/pgtable-2level-hwdef.h> > +#include <asm/pgtable-3level-hwdef.h> > #else > #include <asm/pgtable-4level-hwdef.h> > #endif So 2-level-hwdef.h is no longer used, and 4k pages definitely only works in 4-level configurations... > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index cc764e5..2c67b79 100644 > +#ifdef CONFIG_ARM64_64K_PAGES > +/* > + * Macro to populate the PMD for the corresponding block entry in the next > + * level (tbl) for the given virtual address. > + * > + * Preserves: pmd, tbl, virt > + * Corrupts: tmp1, tmp2 > + */ > + .macro create_pmd_entry, pud, tbl, virt, tmp1, tmp2 > + lsr \tmp1, \virt, #PMD_SHIFT > + and \tmp1, \tmp1, #PTRS_PER_PMD - 1 // PMD index > + orr \tmp2, \tbl, #3 // PMD entry table type > + str \tmp2, [\pud, \tmp1, lsl #3] > + .endm s/pud/pmd/ [...] > @@ -489,7 +509,11 @@ __create_page_tables: > * later based earlyprintk kernel parameter. > */ > ldr x5, =EARLYCON_IOBASE // UART virtual address > +#ifndef CONFIG_ARM64_64K_PAGES > add x0, x26, #PAGE_SIZE // section table address > +#else > + add x0, x26, #2 * PAGE_SIZE // section table address That comment should explain what's going on here. Currently its redundant and unhelpful. [...] > +#ifndef CONFIG_ARM64_64K_PAGES > void __pud_error(const char *file, int line, unsigned long val) > { > printk("%s:%d: bad pud %016lx.\n", file, line, val); > } > +#endif If __p*_error were unified we wouldn't need to #ifdef this here. Thanks, Mark. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-27 7:34 [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses mohun106 at gmail.com 2013-11-27 7:34 ` [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size mohun106 at gmail.com 2013-11-27 7:34 ` [RFC PATCH 2/2] arm64: Add 48-bit PA support for 64KB " mohun106 at gmail.com @ 2013-11-27 11:04 ` Marc Zyngier 2013-11-27 15:33 ` Radha Mohan 2013-11-28 9:29 ` Steve Capper 3 siblings, 1 reply; 19+ messages in thread From: Marc Zyngier @ 2013-11-27 11:04 UTC (permalink / raw) To: linux-arm-kernel [CC-ing the maintainers - seems odd they were not cc-ed the first place] On 27/11/13 07:34, mohun106 at gmail.com wrote: > From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > > This patch series provides an implementation of supporting 48-bit > Physical Addresses for ARMv8 platforms. It is the maximum width that > any ARMv8 based processor can support. > > The implementation extends the existing support of 40-bit PA.The kernel > and user space will now be able to access 128TB each. With 4KB page size > the Linux now will be using 4 levels of page tables by making use of > 'pud'. And with 64KB page size the Linux will be using 3 levels of page > tables. > > The code has been tested with LTP. Aside from finding out whether or not this is a useful change, this breaks KVM, more specifically the way kernel pages are mapped into HYP. Also, guests will still be limited to 40-bit IPAs, and the stage-2 output range needs to be addressed as well. M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-27 11:04 ` [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses Marc Zyngier @ 2013-11-27 15:33 ` Radha Mohan 2013-11-27 15:50 ` Will Deacon 2013-11-27 17:59 ` Marc Zyngier 0 siblings, 2 replies; 19+ messages in thread From: Radha Mohan @ 2013-11-27 15:33 UTC (permalink / raw) To: linux-arm-kernel On Wed, Nov 27, 2013 at 4:34 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > [CC-ing the maintainers - seems odd they were not cc-ed the first place] > > On 27/11/13 07:34, mohun106 at gmail.com wrote: >> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >> >> This patch series provides an implementation of supporting 48-bit >> Physical Addresses for ARMv8 platforms. It is the maximum width that >> any ARMv8 based processor can support. >> >> The implementation extends the existing support of 40-bit PA.The kernel >> and user space will now be able to access 128TB each. With 4KB page size >> the Linux now will be using 4 levels of page tables by making use of >> 'pud'. And with 64KB page size the Linux will be using 3 levels of page >> tables. >> >> The code has been tested with LTP. > > Aside from finding out whether or not this is a useful change, this > breaks KVM, more specifically the way kernel pages are mapped into HYP. > Also, guests will still be limited to 40-bit IPAs, and the stage-2 > output range needs to be addressed as well. This is useful for platforms (like ours) that will use 48-bit PAs. Sooner or later there will be such processors. And yes, I haven't checked on KVM. > > M. > -- > Jazz is not dead. It just smells funny... > ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-27 15:33 ` Radha Mohan @ 2013-11-27 15:50 ` Will Deacon 2013-11-27 17:59 ` Marc Zyngier 1 sibling, 0 replies; 19+ messages in thread From: Will Deacon @ 2013-11-27 15:50 UTC (permalink / raw) To: linux-arm-kernel On Wed, Nov 27, 2013 at 03:33:45PM +0000, Radha Mohan wrote: > On Wed, Nov 27, 2013 at 4:34 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > > [CC-ing the maintainers - seems odd they were not cc-ed the first place] > > > > On 27/11/13 07:34, mohun106 at gmail.com wrote: > >> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > >> > >> This patch series provides an implementation of supporting 48-bit > >> Physical Addresses for ARMv8 platforms. It is the maximum width that > >> any ARMv8 based processor can support. > >> > >> The implementation extends the existing support of 40-bit PA.The kernel > >> and user space will now be able to access 128TB each. With 4KB page size > >> the Linux now will be using 4 levels of page tables by making use of > >> 'pud'. And with 64KB page size the Linux will be using 3 levels of page > >> tables. > >> > >> The code has been tested with LTP. > > > > Aside from finding out whether or not this is a useful change, this > > breaks KVM, more specifically the way kernel pages are mapped into HYP. > > Also, guests will still be limited to 40-bit IPAs, and the stage-2 > > output range needs to be addressed as well. > > This is useful for platforms (like ours) that will use 48-bit PAs. > Sooner or later there will > be such processors. > And yes, I haven't checked on KVM. It may well break the ARM SMMU driver without further changes there, too. Will ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-27 15:33 ` Radha Mohan 2013-11-27 15:50 ` Will Deacon @ 2013-11-27 17:59 ` Marc Zyngier 2013-11-28 2:14 ` Radha Mohan 1 sibling, 1 reply; 19+ messages in thread From: Marc Zyngier @ 2013-11-27 17:59 UTC (permalink / raw) To: linux-arm-kernel On 27/11/13 15:33, Radha Mohan wrote: > On Wed, Nov 27, 2013 at 4:34 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >> [CC-ing the maintainers - seems odd they were not cc-ed the first place] >> >> On 27/11/13 07:34, mohun106 at gmail.com wrote: >>> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >>> >>> This patch series provides an implementation of supporting 48-bit >>> Physical Addresses for ARMv8 platforms. It is the maximum width that >>> any ARMv8 based processor can support. >>> >>> The implementation extends the existing support of 40-bit PA.The kernel >>> and user space will now be able to access 128TB each. With 4KB page size >>> the Linux now will be using 4 levels of page tables by making use of >>> 'pud'. And with 64KB page size the Linux will be using 3 levels of page >>> tables. >>> >>> The code has been tested with LTP. >> >> Aside from finding out whether or not this is a useful change, this >> breaks KVM, more specifically the way kernel pages are mapped into HYP. >> Also, guests will still be limited to 40-bit IPAs, and the stage-2 >> output range needs to be addressed as well. > > This is useful for platforms (like ours) that will use 48-bit PAs. > Sooner or later there will > be such processors. So should the 4-level page table cost be unconditionally forced onto all implementations? > And yes, I haven't checked on KVM. Yeah, tiny detail... ;-) M. -- Jazz is not dead. It just smells funny... ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-27 17:59 ` Marc Zyngier @ 2013-11-28 2:14 ` Radha Mohan 2013-12-05 14:35 ` Radha Mohan 2013-12-05 16:53 ` Catalin Marinas 0 siblings, 2 replies; 19+ messages in thread From: Radha Mohan @ 2013-11-28 2:14 UTC (permalink / raw) To: linux-arm-kernel On Wed, Nov 27, 2013 at 11:29 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > On 27/11/13 15:33, Radha Mohan wrote: >> On Wed, Nov 27, 2013 at 4:34 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >>> [CC-ing the maintainers - seems odd they were not cc-ed the first place] >>> >>> On 27/11/13 07:34, mohun106 at gmail.com wrote: >>>> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >>>> >>>> This patch series provides an implementation of supporting 48-bit >>>> Physical Addresses for ARMv8 platforms. It is the maximum width that >>>> any ARMv8 based processor can support. >>>> >>>> The implementation extends the existing support of 40-bit PA.The kernel >>>> and user space will now be able to access 128TB each. With 4KB page size >>>> the Linux now will be using 4 levels of page tables by making use of >>>> 'pud'. And with 64KB page size the Linux will be using 3 levels of page >>>> tables. >>>> >>>> The code has been tested with LTP. >>> >>> Aside from finding out whether or not this is a useful change, this >>> breaks KVM, more specifically the way kernel pages are mapped into HYP. >>> Also, guests will still be limited to 40-bit IPAs, and the stage-2 >>> output range needs to be addressed as well. >> >> This is useful for platforms (like ours) that will use 48-bit PAs. >> Sooner or later there will >> be such processors. > > So should the 4-level page table cost be unconditionally forced onto all > implementations? > We can have a kernel compile-time config option so that platforms having a 48-bit PA can select that option. But having it will create the code something like below #ifndef CONFIG_ARM64_64KB_PAGES #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA ... .. #else .. .. #endif #else /* !CONFIG_ARM64_64KB_PAGES */ #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA .. .. #else .. .. #endif #endif /* CONFIG_ARM64_64KB_PAGES */ And we might need to do this at lot of places. Is this ok? >> And yes, I haven't checked on KVM. > > Yeah, tiny detail... ;-) > > M. > -- > Jazz is not dead. It just smells funny... > ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-28 2:14 ` Radha Mohan @ 2013-12-05 14:35 ` Radha Mohan 2013-12-05 15:10 ` Marc Zyngier 2013-12-05 17:30 ` Catalin Marinas 2013-12-05 16:53 ` Catalin Marinas 1 sibling, 2 replies; 19+ messages in thread From: Radha Mohan @ 2013-12-05 14:35 UTC (permalink / raw) To: linux-arm-kernel On Thu, Nov 28, 2013 at 7:44 AM, Radha Mohan <mohun106@gmail.com> wrote: > On Wed, Nov 27, 2013 at 11:29 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >> On 27/11/13 15:33, Radha Mohan wrote: >>> On Wed, Nov 27, 2013 at 4:34 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: >>>> [CC-ing the maintainers - seems odd they were not cc-ed the first place] >>>> >>>> On 27/11/13 07:34, mohun106 at gmail.com wrote: >>>>> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >>>>> >>>>> This patch series provides an implementation of supporting 48-bit >>>>> Physical Addresses for ARMv8 platforms. It is the maximum width that >>>>> any ARMv8 based processor can support. >>>>> >>>>> The implementation extends the existing support of 40-bit PA.The kernel >>>>> and user space will now be able to access 128TB each. With 4KB page size >>>>> the Linux now will be using 4 levels of page tables by making use of >>>>> 'pud'. And with 64KB page size the Linux will be using 3 levels of page >>>>> tables. >>>>> >>>>> The code has been tested with LTP. >>>> >>>> Aside from finding out whether or not this is a useful change, this >>>> breaks KVM, more specifically the way kernel pages are mapped into HYP. >>>> Also, guests will still be limited to 40-bit IPAs, and the stage-2 >>>> output range needs to be addressed as well. >>> >>> This is useful for platforms (like ours) that will use 48-bit PAs. >>> Sooner or later there will >>> be such processors. >> >> So should the 4-level page table cost be unconditionally forced onto all >> implementations? >> > > We can have a kernel compile-time config option so that platforms having > a 48-bit PA can select that option. But having it will create the code something > like below > > #ifndef CONFIG_ARM64_64KB_PAGES > #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA > ... > .. > #else > .. > .. > #endif > #else /* !CONFIG_ARM64_64KB_PAGES */ > #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA > .. > .. > #else > .. > .. > #endif > #endif /* CONFIG_ARM64_64KB_PAGES */ > > And we might need to do this at lot of places. Is this ok? > Before going for a next version of patch to fix breaking SMMU and KVM, I would like to know about the best possible implementation. Should I go for a compile time option? I am not in favor of this. What are options if in future there's 16KB page support in the kernel. Penalties for platforms that do not have a 48-bit PA are an extra lookup and extra memory space for another pagetable. We don't have an actual hardware to get an reliable data. If someone can try this on a real hardware I would like to know. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-12-05 14:35 ` Radha Mohan @ 2013-12-05 15:10 ` Marc Zyngier 2013-12-05 17:30 ` Catalin Marinas 1 sibling, 0 replies; 19+ messages in thread From: Marc Zyngier @ 2013-12-05 15:10 UTC (permalink / raw) To: linux-arm-kernel On 2013-12-05 14:35, Radha Mohan wrote: > On Thu, Nov 28, 2013 at 7:44 AM, Radha Mohan <mohun106@gmail.com> > wrote: >> On Wed, Nov 27, 2013 at 11:29 PM, Marc Zyngier >> <marc.zyngier@arm.com> wrote: >>> On 27/11/13 15:33, Radha Mohan wrote: >>>> On Wed, Nov 27, 2013 at 4:34 PM, Marc Zyngier >>>> <marc.zyngier@arm.com> wrote: >>>>> [CC-ing the maintainers - seems odd they were not cc-ed the first >>>>> place] >>>>> >>>>> On 27/11/13 07:34, mohun106 at gmail.com wrote: >>>>>> From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> >>>>>> >>>>>> This patch series provides an implementation of supporting >>>>>> 48-bit >>>>>> Physical Addresses for ARMv8 platforms. It is the maximum width >>>>>> that >>>>>> any ARMv8 based processor can support. >>>>>> >>>>>> The implementation extends the existing support of 40-bit PA.The >>>>>> kernel >>>>>> and user space will now be able to access 128TB each. With 4KB >>>>>> page size >>>>>> the Linux now will be using 4 levels of page tables by making >>>>>> use of >>>>>> 'pud'. And with 64KB page size the Linux will be using 3 levels >>>>>> of page >>>>>> tables. >>>>>> >>>>>> The code has been tested with LTP. >>>>> >>>>> Aside from finding out whether or not this is a useful change, >>>>> this >>>>> breaks KVM, more specifically the way kernel pages are mapped >>>>> into HYP. >>>>> Also, guests will still be limited to 40-bit IPAs, and the >>>>> stage-2 >>>>> output range needs to be addressed as well. >>>> >>>> This is useful for platforms (like ours) that will use 48-bit PAs. >>>> Sooner or later there will >>>> be such processors. >>> >>> So should the 4-level page table cost be unconditionally forced >>> onto all >>> implementations? >>> >> >> We can have a kernel compile-time config option so that platforms >> having >> a 48-bit PA can select that option. But having it will create the >> code something >> like below >> >> #ifndef CONFIG_ARM64_64KB_PAGES >> #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA >> ... >> .. >> #else >> .. >> .. >> #endif >> #else /* !CONFIG_ARM64_64KB_PAGES */ >> #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA >> .. >> .. >> #else >> .. >> .. >> #endif >> #endif /* CONFIG_ARM64_64KB_PAGES */ >> >> And we might need to do this at lot of places. Is this ok? >> > > Before going for a next version of patch to fix breaking SMMU and > KVM, > I would like to know about the best possible implementation. > Should I go for a compile time option? I am not in favor of this. > What > are options if in future there's 16KB page support in the kernel. > > Penalties for platforms that do not have a 48-bit PA are an extra > lookup and extra memory space for another pagetable. We don't have an > actual hardware to get an reliable data. > If someone can try this on a real hardware I would like to know. This is really a questions for Catalin and Will, but my first approach would be that because 3 level page-tables are well understood and tested, they should remain the default. 48bit PA system are likely to be rare for quite a while, and that leaves us some time to evaluate if we're happy to take the overhead on all of the arm64 platforms or not. Over time, and assuming we grow confident enough about the stability of the solution and that the performance hit is within acceptable limits, we can look at making it the default. But in my opinion, the basic approach should be "this is optional". Cheers, M. -- Fast, cheap, reliable. Pick two. ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-12-05 14:35 ` Radha Mohan 2013-12-05 15:10 ` Marc Zyngier @ 2013-12-05 17:30 ` Catalin Marinas 2013-12-10 16:53 ` Radha Mohan 1 sibling, 1 reply; 19+ messages in thread From: Catalin Marinas @ 2013-12-05 17:30 UTC (permalink / raw) To: linux-arm-kernel On Thu, Dec 05, 2013 at 02:35:52PM +0000, Radha Mohan wrote: > Before going for a next version of patch to fix breaking SMMU and KVM, > I would like to know about the best possible implementation. > Should I go for a compile time option? I am not in favor of this. What > are options if in future there's 16KB page support in the kernel. As I said in the other email, separate the shifts definitions (primarily page size dependent) from the type definitions (dependent on the number of levels). Just to understand your requirements better. Does your platform have more than 256GB of RAM? With the current configuration, the VA space is 39-bit with 4K (which would allow 512GB minus some vmalloc and modules space) and 42-bit with 64K pages (2TB minus vmalloc/modules). -- Catalin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-12-05 17:30 ` Catalin Marinas @ 2013-12-10 16:53 ` Radha Mohan 2013-12-10 17:02 ` Catalin Marinas 0 siblings, 1 reply; 19+ messages in thread From: Radha Mohan @ 2013-12-10 16:53 UTC (permalink / raw) To: linux-arm-kernel On Thu, Dec 5, 2013 at 11:00 PM, Catalin Marinas <catalin.marinas@arm.com> wrote: > On Thu, Dec 05, 2013 at 02:35:52PM +0000, Radha Mohan wrote: >> Before going for a next version of patch to fix breaking SMMU and KVM, >> I would like to know about the best possible implementation. >> Should I go for a compile time option? I am not in favor of this. What >> are options if in future there's 16KB page support in the kernel. > > As I said in the other email, separate the shifts definitions (primarily > page size dependent) from the type definitions (dependent on the number > of levels). > > Just to understand your requirements better. Does your platform have > more than 256GB of RAM? With the current configuration, the VA space is > 39-bit with 4K (which would allow 512GB minus some vmalloc and modules > space) and 42-bit with 64K pages (2TB minus vmalloc/modules). Sorry for the delay in responding. More than 256GB DRAM is a likely possibility. But I am unsure of the exact page size that would be used. That is the reason for expanding the VA bits to full and have flexibility for customers. As Marc mentioned this is a rare requirement for now in ARMv8 processors, but would be nice to have a plan for long term. I would prefer a runtime support for this i.e reading the ID_AA64MMFR0_EL1.PARange value but I think this would be a major rework. At the least I plan to support this with current implementation i.e, 3-levels(4K) / 2-levels(64K) itself. This way it would need minimal changes for a 48-bit PA. > > -- > Catalin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-12-10 16:53 ` Radha Mohan @ 2013-12-10 17:02 ` Catalin Marinas 2013-12-11 9:18 ` Radha Mohan 0 siblings, 1 reply; 19+ messages in thread From: Catalin Marinas @ 2013-12-10 17:02 UTC (permalink / raw) To: linux-arm-kernel On Tue, Dec 10, 2013 at 04:53:24PM +0000, Radha Mohan wrote: > On Thu, Dec 5, 2013 at 11:00 PM, Catalin Marinas > <catalin.marinas@arm.com> wrote: > > On Thu, Dec 05, 2013 at 02:35:52PM +0000, Radha Mohan wrote: > >> Before going for a next version of patch to fix breaking SMMU and KVM, > >> I would like to know about the best possible implementation. > >> Should I go for a compile time option? I am not in favor of this. What > >> are options if in future there's 16KB page support in the kernel. > > > > As I said in the other email, separate the shifts definitions (primarily > > page size dependent) from the type definitions (dependent on the number > > of levels). > > > > Just to understand your requirements better. Does your platform have > > more than 256GB of RAM? With the current configuration, the VA space is > > 39-bit with 4K (which would allow 512GB minus some vmalloc and modules > > space) and 42-bit with 64K pages (2TB minus vmalloc/modules). > > Sorry for the delay in responding. > More than 256GB DRAM is a likely possibility. But I am unsure of the > exact page size that would be used. That is the reason for expanding > the VA bits to full and have flexibility for customers. As Marc > mentioned this is a rare requirement for now in ARMv8 processors, but > would be nice to have a plan for long term. I would prefer a runtime > support for this i.e reading the ID_AA64MMFR0_EL1.PARange value but I > think this would be a major rework. Don't confuse the physical address range (given by PARange) with the virtual address range. On arm64 we don't have to use highmem to access the RAM, so if you have lots of RAM (maximum architecture limit with 3-levels and 4K pages is 512GB) we need to expand the VA space to map it. But just having 48-bit PA space doesn't necessarily require the same VA space (unless for some weird reason all the RAM lives beyond bit 39 which would require the idmap to have 4 levels, but even then we could do this just for a static idmap table). -- Catalin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-12-10 17:02 ` Catalin Marinas @ 2013-12-11 9:18 ` Radha Mohan 0 siblings, 0 replies; 19+ messages in thread From: Radha Mohan @ 2013-12-11 9:18 UTC (permalink / raw) To: linux-arm-kernel On Tue, Dec 10, 2013 at 10:32 PM, Catalin Marinas <catalin.marinas@arm.com> wrote: > On Tue, Dec 10, 2013 at 04:53:24PM +0000, Radha Mohan wrote: >> On Thu, Dec 5, 2013 at 11:00 PM, Catalin Marinas >> <catalin.marinas@arm.com> wrote: >> > On Thu, Dec 05, 2013 at 02:35:52PM +0000, Radha Mohan wrote: >> >> Before going for a next version of patch to fix breaking SMMU and KVM, >> >> I would like to know about the best possible implementation. >> >> Should I go for a compile time option? I am not in favor of this. What >> >> are options if in future there's 16KB page support in the kernel. >> > >> > As I said in the other email, separate the shifts definitions (primarily >> > page size dependent) from the type definitions (dependent on the number >> > of levels). >> > >> > Just to understand your requirements better. Does your platform have >> > more than 256GB of RAM? With the current configuration, the VA space is >> > 39-bit with 4K (which would allow 512GB minus some vmalloc and modules >> > space) and 42-bit with 64K pages (2TB minus vmalloc/modules). >> >> Sorry for the delay in responding. >> More than 256GB DRAM is a likely possibility. But I am unsure of the >> exact page size that would be used. That is the reason for expanding >> the VA bits to full and have flexibility for customers. As Marc >> mentioned this is a rare requirement for now in ARMv8 processors, but >> would be nice to have a plan for long term. I would prefer a runtime >> support for this i.e reading the ID_AA64MMFR0_EL1.PARange value but I >> think this would be a major rework. > > Don't confuse the physical address range (given by PARange) with the > virtual address range. On arm64 we don't have to use highmem to access > the RAM, so if you have lots of RAM (maximum architecture limit with > 3-levels and 4K pages is 512GB) we need to expand the VA space to map > it. But just having 48-bit PA space doesn't necessarily require the same > VA space (unless for some weird reason all the RAM lives beyond bit 39 > which would require the idmap to have 4 levels, but even then we could > do this just for a static idmap table). OK. I will stick to 3-levels for now and redo the patch for 48-bit PA. > > -- > Catalin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-28 2:14 ` Radha Mohan 2013-12-05 14:35 ` Radha Mohan @ 2013-12-05 16:53 ` Catalin Marinas 1 sibling, 0 replies; 19+ messages in thread From: Catalin Marinas @ 2013-12-05 16:53 UTC (permalink / raw) To: linux-arm-kernel On Thu, Nov 28, 2013 at 02:14:30AM +0000, Radha Mohan wrote: > On Wed, Nov 27, 2013 at 11:29 PM, Marc Zyngier <marc.zyngier@arm.com> wrote: > > So should the 4-level page table cost be unconditionally forced onto all > > implementations? Definitely not. > We can have a kernel compile-time config option so that platforms having > a 48-bit PA can select that option. But having it will create the code something > like below > > #ifndef CONFIG_ARM64_64KB_PAGES > #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA > ... > .. > #else > .. > .. > #endif > #else /* !CONFIG_ARM64_64KB_PAGES */ > #ifdef CONFIG_ARCH_SUPPORTS_48BIT_PA > .. > .. > #else > .. > .. > #endif > #endif /* CONFIG_ARM64_64KB_PAGES */ > > And we might need to do this at lot of places. Is this ok? We have to figure a way to avoid this. I haven't looked in detail at the patches (BTW, please cc the arm64 maintainers on the next version) but I think we could restrict the #ifdef's to much fewer headers (primarily asm/page.h and asm/pgtable-hwdef.h). What I'm thinking is to have: pgtable-2level-*.h 64K pages only pgtable-3level-*.h 4K and 64K pages support pgtable-4level-*.h 4K pages only You can move the PTRS_PER_* and PMD_SHIFT etc. macros to some common header as these depend on the page size. The other definitions like pmd_t depend on the number of page table levels. With some care you could define PMD_SHIFT for 64K into a single file with some #ifdef's around __PAGETABLE_PMD_FOLDED. Same for PUD_SHIFT with 4K pages. -- Catalin ^ permalink raw reply [flat|nested] 19+ messages in thread
* [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses 2013-11-27 7:34 [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses mohun106 at gmail.com ` (2 preceding siblings ...) 2013-11-27 11:04 ` [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses Marc Zyngier @ 2013-11-28 9:29 ` Steve Capper 3 siblings, 0 replies; 19+ messages in thread From: Steve Capper @ 2013-11-28 9:29 UTC (permalink / raw) To: linux-arm-kernel On Wed, Nov 27, 2013 at 01:04:23PM +0530, mohun106 at gmail.com wrote: > From: Radha Mohan Chintakuntla <rchintakuntla@cavium.com> > > This patch series provides an implementation of supporting 48-bit > Physical Addresses for ARMv8 platforms. It is the maximum width that > any ARMv8 based processor can support. > > The implementation extends the existing support of 40-bit PA.The kernel > and user space will now be able to access 128TB each. With 4KB page size > the Linux now will be using 4 levels of page tables by making use of > 'pud'. And with 64KB page size the Linux will be using 3 levels of page > tables. > > The code has been tested with LTP. Hello, Could you please test how this series interacts with huge pages? So please run LTP with THP enabled and always active: TRANSPARENT_HUGEPAGE=y TRANSPARENT_HUGEPAGE_ALWAYS=y (for both 4KB and 64KB base page sizes). This will give the pmd level code a workout. Then could you please try libhugetlbfs to test HugeTLB? git://git.code.sf.net/p/libhugetlbfs/code (next branch) with the following enabled: HUGETLBFS=y Please run the test suite against 2MB, 1GB (4KB base pages) and 512MB (64KB base pages) huge pages. To set aside 1GB huge pages at boot time, please add the following to the kernel command line: hugepagesz=1G hugepages=x where x is the number of 1GB huge pages you can get away with :-). For 2MB and 512MB: echo x > /proc/sys/vm/nr_hugepages again where x is the number of huge pages to set aside. Ensure that the hugetlbfs filesystem is mounted: mount -t hugetlbfs hugetlbfs /mnt/hugepages Then make check The libhugetlbfs tests above will test out the pud and pmd interactions, please let me know whether or not anything breaks. Thanks, -- Steve ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2013-12-11 9:18 UTC | newest] Thread overview: 19+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-11-27 7:34 [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses mohun106 at gmail.com 2013-11-27 7:34 ` [RFC PATCH 1/2] arm64: Add 48-bit PA support for 4KB page size mohun106 at gmail.com 2013-11-27 11:14 ` Mark Rutland 2013-11-27 16:00 ` Radha Mohan 2013-11-27 7:34 ` [RFC PATCH 2/2] arm64: Add 48-bit PA support for 64KB " mohun106 at gmail.com 2013-11-27 11:30 ` Mark Rutland 2013-11-27 11:04 ` [RFC PATCH 0/2] arm64: Add support for 48-bit Physical Addresses Marc Zyngier 2013-11-27 15:33 ` Radha Mohan 2013-11-27 15:50 ` Will Deacon 2013-11-27 17:59 ` Marc Zyngier 2013-11-28 2:14 ` Radha Mohan 2013-12-05 14:35 ` Radha Mohan 2013-12-05 15:10 ` Marc Zyngier 2013-12-05 17:30 ` Catalin Marinas 2013-12-10 16:53 ` Radha Mohan 2013-12-10 17:02 ` Catalin Marinas 2013-12-11 9:18 ` Radha Mohan 2013-12-05 16:53 ` Catalin Marinas 2013-11-28 9:29 ` Steve Capper
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).