* [PATCH v2 0/8] RISCV device tree mapping
@ 2024-07-12 16:22 Oleksii Kurochko
2024-07-12 16:22 ` [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common Oleksii Kurochko
` (7 more replies)
0 siblings, 8 replies; 51+ messages in thread
From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw)
To: xen-devel
Cc: Oleksii Kurochko, Andrew Cooper, Jan Beulich, Julien Grall,
Stefano Stabellini, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk, Daniel P. Smith, Alistair Francis,
Bob Eshleman, Connor Davis
Current patch series introduces device tree mapping for RISC-V
and necessary things for that such as:
- Fixmap mapping
- pmap
- Xen page table processing
Also, it introduces common stuff for working with fdt which is
based on the patches from [1]:
[PATCH v4 2/6] xen/device-tree: Move Arm's setup.c bootinfo functions to common
[PATCH v4 3/6] xen/common: Move Arm's bootfdt.c
All changes which were done on top of Shawn's patches please find in "Changes" section
of each patch.
Also there is another one depenency:
RISCV basic exception handling implementation [2]
[1] https://lore.kernel.org/xen-devel/cover.1712893887.git.sanastasio@raptorengineering.com/
[2] https://lore.kernel.org/xen-devel/cover.1720799487.git.oleksii.kurochko@gmail.com/T/#t
---
Changes in v2:
- Update the cover letter message
- introduce fixmap mapping
- introduce pmap
- introduce CONFIG_GENREIC_PT
- update use early_fdt_map() after MMU is enabled.
---
Oleksii Kurochko (6):
xen/riscv: enable CONFIG_HAS_DEVICE_TREE
xen/riscv: setup fixmap mapping
xen/riscv: introduce asm/pmap.h header
xen/riscv: introduce generic Xen page table handling
xen/riscv: select CONFIG_GENREIC_PT
xen/riscv: introduce early_fdt_map()
Shawn Anastasio (2):
xen/device-tree: Move Arm's setup.c bootinfo functions to common
xen/common: Move Arm's bootfdt.c to common
MAINTAINERS | 2 +
xen/arch/arm/Makefile | 1 -
xen/arch/arm/bootfdt.c | 622 ------------------------
xen/arch/arm/include/asm/setup.h | 200 +-------
xen/arch/arm/setup.c | 432 -----------------
xen/arch/riscv/Kconfig | 3 +
xen/arch/riscv/include/asm/config.h | 9 +
xen/arch/riscv/include/asm/fixmap.h | 48 ++
xen/arch/riscv/include/asm/mm.h | 13 +
xen/arch/riscv/include/asm/page-bits.h | 45 ++
xen/arch/riscv/include/asm/page.h | 82 +++-
xen/arch/riscv/include/asm/pmap.h | 28 ++
xen/arch/riscv/mm.c | 167 ++++++-
xen/arch/riscv/setup.c | 11 +
xen/arch/riscv/xen.lds.S | 2 +-
xen/common/Kconfig | 5 +
xen/common/Makefile | 2 +
xen/common/device-tree/Makefile | 2 +
xen/common/device-tree/bootfdt.c | 635 +++++++++++++++++++++++++
xen/common/device-tree/bootinfo.c | 459 ++++++++++++++++++
xen/common/mmu/pt.c | 441 +++++++++++++++++
xen/include/xen/bootfdt.h | 210 ++++++++
xen/include/xen/mm.h | 24 +
23 files changed, 2181 insertions(+), 1262 deletions(-)
delete mode 100644 xen/arch/arm/bootfdt.c
create mode 100644 xen/arch/riscv/include/asm/fixmap.h
create mode 100644 xen/arch/riscv/include/asm/pmap.h
create mode 100644 xen/common/device-tree/Makefile
create mode 100644 xen/common/device-tree/bootfdt.c
create mode 100644 xen/common/device-tree/bootinfo.c
create mode 100644 xen/common/mmu/pt.c
create mode 100644 xen/include/xen/bootfdt.h
--
2.45.2
^ permalink raw reply [flat|nested] 51+ messages in thread* [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-15 8:36 ` Jan Beulich 2024-07-15 10:15 ` Michal Orzel 2024-07-12 16:22 ` [PATCH v6 2/8] xen/common: Move Arm's bootfdt.c " Oleksii Kurochko ` (6 subsequent siblings) 7 siblings, 2 replies; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Shawn Anastasio, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Daniel P. Smith, Oleksii Kurochko From: Shawn Anastasio <sanastasio@raptorengineering.com> Arm's setup.c contains a collection of functions for parsing memory map and other boot information from a device tree. Since these routines are generally useful on any architecture that supports device tree booting, move them into xen/common/device-tree. Suggested-by: Julien Grall <julien@xen.org> Signed-off-by: Shawn Anastasio <sanastasio@raptorengineering.com> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V6: - update the version of the patch to v6, to show that it is based on the work done by Shawn in the patch v4. --- Changes in V5: - add xen/include/xen/bootfdt.h to MAINTAINERS file. - drop message "Early device tree parsing and". - After rebase on top of the current staging the following changes were done: - init bootinfo variable in <common/device-tree/bootinfo.c> with BOOTINFO_INIT; - update the code of dt_unreserved_regions(): CONFIG_STATIC_SHM related changes and getting of reserved_mem bootinfo_get_shmem() ?? - update the code of meminfo_overlap_check(): add check ( INVALID_PADDR == bank_start ) to if case. - update the code of check_reserved_regions_overlap(): CONFIG_STATIC_SHM related changes. - struct bootinfo was updated ( CONFIG_STATIC_SHM changes ) - add shared_meminfo ( because of CONFIG_STATIC_SHM ) - struct struct membanks was update with __struct group so <xen/kernel> is neeeded to be included in bootfdt.h - move BOOTINFO_ACPI_INIT, BOOTINFO_SHMEM_INIT, BOOTINFO_INIT to generic bootfdt.h - bootinfo_get_reserved_mem(), bootinfo_get_mem(), bootinfo_get_acpi(), bootinfo_get_shmem() and bootinfo_get_shmem_extra() were moved to xen/bootfdt.h - s/arm32/CONFIG_SEPARATE_XENHEAP/ - add inclusion of <xen/macros.h> because there are function in <xen/bootfdt.h> which are using container_of(). --- Changes in v4: - create new xen/include/bootinfo.h rather than relying on arch's asm/setup.h to provide required definitions for bootinfo.c - build bootinfo.c as .init.o - clean up and sort bootinfo.c's #includes - use CONFIG_SEPARATE_XENHEAP rather than CONFIG_ARM_32 to guard xenheap-specific behavior of populate_boot_allocator - (MAINTAINERS) include all of common/device-tree rather than just bootinfo.c --- MAINTAINERS | 2 + xen/arch/arm/include/asm/setup.h | 187 +----------- xen/arch/arm/setup.c | 432 ---------------------------- xen/common/Makefile | 1 + xen/common/device-tree/Makefile | 1 + xen/common/device-tree/bootinfo.c | 459 ++++++++++++++++++++++++++++++ xen/include/xen/bootfdt.h | 196 +++++++++++++ 7 files changed, 660 insertions(+), 618 deletions(-) create mode 100644 xen/common/device-tree/Makefile create mode 100644 xen/common/device-tree/bootinfo.c create mode 100644 xen/include/xen/bootfdt.h diff --git a/MAINTAINERS b/MAINTAINERS index 2b0c894527..505915b6b6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -295,9 +295,11 @@ M: Stefano Stabellini <sstabellini@kernel.org> M: Julien Grall <julien@xen.org> S: Supported F: xen/common/libfdt/ +F: xen/common/device-tree/ F: xen/common/device_tree.c F: xen/common/dt-overlay.c F: xen/include/xen/libfdt/ +F: xen/include/xen/bootfdt.h F: xen/include/xen/device_tree.h F: xen/drivers/passthrough/device_tree.c diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index c34179da93..051e796716 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -3,159 +3,9 @@ #include <public/version.h> #include <asm/p2m.h> +#include <xen/bootfdt.h> #include <xen/device_tree.h> -#define MIN_FDT_ALIGN 8 -#define MAX_FDT_SIZE SZ_2M - -#define NR_MEM_BANKS 256 -#define NR_SHMEM_BANKS 32 - -#define MAX_MODULES 32 /* Current maximum useful modules */ - -typedef enum { - BOOTMOD_XEN, - BOOTMOD_FDT, - BOOTMOD_KERNEL, - BOOTMOD_RAMDISK, - BOOTMOD_XSM, - BOOTMOD_GUEST_DTB, - BOOTMOD_UNKNOWN -} bootmodule_kind; - -enum membank_type { - /* - * The MEMBANK_DEFAULT type refers to either reserved memory for the - * device/firmware (when the bank is in 'reserved_mem') or any RAM (when - * the bank is in 'mem'). - */ - MEMBANK_DEFAULT, - /* - * The MEMBANK_STATIC_DOMAIN type is used to indicate whether the memory - * bank is bound to a static Xen domain. It is only valid when the bank - * is in reserved_mem. - */ - MEMBANK_STATIC_DOMAIN, - /* - * The MEMBANK_STATIC_HEAP type is used to indicate whether the memory - * bank is reserved as static heap. It is only valid when the bank is - * in reserved_mem. - */ - MEMBANK_STATIC_HEAP, - /* - * The MEMBANK_FDT_RESVMEM type is used to indicate whether the memory - * bank is from the FDT reserve map. - */ - MEMBANK_FDT_RESVMEM, -}; - -/* Indicates the maximum number of characters(\0 included) for shm_id */ -#define MAX_SHM_ID_LENGTH 16 - -struct shmem_membank_extra { - char shm_id[MAX_SHM_ID_LENGTH]; - unsigned int nr_shm_borrowers; -}; - -struct membank { - paddr_t start; - paddr_t size; - union { - enum membank_type type; -#ifdef CONFIG_STATIC_SHM - struct shmem_membank_extra *shmem_extra; -#endif - }; -}; - -struct membanks { - __struct_group(membanks_hdr, common, , - unsigned int nr_banks; - unsigned int max_banks; - ); - struct membank bank[]; -}; - -struct meminfo { - struct membanks_hdr common; - struct membank bank[NR_MEM_BANKS]; -}; - -struct shared_meminfo { - struct membanks_hdr common; - struct membank bank[NR_SHMEM_BANKS]; - struct shmem_membank_extra extra[NR_SHMEM_BANKS]; -}; - -/* - * The domU flag is set for kernels and ramdisks of "xen,domain" nodes. - * The purpose of the domU flag is to avoid getting confused in - * kernel_probe, where we try to guess which is the dom0 kernel and - * initrd to be compatible with all versions of the multiboot spec. - */ -#define BOOTMOD_MAX_CMDLINE 1024 -struct bootmodule { - bootmodule_kind kind; - bool domU; - paddr_t start; - paddr_t size; -}; - -/* DT_MAX_NAME is the node name max length according the DT spec */ -#define DT_MAX_NAME 41 -struct bootcmdline { - bootmodule_kind kind; - bool domU; - paddr_t start; - char dt_name[DT_MAX_NAME]; - char cmdline[BOOTMOD_MAX_CMDLINE]; -}; - -struct bootmodules { - int nr_mods; - struct bootmodule module[MAX_MODULES]; -}; - -struct bootcmdlines { - unsigned int nr_mods; - struct bootcmdline cmdline[MAX_MODULES]; -}; - -struct bootinfo { - struct meminfo mem; - /* The reserved regions are only used when booting using Device-Tree */ - struct meminfo reserved_mem; - struct bootmodules modules; - struct bootcmdlines cmdlines; -#ifdef CONFIG_ACPI - struct meminfo acpi; -#endif -#ifdef CONFIG_STATIC_SHM - struct shared_meminfo shmem; -#endif - bool static_heap; -}; - -#ifdef CONFIG_ACPI -#define BOOTINFO_ACPI_INIT .acpi.common.max_banks = NR_MEM_BANKS, -#else -#define BOOTINFO_ACPI_INIT -#endif - -#ifdef CONFIG_STATIC_SHM -#define BOOTINFO_SHMEM_INIT .shmem.common.max_banks = NR_SHMEM_BANKS, -#else -#define BOOTINFO_SHMEM_INIT -#endif - -#define BOOTINFO_INIT \ -{ \ - .mem.common.max_banks = NR_MEM_BANKS, \ - .reserved_mem.common.max_banks = NR_MEM_BANKS, \ - BOOTINFO_ACPI_INIT \ - BOOTINFO_SHMEM_INIT \ -} - struct map_range_data { struct domain *d; @@ -167,39 +17,8 @@ struct map_range_data struct rangeset *irq_ranges; }; -extern struct bootinfo bootinfo; - extern domid_t max_init_domid; -static inline struct membanks *bootinfo_get_mem(void) -{ - return container_of(&bootinfo.mem.common, struct membanks, common); -} - -static inline struct membanks *bootinfo_get_reserved_mem(void) -{ - return container_of(&bootinfo.reserved_mem.common, struct membanks, common); -} - -#ifdef CONFIG_ACPI -static inline struct membanks *bootinfo_get_acpi(void) -{ - return container_of(&bootinfo.acpi.common, struct membanks, common); -} -#endif - -#ifdef CONFIG_STATIC_SHM -static inline struct membanks *bootinfo_get_shmem(void) -{ - return container_of(&bootinfo.shmem.common, struct membanks, common); -} - -static inline struct shmem_membank_extra *bootinfo_get_shmem_extra(void) -{ - return bootinfo.shmem.extra; -} -#endif - void copy_from_paddr(void *dst, paddr_t paddr, unsigned long len); size_t estimate_efi_size(unsigned int mem_nr_banks); @@ -220,9 +39,6 @@ void fw_unreserved_regions(paddr_t s, paddr_t e, void (*cb)(paddr_t ps, paddr_t pe), unsigned int first); -size_t boot_fdt_info(const void *fdt, paddr_t paddr); -const char *boot_fdt_cmdline(const void *fdt); - bool check_reserved_regions_overlap(paddr_t region_start, paddr_t region_size); struct bootmodule *add_boot_module(bootmodule_kind kind, @@ -237,7 +53,6 @@ struct bootcmdline * boot_cmdline_find_by_name(const char *name); const char *boot_module_kind_as_string(bootmodule_kind kind); void init_pdx(void); -void populate_boot_allocator(void); void setup_mm(void); extern uint32_t hyp_traps_vector[]; diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 0c2fdaceaf..cb2c0a16b8 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -48,8 +48,6 @@ #include <xsm/xsm.h> #include <asm/acpi.h> -struct bootinfo __initdata bootinfo = BOOTINFO_INIT; - /* * Sanitized version of cpuinfo containing only features available on all * cores (only on arm64 as there is no sanitization support on arm32). @@ -203,321 +201,6 @@ static void __init processor_id(void) processor_setup(); } -static void __init dt_unreserved_regions(paddr_t s, paddr_t e, - void (*cb)(paddr_t ps, paddr_t pe), - unsigned int first) -{ - const struct membanks *reserved_mem = bootinfo_get_reserved_mem(); -#ifdef CONFIG_STATIC_SHM - const struct membanks *shmem = bootinfo_get_shmem(); - unsigned int offset; -#endif - unsigned int i; - - /* - * i is the current bootmodule we are evaluating across all possible - * kinds. - */ - for ( i = first; i < reserved_mem->nr_banks; i++ ) - { - paddr_t r_s = reserved_mem->bank[i].start; - paddr_t r_e = r_s + reserved_mem->bank[i].size; - - if ( s < r_e && r_s < e ) - { - dt_unreserved_regions(r_e, e, cb, i + 1); - dt_unreserved_regions(s, r_s, cb, i + 1); - return; - } - } - -#ifdef CONFIG_STATIC_SHM - /* - * When retrieving the corresponding shared memory addresses - * below, we need to index the shmem->bank starting from 0, hence - * we need to use i - reserved_mem->nr_banks. - */ - offset = reserved_mem->nr_banks; - for ( ; i - offset < shmem->nr_banks; i++ ) - { - paddr_t r_s, r_e; - - r_s = shmem->bank[i - offset].start; - - /* Shared memory banks can contain INVALID_PADDR as start */ - if ( INVALID_PADDR == r_s ) - continue; - - r_e = r_s + shmem->bank[i - offset].size; - - if ( s < r_e && r_s < e ) - { - dt_unreserved_regions(r_e, e, cb, i + 1); - dt_unreserved_regions(s, r_s, cb, i + 1); - return; - } - } -#endif - - cb(s, e); -} - -/* - * TODO: '*_end' could be 0 if the bank/region is at the end of the physical - * address space. This is for now not handled as it requires more rework. - */ -static bool __init meminfo_overlap_check(const struct membanks *mem, - paddr_t region_start, - paddr_t region_size) -{ - paddr_t bank_start = INVALID_PADDR, bank_end = 0; - paddr_t region_end = region_start + region_size; - unsigned int i, bank_num = mem->nr_banks; - - for ( i = 0; i < bank_num; i++ ) - { - bank_start = mem->bank[i].start; - bank_end = bank_start + mem->bank[i].size; - - if ( INVALID_PADDR == bank_start || region_end <= bank_start || - region_start >= bank_end ) - continue; - else - { - printk("Region: [%#"PRIpaddr", %#"PRIpaddr") overlapping with bank[%u]: [%#"PRIpaddr", %#"PRIpaddr")\n", - region_start, region_end, i, bank_start, bank_end); - return true; - } - } - - return false; -} - -/* - * TODO: '*_end' could be 0 if the module/region is at the end of the physical - * address space. This is for now not handled as it requires more rework. - */ -static bool __init bootmodules_overlap_check(struct bootmodules *bootmodules, - paddr_t region_start, - paddr_t region_size) -{ - paddr_t mod_start = INVALID_PADDR, mod_end = 0; - paddr_t region_end = region_start + region_size; - unsigned int i, mod_num = bootmodules->nr_mods; - - for ( i = 0; i < mod_num; i++ ) - { - mod_start = bootmodules->module[i].start; - mod_end = mod_start + bootmodules->module[i].size; - - if ( region_end <= mod_start || region_start >= mod_end ) - continue; - else - { - printk("Region: [%#"PRIpaddr", %#"PRIpaddr") overlapping with mod[%u]: [%#"PRIpaddr", %#"PRIpaddr")\n", - region_start, region_end, i, mod_start, mod_end); - return true; - } - } - - return false; -} - -void __init fw_unreserved_regions(paddr_t s, paddr_t e, - void (*cb)(paddr_t ps, paddr_t pe), - unsigned int first) -{ - if ( acpi_disabled ) - dt_unreserved_regions(s, e, cb, first); - else - cb(s, e); -} - -/* - * Given an input physical address range, check if this range is overlapping - * with the existing reserved memory regions defined in bootinfo. - * Return true if the input physical address range is overlapping with any - * existing reserved memory regions, otherwise false. - */ -bool __init check_reserved_regions_overlap(paddr_t region_start, - paddr_t region_size) -{ - const struct membanks *mem_banks[] = { - bootinfo_get_reserved_mem(), -#ifdef CONFIG_ACPI - bootinfo_get_acpi(), -#endif -#ifdef CONFIG_STATIC_SHM - bootinfo_get_shmem(), -#endif - }; - unsigned int i; - - /* - * Check if input region is overlapping with reserved memory banks or - * ACPI EfiACPIReclaimMemory (when ACPI feature is enabled) or static - * shared memory banks (when static shared memory feature is enabled) - */ - for ( i = 0; i < ARRAY_SIZE(mem_banks); i++ ) - if ( meminfo_overlap_check(mem_banks[i], region_start, region_size) ) - return true; - - /* Check if input region is overlapping with bootmodules */ - if ( bootmodules_overlap_check(&bootinfo.modules, - region_start, region_size) ) - return true; - - return false; -} - -struct bootmodule __init *add_boot_module(bootmodule_kind kind, - paddr_t start, paddr_t size, - bool domU) -{ - struct bootmodules *mods = &bootinfo.modules; - struct bootmodule *mod; - unsigned int i; - - if ( mods->nr_mods == MAX_MODULES ) - { - printk("Ignoring %s boot module at %"PRIpaddr"-%"PRIpaddr" (too many)\n", - boot_module_kind_as_string(kind), start, start + size); - return NULL; - } - - if ( check_reserved_regions_overlap(start, size) ) - return NULL; - - for ( i = 0 ; i < mods->nr_mods ; i++ ) - { - mod = &mods->module[i]; - if ( mod->kind == kind && mod->start == start ) - { - if ( !domU ) - mod->domU = false; - return mod; - } - } - - mod = &mods->module[mods->nr_mods++]; - mod->kind = kind; - mod->start = start; - mod->size = size; - mod->domU = domU; - - return mod; -} - -/* - * boot_module_find_by_kind can only be used to return Xen modules (e.g - * XSM, DTB) or Dom0 modules. This is not suitable for looking up guest - * modules. - */ -struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind) -{ - struct bootmodules *mods = &bootinfo.modules; - struct bootmodule *mod; - int i; - for (i = 0 ; i < mods->nr_mods ; i++ ) - { - mod = &mods->module[i]; - if ( mod->kind == kind && !mod->domU ) - return mod; - } - return NULL; -} - -void __init add_boot_cmdline(const char *name, const char *cmdline, - bootmodule_kind kind, paddr_t start, bool domU) -{ - struct bootcmdlines *cmds = &bootinfo.cmdlines; - struct bootcmdline *cmd; - - if ( cmds->nr_mods == MAX_MODULES ) - { - printk("Ignoring %s cmdline (too many)\n", name); - return; - } - - cmd = &cmds->cmdline[cmds->nr_mods++]; - cmd->kind = kind; - cmd->domU = domU; - cmd->start = start; - - ASSERT(strlen(name) <= DT_MAX_NAME); - safe_strcpy(cmd->dt_name, name); - - if ( strlen(cmdline) > BOOTMOD_MAX_CMDLINE ) - panic("module %s command line too long\n", name); - safe_strcpy(cmd->cmdline, cmdline); -} - -/* - * boot_cmdline_find_by_kind can only be used to return Xen modules (e.g - * XSM, DTB) or Dom0 modules. This is not suitable for looking up guest - * modules. - */ -struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind) -{ - struct bootcmdlines *cmds = &bootinfo.cmdlines; - struct bootcmdline *cmd; - int i; - - for ( i = 0 ; i < cmds->nr_mods ; i++ ) - { - cmd = &cmds->cmdline[i]; - if ( cmd->kind == kind && !cmd->domU ) - return cmd; - } - return NULL; -} - -struct bootcmdline * __init boot_cmdline_find_by_name(const char *name) -{ - struct bootcmdlines *mods = &bootinfo.cmdlines; - struct bootcmdline *mod; - unsigned int i; - - for (i = 0 ; i < mods->nr_mods ; i++ ) - { - mod = &mods->cmdline[i]; - if ( strcmp(mod->dt_name, name) == 0 ) - return mod; - } - return NULL; -} - -struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind, - paddr_t start) -{ - struct bootmodules *mods = &bootinfo.modules; - struct bootmodule *mod; - unsigned int i; - - for (i = 0 ; i < mods->nr_mods ; i++ ) - { - mod = &mods->module[i]; - if ( mod->kind == kind && mod->start == start ) - return mod; - } - return NULL; -} - -const char * __init boot_module_kind_as_string(bootmodule_kind kind) -{ - switch ( kind ) - { - case BOOTMOD_XEN: return "Xen"; - case BOOTMOD_FDT: return "Device Tree"; - case BOOTMOD_KERNEL: return "Kernel"; - case BOOTMOD_RAMDISK: return "Ramdisk"; - case BOOTMOD_XSM: return "XSM"; - case BOOTMOD_GUEST_DTB: return "DTB"; - case BOOTMOD_UNKNOWN: return "Unknown"; - default: BUG(); - } -} - void __init discard_initial_modules(void) { struct bootmodules *mi = &bootinfo.modules; @@ -556,40 +239,6 @@ static void * __init relocate_fdt(paddr_t dtb_paddr, size_t dtb_size) return fdt; } -/* - * Return the end of the non-module region starting at s. In other - * words return s the start of the next modules after s. - * - * On input *end is the end of the region which should be considered - * and it is updated to reflect the end of the module, clipped to the - * end of the region if it would run over. - */ -static paddr_t __init next_module(paddr_t s, paddr_t *end) -{ - struct bootmodules *mi = &bootinfo.modules; - paddr_t lowest = ~(paddr_t)0; - int i; - - for ( i = 0; i < mi->nr_mods; i++ ) - { - paddr_t mod_s = mi->module[i].start; - paddr_t mod_e = mod_s + mi->module[i].size; - - if ( !mi->module[i].size ) - continue; - - if ( mod_s < s ) - continue; - if ( mod_s > lowest ) - continue; - if ( mod_s > *end ) - continue; - lowest = mod_s; - *end = min(*end, mod_e); - } - return lowest; -} - void __init init_pdx(void) { const struct membanks *mem = bootinfo_get_mem(); @@ -635,87 +284,6 @@ void __init init_pdx(void) } } -/* - * Populate the boot allocator. - * If a static heap was not provided by the admin, all the RAM but the - * following regions will be added: - * - Modules (e.g., Xen, Kernel) - * - Reserved regions - * - Xenheap (arm32 only) - * If a static heap was provided by the admin, populate the boot - * allocator with the corresponding regions only, but with Xenheap excluded - * on arm32. - */ -void __init populate_boot_allocator(void) -{ - unsigned int i; - const struct membanks *banks = bootinfo_get_mem(); - const struct membanks *reserved_mem = bootinfo_get_reserved_mem(); - paddr_t s, e; - - if ( bootinfo.static_heap ) - { - for ( i = 0 ; i < reserved_mem->nr_banks; i++ ) - { - if ( reserved_mem->bank[i].type != MEMBANK_STATIC_HEAP ) - continue; - - s = reserved_mem->bank[i].start; - e = s + reserved_mem->bank[i].size; -#ifdef CONFIG_ARM_32 - /* Avoid the xenheap, note that the xenheap cannot across a bank */ - if ( s <= mfn_to_maddr(directmap_mfn_start) && - e >= mfn_to_maddr(directmap_mfn_end) ) - { - init_boot_pages(s, mfn_to_maddr(directmap_mfn_start)); - init_boot_pages(mfn_to_maddr(directmap_mfn_end), e); - } - else -#endif - init_boot_pages(s, e); - } - - return; - } - - for ( i = 0; i < banks->nr_banks; i++ ) - { - const struct membank *bank = &banks->bank[i]; - paddr_t bank_end = bank->start + bank->size; - - s = bank->start; - while ( s < bank_end ) - { - paddr_t n = bank_end; - - e = next_module(s, &n); - - if ( e == ~(paddr_t)0 ) - e = n = bank_end; - - /* - * Module in a RAM bank other than the one which we are - * not dealing with here. - */ - if ( e > bank_end ) - e = bank_end; - -#ifdef CONFIG_ARM_32 - /* Avoid the xenheap */ - if ( s < mfn_to_maddr(directmap_mfn_end) && - mfn_to_maddr(directmap_mfn_start) < e ) - { - e = mfn_to_maddr(directmap_mfn_start); - n = mfn_to_maddr(directmap_mfn_end); - } -#endif - - fw_unreserved_regions(s, e, init_boot_pages, 0); - s = n; - } - } -} - size_t __read_mostly dcache_line_bytes; /* C entry point for boot CPU */ diff --git a/xen/common/Makefile b/xen/common/Makefile index f12a474d40..21359bab02 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -77,6 +77,7 @@ obj-$(CONFIG_UBSAN) += ubsan/ obj-$(CONFIG_NEEDS_LIBELF) += libelf/ obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/ +obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/ CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(objtree)/)$(KCONFIG_CONFIG) $(obj)/config.gz: $(CONF_FILE) diff --git a/xen/common/device-tree/Makefile b/xen/common/device-tree/Makefile new file mode 100644 index 0000000000..947bad979c --- /dev/null +++ b/xen/common/device-tree/Makefile @@ -0,0 +1 @@ +obj-y += bootinfo.init.o diff --git a/xen/common/device-tree/bootinfo.c b/xen/common/device-tree/bootinfo.c new file mode 100644 index 0000000000..dcac9a40a0 --- /dev/null +++ b/xen/common/device-tree/bootinfo.c @@ -0,0 +1,459 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Derived from $xen/arch/arm/setup.c. + * + * bookkeeping routines. + * + * Tim Deegan <tim@xen.org> + * Copyright (c) 2011 Citrix Systems. + * Copyright (c) 2024 Raptor Engineering LLC + */ + +#include <xen/acpi.h> +#include <xen/bootfdt.h> +#include <xen/bug.h> +#include <xen/device_tree.h> +#include <xen/init.h> +#include <xen/libfdt/libfdt-xen.h> +#include <xen/mm.h> + +struct bootinfo __initdata bootinfo = BOOTINFO_INIT; + +const char * __init boot_module_kind_as_string(bootmodule_kind kind) +{ + switch ( kind ) + { + case BOOTMOD_XEN: return "Xen"; + case BOOTMOD_FDT: return "Device Tree"; + case BOOTMOD_KERNEL: return "Kernel"; + case BOOTMOD_RAMDISK: return "Ramdisk"; + case BOOTMOD_XSM: return "XSM"; + case BOOTMOD_GUEST_DTB: return "DTB"; + case BOOTMOD_UNKNOWN: return "Unknown"; + default: BUG(); + } +} + +static void __init dt_unreserved_regions(paddr_t s, paddr_t e, + void (*cb)(paddr_t ps, paddr_t pe), + unsigned int first) +{ + const struct membanks *reserved_mem = bootinfo_get_reserved_mem(); +#ifdef CONFIG_STATIC_SHM + const struct membanks *shmem = bootinfo_get_shmem(); + unsigned int offset; +#endif + unsigned int i; + + /* + * i is the current bootmodule we are evaluating across all possible + * kinds. + */ + for ( i = first; i < reserved_mem->nr_banks; i++ ) + { + paddr_t r_s = reserved_mem->bank[i].start; + paddr_t r_e = r_s + reserved_mem->bank[i].size; + + if ( s < r_e && r_s < e ) + { + dt_unreserved_regions(r_e, e, cb, i + 1); + dt_unreserved_regions(s, r_s, cb, i + 1); + return; + } + } + +#ifdef CONFIG_STATIC_SHM + /* + * When retrieving the corresponding shared memory addresses + * below, we need to index the shmem->bank starting from 0, hence + * we need to use i - reserved_mem->nr_banks. + */ + offset = reserved_mem->nr_banks; + for ( ; i - offset < shmem->nr_banks; i++ ) + { + paddr_t r_s, r_e; + + r_s = shmem->bank[i - offset].start; + + /* Shared memory banks can contain INVALID_PADDR as start */ + if ( INVALID_PADDR == r_s ) + continue; + + r_e = r_s + shmem->bank[i - offset].size; + + if ( s < r_e && r_s < e ) + { + dt_unreserved_regions(r_e, e, cb, i + 1); + dt_unreserved_regions(s, r_s, cb, i + 1); + return; + } + } +#endif + + cb(s, e); +} + +/* + * TODO: '*_end' could be 0 if the bank/region is at the end of the physical + * address space. This is for now not handled as it requires more rework. + */ +static bool __init meminfo_overlap_check(const struct membanks *mem, + paddr_t region_start, + paddr_t region_size) +{ + paddr_t bank_start = INVALID_PADDR, bank_end = 0; + paddr_t region_end = region_start + region_size; + unsigned int i, bank_num = mem->nr_banks; + + for ( i = 0; i < bank_num; i++ ) + { + bank_start = mem->bank[i].start; + bank_end = bank_start + mem->bank[i].size; + + if ( INVALID_PADDR == bank_start || region_end <= bank_start || + region_start >= bank_end ) + continue; + else + { + printk("Region: [%#"PRIpaddr", %#"PRIpaddr") overlapping with bank[%u]: [%#"PRIpaddr", %#"PRIpaddr")\n", + region_start, region_end, i, bank_start, bank_end); + return true; + } + } + + return false; +} + +/* + * TODO: '*_end' could be 0 if the module/region is at the end of the physical + * address space. This is for now not handled as it requires more rework. + */ +static bool __init bootmodules_overlap_check(struct bootmodules *bootmodules, + paddr_t region_start, + paddr_t region_size) +{ + paddr_t mod_start = INVALID_PADDR, mod_end = 0; + paddr_t region_end = region_start + region_size; + unsigned int i, mod_num = bootmodules->nr_mods; + + for ( i = 0; i < mod_num; i++ ) + { + mod_start = bootmodules->module[i].start; + mod_end = mod_start + bootmodules->module[i].size; + + if ( region_end <= mod_start || region_start >= mod_end ) + continue; + else + { + printk("Region: [%#"PRIpaddr", %#"PRIpaddr") overlapping with mod[%u]: [%#"PRIpaddr", %#"PRIpaddr")\n", + region_start, region_end, i, mod_start, mod_end); + return true; + } + } + + return false; +} + +void __init fw_unreserved_regions(paddr_t s, paddr_t e, + void (*cb)(paddr_t ps, paddr_t pe), + unsigned int first) +{ + if ( acpi_disabled ) + dt_unreserved_regions(s, e, cb, first); + else + cb(s, e); +} + +/* + * Given an input physical address range, check if this range is overlapping + * with the existing reserved memory regions defined in bootinfo. + * Return true if the input physical address range is overlapping with any + * existing reserved memory regions, otherwise false. + */ +bool __init check_reserved_regions_overlap(paddr_t region_start, + paddr_t region_size) +{ + const struct membanks *mem_banks[] = { + bootinfo_get_reserved_mem(), +#ifdef CONFIG_ACPI + bootinfo_get_acpi(), +#endif +#ifdef CONFIG_STATIC_SHM + bootinfo_get_shmem(), +#endif + }; + unsigned int i; + + /* + * Check if input region is overlapping with reserved memory banks or + * ACPI EfiACPIReclaimMemory (when ACPI feature is enabled) or static + * shared memory banks (when static shared memory feature is enabled) + */ + for ( i = 0; i < ARRAY_SIZE(mem_banks); i++ ) + if ( meminfo_overlap_check(mem_banks[i], region_start, region_size) ) + return true; + + /* Check if input region is overlapping with bootmodules */ + if ( bootmodules_overlap_check(&bootinfo.modules, + region_start, region_size) ) + return true; + + return false; +} + +struct bootmodule __init *add_boot_module(bootmodule_kind kind, + paddr_t start, paddr_t size, + bool domU) +{ + struct bootmodules *mods = &bootinfo.modules; + struct bootmodule *mod; + unsigned int i; + + if ( mods->nr_mods == MAX_MODULES ) + { + printk("Ignoring %s boot module at %"PRIpaddr"-%"PRIpaddr" (too many)\n", + boot_module_kind_as_string(kind), start, start + size); + return NULL; + } + + if ( check_reserved_regions_overlap(start, size) ) + return NULL; + + for ( i = 0 ; i < mods->nr_mods ; i++ ) + { + mod = &mods->module[i]; + if ( mod->kind == kind && mod->start == start ) + { + if ( !domU ) + mod->domU = false; + return mod; + } + } + + mod = &mods->module[mods->nr_mods++]; + mod->kind = kind; + mod->start = start; + mod->size = size; + mod->domU = domU; + + return mod; +} + +/* + * boot_module_find_by_kind can only be used to return Xen modules (e.g + * XSM, DTB) or Dom0 modules. This is not suitable for looking up guest + * modules. + */ +struct bootmodule * __init boot_module_find_by_kind(bootmodule_kind kind) +{ + struct bootmodules *mods = &bootinfo.modules; + struct bootmodule *mod; + int i; + for (i = 0 ; i < mods->nr_mods ; i++ ) + { + mod = &mods->module[i]; + if ( mod->kind == kind && !mod->domU ) + return mod; + } + return NULL; +} + +void __init add_boot_cmdline(const char *name, const char *cmdline, + bootmodule_kind kind, paddr_t start, bool domU) +{ + struct bootcmdlines *cmds = &bootinfo.cmdlines; + struct bootcmdline *cmd; + + if ( cmds->nr_mods == MAX_MODULES ) + { + printk("Ignoring %s cmdline (too many)\n", name); + return; + } + + cmd = &cmds->cmdline[cmds->nr_mods++]; + cmd->kind = kind; + cmd->domU = domU; + cmd->start = start; + + ASSERT(strlen(name) <= DT_MAX_NAME); + safe_strcpy(cmd->dt_name, name); + + if ( strlen(cmdline) > BOOTMOD_MAX_CMDLINE ) + panic("module %s command line too long\n", name); + safe_strcpy(cmd->cmdline, cmdline); +} + +/* + * boot_cmdline_find_by_kind can only be used to return Xen modules (e.g + * XSM, DTB) or Dom0 modules. This is not suitable for looking up guest + * modules. + */ +struct bootcmdline * __init boot_cmdline_find_by_kind(bootmodule_kind kind) +{ + struct bootcmdlines *cmds = &bootinfo.cmdlines; + struct bootcmdline *cmd; + int i; + + for ( i = 0 ; i < cmds->nr_mods ; i++ ) + { + cmd = &cmds->cmdline[i]; + if ( cmd->kind == kind && !cmd->domU ) + return cmd; + } + return NULL; +} + +struct bootcmdline * __init boot_cmdline_find_by_name(const char *name) +{ + struct bootcmdlines *mods = &bootinfo.cmdlines; + struct bootcmdline *mod; + unsigned int i; + + for (i = 0 ; i < mods->nr_mods ; i++ ) + { + mod = &mods->cmdline[i]; + if ( strcmp(mod->dt_name, name) == 0 ) + return mod; + } + return NULL; +} + +struct bootmodule * __init boot_module_find_by_addr_and_kind(bootmodule_kind kind, + paddr_t start) +{ + struct bootmodules *mods = &bootinfo.modules; + struct bootmodule *mod; + unsigned int i; + + for (i = 0 ; i < mods->nr_mods ; i++ ) + { + mod = &mods->module[i]; + if ( mod->kind == kind && mod->start == start ) + return mod; + } + return NULL; +} + +/* + * Return the end of the non-module region starting at s. In other + * words return s the start of the next modules after s. + * + * On input *end is the end of the region which should be considered + * and it is updated to reflect the end of the module, clipped to the + * end of the region if it would run over. + */ +static paddr_t __init next_module(paddr_t s, paddr_t *end) +{ + struct bootmodules *mi = &bootinfo.modules; + paddr_t lowest = ~(paddr_t)0; + int i; + + for ( i = 0; i < mi->nr_mods; i++ ) + { + paddr_t mod_s = mi->module[i].start; + paddr_t mod_e = mod_s + mi->module[i].size; + + if ( !mi->module[i].size ) + continue; + + if ( mod_s < s ) + continue; + if ( mod_s > lowest ) + continue; + if ( mod_s > *end ) + continue; + lowest = mod_s; + *end = min(*end, mod_e); + } + return lowest; +} + +/* + * Populate the boot allocator. + * If a static heap was not provided by the admin, all the RAM but the + * following regions will be added: + * - Modules (e.g., Xen, Kernel) + * - Reserved regions + * - Xenheap (CONFIG_SEPARATE_XENHEAP only) + * If a static heap was provided by the admin, populate the boot + * allocator with the corresponding regions only, but with Xenheap excluded + * on CONFIG_SEPARATE_XENHEAP. + */ +void __init populate_boot_allocator(void) +{ + unsigned int i; + const struct membanks *banks = bootinfo_get_mem(); + const struct membanks *reserved_mem = bootinfo_get_reserved_mem(); + paddr_t s, e; + + if ( bootinfo.static_heap ) + { + for ( i = 0 ; i < reserved_mem->nr_banks; i++ ) + { + if ( reserved_mem->bank[i].type != MEMBANK_STATIC_HEAP ) + continue; + + s = reserved_mem->bank[i].start; + e = s + reserved_mem->bank[i].size; +#ifdef CONFIG_SEPARATE_XENHEAP + /* Avoid the xenheap, note that the xenheap cannot across a bank */ + if ( s <= mfn_to_maddr(directmap_mfn_start) && + e >= mfn_to_maddr(directmap_mfn_end) ) + { + init_boot_pages(s, mfn_to_maddr(directmap_mfn_start)); + init_boot_pages(mfn_to_maddr(directmap_mfn_end), e); + } + else +#endif + init_boot_pages(s, e); + } + + return; + } + + for ( i = 0; i < banks->nr_banks; i++ ) + { + const struct membank *bank = &banks->bank[i]; + paddr_t bank_end = bank->start + bank->size; + + s = bank->start; + while ( s < bank_end ) + { + paddr_t n = bank_end; + + e = next_module(s, &n); + + if ( e == ~(paddr_t)0 ) + e = n = bank_end; + + /* + * Module in a RAM bank other than the one which we are + * not dealing with here. + */ + if ( e > bank_end ) + e = bank_end; + +#ifdef CONFIG_SEPARATE_XENHEAP + /* Avoid the xenheap */ + if ( s < mfn_to_maddr(directmap_mfn_end) && + mfn_to_maddr(directmap_mfn_start) < e ) + { + e = mfn_to_maddr(directmap_mfn_start); + n = mfn_to_maddr(directmap_mfn_end); + } +#endif + + fw_unreserved_regions(s, e, init_boot_pages, 0); + s = n; + } + } +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/bootfdt.h b/xen/include/xen/bootfdt.h new file mode 100644 index 0000000000..7cd45b3d4b --- /dev/null +++ b/xen/include/xen/bootfdt.h @@ -0,0 +1,196 @@ +#ifndef __XEN_BOOTFDT_H__ +#define __XEN_BOOTFDT_H__ + +#include <xen/types.h> +#include <xen/kernel.h> +#include <xen/macros.h> + +#define MIN_FDT_ALIGN 8 +#define MAX_FDT_SIZE SZ_2M + +#define NR_MEM_BANKS 256 +#define NR_SHMEM_BANKS 32 + +#define MAX_MODULES 32 /* Current maximum useful modules */ + +typedef enum { + BOOTMOD_XEN, + BOOTMOD_FDT, + BOOTMOD_KERNEL, + BOOTMOD_RAMDISK, + BOOTMOD_XSM, + BOOTMOD_GUEST_DTB, + BOOTMOD_UNKNOWN +} bootmodule_kind; + +enum membank_type { + /* + * The MEMBANK_DEFAULT type refers to either reserved memory for the + * device/firmware (when the bank is in 'reserved_mem') or any RAM (when + * the bank is in 'mem'). + */ + MEMBANK_DEFAULT, + /* + * The MEMBANK_STATIC_DOMAIN type is used to indicate whether the memory + * bank is bound to a static Xen domain. It is only valid when the bank + * is in reserved_mem. + */ + MEMBANK_STATIC_DOMAIN, + /* + * The MEMBANK_STATIC_HEAP type is used to indicate whether the memory + * bank is reserved as static heap. It is only valid when the bank is + * in reserved_mem. + */ + MEMBANK_STATIC_HEAP, + /* + * The MEMBANK_FDT_RESVMEM type is used to indicate whether the memory + * bank is from the FDT reserve map. + */ + MEMBANK_FDT_RESVMEM, +}; + +/* Indicates the maximum number of characters(\0 included) for shm_id */ +#define MAX_SHM_ID_LENGTH 16 + +struct shmem_membank_extra { + char shm_id[MAX_SHM_ID_LENGTH]; + unsigned int nr_shm_borrowers; +}; + +struct membank { + paddr_t start; + paddr_t size; + union { + enum membank_type type; +#ifdef CONFIG_STATIC_SHM + struct shmem_membank_extra *shmem_extra; +#endif + }; +}; + +struct membanks { + __struct_group(membanks_hdr, common, , + unsigned int nr_banks; + unsigned int max_banks; + ); + struct membank bank[]; +}; + +struct meminfo { + struct membanks_hdr common; + struct membank bank[NR_MEM_BANKS]; +}; + +struct shared_meminfo { + struct membanks_hdr common; + struct membank bank[NR_SHMEM_BANKS]; + struct shmem_membank_extra extra[NR_SHMEM_BANKS]; +}; + +/* + * The domU flag is set for kernels and ramdisks of "xen,domain" nodes. + * The purpose of the domU flag is to avoid getting confused in + * kernel_probe, where we try to guess which is the dom0 kernel and + * initrd to be compatible with all versions of the multiboot spec. + */ +#define BOOTMOD_MAX_CMDLINE 1024 +struct bootmodule { + bootmodule_kind kind; + bool domU; + paddr_t start; + paddr_t size; +}; + +/* DT_MAX_NAME is the node name max length according the DT spec */ +#define DT_MAX_NAME 41 +struct bootcmdline { + bootmodule_kind kind; + bool domU; + paddr_t start; + char dt_name[DT_MAX_NAME]; + char cmdline[BOOTMOD_MAX_CMDLINE]; +}; + +struct bootmodules { + int nr_mods; + struct bootmodule module[MAX_MODULES]; +}; + +struct bootcmdlines { + unsigned int nr_mods; + struct bootcmdline cmdline[MAX_MODULES]; +}; + +struct bootinfo { + struct meminfo mem; + /* The reserved regions are only used when booting using Device-Tree */ + struct meminfo reserved_mem; + struct bootmodules modules; + struct bootcmdlines cmdlines; +#ifdef CONFIG_ACPI + struct meminfo acpi; +#endif +#ifdef CONFIG_STATIC_SHM + struct shared_meminfo shmem; +#endif + bool static_heap; +}; + +#ifdef CONFIG_ACPI +#define BOOTINFO_ACPI_INIT .acpi.common.max_banks = NR_MEM_BANKS, +#else +#define BOOTINFO_ACPI_INIT +#endif + +#ifdef CONFIG_STATIC_SHM +#define BOOTINFO_SHMEM_INIT .shmem.common.max_banks = NR_SHMEM_BANKS, +#else +#define BOOTINFO_SHMEM_INIT +#endif + +#define BOOTINFO_INIT \ +{ \ + .mem.common.max_banks = NR_MEM_BANKS, \ + .reserved_mem.common.max_banks = NR_MEM_BANKS, \ + BOOTINFO_ACPI_INIT \ + BOOTINFO_SHMEM_INIT \ +} + +extern struct bootinfo bootinfo; + +void populate_boot_allocator(void); + +size_t boot_fdt_info(const void *fdt, paddr_t paddr); + +const char *boot_fdt_cmdline(const void *fdt); + +static inline struct membanks *bootinfo_get_reserved_mem(void) +{ + return container_of(&bootinfo.reserved_mem.common, struct membanks, common); +} + +static inline struct membanks *bootinfo_get_mem(void) +{ + return container_of(&bootinfo.mem.common, struct membanks, common); +} + +#ifdef CONFIG_ACPI +static inline struct membanks *bootinfo_get_acpi(void) +{ + return container_of(&bootinfo.acpi.common, struct membanks, common); +} +#endif + +#ifdef CONFIG_STATIC_SHM +static inline struct membanks *bootinfo_get_shmem(void) +{ + return container_of(&bootinfo.shmem.common, struct membanks, common); +} + +static inline struct shmem_membank_extra *bootinfo_get_shmem_extra(void) +{ + return bootinfo.shmem.extra; +} +#endif + +#endif /* __XEN_BOOTFDT_H__ */ -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common 2024-07-12 16:22 ` [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common Oleksii Kurochko @ 2024-07-15 8:36 ` Jan Beulich 2024-07-15 10:15 ` Michal Orzel 1 sibling, 0 replies; 51+ messages in thread From: Jan Beulich @ 2024-07-15 8:36 UTC (permalink / raw) To: Oleksii Kurochko Cc: Shawn Anastasio, Andrew Cooper, Julien Grall, Stefano Stabellini, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Daniel P. Smith, xen-devel On 12.07.2024 18:22, Oleksii Kurochko wrote:> --- a/xen/common/Makefile > +++ b/xen/common/Makefile > @@ -77,6 +77,7 @@ obj-$(CONFIG_UBSAN) += ubsan/ > > obj-$(CONFIG_NEEDS_LIBELF) += libelf/ > obj-$(CONFIG_HAS_DEVICE_TREE) += libfdt/ > +obj-$(CONFIG_HAS_DEVICE_TREE) += device-tree/ > > CONF_FILE := $(if $(patsubst /%,,$(KCONFIG_CONFIG)),$(objtree)/)$(KCONFIG_CONFIG) > $(obj)/config.gz: $(CONF_FILE) I think it would be nice for the bad placement of the two lib*/ entries to not lead to further bad placements. Imo your addition wants to go ahead of the obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o line. (Eventually I expect common/device_tree.c would then also want to move there, probably at the same time renaming it.) Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common 2024-07-12 16:22 ` [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common Oleksii Kurochko 2024-07-15 8:36 ` Jan Beulich @ 2024-07-15 10:15 ` Michal Orzel 2024-07-15 13:42 ` Oleksii 1 sibling, 1 reply; 51+ messages in thread From: Michal Orzel @ 2024-07-15 10:15 UTC (permalink / raw) To: Oleksii Kurochko, xen-devel Cc: Shawn Anastasio, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Bertrand Marquis, Volodymyr Babchuk, Daniel P. Smith Hi Oleksii, In general, the patch looks ok (apart from Jan comments). Just a couple of remarks. On 12/07/2024 18:22, Oleksii Kurochko wrote: > > > From: Shawn Anastasio <sanastasio@raptorengineering.com> > > Arm's setup.c contains a collection of functions for parsing memory map > and other boot information from a device tree. Since these routines are > generally useful on any architecture that supports device tree booting, > move them into xen/common/device-tree. > > Suggested-by: Julien Grall <julien@xen.org> > Signed-off-by: Shawn Anastasio <sanastasio@raptorengineering.com> > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > --- > Changes in V6: > - update the version of the patch to v6, to show that it is based > on the work done by Shawn in the patch v4. > --- > Changes in V5: > - add xen/include/xen/bootfdt.h to MAINTAINERS file. > - drop message "Early device tree parsing and". > - After rebase on top of the current staging the following changes were done: > - init bootinfo variable in <common/device-tree/bootinfo.c> with BOOTINFO_INIT; > - update the code of dt_unreserved_regions(): > CONFIG_STATIC_SHM related changes and getting of reserved_mem > bootinfo_get_shmem() ?? > - update the code of meminfo_overlap_check(): > add check ( INVALID_PADDR == bank_start ) to if case. > - update the code of check_reserved_regions_overlap(): > CONFIG_STATIC_SHM related changes. > - struct bootinfo was updated ( CONFIG_STATIC_SHM changes ) > - add shared_meminfo ( because of CONFIG_STATIC_SHM ) > - struct struct membanks was update with __struct group so <xen/kernel> is > neeeded to be included in bootfdt.h > - move BOOTINFO_ACPI_INIT, BOOTINFO_SHMEM_INIT, BOOTINFO_INIT to generic bootfdt.h > - bootinfo_get_reserved_mem(), bootinfo_get_mem(), bootinfo_get_acpi(), > bootinfo_get_shmem() and bootinfo_get_shmem_extra() were moved to xen/bootfdt.h > - s/arm32/CONFIG_SEPARATE_XENHEAP/ > - add inclusion of <xen/macros.h> because there are function in <xen/bootfdt.h> which > are using container_of(). > --- > Changes in v4: > - create new xen/include/bootinfo.h rather than relying on arch's > asm/setup.h to provide required definitions for bootinfo.c > - build bootinfo.c as .init.o > - clean up and sort bootinfo.c's #includes > - use CONFIG_SEPARATE_XENHEAP rather than CONFIG_ARM_32 to guard > xenheap-specific behavior of populate_boot_allocator > - (MAINTAINERS) include all of common/device-tree rather than just > bootinfo.c > --- > MAINTAINERS | 2 + > xen/arch/arm/include/asm/setup.h | 187 +----------- > xen/arch/arm/setup.c | 432 ---------------------------- > xen/common/Makefile | 1 + > xen/common/device-tree/Makefile | 1 + > xen/common/device-tree/bootinfo.c | 459 ++++++++++++++++++++++++++++++ > xen/include/xen/bootfdt.h | 196 +++++++++++++ > 7 files changed, 660 insertions(+), 618 deletions(-) > create mode 100644 xen/common/device-tree/Makefile > create mode 100644 xen/common/device-tree/bootinfo.c > create mode 100644 xen/include/xen/bootfdt.h > [...] > diff --git a/xen/include/xen/bootfdt.h b/xen/include/xen/bootfdt.h > new file mode 100644 > index 0000000000..7cd45b3d4b > --- /dev/null > +++ b/xen/include/xen/bootfdt.h > @@ -0,0 +1,196 @@ > +#ifndef __XEN_BOOTFDT_H__ AFAIR, to avoid violating MISRA rule 21.1, we should avoid introducing new macros with double underscore. > +#define __XEN_BOOTFDT_H__ > + > +#include <xen/types.h> > +#include <xen/kernel.h> > +#include <xen/macros.h> > + > +#define MIN_FDT_ALIGN 8 > +#define MAX_FDT_SIZE SZ_2M 2M blob limit is Arm64 specific. What will be the limit on RISCV? Shouldn't it be moved to some arch specific file? ~Michal ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common 2024-07-15 10:15 ` Michal Orzel @ 2024-07-15 13:42 ` Oleksii 0 siblings, 0 replies; 51+ messages in thread From: Oleksii @ 2024-07-15 13:42 UTC (permalink / raw) To: Michal Orzel, xen-devel Cc: Shawn Anastasio, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Bertrand Marquis, Volodymyr Babchuk, Daniel P. Smith Hi Michal, On Mon, 2024-07-15 at 12:15 +0200, Michal Orzel wrote: > Hi Oleksii, > > In general, the patch looks ok (apart from Jan comments). Just a > couple of remarks. > > On 12/07/2024 18:22, Oleksii Kurochko wrote: > > > > > > From: Shawn Anastasio <sanastasio@raptorengineering.com> > > > > Arm's setup.c contains a collection of functions for parsing memory > > map > > and other boot information from a device tree. Since these routines > > are > > generally useful on any architecture that supports device tree > > booting, > > move them into xen/common/device-tree. > > > > Suggested-by: Julien Grall <julien@xen.org> > > Signed-off-by: Shawn Anastasio <sanastasio@raptorengineering.com> > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > --- > > Changes in V6: > > - update the version of the patch to v6, to show that it is based > > on the work done by Shawn in the patch v4. > > --- > > Changes in V5: > > - add xen/include/xen/bootfdt.h to MAINTAINERS file. > > - drop message "Early device tree parsing and". > > - After rebase on top of the current staging the following changes > > were done: > > - init bootinfo variable in <common/device-tree/bootinfo.c> with > > BOOTINFO_INIT; > > - update the code of dt_unreserved_regions(): > > CONFIG_STATIC_SHM related changes and getting of > > reserved_mem > > bootinfo_get_shmem() ?? > > - update the code of meminfo_overlap_check(): > > add check ( INVALID_PADDR == bank_start ) to if case. > > - update the code of check_reserved_regions_overlap(): > > CONFIG_STATIC_SHM related changes. > > - struct bootinfo was updated ( CONFIG_STATIC_SHM changes ) > > - add shared_meminfo ( because of CONFIG_STATIC_SHM ) > > - struct struct membanks was update with __struct group so > > <xen/kernel> is > > neeeded to be included in bootfdt.h > > - move BOOTINFO_ACPI_INIT, BOOTINFO_SHMEM_INIT, BOOTINFO_INIT to > > generic bootfdt.h > > - bootinfo_get_reserved_mem(), bootinfo_get_mem(), > > bootinfo_get_acpi(), > > bootinfo_get_shmem() and bootinfo_get_shmem_extra() were moved > > to xen/bootfdt.h > > - s/arm32/CONFIG_SEPARATE_XENHEAP/ > > - add inclusion of <xen/macros.h> because there are function in > > <xen/bootfdt.h> which > > are using container_of(). > > --- > > Changes in v4: > > - create new xen/include/bootinfo.h rather than relying on arch's > > asm/setup.h to provide required definitions for bootinfo.c > > - build bootinfo.c as .init.o > > - clean up and sort bootinfo.c's #includes > > - use CONFIG_SEPARATE_XENHEAP rather than CONFIG_ARM_32 to guard > > xenheap-specific behavior of populate_boot_allocator > > - (MAINTAINERS) include all of common/device-tree rather than > > just > > bootinfo.c > > --- > > MAINTAINERS | 2 + > > xen/arch/arm/include/asm/setup.h | 187 +----------- > > xen/arch/arm/setup.c | 432 -------------------------- > > -- > > xen/common/Makefile | 1 + > > xen/common/device-tree/Makefile | 1 + > > xen/common/device-tree/bootinfo.c | 459 > > ++++++++++++++++++++++++++++++ > > xen/include/xen/bootfdt.h | 196 +++++++++++++ > > 7 files changed, 660 insertions(+), 618 deletions(-) > > create mode 100644 xen/common/device-tree/Makefile > > create mode 100644 xen/common/device-tree/bootinfo.c > > create mode 100644 xen/include/xen/bootfdt.h > > > > [...] > > > diff --git a/xen/include/xen/bootfdt.h b/xen/include/xen/bootfdt.h > > new file mode 100644 > > index 0000000000..7cd45b3d4b > > --- /dev/null > > +++ b/xen/include/xen/bootfdt.h > > @@ -0,0 +1,196 @@ > > +#ifndef __XEN_BOOTFDT_H__ > AFAIR, to avoid violating MISRA rule 21.1, we should avoid > introducing new macros with double underscore. I will update that in the next patch version. > > > +#define __XEN_BOOTFDT_H__ > > + > > +#include <xen/types.h> > > +#include <xen/kernel.h> > > +#include <xen/macros.h> > > + > > +#define MIN_FDT_ALIGN 8 > > +#define MAX_FDT_SIZE SZ_2M > 2M blob limit is Arm64 specific. What will be the limit on RISCV? > Shouldn't it be moved to some arch specific file? Agree, good point. I should drop that definition here and make it arch specific. I have to update my patch series where I am introducing BOOTFDT_MAX_SIZE ( or something similar ) in riscv/config.h. Thanks. ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH v6 2/8] xen/common: Move Arm's bootfdt.c to common 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v2 3/8] xen/riscv: enable CONFIG_HAS_DEVICE_TREE Oleksii Kurochko ` (5 subsequent siblings) 7 siblings, 0 replies; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Shawn Anastasio, Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Daniel P. Smith, Oleksii Kurochko From: Shawn Anastasio <sanastasio@raptorengineering.com> Move Arm's bootfdt.c to xen/common so that it can be used by other device tree architectures like PPC and RISCV. Suggested-by: Julien Grall <julien@xen.org> Signed-off-by: Shawn Anastasio <sanastasio@raptorengineering.com> Acked-by: Julien Grall <julien@xen.org> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V6: - update the version of the patch to v6, to show that it is based on the work done by Shawn in the patch v4. --- Changes in v5: - add guard #ifdef CONFIG_STATIC_SHM around inclusion of <asm/static-shmem.h> in common/device-tree/bootfdt.c. - add stub for process_shm_node() in case CONFIG_STATIC_SHM isn't enabled. - add guard around #ifdef CONFIG_STATIC_SHM aroud early_print_info_shmem() in early_print_info(). --- Changes in v4: - move function prototypes to patch 2's xen/include/bootfdt.h - clean up #includes --- xen/arch/arm/Makefile | 1 - xen/arch/arm/bootfdt.c | 622 ------------------------------ xen/arch/arm/include/asm/setup.h | 13 - xen/common/device-tree/Makefile | 1 + xen/common/device-tree/bootfdt.c | 635 +++++++++++++++++++++++++++++++ xen/include/xen/bootfdt.h | 14 + 6 files changed, 650 insertions(+), 636 deletions(-) delete mode 100644 xen/arch/arm/bootfdt.c create mode 100644 xen/common/device-tree/bootfdt.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 45dc29ea53..da9c979dc4 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -10,7 +10,6 @@ obj-$(CONFIG_TEE) += tee/ obj-$(CONFIG_HAS_VPCI) += vpci.o obj-$(CONFIG_HAS_ALTERNATIVE) += alternative.o -obj-y += bootfdt.init.o obj-y += cpuerrata.o obj-y += cpufeature.o obj-y += decode.o diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c deleted file mode 100644 index 6e060111d9..0000000000 --- a/xen/arch/arm/bootfdt.c +++ /dev/null @@ -1,622 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * Early Device Tree - * - * Copyright (C) 2012-2014 Citrix Systems, Inc. - */ -#include <xen/types.h> -#include <xen/lib.h> -#include <xen/kernel.h> -#include <xen/init.h> -#include <xen/efi.h> -#include <xen/device_tree.h> -#include <xen/lib.h> -#include <xen/libfdt/libfdt-xen.h> -#include <xen/sort.h> -#include <xsm/xsm.h> -#include <asm/setup.h> -#include <asm/static-shmem.h> - -static void __init __maybe_unused build_assertions(void) -{ - /* - * Check that no padding is between struct membanks "bank" flexible array - * member and struct meminfo "bank" member - */ - BUILD_BUG_ON((offsetof(struct membanks, bank) != - offsetof(struct meminfo, bank))); - /* Ensure "struct membanks" is 8-byte aligned */ - BUILD_BUG_ON(alignof(struct membanks) != 8); -} - -static bool __init device_tree_node_is_available(const void *fdt, int node) -{ - const char *status; - int len; - - status = fdt_getprop(fdt, node, "status", &len); - if ( !status ) - return true; - - if ( len > 0 ) - { - if ( !strcmp(status, "ok") || !strcmp(status, "okay") ) - return true; - } - - return false; -} - -static bool __init device_tree_node_matches(const void *fdt, int node, - const char *match) -{ - const char *name; - size_t match_len; - - name = fdt_get_name(fdt, node, NULL); - match_len = strlen(match); - - /* Match both "match" and "match@..." patterns but not - "match-foo". */ - return strncmp(name, match, match_len) == 0 - && (name[match_len] == '@' || name[match_len] == '\0'); -} - -static bool __init device_tree_node_compatible(const void *fdt, int node, - const char *match) -{ - int len, l; - const void *prop; - - prop = fdt_getprop(fdt, node, "compatible", &len); - if ( prop == NULL ) - return false; - - while ( len > 0 ) { - if ( !dt_compat_cmp(prop, match) ) - return true; - l = strlen(prop) + 1; - prop += l; - len -= l; - } - - return false; -} - -void __init device_tree_get_reg(const __be32 **cell, uint32_t address_cells, - uint32_t size_cells, paddr_t *start, - paddr_t *size) -{ - uint64_t dt_start, dt_size; - - /* - * dt_next_cell will return uint64_t whereas paddr_t may not be 64-bit. - * Thus, there is an implicit cast from uint64_t to paddr_t. - */ - dt_start = dt_next_cell(address_cells, cell); - dt_size = dt_next_cell(size_cells, cell); - - if ( dt_start != (paddr_t)dt_start ) - { - printk("Physical address greater than max width supported\n"); - WARN(); - } - - if ( dt_size != (paddr_t)dt_size ) - { - printk("Physical size greater than max width supported\n"); - WARN(); - } - - /* - * Xen will truncate the address/size if it is greater than the maximum - * supported width and it will give an appropriate warning. - */ - *start = dt_start; - *size = dt_size; -} - -static int __init device_tree_get_meminfo(const void *fdt, int node, - const char *prop_name, - u32 address_cells, u32 size_cells, - struct membanks *mem, - enum membank_type type) -{ - const struct fdt_property *prop; - unsigned int i, banks; - const __be32 *cell; - u32 reg_cells = address_cells + size_cells; - paddr_t start, size; - - if ( !device_tree_node_is_available(fdt, node) ) - return 0; - - if ( address_cells < 1 || size_cells < 1 ) - { - printk("fdt: property `%s': invalid #address-cells or #size-cells", - prop_name); - return -EINVAL; - } - - prop = fdt_get_property(fdt, node, prop_name, NULL); - if ( !prop ) - return -ENOENT; - - cell = (const __be32 *)prop->data; - banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32)); - - for ( i = 0; i < banks && mem->nr_banks < mem->max_banks; i++ ) - { - device_tree_get_reg(&cell, address_cells, size_cells, &start, &size); - if ( mem == bootinfo_get_reserved_mem() && - check_reserved_regions_overlap(start, size) ) - return -EINVAL; - /* Some DT may describe empty bank, ignore them */ - if ( !size ) - continue; - mem->bank[mem->nr_banks].start = start; - mem->bank[mem->nr_banks].size = size; - mem->bank[mem->nr_banks].type = type; - mem->nr_banks++; - } - - if ( i < banks ) - { - printk("Warning: Max number of supported memory regions reached.\n"); - return -ENOSPC; - } - - return 0; -} - -u32 __init device_tree_get_u32(const void *fdt, int node, - const char *prop_name, u32 dflt) -{ - const struct fdt_property *prop; - - prop = fdt_get_property(fdt, node, prop_name, NULL); - if ( !prop || prop->len < sizeof(u32) ) - return dflt; - - return fdt32_to_cpu(*(uint32_t*)prop->data); -} - -/** - * device_tree_for_each_node - iterate over all device tree sub-nodes - * @fdt: flat device tree. - * @node: parent node to start the search from - * @func: function to call for each sub-node. - * @data: data to pass to @func. - * - * Any nodes nested at DEVICE_TREE_MAX_DEPTH or deeper are ignored. - * - * Returns 0 if all nodes were iterated over successfully. If @func - * returns a value different from 0, that value is returned immediately. - */ -int __init device_tree_for_each_node(const void *fdt, int node, - device_tree_node_func func, - void *data) -{ - /* - * We only care about relative depth increments, assume depth of - * node is 0 for simplicity. - */ - int depth = 0; - const int first_node = node; - u32 address_cells[DEVICE_TREE_MAX_DEPTH]; - u32 size_cells[DEVICE_TREE_MAX_DEPTH]; - int ret; - - do { - const char *name = fdt_get_name(fdt, node, NULL); - u32 as, ss; - - if ( depth >= DEVICE_TREE_MAX_DEPTH ) - { - printk("Warning: device tree node `%s' is nested too deep\n", - name); - continue; - } - - as = depth > 0 ? address_cells[depth-1] : DT_ROOT_NODE_ADDR_CELLS_DEFAULT; - ss = depth > 0 ? size_cells[depth-1] : DT_ROOT_NODE_SIZE_CELLS_DEFAULT; - - address_cells[depth] = device_tree_get_u32(fdt, node, - "#address-cells", as); - size_cells[depth] = device_tree_get_u32(fdt, node, - "#size-cells", ss); - - /* skip the first node */ - if ( node != first_node ) - { - ret = func(fdt, node, name, depth, as, ss, data); - if ( ret != 0 ) - return ret; - } - - node = fdt_next_node(fdt, node, &depth); - } while ( node >= 0 && depth > 0 ); - - return 0; -} - -static int __init process_memory_node(const void *fdt, int node, - const char *name, int depth, - u32 address_cells, u32 size_cells, - struct membanks *mem) -{ - return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, - mem, MEMBANK_DEFAULT); -} - -static int __init process_reserved_memory_node(const void *fdt, int node, - const char *name, int depth, - u32 address_cells, - u32 size_cells, - void *data) -{ - int rc = process_memory_node(fdt, node, name, depth, address_cells, - size_cells, data); - - if ( rc == -ENOSPC ) - panic("Max number of supported reserved-memory regions reached.\n"); - else if ( rc != -ENOENT ) - return rc; - return 0; -} - -static int __init process_reserved_memory(const void *fdt, int node, - const char *name, int depth, - u32 address_cells, u32 size_cells) -{ - return device_tree_for_each_node(fdt, node, - process_reserved_memory_node, - bootinfo_get_reserved_mem()); -} - -static void __init process_multiboot_node(const void *fdt, int node, - const char *name, - u32 address_cells, u32 size_cells) -{ - static int __initdata kind_guess = 0; - const struct fdt_property *prop; - const __be32 *cell; - bootmodule_kind kind; - paddr_t start, size; - int len; - /* sizeof("/chosen/") + DT_MAX_NAME + '/' + DT_MAX_NAME + '/0' => 92 */ - char path[92]; - int parent_node, ret; - bool domU; - - parent_node = fdt_parent_offset(fdt, node); - ASSERT(parent_node >= 0); - - /* Check that the node is under "/chosen" (first 7 chars of path) */ - ret = fdt_get_path(fdt, node, path, sizeof (path)); - if ( ret != 0 || strncmp(path, "/chosen", 7) ) - return; - - prop = fdt_get_property(fdt, node, "reg", &len); - if ( !prop ) - panic("node %s missing `reg' property\n", name); - - if ( len < dt_cells_to_size(address_cells + size_cells) ) - panic("fdt: node `%s': `reg` property length is too short\n", - name); - - cell = (const __be32 *)prop->data; - device_tree_get_reg(&cell, address_cells, size_cells, &start, &size); - - if ( fdt_node_check_compatible(fdt, node, "xen,linux-zimage") == 0 || - fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 ) - kind = BOOTMOD_KERNEL; - else if ( fdt_node_check_compatible(fdt, node, "xen,linux-initrd") == 0 || - fdt_node_check_compatible(fdt, node, "multiboot,ramdisk") == 0 ) - kind = BOOTMOD_RAMDISK; - else if ( fdt_node_check_compatible(fdt, node, "xen,xsm-policy") == 0 ) - kind = BOOTMOD_XSM; - else if ( fdt_node_check_compatible(fdt, node, "multiboot,device-tree") == 0 ) - kind = BOOTMOD_GUEST_DTB; - else - kind = BOOTMOD_UNKNOWN; - - /** - * Guess the kind of these first two unknowns respectively: - * (1) The first unknown must be kernel. - * (2) Detect the XSM Magic from the 2nd unknown: - * a. If it's XSM, set the kind as XSM, and that also means we - * won't load ramdisk; - * b. if it's not XSM, set the kind as ramdisk. - * So if user want to load ramdisk, it must be the 2nd unknown. - * We also detect the XSM Magic for the following unknowns, - * then set its kind according to the return value of has_xsm_magic. - */ - if ( kind == BOOTMOD_UNKNOWN ) - { - switch ( kind_guess++ ) - { - case 0: kind = BOOTMOD_KERNEL; break; - case 1: kind = BOOTMOD_RAMDISK; break; - default: break; - } - if ( kind_guess > 1 && has_xsm_magic(start) ) - kind = BOOTMOD_XSM; - } - - domU = fdt_node_check_compatible(fdt, parent_node, "xen,domain") == 0; - add_boot_module(kind, start, size, domU); - - prop = fdt_get_property(fdt, node, "bootargs", &len); - if ( !prop ) - return; - add_boot_cmdline(fdt_get_name(fdt, parent_node, &len), prop->data, - kind, start, domU); -} - -static int __init process_chosen_node(const void *fdt, int node, - const char *name, - u32 address_cells, u32 size_cells) -{ - const struct fdt_property *prop; - paddr_t start, end; - int len; - - if ( fdt_get_property(fdt, node, "xen,static-heap", NULL) ) - { - int rc; - - printk("Checking for static heap in /chosen\n"); - - rc = device_tree_get_meminfo(fdt, node, "xen,static-heap", - address_cells, size_cells, - bootinfo_get_reserved_mem(), - MEMBANK_STATIC_HEAP); - if ( rc ) - return rc; - - bootinfo.static_heap = true; - } - - printk("Checking for initrd in /chosen\n"); - - prop = fdt_get_property(fdt, node, "linux,initrd-start", &len); - if ( !prop ) - /* No initrd present. */ - return 0; - if ( len != sizeof(u32) && len != sizeof(u64) ) - { - printk("linux,initrd-start property has invalid length %d\n", len); - return -EINVAL; - } - start = dt_read_paddr((const void *)&prop->data, dt_size_to_cells(len)); - - prop = fdt_get_property(fdt, node, "linux,initrd-end", &len); - if ( !prop ) - { - printk("linux,initrd-end not present but -start was\n"); - return -EINVAL; - } - if ( len != sizeof(u32) && len != sizeof(u64) ) - { - printk("linux,initrd-end property has invalid length %d\n", len); - return -EINVAL; - } - end = dt_read_paddr((const void *)&prop->data, dt_size_to_cells(len)); - - if ( start >= end ) - { - printk("linux,initrd limits invalid: %"PRIpaddr" >= %"PRIpaddr"\n", - start, end); - return -EINVAL; - } - - printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end); - - add_boot_module(BOOTMOD_RAMDISK, start, end-start, false); - - return 0; -} - -static int __init process_domain_node(const void *fdt, int node, - const char *name, - u32 address_cells, u32 size_cells) -{ - const struct fdt_property *prop; - - printk("Checking for \"xen,static-mem\" in domain node\n"); - - prop = fdt_get_property(fdt, node, "xen,static-mem", NULL); - if ( !prop ) - /* No "xen,static-mem" present. */ - return 0; - - return device_tree_get_meminfo(fdt, node, "xen,static-mem", address_cells, - size_cells, bootinfo_get_reserved_mem(), - MEMBANK_STATIC_DOMAIN); -} - -static int __init early_scan_node(const void *fdt, - int node, const char *name, int depth, - u32 address_cells, u32 size_cells, - void *data) -{ - int rc = 0; - - /* - * If Xen has been booted via UEFI, the memory banks are - * populated. So we should skip the parsing. - */ - if ( !efi_enabled(EFI_BOOT) && - device_tree_node_matches(fdt, node, "memory") ) - rc = process_memory_node(fdt, node, name, depth, - address_cells, size_cells, bootinfo_get_mem()); - else if ( depth == 1 && !dt_node_cmp(name, "reserved-memory") ) - rc = process_reserved_memory(fdt, node, name, depth, - address_cells, size_cells); - else if ( depth <= 3 && (device_tree_node_compatible(fdt, node, "xen,multiboot-module" ) || - device_tree_node_compatible(fdt, node, "multiboot,module" ))) - process_multiboot_node(fdt, node, name, address_cells, size_cells); - else if ( depth == 1 && device_tree_node_matches(fdt, node, "chosen") ) - rc = process_chosen_node(fdt, node, name, address_cells, size_cells); - else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") ) - rc = process_domain_node(fdt, node, name, address_cells, size_cells); - else if ( depth <= 3 && device_tree_node_compatible(fdt, node, "xen,domain-shared-memory-v1") ) - rc = process_shm_node(fdt, node, address_cells, size_cells); - - if ( rc < 0 ) - printk("fdt: node `%s': parsing failed\n", name); - return rc; -} - -static void __init early_print_info(void) -{ - const struct membanks *mi = bootinfo_get_mem(); - const struct membanks *mem_resv = bootinfo_get_reserved_mem(); - struct bootmodules *mods = &bootinfo.modules; - struct bootcmdlines *cmds = &bootinfo.cmdlines; - unsigned int i; - - for ( i = 0; i < mi->nr_banks; i++ ) - printk("RAM: %"PRIpaddr" - %"PRIpaddr"\n", - mi->bank[i].start, - mi->bank[i].start + mi->bank[i].size - 1); - printk("\n"); - for ( i = 0 ; i < mods->nr_mods; i++ ) - printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s\n", - i, - mods->module[i].start, - mods->module[i].start + mods->module[i].size, - boot_module_kind_as_string(mods->module[i].kind)); - - for ( i = 0; i < mem_resv->nr_banks; i++ ) - { - printk(" RESVD[%u]: %"PRIpaddr" - %"PRIpaddr"\n", i, - mem_resv->bank[i].start, - mem_resv->bank[i].start + mem_resv->bank[i].size - 1); - } - early_print_info_shmem(); - printk("\n"); - for ( i = 0 ; i < cmds->nr_mods; i++ ) - printk("CMDLINE[%"PRIpaddr"]:%s %s\n", cmds->cmdline[i].start, - cmds->cmdline[i].dt_name, - &cmds->cmdline[i].cmdline[0]); - printk("\n"); -} - -/* This function assumes that memory regions are not overlapped */ -static int __init cmp_memory_node(const void *key, const void *elem) -{ - const struct membank *handler0 = key; - const struct membank *handler1 = elem; - - if ( handler0->start < handler1->start ) - return -1; - - if ( handler0->start >= (handler1->start + handler1->size) ) - return 1; - - return 0; -} - -static void __init swap_memory_node(void *_a, void *_b, size_t size) -{ - struct membank *a = _a, *b = _b; - - SWAP(*a, *b); -} - -/** - * boot_fdt_info - initialize bootinfo from a DTB - * @fdt: flattened device tree binary - * - * Returns the size of the DTB. - */ -size_t __init boot_fdt_info(const void *fdt, paddr_t paddr) -{ - struct membanks *reserved_mem = bootinfo_get_reserved_mem(); - struct membanks *mem = bootinfo_get_mem(); - unsigned int i; - int nr_rsvd; - int ret; - - ret = fdt_check_header(fdt); - if ( ret < 0 ) - panic("No valid device tree\n"); - - add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), false); - - nr_rsvd = fdt_num_mem_rsv(fdt); - if ( nr_rsvd < 0 ) - panic("Parsing FDT memory reserve map failed (%d)\n", nr_rsvd); - - for ( i = 0; i < nr_rsvd; i++ ) - { - struct membank *bank; - paddr_t s, sz; - - if ( fdt_get_mem_rsv_paddr(device_tree_flattened, i, &s, &sz) < 0 ) - continue; - - if ( reserved_mem->nr_banks < reserved_mem->max_banks ) - { - bank = &reserved_mem->bank[reserved_mem->nr_banks]; - bank->start = s; - bank->size = sz; - bank->type = MEMBANK_FDT_RESVMEM; - reserved_mem->nr_banks++; - } - else - panic("Cannot allocate reserved memory bank\n"); - } - - ret = device_tree_for_each_node(fdt, 0, early_scan_node, NULL); - if ( ret ) - panic("Early FDT parsing failed (%d)\n", ret); - - /* - * On Arm64 setup_directmap_mappings() expects to be called with the lowest - * bank in memory first. There is no requirement that the DT will provide - * the banks sorted in ascending order. So sort them through. - */ - sort(mem->bank, mem->nr_banks, sizeof(struct membank), - cmp_memory_node, swap_memory_node); - - early_print_info(); - - return fdt_totalsize(fdt); -} - -const __init char *boot_fdt_cmdline(const void *fdt) -{ - int node; - const struct fdt_property *prop; - - node = fdt_path_offset(fdt, "/chosen"); - if ( node < 0 ) - return NULL; - - prop = fdt_get_property(fdt, node, "xen,xen-bootargs", NULL); - if ( prop == NULL ) - { - struct bootcmdline *dom0_cmdline = - boot_cmdline_find_by_kind(BOOTMOD_KERNEL); - - if (fdt_get_property(fdt, node, "xen,dom0-bootargs", NULL) || - ( dom0_cmdline && dom0_cmdline->cmdline[0] ) ) - prop = fdt_get_property(fdt, node, "bootargs", NULL); - } - if ( prop == NULL ) - return NULL; - - return prop->data; -} - -/* - * Local variables: - * mode: C - * c-file-style: "BSD" - * c-basic-offset: 4 - * indent-tabs-mode: nil - * End: - */ diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h index 051e796716..5ee690aeb2 100644 --- a/xen/arch/arm/include/asm/setup.h +++ b/xen/arch/arm/include/asm/setup.h @@ -39,19 +39,6 @@ void fw_unreserved_regions(paddr_t s, paddr_t e, void (*cb)(paddr_t ps, paddr_t pe), unsigned int first); -bool check_reserved_regions_overlap(paddr_t region_start, paddr_t region_size); - -struct bootmodule *add_boot_module(bootmodule_kind kind, - paddr_t start, paddr_t size, bool domU); -struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind); -struct bootmodule * boot_module_find_by_addr_and_kind(bootmodule_kind kind, - paddr_t start); -void add_boot_cmdline(const char *name, const char *cmdline, - bootmodule_kind kind, paddr_t start, bool domU); -struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind); -struct bootcmdline * boot_cmdline_find_by_name(const char *name); -const char *boot_module_kind_as_string(bootmodule_kind kind); - void init_pdx(void); void setup_mm(void); diff --git a/xen/common/device-tree/Makefile b/xen/common/device-tree/Makefile index 947bad979c..ff2de71c96 100644 --- a/xen/common/device-tree/Makefile +++ b/xen/common/device-tree/Makefile @@ -1 +1,2 @@ +obj-y += bootfdt.init.o obj-y += bootinfo.init.o diff --git a/xen/common/device-tree/bootfdt.c b/xen/common/device-tree/bootfdt.c new file mode 100644 index 0000000000..748b5f7c69 --- /dev/null +++ b/xen/common/device-tree/bootfdt.c @@ -0,0 +1,635 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Early Device Tree + * + * Copyright (C) 2012-2014 Citrix Systems, Inc. + */ + +#include <xen/bootfdt.h> +#include <xen/device_tree.h> +#include <xen/efi.h> +#include <xen/init.h> +#include <xen/kernel.h> +#include <xen/libfdt/libfdt-xen.h> +#include <xen/sort.h> +#include <xsm/xsm.h> +#include <asm/setup.h> +#ifdef CONFIG_STATIC_SHM +#include <asm/static-shmem.h> +#endif + +static void __init __maybe_unused build_assertions(void) +{ + /* + * Check that no padding is between struct membanks "bank" flexible array + * member and struct meminfo "bank" member + */ + BUILD_BUG_ON((offsetof(struct membanks, bank) != + offsetof(struct meminfo, bank))); + /* Ensure "struct membanks" is 8-byte aligned */ + BUILD_BUG_ON(alignof(struct membanks) != 8); +} + +static bool __init device_tree_node_is_available(const void *fdt, int node) +{ + const char *status; + int len; + + status = fdt_getprop(fdt, node, "status", &len); + if ( !status ) + return true; + + if ( len > 0 ) + { + if ( !strcmp(status, "ok") || !strcmp(status, "okay") ) + return true; + } + + return false; +} + +static bool __init device_tree_node_matches(const void *fdt, int node, + const char *match) +{ + const char *name; + size_t match_len; + + name = fdt_get_name(fdt, node, NULL); + match_len = strlen(match); + + /* Match both "match" and "match@..." patterns but not + "match-foo". */ + return strncmp(name, match, match_len) == 0 + && (name[match_len] == '@' || name[match_len] == '\0'); +} + +static bool __init device_tree_node_compatible(const void *fdt, int node, + const char *match) +{ + int len, l; + const void *prop; + + prop = fdt_getprop(fdt, node, "compatible", &len); + if ( prop == NULL ) + return false; + + while ( len > 0 ) { + if ( !dt_compat_cmp(prop, match) ) + return true; + l = strlen(prop) + 1; + prop += l; + len -= l; + } + + return false; +} + +void __init device_tree_get_reg(const __be32 **cell, uint32_t address_cells, + uint32_t size_cells, paddr_t *start, + paddr_t *size) +{ + uint64_t dt_start, dt_size; + + /* + * dt_next_cell will return uint64_t whereas paddr_t may not be 64-bit. + * Thus, there is an implicit cast from uint64_t to paddr_t. + */ + dt_start = dt_next_cell(address_cells, cell); + dt_size = dt_next_cell(size_cells, cell); + + if ( dt_start != (paddr_t)dt_start ) + { + printk("Physical address greater than max width supported\n"); + WARN(); + } + + if ( dt_size != (paddr_t)dt_size ) + { + printk("Physical size greater than max width supported\n"); + WARN(); + } + + /* + * Xen will truncate the address/size if it is greater than the maximum + * supported width and it will give an appropriate warning. + */ + *start = dt_start; + *size = dt_size; +} + +static int __init device_tree_get_meminfo(const void *fdt, int node, + const char *prop_name, + u32 address_cells, u32 size_cells, + struct membanks *mem, + enum membank_type type) +{ + const struct fdt_property *prop; + unsigned int i, banks; + const __be32 *cell; + u32 reg_cells = address_cells + size_cells; + paddr_t start, size; + + if ( !device_tree_node_is_available(fdt, node) ) + return 0; + + if ( address_cells < 1 || size_cells < 1 ) + { + printk("fdt: property `%s': invalid #address-cells or #size-cells", + prop_name); + return -EINVAL; + } + + prop = fdt_get_property(fdt, node, prop_name, NULL); + if ( !prop ) + return -ENOENT; + + cell = (const __be32 *)prop->data; + banks = fdt32_to_cpu(prop->len) / (reg_cells * sizeof (u32)); + + for ( i = 0; i < banks && mem->nr_banks < mem->max_banks; i++ ) + { + device_tree_get_reg(&cell, address_cells, size_cells, &start, &size); + if ( mem == bootinfo_get_reserved_mem() && + check_reserved_regions_overlap(start, size) ) + return -EINVAL; + /* Some DT may describe empty bank, ignore them */ + if ( !size ) + continue; + mem->bank[mem->nr_banks].start = start; + mem->bank[mem->nr_banks].size = size; + mem->bank[mem->nr_banks].type = type; + mem->nr_banks++; + } + + if ( i < banks ) + { + printk("Warning: Max number of supported memory regions reached.\n"); + return -ENOSPC; + } + + return 0; +} + +u32 __init device_tree_get_u32(const void *fdt, int node, + const char *prop_name, u32 dflt) +{ + const struct fdt_property *prop; + + prop = fdt_get_property(fdt, node, prop_name, NULL); + if ( !prop || prop->len < sizeof(u32) ) + return dflt; + + return fdt32_to_cpu(*(uint32_t*)prop->data); +} + +/** + * device_tree_for_each_node - iterate over all device tree sub-nodes + * @fdt: flat device tree. + * @node: parent node to start the search from + * @func: function to call for each sub-node. + * @data: data to pass to @func. + * + * Any nodes nested at DEVICE_TREE_MAX_DEPTH or deeper are ignored. + * + * Returns 0 if all nodes were iterated over successfully. If @func + * returns a value different from 0, that value is returned immediately. + */ +int __init device_tree_for_each_node(const void *fdt, int node, + device_tree_node_func func, + void *data) +{ + /* + * We only care about relative depth increments, assume depth of + * node is 0 for simplicity. + */ + int depth = 0; + const int first_node = node; + u32 address_cells[DEVICE_TREE_MAX_DEPTH]; + u32 size_cells[DEVICE_TREE_MAX_DEPTH]; + int ret; + + do { + const char *name = fdt_get_name(fdt, node, NULL); + u32 as, ss; + + if ( depth >= DEVICE_TREE_MAX_DEPTH ) + { + printk("Warning: device tree node `%s' is nested too deep\n", + name); + continue; + } + + as = depth > 0 ? address_cells[depth-1] : DT_ROOT_NODE_ADDR_CELLS_DEFAULT; + ss = depth > 0 ? size_cells[depth-1] : DT_ROOT_NODE_SIZE_CELLS_DEFAULT; + + address_cells[depth] = device_tree_get_u32(fdt, node, + "#address-cells", as); + size_cells[depth] = device_tree_get_u32(fdt, node, + "#size-cells", ss); + + /* skip the first node */ + if ( node != first_node ) + { + ret = func(fdt, node, name, depth, as, ss, data); + if ( ret != 0 ) + return ret; + } + + node = fdt_next_node(fdt, node, &depth); + } while ( node >= 0 && depth > 0 ); + + return 0; +} + +static int __init process_memory_node(const void *fdt, int node, + const char *name, int depth, + u32 address_cells, u32 size_cells, + struct membanks *mem) +{ + return device_tree_get_meminfo(fdt, node, "reg", address_cells, size_cells, + mem, MEMBANK_DEFAULT); +} + +static int __init process_reserved_memory_node(const void *fdt, int node, + const char *name, int depth, + u32 address_cells, + u32 size_cells, + void *data) +{ + int rc = process_memory_node(fdt, node, name, depth, address_cells, + size_cells, data); + + if ( rc == -ENOSPC ) + panic("Max number of supported reserved-memory regions reached.\n"); + else if ( rc != -ENOENT ) + return rc; + return 0; +} + +static int __init process_reserved_memory(const void *fdt, int node, + const char *name, int depth, + u32 address_cells, u32 size_cells) +{ + return device_tree_for_each_node(fdt, node, + process_reserved_memory_node, + bootinfo_get_reserved_mem()); +} + +static void __init process_multiboot_node(const void *fdt, int node, + const char *name, + u32 address_cells, u32 size_cells) +{ + static int __initdata kind_guess = 0; + const struct fdt_property *prop; + const __be32 *cell; + bootmodule_kind kind; + paddr_t start, size; + int len; + /* sizeof("/chosen/") + DT_MAX_NAME + '/' + DT_MAX_NAME + '/0' => 92 */ + char path[92]; + int parent_node, ret; + bool domU; + + parent_node = fdt_parent_offset(fdt, node); + ASSERT(parent_node >= 0); + + /* Check that the node is under "/chosen" (first 7 chars of path) */ + ret = fdt_get_path(fdt, node, path, sizeof (path)); + if ( ret != 0 || strncmp(path, "/chosen", 7) ) + return; + + prop = fdt_get_property(fdt, node, "reg", &len); + if ( !prop ) + panic("node %s missing `reg' property\n", name); + + if ( len < dt_cells_to_size(address_cells + size_cells) ) + panic("fdt: node `%s': `reg` property length is too short\n", + name); + + cell = (const __be32 *)prop->data; + device_tree_get_reg(&cell, address_cells, size_cells, &start, &size); + + if ( fdt_node_check_compatible(fdt, node, "xen,linux-zimage") == 0 || + fdt_node_check_compatible(fdt, node, "multiboot,kernel") == 0 ) + kind = BOOTMOD_KERNEL; + else if ( fdt_node_check_compatible(fdt, node, "xen,linux-initrd") == 0 || + fdt_node_check_compatible(fdt, node, "multiboot,ramdisk") == 0 ) + kind = BOOTMOD_RAMDISK; + else if ( fdt_node_check_compatible(fdt, node, "xen,xsm-policy") == 0 ) + kind = BOOTMOD_XSM; + else if ( fdt_node_check_compatible(fdt, node, "multiboot,device-tree") == 0 ) + kind = BOOTMOD_GUEST_DTB; + else + kind = BOOTMOD_UNKNOWN; + + /** + * Guess the kind of these first two unknowns respectively: + * (1) The first unknown must be kernel. + * (2) Detect the XSM Magic from the 2nd unknown: + * a. If it's XSM, set the kind as XSM, and that also means we + * won't load ramdisk; + * b. if it's not XSM, set the kind as ramdisk. + * So if user want to load ramdisk, it must be the 2nd unknown. + * We also detect the XSM Magic for the following unknowns, + * then set its kind according to the return value of has_xsm_magic. + */ + if ( kind == BOOTMOD_UNKNOWN ) + { + switch ( kind_guess++ ) + { + case 0: kind = BOOTMOD_KERNEL; break; + case 1: kind = BOOTMOD_RAMDISK; break; + default: break; + } + if ( kind_guess > 1 && has_xsm_magic(start) ) + kind = BOOTMOD_XSM; + } + + domU = fdt_node_check_compatible(fdt, parent_node, "xen,domain") == 0; + add_boot_module(kind, start, size, domU); + + prop = fdt_get_property(fdt, node, "bootargs", &len); + if ( !prop ) + return; + add_boot_cmdline(fdt_get_name(fdt, parent_node, &len), prop->data, + kind, start, domU); +} + +static int __init process_chosen_node(const void *fdt, int node, + const char *name, + u32 address_cells, u32 size_cells) +{ + const struct fdt_property *prop; + paddr_t start, end; + int len; + + if ( fdt_get_property(fdt, node, "xen,static-heap", NULL) ) + { + int rc; + + printk("Checking for static heap in /chosen\n"); + + rc = device_tree_get_meminfo(fdt, node, "xen,static-heap", + address_cells, size_cells, + bootinfo_get_reserved_mem(), + MEMBANK_STATIC_HEAP); + if ( rc ) + return rc; + + bootinfo.static_heap = true; + } + + printk("Checking for initrd in /chosen\n"); + + prop = fdt_get_property(fdt, node, "linux,initrd-start", &len); + if ( !prop ) + /* No initrd present. */ + return 0; + if ( len != sizeof(u32) && len != sizeof(u64) ) + { + printk("linux,initrd-start property has invalid length %d\n", len); + return -EINVAL; + } + start = dt_read_paddr((const void *)&prop->data, dt_size_to_cells(len)); + + prop = fdt_get_property(fdt, node, "linux,initrd-end", &len); + if ( !prop ) + { + printk("linux,initrd-end not present but -start was\n"); + return -EINVAL; + } + if ( len != sizeof(u32) && len != sizeof(u64) ) + { + printk("linux,initrd-end property has invalid length %d\n", len); + return -EINVAL; + } + end = dt_read_paddr((const void *)&prop->data, dt_size_to_cells(len)); + + if ( start >= end ) + { + printk("linux,initrd limits invalid: %"PRIpaddr" >= %"PRIpaddr"\n", + start, end); + return -EINVAL; + } + + printk("Initrd %"PRIpaddr"-%"PRIpaddr"\n", start, end); + + add_boot_module(BOOTMOD_RAMDISK, start, end-start, false); + + return 0; +} + +static int __init process_domain_node(const void *fdt, int node, + const char *name, + u32 address_cells, u32 size_cells) +{ + const struct fdt_property *prop; + + printk("Checking for \"xen,static-mem\" in domain node\n"); + + prop = fdt_get_property(fdt, node, "xen,static-mem", NULL); + if ( !prop ) + /* No "xen,static-mem" present. */ + return 0; + + return device_tree_get_meminfo(fdt, node, "xen,static-mem", address_cells, + size_cells, bootinfo_get_reserved_mem(), + MEMBANK_STATIC_DOMAIN); +} + +#ifndef CONFIG_STATIC_SHM +static inline int process_shm_node(const void *fdt, int node, + uint32_t address_cells, uint32_t size_cells) +{ + printk("CONFIG_STATIC_SHM must be enabled for parsing static shared" + " memory nodes\n"); + return -EINVAL; +} +#endif + +static int __init early_scan_node(const void *fdt, + int node, const char *name, int depth, + u32 address_cells, u32 size_cells, + void *data) +{ + int rc = 0; + + /* + * If Xen has been booted via UEFI, the memory banks are + * populated. So we should skip the parsing. + */ + if ( !efi_enabled(EFI_BOOT) && + device_tree_node_matches(fdt, node, "memory") ) + rc = process_memory_node(fdt, node, name, depth, + address_cells, size_cells, bootinfo_get_mem()); + else if ( depth == 1 && !dt_node_cmp(name, "reserved-memory") ) + rc = process_reserved_memory(fdt, node, name, depth, + address_cells, size_cells); + else if ( depth <= 3 && (device_tree_node_compatible(fdt, node, "xen,multiboot-module" ) || + device_tree_node_compatible(fdt, node, "multiboot,module" ))) + process_multiboot_node(fdt, node, name, address_cells, size_cells); + else if ( depth == 1 && device_tree_node_matches(fdt, node, "chosen") ) + rc = process_chosen_node(fdt, node, name, address_cells, size_cells); + else if ( depth == 2 && device_tree_node_compatible(fdt, node, "xen,domain") ) + rc = process_domain_node(fdt, node, name, address_cells, size_cells); + else if ( depth <= 3 && device_tree_node_compatible(fdt, node, "xen,domain-shared-memory-v1") ) + rc = process_shm_node(fdt, node, address_cells, size_cells); + + if ( rc < 0 ) + printk("fdt: node `%s': parsing failed\n", name); + return rc; +} + +static void __init early_print_info(void) +{ + const struct membanks *mi = bootinfo_get_mem(); + const struct membanks *mem_resv = bootinfo_get_reserved_mem(); + struct bootmodules *mods = &bootinfo.modules; + struct bootcmdlines *cmds = &bootinfo.cmdlines; + unsigned int i; + + for ( i = 0; i < mi->nr_banks; i++ ) + printk("RAM: %"PRIpaddr" - %"PRIpaddr"\n", + mi->bank[i].start, + mi->bank[i].start + mi->bank[i].size - 1); + printk("\n"); + for ( i = 0 ; i < mods->nr_mods; i++ ) + printk("MODULE[%d]: %"PRIpaddr" - %"PRIpaddr" %-12s\n", + i, + mods->module[i].start, + mods->module[i].start + mods->module[i].size, + boot_module_kind_as_string(mods->module[i].kind)); + + for ( i = 0; i < mem_resv->nr_banks; i++ ) + { + printk(" RESVD[%u]: %"PRIpaddr" - %"PRIpaddr"\n", i, + mem_resv->bank[i].start, + mem_resv->bank[i].start + mem_resv->bank[i].size - 1); + } +#ifdef CONFIG_STATIC_SHM + early_print_info_shmem(); +#endif + printk("\n"); + for ( i = 0 ; i < cmds->nr_mods; i++ ) + printk("CMDLINE[%"PRIpaddr"]:%s %s\n", cmds->cmdline[i].start, + cmds->cmdline[i].dt_name, + &cmds->cmdline[i].cmdline[0]); + printk("\n"); +} + +/* This function assumes that memory regions are not overlapped */ +static int __init cmp_memory_node(const void *key, const void *elem) +{ + const struct membank *handler0 = key; + const struct membank *handler1 = elem; + + if ( handler0->start < handler1->start ) + return -1; + + if ( handler0->start >= (handler1->start + handler1->size) ) + return 1; + + return 0; +} + +static void __init swap_memory_node(void *_a, void *_b, size_t size) +{ + struct membank *a = _a, *b = _b; + + SWAP(*a, *b); +} + +/** + * boot_fdt_info - initialize bootinfo from a DTB + * @fdt: flattened device tree binary + * + * Returns the size of the DTB. + */ +size_t __init boot_fdt_info(const void *fdt, paddr_t paddr) +{ + struct membanks *reserved_mem = bootinfo_get_reserved_mem(); + struct membanks *mem = bootinfo_get_mem(); + unsigned int i; + int nr_rsvd; + int ret; + + ret = fdt_check_header(fdt); + if ( ret < 0 ) + panic("No valid device tree\n"); + + add_boot_module(BOOTMOD_FDT, paddr, fdt_totalsize(fdt), false); + + nr_rsvd = fdt_num_mem_rsv(fdt); + if ( nr_rsvd < 0 ) + panic("Parsing FDT memory reserve map failed (%d)\n", nr_rsvd); + + for ( i = 0; i < nr_rsvd; i++ ) + { + struct membank *bank; + paddr_t s, sz; + + if ( fdt_get_mem_rsv_paddr(device_tree_flattened, i, &s, &sz) < 0 ) + continue; + + if ( reserved_mem->nr_banks < reserved_mem->max_banks ) + { + bank = &reserved_mem->bank[reserved_mem->nr_banks]; + bank->start = s; + bank->size = sz; + bank->type = MEMBANK_FDT_RESVMEM; + reserved_mem->nr_banks++; + } + else + panic("Cannot allocate reserved memory bank\n"); + } + + ret = device_tree_for_each_node(fdt, 0, early_scan_node, NULL); + if ( ret ) + panic("Early FDT parsing failed (%d)\n", ret); + + /* + * On Arm64 setup_directmap_mappings() expects to be called with the lowest + * bank in memory first. There is no requirement that the DT will provide + * the banks sorted in ascending order. So sort them through. + */ + sort(mem->bank, mem->nr_banks, sizeof(struct membank), + cmp_memory_node, swap_memory_node); + + early_print_info(); + + return fdt_totalsize(fdt); +} + +const __init char *boot_fdt_cmdline(const void *fdt) +{ + int node; + const struct fdt_property *prop; + + node = fdt_path_offset(fdt, "/chosen"); + if ( node < 0 ) + return NULL; + + prop = fdt_get_property(fdt, node, "xen,xen-bootargs", NULL); + if ( prop == NULL ) + { + struct bootcmdline *dom0_cmdline = + boot_cmdline_find_by_kind(BOOTMOD_KERNEL); + + if (fdt_get_property(fdt, node, "xen,dom0-bootargs", NULL) || + ( dom0_cmdline && dom0_cmdline->cmdline[0] ) ) + prop = fdt_get_property(fdt, node, "bootargs", NULL); + } + if ( prop == NULL ) + return NULL; + + return prop->data; +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/include/xen/bootfdt.h b/xen/include/xen/bootfdt.h index 7cd45b3d4b..5943bc5fe3 100644 --- a/xen/include/xen/bootfdt.h +++ b/xen/include/xen/bootfdt.h @@ -158,6 +158,20 @@ struct bootinfo { extern struct bootinfo bootinfo; +bool check_reserved_regions_overlap(paddr_t region_start, + paddr_t region_size); + +struct bootmodule *add_boot_module(bootmodule_kind kind, + paddr_t start, paddr_t size, bool domU); +struct bootmodule *boot_module_find_by_kind(bootmodule_kind kind); +struct bootmodule * boot_module_find_by_addr_and_kind(bootmodule_kind kind, + paddr_t start); +void add_boot_cmdline(const char *name, const char *cmdline, + bootmodule_kind kind, paddr_t start, bool domU); +struct bootcmdline *boot_cmdline_find_by_kind(bootmodule_kind kind); +struct bootcmdline * boot_cmdline_find_by_name(const char *name); +const char *boot_module_kind_as_string(bootmodule_kind kind); + void populate_boot_allocator(void); size_t boot_fdt_info(const void *fdt, paddr_t paddr); -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* [PATCH v2 3/8] xen/riscv: enable CONFIG_HAS_DEVICE_TREE 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v6 2/8] xen/common: Move Arm's bootfdt.c " Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-15 8:40 ` Jan Beulich 2024-07-12 16:22 ` [PATCH v2 4/8] xen/riscv: setup fixmap mapping Oleksii Kurochko ` (4 subsequent siblings) 7 siblings, 1 reply; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Oleksii Kurochko, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V2: - move 'select HAS_DEVICE_TREE' to CONFIG_RISCV. --- Changes in V1: - new patch --- xen/arch/riscv/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig index f531e96657..259eea8d3b 100644 --- a/xen/arch/riscv/Kconfig +++ b/xen/arch/riscv/Kconfig @@ -2,6 +2,7 @@ config RISCV def_bool y select FUNCTION_ALIGNMENT_16B select GENERIC_BUG_FRAME + select HAS_DEVICE_TREE config RISCV_64 def_bool y -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [PATCH v2 3/8] xen/riscv: enable CONFIG_HAS_DEVICE_TREE 2024-07-12 16:22 ` [PATCH v2 3/8] xen/riscv: enable CONFIG_HAS_DEVICE_TREE Oleksii Kurochko @ 2024-07-15 8:40 ` Jan Beulich 0 siblings, 0 replies; 51+ messages in thread From: Jan Beulich @ 2024-07-15 8:40 UTC (permalink / raw) To: Oleksii Kurochko Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 12.07.2024 18:22, Oleksii Kurochko wrote: > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> Acked-by: Jan Beulich <jbeulich@suse.com> Albeit a little reluctantly, seeing the once again entirely missing description. Please can you get used to not leaving that out, unless the title really says _all_ that's wanted / needed? Jan > --- a/xen/arch/riscv/Kconfig > +++ b/xen/arch/riscv/Kconfig > @@ -2,6 +2,7 @@ config RISCV > def_bool y > select FUNCTION_ALIGNMENT_16B > select GENERIC_BUG_FRAME > + select HAS_DEVICE_TREE > > config RISCV_64 > def_bool y ^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko ` (2 preceding siblings ...) 2024-07-12 16:22 ` [PATCH v2 3/8] xen/riscv: enable CONFIG_HAS_DEVICE_TREE Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-21 8:46 ` Julien Grall 2024-07-22 12:42 ` Jan Beulich 2024-07-12 16:22 ` [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header Oleksii Kurochko ` (3 subsequent siblings) 7 siblings, 2 replies; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Oleksii Kurochko, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini Introduce a function to set up fixmap mappings and L0 page table for fixmap. Additionally, defines were introduced in riscv/config.h to calculate the FIXMAP_BASE address. This involved introducing BOOT_FDT_VIRT_{START, SIZE} and XEN_SIZE, XEN_VIRT_END. Also, the check of Xen size was updated in the riscv/lds.S script to use XEN_SIZE instead of a hardcoded constant. Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V2: - newly introduced patch --- xen/arch/riscv/include/asm/config.h | 9 ++++++ xen/arch/riscv/include/asm/fixmap.h | 48 +++++++++++++++++++++++++++++ xen/arch/riscv/include/asm/mm.h | 2 ++ xen/arch/riscv/include/asm/page.h | 7 +++++ xen/arch/riscv/mm.c | 35 +++++++++++++++++++++ xen/arch/riscv/setup.c | 2 ++ xen/arch/riscv/xen.lds.S | 2 +- 7 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 xen/arch/riscv/include/asm/fixmap.h diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h index 50583aafdc..3275477c17 100644 --- a/xen/arch/riscv/include/asm/config.h +++ b/xen/arch/riscv/include/asm/config.h @@ -74,11 +74,20 @@ #error "unsupported RV_STAGE1_MODE" #endif +#define XEN_SIZE MB(2) +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) + +#define BOOT_FDT_VIRT_START XEN_VIRT_END +#define BOOT_FDT_VIRT_SIZE MB(4) + #define DIRECTMAP_SLOT_END 509 #define DIRECTMAP_SLOT_START 200 #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - SLOTN(DIRECTMAP_SLOT_START)) +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE) +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) + #define FRAMETABLE_SCALE_FACTOR (PAGE_SIZE/sizeof(struct page_info)) #define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / FRAMETABLE_SCALE_FACTOR) + 1) diff --git a/xen/arch/riscv/include/asm/fixmap.h b/xen/arch/riscv/include/asm/fixmap.h new file mode 100644 index 0000000000..fcfb82d69c --- /dev/null +++ b/xen/arch/riscv/include/asm/fixmap.h @@ -0,0 +1,48 @@ +/* + * fixmap.h: compile-time virtual memory allocation + */ +#ifndef __ASM_FIXMAP_H +#define __ASM_FIXMAP_H + +#include <xen/bug.h> +#include <xen/page-size.h> +#include <xen/pmap.h> + +#include <asm/page.h> + +/* Fixmap slots */ +#define FIX_PMAP_BEGIN (0) /* Start of PMAP */ +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of PMAP */ +#define FIX_MISC (FIX_PMAP_END + 1) /* Ephemeral mappings of hardware */ + +#define FIX_LAST FIX_MISC + +#define FIXADDR_START FIXMAP_ADDR(0) +#define FIXADDR_TOP FIXMAP_ADDR(FIX_LAST) + +#ifndef __ASSEMBLY__ + +/* + * Direct access to xen_fixmap[] should only happen when {set, + * clear}_fixmap() is unusable (e.g. where we would end up to + * recursively call the helpers). + */ +extern pte_t xen_fixmap[]; + +/* Map a page in a fixmap entry */ +extern void set_fixmap(unsigned int map, mfn_t mfn, unsigned int attributes); +/* Remove a mapping from a fixmap entry */ +extern void clear_fixmap(unsigned int map); + +#define fix_to_virt(slot) ((void *)FIXMAP_ADDR(slot)) + +static inline unsigned int virt_to_fix(vaddr_t vaddr) +{ + BUG_ON(vaddr >= FIXADDR_TOP || vaddr < FIXADDR_START); + + return ((vaddr - FIXADDR_START) >> PAGE_SHIFT); +} + +#endif /* __ASSEMBLY__ */ + +#endif /* __ASM_FIXMAP_H */ diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h index 25af9e1aaa..a0bdc2bc3a 100644 --- a/xen/arch/riscv/include/asm/mm.h +++ b/xen/arch/riscv/include/asm/mm.h @@ -255,4 +255,6 @@ static inline unsigned int arch_get_dma_bitsize(void) return 32; /* TODO */ } +void setup_fixmap_mappings(void); + #endif /* _ASM_RISCV_MM_H */ diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h index c831e16417..cbbf3656d1 100644 --- a/xen/arch/riscv/include/asm/page.h +++ b/xen/arch/riscv/include/asm/page.h @@ -81,6 +81,13 @@ static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) BUG_ON("unimplemented"); } +/* Write a pagetable entry. */ +static inline void write_pte(pte_t *p, pte_t pte) +{ + *p = pte; + asm volatile ("sfence.vma"); +} + #endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_PAGE_H */ diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 7d09e781bf..d69a174b5d 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) +xen_fixmap[PAGETABLE_ENTRIES]; + #define HANDLE_PGTBL(curr_lvl_num) \ index = pt_index(curr_lvl_num, page_addr); \ if ( pte_is_valid(pgtbl[index]) ) \ @@ -191,6 +194,38 @@ static bool __init check_pgtbl_mode_support(struct mmu_desc *mmu_desc, return is_mode_supported; } +void __init setup_fixmap_mappings(void) +{ + pte_t *pte; + unsigned int i; + + pte = &stage1_pgtbl_root[pt_index(HYP_PT_ROOT_LEVEL, FIXMAP_ADDR(0))]; + + for ( i = HYP_PT_ROOT_LEVEL - 1; i != 0; i-- ) + { + BUG_ON(!pte_is_valid(*pte)); + + pte = (pte_t *)LOAD_TO_LINK(pte_to_paddr(*pte)); + pte = &pte[pt_index(i, FIXMAP_ADDR(0))]; + } + + BUG_ON( pte_is_valid(*pte) ); + + if ( !pte_is_valid(*pte) ) + { + pte_t tmp = paddr_to_pte(LINK_TO_LOAD((unsigned long)&xen_fixmap), PTE_TABLE); + + write_pte(pte, tmp); + + printk("(XEN) fixmap is mapped\n"); + } + + /* + * We only need the zeroeth table allocated, but not the PTEs set, because + * set_fixmap() will set them on the fly. + */ +} + /* * setup_initial_pagetables: * diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c index 4defad68f4..13f0e8c77d 100644 --- a/xen/arch/riscv/setup.c +++ b/xen/arch/riscv/setup.c @@ -46,6 +46,8 @@ void __init noreturn start_xen(unsigned long bootcpu_id, test_macros_from_bug_h(); #endif + setup_fixmap_mappings(); + printk("All set up\n"); for ( ;; ) diff --git a/xen/arch/riscv/xen.lds.S b/xen/arch/riscv/xen.lds.S index 070b19d915..63b1dd7bb6 100644 --- a/xen/arch/riscv/xen.lds.S +++ b/xen/arch/riscv/xen.lds.S @@ -181,6 +181,6 @@ ASSERT(!SIZEOF(.got.plt), ".got.plt non-empty") * Changing the size of Xen binary can require an update of * PGTBL_INITIAL_COUNT. */ -ASSERT(_end - _start <= MB(2), "Xen too large for early-boot assumptions") +ASSERT(_end - _start <= XEN_SIZE, "Xen too large for early-boot assumptions") ASSERT(_ident_end - _ident_start <= IDENT_AREA_SIZE, "identity region is too big"); -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-12 16:22 ` [PATCH v2 4/8] xen/riscv: setup fixmap mapping Oleksii Kurochko @ 2024-07-21 8:46 ` Julien Grall 2024-07-22 14:31 ` Oleksii ` (2 more replies) 2024-07-22 12:42 ` Jan Beulich 1 sibling, 3 replies; 51+ messages in thread From: Julien Grall @ 2024-07-21 8:46 UTC (permalink / raw) To: Oleksii Kurochko, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini Hi Oleksii, On 12/07/2024 17:22, Oleksii Kurochko wrote: > Introduce a function to set up fixmap mappings and L0 page > table for fixmap. > > Additionally, defines were introduced in riscv/config.h to > calculate the FIXMAP_BASE address. > This involved introducing BOOT_FDT_VIRT_{START, SIZE} and > XEN_SIZE, XEN_VIRT_END. > > Also, the check of Xen size was updated in the riscv/lds.S > script to use XEN_SIZE instead of a hardcoded constant. > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > --- > Changes in V2: > - newly introduced patch > --- > xen/arch/riscv/include/asm/config.h | 9 ++++++ > xen/arch/riscv/include/asm/fixmap.h | 48 +++++++++++++++++++++++++++++ > xen/arch/riscv/include/asm/mm.h | 2 ++ > xen/arch/riscv/include/asm/page.h | 7 +++++ > xen/arch/riscv/mm.c | 35 +++++++++++++++++++++ > xen/arch/riscv/setup.c | 2 ++ > xen/arch/riscv/xen.lds.S | 2 +- > 7 files changed, 104 insertions(+), 1 deletion(-) > create mode 100644 xen/arch/riscv/include/asm/fixmap.h > > diff --git a/xen/arch/riscv/include/asm/config.h b/xen/arch/riscv/include/asm/config.h > index 50583aafdc..3275477c17 100644 > --- a/xen/arch/riscv/include/asm/config.h > +++ b/xen/arch/riscv/include/asm/config.h > @@ -74,11 +74,20 @@ > #error "unsupported RV_STAGE1_MODE" > #endif > > +#define XEN_SIZE MB(2) NIT: I would name it XEN_VIRT_SIZE to be consistent with the start/end. > +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) Can we get away with not introducing *_END and just use START, SIZE? The reason I am asking is with "end" it is never clear whether it is inclusive or exclusive. For instance, here you use an exclusive range but ... > + > +#define BOOT_FDT_VIRT_START XEN_VIRT_END > +#define BOOT_FDT_VIRT_SIZE MB(4) > + > #define DIRECTMAP_SLOT_END 509 > #define DIRECTMAP_SLOT_START 200 > #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) > #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - SLOTN(DIRECTMAP_SLOT_START)) > > +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE) > +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) > + > #define FRAMETABLE_SCALE_FACTOR (PAGE_SIZE/sizeof(struct page_info)) > #define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / FRAMETABLE_SCALE_FACTOR) + 1) > > diff --git a/xen/arch/riscv/include/asm/fixmap.h b/xen/arch/riscv/include/asm/fixmap.h > new file mode 100644 > index 0000000000..fcfb82d69c > --- /dev/null > +++ b/xen/arch/riscv/include/asm/fixmap.h > @@ -0,0 +1,48 @@ > +/* > + * fixmap.h: compile-time virtual memory allocation > + */ > +#ifndef __ASM_FIXMAP_H > +#define __ASM_FIXMAP_H > + > +#include <xen/bug.h> > +#include <xen/page-size.h> > +#include <xen/pmap.h> > + > +#include <asm/page.h> > + > +/* Fixmap slots */ > +#define FIX_PMAP_BEGIN (0) /* Start of PMAP */ > +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of PMAP */ ... here is seems to be inclusive. Furthermore if you had 32-bit address space, it is also quite easy to have to create a region right at the top of it. So when END is exclusive, it would become 0. So on Arm, we decided to start to get rid of "end". I would consider to do the same on RISC-V for new functions. > +#define FIX_MISC (FIX_PMAP_END + 1) /* Ephemeral mappings of hardware */ Are you going to use this fixmap? If not, then I would consider to remove it. > + > +#define FIX_LAST FIX_MISC > + > +#define FIXADDR_START FIXMAP_ADDR(0) > +#define FIXADDR_TOP FIXMAP_ADDR(FIX_LAST) > + > +#ifndef __ASSEMBLY__ > + > +/* > + * Direct access to xen_fixmap[] should only happen when {set, > + * clear}_fixmap() is unusable (e.g. where we would end up to > + * recursively call the helpers). > + */ > +extern pte_t xen_fixmap[]; > + > +/* Map a page in a fixmap entry */ > +extern void set_fixmap(unsigned int map, mfn_t mfn, unsigned int attributes); > +/* Remove a mapping from a fixmap entry */ > +extern void clear_fixmap(unsigned int map); Neither of the functions seem to be implemented in this patch. Can you clarify what's the plan for them? Also, I know that for x86/arm, we have some function prefixed with extern. But AFAIK, we are trying to get rid of them. In any case, I think for RISC-V we need some consistency. For instance, here you define with extern but... > + > +#define fix_to_virt(slot) ((void *)FIXMAP_ADDR(slot)) > + > +static inline unsigned int virt_to_fix(vaddr_t vaddr) > +{ > + BUG_ON(vaddr >= FIXADDR_TOP || vaddr < FIXADDR_START); > + > + return ((vaddr - FIXADDR_START) >> PAGE_SHIFT); > +} > + > +#endif /* __ASSEMBLY__ */ > + > +#endif /* __ASM_FIXMAP_H */ > diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h > index 25af9e1aaa..a0bdc2bc3a 100644 > --- a/xen/arch/riscv/include/asm/mm.h > +++ b/xen/arch/riscv/include/asm/mm.h > @@ -255,4 +255,6 @@ static inline unsigned int arch_get_dma_bitsize(void) > return 32; /* TODO */ > } > > +void setup_fixmap_mappings(void); ... here it is without. > + > #endif /* _ASM_RISCV_MM_H */ > diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h > index c831e16417..cbbf3656d1 100644 > --- a/xen/arch/riscv/include/asm/page.h > +++ b/xen/arch/riscv/include/asm/page.h > @@ -81,6 +81,13 @@ static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) > BUG_ON("unimplemented"); > } > > +/* Write a pagetable entry. */ > +static inline void write_pte(pte_t *p, pte_t pte) > +{ > + *p = pte; > + asm volatile ("sfence.vma"); > +} > + > #endif /* __ASSEMBLY__ */ > > #endif /* _ASM_RISCV_PAGE_H */ > diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c > index 7d09e781bf..d69a174b5d 100644 > --- a/xen/arch/riscv/mm.c > +++ b/xen/arch/riscv/mm.c > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > +xen_fixmap[PAGETABLE_ENTRIES]; Can you add a BUILD_BUG_ON() to check that the number of entries in the fixmap will never be above PAGETABLE_ENTRIES? > + > #define HANDLE_PGTBL(curr_lvl_num) \ > index = pt_index(curr_lvl_num, page_addr); \ > if ( pte_is_valid(pgtbl[index]) ) \ > @@ -191,6 +194,38 @@ static bool __init check_pgtbl_mode_support(struct mmu_desc *mmu_desc, > return is_mode_supported; > } > > +void __init setup_fixmap_mappings(void) > +{ > + pte_t *pte; > + unsigned int i; > + > + pte = &stage1_pgtbl_root[pt_index(HYP_PT_ROOT_LEVEL, FIXMAP_ADDR(0))]; > + > + for ( i = HYP_PT_ROOT_LEVEL - 1; i != 0; i-- ) I am a little bit confused with the - 1. Is this because you only want to map at L1 (I am not sure if this is the correct naming for RISC-V)? In any case, I think it would be worth a comment. > + { > + BUG_ON(!pte_is_valid(*pte)); > + > + pte = (pte_t *)LOAD_TO_LINK(pte_to_paddr(*pte)); > + pte = &pte[pt_index(i, FIXMAP_ADDR(0))]; > + } > + > + BUG_ON( pte_is_valid(*pte) ); Coding style: BUG_ON(pte_is_valid(*pte)); > + > + if ( !pte_is_valid(*pte) ) I am a bit confused with this check. Above, Xen will crash if the PTE is valid. So why do we need a runtime check? > + { > + pte_t tmp = paddr_to_pte(LINK_TO_LOAD((unsigned long)&xen_fixmap), PTE_TABLE); > + > + write_pte(pte, tmp); > + > + printk("(XEN) fixmap is mapped\n"); > + } > + > + /* > + * We only need the zeroeth table allocated, but not the PTEs set, because > + * set_fixmap() will set them on the fly. This function doesn't seem to exists yet (?). > + */ > +} > + > /* > * setup_initial_pagetables: > * > diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c > index 4defad68f4..13f0e8c77d 100644 > --- a/xen/arch/riscv/setup.c > +++ b/xen/arch/riscv/setup.c > @@ -46,6 +46,8 @@ void __init noreturn start_xen(unsigned long bootcpu_id, > test_macros_from_bug_h(); > #endif > > + setup_fixmap_mappings(); > + > printk("All set up\n"); > > for ( ;; ) > diff --git a/xen/arch/riscv/xen.lds.S b/xen/arch/riscv/xen.lds.S > index 070b19d915..63b1dd7bb6 100644 > --- a/xen/arch/riscv/xen.lds.S > +++ b/xen/arch/riscv/xen.lds.S > @@ -181,6 +181,6 @@ ASSERT(!SIZEOF(.got.plt), ".got.plt non-empty") > * Changing the size of Xen binary can require an update of > * PGTBL_INITIAL_COUNT. > */ > -ASSERT(_end - _start <= MB(2), "Xen too large for early-boot assumptions") > +ASSERT(_end - _start <= XEN_SIZE, "Xen too large for early-boot assumptions") > > ASSERT(_ident_end - _ident_start <= IDENT_AREA_SIZE, "identity region is too big"); Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-21 8:46 ` Julien Grall @ 2024-07-22 14:31 ` Oleksii 2024-07-22 14:44 ` Julien Grall 2024-07-23 12:58 ` oleksii.kurochko 2024-07-23 13:27 ` oleksii.kurochko 2 siblings, 1 reply; 51+ messages in thread From: Oleksii @ 2024-07-22 14:31 UTC (permalink / raw) To: Julien Grall, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini Hi Julien, On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: > Hi Oleksii, > > On 12/07/2024 17:22, Oleksii Kurochko wrote: > > Introduce a function to set up fixmap mappings and L0 page > > table for fixmap. > > > > Additionally, defines were introduced in riscv/config.h to > > calculate the FIXMAP_BASE address. > > This involved introducing BOOT_FDT_VIRT_{START, SIZE} and > > XEN_SIZE, XEN_VIRT_END. > > > > Also, the check of Xen size was updated in the riscv/lds.S > > script to use XEN_SIZE instead of a hardcoded constant. > > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > --- > > Changes in V2: > > - newly introduced patch > > --- > > xen/arch/riscv/include/asm/config.h | 9 ++++++ > > xen/arch/riscv/include/asm/fixmap.h | 48 > > +++++++++++++++++++++++++++++ > > xen/arch/riscv/include/asm/mm.h | 2 ++ > > xen/arch/riscv/include/asm/page.h | 7 +++++ > > xen/arch/riscv/mm.c | 35 +++++++++++++++++++++ > > xen/arch/riscv/setup.c | 2 ++ > > xen/arch/riscv/xen.lds.S | 2 +- > > 7 files changed, 104 insertions(+), 1 deletion(-) > > create mode 100644 xen/arch/riscv/include/asm/fixmap.h > > > > diff --git a/xen/arch/riscv/include/asm/config.h > > b/xen/arch/riscv/include/asm/config.h > > index 50583aafdc..3275477c17 100644 > > --- a/xen/arch/riscv/include/asm/config.h > > +++ b/xen/arch/riscv/include/asm/config.h > > @@ -74,11 +74,20 @@ > > #error "unsupported RV_STAGE1_MODE" > > #endif > > > > +#define XEN_SIZE MB(2) > > NIT: I would name it XEN_VIRT_SIZE to be consistent with the > start/end. > > > +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) > Can we get away with not introducing *_END and just use START, SIZE? > The > reason I am asking is with "end" it is never clear whether it is > inclusive or exclusive. For instance, here you use an exclusive range > but ... > > > + > > +#define BOOT_FDT_VIRT_START XEN_VIRT_END > > +#define BOOT_FDT_VIRT_SIZE MB(4) > > + > > #define DIRECTMAP_SLOT_END 509 > > #define DIRECTMAP_SLOT_START 200 > > #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) > > #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - > > SLOTN(DIRECTMAP_SLOT_START)) > > > > +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + > > BOOT_FDT_VIRT_SIZE) > > +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) > > + > > #define FRAMETABLE_SCALE_FACTOR (PAGE_SIZE/sizeof(struct > > page_info)) > > #define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / > > FRAMETABLE_SCALE_FACTOR) + 1) > > > > diff --git a/xen/arch/riscv/include/asm/fixmap.h > > b/xen/arch/riscv/include/asm/fixmap.h > > new file mode 100644 > > index 0000000000..fcfb82d69c > > --- /dev/null > > +++ b/xen/arch/riscv/include/asm/fixmap.h > > @@ -0,0 +1,48 @@ > > +/* > > + * fixmap.h: compile-time virtual memory allocation > > + */ > > +#ifndef __ASM_FIXMAP_H > > +#define __ASM_FIXMAP_H > > + > > +#include <xen/bug.h> > > +#include <xen/page-size.h> > > +#include <xen/pmap.h> > > + > > +#include <asm/page.h> > > + > > +/* Fixmap slots */ > > +#define FIX_PMAP_BEGIN (0) /* Start of PMAP */ > > +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of > > PMAP */ > > ... here is seems to be inclusive. Furthermore if you had 32-bit > address > space, it is also quite easy to have to create a region right at the > top > of it. So when END is exclusive, it would become 0. > > So on Arm, we decided to start to get rid of "end". I would consider > to > do the same on RISC-V for new functions. I will refactor the code and get rid of "end". > > > +#define FIX_MISC (FIX_PMAP_END + 1) /* Ephemeral mappings of > > hardware */ > > Are you going to use this fixmap? If not, then I would consider to > remove it. Yes, it used now in copy_from_paddr(): /** * copy_from_paddr - copy data from a physical address * @dst: destination virtual address * @paddr: source physical address * @len: length to copy */ void __init copy_from_paddr(void *dst, paddr_t paddr, unsigned long len) { void *src = (void *)FIXMAP_ADDR(FIXMAP_MISC); while (len) { unsigned long l, s; s = paddr & (PAGE_SIZE-1); l = min(PAGE_SIZE - s, len); set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), PAGE_HYPERVISOR_WC); memcpy(dst, src + s, l); clear_fixmap(FIXMAP_MISC); paddr += l; dst += l; len -= l; } } > > > + > > +#define FIX_LAST FIX_MISC > > + > > +#define FIXADDR_START FIXMAP_ADDR(0) > > +#define FIXADDR_TOP FIXMAP_ADDR(FIX_LAST) > > + > > +#ifndef __ASSEMBLY__ > > + > > +/* > > + * Direct access to xen_fixmap[] should only happen when {set, > > + * clear}_fixmap() is unusable (e.g. where we would end up to > > + * recursively call the helpers). > > + */ > > +extern pte_t xen_fixmap[]; > > + > > +/* Map a page in a fixmap entry */ > > +extern void set_fixmap(unsigned int map, mfn_t mfn, unsigned int > > attributes); > > +/* Remove a mapping from a fixmap entry */ > > +extern void clear_fixmap(unsigned int map); > > Neither of the functions seem to be implemented in this patch. Can > you > clarify what's the plan for them? You are right, it could be dropped now. But in future this functions are used for copy_from_paddr(). Look at the code above. > > Also, I know that for x86/arm, we have some function prefixed with > extern. But AFAIK, we are trying to get rid of them. > > In any case, I think for RISC-V we need some consistency. For > instance, > here you define with extern but... > > > + > > +#define fix_to_virt(slot) ((void *)FIXMAP_ADDR(slot)) > > + > > +static inline unsigned int virt_to_fix(vaddr_t vaddr) > > +{ > > + BUG_ON(vaddr >= FIXADDR_TOP || vaddr < FIXADDR_START); > > + > > + return ((vaddr - FIXADDR_START) >> PAGE_SHIFT); > > +} > > + > > +#endif /* __ASSEMBLY__ */ > > + > > +#endif /* __ASM_FIXMAP_H */ > > diff --git a/xen/arch/riscv/include/asm/mm.h > > b/xen/arch/riscv/include/asm/mm.h > > index 25af9e1aaa..a0bdc2bc3a 100644 > > --- a/xen/arch/riscv/include/asm/mm.h > > +++ b/xen/arch/riscv/include/asm/mm.h > > @@ -255,4 +255,6 @@ static inline unsigned int > > arch_get_dma_bitsize(void) > > return 32; /* TODO */ > > } > > > > +void setup_fixmap_mappings(void); > > ... here it is without. > > > + > > #endif /* _ASM_RISCV_MM_H */ > > diff --git a/xen/arch/riscv/include/asm/page.h > > b/xen/arch/riscv/include/asm/page.h > > index c831e16417..cbbf3656d1 100644 > > --- a/xen/arch/riscv/include/asm/page.h > > +++ b/xen/arch/riscv/include/asm/page.h > > @@ -81,6 +81,13 @@ static inline void flush_page_to_ram(unsigned > > long mfn, bool sync_icache) > > BUG_ON("unimplemented"); > > } > > > > +/* Write a pagetable entry. */ > > +static inline void write_pte(pte_t *p, pte_t pte) > > +{ > > + *p = pte; > > + asm volatile ("sfence.vma"); > > +} > > + > > #endif /* __ASSEMBLY__ */ > > > > #endif /* _ASM_RISCV_PAGE_H */ > > diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c > > index 7d09e781bf..d69a174b5d 100644 > > --- a/xen/arch/riscv/mm.c > > +++ b/xen/arch/riscv/mm.c > > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; > > > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > +xen_fixmap[PAGETABLE_ENTRIES]; > > Can you add a BUILD_BUG_ON() to check that the number of entries in > the > fixmap will never be above PAGETABLE_ENTRIES? Sure. What is the best place? Somewhere in setup_fixmap_mappings()? > > > + > > #define > > HANDLE_PGTBL(curr_lvl_num) > > \ > > index = pt_index(curr_lvl_num, > > page_addr); \ > > if ( pte_is_valid(pgtbl[index]) > > ) \ > > @@ -191,6 +194,38 @@ static bool __init > > check_pgtbl_mode_support(struct mmu_desc *mmu_desc, > > return is_mode_supported; > > } > > > > +void __init setup_fixmap_mappings(void) > > +{ > > + pte_t *pte; > > + unsigned int i; > > + > > + pte = &stage1_pgtbl_root[pt_index(HYP_PT_ROOT_LEVEL, > > FIXMAP_ADDR(0))]; > > + > > + for ( i = HYP_PT_ROOT_LEVEL - 1; i != 0; i-- ) > > I am a little bit confused with the - 1. Is this because you only > want > to map at L1 (I am not sure if this is the correct naming for RISC- > V)? Yes, the idea is that I want to stop in L1 ( 2Mb pages ) as this mapping is already exist and there will not be need to create a new table. ( what will fail because boot allocator isn't initialized yet and alloc_boot_pages() will start to alarm because of BUG_ON(!nr_bootmem_regions) ). RISC-V also uses word levels, but the order is an opposite to Arm. > > In any case, I think it would be worth a comment. Sure, I will add it. > > > + { > > + BUG_ON(!pte_is_valid(*pte)); > > + > > + pte = (pte_t *)LOAD_TO_LINK(pte_to_paddr(*pte)); > > + pte = &pte[pt_index(i, FIXMAP_ADDR(0))]; > > + } > > + > > + BUG_ON( pte_is_valid(*pte) ); > > Coding style: BUG_ON(pte_is_valid(*pte)); > > > + > > + if ( !pte_is_valid(*pte) ) > > I am a bit confused with this check. Above, Xen will crash if the PTE > is > valid. So why do we need a runtime check? You are right, there is no any sense. We should drop it. > > > + { > > + pte_t tmp = paddr_to_pte(LINK_TO_LOAD((unsigned > > long)&xen_fixmap), PTE_TABLE); > > + > > + write_pte(pte, tmp); > > + > > + printk("(XEN) fixmap is mapped\n"); > > + } > > + > > + /* > > + * We only need the zeroeth table allocated, but not the PTEs > > set, because > > + * set_fixmap() will set them on the fly. > > This function doesn't seem to exists yet (?). Not yet. It will be introduced later... ~ Oleksii > > > + */ > > +} > > + > > /* > > * setup_initial_pagetables: > > * > > diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c > > index 4defad68f4..13f0e8c77d 100644 > > --- a/xen/arch/riscv/setup.c > > +++ b/xen/arch/riscv/setup.c > > @@ -46,6 +46,8 @@ void __init noreturn start_xen(unsigned long > > bootcpu_id, > > test_macros_from_bug_h(); > > #endif > > > > + setup_fixmap_mappings(); > > + > > printk("All set up\n"); > > > > for ( ;; ) > > diff --git a/xen/arch/riscv/xen.lds.S b/xen/arch/riscv/xen.lds.S > > index 070b19d915..63b1dd7bb6 100644 > > --- a/xen/arch/riscv/xen.lds.S > > +++ b/xen/arch/riscv/xen.lds.S > > @@ -181,6 +181,6 @@ ASSERT(!SIZEOF(.got.plt), ".got.plt non- > > empty") > > * Changing the size of Xen binary can require an update of > > * PGTBL_INITIAL_COUNT. > > */ > > -ASSERT(_end - _start <= MB(2), "Xen too large for early-boot > > assumptions") > > +ASSERT(_end - _start <= XEN_SIZE, "Xen too large for early-boot > > assumptions") > > > > ASSERT(_ident_end - _ident_start <= IDENT_AREA_SIZE, "identity > > region is too big"); > > Cheers, > ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-22 14:31 ` Oleksii @ 2024-07-22 14:44 ` Julien Grall 0 siblings, 0 replies; 51+ messages in thread From: Julien Grall @ 2024-07-22 14:44 UTC (permalink / raw) To: Oleksii, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini On 22/07/2024 15:31, Oleksii wrote: > Hi Julien, Hi Oleksii, > On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: >> Hi Oleksii, >> >> On 12/07/2024 17:22, Oleksii Kurochko wrote: >>> Introduce a function to set up fixmap mappings and L0 page >>> table for fixmap. >>> >>> Additionally, defines were introduced in riscv/config.h to >>> calculate the FIXMAP_BASE address. >>> This involved introducing BOOT_FDT_VIRT_{START, SIZE} and >>> XEN_SIZE, XEN_VIRT_END. >>> >>> Also, the check of Xen size was updated in the riscv/lds.S >>> script to use XEN_SIZE instead of a hardcoded constant. >>> >>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> >>> --- >>> Changes in V2: >>> - newly introduced patch >>> --- >>> xen/arch/riscv/include/asm/config.h | 9 ++++++ >>> xen/arch/riscv/include/asm/fixmap.h | 48 >>> +++++++++++++++++++++++++++++ >>> xen/arch/riscv/include/asm/mm.h | 2 ++ >>> xen/arch/riscv/include/asm/page.h | 7 +++++ >>> xen/arch/riscv/mm.c | 35 +++++++++++++++++++++ >>> xen/arch/riscv/setup.c | 2 ++ >>> xen/arch/riscv/xen.lds.S | 2 +- >>> 7 files changed, 104 insertions(+), 1 deletion(-) >>> create mode 100644 xen/arch/riscv/include/asm/fixmap.h >>> >>> diff --git a/xen/arch/riscv/include/asm/config.h >>> b/xen/arch/riscv/include/asm/config.h >>> index 50583aafdc..3275477c17 100644 >>> --- a/xen/arch/riscv/include/asm/config.h >>> +++ b/xen/arch/riscv/include/asm/config.h >>> @@ -74,11 +74,20 @@ >>> #error "unsupported RV_STAGE1_MODE" >>> #endif >>> >>> +#define XEN_SIZE MB(2) >> >> NIT: I would name it XEN_VIRT_SIZE to be consistent with the >> start/end. >> >>> +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) >> Can we get away with not introducing *_END and just use START, SIZE? >> The >> reason I am asking is with "end" it is never clear whether it is >> inclusive or exclusive. For instance, here you use an exclusive range >> but ... >> >>> + >>> +#define BOOT_FDT_VIRT_START XEN_VIRT_END >>> +#define BOOT_FDT_VIRT_SIZE MB(4) >>> + >>> #define DIRECTMAP_SLOT_END 509 >>> #define DIRECTMAP_SLOT_START 200 >>> #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) >>> #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - >>> SLOTN(DIRECTMAP_SLOT_START)) >>> >>> +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + >>> BOOT_FDT_VIRT_SIZE) >>> +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) >>> + >>> #define FRAMETABLE_SCALE_FACTOR (PAGE_SIZE/sizeof(struct >>> page_info)) >>> #define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / >>> FRAMETABLE_SCALE_FACTOR) + 1) >>> >>> diff --git a/xen/arch/riscv/include/asm/fixmap.h >>> b/xen/arch/riscv/include/asm/fixmap.h >>> new file mode 100644 >>> index 0000000000..fcfb82d69c >>> --- /dev/null >>> +++ b/xen/arch/riscv/include/asm/fixmap.h >>> @@ -0,0 +1,48 @@ >>> +/* >>> + * fixmap.h: compile-time virtual memory allocation >>> + */ >>> +#ifndef __ASM_FIXMAP_H >>> +#define __ASM_FIXMAP_H >>> + >>> +#include <xen/bug.h> >>> +#include <xen/page-size.h> >>> +#include <xen/pmap.h> >>> + >>> +#include <asm/page.h> >>> + >>> +/* Fixmap slots */ >>> +#define FIX_PMAP_BEGIN (0) /* Start of PMAP */ >>> +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of >>> PMAP */ >> >> ... here is seems to be inclusive. Furthermore if you had 32-bit >> address >> space, it is also quite easy to have to create a region right at the >> top >> of it. So when END is exclusive, it would become 0. >> >> So on Arm, we decided to start to get rid of "end". I would consider >> to >> do the same on RISC-V for new functions. > I will refactor the code and get rid of "end". > >> >>> +#define FIX_MISC (FIX_PMAP_END + 1) /* Ephemeral mappings of >>> hardware */ >> >> Are you going to use this fixmap? If not, then I would consider to >> remove it. > Yes, it used now in copy_from_paddr(): > /** > * copy_from_paddr - copy data from a physical address > * @dst: destination virtual address > * @paddr: source physical address > * @len: length to copy > */ > void __init copy_from_paddr(void *dst, paddr_t paddr, unsigned long > len) > { > void *src = (void *)FIXMAP_ADDR(FIXMAP_MISC); > > while (len) { > unsigned long l, s; > > s = paddr & (PAGE_SIZE-1); > l = min(PAGE_SIZE - s, len); > > set_fixmap(FIXMAP_MISC, maddr_to_mfn(paddr), > PAGE_HYPERVISOR_WC); > memcpy(dst, src + s, l); > clear_fixmap(FIXMAP_MISC); > > paddr += l; > dst += l; > len -= l; > } > } > >> >>> + >>> +#define FIX_LAST FIX_MISC >>> + >>> +#define FIXADDR_START FIXMAP_ADDR(0) >>> +#define FIXADDR_TOP FIXMAP_ADDR(FIX_LAST) >>> + >>> +#ifndef __ASSEMBLY__ >>> + >>> +/* >>> + * Direct access to xen_fixmap[] should only happen when {set, >>> + * clear}_fixmap() is unusable (e.g. where we would end up to >>> + * recursively call the helpers). >>> + */ >>> +extern pte_t xen_fixmap[]; >>> + >>> +/* Map a page in a fixmap entry */ >>> +extern void set_fixmap(unsigned int map, mfn_t mfn, unsigned int >>> attributes); >>> +/* Remove a mapping from a fixmap entry */ >>> +extern void clear_fixmap(unsigned int map); >> >> Neither of the functions seem to be implemented in this patch. Can >> you >> clarify what's the plan for them? > You are right, it could be dropped now. But in future this functions > are used for copy_from_paddr(). Look at the code above. Right, to me it is just odd we are definition prototype for functions that don't yet exist. >> >> Also, I know that for x86/arm, we have some function prefixed with >> extern. But AFAIK, we are trying to get rid of them. >> >> In any case, I think for RISC-V we need some consistency. For >> instance, >> here you define with extern but... >> >>> + >>> +#define fix_to_virt(slot) ((void *)FIXMAP_ADDR(slot)) >>> + >>> +static inline unsigned int virt_to_fix(vaddr_t vaddr) >>> +{ >>> + BUG_ON(vaddr >= FIXADDR_TOP || vaddr < FIXADDR_START); >>> + >>> + return ((vaddr - FIXADDR_START) >> PAGE_SHIFT); >>> +} >>> + >>> +#endif /* __ASSEMBLY__ */ >>> + >>> +#endif /* __ASM_FIXMAP_H */ >>> diff --git a/xen/arch/riscv/include/asm/mm.h >>> b/xen/arch/riscv/include/asm/mm.h >>> index 25af9e1aaa..a0bdc2bc3a 100644 >>> --- a/xen/arch/riscv/include/asm/mm.h >>> +++ b/xen/arch/riscv/include/asm/mm.h >>> @@ -255,4 +255,6 @@ static inline unsigned int >>> arch_get_dma_bitsize(void) >>> return 32; /* TODO */ >>> } >>> >>> +void setup_fixmap_mappings(void); >> >> ... here it is without. >> >>> + >>> #endif /* _ASM_RISCV_MM_H */ >>> diff --git a/xen/arch/riscv/include/asm/page.h >>> b/xen/arch/riscv/include/asm/page.h >>> index c831e16417..cbbf3656d1 100644 >>> --- a/xen/arch/riscv/include/asm/page.h >>> +++ b/xen/arch/riscv/include/asm/page.h >>> @@ -81,6 +81,13 @@ static inline void flush_page_to_ram(unsigned >>> long mfn, bool sync_icache) >>> BUG_ON("unimplemented"); >>> } >>> >>> +/* Write a pagetable entry. */ >>> +static inline void write_pte(pte_t *p, pte_t pte) >>> +{ >>> + *p = pte; >>> + asm volatile ("sfence.vma"); >>> +} >>> + >>> #endif /* __ASSEMBLY__ */ >>> >>> #endif /* _ASM_RISCV_PAGE_H */ >>> diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c >>> index 7d09e781bf..d69a174b5d 100644 >>> --- a/xen/arch/riscv/mm.c >>> +++ b/xen/arch/riscv/mm.c >>> @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; >>> pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) >>> stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; >>> >>> +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) >>> +xen_fixmap[PAGETABLE_ENTRIES]; >> >> Can you add a BUILD_BUG_ON() to check that the number of entries in >> the >> fixmap will never be above PAGETABLE_ENTRIES? > Sure. What is the best place? Somewhere in setup_fixmap_mappings()? I think so. Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-21 8:46 ` Julien Grall 2024-07-22 14:31 ` Oleksii @ 2024-07-23 12:58 ` oleksii.kurochko 2024-07-23 13:32 ` Julien Grall 2024-07-23 13:27 ` oleksii.kurochko 2 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 12:58 UTC (permalink / raw) To: Julien Grall, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini Hi Julien, On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: > > +/* Fixmap slots */ > > +#define FIX_PMAP_BEGIN (0) /* Start of PMAP */ > > +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of > > PMAP */ > > ... here is seems to be inclusive. Furthermore if you had 32-bit > address > space, it is also quite easy to have to create a region right at the > top > of it. So when END is exclusive, it would become 0. > > So on Arm, we decided to start to get rid of "end". I would consider > to > do the same on RISC-V for new functions. I assume that you wrote here just as an example of confusion occurs because of using *_END but just to be clear I have to leave FIXMAP_MAP_END as-is because it is used now by common code. ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-23 12:58 ` oleksii.kurochko @ 2024-07-23 13:32 ` Julien Grall 0 siblings, 0 replies; 51+ messages in thread From: Julien Grall @ 2024-07-23 13:32 UTC (permalink / raw) To: oleksii.kurochko, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini On 23/07/2024 13:58, oleksii.kurochko@gmail.com wrote: > Hi Julien, Hi Oleksii, > On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: >>> +/* Fixmap slots */ >>> +#define FIX_PMAP_BEGIN (0) /* Start of PMAP */ >>> +#define FIX_PMAP_END (FIX_PMAP_BEGIN + NUM_FIX_PMAP - 1) /* End of >>> PMAP */ >> >> ... here is seems to be inclusive. Furthermore if you had 32-bit >> address >> space, it is also quite easy to have to create a region right at the >> top >> of it. So when END is exclusive, it would become 0. >> >> So on Arm, we decided to start to get rid of "end". I would consider >> to >> do the same on RISC-V for new functions. > I assume that you wrote here just as an example of confusion occurs > because of using *_END but just to be clear I have to leave > FIXMAP_MAP_END as-is because it is used now by common code. Indeed. FIXMAP_PMAP_END should stay for now. Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-21 8:46 ` Julien Grall 2024-07-22 14:31 ` Oleksii 2024-07-23 12:58 ` oleksii.kurochko @ 2024-07-23 13:27 ` oleksii.kurochko 2024-07-23 13:33 ` Julien Grall 2 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 13:27 UTC (permalink / raw) To: Julien Grall, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini Hello Julien, On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: > > diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c > > index 7d09e781bf..d69a174b5d 100644 > > --- a/xen/arch/riscv/mm.c > > +++ b/xen/arch/riscv/mm.c > > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; > > > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > +xen_fixmap[PAGETABLE_ENTRIES]; > > Can you add a BUILD_BUG_ON() to check that the number of entries in > the > fixmap will never be above PAGETABLE_ENTRIES? I just realized that we don't have the information about how many entries has been used. Am I confusing something? ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-23 13:27 ` oleksii.kurochko @ 2024-07-23 13:33 ` Julien Grall 2024-07-23 13:38 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: Julien Grall @ 2024-07-23 13:33 UTC (permalink / raw) To: oleksii.kurochko, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini On 23/07/2024 14:27, oleksii.kurochko@gmail.com wrote: > Hello Julien, Hi Oleksii, > On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: >>> diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c >>> index 7d09e781bf..d69a174b5d 100644 >>> --- a/xen/arch/riscv/mm.c >>> +++ b/xen/arch/riscv/mm.c >>> @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; >>> pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) >>> stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; >>> >>> +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) >>> +xen_fixmap[PAGETABLE_ENTRIES]; >> >> Can you add a BUILD_BUG_ON() to check that the number of entries in >> the >> fixmap will never be above PAGETABLE_ENTRIES? > I just realized that we don't have the information about how many > entries has been used. Am I confusing something? I think we do. It is FIX_LAST. Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-23 13:33 ` Julien Grall @ 2024-07-23 13:38 ` oleksii.kurochko 0 siblings, 0 replies; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 13:38 UTC (permalink / raw) To: Julien Grall, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini On Tue, 2024-07-23 at 14:33 +0100, Julien Grall wrote: > On 23/07/2024 14:27, oleksii.kurochko@gmail.com wrote: > > Hello Julien, > > Hi Oleksii, > > > > On Sun, 2024-07-21 at 09:46 +0100, Julien Grall wrote: > > > > diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c > > > > index 7d09e781bf..d69a174b5d 100644 > > > > --- a/xen/arch/riscv/mm.c > > > > +++ b/xen/arch/riscv/mm.c > > > > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > > > > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > > > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * > > > > PAGETABLE_ENTRIES]; > > > > > > > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > > > +xen_fixmap[PAGETABLE_ENTRIES]; > > > > > > Can you add a BUILD_BUG_ON() to check that the number of entries > > > in > > > the > > > fixmap will never be above PAGETABLE_ENTRIES? > > I just realized that we don't have the information about how many > > entries has been used. Am I confusing something? > > I think we do. It is FIX_LAST. Sure. We have FIX_LAST. Thanks ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-12 16:22 ` [PATCH v2 4/8] xen/riscv: setup fixmap mapping Oleksii Kurochko 2024-07-21 8:46 ` Julien Grall @ 2024-07-22 12:42 ` Jan Beulich 2024-07-22 14:36 ` Oleksii 1 sibling, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-22 12:42 UTC (permalink / raw) To: Oleksii Kurochko Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 12.07.2024 18:22, Oleksii Kurochko wrote: > --- a/xen/arch/riscv/include/asm/config.h > +++ b/xen/arch/riscv/include/asm/config.h > @@ -74,11 +74,20 @@ > #error "unsupported RV_STAGE1_MODE" > #endif > > +#define XEN_SIZE MB(2) > +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) > + > +#define BOOT_FDT_VIRT_START XEN_VIRT_END > +#define BOOT_FDT_VIRT_SIZE MB(4) > + > #define DIRECTMAP_SLOT_END 509 > #define DIRECTMAP_SLOT_START 200 > #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) > #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - SLOTN(DIRECTMAP_SLOT_START)) > > +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + BOOT_FDT_VIRT_SIZE) > +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) Why exactly do you insert this here, and not adjacent to BOOT_FDT_VIRT_*, which it actually is adjacent with? Then ... > #define FRAMETABLE_SCALE_FACTOR (PAGE_SIZE/sizeof(struct page_info)) > #define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / FRAMETABLE_SCALE_FACTOR) + 1) ... this would also stay next to DIRECTMAP_*, which it uses. > --- a/xen/arch/riscv/include/asm/page.h > +++ b/xen/arch/riscv/include/asm/page.h > @@ -81,6 +81,13 @@ static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache) > BUG_ON("unimplemented"); > } > > +/* Write a pagetable entry. */ > +static inline void write_pte(pte_t *p, pte_t pte) > +{ > + *p = pte; > + asm volatile ("sfence.vma"); When they don't have operands, asm()-s are volatile anyway (being explicit about this may still be desirable, yes). But: Can you get away without operands here? Don't you need a memory clobber for the fence to actually remain where it is needed? Also, nit (style): Blanks missing. > --- a/xen/arch/riscv/mm.c > +++ b/xen/arch/riscv/mm.c > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > +xen_fixmap[PAGETABLE_ENTRIES]; Any reason this cannot be static? Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-22 12:42 ` Jan Beulich @ 2024-07-22 14:36 ` Oleksii 2024-07-22 15:25 ` Jan Beulich 0 siblings, 1 reply; 51+ messages in thread From: Oleksii @ 2024-07-22 14:36 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On Mon, 2024-07-22 at 14:42 +0200, Jan Beulich wrote: > On 12.07.2024 18:22, Oleksii Kurochko wrote: > > --- a/xen/arch/riscv/include/asm/config.h > > +++ b/xen/arch/riscv/include/asm/config.h > > @@ -74,11 +74,20 @@ > > #error "unsupported RV_STAGE1_MODE" > > #endif > > > > +#define XEN_SIZE MB(2) > > +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) > > + > > +#define BOOT_FDT_VIRT_START XEN_VIRT_END > > +#define BOOT_FDT_VIRT_SIZE MB(4) > > + > > #define DIRECTMAP_SLOT_END 509 > > #define DIRECTMAP_SLOT_START 200 > > #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) > > #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - > > SLOTN(DIRECTMAP_SLOT_START)) > > > > +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + > > BOOT_FDT_VIRT_SIZE) > > +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) > > Why exactly do you insert this here, and not adjacent to > BOOT_FDT_VIRT_*, > which it actually is adjacent with? I tried to follow alphabetical order. > Then ... > > > #define FRAMETABLE_SCALE_FACTOR (PAGE_SIZE/sizeof(struct > > page_info)) > > #define FRAMETABLE_SIZE_IN_SLOTS (((DIRECTMAP_SIZE / SLOTN(1)) / > > FRAMETABLE_SCALE_FACTOR) + 1) > > ... this would also stay next to DIRECTMAP_*, which it uses. > > > --- a/xen/arch/riscv/include/asm/page.h > > +++ b/xen/arch/riscv/include/asm/page.h > > @@ -81,6 +81,13 @@ static inline void flush_page_to_ram(unsigned > > long mfn, bool sync_icache) > > BUG_ON("unimplemented"); > > } > > > > +/* Write a pagetable entry. */ > > +static inline void write_pte(pte_t *p, pte_t pte) > > +{ > > + *p = pte; > > + asm volatile ("sfence.vma"); > > When they don't have operands, asm()-s are volatile anyway (being > explicit > about this may still be desirable, yes). But: Can you get away > without > operands here? Don't you need a memory clobber for the fence to > actually > remain where it is needed? It should be "::memory" here. Thanks. > > Also, nit (style): Blanks missing. > > > --- a/xen/arch/riscv/mm.c > > +++ b/xen/arch/riscv/mm.c > > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; > > > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > +xen_fixmap[PAGETABLE_ENTRIES]; > > Any reason this cannot be static? It will be used by pmap: static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) { pte_t *entry = &xen_fixmap[slot]; pte_t pte; ASSERT(!pte_is_valid(*entry)); pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); pte.pte |= PTE_LEAF_DEFAULT; write_pte(entry, pte); } static inline void arch_pmap_unmap(unsigned int slot) { pte_t pte = {}; write_pte(&xen_fixmap[slot], pte); } ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-22 14:36 ` Oleksii @ 2024-07-22 15:25 ` Jan Beulich 2024-07-22 17:04 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-22 15:25 UTC (permalink / raw) To: Oleksii Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 22.07.2024 16:36, Oleksii wrote: > On Mon, 2024-07-22 at 14:42 +0200, Jan Beulich wrote: >> On 12.07.2024 18:22, Oleksii Kurochko wrote: >>> --- a/xen/arch/riscv/include/asm/config.h >>> +++ b/xen/arch/riscv/include/asm/config.h >>> @@ -74,11 +74,20 @@ >>> #error "unsupported RV_STAGE1_MODE" >>> #endif >>> >>> +#define XEN_SIZE MB(2) >>> +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) >>> + >>> +#define BOOT_FDT_VIRT_START XEN_VIRT_END >>> +#define BOOT_FDT_VIRT_SIZE MB(4) >>> + >>> #define DIRECTMAP_SLOT_END 509 >>> #define DIRECTMAP_SLOT_START 200 >>> #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) >>> #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - >>> SLOTN(DIRECTMAP_SLOT_START)) >>> >>> +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + >>> BOOT_FDT_VIRT_SIZE) >>> +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * PAGE_SIZE) >> >> Why exactly do you insert this here, and not adjacent to >> BOOT_FDT_VIRT_*, >> which it actually is adjacent with? > I tried to follow alphabetical order. Oh, X before B (just making fun) ... Anyway, my take here is that sorting by address is going to be more helpful. >>> --- a/xen/arch/riscv/mm.c >>> +++ b/xen/arch/riscv/mm.c >>> @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; >>> pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) >>> stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; >>> >>> +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) >>> +xen_fixmap[PAGETABLE_ENTRIES]; >> >> Any reason this cannot be static? > It will be used by pmap: > static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > { > pte_t *entry = &xen_fixmap[slot]; > pte_t pte; > > ASSERT(!pte_is_valid(*entry)); > > pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > pte.pte |= PTE_LEAF_DEFAULT; > write_pte(entry, pte); > } > > static inline void arch_pmap_unmap(unsigned int slot) > { > pte_t pte = {}; > > write_pte(&xen_fixmap[slot], pte); > } Yet as asked there - shouldn't that be set_fixmap() and clear_fixmap()? Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-22 15:25 ` Jan Beulich @ 2024-07-22 17:04 ` oleksii.kurochko 2024-07-23 13:34 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-22 17:04 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On Mon, 2024-07-22 at 17:25 +0200, Jan Beulich wrote: > On 22.07.2024 16:36, Oleksii wrote: > > On Mon, 2024-07-22 at 14:42 +0200, Jan Beulich wrote: > > > On 12.07.2024 18:22, Oleksii Kurochko wrote: > > > > --- a/xen/arch/riscv/include/asm/config.h > > > > +++ b/xen/arch/riscv/include/asm/config.h > > > > @@ -74,11 +74,20 @@ > > > > #error "unsupported RV_STAGE1_MODE" > > > > #endif > > > > > > > > +#define XEN_SIZE MB(2) > > > > +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) > > > > + > > > > +#define BOOT_FDT_VIRT_START XEN_VIRT_END > > > > +#define BOOT_FDT_VIRT_SIZE MB(4) > > > > + > > > > #define DIRECTMAP_SLOT_END 509 > > > > #define DIRECTMAP_SLOT_START 200 > > > > #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) > > > > #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - > > > > SLOTN(DIRECTMAP_SLOT_START)) > > > > > > > > +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + > > > > BOOT_FDT_VIRT_SIZE) > > > > +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * > > > > PAGE_SIZE) > > > > > > Why exactly do you insert this here, and not adjacent to > > > BOOT_FDT_VIRT_*, > > > which it actually is adjacent with? > > I tried to follow alphabetical order. > > Oh, X before B (just making fun) ... Anyway, my take here is that > sorting > by address is going to be more helpful. > > > > > --- a/xen/arch/riscv/mm.c > > > > +++ b/xen/arch/riscv/mm.c > > > > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > > > > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > > > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * PAGETABLE_ENTRIES]; > > > > > > > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > > > +xen_fixmap[PAGETABLE_ENTRIES]; > > > > > > Any reason this cannot be static? > > It will be used by pmap: > > static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > > { > > pte_t *entry = &xen_fixmap[slot]; > > pte_t pte; > > > > ASSERT(!pte_is_valid(*entry)); > > > > pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > > pte.pte |= PTE_LEAF_DEFAULT; > > write_pte(entry, pte); > > } > > > > static inline void arch_pmap_unmap(unsigned int slot) > > { > > pte_t pte = {}; > > > > write_pte(&xen_fixmap[slot], pte); > > } > > Yet as asked there - shouldn't that be set_fixmap() and > clear_fixmap()? It should be, I'll rework that in the next patch version. ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 4/8] xen/riscv: setup fixmap mapping 2024-07-22 17:04 ` oleksii.kurochko @ 2024-07-23 13:34 ` oleksii.kurochko 0 siblings, 0 replies; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 13:34 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On Mon, 2024-07-22 at 19:04 +0200, oleksii.kurochko@gmail.com wrote: > On Mon, 2024-07-22 at 17:25 +0200, Jan Beulich wrote: > > On 22.07.2024 16:36, Oleksii wrote: > > > On Mon, 2024-07-22 at 14:42 +0200, Jan Beulich wrote: > > > > On 12.07.2024 18:22, Oleksii Kurochko wrote: > > > > > --- a/xen/arch/riscv/include/asm/config.h > > > > > +++ b/xen/arch/riscv/include/asm/config.h > > > > > @@ -74,11 +74,20 @@ > > > > > #error "unsupported RV_STAGE1_MODE" > > > > > #endif > > > > > > > > > > +#define XEN_SIZE MB(2) > > > > > +#define XEN_VIRT_END (XEN_VIRT_START + XEN_SIZE) > > > > > + > > > > > +#define BOOT_FDT_VIRT_START XEN_VIRT_END > > > > > +#define BOOT_FDT_VIRT_SIZE MB(4) > > > > > + > > > > > #define DIRECTMAP_SLOT_END 509 > > > > > #define DIRECTMAP_SLOT_START 200 > > > > > #define DIRECTMAP_VIRT_START SLOTN(DIRECTMAP_SLOT_START) > > > > > #define DIRECTMAP_SIZE (SLOTN(DIRECTMAP_SLOT_END) - > > > > > SLOTN(DIRECTMAP_SLOT_START)) > > > > > > > > > > +#define FIXMAP_BASE (BOOT_FDT_VIRT_START + > > > > > BOOT_FDT_VIRT_SIZE) > > > > > +#define FIXMAP_ADDR(n) (FIXMAP_BASE + (n) * > > > > > PAGE_SIZE) > > > > > > > > Why exactly do you insert this here, and not adjacent to > > > > BOOT_FDT_VIRT_*, > > > > which it actually is adjacent with? > > > I tried to follow alphabetical order. > > > > Oh, X before B (just making fun) ... Anyway, my take here is that > > sorting > > by address is going to be more helpful. > > > > > > > --- a/xen/arch/riscv/mm.c > > > > > +++ b/xen/arch/riscv/mm.c > > > > > @@ -49,6 +49,9 @@ stage1_pgtbl_root[PAGETABLE_ENTRIES]; > > > > > pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > > > > stage1_pgtbl_nonroot[PGTBL_INITIAL_COUNT * > > > > > PAGETABLE_ENTRIES]; > > > > > > > > > > +pte_t __section(".bss.page_aligned") __aligned(PAGE_SIZE) > > > > > +xen_fixmap[PAGETABLE_ENTRIES]; > > > > > > > > Any reason this cannot be static? > > > It will be used by pmap: > > > static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > > > { > > > pte_t *entry = &xen_fixmap[slot]; > > > pte_t pte; > > > > > > ASSERT(!pte_is_valid(*entry)); > > > > > > pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > > > pte.pte |= PTE_LEAF_DEFAULT; > > > write_pte(entry, pte); > > > } > > > > > > static inline void arch_pmap_unmap(unsigned int slot) > > > { > > > pte_t pte = {}; > > > > > > write_pte(&xen_fixmap[slot], pte); > > > } > > > > Yet as asked there - shouldn't that be set_fixmap() and > > clear_fixmap()? > It should be, I'll rework that in the next patch version. It couldn't be set_fixmap() and clear_fixmap() as I am going to implement them using map_pages_to_xen() because: ... /* * We cannot use set_fixmap() here. We use PMAP when the domain map * page infrastructure is not yet initialized, so map_pages_to_xen() called * by set_fixmap() needs to map pages on demand, which then calls pmap() * again, resulting in a loop. Modify the PTEs directly instead. The same * is true for pmap_unmap(). */ arch_pmap_map(slot, mfn); ... ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko ` (3 preceding siblings ...) 2024-07-12 16:22 ` [PATCH v2 4/8] xen/riscv: setup fixmap mapping Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-21 8:51 ` Julien Grall 2024-07-22 12:54 ` Jan Beulich 2024-07-12 16:22 ` [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling Oleksii Kurochko ` (2 subsequent siblings) 7 siblings, 2 replies; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Oleksii Kurochko, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini Introduces arch_pmap_{un}map functions and select HAS_PMAP for CONFIG_RISCV. Additionaly it was necessary to introduce functions: - mfn_to_xen_entry - mfn_to_pte Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V2: - newly introduced patch --- xen/arch/riscv/Kconfig | 1 + xen/arch/riscv/include/asm/page.h | 2 ++ xen/arch/riscv/include/asm/pmap.h | 28 ++++++++++++++++++++++++++++ xen/arch/riscv/mm.c | 14 ++++++++++++++ 4 files changed, 45 insertions(+) create mode 100644 xen/arch/riscv/include/asm/pmap.h diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig index 259eea8d3b..0112aa8778 100644 --- a/xen/arch/riscv/Kconfig +++ b/xen/arch/riscv/Kconfig @@ -3,6 +3,7 @@ config RISCV select FUNCTION_ALIGNMENT_16B select GENERIC_BUG_FRAME select HAS_DEVICE_TREE + select HAS_PMAP config RISCV_64 def_bool y diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h index cbbf3656d1..339074d502 100644 --- a/xen/arch/riscv/include/asm/page.h +++ b/xen/arch/riscv/include/asm/page.h @@ -51,6 +51,8 @@ typedef struct { #endif } pte_t; +pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr); + static inline pte_t paddr_to_pte(paddr_t paddr, unsigned int permissions) { diff --git a/xen/arch/riscv/include/asm/pmap.h b/xen/arch/riscv/include/asm/pmap.h new file mode 100644 index 0000000000..eb4c48515c --- /dev/null +++ b/xen/arch/riscv/include/asm/pmap.h @@ -0,0 +1,28 @@ +#ifndef __ASM_PMAP_H__ +#define __ASM_PMAP_H__ + +#include <xen/bug.h> +#include <xen/mm.h> + +#include <asm/fixmap.h> + +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) +{ + pte_t *entry = &xen_fixmap[slot]; + pte_t pte; + + ASSERT(!pte_is_valid(*entry)); + + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); + pte.pte |= PTE_LEAF_DEFAULT; + write_pte(entry, pte); +} + +static inline void arch_pmap_unmap(unsigned int slot) +{ + pte_t pte = {}; + + write_pte(&xen_fixmap[slot], pte); +} + +#endif /* __ASM_PMAP_H__ */ diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index d69a174b5d..445319af08 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -370,3 +370,17 @@ int map_pages_to_xen(unsigned long virt, BUG_ON("unimplemented"); return -1; } + +static inline pte_t mfn_to_pte(mfn_t mfn) +{ + unsigned long pte = mfn_x(mfn) << PTE_PPN_SHIFT; + return (pte_t){ .pte = pte}; +} + +inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) +{ + /* there is no attr field in RISC-V's pte */ + (void) attr; + + return mfn_to_pte(mfn); +} -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-12 16:22 ` [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header Oleksii Kurochko @ 2024-07-21 8:51 ` Julien Grall 2024-07-22 12:58 ` Jan Beulich 2024-07-22 14:40 ` Oleksii 2024-07-22 12:54 ` Jan Beulich 1 sibling, 2 replies; 51+ messages in thread From: Julien Grall @ 2024-07-21 8:51 UTC (permalink / raw) To: Oleksii Kurochko, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini Hi Oleksii, On 12/07/2024 17:22, Oleksii Kurochko wrote: > Introduces arch_pmap_{un}map functions and select HAS_PMAP > for CONFIG_RISCV. > > Additionaly it was necessary to introduce functions: > - mfn_to_xen_entry > - mfn_to_pte > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > --- > Changes in V2: > - newly introduced patch > --- > xen/arch/riscv/Kconfig | 1 + > xen/arch/riscv/include/asm/page.h | 2 ++ > xen/arch/riscv/include/asm/pmap.h | 28 ++++++++++++++++++++++++++++ > xen/arch/riscv/mm.c | 14 ++++++++++++++ > 4 files changed, 45 insertions(+) > create mode 100644 xen/arch/riscv/include/asm/pmap.h > > diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig > index 259eea8d3b..0112aa8778 100644 > --- a/xen/arch/riscv/Kconfig > +++ b/xen/arch/riscv/Kconfig > @@ -3,6 +3,7 @@ config RISCV > select FUNCTION_ALIGNMENT_16B > select GENERIC_BUG_FRAME > select HAS_DEVICE_TREE > + select HAS_PMAP > > config RISCV_64 > def_bool y > diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h > index cbbf3656d1..339074d502 100644 > --- a/xen/arch/riscv/include/asm/page.h > +++ b/xen/arch/riscv/include/asm/page.h > @@ -51,6 +51,8 @@ typedef struct { > #endif > } pte_t; > > +pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr); > + > static inline pte_t paddr_to_pte(paddr_t paddr, > unsigned int permissions) > { > diff --git a/xen/arch/riscv/include/asm/pmap.h b/xen/arch/riscv/include/asm/pmap.h > new file mode 100644 > index 0000000000..eb4c48515c > --- /dev/null > +++ b/xen/arch/riscv/include/asm/pmap.h > @@ -0,0 +1,28 @@ > +#ifndef __ASM_PMAP_H__ > +#define __ASM_PMAP_H__ > + > +#include <xen/bug.h> > +#include <xen/mm.h> > + > +#include <asm/fixmap.h> > + > +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > +{ > + pte_t *entry = &xen_fixmap[slot]; > + pte_t pte; > + > + ASSERT(!pte_is_valid(*entry)); > + > + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > + pte.pte |= PTE_LEAF_DEFAULT; > + write_pte(entry, pte); > +} > + > +static inline void arch_pmap_unmap(unsigned int slot) > +{ > + pte_t pte = {}; > + > + write_pte(&xen_fixmap[slot], pte); > +} > + > +#endif /* __ASM_PMAP_H__ */ > diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c > index d69a174b5d..445319af08 100644 > --- a/xen/arch/riscv/mm.c > +++ b/xen/arch/riscv/mm.c > @@ -370,3 +370,17 @@ int map_pages_to_xen(unsigned long virt, > BUG_ON("unimplemented"); > return -1; > } > + > +static inline pte_t mfn_to_pte(mfn_t mfn) > +{ > + unsigned long pte = mfn_x(mfn) << PTE_PPN_SHIFT; > + return (pte_t){ .pte = pte}; > +} > + > +inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) > +{ > + /* there is no attr field in RISC-V's pte */ > + (void) attr; Surely you have a way to say indicate whether an entry is readable/writable? > + > + return mfn_to_pte(mfn); > +} Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-21 8:51 ` Julien Grall @ 2024-07-22 12:58 ` Jan Beulich 2024-07-22 14:57 ` Julien Grall 2024-07-22 14:40 ` Oleksii 1 sibling, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-22 12:58 UTC (permalink / raw) To: Julien Grall Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, Oleksii Kurochko, xen-devel On 21.07.2024 10:51, Julien Grall wrote: > On 12/07/2024 17:22, Oleksii Kurochko wrote: >> +inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) >> +{ >> + /* there is no attr field in RISC-V's pte */ >> + (void) attr; > > Surely you have a way to say indicate whether an entry is readable/writable? I'm puzzled by this question. The sole outlier in Arm code is pmap.h, in passing PAGE_HYPERVISOR_RW to this function when all others pass memory types (MT_*). Afaics only the low three bits are then used in the function, discarding access control bits altogether. R/W access appears to be implied. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-22 12:58 ` Jan Beulich @ 2024-07-22 14:57 ` Julien Grall 0 siblings, 0 replies; 51+ messages in thread From: Julien Grall @ 2024-07-22 14:57 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, Oleksii Kurochko, xen-devel On 22/07/2024 13:58, Jan Beulich wrote: > On 21.07.2024 10:51, Julien Grall wrote: >> On 12/07/2024 17:22, Oleksii Kurochko wrote: >>> +inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) >>> +{ >>> + /* there is no attr field in RISC-V's pte */ >>> + (void) attr; >> >> Surely you have a way to say indicate whether an entry is readable/writable? > > I'm puzzled by this question. The sole outlier in Arm code is pmap.h, in > passing PAGE_HYPERVISOR_RW to this function when all others pass memory > types (MT_*). > Afaics only the low three bits are then used in the > function, discarding access control bits altogether. R/W access appears > to be implied. Arm is not exempt of odd interfaces. However, from just this patch, it is not clear why RISC-V would continue to use the same appropach if the attributes doesn't exist. Looking at the rest of the series, it seems to be because we wanted to have a generic page-table code. If there are a desire to continue towards this direction, then I would assume we would need a arch specific way to set the read/write bit. At which point it makes a lot more sense to push setting the access bits in mfn_to_xen_entry(). Even if we don't do any code consolidation, I think it is odd for Arm that a caller will assume mfn_to_xen_entry() will always return a read-writable page and update as necessary. It would be nicer to push this decision to mfn_to_xen_entry(). Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-21 8:51 ` Julien Grall 2024-07-22 12:58 ` Jan Beulich @ 2024-07-22 14:40 ` Oleksii 1 sibling, 0 replies; 51+ messages in thread From: Oleksii @ 2024-07-22 14:40 UTC (permalink / raw) To: Julien Grall, xen-devel Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Stefano Stabellini Hi Julien, On Sun, 2024-07-21 at 09:51 +0100, Julien Grall wrote: > Hi Oleksii, > > On 12/07/2024 17:22, Oleksii Kurochko wrote: > > Introduces arch_pmap_{un}map functions and select HAS_PMAP > > for CONFIG_RISCV. > > > > Additionaly it was necessary to introduce functions: > > - mfn_to_xen_entry > > - mfn_to_pte > > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > --- > > Changes in V2: > > - newly introduced patch > > --- > > xen/arch/riscv/Kconfig | 1 + > > xen/arch/riscv/include/asm/page.h | 2 ++ > > xen/arch/riscv/include/asm/pmap.h | 28 > > ++++++++++++++++++++++++++++ > > xen/arch/riscv/mm.c | 14 ++++++++++++++ > > 4 files changed, 45 insertions(+) > > create mode 100644 xen/arch/riscv/include/asm/pmap.h > > > > diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig > > index 259eea8d3b..0112aa8778 100644 > > --- a/xen/arch/riscv/Kconfig > > +++ b/xen/arch/riscv/Kconfig > > @@ -3,6 +3,7 @@ config RISCV > > select FUNCTION_ALIGNMENT_16B > > select GENERIC_BUG_FRAME > > select HAS_DEVICE_TREE > > + select HAS_PMAP > > > > config RISCV_64 > > def_bool y > > diff --git a/xen/arch/riscv/include/asm/page.h > > b/xen/arch/riscv/include/asm/page.h > > index cbbf3656d1..339074d502 100644 > > --- a/xen/arch/riscv/include/asm/page.h > > +++ b/xen/arch/riscv/include/asm/page.h > > @@ -51,6 +51,8 @@ typedef struct { > > #endif > > } pte_t; > > > > +pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr); > > + > > static inline pte_t paddr_to_pte(paddr_t paddr, > > unsigned int permissions) > > { > > diff --git a/xen/arch/riscv/include/asm/pmap.h > > b/xen/arch/riscv/include/asm/pmap.h > > new file mode 100644 > > index 0000000000..eb4c48515c > > --- /dev/null > > +++ b/xen/arch/riscv/include/asm/pmap.h > > @@ -0,0 +1,28 @@ > > +#ifndef __ASM_PMAP_H__ > > +#define __ASM_PMAP_H__ > > + > > +#include <xen/bug.h> > > +#include <xen/mm.h> > > + > > +#include <asm/fixmap.h> > > + > > +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > > +{ > > + pte_t *entry = &xen_fixmap[slot]; > > + pte_t pte; > > + > > + ASSERT(!pte_is_valid(*entry)); > > + > > + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > > + pte.pte |= PTE_LEAF_DEFAULT; > > + write_pte(entry, pte); > > +} > > + > > +static inline void arch_pmap_unmap(unsigned int slot) > > +{ > > + pte_t pte = {}; > > + > > + write_pte(&xen_fixmap[slot], pte); > > +} > > + > > +#endif /* __ASM_PMAP_H__ */ > > diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c > > index d69a174b5d..445319af08 100644 > > --- a/xen/arch/riscv/mm.c > > +++ b/xen/arch/riscv/mm.c > > @@ -370,3 +370,17 @@ int map_pages_to_xen(unsigned long virt, > > BUG_ON("unimplemented"); > > return -1; > > } > > + > > +static inline pte_t mfn_to_pte(mfn_t mfn) > > +{ > > + unsigned long pte = mfn_x(mfn) << PTE_PPN_SHIFT; > > + return (pte_t){ .pte = pte}; > > +} > > + > > +inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) > > +{ > > + /* there is no attr field in RISC-V's pte */ > > + (void) attr; > > Surely you have a way to say indicate whether an entry is > readable/writable? Sure, there is a way. But probably I misinterpreted attr for Arm and decided not to use them here. By that I mean, that Arm has MT_NORMAL, MT_DEVICE which RISC-V doesn't have. If it is about readable/writable then for sure, I will start to use attr. ~ Oleksii > > > + > > + return mfn_to_pte(mfn); > > +} > > Cheers, > ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-12 16:22 ` [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header Oleksii Kurochko 2024-07-21 8:51 ` Julien Grall @ 2024-07-22 12:54 ` Jan Beulich 2024-07-22 14:44 ` Oleksii Kurochko 1 sibling, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-22 12:54 UTC (permalink / raw) To: Oleksii Kurochko Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 12.07.2024 18:22, Oleksii Kurochko wrote: > --- /dev/null > +++ b/xen/arch/riscv/include/asm/pmap.h > @@ -0,0 +1,28 @@ > +#ifndef __ASM_PMAP_H__ > +#define __ASM_PMAP_H__ > + > +#include <xen/bug.h> > +#include <xen/mm.h> > + > +#include <asm/fixmap.h> > + > +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > +{ > + pte_t *entry = &xen_fixmap[slot]; > + pte_t pte; > + > + ASSERT(!pte_is_valid(*entry)); > + > + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > + pte.pte |= PTE_LEAF_DEFAULT; > + write_pte(entry, pte); > +} > + > +static inline void arch_pmap_unmap(unsigned int slot) > +{ > + pte_t pte = {}; > + > + write_pte(&xen_fixmap[slot], pte); > +} Why are these not using set_fixmap() / clear_fixmap() respectively? > --- a/xen/arch/riscv/mm.c > +++ b/xen/arch/riscv/mm.c > @@ -370,3 +370,17 @@ int map_pages_to_xen(unsigned long virt, > BUG_ON("unimplemented"); > return -1; > } > + > +static inline pte_t mfn_to_pte(mfn_t mfn) This name suggests (to me) that you're getting _the_ (single) PTE for a given MFN. However, what the function is doing is make a PTE using the given MFN. On x86 at least the common way to name such a function would be pte_from_mfn(). > +{ > + unsigned long pte = mfn_x(mfn) << PTE_PPN_SHIFT; > + return (pte_t){ .pte = pte}; Nit: Blank missing. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-22 12:54 ` Jan Beulich @ 2024-07-22 14:44 ` Oleksii Kurochko 2024-07-22 14:48 ` Julien Grall 0 siblings, 1 reply; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-22 14:44 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On Mon, 2024-07-22 at 14:54 +0200, Jan Beulich wrote: > On 12.07.2024 18:22, Oleksii Kurochko wrote: > > --- /dev/null > > +++ b/xen/arch/riscv/include/asm/pmap.h > > @@ -0,0 +1,28 @@ > > +#ifndef __ASM_PMAP_H__ > > +#define __ASM_PMAP_H__ > > + > > +#include <xen/bug.h> > > +#include <xen/mm.h> > > + > > +#include <asm/fixmap.h> > > + > > +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > > +{ > > + pte_t *entry = &xen_fixmap[slot]; > > + pte_t pte; > > + > > + ASSERT(!pte_is_valid(*entry)); > > + > > + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > > + pte.pte |= PTE_LEAF_DEFAULT; > > + write_pte(entry, pte); > > +} > > + > > +static inline void arch_pmap_unmap(unsigned int slot) > > +{ > > + pte_t pte = {}; > > + > > + write_pte(&xen_fixmap[slot], pte); > > +} > > Why are these not using set_fixmap() / clear_fixmap() respectively? They haven't been introduced yet. And I thought that these fucntion are used only in pmap_{un}map() and that is the reason why I decided to not introduce them. But while writing the answer on another comment, I found other places where set_fixmap() / clear_fixmap() are used, so I will introduce them and reuse here. > > > --- a/xen/arch/riscv/mm.c > > +++ b/xen/arch/riscv/mm.c > > @@ -370,3 +370,17 @@ int map_pages_to_xen(unsigned long virt, > > BUG_ON("unimplemented"); > > return -1; > > } > > + > > +static inline pte_t mfn_to_pte(mfn_t mfn) > > This name suggests (to me) that you're getting _the_ (single) PTE for > a given MFN. However, what the function is doing is make a PTE using > the given MFN. On x86 at least the common way to name such a function > would be pte_from_mfn(). If it is a common way then I will rename it. Thanks. ~ Oleksii > > > +{ > > + unsigned long pte = mfn_x(mfn) << PTE_PPN_SHIFT; > > + return (pte_t){ .pte = pte}; > > Nit: Blank missing. > > Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-22 14:44 ` Oleksii Kurochko @ 2024-07-22 14:48 ` Julien Grall 2024-07-22 17:09 ` Oleksii Kurochko 0 siblings, 1 reply; 51+ messages in thread From: Julien Grall @ 2024-07-22 14:48 UTC (permalink / raw) To: Oleksii Kurochko, Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel Hi, On 22/07/2024 15:44, Oleksii Kurochko wrote: > On Mon, 2024-07-22 at 14:54 +0200, Jan Beulich wrote: >> On 12.07.2024 18:22, Oleksii Kurochko wrote: >>> --- /dev/null >>> +++ b/xen/arch/riscv/include/asm/pmap.h >>> @@ -0,0 +1,28 @@ >>> +#ifndef __ASM_PMAP_H__ >>> +#define __ASM_PMAP_H__ >>> + >>> +#include <xen/bug.h> >>> +#include <xen/mm.h> >>> + >>> +#include <asm/fixmap.h> >>> + >>> +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) >>> +{ >>> + pte_t *entry = &xen_fixmap[slot]; >>> + pte_t pte; >>> + >>> + ASSERT(!pte_is_valid(*entry)); >>> + >>> + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); >>> + pte.pte |= PTE_LEAF_DEFAULT; >>> + write_pte(entry, pte); >>> +} >>> + >>> +static inline void arch_pmap_unmap(unsigned int slot) >>> +{ >>> + pte_t pte = {}; >>> + >>> + write_pte(&xen_fixmap[slot], pte); >>> +} >> >> Why are these not using set_fixmap() / clear_fixmap() respectively? > They haven't been introduced yet. And I thought that these fucntion are > used only in pmap_{un}map() and that is the reason why I decided to not > introduce them. But while writing the answer on another comment, I > found other places where set_fixmap() / clear_fixmap() are used, so I > will introduce them and reuse here. I am guessing you are going to implement set_fixmap()/clear_fixmap() using map_pages_to_xen(). If so, for early boot you are going to end up in a circular loop because map_pages_to_xen() will likely use pmap() which will call set_fixmap(). There is a big comment in common/pmap.c which explain why arch_pmap_* was introduced rather than calling *_fixmap() directly: /* * We cannot use set_fixmap() here. We use PMAP when the domain map * page infrastructure is not yet initialized, so map_pages_to_xen() called * by set_fixmap() needs to map pages on demand, which then calls pmap() * again, resulting in a loop. Modify the PTEs directly instead. The same * is true for pmap_unmap(). */ Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-22 14:48 ` Julien Grall @ 2024-07-22 17:09 ` Oleksii Kurochko 2024-07-22 17:21 ` Julien Grall 0 siblings, 1 reply; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-22 17:09 UTC (permalink / raw) To: Julien Grall, Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel On Mon, 2024-07-22 at 15:48 +0100, Julien Grall wrote: > Hi, > > On 22/07/2024 15:44, Oleksii Kurochko wrote: > > On Mon, 2024-07-22 at 14:54 +0200, Jan Beulich wrote: > > > On 12.07.2024 18:22, Oleksii Kurochko wrote: > > > > --- /dev/null > > > > +++ b/xen/arch/riscv/include/asm/pmap.h > > > > @@ -0,0 +1,28 @@ > > > > +#ifndef __ASM_PMAP_H__ > > > > +#define __ASM_PMAP_H__ > > > > + > > > > +#include <xen/bug.h> > > > > +#include <xen/mm.h> > > > > + > > > > +#include <asm/fixmap.h> > > > > + > > > > +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > > > > +{ > > > > + pte_t *entry = &xen_fixmap[slot]; > > > > + pte_t pte; > > > > + > > > > + ASSERT(!pte_is_valid(*entry)); > > > > + > > > > + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > > > > + pte.pte |= PTE_LEAF_DEFAULT; > > > > + write_pte(entry, pte); > > > > +} > > > > + > > > > +static inline void arch_pmap_unmap(unsigned int slot) > > > > +{ > > > > + pte_t pte = {}; > > > > + > > > > + write_pte(&xen_fixmap[slot], pte); > > > > +} > > > > > > Why are these not using set_fixmap() / clear_fixmap() > > > respectively? > > They haven't been introduced yet. And I thought that these fucntion > > are > > used only in pmap_{un}map() and that is the reason why I decided to > > not > > introduce them. But while writing the answer on another comment, I > > found other places where set_fixmap() / clear_fixmap() are used, so > > I > > will introduce them and reuse here. > > I am guessing you are going to implement set_fixmap()/clear_fixmap() > using map_pages_to_xen(). If so, for early boot you are going to end > up > in a circular loop because map_pages_to_xen() will likely use pmap() > which will call set_fixmap(). I am going to implement that in the following way as I faced the described by you issue when I first time tried to implement it using map_pages_to_xen(): /* Map a 4k page in a fixmap entry */ void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags) { pte_t pte; pte = mfn_to_xen_entry(mfn, flags); pte.pte |= PTE_LEAF_DEFAULT; write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); } /* Remove a mapping from a fixmap entry */ void clear_fixmap(unsigned map) { pte_t pte = {0}; write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); } ~ Oleksii > > There is a big comment in common/pmap.c which explain why arch_pmap_* > was introduced rather than calling *_fixmap() directly: > > /* > * We cannot use set_fixmap() here. We use PMAP when the domain > map > * page infrastructure is not yet initialized, so > map_pages_to_xen() called > * by set_fixmap() needs to map pages on demand, which then > calls > pmap() > * again, resulting in a loop. Modify the PTEs directly instead. > The same > * is true for pmap_unmap(). > */ > > Cheers, > ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-22 17:09 ` Oleksii Kurochko @ 2024-07-22 17:21 ` Julien Grall 2024-07-23 8:02 ` Oleksii Kurochko 0 siblings, 1 reply; 51+ messages in thread From: Julien Grall @ 2024-07-22 17:21 UTC (permalink / raw) To: Oleksii Kurochko, Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel Hi, On 22/07/2024 18:09, Oleksii Kurochko wrote: > On Mon, 2024-07-22 at 15:48 +0100, Julien Grall wrote: >> Hi, >> >> On 22/07/2024 15:44, Oleksii Kurochko wrote: >>> On Mon, 2024-07-22 at 14:54 +0200, Jan Beulich wrote: >>>> On 12.07.2024 18:22, Oleksii Kurochko wrote: >>>>> --- /dev/null >>>>> +++ b/xen/arch/riscv/include/asm/pmap.h >>>>> @@ -0,0 +1,28 @@ >>>>> +#ifndef __ASM_PMAP_H__ >>>>> +#define __ASM_PMAP_H__ >>>>> + >>>>> +#include <xen/bug.h> >>>>> +#include <xen/mm.h> >>>>> + >>>>> +#include <asm/fixmap.h> >>>>> + >>>>> +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) >>>>> +{ >>>>> + pte_t *entry = &xen_fixmap[slot]; >>>>> + pte_t pte; >>>>> + >>>>> + ASSERT(!pte_is_valid(*entry)); >>>>> + >>>>> + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); >>>>> + pte.pte |= PTE_LEAF_DEFAULT; >>>>> + write_pte(entry, pte); >>>>> +} >>>>> + >>>>> +static inline void arch_pmap_unmap(unsigned int slot) >>>>> +{ >>>>> + pte_t pte = {}; >>>>> + >>>>> + write_pte(&xen_fixmap[slot], pte); >>>>> +} >>>> >>>> Why are these not using set_fixmap() / clear_fixmap() >>>> respectively? >>> They haven't been introduced yet. And I thought that these fucntion >>> are >>> used only in pmap_{un}map() and that is the reason why I decided to >>> not >>> introduce them. But while writing the answer on another comment, I >>> found other places where set_fixmap() / clear_fixmap() are used, so >>> I >>> will introduce them and reuse here. >> >> I am guessing you are going to implement set_fixmap()/clear_fixmap() >> using map_pages_to_xen(). If so, for early boot you are going to end >> up >> in a circular loop because map_pages_to_xen() will likely use pmap() >> which will call set_fixmap(). > I am going to implement that in the following way as I faced the > described by you issue when I first time tried to implement it using > map_pages_to_xen(): What's wrong with keeping the arch_pmap_*() as-is and call map_pages_to_xen() in the fixmap? At least this would be consistent with what other architectures does and less risky (see below). > /* Map a 4k page in a fixmap entry */ > void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags) > { > pte_t pte; > > pte = mfn_to_xen_entry(mfn, flags); > pte.pte |= PTE_LEAF_DEFAULT; > write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); It would be saner to check if you are not overwriting any existing mapping as otherwise you will probably need a TLB flush. > } > > /* Remove a mapping from a fixmap entry */ > void clear_fixmap(unsigned map) > { > pte_t pte = {0}; > write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); Don't you need a TLB flush? Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-22 17:21 ` Julien Grall @ 2024-07-23 8:02 ` Oleksii Kurochko 2024-07-23 8:36 ` Jan Beulich 0 siblings, 1 reply; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-23 8:02 UTC (permalink / raw) To: Julien Grall Cc: Oleksii Kurochko, Jan Beulich, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel [-- Attachment #1: Type: text/plain, Size: 3448 bytes --] Hi Julien, On Mon, Jul 22, 2024 at 7:27 PM Julien Grall <julien@xen.org> wrote: > Hi, > > >> On 22/07/2024 15:44, Oleksii Kurochko wrote: > >>> On Mon, 2024-07-22 at 14:54 +0200, Jan Beulich wrote: > >>>> On 12.07.2024 18:22, Oleksii Kurochko wrote: > >>>>> --- /dev/null > >>>>> +++ b/xen/arch/riscv/include/asm/pmap.h > >>>>> @@ -0,0 +1,28 @@ > >>>>> +#ifndef __ASM_PMAP_H__ > >>>>> +#define __ASM_PMAP_H__ > >>>>> + > >>>>> +#include <xen/bug.h> > >>>>> +#include <xen/mm.h> > >>>>> + > >>>>> +#include <asm/fixmap.h> > >>>>> + > >>>>> +static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > >>>>> +{ > >>>>> + pte_t *entry = &xen_fixmap[slot]; > >>>>> + pte_t pte; > >>>>> + > >>>>> + ASSERT(!pte_is_valid(*entry)); > >>>>> + > >>>>> + pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > >>>>> + pte.pte |= PTE_LEAF_DEFAULT; > >>>>> + write_pte(entry, pte); > >>>>> +} > >>>>> + > >>>>> +static inline void arch_pmap_unmap(unsigned int slot) > >>>>> +{ > >>>>> + pte_t pte = {}; > >>>>> + > >>>>> + write_pte(&xen_fixmap[slot], pte); > >>>>> +} > >>>> > >>>> Why are these not using set_fixmap() / clear_fixmap() > >>>> respectively? > >>> They haven't been introduced yet. And I thought that these fucntion > >>> are > >>> used only in pmap_{un}map() and that is the reason why I decided to > >>> not > >>> introduce them. But while writing the answer on another comment, I > >>> found other places where set_fixmap() / clear_fixmap() are used, so > >>> I > >>> will introduce them and reuse here. > >> > >> I am guessing you are going to implement set_fixmap()/clear_fixmap() > >> using map_pages_to_xen(). If so, for early boot you are going to end > >> up > >> in a circular loop because map_pages_to_xen() will likely use pmap() > >> which will call set_fixmap(). > > I am going to implement that in the following way as I faced the > > described by you issue when I first time tried to implement it using > > map_pages_to_xen(): > What's wrong with keeping the arch_pmap_*() as-is and call > map_pages_to_xen() in the fixmap? > > At least this would be consistent with what other architectures does and > less risky (see below). > Then I misunderstood you, if not to use {set/clear}_fixmap() in arch_pmap() then everything should be fine. Then I think it is needed to add the comment also above arch_pmap_*() function why it isn't used {set/clear}_fixmap() inside. ( or update the commit message ) > > > /* Map a 4k page in a fixmap entry */ > > void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags) > > { > > pte_t pte; > > > > pte = mfn_to_xen_entry(mfn, flags); > > pte.pte |= PTE_LEAF_DEFAULT; > > write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); > > It would be saner to check if you are not overwriting any existing > mapping as otherwise you will probably need a TLB flush. > > > } > > > > /* Remove a mapping from a fixmap entry */ > > void clear_fixmap(unsigned map) > > { > > pte_t pte = {0}; > > write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); > > Don't you need a TLB flush? > Inside write_pte() there is "sfence.vma". But probably it would be better to add flush_xen_tlb_range_va_local() or something similar here in case if someone will decide to update write_pte(). ~ Oleksii [-- Attachment #2: Type: text/html, Size: 4935 bytes --] ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 8:02 ` Oleksii Kurochko @ 2024-07-23 8:36 ` Jan Beulich 2024-07-23 8:55 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-23 8:36 UTC (permalink / raw) To: Oleksii Kurochko Cc: Oleksii Kurochko, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel, Julien Grall On 23.07.2024 10:02, Oleksii Kurochko wrote: > On Mon, Jul 22, 2024 at 7:27 PM Julien Grall <julien@xen.org> wrote: >>>> On 22/07/2024 15:44, Oleksii Kurochko wrote: >>> /* Map a 4k page in a fixmap entry */ >>> void set_fixmap(unsigned map, mfn_t mfn, unsigned int flags) >>> { >>> pte_t pte; >>> >>> pte = mfn_to_xen_entry(mfn, flags); >>> pte.pte |= PTE_LEAF_DEFAULT; >>> write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); >> >> It would be saner to check if you are not overwriting any existing >> mapping as otherwise you will probably need a TLB flush. >> >>> } >>> >>> /* Remove a mapping from a fixmap entry */ >>> void clear_fixmap(unsigned map) >>> { >>> pte_t pte = {0}; >>> write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], pte); >> >> Don't you need a TLB flush? >> > Inside write_pte() there is "sfence.vma". That's just a fence though, not a TLB flush. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 8:36 ` Jan Beulich @ 2024-07-23 8:55 ` oleksii.kurochko 2024-07-23 10:02 ` Jan Beulich 0 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 8:55 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel, Julien Grall On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: > On 23.07.2024 10:02, Oleksii Kurochko wrote: > > On Mon, Jul 22, 2024 at 7:27 PM Julien Grall <julien@xen.org> > > wrote: > > > > > On 22/07/2024 15:44, Oleksii Kurochko wrote: > > > > /* Map a 4k page in a fixmap entry */ > > > > void set_fixmap(unsigned map, mfn_t mfn, unsigned int > > > > flags) > > > > { > > > > pte_t pte; > > > > > > > > pte = mfn_to_xen_entry(mfn, flags); > > > > pte.pte |= PTE_LEAF_DEFAULT; > > > > write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], > > > > pte); > > > > > > It would be saner to check if you are not overwriting any > > > existing > > > mapping as otherwise you will probably need a TLB flush. > > > > > > > } > > > > > > > > /* Remove a mapping from a fixmap entry */ > > > > void clear_fixmap(unsigned map) > > > > { > > > > pte_t pte = {0}; > > > > write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], > > > > pte); > > > > > > Don't you need a TLB flush? > > > > > Inside write_pte() there is "sfence.vma". > > That's just a fence though, not a TLB flush. From the privileged doc: ``` SFENCE.VMA is also used to invalidate entries in the address-translation cache associated with a hart (see Section 4.3.2). ... The SFENCE.VMA is used to flush any local hardware caches related to address translation. It is specified as a fence rather than a TLB flush to provide cleaner semantics with respect to which instructions are affected by the flush operation and to support a wider variety of dynamic caching structures and memory-management schemes. SFENCE.VMA is also used by higher privilege levels to synchronize page table writes and the address translation hardware. ... ``` I read this as SFENCE.VMA is used not only for ordering of load/stores, but also to flush TLB ( which is a type of more general term as address-translation cache, IIUIC ). Also, Linux kernel uses sfence.vma to flush TLB: https://elixir.bootlin.com/linux/v6.0/source/arch/riscv/include/asm/tlbflush.h#L23 ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 8:55 ` oleksii.kurochko @ 2024-07-23 10:02 ` Jan Beulich 2024-07-23 15:36 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-23 10:02 UTC (permalink / raw) To: oleksii.kurochko Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel, Julien Grall On 23.07.2024 10:55, oleksii.kurochko@gmail.com wrote: > On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: >> On 23.07.2024 10:02, Oleksii Kurochko wrote: >>> On Mon, Jul 22, 2024 at 7:27 PM Julien Grall <julien@xen.org> >>> wrote: >>>>>> On 22/07/2024 15:44, Oleksii Kurochko wrote: >>>>> /* Map a 4k page in a fixmap entry */ >>>>> void set_fixmap(unsigned map, mfn_t mfn, unsigned int >>>>> flags) >>>>> { >>>>> pte_t pte; >>>>> >>>>> pte = mfn_to_xen_entry(mfn, flags); >>>>> pte.pte |= PTE_LEAF_DEFAULT; >>>>> write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], >>>>> pte); >>>> >>>> It would be saner to check if you are not overwriting any >>>> existing >>>> mapping as otherwise you will probably need a TLB flush. >>>> >>>>> } >>>>> >>>>> /* Remove a mapping from a fixmap entry */ >>>>> void clear_fixmap(unsigned map) >>>>> { >>>>> pte_t pte = {0}; >>>>> write_pte(&xen_fixmap[pt_index(0, FIXMAP_ADDR(map))], >>>>> pte); >>>> >>>> Don't you need a TLB flush? >>>> >>> Inside write_pte() there is "sfence.vma". >> >> That's just a fence though, not a TLB flush. > From the privileged doc: > ``` > SFENCE.VMA is also used to invalidate entries in the > address-translation cache associated with a hart (see Section 4.3.2). > ... > The SFENCE.VMA is used to flush any local hardware caches related to > address translation. > It is specified as a fence rather than a TLB flush to provide cleaner > semantics with respect to > which instructions are affected by the flush operation and to support a > wider variety of dynamic > caching structures and memory-management schemes. SFENCE.VMA is also > used by higher > privilege levels to synchronize page table writes and the address > translation hardware. > ... > ``` > I read this as SFENCE.VMA is used not only for ordering of load/stores, > but also to flush TLB ( which is a type of more general term as > address-translation cache, IIUIC ). Oh, I see. Kind of unexpected for an instruction of that name. Yet note how they talk about the local hart only. You need a wider scope TLB flush here. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 10:02 ` Jan Beulich @ 2024-07-23 15:36 ` oleksii.kurochko 2024-07-23 15:49 ` Julien Grall 0 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 15:36 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel, Julien Grall On Tue, 2024-07-23 at 12:02 +0200, Jan Beulich wrote: > On 23.07.2024 10:55, oleksii.kurochko@gmail.com wrote: > > On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: > > > On 23.07.2024 10:02, Oleksii Kurochko wrote: > > > > On Mon, Jul 22, 2024 at 7:27 PM Julien Grall <julien@xen.org> > > > > wrote: > > > > > > > On 22/07/2024 15:44, Oleksii Kurochko wrote: > > > > > > /* Map a 4k page in a fixmap entry */ > > > > > > void set_fixmap(unsigned map, mfn_t mfn, unsigned int > > > > > > flags) > > > > > > { > > > > > > pte_t pte; > > > > > > > > > > > > pte = mfn_to_xen_entry(mfn, flags); > > > > > > pte.pte |= PTE_LEAF_DEFAULT; > > > > > > write_pte(&xen_fixmap[pt_index(0, > > > > > > FIXMAP_ADDR(map))], > > > > > > pte); > > > > > > > > > > It would be saner to check if you are not overwriting any > > > > > existing > > > > > mapping as otherwise you will probably need a TLB flush. > > > > > > > > > > > } > > > > > > > > > > > > /* Remove a mapping from a fixmap entry */ > > > > > > void clear_fixmap(unsigned map) > > > > > > { > > > > > > pte_t pte = {0}; > > > > > > write_pte(&xen_fixmap[pt_index(0, > > > > > > FIXMAP_ADDR(map))], > > > > > > pte); > > > > > > > > > > Don't you need a TLB flush? > > > > > > > > > Inside write_pte() there is "sfence.vma". > > > > > > That's just a fence though, not a TLB flush. > > From the privileged doc: > > ``` > > SFENCE.VMA is also used to invalidate entries in the > > address-translation cache associated with a hart (see Section > > 4.3.2). > > ... > > The SFENCE.VMA is used to flush any local hardware caches > > related to > > address translation. > > It is specified as a fence rather than a TLB flush to provide > > cleaner > > semantics with respect to > > which instructions are affected by the flush operation and to > > support a > > wider variety of dynamic > > caching structures and memory-management schemes. SFENCE.VMA is > > also > > used by higher > > privilege levels to synchronize page table writes and the > > address > > translation hardware. > > ... > > ``` > > I read this as SFENCE.VMA is used not only for ordering of > > load/stores, > > but also to flush TLB ( which is a type of more general term as > > address-translation cache, IIUIC ). > > Oh, I see. Kind of unexpected for an instruction of that name. Yet > note > how they talk about the local hart only. You need a wider scope TLB > flush here. Could you please clarify why it is needed wider? Arm Xen flushed only local TLB. RISC-V Linux kernel for fixmap also uses: local_flush_tlb_page(). ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 15:36 ` oleksii.kurochko @ 2024-07-23 15:49 ` Julien Grall 2024-07-23 17:25 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: Julien Grall @ 2024-07-23 15:49 UTC (permalink / raw) To: oleksii.kurochko, Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel Hi Oleksii, On 23/07/2024 16:36, oleksii.kurochko@gmail.com wrote: > On Tue, 2024-07-23 at 12:02 +0200, Jan Beulich wrote: >> On 23.07.2024 10:55, oleksii.kurochko@gmail.com wrote: >>> On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: >>>> On 23.07.2024 10:02, Oleksii Kurochko wrote: >>>>> On Mon, Jul 22, 2024 at 7:27 PM Julien Grall <julien@xen.org> >>>>> wrote: >>>>>>>> On 22/07/2024 15:44, Oleksii Kurochko wrote: >>>>>>> /* Map a 4k page in a fixmap entry */ >>>>>>> void set_fixmap(unsigned map, mfn_t mfn, unsigned int >>>>>>> flags) >>>>>>> { >>>>>>> pte_t pte; >>>>>>> >>>>>>> pte = mfn_to_xen_entry(mfn, flags); >>>>>>> pte.pte |= PTE_LEAF_DEFAULT; >>>>>>> write_pte(&xen_fixmap[pt_index(0, >>>>>>> FIXMAP_ADDR(map))], >>>>>>> pte); >>>>>> >>>>>> It would be saner to check if you are not overwriting any >>>>>> existing >>>>>> mapping as otherwise you will probably need a TLB flush. >>>>>> >>>>>>> } >>>>>>> >>>>>>> /* Remove a mapping from a fixmap entry */ >>>>>>> void clear_fixmap(unsigned map) >>>>>>> { >>>>>>> pte_t pte = {0}; >>>>>>> write_pte(&xen_fixmap[pt_index(0, >>>>>>> FIXMAP_ADDR(map))], >>>>>>> pte); >>>>>> >>>>>> Don't you need a TLB flush? >>>>>> >>>>> Inside write_pte() there is "sfence.vma". >>>> >>>> That's just a fence though, not a TLB flush. >>> From the privileged doc: >>> ``` >>> SFENCE.VMA is also used to invalidate entries in the >>> address-translation cache associated with a hart (see Section >>> 4.3.2). >>> ... >>> The SFENCE.VMA is used to flush any local hardware caches >>> related to >>> address translation. >>> It is specified as a fence rather than a TLB flush to provide >>> cleaner >>> semantics with respect to >>> which instructions are affected by the flush operation and to >>> support a >>> wider variety of dynamic >>> caching structures and memory-management schemes. SFENCE.VMA is >>> also >>> used by higher >>> privilege levels to synchronize page table writes and the >>> address >>> translation hardware. >>> ... >>> ``` >>> I read this as SFENCE.VMA is used not only for ordering of >>> load/stores, >>> but also to flush TLB ( which is a type of more general term as >>> address-translation cache, IIUIC ). I have to admit, I am a little because concerned with calling sfence.vma in write_pte() (this may only be because I am not very familiar with RISC-V). We have cases where multiple entry will be written in a single map_pages_to_xen() call. So wouldn't this means that the local TLBs would be nuked for every write rather than once? >> >> Oh, I see. Kind of unexpected for an instruction of that name. Yet >> note >> how they talk about the local hart only. You need a wider scope TLB >> flush here. > Could you please clarify why it is needed wider? > > Arm Xen flushed only local TLB. Which code are you looking at? set_fixmap() will propagate the TLB flush to all innershareable CPUs. The PMAP interface will do a local TLB flush because the interface can only be used during early boot where there is a single CPU running. > RISC-V Linux kernel for fixmap also uses: local_flush_tlb_page(). I don't know how Linux is using set_fixmap(). But what matters is how Xen is using set_fixmap(). We have a couple of places in Xen where the fixmap needs to be accessed by all the CPUs. Given this is a common interface in Xen, I think it makes sense to follow the same approach to avoid any confusion. Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 15:49 ` Julien Grall @ 2024-07-23 17:25 ` oleksii.kurochko 2024-07-23 17:28 ` oleksii.kurochko 0 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 17:25 UTC (permalink / raw) To: Julien Grall, Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel On Tue, 2024-07-23 at 16:49 +0100, Julien Grall wrote: > Hi Oleksii, > > On 23/07/2024 16:36, oleksii.kurochko@gmail.com wrote: > > On Tue, 2024-07-23 at 12:02 +0200, Jan Beulich wrote: > > > On 23.07.2024 10:55, oleksii.kurochko@gmail.com wrote: > > > > On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: > > > > > On 23.07.2024 10:02, Oleksii Kurochko wrote: > > > > > > On Mon, Jul 22, 2024 at 7:27 PM Julien Grall > > > > > > <julien@xen.org> > > > > > > wrote: > > > > > > > > > On 22/07/2024 15:44, Oleksii Kurochko wrote: > > > > > > > > /* Map a 4k page in a fixmap entry */ > > > > > > > > void set_fixmap(unsigned map, mfn_t mfn, unsigned > > > > > > > > int > > > > > > > > flags) > > > > > > > > { > > > > > > > > pte_t pte; > > > > > > > > > > > > > > > > pte = mfn_to_xen_entry(mfn, flags); > > > > > > > > pte.pte |= PTE_LEAF_DEFAULT; > > > > > > > > write_pte(&xen_fixmap[pt_index(0, > > > > > > > > FIXMAP_ADDR(map))], > > > > > > > > pte); > > > > > > > > > > > > > > It would be saner to check if you are not overwriting any > > > > > > > existing > > > > > > > mapping as otherwise you will probably need a TLB flush. > > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > /* Remove a mapping from a fixmap entry */ > > > > > > > > void clear_fixmap(unsigned map) > > > > > > > > { > > > > > > > > pte_t pte = {0}; > > > > > > > > write_pte(&xen_fixmap[pt_index(0, > > > > > > > > FIXMAP_ADDR(map))], > > > > > > > > pte); > > > > > > > > > > > > > > Don't you need a TLB flush? > > > > > > > > > > > > > Inside write_pte() there is "sfence.vma". > > > > > > > > > > That's just a fence though, not a TLB flush. > > > > From the privileged doc: > > > > ``` > > > > SFENCE.VMA is also used to invalidate entries in the > > > > address-translation cache associated with a hart (see > > > > Section > > > > 4.3.2). > > > > ... > > > > The SFENCE.VMA is used to flush any local hardware caches > > > > related to > > > > address translation. > > > > It is specified as a fence rather than a TLB flush to > > > > provide > > > > cleaner > > > > semantics with respect to > > > > which instructions are affected by the flush operation and > > > > to > > > > support a > > > > wider variety of dynamic > > > > caching structures and memory-management schemes. > > > > SFENCE.VMA is > > > > also > > > > used by higher > > > > privilege levels to synchronize page table writes and the > > > > address > > > > translation hardware. > > > > ... > > > > ``` > > > > I read this as SFENCE.VMA is used not only for ordering of > > > > load/stores, > > > > but also to flush TLB ( which is a type of more general term as > > > > address-translation cache, IIUIC ). > I have to admit, I am a little because concerned with calling > sfence.vma > in write_pte() (this may only be because I am not very familiar with > RISC-V). > > We have cases where multiple entry will be written in a single > map_pages_to_xen() call. So wouldn't this means that the local TLBs > would be nuked for every write rather than once? Yes, it will be nuked. It is bad from perfomance point of view. I just wanted to be sure that I won't miss to put sfence.vma when it is necessary and then reworked that a little bit after. But it seems it would be better not to call sfence.vma in write_pte() just from the start. > > > > > > > > Oh, I see. Kind of unexpected for an instruction of that name. > > > Yet > > > note > > > how they talk about the local hart only. You need a wider scope > > > TLB > > > flush here. > > Could you please clarify why it is needed wider? > > > > Arm Xen flushed only local TLB. > > Which code are you looking at? set_fixmap() will propagate the TLB > flush > to all innershareable CPUs. Yes, here I agree that set_fixmap() uses map_pages_to_xen which somewhere inside uses flush_xen_tlb_range_va() ( not flush_xen_tlb_range_va() ) so TLB flush will happen for all innershareable CPUs. > > The PMAP interface will do a local TLB flush because the interface > can > only be used during early boot where there is a single CPU running. Yes, I am looking at PMAP: static inline void arch_pmap_unmap(unsigned int slot) { lpae_t pte = {}; write_pte(&xen_fixmap[slot], pte); flush_xen_tlb_range_va(FIXMAP_ADDR(slot), PAGE_SIZE); } IIUC, originaly Jan told about arch_pmap_unmap() case so that is why I decided to clarify additionaly. > > > RISC-V Linux kernel for fixmap also uses: local_flush_tlb_page(). > > I don't know how Linux is using set_fixmap(). But what matters is how > Xen is using set_fixmap(). We have a couple of places in Xen where > the > fixmap needs to be accessed by all the CPUs. > > Given this is a common interface in Xen, I think it makes sense to > follow the same approach to avoid any confusion. Sure. Thanks for claryfying. I will flush_xen_tlb_range_va() for set_fixmap(). ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 17:25 ` oleksii.kurochko @ 2024-07-23 17:28 ` oleksii.kurochko 2024-07-23 18:44 ` Julien Grall 0 siblings, 1 reply; 51+ messages in thread From: oleksii.kurochko @ 2024-07-23 17:28 UTC (permalink / raw) To: Julien Grall Cc: Jan Beulich, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel On Tue, 2024-07-23 at 19:25 +0200, oleksii.kurochko@gmail.com wrote: > On Tue, 2024-07-23 at 16:49 +0100, Julien Grall wrote: > > Hi Oleksii, > > > > On 23/07/2024 16:36, oleksii.kurochko@gmail.com wrote: > > > On Tue, 2024-07-23 at 12:02 +0200, Jan Beulich wrote: > > > > On 23.07.2024 10:55, oleksii.kurochko@gmail.com wrote: > > > > > On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: > > > > > > On 23.07.2024 10:02, Oleksii Kurochko wrote: > > > > > > > On Mon, Jul 22, 2024 at 7:27 PM Julien Grall > > > > > > > <julien@xen.org> > > > > > > > wrote: > > > > > > > > > > On 22/07/2024 15:44, Oleksii Kurochko wrote: > > > > > > > > > /* Map a 4k page in a fixmap entry */ > > > > > > > > > void set_fixmap(unsigned map, mfn_t mfn, > > > > > > > > > unsigned > > > > > > > > > int > > > > > > > > > flags) > > > > > > > > > { > > > > > > > > > pte_t pte; > > > > > > > > > > > > > > > > > > pte = mfn_to_xen_entry(mfn, flags); > > > > > > > > > pte.pte |= PTE_LEAF_DEFAULT; > > > > > > > > > write_pte(&xen_fixmap[pt_index(0, > > > > > > > > > FIXMAP_ADDR(map))], > > > > > > > > > pte); > > > > > > > > > > > > > > > > It would be saner to check if you are not overwriting > > > > > > > > any > > > > > > > > existing > > > > > > > > mapping as otherwise you will probably need a TLB > > > > > > > > flush. > > > > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > /* Remove a mapping from a fixmap entry */ > > > > > > > > > void clear_fixmap(unsigned map) > > > > > > > > > { > > > > > > > > > pte_t pte = {0}; > > > > > > > > > write_pte(&xen_fixmap[pt_index(0, > > > > > > > > > FIXMAP_ADDR(map))], > > > > > > > > > pte); > > > > > > > > > > > > > > > > Don't you need a TLB flush? > > > > > > > > > > > > > > > Inside write_pte() there is "sfence.vma". > > > > > > > > > > > > That's just a fence though, not a TLB flush. > > > > > From the privileged doc: > > > > > ``` > > > > > SFENCE.VMA is also used to invalidate entries in the > > > > > address-translation cache associated with a hart (see > > > > > Section > > > > > 4.3.2). > > > > > ... > > > > > The SFENCE.VMA is used to flush any local hardware caches > > > > > related to > > > > > address translation. > > > > > It is specified as a fence rather than a TLB flush to > > > > > provide > > > > > cleaner > > > > > semantics with respect to > > > > > which instructions are affected by the flush operation > > > > > and > > > > > to > > > > > support a > > > > > wider variety of dynamic > > > > > caching structures and memory-management schemes. > > > > > SFENCE.VMA is > > > > > also > > > > > used by higher > > > > > privilege levels to synchronize page table writes and the > > > > > address > > > > > translation hardware. > > > > > ... > > > > > ``` > > > > > I read this as SFENCE.VMA is used not only for ordering of > > > > > load/stores, > > > > > but also to flush TLB ( which is a type of more general term > > > > > as > > > > > address-translation cache, IIUIC ). > > I have to admit, I am a little because concerned with calling > > sfence.vma > > in write_pte() (this may only be because I am not very familiar > > with > > RISC-V). > > > > We have cases where multiple entry will be written in a single > > map_pages_to_xen() call. So wouldn't this means that the local TLBs > > would be nuked for every write rather than once? > Yes, it will be nuked. It is bad from perfomance point of view. > I just wanted to be sure that I won't miss to put sfence.vma when it > is > necessary and then reworked that a little bit after. But it seems it > would be better not to call sfence.vma in write_pte() just from the > start. > > > > > > > > > > > > > > Oh, I see. Kind of unexpected for an instruction of that name. > > > > Yet > > > > note > > > > how they talk about the local hart only. You need a wider scope > > > > TLB > > > > flush here. > > > Could you please clarify why it is needed wider? > > > > > > Arm Xen flushed only local TLB. > > > > Which code are you looking at? set_fixmap() will propagate the TLB > > flush > > to all innershareable CPUs. > Yes, here I agree that set_fixmap() uses map_pages_to_xen which > somewhere inside uses flush_xen_tlb_range_va() ( not > flush_xen_tlb_range_va() ) so TLB flush will happen for all > innershareable CPUs. > > > > The PMAP interface will do a local TLB flush because the interface > > can > > only be used during early boot where there is a single CPU running. > > Yes, I am looking at PMAP: > static inline void arch_pmap_unmap(unsigned int slot) > { > lpae_t pte = {}; > > write_pte(&xen_fixmap[slot], pte); > > flush_xen_tlb_range_va(FIXMAP_ADDR(slot), PAGE_SIZE); > } > IIUC, originaly Jan told about arch_pmap_unmap() case so that is why > I > decided to clarify additionaly. Julien, I have a questation related to Arm's version of arch_pmap_map(): static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) { lpae_t *entry = &xen_fixmap[slot]; lpae_t pte; ASSERT(!lpae_is_valid(*entry)); pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); pte.pt.table = 1; write_pte(entry, pte); /* * The new entry will be used very soon after arch_pmap_map() returns. * So ensure the DSB in write_pte() has completed before continuing. */ isb(); } Is the code above isb() is correct? is it insure the DSB not ISB? And isn't need to do TLB flush here? ~ Oleksii > > > > > > RISC-V Linux kernel for fixmap also uses: local_flush_tlb_page(). > > > > I don't know how Linux is using set_fixmap(). But what matters is > > how > > Xen is using set_fixmap(). We have a couple of places in Xen where > > the > > fixmap needs to be accessed by all the CPUs. > > > > Given this is a common interface in Xen, I think it makes sense to > > follow the same approach to avoid any confusion. > Sure. Thanks for claryfying. I will flush_xen_tlb_range_va() for > set_fixmap(). > > > ~ Oleksii ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header 2024-07-23 17:28 ` oleksii.kurochko @ 2024-07-23 18:44 ` Julien Grall 0 siblings, 0 replies; 51+ messages in thread From: Julien Grall @ 2024-07-23 18:44 UTC (permalink / raw) To: oleksii.kurochko Cc: Jan Beulich, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Stefano Stabellini, xen-devel On 23/07/2024 18:28, oleksii.kurochko@gmail.com wrote: > On Tue, 2024-07-23 at 19:25 +0200, oleksii.kurochko@gmail.com wrote: >> On Tue, 2024-07-23 at 16:49 +0100, Julien Grall wrote: >>> Hi Oleksii, >>> >>> On 23/07/2024 16:36, oleksii.kurochko@gmail.com wrote: >>>> On Tue, 2024-07-23 at 12:02 +0200, Jan Beulich wrote: >>>>> On 23.07.2024 10:55, oleksii.kurochko@gmail.com wrote: >>>>>> On Tue, 2024-07-23 at 10:36 +0200, Jan Beulich wrote: >>>>>>> On 23.07.2024 10:02, Oleksii Kurochko wrote: >>>>>>>> On Mon, Jul 22, 2024 at 7:27 PM Julien Grall >>>>>>>> <julien@xen.org> >>>>>>>> wrote: >>>>>>>>>>> On 22/07/2024 15:44, Oleksii Kurochko wrote: >>>>>>>>>> /* Map a 4k page in a fixmap entry */ >>>>>>>>>> void set_fixmap(unsigned map, mfn_t mfn, >>>>>>>>>> unsigned >>>>>>>>>> int >>>>>>>>>> flags) >>>>>>>>>> { >>>>>>>>>> pte_t pte; >>>>>>>>>> >>>>>>>>>> pte = mfn_to_xen_entry(mfn, flags); >>>>>>>>>> pte.pte |= PTE_LEAF_DEFAULT; >>>>>>>>>> write_pte(&xen_fixmap[pt_index(0, >>>>>>>>>> FIXMAP_ADDR(map))], >>>>>>>>>> pte); >>>>>>>>> >>>>>>>>> It would be saner to check if you are not overwriting >>>>>>>>> any >>>>>>>>> existing >>>>>>>>> mapping as otherwise you will probably need a TLB >>>>>>>>> flush. >>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> /* Remove a mapping from a fixmap entry */ >>>>>>>>>> void clear_fixmap(unsigned map) >>>>>>>>>> { >>>>>>>>>> pte_t pte = {0}; >>>>>>>>>> write_pte(&xen_fixmap[pt_index(0, >>>>>>>>>> FIXMAP_ADDR(map))], >>>>>>>>>> pte); >>>>>>>>> >>>>>>>>> Don't you need a TLB flush? >>>>>>>>> >>>>>>>> Inside write_pte() there is "sfence.vma". >>>>>>> >>>>>>> That's just a fence though, not a TLB flush. >>>>>> From the privileged doc: >>>>>> ``` >>>>>> SFENCE.VMA is also used to invalidate entries in the >>>>>> address-translation cache associated with a hart (see >>>>>> Section >>>>>> 4.3.2). >>>>>> ... >>>>>> The SFENCE.VMA is used to flush any local hardware caches >>>>>> related to >>>>>> address translation. >>>>>> It is specified as a fence rather than a TLB flush to >>>>>> provide >>>>>> cleaner >>>>>> semantics with respect to >>>>>> which instructions are affected by the flush operation >>>>>> and >>>>>> to >>>>>> support a >>>>>> wider variety of dynamic >>>>>> caching structures and memory-management schemes. >>>>>> SFENCE.VMA is >>>>>> also >>>>>> used by higher >>>>>> privilege levels to synchronize page table writes and the >>>>>> address >>>>>> translation hardware. >>>>>> ... >>>>>> ``` >>>>>> I read this as SFENCE.VMA is used not only for ordering of >>>>>> load/stores, >>>>>> but also to flush TLB ( which is a type of more general term >>>>>> as >>>>>> address-translation cache, IIUIC ). >>> I have to admit, I am a little because concerned with calling >>> sfence.vma >>> in write_pte() (this may only be because I am not very familiar >>> with >>> RISC-V). >>> >>> We have cases where multiple entry will be written in a single >>> map_pages_to_xen() call. So wouldn't this means that the local TLBs >>> would be nuked for every write rather than once? >> Yes, it will be nuked. It is bad from perfomance point of view. >> I just wanted to be sure that I won't miss to put sfence.vma when it >> is >> necessary and then reworked that a little bit after. But it seems it >> would be better not to call sfence.vma in write_pte() just from the >> start. >> >> >>> >>> >>>>> >>>>> Oh, I see. Kind of unexpected for an instruction of that name. >>>>> Yet >>>>> note >>>>> how they talk about the local hart only. You need a wider scope >>>>> TLB >>>>> flush here. >>>> Could you please clarify why it is needed wider? >>>> >>>> Arm Xen flushed only local TLB. >>> >>> Which code are you looking at? set_fixmap() will propagate the TLB >>> flush >>> to all innershareable CPUs. >> Yes, here I agree that set_fixmap() uses map_pages_to_xen which >> somewhere inside uses flush_xen_tlb_range_va() ( not >> flush_xen_tlb_range_va() ) so TLB flush will happen for all >> innershareable CPUs. >>> >>> The PMAP interface will do a local TLB flush because the interface >>> can >>> only be used during early boot where there is a single CPU running. >> >> Yes, I am looking at PMAP: >> static inline void arch_pmap_unmap(unsigned int slot) >> { >> lpae_t pte = {}; >> >> write_pte(&xen_fixmap[slot], pte); >> >> flush_xen_tlb_range_va(FIXMAP_ADDR(slot), PAGE_SIZE); >> } >> IIUC, originaly Jan told about arch_pmap_unmap() case so that is why >> I >> decided to clarify additionaly. > > Julien, > I have a questation related to Arm's version of arch_pmap_map(): > > static inline void arch_pmap_map(unsigned int slot, mfn_t mfn) > { > lpae_t *entry = &xen_fixmap[slot]; > lpae_t pte; > > ASSERT(!lpae_is_valid(*entry)); > > pte = mfn_to_xen_entry(mfn, PAGE_HYPERVISOR_RW); > pte.pt.table = 1; > write_pte(entry, pte); > /* > * The new entry will be used very soon after arch_pmap_map() > returns. > * So ensure the DSB in write_pte() has completed before > continuing. > */ > isb(); > } > > Is the code above isb() is correct? is it insure the DSB not ISB? I guess you mean comment. If so, yes it is correct. write_pte() has a dsb() just after the PTE. This is to guarantee that implicit memory access (instruction fetch or from the Hardware translation table access) will only happen *after* the dsb() has completed. I am not 100% sure whether the isb() is required for arch_pmap_map(). But I took the safest approach when implementing it. > And isn't need to do TLB flush here? It is not. On Arm, we have a strict policy that every unmap will be followed by a TLB flush and you can't modify an existing entry (see ASSERT a few lines above). So the TLBs will not contain an entry for mapping. This policy was mainly dictacted by the Arm Arm because when modifying the output address you need to follow the break-before-make sequence which means you have to transition to an invalid mapping and flush the TLBs before the new entry is added. For simplicity, we decided to just not bother with trying to implement break-before-make for the hypervisor page-tables. But we do for the P2M. Note that newer revision of the Armv8 spec relaxed the requirement (see FEAT_BBM). Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko ` (4 preceding siblings ...) 2024-07-12 16:22 ` [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-21 9:02 ` Julien Grall 2024-07-12 16:22 ` [PATCH v2 7/8] xen/riscv: select CONFIG_GENREIC_PT Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() Oleksii Kurochko 7 siblings, 1 reply; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Oleksii Kurochko, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk At least, between Arm and RISC-V most of the code related to Xen page table handling are common. This GENERIC_PT code is based on Arm's arm/mmu/pt.c except some minor changes such as introduction of the following functions: * get_root_page() * xen_pt_check_contig() * set_pte_table_bit() * sanity_arch_specific_pte_checks() * get_contig_bit() * set_pte_permissions() * flush_xen_tlb_range_va() It was done because not every functions has the generic pte_flags and it could be a different positions of the PTE bits in a PTE. Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V2: - newly introduced patch --- xen/common/Kconfig | 5 + xen/common/Makefile | 1 + xen/common/mmu/pt.c | 441 +++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/mm.h | 24 +++ 4 files changed, 471 insertions(+) create mode 100644 xen/common/mmu/pt.c diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 565ceda741..6534b77cc9 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -47,6 +47,11 @@ config ARCH_MAP_DOMAIN_PAGE config GENERIC_BUG_FRAME bool +config GENERIC_PT + bool + help + Introduces common function to work with page table when MMU is enabled. + config HAS_ALTERNATIVE bool diff --git a/xen/common/Makefile b/xen/common/Makefile index 21359bab02..e1fb6a5fe8 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -2,6 +2,7 @@ obj-$(CONFIG_ARGO) += argo.o obj-y += bitmap.o obj-bin-$(CONFIG_SELF_TESTS) += bitops.init.o obj-$(CONFIG_GENERIC_BUG_FRAME) += bug.o +obj-$(CONFIG_GENERIC_PT) += mmu/pt.o obj-$(CONFIG_HYPFS_CONFIG) += config_data.o obj-$(CONFIG_CORE_PARKING) += core_parking.o obj-y += cpu.o diff --git a/xen/common/mmu/pt.c b/xen/common/mmu/pt.c new file mode 100644 index 0000000000..7e488eba24 --- /dev/null +++ b/xen/common/mmu/pt.c @@ -0,0 +1,441 @@ +#include <xen/bug.h> +#include <xen/domain_page.h> +#include <xen/errno.h> +#include <xen/mm.h> +#include <xen/mm-frame.h> +#include <xen/pmap.h> +#include <xen/spinlock.h> + +/* Sanity check of the entry */ +static bool xen_pt_check_entry(pte_t entry, mfn_t mfn, unsigned int level, + unsigned int flags) +{ + /* Sanity check when modifying an entry. */ + if ( (flags & _PAGE_PRESENT) && mfn_eq(mfn, INVALID_MFN) ) + { + /* We don't allow modifying an invalid entry. */ + if ( !pte_is_valid(entry) ) + { + printk("Modifying invalid entry is not allowed.\n"); + return false; + } + + /* We don't allow modifying a table entry */ + if ( !pte_is_mapping(entry, level) ) + { + printk("Modifying a table entry is not allowed.\n"); + return false; + } + + if ( !sanity_arch_specific_pte_checks(entry) ) + { + printk("sanity check failed\n"); + return false; + } + } + /* Sanity check when inserting a mapping */ + else if ( flags & _PAGE_PRESENT ) + { + /* We should be here with a valid MFN. */ + ASSERT(!mfn_eq(mfn, INVALID_MFN)); + + /* + * We don't allow replacing any valid entry. + * + * Note that the function xen_pt_update() relies on this + * assumption and will skip the TLB flush. The function will need + * to be updated if the check is relaxed. + */ + if ( pte_is_valid(entry) ) + { + if ( pte_is_mapping(entry, level) ) + printk("Changing MFN for a valid entry is not allowed (%#"PRI_mfn" -> %#"PRI_mfn").\n", + mfn_x(pte_get_mfn(entry)), mfn_x(mfn)); + else + printk("Trying to replace a table with a mapping.\n"); + return false; + } + } + /* Sanity check when removing a mapping. */ + else if ( (flags & (_PAGE_PRESENT|_PAGE_POPULATE)) == 0 ) + { + /* We should be here with an invalid MFN. */ + ASSERT(mfn_eq(mfn, INVALID_MFN)); + + /* We don't allow removing a table */ + if ( pte_is_table(entry, level) ) + { + printk("Removing a table is not allowed.\n"); + return false; + } + + if ( get_contig_bit(entry) ) + { + printk("Removing entry with contiguous bit set is not allowed.\n"); + return false; + } + } + /* Sanity check when populating the page-table. No check so far. */ + else + { + ASSERT(flags & _PAGE_POPULATE); + /* We should be here with an invalid MFN */ + ASSERT(mfn_eq(mfn, INVALID_MFN)); + } + + return true; +} + +static pte_t *xen_map_table(mfn_t mfn) +{ + /* + * During early boot, map_domain_page() may be unusable. Use the + * PMAP to map temporarily a page-table. + */ + if ( system_state == SYS_STATE_early_boot ) + return pmap_map(mfn); + + return map_domain_page(mfn); +} + +static void xen_unmap_table(const pte_t *table) +{ + /* + * During early boot, xen_map_table() will not use map_domain_page() + * but the PMAP. + */ + if ( system_state == SYS_STATE_early_boot ) + pmap_unmap(table); + else + unmap_domain_page(table); +} + +static int create_xen_table(pte_t *entry) +{ + mfn_t mfn; + void *p; + pte_t pte; + + if ( system_state != SYS_STATE_early_boot ) + { + struct page_info *pg = alloc_domheap_page(NULL, 0); + + if ( pg == NULL ) + return -ENOMEM; + + mfn = page_to_mfn(pg); + } + else + mfn = alloc_boot_pages(1, 1); + + p = xen_map_table(mfn); + clear_page(p); + xen_unmap_table(p); + + pte = mfn_to_xen_entry(mfn, MT_NORMAL); + + set_pte_table_bit(&pte, 1); + + write_pte(entry, pte); + + return 0; +} + +#define XEN_TABLE_MAP_FAILED 0 +#define XEN_TABLE_SUPER_PAGE 1 +#define XEN_TABLE_NORMAL_PAGE 2 + +/* + * Take the currently mapped table, find the corresponding entry, + * and map the next table, if available. + * + * The read_only parameters indicates whether intermediate tables should + * be allocated when not present. + * + * Return values: + * XEN_TABLE_MAP_FAILED: Either read_only was set and the entry + * was empty, or allocating a new page failed. + * XEN_TABLE_NORMAL_PAGE: next level mapped normally + * XEN_TABLE_SUPER_PAGE: The next entry points to a superpage. + */ +static int xen_pt_next_level(bool read_only, unsigned int level, + pte_t **table, unsigned int offset) +{ + pte_t *entry; + int ret; + mfn_t mfn; + + entry = *table + offset; + + if ( !pte_is_valid(*entry) ) + { + if ( read_only ) + return XEN_TABLE_MAP_FAILED; + + ret = create_xen_table(entry); + if ( ret ) + return XEN_TABLE_MAP_FAILED; + } + + if ( pte_is_mapping(*entry, level) ) + { + return XEN_TABLE_SUPER_PAGE; + } + + mfn = pte_get_mfn(*entry); + + xen_unmap_table(*table); + *table = xen_map_table(mfn); + + return XEN_TABLE_NORMAL_PAGE; +} + +/* Update an entry at the level @target. */ +static int xen_pt_update_entry(mfn_t root, unsigned long virt, + mfn_t mfn, unsigned int arch_target, + unsigned int flags) +{ + int rc; + unsigned int level = convert_level(HYP_PT_ROOT_LEVEL); + unsigned int arch_level = convert_level(level); + unsigned int target = convert_level(arch_target); + pte_t *table; + /* + * The intermediate page tables are read-only when the MFN is not valid + * and we are not populating page table. + * This means we either modify permissions or remove an entry. + */ + bool read_only = mfn_eq(mfn, INVALID_MFN) && !(flags & _PAGE_POPULATE); + pte_t pte, *entry; + + /* convenience aliases */ + DECLARE_OFFSETS(offsets, (paddr_t)virt); + + /* _PAGE_POPULATE and _PAGE_PRESENT should never be set together. */ + ASSERT((flags & (_PAGE_POPULATE|_PAGE_PRESENT)) != (_PAGE_POPULATE|_PAGE_PRESENT)); + + table = xen_map_table(root); + for ( ; level < target; level++, arch_level = convert_level(level) ) + { + rc = xen_pt_next_level(read_only, arch_level, &table, offsets[arch_level]); + if ( rc == XEN_TABLE_MAP_FAILED ) + { + /* + * We are here because xen_pt_next_level has failed to map + * the intermediate page table (e.g the table does not exist + * and the pt is read-only). It is a valid case when + * removing a mapping as it may not exist in the page table. + * In this case, just ignore it. + */ + if ( flags & (_PAGE_PRESENT | _PAGE_POPULATE) ) + { + printk("%s: Unable to map level %u\n", __func__, arch_level); + rc = -ENOENT; + goto out; + } + else + { + rc = 0; + goto out; + } + } + else if ( rc != XEN_TABLE_NORMAL_PAGE ) { + break; + } + } + + if ( arch_level != arch_target ) + { + printk("%s: Shattering superpage is not supported\n", __func__); + rc = -EOPNOTSUPP; + goto out; + } + + entry = table + offsets[arch_level]; + + rc = -EINVAL; + if ( !xen_pt_check_entry(*entry, mfn, arch_level, flags) ) + goto out; + + /* If we are only populating page-table, then we are done. */ + rc = 0; + if ( flags & _PAGE_POPULATE ) + goto out; + + /* We are removing the page */ + if ( !(flags & _PAGE_PRESENT) ) + memset(&pte, 0x00, sizeof(pte)); + else + { + /* We are inserting a mapping => Create new pte. */ + if ( !mfn_eq(mfn, INVALID_MFN) ) + { + pte = mfn_to_xen_entry(mfn, PAGE_AI_MASK(flags)); + + set_pte_table_bit(&pte, (arch_level == 3)); + } + else /* We are updating the permission => Copy the current pte. */ + pte = *entry; + + set_pte_permissions(&pte, flags); + } + + write_pte(entry, pte); + + rc = 0; + +out: + xen_unmap_table(table); + + return rc; +} + +static DEFINE_SPINLOCK(xen_pt_lock); + +/* Return the level where mapping should be done */ +int xen_pt_mapping_level(unsigned long vfn, mfn_t mfn, unsigned long nr, + unsigned int flags) +{ + unsigned int level = 0; + unsigned long mask; + unsigned int i = 0; + + /* + * Don't take into account the MFN when removing mapping (i.e + * MFN_INVALID) to calculate the correct target order. + * + * Per the Arm Arm, `vfn` and `mfn` must be both superpage aligned. + * They are or-ed together and then checked against the size of + * each level. + * + * `left` is not included and checked separately to allow + * superpage mapping even if it is not properly aligned (the + * user may have asked to map 2MB + 4k). + */ + mask = !mfn_eq(mfn, INVALID_MFN) ? mfn_x(mfn) : 0; + mask |= vfn; + + /* + * Always use level 3 mapping unless the caller request block + * mapping. + */ + if ( likely(!(flags & _PAGE_BLOCK)) ) + return level; + + for ( i = 0; i < CONFIG_PAGING_LEVELS; i++ ) + { + if ( !(mask & (BIT(XEN_PT_LEVEL_ORDER(convert_level(i)), UL) - 1)) && + (nr >= BIT(XEN_PT_LEVEL_ORDER(convert_level(i)), UL)) ) + { + level = convert_level(i); + break; + } + } + + return level; +} + +static int xen_pt_update(unsigned long virt, + mfn_t mfn, + /* const on purpose as it is used for TLB flush */ + const unsigned long nr_mfns, + unsigned int flags) +{ + int rc = 0; + unsigned long vfn = virt >> PAGE_SHIFT; + unsigned long left = nr_mfns; + + /* + * For arm32, page-tables are different on each CPUs. Yet, they share + * some common mappings. It is assumed that only common mappings + * will be modified with this function. + * + * XXX: Add a check. + */ + const mfn_t root = get_root_page(); + + /* + * The hardware was configured to forbid mapping both writeable and + * executable. + * When modifying/creating mapping (i.e _PAGE_PRESENT is set), + * prevent any update if this happen. + */ + if ( (flags & _PAGE_PRESENT) && !PAGE_RO_MASK(flags) && + !PAGE_XN_MASK(flags) ) + { + printk("Mappings should not be both Writeable and Executable.\n"); + return -EINVAL; + } + + if ( flags & _PAGE_CONTIG ) + { + printk("_PAGE_CONTIG is an internal only flag.\n"); + return -EINVAL; + } + + if ( !IS_ALIGNED(virt, PAGE_SIZE) ) + { + printk("The virtual address is not aligned to the page-size.\n"); + return -EINVAL; + } + + spin_lock(&xen_pt_lock); + + while ( left ) + { + unsigned int order, level, nr_contig, new_flags; + + level = xen_pt_mapping_level(vfn, mfn, left, flags); + order = XEN_PT_LEVEL_ORDER(level); + + ASSERT(left >= BIT(order, UL)); + + /* + * Check if we can set the contiguous mapping and update the + * flags accordingly. + */ + nr_contig = xen_pt_check_contig(vfn, mfn, level, left, flags); + new_flags = flags | ((nr_contig > 1) ? _PAGE_CONTIG : 0); + + for ( ; nr_contig > 0; nr_contig-- ) + { + rc = xen_pt_update_entry(root, vfn << PAGE_SHIFT, mfn, level, + new_flags); + if ( rc ) + break; + + vfn += 1U << order; + if ( !mfn_eq(mfn, INVALID_MFN) ) + mfn = mfn_add(mfn, 1U << order); + + left -= (1U << order); + } + + if ( rc ) + break; + } + + /* + * The TLBs flush can be safely skipped when a mapping is inserted + * as we don't allow mapping replacement (see xen_pt_check_entry()). + * + * For all the other cases, the TLBs will be flushed unconditionally + * even if the mapping has failed. This is because we may have + * partially modified the PT. This will prevent any unexpected + * behavior afterwards. + */ + if ( !((flags & _PAGE_PRESENT) && !mfn_eq(mfn, INVALID_MFN)) ) + flush_xen_tlb_range_va(virt, PAGE_SIZE * nr_mfns); + + spin_unlock(&xen_pt_lock); + + return rc; +} + +int map_pages_to_xen(unsigned long virt, + mfn_t mfn, + unsigned long nr_mfns, + unsigned int flags) +{ + return xen_pt_update(virt, mfn, nr_mfns, flags); +} diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index 7561297a75..47cd5326da 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -648,4 +648,28 @@ static inline void put_page_alloc_ref(struct page_info *page) } } +#ifdef CONFIG_GENERIC_PT + +/* is needed because of pte_t. */ +#include <asm/page.h> + +const mfn_t get_root_page(void); + +unsigned int xen_pt_check_contig(unsigned long vfn, mfn_t mfn, + unsigned int level, unsigned long left, + unsigned int flags); + +void set_pte_table_bit(pte_t *pte, unsigned int tbl_bit_val); + +bool sanity_arch_specific_pte_checks(pte_t entry); + +unsigned int get_contig_bit(pte_t entry); + +void set_pte_permissions(pte_t *pte, unsigned int flags); + +void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size); + +#endif /* CONFIG_GENERIC_PT */ + #endif /* __XEN_MM_H__ */ -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling 2024-07-12 16:22 ` [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling Oleksii Kurochko @ 2024-07-21 9:02 ` Julien Grall 2024-07-22 6:59 ` Jan Beulich 2024-07-22 14:51 ` Oleksii 0 siblings, 2 replies; 51+ messages in thread From: Julien Grall @ 2024-07-21 9:02 UTC (permalink / raw) To: Oleksii Kurochko, xen-devel Cc: Andrew Cooper, Jan Beulich, Stefano Stabellini, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk Hi, On 12/07/2024 17:22, Oleksii Kurochko wrote: > At least, between Arm and RISC-V most of the code related to Xen page > table handling are common. > > This GENERIC_PT code is based on Arm's arm/mmu/pt.c except some minor > changes such as introduction of the following functions: > * get_root_page() > * xen_pt_check_contig() > * set_pte_table_bit() > * sanity_arch_specific_pte_checks() > * get_contig_bit() > * set_pte_permissions() > * flush_xen_tlb_range_va() > It was done because not every functions has the generic pte_flags and > it could be a different positions of the PTE bits in a PTE. While I am always in favor of trying to avoid code duplication, I am not sure the page-tables are one that should be. For instance, you don't have the concept of contiguous page in RISC-V (I see you introduce dummy flags, but IMHO this is a bit of a hack). All the code was also written in a way to avoid temporary conflict mappings. This is to avoid using Break-Before-Make. But on newer hardware this could be relaxed. I am interested to know what the others thinks. Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling 2024-07-21 9:02 ` Julien Grall @ 2024-07-22 6:59 ` Jan Beulich 2024-07-22 14:51 ` Oleksii 1 sibling, 0 replies; 51+ messages in thread From: Jan Beulich @ 2024-07-22 6:59 UTC (permalink / raw) To: Julien Grall Cc: Andrew Cooper, Stefano Stabellini, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Oleksii Kurochko, xen-devel On 21.07.2024 11:02, Julien Grall wrote: > Hi, > > On 12/07/2024 17:22, Oleksii Kurochko wrote: >> At least, between Arm and RISC-V most of the code related to Xen page >> table handling are common. >> >> This GENERIC_PT code is based on Arm's arm/mmu/pt.c except some minor >> changes such as introduction of the following functions: >> * get_root_page() >> * xen_pt_check_contig() >> * set_pte_table_bit() >> * sanity_arch_specific_pte_checks() >> * get_contig_bit() >> * set_pte_permissions() >> * flush_xen_tlb_range_va() >> It was done because not every functions has the generic pte_flags and >> it could be a different positions of the PTE bits in a PTE. > While I am always in favor of trying to avoid code duplication, I am not > sure the page-tables are one that should be. > > For instance, you don't have the concept of contiguous page in RISC-V (I > see you introduce dummy flags, but IMHO this is a bit of a hack). All > the code was also written in a way to avoid temporary conflict mappings. > This is to avoid using Break-Before-Make. But on newer hardware this > could be relaxed. > > I am interested to know what the others thinks. I share the concern; I was rather surprised to see the title, but - not having had the time to look at the patch yet - was kind of guessing (hoping) it wasn't really what the title said. Page table handling quite likely wants to remain arch-specific. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling 2024-07-21 9:02 ` Julien Grall 2024-07-22 6:59 ` Jan Beulich @ 2024-07-22 14:51 ` Oleksii 1 sibling, 0 replies; 51+ messages in thread From: Oleksii @ 2024-07-22 14:51 UTC (permalink / raw) To: Julien Grall, xen-devel Cc: Andrew Cooper, Jan Beulich, Stefano Stabellini, Bertrand Marquis, Michal Orzel, Volodymyr Babchuk Hi Julien, On Sun, 2024-07-21 at 10:02 +0100, Julien Grall wrote: > Hi, > > On 12/07/2024 17:22, Oleksii Kurochko wrote: > > At least, between Arm and RISC-V most of the code related to Xen > > page > > table handling are common. > > > > This GENERIC_PT code is based on Arm's arm/mmu/pt.c except some > > minor > > changes such as introduction of the following functions: > > * get_root_page() > > * xen_pt_check_contig() > > * set_pte_table_bit() > > * sanity_arch_specific_pte_checks() > > * get_contig_bit() > > * set_pte_permissions() > > * flush_xen_tlb_range_va() > > It was done because not every functions has the generic pte_flags > > and > > it could be a different positions of the PTE bits in a PTE. > While I am always in favor of trying to avoid code duplication, I am > not > sure the page-tables are one that should be. Probably it wasn't the best one abstraction provided. But I think that there are still some function which could be generic: - xen_{un}map_table() - map_pages_to_xen - probably some page table walking? But I am okay, if not to abstract that, I just wanted to here an opinion if it is a sense to work in this direction or not. I will then move everything to RISC-V specific folder and clean up some places related to Arm. ~ Oleksii > > For instance, you don't have the concept of contiguous page in RISC-V > (I > see you introduce dummy flags, but IMHO this is a bit of a hack). All > the code was also written in a way to avoid temporary conflict > mappings. > This is to avoid using Break-Before-Make. But on newer hardware this > could be relaxed. > > I am interested to know what the others thinks. > > Cheers, > ^ permalink raw reply [flat|nested] 51+ messages in thread
* [PATCH v2 7/8] xen/riscv: select CONFIG_GENREIC_PT 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko ` (5 preceding siblings ...) 2024-07-12 16:22 ` [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() Oleksii Kurochko 7 siblings, 0 replies; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Oleksii Kurochko, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini Enable GENERIC_PT functionalities for RISC-V and also introduce some RISC-V specific functions necessary to make the GENERIC_PT code work and compilable. Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V2: - newly introduced patch --- xen/arch/riscv/Kconfig | 1 + xen/arch/riscv/include/asm/mm.h | 9 ++++ xen/arch/riscv/include/asm/page-bits.h | 45 ++++++++++++++++ xen/arch/riscv/include/asm/page.h | 73 +++++++++++++++++++++++++- xen/arch/riscv/mm.c | 63 +++++++++++++++++++--- 5 files changed, 184 insertions(+), 7 deletions(-) diff --git a/xen/arch/riscv/Kconfig b/xen/arch/riscv/Kconfig index 0112aa8778..9827a12d34 100644 --- a/xen/arch/riscv/Kconfig +++ b/xen/arch/riscv/Kconfig @@ -2,6 +2,7 @@ config RISCV def_bool y select FUNCTION_ALIGNMENT_16B select GENERIC_BUG_FRAME + select GENERIC_PT select HAS_DEVICE_TREE select HAS_PMAP diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h index a0bdc2bc3a..c54546c275 100644 --- a/xen/arch/riscv/include/asm/mm.h +++ b/xen/arch/riscv/include/asm/mm.h @@ -42,6 +42,8 @@ static inline void *maddr_to_virt(paddr_t ma) #define virt_to_mfn(va) __virt_to_mfn(va) #define mfn_to_virt(mfn) __mfn_to_virt(mfn) +#define pte_get_mfn(pte) maddr_to_mfn(pte_to_paddr(pte)) + struct page_info { /* Each frame can be threaded onto a doubly-linked list. */ @@ -238,6 +240,13 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) #define PFN_ORDER(pg) ((pg)->v.free.order) +/* + * Generic code that works with page tables expects the page table + * levels to be numbered in the order L0 -> L1 -> ...; + * RISC-V uses the opposite enumeration: Lx -> L1 -> L0 + */ +#define convert_level(level) (HYP_PT_ROOT_LEVEL - level) + extern unsigned char cpu0_boot_stack[]; void setup_initial_pagetables(void); diff --git a/xen/arch/riscv/include/asm/page-bits.h b/xen/arch/riscv/include/asm/page-bits.h index 8f1f474371..1d3a0adda6 100644 --- a/xen/arch/riscv/include/asm/page-bits.h +++ b/xen/arch/riscv/include/asm/page-bits.h @@ -3,6 +3,51 @@ #ifndef __RISCV_PAGE_BITS_H__ #define __RISCV_PAGE_BITS_H__ +/* + * PTE format: + * | XLEN-1 10 | 9 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 + * PFN reserved for SW D A G U X W R V + */ + +#define _PAGE_PRESENT BIT(0, UL) +#define _PAGE_READ BIT(1, UL) /* Readable */ +#define _PAGE_WRITE BIT(2, UL) /* Writable */ +#define _PAGE_EXEC BIT(3, UL) /* Executable */ +#define _PAGE_USER BIT(4, UL) /* User */ +#define _PAGE_GLOBAL BIT(5, UL) /* Global */ +#define _PAGE_ACCESSED BIT(6, UL) /* Set by hardware on any access */ +#define _PAGE_DIRTY BIT(7, UL) /* Set by hardware on any write */ +#define _PAGE_SOFT BIT(8, UL) /* Reserved for software */ + +/* + * There is no such bits in PTE format for RISC-V. + * Most of the definition below are just to make PT generic code happy, + * except _PAGE_BLOCK which is used to map 2 MB page table entries. + */ +#define _PAGE_BLOCK BIT(9, UL) +#define _PAGE_CONTIG BIT(10, UL) +#define _PAGE_POPULATE BIT(11, UL) +#define MT_NORMAL BIT(12, UL) + +/* Set of bits to preserve across pte_modify() */ +#define _PAGE_CHG_MASK (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ | \ + _PAGE_WRITE | _PAGE_EXEC | \ + _PAGE_USER | _PAGE_GLOBAL)) + +#define PAGE_AI_MASK(x) ((x) & _PAGE_CHG_MASK) + +#define _PAGE_W_BIT 2 +#define _PAGE_XN_BIT 3 +#define _PAGE_RO_BIT 1 + +/* TODO: move to somewhere generic part/header ? */ +#define _PAGE_XN (1U << _PAGE_XN_BIT) +#define _PAGE_RO (1U << _PAGE_RO_BIT) +#define _PAGE_W (1U << _PAGE_W_BIT) +#define PAGE_XN_MASK(x) (((x) >> _PAGE_XN_BIT) & 0x1U) +#define PAGE_RO_MASK(x) (((x) >> _PAGE_RO_BIT) & 0x1U) +#define PAGE_W_MASK(x) (((x) >> _PAGE_W_BIT) & 0x1U) + #define PAGE_SHIFT 12 /* 4 KiB Pages */ #define PADDR_BITS 56 /* 44-bit PPN */ diff --git a/xen/arch/riscv/include/asm/page.h b/xen/arch/riscv/include/asm/page.h index 339074d502..bb2da05693 100644 --- a/xen/arch/riscv/include/asm/page.h +++ b/xen/arch/riscv/include/asm/page.h @@ -33,6 +33,7 @@ #define PTE_LEAF_DEFAULT (PTE_VALID | PTE_READABLE | PTE_WRITABLE) #define PTE_TABLE (PTE_VALID) +#define PAGE_HYPERVISOR_RO (PTE_VALID | PTE_READABLE) #define PAGE_HYPERVISOR_RW (PTE_VALID | PTE_READABLE | PTE_WRITABLE) #define PAGE_HYPERVISOR PAGE_HYPERVISOR_RW @@ -42,13 +43,68 @@ #define pt_index(lvl, va) (pt_linear_offset((lvl), (va)) & VPN_MASK) -/* Page Table entry */ +#define FIRST_SIZE (XEN_PT_LEVEL_SIZE(2)) + +#define TABLE_OFFSET(offs) (_AT(unsigned int, offs) & ((_AC(1, U) << PAGETABLE_ORDER) - 1)) + +#if RV_STAGE1_MODE > SATP_MODE_SV48 +#error "need to to update DECLARE_OFFSETS macros" +#else + +#define l0_table_offset(va) TABLE_OFFSET(pt_linear_offset(0, va)) +#define l1_table_offset(va) TABLE_OFFSET(pt_linear_offset(1, va)) +#define l2_table_offset(va) TABLE_OFFSET(pt_linear_offset(2, va)) +#define l3_table_offset(va) TABLE_OFFSET(pt_linear_offset(3, va)) + +/* Generate an array @var containing the offset for each level from @addr */ +#define DECLARE_OFFSETS(var, addr) \ + const unsigned int var[4] = { \ + l0_table_offset(addr), \ + l1_table_offset(addr), \ + l2_table_offset(addr), \ + l3_table_offset(addr) \ + } + +#endif + typedef struct { + unsigned long v:1; + unsigned long r:1; + unsigned long w:1; + unsigned long x:1; + unsigned long u:1; + unsigned long g:1; + unsigned long a:1; + unsigned long d:1; + unsigned long rsw:2; +#if RV_STAGE1_MODE == SATP_MODE_SV39 + unsigned long ppn0:9; + unsigned long ppn1:9; + unsigned long ppn2:26; + unsigned long rsw2:7; + unsigned long pbmt:2; + unsigned long n:1; +#elif RV_STAGE1_MODE == SATP_MODE_SV48 + unsigned long ppn0:9; + unsigned long ppn1:9; + unsigned long ppn2:9; + unsigned long ppn3:17; + unsigned long rsw2:7; + unsigned long pbmt:2; + unsigned long n:1; +#else +#error "Add proper bits for SATP_MODE" +#endif +} pt_t; + +/* Page Table entry */ +typedef union { #ifdef CONFIG_RISCV_64 uint64_t pte; #else uint32_t pte; #endif +pt_t bits; } pte_t; pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr); @@ -69,6 +125,21 @@ static inline bool pte_is_valid(pte_t p) return p.pte & PTE_VALID; } +inline bool pte_is_table(const pte_t p, unsigned int level) +{ + (void) level; + + return (((p.pte) & (PTE_VALID + | PTE_READABLE + | PTE_WRITABLE + | PTE_EXECUTABLE)) == PTE_VALID); +} + +static inline bool pte_is_mapping(const pte_t pte, unsigned int level) +{ + return !pte_is_table(pte, level); +} + static inline void invalidate_icache(void) { BUG_ON("unimplemented"); diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 445319af08..44f36359c8 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -362,13 +362,64 @@ int destroy_xen_mappings(unsigned long s, unsigned long e) return -1; } -int map_pages_to_xen(unsigned long virt, - mfn_t mfn, - unsigned long nr_mfns, - unsigned int flags) +const mfn_t get_root_page(void) { - BUG_ON("unimplemented"); - return -1; + unsigned long root_maddr = csr_read(CSR_SATP) << PAGE_SHIFT; + + return maddr_to_mfn(root_maddr); +} + +/* + * Check whether the contiguous bit can be set. Return the number of + * contiguous entry allowed. If not allowed, return 1. + */ +unsigned int xen_pt_check_contig(unsigned long vfn, mfn_t mfn, + unsigned int level, unsigned long left, + unsigned int flags) +{ + /* there is no contig bit in RISC-V */ + return 1; +} + +void set_pte_table_bit(pte_t *pte, unsigned int tbl_bit_val) +{ + /* table bit for RISC-V is always equal to PTE_TABLE */ + (void) tbl_bit_val; + + pte->pte |= PTE_TABLE; +} + +bool sanity_arch_specific_pte_checks(pte_t entry) +{ + /* there is no RISC-V specific PTE checks */ + return true; +} + +unsigned int get_contig_bit(pte_t entry) +{ + /* there is no contig bit */ + (void) entry; + + return 0; +} + +void set_pte_permissions(pte_t *pte, unsigned int flags) +{ + pte->bits.r = PAGE_RO_MASK(flags); + pte->bits.x = ~PAGE_XN_MASK(flags); + pte->bits.w = PAGE_W_MASK(flags); + + pte->pte |= PTE_ACCESSED | PTE_DIRTY; +} + +inline void flush_xen_tlb_range_va(vaddr_t va, + unsigned long size) +{ + /* TODO: implement flush of specific range va */ + (void) va; + (void) size; + + asm volatile("sfence.vma"); } static inline pte_t mfn_to_pte(mfn_t mfn) -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko ` (6 preceding siblings ...) 2024-07-12 16:22 ` [PATCH v2 7/8] xen/riscv: select CONFIG_GENREIC_PT Oleksii Kurochko @ 2024-07-12 16:22 ` Oleksii Kurochko 2024-07-15 8:52 ` Jan Beulich 7 siblings, 1 reply; 51+ messages in thread From: Oleksii Kurochko @ 2024-07-12 16:22 UTC (permalink / raw) To: xen-devel Cc: Oleksii Kurochko, Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Jan Beulich, Julien Grall, Stefano Stabellini Introduce function which allows to map FDT to Xen. Also, initialization of device_tree_flattened happens using early_fdt_map. Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- Changes in V2: - rework early_fdt_map to use map_pages_to_xen() - move call early_fdt_map() to C code after MMU is enabled. --- xen/arch/riscv/include/asm/mm.h | 2 ++ xen/arch/riscv/mm.c | 55 +++++++++++++++++++++++++++++++++ xen/arch/riscv/setup.c | 9 ++++++ 3 files changed, 66 insertions(+) diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h index c54546c275..a12ef5bb63 100644 --- a/xen/arch/riscv/include/asm/mm.h +++ b/xen/arch/riscv/include/asm/mm.h @@ -266,4 +266,6 @@ static inline unsigned int arch_get_dma_bitsize(void) void setup_fixmap_mappings(void); +void* early_fdt_map(paddr_t fdt_paddr); + #endif /* _ASM_RISCV_MM_H */ diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 44f36359c8..428c26b636 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -1,13 +1,16 @@ /* SPDX-License-Identifier: GPL-2.0-only */ +#include <xen/bootfdt.h> #include <xen/bug.h> #include <xen/compiler.h> #include <xen/init.h> #include <xen/kernel.h> +#include <xen/libfdt/libfdt.h> #include <xen/macros.h> #include <xen/mm.h> #include <xen/pfn.h> #include <xen/sections.h> +#include <xen/sizes.h> #include <asm/early_printk.h> #include <asm/csr.h> @@ -435,3 +438,55 @@ inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) return mfn_to_pte(mfn); } + +void * __init early_fdt_map(paddr_t fdt_paddr) +{ + /* We are using 2MB superpage for mapping the FDT */ + paddr_t base_paddr = fdt_paddr & XEN_PT_LEVEL_MAP_MASK(1); + paddr_t offset; + void *fdt_virt; + uint32_t size; + int rc; + + /* + * Check whether the physical FDT address is set and meets the minimum + * alignment requirement. Since we are relying on MIN_FDT_ALIGN to be at + * least 8 bytes so that we always access the magic and size fields + * of the FDT header after mapping the first chunk, double check if + * that is indeed the case. + */ + BUILD_BUG_ON(MIN_FDT_ALIGN < 8); + if ( !fdt_paddr || fdt_paddr % MIN_FDT_ALIGN ) + return NULL; + + /* The FDT is mapped using 2MB superpage */ + BUILD_BUG_ON(BOOT_FDT_VIRT_START % SZ_2M); + + rc = map_pages_to_xen(BOOT_FDT_VIRT_START, maddr_to_mfn(base_paddr), + SZ_2M >> PAGE_SHIFT, + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); + if ( rc ) + panic("Unable to map the device-tree.\n"); + + offset = fdt_paddr % XEN_PT_LEVEL_SIZE(1); + fdt_virt = (void *)BOOT_FDT_VIRT_START + offset; + + if ( fdt_magic(fdt_virt) != FDT_MAGIC ) + return NULL; + + size = fdt_totalsize(fdt_virt); + if ( size > BOOT_FDT_VIRT_SIZE ) + return NULL; + + if ( (offset + size) > SZ_2M ) + { + rc = map_pages_to_xen(BOOT_FDT_VIRT_START + SZ_2M, + maddr_to_mfn(base_paddr + SZ_2M), + SZ_2M >> PAGE_SHIFT, + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); + if ( rc ) + panic("Unable to map the device-tree\n"); + } + + return fdt_virt; +} diff --git a/xen/arch/riscv/setup.c b/xen/arch/riscv/setup.c index 13f0e8c77d..21628b7300 100644 --- a/xen/arch/riscv/setup.c +++ b/xen/arch/riscv/setup.c @@ -2,6 +2,7 @@ #include <xen/bug.h> #include <xen/compile.h> +#include <xen/device_tree.h> #include <xen/init.h> #include <xen/mm.h> @@ -48,6 +49,14 @@ void __init noreturn start_xen(unsigned long bootcpu_id, setup_fixmap_mappings(); + device_tree_flattened = early_fdt_map(dtb_addr); + if ( device_tree_flattened ) + panic("Invalid device tree blob at physical address %#lx.\n" + "The DTB must be 8-byte aligned and must not exceed %lld " + "bytes in size.\n\n" + "Please check your bootloader.\n", + dtb_addr, BOOT_FDT_VIRT_SIZE); + printk("All set up\n"); for ( ;; ) -- 2.45.2 ^ permalink raw reply related [flat|nested] 51+ messages in thread
* Re: [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() 2024-07-12 16:22 ` [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() Oleksii Kurochko @ 2024-07-15 8:52 ` Jan Beulich 2024-07-15 13:58 ` Oleksii 0 siblings, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-15 8:52 UTC (permalink / raw) To: Oleksii Kurochko Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 12.07.2024 18:22, Oleksii Kurochko wrote: > --- a/xen/arch/riscv/include/asm/mm.h > +++ b/xen/arch/riscv/include/asm/mm.h > @@ -266,4 +266,6 @@ static inline unsigned int arch_get_dma_bitsize(void) > > void setup_fixmap_mappings(void); > > +void* early_fdt_map(paddr_t fdt_paddr); Please can you take care to address comments on earlier versions before submitting a new one? > @@ -435,3 +438,55 @@ inline pte_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr) > > return mfn_to_pte(mfn); > } > + > +void * __init early_fdt_map(paddr_t fdt_paddr) > +{ > + /* We are using 2MB superpage for mapping the FDT */ > + paddr_t base_paddr = fdt_paddr & XEN_PT_LEVEL_MAP_MASK(1); > + paddr_t offset; > + void *fdt_virt; > + uint32_t size; > + int rc; > + > + /* > + * Check whether the physical FDT address is set and meets the minimum > + * alignment requirement. Since we are relying on MIN_FDT_ALIGN to be at > + * least 8 bytes so that we always access the magic and size fields > + * of the FDT header after mapping the first chunk, double check if > + * that is indeed the case. > + */ > + BUILD_BUG_ON(MIN_FDT_ALIGN < 8); > + if ( !fdt_paddr || fdt_paddr % MIN_FDT_ALIGN ) > + return NULL; > + > + /* The FDT is mapped using 2MB superpage */ > + BUILD_BUG_ON(BOOT_FDT_VIRT_START % SZ_2M); May I suggest that you use MB(2) instead of SZ_2M (not just here)? I think I had voiced opposition to the introduction of xen/sizes.h about 10 years back, yet sadly it still landed in the tree. I for one think that our KB(), MB(), and GB() constructs are superior, and (I hope) free of Misra issues (unlike SZ_2G). > + rc = map_pages_to_xen(BOOT_FDT_VIRT_START, maddr_to_mfn(base_paddr), > + SZ_2M >> PAGE_SHIFT, > + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); > + if ( rc ) > + panic("Unable to map the device-tree.\n"); > + > + offset = fdt_paddr % XEN_PT_LEVEL_SIZE(1); > + fdt_virt = (void *)BOOT_FDT_VIRT_START + offset; > + > + if ( fdt_magic(fdt_virt) != FDT_MAGIC ) > + return NULL; > + > + size = fdt_totalsize(fdt_virt); > + if ( size > BOOT_FDT_VIRT_SIZE ) > + return NULL; > + > + if ( (offset + size) > SZ_2M ) > + { > + rc = map_pages_to_xen(BOOT_FDT_VIRT_START + SZ_2M, > + maddr_to_mfn(base_paddr + SZ_2M), > + SZ_2M >> PAGE_SHIFT, > + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); > + if ( rc ) > + panic("Unable to map the device-tree\n"); > + } Why this two part mapping? And why are you mapping perhaps much more than "size"? > @@ -48,6 +49,14 @@ void __init noreturn start_xen(unsigned long bootcpu_id, > > setup_fixmap_mappings(); > > + device_tree_flattened = early_fdt_map(dtb_addr); > + if ( device_tree_flattened ) Is this condition perhaps inverted? Jan > + panic("Invalid device tree blob at physical address %#lx.\n" > + "The DTB must be 8-byte aligned and must not exceed %lld " > + "bytes in size.\n\n" > + "Please check your bootloader.\n", > + dtb_addr, BOOT_FDT_VIRT_SIZE); > + > printk("All set up\n"); > > for ( ;; ) ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() 2024-07-15 8:52 ` Jan Beulich @ 2024-07-15 13:58 ` Oleksii 2024-07-15 14:58 ` Jan Beulich 0 siblings, 1 reply; 51+ messages in thread From: Oleksii @ 2024-07-15 13:58 UTC (permalink / raw) To: Jan Beulich Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On Mon, 2024-07-15 at 10:52 +0200, Jan Beulich wrote: > On 12.07.2024 18:22, Oleksii Kurochko wrote: > > --- a/xen/arch/riscv/include/asm/mm.h > > +++ b/xen/arch/riscv/include/asm/mm.h > > @@ -266,4 +266,6 @@ static inline unsigned int > > arch_get_dma_bitsize(void) > > > > void setup_fixmap_mappings(void); > > > > +void* early_fdt_map(paddr_t fdt_paddr); > > Please can you take care to address comments on earlier versions > before > submitting a new one? Sorry, missed that Nit comment where you suggested to switch space/block and *. > > > @@ -435,3 +438,55 @@ inline pte_t mfn_to_xen_entry(mfn_t mfn, > > unsigned int attr) > > > > return mfn_to_pte(mfn); > > } > > + > > +void * __init early_fdt_map(paddr_t fdt_paddr) > > +{ > > + /* We are using 2MB superpage for mapping the FDT */ > > + paddr_t base_paddr = fdt_paddr & XEN_PT_LEVEL_MAP_MASK(1); > > + paddr_t offset; > > + void *fdt_virt; > > + uint32_t size; > > + int rc; > > + > > + /* > > + * Check whether the physical FDT address is set and meets the > > minimum > > + * alignment requirement. Since we are relying on > > MIN_FDT_ALIGN to be at > > + * least 8 bytes so that we always access the magic and size > > fields > > + * of the FDT header after mapping the first chunk, double > > check if > > + * that is indeed the case. > > + */ > > + BUILD_BUG_ON(MIN_FDT_ALIGN < 8); > > + if ( !fdt_paddr || fdt_paddr % MIN_FDT_ALIGN ) > > + return NULL; > > + > > + /* The FDT is mapped using 2MB superpage */ > > + BUILD_BUG_ON(BOOT_FDT_VIRT_START % SZ_2M); > > May I suggest that you use MB(2) instead of SZ_2M (not just here)? I > think > I had voiced opposition to the introduction of xen/sizes.h about 10 > years > back, yet sadly it still landed in the tree. I for one think that our > KB(), > MB(), and GB() constructs are superior, and (I hope) free of Misra > issues > (unlike SZ_2G). > > > + rc = map_pages_to_xen(BOOT_FDT_VIRT_START, > > maddr_to_mfn(base_paddr), > > + SZ_2M >> PAGE_SHIFT, > > + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); > > + if ( rc ) > > + panic("Unable to map the device-tree.\n"); > > + > > + offset = fdt_paddr % XEN_PT_LEVEL_SIZE(1); > > + fdt_virt = (void *)BOOT_FDT_VIRT_START + offset; > > + > > + if ( fdt_magic(fdt_virt) != FDT_MAGIC ) > > + return NULL; > > + > > + size = fdt_totalsize(fdt_virt); > > + if ( size > BOOT_FDT_VIRT_SIZE ) > > + return NULL; > > + > > + if ( (offset + size) > SZ_2M ) > > + { > > + rc = map_pages_to_xen(BOOT_FDT_VIRT_START + SZ_2M, > > + maddr_to_mfn(base_paddr + SZ_2M), > > + SZ_2M >> PAGE_SHIFT, > > + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); > > + if ( rc ) > > + panic("Unable to map the device-tree\n"); > > + } > > Why this two part mapping? And why are you mapping perhaps much more > than "size"? I wasn't able to find if RISC-V has a requirement for alignment of FDT address so I decided to follow Arm where FDT is required to be placed on a 8-byte boundary, so FDT can cross a 2MB boundary, so the second 2MB page should be mapped if the FDT is crossing the 2MB boundary. > > > @@ -48,6 +49,14 @@ void __init noreturn start_xen(unsigned long > > bootcpu_id, > > > > setup_fixmap_mappings(); > > > > + device_tree_flattened = early_fdt_map(dtb_addr); > > + if ( device_tree_flattened ) > > Is this condition perhaps inverted? Yes, you are right. It should be inverted. Thanks. ~ Oleksii > > > + panic("Invalid device tree blob at physical address > > %#lx.\n" > > + "The DTB must be 8-byte aligned and must not exceed > > %lld " > > + "bytes in size.\n\n" > > + "Please check your bootloader.\n", > > + dtb_addr, BOOT_FDT_VIRT_SIZE); > > + > > printk("All set up\n"); > > > > for ( ;; ) > ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() 2024-07-15 13:58 ` Oleksii @ 2024-07-15 14:58 ` Jan Beulich 2024-07-15 15:00 ` Jan Beulich 0 siblings, 1 reply; 51+ messages in thread From: Jan Beulich @ 2024-07-15 14:58 UTC (permalink / raw) To: Oleksii Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 15.07.2024 15:58, Oleksii wrote: > On Mon, 2024-07-15 at 10:52 +0200, Jan Beulich wrote: >> On 12.07.2024 18:22, Oleksii Kurochko wrote: >>> + rc = map_pages_to_xen(BOOT_FDT_VIRT_START, >>> maddr_to_mfn(base_paddr), >>> + SZ_2M >> PAGE_SHIFT, >>> + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); >>> + if ( rc ) >>> + panic("Unable to map the device-tree.\n"); >>> + >>> + offset = fdt_paddr % XEN_PT_LEVEL_SIZE(1); >>> + fdt_virt = (void *)BOOT_FDT_VIRT_START + offset; >>> + >>> + if ( fdt_magic(fdt_virt) != FDT_MAGIC ) >>> + return NULL; >>> + >>> + size = fdt_totalsize(fdt_virt); >>> + if ( size > BOOT_FDT_VIRT_SIZE ) >>> + return NULL; >>> + >>> + if ( (offset + size) > SZ_2M ) >>> + { >>> + rc = map_pages_to_xen(BOOT_FDT_VIRT_START + SZ_2M, >>> + maddr_to_mfn(base_paddr + SZ_2M), >>> + SZ_2M >> PAGE_SHIFT, >>> + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); >>> + if ( rc ) >>> + panic("Unable to map the device-tree\n"); >>> + } >> >> Why this two part mapping? And why are you mapping perhaps much more >> than "size"? > I wasn't able to find if RISC-V has a requirement for alignment of FDT > address so I decided to follow Arm where FDT is required to be placed > on a 8-byte boundary, so FDT can cross a 2MB boundary, so the second > 2MB page should be mapped if the FDT is crossing the 2MB boundary. This explains why you may need to map more than 2Mb (which wasn't the question), but it doesn't explain why you need to do it in two steps. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
* Re: [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() 2024-07-15 14:58 ` Jan Beulich @ 2024-07-15 15:00 ` Jan Beulich 0 siblings, 0 replies; 51+ messages in thread From: Jan Beulich @ 2024-07-15 15:00 UTC (permalink / raw) To: Oleksii Cc: Alistair Francis, Bob Eshleman, Connor Davis, Andrew Cooper, Julien Grall, Stefano Stabellini, xen-devel On 15.07.2024 16:58, Jan Beulich wrote: > On 15.07.2024 15:58, Oleksii wrote: >> On Mon, 2024-07-15 at 10:52 +0200, Jan Beulich wrote: >>> On 12.07.2024 18:22, Oleksii Kurochko wrote: >>>> + rc = map_pages_to_xen(BOOT_FDT_VIRT_START, >>>> maddr_to_mfn(base_paddr), >>>> + SZ_2M >> PAGE_SHIFT, >>>> + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); >>>> + if ( rc ) >>>> + panic("Unable to map the device-tree.\n"); >>>> + >>>> + offset = fdt_paddr % XEN_PT_LEVEL_SIZE(1); >>>> + fdt_virt = (void *)BOOT_FDT_VIRT_START + offset; >>>> + >>>> + if ( fdt_magic(fdt_virt) != FDT_MAGIC ) >>>> + return NULL; >>>> + >>>> + size = fdt_totalsize(fdt_virt); >>>> + if ( size > BOOT_FDT_VIRT_SIZE ) >>>> + return NULL; >>>> + >>>> + if ( (offset + size) > SZ_2M ) >>>> + { >>>> + rc = map_pages_to_xen(BOOT_FDT_VIRT_START + SZ_2M, >>>> + maddr_to_mfn(base_paddr + SZ_2M), >>>> + SZ_2M >> PAGE_SHIFT, >>>> + PAGE_HYPERVISOR_RO | _PAGE_BLOCK); >>>> + if ( rc ) >>>> + panic("Unable to map the device-tree\n"); >>>> + } >>> >>> Why this two part mapping? And why are you mapping perhaps much more >>> than "size"? >> I wasn't able to find if RISC-V has a requirement for alignment of FDT >> address so I decided to follow Arm where FDT is required to be placed >> on a 8-byte boundary, so FDT can cross a 2MB boundary, so the second >> 2MB page should be mapped if the FDT is crossing the 2MB boundary. > > This explains why you may need to map more than 2Mb (which wasn't the > question), but it doesn't explain why you need to do it in two steps. Oh, wait - you know the full size only after having mapped the initial part. I'm sorry, I didn't spot that early enough. Jan ^ permalink raw reply [flat|nested] 51+ messages in thread
end of thread, other threads:[~2024-07-23 18:44 UTC | newest] Thread overview: 51+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-07-12 16:22 [PATCH v2 0/8] RISCV device tree mapping Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v6 1/8] xen/device-tree: Move Arm's setup.c bootinfo functions to common Oleksii Kurochko 2024-07-15 8:36 ` Jan Beulich 2024-07-15 10:15 ` Michal Orzel 2024-07-15 13:42 ` Oleksii 2024-07-12 16:22 ` [PATCH v6 2/8] xen/common: Move Arm's bootfdt.c " Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v2 3/8] xen/riscv: enable CONFIG_HAS_DEVICE_TREE Oleksii Kurochko 2024-07-15 8:40 ` Jan Beulich 2024-07-12 16:22 ` [PATCH v2 4/8] xen/riscv: setup fixmap mapping Oleksii Kurochko 2024-07-21 8:46 ` Julien Grall 2024-07-22 14:31 ` Oleksii 2024-07-22 14:44 ` Julien Grall 2024-07-23 12:58 ` oleksii.kurochko 2024-07-23 13:32 ` Julien Grall 2024-07-23 13:27 ` oleksii.kurochko 2024-07-23 13:33 ` Julien Grall 2024-07-23 13:38 ` oleksii.kurochko 2024-07-22 12:42 ` Jan Beulich 2024-07-22 14:36 ` Oleksii 2024-07-22 15:25 ` Jan Beulich 2024-07-22 17:04 ` oleksii.kurochko 2024-07-23 13:34 ` oleksii.kurochko 2024-07-12 16:22 ` [PATCH v2 5/8] xen/riscv: introduce asm/pmap.h header Oleksii Kurochko 2024-07-21 8:51 ` Julien Grall 2024-07-22 12:58 ` Jan Beulich 2024-07-22 14:57 ` Julien Grall 2024-07-22 14:40 ` Oleksii 2024-07-22 12:54 ` Jan Beulich 2024-07-22 14:44 ` Oleksii Kurochko 2024-07-22 14:48 ` Julien Grall 2024-07-22 17:09 ` Oleksii Kurochko 2024-07-22 17:21 ` Julien Grall 2024-07-23 8:02 ` Oleksii Kurochko 2024-07-23 8:36 ` Jan Beulich 2024-07-23 8:55 ` oleksii.kurochko 2024-07-23 10:02 ` Jan Beulich 2024-07-23 15:36 ` oleksii.kurochko 2024-07-23 15:49 ` Julien Grall 2024-07-23 17:25 ` oleksii.kurochko 2024-07-23 17:28 ` oleksii.kurochko 2024-07-23 18:44 ` Julien Grall 2024-07-12 16:22 ` [PATCH v2 6/8] xen/riscv: introduce generic Xen page table handling Oleksii Kurochko 2024-07-21 9:02 ` Julien Grall 2024-07-22 6:59 ` Jan Beulich 2024-07-22 14:51 ` Oleksii 2024-07-12 16:22 ` [PATCH v2 7/8] xen/riscv: select CONFIG_GENREIC_PT Oleksii Kurochko 2024-07-12 16:22 ` [PATCH v2 8/8] xen/riscv: introduce early_fdt_map() Oleksii Kurochko 2024-07-15 8:52 ` Jan Beulich 2024-07-15 13:58 ` Oleksii 2024-07-15 14:58 ` Jan Beulich 2024-07-15 15:00 ` Jan Beulich
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.