* [PATCH 31/36] arm: Implement thread_struct whitelist for hardened usercopy
From: Kees Cook @ 2018-01-09 20:56 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515531365-37423-1-git-send-email-keescook@chromium.org>
ARM does not carry FPU state in the thread structure, so it can declare
no usercopy whitelist at all.
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc: linux-arm-kernel at lists.infradead.org
Signed-off-by: Kees Cook <keescook@chromium.org>
---
arch/arm/Kconfig | 1 +
arch/arm/include/asm/processor.h | 7 +++++++
2 files changed, 8 insertions(+)
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 51c8df561077..3ea00d65f35d 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -50,6 +50,7 @@ config ARM
select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU
select HAVE_ARCH_MMAP_RND_BITS if MMU
select HAVE_ARCH_SECCOMP_FILTER if (AEABI && !OABI_COMPAT)
+ select HAVE_ARCH_THREAD_STRUCT_WHITELIST
select HAVE_ARCH_TRACEHOOK
select HAVE_ARM_SMCCC if CPU_V7
select HAVE_EBPF_JIT if !CPU_ENDIAN_BE32
diff --git a/arch/arm/include/asm/processor.h b/arch/arm/include/asm/processor.h
index 338cbe0a18ef..01a41be58d43 100644
--- a/arch/arm/include/asm/processor.h
+++ b/arch/arm/include/asm/processor.h
@@ -45,6 +45,13 @@ struct thread_struct {
struct debug_info debug;
};
+/* Nothing needs to be usercopy-whitelisted from thread_struct. */
+static inline void arch_thread_struct_whitelist(unsigned long *offset,
+ unsigned long *size)
+{
+ *offset = *size = 0;
+}
+
#define INIT_THREAD { }
#define start_thread(regs,pc,sp) \
--
2.7.4
^ permalink raw reply related
* [PATCH 30/36] arm64: Implement thread_struct whitelist for hardened usercopy
From: Kees Cook @ 2018-01-09 20:55 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515531365-37423-1-git-send-email-keescook@chromium.org>
This whitelists the FPU register state portion of the thread_struct for
copying to userspace, instead of the default entire structure.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: James Morse <james.morse@arm.com>
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc: Dave Martin <Dave.Martin@arm.com>
Cc: zijun_hu <zijun_hu@htc.com>
Cc: linux-arm-kernel at lists.infradead.org
Signed-off-by: Kees Cook <keescook@chromium.org>
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/processor.h | 8 ++++++++
2 files changed, 9 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a93339f5178f..c84477e6a884 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -90,6 +90,7 @@ config ARM64
select HAVE_ARCH_MMAP_RND_BITS
select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
select HAVE_ARCH_SECCOMP_FILTER
+ select HAVE_ARCH_THREAD_STRUCT_WHITELIST
select HAVE_ARCH_TRACEHOOK
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
select HAVE_ARCH_VMAP_STACK
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 023cacb946c3..e58a5864ec89 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -113,6 +113,14 @@ struct thread_struct {
struct debug_info debug; /* debugging */
};
+/* Whitelist the fpsimd_state for copying to userspace. */
+static inline void arch_thread_struct_whitelist(unsigned long *offset,
+ unsigned long *size)
+{
+ *offset = offsetof(struct thread_struct, fpsimd_state);
+ *size = sizeof(struct fpsimd_state);
+}
+
#ifdef CONFIG_COMPAT
#define task_user_tls(t) \
({ \
--
2.7.4
^ permalink raw reply related
* [PATCH v3 6/6] coresight: etm4x: Support panic kdump
From: Mathieu Poirier @ 2018-01-09 20:21 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1513844415-11427-7-git-send-email-leo.yan@linaro.org>
On Thu, Dec 21, 2017 at 04:20:15PM +0800, Leo Yan wrote:
> ETMv4 hardware information and configuration needs to be saved as
> metadata; these metadata should be compatible with tool 'perf' and
> can be used for tracing data analysis. ETMv4 usually works as tracer
> per CPU, we cannot wait to gather ETM info after the CPU has been panic
> and cannot execute dump operations for itself; so should gather
> metadata when the corresponding CPU is alive.
>
> Since values in TRCIDR{0, 1, 2, 8} and TRCAUTHSTATUS are read-only and
> won't change at the runtime. Those registers value are filled when
> tracers are instantiated.
>
> The configuration and control registers TRCCONFIGR and TRCTRACEIDR are
> dynamically configured, we record their value when enabling coresight
> path. When operating from sysFS tracer these two registers are recorded
> in etm4_enable_sysfs() and add kdump node into list, and remove the
> kdump node in etm4_disable_sysfs(). When operating from perf,
> etm_setup_aux() adds all tracers to the dump list and etm4_enable_perf()
> is used to record configuration registers and update dump buffer info,
> this can avoid unnecessary list addition and deletion operations.
> Removal of the tracers from the dump list is done in function
> free_event_data().
>
> Suggested-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> Signed-off-by: Leo Yan <leo.yan@linaro.org>
> ---
> drivers/hwtracing/coresight/coresight-etm-perf.c | 12 +++++++++++-
> drivers/hwtracing/coresight/coresight-etm4x.c | 23 +++++++++++++++++++++++
> drivers/hwtracing/coresight/coresight-etm4x.h | 15 +++++++++++++++
> 3 files changed, 49 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
> index 8a0ad77..fec779b 100644
> --- a/drivers/hwtracing/coresight/coresight-etm-perf.c
> +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
> @@ -137,6 +137,12 @@ static void free_event_data(struct work_struct *work)
> }
>
> for_each_cpu(cpu, mask) {
> + struct coresight_device *csdev;
> +
> + csdev = per_cpu(csdev_src, cpu);
> + if (csdev)
> + coresight_kdump_del(csdev);
> +
> if (!(IS_ERR_OR_NULL(event_data->path[cpu])))
> coresight_release_path(event_data->path[cpu]);
> }
> @@ -195,7 +201,7 @@ static void etm_free_aux(void *data)
> static void *etm_setup_aux(int event_cpu, void **pages,
> int nr_pages, bool overwrite)
> {
> - int cpu;
> + int cpu, ret;
> cpumask_t *mask;
> struct coresight_device *sink;
> struct etm_event_data *event_data = NULL;
> @@ -238,6 +244,10 @@ static void *etm_setup_aux(int event_cpu, void **pages,
> event_data->path[cpu] = coresight_build_path(csdev, sink);
> if (IS_ERR(event_data->path[cpu]))
> goto err;
> +
> + ret = coresight_kdump_add(csdev, cpu);
Aren't you missing the configuration for trcconfigr and trctraceidr?
> + if (ret)
> + goto err;
> }
>
> if (!sink_ops(sink)->alloc_buffer)
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c b/drivers/hwtracing/coresight/coresight-etm4x.c
> index cf364a5..cbde398 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x.c
> @@ -258,10 +258,19 @@ static int etm4_enable_perf(struct coresight_device *csdev,
> static int etm4_enable_sysfs(struct coresight_device *csdev)
> {
> struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
> + struct etmv4_config *config = &drvdata->config;
> + struct etmv4_metadata *metadata = &drvdata->metadata;
> int ret;
>
> spin_lock(&drvdata->spinlock);
>
> + /* Update meta data and add into kdump list */
> + metadata->trcconfigr = config->cfg;
> + metadata->trctraceidr = drvdata->trcid;
> +
> + coresight_kdump_add(csdev, drvdata->cpu);
> + coresight_kdump_update(csdev, (char *)metadata, sizeof(*metadata));
> +
> /*
> * Executing etm4_enable_hw on the cpu whose ETM is being enabled
> * ensures that register writes occur when cpu is powered.
> @@ -384,6 +393,9 @@ static void etm4_disable_sysfs(struct coresight_device *csdev)
> */
> smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1);
>
> + /* Delete from kdump list */
> + coresight_kdump_del(csdev);
> +
> spin_unlock(&drvdata->spinlock);
> cpus_read_unlock();
>
> @@ -438,6 +450,7 @@ static void etm4_init_arch_data(void *info)
> u32 etmidr4;
> u32 etmidr5;
> struct etmv4_drvdata *drvdata = info;
> + struct etmv4_metadata *metadata = &drvdata->metadata;
>
> /* Make sure all registers are accessible */
> etm4_os_unlock(drvdata);
> @@ -590,6 +603,16 @@ static void etm4_init_arch_data(void *info)
> drvdata->nrseqstate = BMVAL(etmidr5, 25, 27);
> /* NUMCNTR, bits[30:28] number of counters available for tracing */
> drvdata->nr_cntr = BMVAL(etmidr5, 28, 30);
> +
> + /* Update metadata */
> + metadata->magic = ETM4_METADATA_MAGIC;
> + metadata->cpu = drvdata->cpu;
> + metadata->trcidr0 = readl_relaxed(drvdata->base + TRCIDR0);
> + metadata->trcidr1 = readl_relaxed(drvdata->base + TRCIDR1);
> + metadata->trcidr2 = readl_relaxed(drvdata->base + TRCIDR2);
> + metadata->trcidr8 = readl_relaxed(drvdata->base + TRCIDR8);
> + metadata->trcauthstatus = readl_relaxed(drvdata->base + TRCAUTHSTATUS);
> +
> CS_LOCK(drvdata->base);
> }
>
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h
> index b3b5ea7..08dc8b7 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x.h
> +++ b/drivers/hwtracing/coresight/coresight-etm4x.h
> @@ -198,6 +198,20 @@
> #define ETM_EXLEVEL_NS_HYP BIT(14)
> #define ETM_EXLEVEL_NS_NA BIT(15)
>
> +#define ETM4_METADATA_MAGIC 0x4040404040404040ULL
This is a duplicate of the magic value found in cs-etm.h but I'm not sure of
what we'll do about that. It is probably time to come up
with a shared file between the kernel and the perf tools, just like
coresight-pmu.h. You can have a stab at it or concentrate on my previous
comments for now - it's entirely up to you.
> +
> +struct etmv4_metadata {
> + u64 magic;
> + u64 cpu;
> + u64 trcconfigr;
> + u64 trctraceidr;
> + u64 trcidr0;
> + u64 trcidr1;
> + u64 trcidr2;
> + u64 trcidr8;
> + u64 trcauthstatus;
> +};
Same here... This is a duplicate of struct etmv4_drvdata. Again not sure about
the best way to handle this. I'll think about it.
> +
> /**
> * struct etmv4_config - configuration information related to an ETMv4
> * @mode: Controls various modes supported by this ETM.
> @@ -393,6 +407,7 @@ struct etmv4_drvdata {
> bool atbtrig;
> bool lpoverride;
> struct etmv4_config config;
> + struct etmv4_metadata metadata;
> };
>
> /* Address comparator access types */
> --
> 2.7.4
>
^ permalink raw reply
* [PATCH 3/4] bcm2835-gpio-exp: Driver for GPIO expander via mailbox service
From: Stefan Wahren @ 2018-01-09 20:15 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109134135.32i6slkhmqsjnmne@tarshish>
Hi Baruch,
> Baruch Siach <baruch@tkos.co.il> hat am 9. Januar 2018 um 14:41 geschrieben:
>
>
> Hi Stefan,
>
> On Tue, Jan 02, 2018 at 07:49:44PM +0100, Stefan Wahren wrote:
>
> [...]
>
> > > + ret = rpi_firmware_property(gpio->fw, RPI_FIRMWARE_GET_GPIO_CONFIG,
> > > + &get, sizeof(get));
> > > + if (ret) {
> > > + dev_err(gpio->dev,
> > > + "Failed to get GPIO %u config (%d)\n", off, ret);
> > > + return ret;
> > > + }
> >
> > Shouldn't we also check the in-bound status at get.gpio?
>
> What is the in-bound status value? May you refer me to the documentation?
unfortunately this mailbox property isn't documented here [1]. So please refer to Phil's explanation [2].
I assume gpio_get_config.gpio of the response would be 0x80000000 in case of an invalid request. But i didn't test it yet.
This was my solution [3] for the RPI_FIRMWARE_GET_CUSTOMER_OTP. But it hadn't been reviewed yet.
Stefan
[1] - https://github.com/raspberrypi/firmware/wiki/Mailbox-property-interface
[2] - http://lists.infradead.org/pipermail/linux-rpi-kernel/2018-January/007237.html
[3] - https://github.com/lategoodbye/rpi-zero/blob/bcm2835-otp/drivers/nvmem/raspberrypi-otp.c#L61
>
> > And in all the other gpio ops?
>
> Thanks,
> baruch
>
> --
> http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
> =}------------------------------------------------ooO--U--Ooo------------{=
> - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH v2 0/8] arm64: 52-bit physical address support
From: Kristina Martsenko @ 2018-01-09 19:33 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20171222152307.11252-1-catalin.marinas@arm.com>
On 22/12/17 15:22, Catalin Marinas wrote:
> Hi,
>
> That's v2 of Kristina's 52-bit PA series, posted here:
>
> http://lkml.kernel.org/r/1513184845-8711-1-git-send-email-kristina.martsenko at arm.com
>
> I addressed the comments raised on the list and I plan to push it into
> -next soon.
>
> Changes in v2:
>
> - Folded patches 7 and 8 from the original series into 1
> - Definitions for TCR_IPS_*
> - Renamed some asm macros and functions
> - __create_hyp_mappings() changed to avoid passing an extra arg
> - More code comments
> - Added Reviewed/Tested tags I've got so far
Apart from the few things I pointed out, the rest of the changes look
good to me. Thanks.
Kristina
>
> Thanks,
>
> Catalin
>
> Kristina Martsenko (8):
> arm64: add kconfig symbol to configure physical address size
> arm64: limit PA size to supported range
> arm64: handle 52-bit addresses in TTBR
> arm64: head.S: handle 52-bit PAs in PTEs in early page table setup
> arm64: don't open code page table entry creation
> arm64: handle 52-bit physical addresses in page table entries
> arm64: allow ID map to be extended to 52 bits
> arm64: enable 52-bit physical address support
>
> arch/arm/include/asm/kvm_mmu.h | 7 ++
> arch/arm64/Kconfig | 29 ++++++++
> arch/arm64/include/asm/assembler.h | 36 +++++++++-
> arch/arm64/include/asm/kvm_mmu.h | 21 +++++-
> arch/arm64/include/asm/mmu_context.h | 20 ++++--
> arch/arm64/include/asm/pgalloc.h | 6 +-
> arch/arm64/include/asm/pgtable-hwdef.h | 25 ++++++-
> arch/arm64/include/asm/pgtable.h | 55 ++++++++++++---
> arch/arm64/include/asm/sparsemem.h | 2 +-
> arch/arm64/include/asm/sysreg.h | 8 +++
> arch/arm64/kernel/head.S | 122 +++++++++++++++++++++------------
> arch/arm64/kernel/hibernate-asm.S | 12 ++--
> arch/arm64/kernel/hibernate.c | 5 +-
> arch/arm64/kvm/hyp-init.S | 26 ++++---
> arch/arm64/kvm/hyp/s2-setup.c | 2 +
> arch/arm64/mm/mmu.c | 15 ++--
> arch/arm64/mm/pgd.c | 8 +++
> arch/arm64/mm/proc.S | 19 ++---
> virt/kvm/arm/arm.c | 2 +-
> virt/kvm/arm/mmu.c | 10 ++-
> 20 files changed, 323 insertions(+), 107 deletions(-)
>
^ permalink raw reply
* [PATCH v2 2/8] arm64: limit PA size to supported range
From: Kristina Martsenko @ 2018-01-09 19:32 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20171222152307.11252-3-catalin.marinas@arm.com>
On 22/12/17 15:23, Catalin Marinas wrote:
> From: Kristina Martsenko <kristina.martsenko@arm.com>
>
> We currently copy the physical address size from
> ID_AA64MMFR0_EL1.PARange directly into TCR.(I)PS. This will not work for
> 4k and 16k granule kernels on systems that support 52-bit physical
> addresses, since 52-bit addresses are only permitted with the 64k
> granule.
>
> To fix this, fall back to 48 bits when configuring the PA size when the
> kernel does not support 52-bit PAs. When it does, fall back to 52, to
> avoid similar problems in the future if the PA size is ever increased
> above 52.
>
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
> Tested-by: Bob Picco <bob.picco@oracle.com>
> Reviewed-by: Bob Picco <bob.picco@oracle.com>
> Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
> [catalin.marinas at arm.com: tcr_set_pa_size macro renamed to tcr_compute_pa_size]
> [catalin.marinas at arm.com: comments added to tcr_compute_pa_size]
> [catalin.marinas at arm.com: definitions added for TCR_*PS_SHIFT]
> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
> ---
> arch/arm64/include/asm/assembler.h | 18 ++++++++++++++++++
> arch/arm64/include/asm/pgtable-hwdef.h | 2 ++
> arch/arm64/include/asm/sysreg.h | 8 ++++++++
> arch/arm64/kvm/hyp-init.S | 6 ++----
> arch/arm64/kvm/hyp/s2-setup.c | 2 ++
> arch/arm64/mm/proc.S | 6 ++----
> 6 files changed, 34 insertions(+), 8 deletions(-)
>
> diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
> index aef72d886677..04a92307e6c1 100644
> --- a/arch/arm64/include/asm/assembler.h
> +++ b/arch/arm64/include/asm/assembler.h
> @@ -351,6 +351,24 @@ alternative_endif
> .endm
>
> /*
> + * tcr_compute_pa_size - set TCR.(I)PS to the highest supported
> + * ID_AA64MMFR0_EL1.PARange value
> + *
> + * tcr: register with the TCR_ELx value to be updated
> + * pos: PARange bitfield position
This should be "(I)PS bitfield position" or something like that.
(Also, regarding the name change tcr_set_pa_size -> tcr_compute_pa_size,
note that there is a similar macro "tcr_set_idmap_t0sz" just above. But
I don't mind either way.)
Kristina
> + * tmp{0,1}: temporary registers
> + */
> + .macro tcr_compute_pa_size, tcr, pos, tmp0, tmp1
> + mrs \tmp0, ID_AA64MMFR0_EL1
> + // Narrow PARange to fit the PS field in TCR_ELx
> + ubfx \tmp0, \tmp0, #ID_AA64MMFR0_PARANGE_SHIFT, #3
> + mov \tmp1, #ID_AA64MMFR0_PARANGE_MAX
> + cmp \tmp0, \tmp1
> + csel \tmp0, \tmp1, \tmp0, hi
> + bfi \tcr, \tmp0, \pos, #3
> + .endm
> +
> +/*
> * Macro to perform a data cache maintenance for the interval
> * [kaddr, kaddr + size)
> *
> diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
> index c1de9f67980b..9be2e9371c52 100644
> --- a/arch/arm64/include/asm/pgtable-hwdef.h
> +++ b/arch/arm64/include/asm/pgtable-hwdef.h
> @@ -272,6 +272,8 @@
> #define TCR_TG1_4K (UL(2) << TCR_TG1_SHIFT)
> #define TCR_TG1_64K (UL(3) << TCR_TG1_SHIFT)
>
> +#define TCR_IPS_SHIFT 32
> +#define TCR_IPS_MASK (UL(7) << TCR_IPS_SHIFT)
> #define TCR_ASID16 (UL(1) << 36)
> #define TCR_TBI0 (UL(1) << 37)
> #define TCR_HA (UL(1) << 39)
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 08cc88574659..ec144f480b39 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -471,6 +471,14 @@
> #define ID_AA64MMFR0_TGRAN64_SUPPORTED 0x0
> #define ID_AA64MMFR0_TGRAN16_NI 0x0
> #define ID_AA64MMFR0_TGRAN16_SUPPORTED 0x1
> +#define ID_AA64MMFR0_PARANGE_48 0x5
> +#define ID_AA64MMFR0_PARANGE_52 0x6
> +
> +#ifdef CONFIG_ARM64_PA_BITS_52
> +#define ID_AA64MMFR0_PARANGE_MAX ID_AA64MMFR0_PARANGE_52
> +#else
> +#define ID_AA64MMFR0_PARANGE_MAX ID_AA64MMFR0_PARANGE_48
> +#endif
>
> /* id_aa64mmfr1 */
> #define ID_AA64MMFR1_PAN_SHIFT 20
> diff --git a/arch/arm64/kvm/hyp-init.S b/arch/arm64/kvm/hyp-init.S
> index 3f9615582377..e2d1fe03662a 100644
> --- a/arch/arm64/kvm/hyp-init.S
> +++ b/arch/arm64/kvm/hyp-init.S
> @@ -90,11 +90,9 @@ __do_hyp_init:
> bfi x4, x5, TCR_T0SZ_OFFSET, TCR_TxSZ_WIDTH
> #endif
> /*
> - * Read the PARange bits from ID_AA64MMFR0_EL1 and set the PS bits in
> - * TCR_EL2.
> + * Set the PS bits in TCR_EL2.
> */
> - mrs x5, ID_AA64MMFR0_EL1
> - bfi x4, x5, #16, #3
> + tcr_compute_pa_size x4, #TCR_EL2_PS_SHIFT, x5, x6
>
> msr tcr_el2, x4
>
> diff --git a/arch/arm64/kvm/hyp/s2-setup.c b/arch/arm64/kvm/hyp/s2-setup.c
> index a81f5e10fc8c..603e1ee83e89 100644
> --- a/arch/arm64/kvm/hyp/s2-setup.c
> +++ b/arch/arm64/kvm/hyp/s2-setup.c
> @@ -32,6 +32,8 @@ u32 __hyp_text __init_stage2_translation(void)
> * PS is only 3. Fortunately, bit 19 is RES0 in VTCR_EL2...
> */
> parange = read_sysreg(id_aa64mmfr0_el1) & 7;
> + if (parange > ID_AA64MMFR0_PARANGE_MAX)
> + parange = ID_AA64MMFR0_PARANGE_MAX;
> val |= parange << 16;
>
> /* Compute the actual PARange... */
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 95233dfc4c39..4f133cb340dc 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -228,11 +228,9 @@ ENTRY(__cpu_setup)
> tcr_set_idmap_t0sz x10, x9
>
> /*
> - * Read the PARange bits from ID_AA64MMFR0_EL1 and set the IPS bits in
> - * TCR_EL1.
> + * Set the IPS bits in TCR_EL1.
> */
> - mrs x9, ID_AA64MMFR0_EL1
> - bfi x10, x9, #32, #3
> + tcr_compute_pa_size x10, #TCR_IPS_SHIFT, x5, x6
> #ifdef CONFIG_ARM64_HW_AFDBM
> /*
> * Hardware update of the Access and Dirty bits.
>
^ permalink raw reply
* [PATCH v2 7/8] arm64: allow ID map to be extended to 52 bits
From: Kristina Martsenko @ 2018-01-09 19:31 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <a53c8d88-18c3-be49-91a1-5e2ff93b277a@arm.com>
On 22/12/17 16:34, Suzuki K Poulose wrote:
> On 22/12/17 15:23, Catalin Marinas wrote:
>> From: Kristina Martsenko <kristina.martsenko@arm.com>
>>
>> Currently, when using VA_BITS < 48, if the ID map text happens to be
>> placed in physical memory above VA_BITS, we increase the VA size (up to
>> 48) and create a new table level, in order to map in the ID map text.
>> This is okay because the system always supports 48 bits of VA.
>>
>> This patch extends the code such that if the system supports 52 bits of
>> VA, and the ID map text is placed that high up, then we increase the VA
>> size accordingly, up to 52.
>>
>> One difference from the current implementation is that so far the
>> condition of VA_BITS < 48 has meant that the top level table is always
>> "full", with the maximum number of entries, and an extra table level is
>> always needed. Now, when VA_BITS = 48 (and using 64k pages), the top
>> level table is not full, and we simply need to increase the number of
>> entries in it, instead of creating a new table level.
>>
>> Tested-by: Bob Picco <bob.picco@oracle.com>
>> Reviewed-by: Bob Picco <bob.picco@oracle.com>
>> Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
>> [catalin.marinas at arm.com: reduce arguments to __create_hyp_mappings()]
>> [catalin.marinas at arm.com: reworked/renamed __cpu_uses_extended_idmap_level()]
>> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
>> ---
>> ? arch/arm/include/asm/kvm_mmu.h?????? |? 5 +++
>> ? arch/arm64/include/asm/assembler.h?? |? 2 -
>> ? arch/arm64/include/asm/kvm_mmu.h???? |? 7 +++-
>> ? arch/arm64/include/asm/mmu_context.h | 18 +++++++--
>> ? arch/arm64/kernel/head.S???????????? | 76 +++++++++++++++++++++---------------
>> ? arch/arm64/kvm/hyp-init.S??????????? | 17 ++++----
>> ? arch/arm64/mm/mmu.c????????????????? |? 1 +
>> ? virt/kvm/arm/mmu.c?????????????????? | 10 ++++-
>> ? 8 files changed, 87 insertions(+), 49 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
>> index 8dbec683638b..8c5643e2eea4 100644
>> --- a/arch/arm/include/asm/kvm_mmu.h
>> +++ b/arch/arm/include/asm/kvm_mmu.h
>> @@ -211,6 +211,11 @@ static inline bool __kvm_cpu_uses_extended_idmap(void)
>> ????? return false;
>> ? }
>> ? +static inline unsigned long __kvm_idmap_ptrs_per_pgd(void)
>> +{
>> +??? return PTRS_PER_PGD;
>> +}
>> +
>> ? static inline void __kvm_extend_hypmap(pgd_t *boot_hyp_pgd,
>> ???????????????????????? pgd_t *hyp_pgd,
>> ???????????????????????? pgd_t *merged_hyp_pgd,
>> diff --git a/arch/arm64/include/asm/assembler.h
>> b/arch/arm64/include/asm/assembler.h
>> index 49ea3def4bd1..942fdb5ef0ad 100644
>> --- a/arch/arm64/include/asm/assembler.h
>> +++ b/arch/arm64/include/asm/assembler.h
>> @@ -344,10 +344,8 @@ alternative_endif
>> ?? * tcr_set_idmap_t0sz - update TCR.T0SZ so that we can load the ID map
>> ?? */
>> ????? .macro??? tcr_set_idmap_t0sz, valreg, tmpreg
>> -#ifndef CONFIG_ARM64_VA_BITS_48
>> ????? ldr_l??? \tmpreg, idmap_t0sz
>> ????? bfi??? \valreg, \tmpreg, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH
>> -#endif
>> ????? .endm
>> ? ? /*
>> diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
>> index b3f7b68b042d..8d663ca0d50c 100644
>> --- a/arch/arm64/include/asm/kvm_mmu.h
>> +++ b/arch/arm64/include/asm/kvm_mmu.h
>> @@ -273,7 +273,12 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled);
>> ? ? static inline bool __kvm_cpu_uses_extended_idmap(void)
>> ? {
>> -??? return __cpu_uses_extended_idmap();
>> +??? return __cpu_uses_extended_idmap_table();
>> +}
>> +
>> +static inline unsigned long __kvm_idmap_ptrs_per_pgd(void)
>> +{
>> +??? return idmap_ptrs_per_pgd;
>> ? }
>> ? ? /*
>> diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h
>> index accc2ff32a0e..7991718890c6 100644
>> --- a/arch/arm64/include/asm/mmu_context.h
>> +++ b/arch/arm64/include/asm/mmu_context.h
>> @@ -63,11 +63,21 @@ static inline void cpu_set_reserved_ttbr0(void)
>> ?? * physical memory, in which case it will be smaller.
>> ?? */
>> ? extern u64 idmap_t0sz;
>> +extern u64 idmap_ptrs_per_pgd;
>> ? -static inline bool __cpu_uses_extended_idmap(void)
>> +static inline bool __cpu_uses_extended_idmap_level(void)
>> ? {
>> -??? return (!IS_ENABLED(CONFIG_ARM64_VA_BITS_48) &&
>> -??????? unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS)));
>> +??? return ARM64_HW_PGTABLE_LEVELS((64 - idmap_t0sz)) > CONFIG_PGTABLE_LEVELS;
>> +}
>> +
>> +/*
>> + * True if the extended ID map requires an extra level of translation
>> table
>> + * to be configured.
>> + */
>> +static inline bool __cpu_uses_extended_idmap_table(void)
>> +{
>> +??? return __cpu_uses_extended_idmap_level() &&
>> +??????? (idmap_ptrs_per_pgd == PTRS_PER_PGD);
>> ? }
>
> As discussed offline, I was talking about changing
>
> ?__cpu_uses_extended_idmap_table =>? __cpu_uses_extended_idmap_level.
>
> And the __cpu_uses_extended_idmap() doesn't need any changes. i.e :
> It could look like :
>
> static inline bool __cpu_uses_extended_idmap(void)
> {
> ????return (!IS_ENABLED(CONFIG_ARM64_VA_BITS_48) &&
> ??????? unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS)));
> }
The changes to __cpu_uses_extended_idmap_level (below) look good to me,
but it seems that __cpu_uses_extended_idmap (above) has mistakenly been
changed too. It should look like this, as it was in v1 of this series:
static inline bool __cpu_uses_extended_idmap(void)
{
return unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS));
}
With the current code, the kernel fails to boot when VA_BITS = 48 and
the idmap is in 52-bit memory.
>
> static inline bool __cpu_uses_extended_idmap_level(void)
> {
> ????return ARM64_HW_PGTABLE_LEVELS((64 - idmap_t0sz)) > CONFIG_PGTABLE_LEVELS;
(double parentheses?)
Thanks,
Kristina
> }
>
> And the __kvm_cpu_uses_extended_idmap() above should use the
>
> __cpu_uses_extended_idmap_level().
>
> Sorry for the confusion.
>
>
> With that:
>
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
^ permalink raw reply
* [PATCH v2 7/8] arm64: allow ID map to be extended to 52 bits
From: Kristina Martsenko @ 2018-01-09 19:29 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20171222152307.11252-8-catalin.marinas@arm.com>
On 22/12/17 15:23, Catalin Marinas wrote:
> From: Kristina Martsenko <kristina.martsenko@arm.com>
>
> Currently, when using VA_BITS < 48, if the ID map text happens to be
> placed in physical memory above VA_BITS, we increase the VA size (up to
> 48) and create a new table level, in order to map in the ID map text.
> This is okay because the system always supports 48 bits of VA.
>
> This patch extends the code such that if the system supports 52 bits of
> VA, and the ID map text is placed that high up, then we increase the VA
> size accordingly, up to 52.
>
> One difference from the current implementation is that so far the
> condition of VA_BITS < 48 has meant that the top level table is always
> "full", with the maximum number of entries, and an extra table level is
> always needed. Now, when VA_BITS = 48 (and using 64k pages), the top
> level table is not full, and we simply need to increase the number of
> entries in it, instead of creating a new table level.
>
> Tested-by: Bob Picco <bob.picco@oracle.com>
> Reviewed-by: Bob Picco <bob.picco@oracle.com>
> Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
> [catalin.marinas at arm.com: reduce arguments to __create_hyp_mappings()]
> [catalin.marinas at arm.com: reworked/renamed __cpu_uses_extended_idmap_level()]
> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
> ---
> arch/arm/include/asm/kvm_mmu.h | 5 +++
> arch/arm64/include/asm/assembler.h | 2 -
> arch/arm64/include/asm/kvm_mmu.h | 7 +++-
> arch/arm64/include/asm/mmu_context.h | 18 +++++++--
> arch/arm64/kernel/head.S | 76 +++++++++++++++++++++---------------
> arch/arm64/kvm/hyp-init.S | 17 ++++----
> arch/arm64/mm/mmu.c | 1 +
> virt/kvm/arm/mmu.c | 10 ++++-
> 8 files changed, 87 insertions(+), 49 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
> index 8dbec683638b..8c5643e2eea4 100644
> --- a/arch/arm/include/asm/kvm_mmu.h
> +++ b/arch/arm/include/asm/kvm_mmu.h
> @@ -211,6 +211,11 @@ static inline bool __kvm_cpu_uses_extended_idmap(void)
> return false;
> }
>
> +static inline unsigned long __kvm_idmap_ptrs_per_pgd(void)
> +{
> + return PTRS_PER_PGD;
> +}
> +
> static inline void __kvm_extend_hypmap(pgd_t *boot_hyp_pgd,
> pgd_t *hyp_pgd,
> pgd_t *merged_hyp_pgd,
> diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
> index 49ea3def4bd1..942fdb5ef0ad 100644
> --- a/arch/arm64/include/asm/assembler.h
> +++ b/arch/arm64/include/asm/assembler.h
> @@ -344,10 +344,8 @@ alternative_endif
> * tcr_set_idmap_t0sz - update TCR.T0SZ so that we can load the ID map
> */
> .macro tcr_set_idmap_t0sz, valreg, tmpreg
> -#ifndef CONFIG_ARM64_VA_BITS_48
> ldr_l \tmpreg, idmap_t0sz
> bfi \valreg, \tmpreg, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH
> -#endif
> .endm
>
> /*
> diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
> index b3f7b68b042d..8d663ca0d50c 100644
> --- a/arch/arm64/include/asm/kvm_mmu.h
> +++ b/arch/arm64/include/asm/kvm_mmu.h
> @@ -273,7 +273,12 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool was_enabled);
>
> static inline bool __kvm_cpu_uses_extended_idmap(void)
> {
> - return __cpu_uses_extended_idmap();
> + return __cpu_uses_extended_idmap_table();
> +}
> +
> +static inline unsigned long __kvm_idmap_ptrs_per_pgd(void)
> +{
> + return idmap_ptrs_per_pgd;
> }
>
> /*
> diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h
> index accc2ff32a0e..7991718890c6 100644
> --- a/arch/arm64/include/asm/mmu_context.h
> +++ b/arch/arm64/include/asm/mmu_context.h
> @@ -63,11 +63,21 @@ static inline void cpu_set_reserved_ttbr0(void)
> * physical memory, in which case it will be smaller.
> */
> extern u64 idmap_t0sz;
> +extern u64 idmap_ptrs_per_pgd;
>
> -static inline bool __cpu_uses_extended_idmap(void)
> +static inline bool __cpu_uses_extended_idmap_level(void)
> {
> - return (!IS_ENABLED(CONFIG_ARM64_VA_BITS_48) &&
> - unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS)));
> + return ARM64_HW_PGTABLE_LEVELS((64 - idmap_t0sz)) > CONFIG_PGTABLE_LEVELS;
> +}
> +
> +/*
> + * True if the extended ID map requires an extra level of translation table
> + * to be configured.
> + */
> +static inline bool __cpu_uses_extended_idmap_table(void)
> +{
> + return __cpu_uses_extended_idmap_level() &&
> + (idmap_ptrs_per_pgd == PTRS_PER_PGD);
> }
>
> /*
> @@ -77,7 +87,7 @@ static inline void __cpu_set_tcr_t0sz(unsigned long t0sz)
> {
> unsigned long tcr;
>
> - if (!__cpu_uses_extended_idmap())
> + if (!__cpu_uses_extended_idmap_level())
> return;
>
> tcr = read_sysreg(tcr_el1);
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index eeec0001e204..66f01869e97c 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -176,7 +176,7 @@ ENDPROC(preserve_boot_args)
> * ptrs: #imm pointers per table page
> *
> * Preserves: virt
> - * Corrupts: tmp1, tmp2
> + * Corrupts: ptrs, tmp1, tmp2
> * Returns: tbl -> next level table page address
> */
> .macro create_table_entry, tbl, virt, shift, ptrs, tmp1, tmp2
> @@ -184,7 +184,8 @@ ENDPROC(preserve_boot_args)
> phys_to_pte \tmp1, \tmp2
> orr \tmp2, \tmp2, #PMD_TYPE_TABLE // address of next table and entry type
> lsr \tmp1, \virt, #\shift
> - and \tmp1, \tmp1, #\ptrs - 1 // table index
> + sub \ptrs, \ptrs, #1
> + and \tmp1, \tmp1, \ptrs // table index
> str \tmp2, [\tbl, \tmp1, lsl #3]
> add \tbl, \tbl, #PAGE_SIZE // next level table page
> .endm
> @@ -194,15 +195,17 @@ ENDPROC(preserve_boot_args)
> * block entry in the next level (tbl) for the given virtual address.
> *
> * Preserves: tbl, next, virt
> - * Corrupts: tmp1, tmp2
> + * Corrupts: ptrs_per_pgd, tmp1, tmp2
> */
> - .macro create_pgd_entry, tbl, virt, tmp1, tmp2
> - create_table_entry \tbl, \virt, PGDIR_SHIFT, PTRS_PER_PGD, \tmp1, \tmp2
> + .macro create_pgd_entry, tbl, virt, ptrs_per_pgd, tmp1, tmp2
> + create_table_entry \tbl, \virt, PGDIR_SHIFT, \ptrs_per_pgd, \tmp1, \tmp2
> #if SWAPPER_PGTABLE_LEVELS > 3
> - create_table_entry \tbl, \virt, PUD_SHIFT, PTRS_PER_PUD, \tmp1, \tmp2
> + mov \ptrs_per_pgd, PTRS_PER_PUD
> + create_table_entry \tbl, \virt, PUD_SHIFT, \ptrs_per_pgd, \tmp1, \tmp2
> #endif
> #if SWAPPER_PGTABLE_LEVELS > 2
> - create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, PTRS_PER_PTE, \tmp1, \tmp2
> + mov \ptrs_per_pgd, PTRS_PER_PTE
> + create_table_entry \tbl, \virt, SWAPPER_TABLE_SHIFT, \ptrs_per_pgd, \tmp1, \tmp2
> #endif
> .endm
>
> @@ -266,26 +269,13 @@ __create_page_tables:
> adrp x0, idmap_pg_dir
> adrp x3, __idmap_text_start // __pa(__idmap_text_start)
>
> -#ifndef CONFIG_ARM64_VA_BITS_48
> -#define EXTRA_SHIFT (PGDIR_SHIFT + PAGE_SHIFT - 3)
> -#define EXTRA_PTRS (1 << (48 - EXTRA_SHIFT))
> -
> - /*
> - * If VA_BITS < 48, it may be too small to allow for an ID mapping to be
> - * created that covers system RAM if that is located sufficiently high
> - * in the physical address space. So for the ID map, use an extended
> - * virtual range in that case, by configuring an additional translation
> - * level.
> - * First, we have to verify our assumption that the current value of
> - * VA_BITS was chosen such that all translation levels are fully
> - * utilised, and that lowering T0SZ will always result in an additional
> - * translation level to be configured.
> - */
> -#if VA_BITS != EXTRA_SHIFT
> -#error "Mismatch between VA_BITS and page size/number of translation levels"
> -#endif
> -
> /*
> + * VA_BITS may be too small to allow for an ID mapping to be created
> + * that covers system RAM if that is located sufficiently high in the
> + * physical address space. So for the ID map, use an extended virtual
> + * range in that case, and configure an additional translation level
> + * if needed.
> + *
> * Calculate the maximum allowed value for TCR_EL1.T0SZ so that the
> * entire ID map region can be mapped. As T0SZ == (64 - #bits used),
> * this number conveniently equals the number of leading zeroes in
> @@ -294,18 +284,41 @@ __create_page_tables:
> adrp x5, __idmap_text_end
> clz x5, x5
> cmp x5, TCR_T0SZ(VA_BITS) // default T0SZ small enough?
> - b.ge 1f // .. then skip additional level
> + b.ge 1f // .. then skip VA range extension
>
> adr_l x6, idmap_t0sz
> str x5, [x6]
> dmb sy
> dc ivac, x6 // Invalidate potentially stale cache line
>
> - create_table_entry x0, x3, EXTRA_SHIFT, EXTRA_PTRS, x5, x6
> -1:
> +#if (VA_BITS < 48)
> +#define EXTRA_SHIFT (PGDIR_SHIFT + PAGE_SHIFT - 3)
> +#define EXTRA_PTRS (1 << (PHYS_MASK_SHIFT - EXTRA_SHIFT))
> +
> + /*
> + * If VA_BITS < 48, we have to configure an additional table level.
> + * First, we have to verify our assumption that the current value of
> + * VA_BITS was chosen such that all translation levels are fully
> + * utilised, and that lowering T0SZ will always result in an additional
> + * translation level to be configured.
> + */
> +#if VA_BITS != EXTRA_SHIFT
> +#error "Mismatch between VA_BITS and page size/number of translation levels"
> #endif
>
> - create_pgd_entry x0, x3, x5, x6
> + mov x4, EXTRA_PTRS
> + create_table_entry x0, x3, EXTRA_SHIFT, x4, x5, x6
> +#else
> + /*
> + * If VA_BITS == 48, we don't have to configure an additional
> + * translation level, but the top-level table has more entries.
> + */
> + mov x4, #1 << (PHYS_MASK_SHIFT - PGDIR_SHIFT)
> + str_l x4, idmap_ptrs_per_pgd, x5
> +#endif
> +1:
> + ldr_l x4, idmap_ptrs_per_pgd
> + create_pgd_entry x0, x3, x4, x5, x6
> mov x5, x3 // __pa(__idmap_text_start)
> adr_l x6, __idmap_text_end // __pa(__idmap_text_end)
> create_block_map x0, x7, x3, x5, x6, x4
> @@ -316,7 +329,8 @@ __create_page_tables:
> adrp x0, swapper_pg_dir
> mov_q x5, KIMAGE_VADDR + TEXT_OFFSET // compile time __va(_text)
> add x5, x5, x23 // add KASLR displacement
> - create_pgd_entry x0, x5, x3, x6
> + mov x4, PTRS_PER_PGD
> + create_pgd_entry x0, x5, x4, x3, x6
> adrp x6, _end // runtime __pa(_end)
> adrp x3, _text // runtime __pa(_text)
> sub x6, x6, x3 // _end - _text
> diff --git a/arch/arm64/kvm/hyp-init.S b/arch/arm64/kvm/hyp-init.S
> index f9681cc00973..33c40b3eea01 100644
> --- a/arch/arm64/kvm/hyp-init.S
> +++ b/arch/arm64/kvm/hyp-init.S
> @@ -72,24 +72,23 @@ __do_hyp_init:
> mov x5, #TCR_EL2_RES1
> orr x4, x4, x5
>
> -#ifndef CONFIG_ARM64_VA_BITS_48
> /*
> - * If we are running with VA_BITS < 48, we may be running with an extra
> - * level of translation in the ID map. This is only the case if system
> - * RAM is out of range for the currently configured page size and number
> - * of translation levels, in which case we will also need the extra
> - * level for the HYP ID map, or we won't be able to enable the EL2 MMU.
> + * The ID map may be configured to use an extended virtual address
> + * range. This is only the case if system RAM is out of range for the
> + * currently configured page size and VA_BITS, in which case we will
> + * also need the extended virtual range for the HYP ID map, or we won't
> + * be able to enable the EL2 MMU.
> *
> * However, at EL2, there is only one TTBR register, and we can't switch
> * between translation tables *and* update TCR_EL2.T0SZ at the same
> - * time. Bottom line: we need the extra level in *both* our translation
> - * tables.
> + * time. Bottom line: we need to use the extended range with *both* our
> + * translation tables.
> *
> * So use the same T0SZ value we use for the ID map.
> */
> ldr_l x5, idmap_t0sz
> bfi x4, x5, TCR_T0SZ_OFFSET, TCR_TxSZ_WIDTH
> -#endif
> +
> /*
> * Set the PS bits in TCR_EL2.
> */
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 0c631a17ae1d..baa34418c3bf 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -50,6 +50,7 @@
> #define NO_CONT_MAPPINGS BIT(1)
>
> u64 idmap_t0sz = TCR_T0SZ(VA_BITS);
> +u64 idmap_ptrs_per_pgd = PTRS_PER_PGD;
>
> u64 kimage_voffset __ro_after_init;
> EXPORT_SYMBOL(kimage_voffset);
> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
> index b36945d49986..761787befd3b 100644
> --- a/virt/kvm/arm/mmu.c
> +++ b/virt/kvm/arm/mmu.c
> @@ -629,14 +629,20 @@ static int __create_hyp_mappings(pgd_t *pgdp,
> {
> pgd_t *pgd;
> pud_t *pud;
> - unsigned long addr, next;
> + unsigned long addr, next, ptrs_per_pgd = PTRS_PER_PGD;
> int err = 0;
>
> + /*
> + * If it's not the hyp_pgd, fall back to the kvm idmap layout.
> + */
> + if (pgdp != hyp_pgd)
> + ptrs_per_pgd = __kvm_idmap_ptrs_per_pgd();
This isn't right. When VA_BITS = 48 and the idmap is in 52-bit memory,
we come here with hyp_pgd, but need to use __kvm_idmap_ptrs_per_pgd. (We
don't have an extra table level, so we take the "else" case in
kvm_mmu_init. But we have a larger number of entries in the top-level
table, so we need __kvm_idmap_ptrs_per_pgd to mask in the extra bits.)
Currently a kernel with the above configuration (and VHE disabled) fails
to boot.
I'm not sure how else to push the changes into __create_hyp_mappings, as
Marc originally requested. One option might be to get rid of the mask
entirely, since the top bits of all hyp addresses should be zero anyway:
pgd = pgdp + (addr >> PGDIR_SHIFT);
but I'm not sure how nice that is. Any other ideas?
If not, we should put this back as it was in v1 of the series, i.e.
change the call sites instead.
Kristina
> +
> mutex_lock(&kvm_hyp_pgd_mutex);
> addr = start & PAGE_MASK;
> end = PAGE_ALIGN(end);
> do {
> - pgd = pgdp + pgd_index(addr);
> + pgd = pgdp + ((addr >> PGDIR_SHIFT) & (ptrs_per_pgd - 1));
>
> if (pgd_none(*pgd)) {
> pud = pud_alloc_one(NULL, addr);
>
^ permalink raw reply
* [PATCH] usb: dwc2: Fix endless deferral probe
From: Stefan Wahren @ 2018-01-09 19:28 UTC (permalink / raw)
To: linux-arm-kernel
The dwc2 USB driver tries to find a generic PHY first and then look
for an old style USB PHY. In case of a valid generic PHY node without
a PHY driver, the PHY layer will return -EPROBE_DEFER forever. So dwc2
will never tries for an USB PHY.
Fix this issue by finding a generic PHY and an old style USB PHY
at once.
Fixes: 6c2dad69163f ("usb: dwc2: Return errors from PHY")
Link: https://marc.info/?l=linux-usb&m=151518314314753&w=2
Signed-off-by: Stefan Wahren <stefan.wahren@i2se.com>
---
drivers/usb/dwc2/platform.c | 42 ++++++++++++++++++++++++------------------
1 file changed, 24 insertions(+), 18 deletions(-)
diff --git a/drivers/usb/dwc2/platform.c b/drivers/usb/dwc2/platform.c
index 3e26550..5279567 100644
--- a/drivers/usb/dwc2/platform.c
+++ b/drivers/usb/dwc2/platform.c
@@ -225,10 +225,11 @@ static int dwc2_lowlevel_hw_init(struct dwc2_hsotg *hsotg)
hsotg->phyif = GUSBCFG_PHYIF16;
/*
- * Attempt to find a generic PHY, then look for an old style
- * USB PHY and then fall back to pdata
+ * Attempt to find a generic PHY or an old style USB PHY at once
+ * otherwise fall back to pdata
*/
hsotg->phy = devm_phy_get(hsotg->dev, "usb2-phy");
+ hsotg->uphy = devm_usb_get_phy(hsotg->dev, USB_PHY_TYPE_USB2);
if (IS_ERR(hsotg->phy)) {
ret = PTR_ERR(hsotg->phy);
switch (ret) {
@@ -237,29 +238,34 @@ static int dwc2_lowlevel_hw_init(struct dwc2_hsotg *hsotg)
hsotg->phy = NULL;
break;
case -EPROBE_DEFER:
- return ret;
+ if (IS_ERR(hsotg->uphy))
+ return ret;
+
+ hsotg->phy = NULL;
+ break;
default:
dev_err(hsotg->dev, "error getting phy %d\n", ret);
return ret;
}
}
- if (!hsotg->phy) {
- hsotg->uphy = devm_usb_get_phy(hsotg->dev, USB_PHY_TYPE_USB2);
- if (IS_ERR(hsotg->uphy)) {
- ret = PTR_ERR(hsotg->uphy);
- switch (ret) {
- case -ENODEV:
- case -ENXIO:
- hsotg->uphy = NULL;
- break;
- case -EPROBE_DEFER:
- return ret;
- default:
- dev_err(hsotg->dev, "error getting usb phy %d\n",
- ret);
+ if (IS_ERR(hsotg->uphy)) {
+ ret = PTR_ERR(hsotg->uphy);
+ switch (ret) {
+ case -ENODEV:
+ case -ENXIO:
+ hsotg->uphy = NULL;
+ break;
+ case -EPROBE_DEFER:
+ if (!hsotg->phy)
return ret;
- }
+
+ hsotg->uphy = NULL;
+ break;
+ default:
+ dev_err(hsotg->dev, "error getting usb phy %d\n",
+ ret);
+ return ret;
}
}
--
2.7.4
^ permalink raw reply related
* [PATCH] arm64: mm: Add additional parameter to uaccess_ttbr0_enable
From: Christoffer Dall @ 2018-01-09 19:26 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <3e459188-f759-6f13-7ab4-1597c6da00a7@arm.com>
On Tue, Jan 09, 2018 at 01:53:56PM +0000, Marc Zyngier wrote:
> On 09/01/18 13:43, Christoffer Dall wrote:
> > Add an extra temporary register parameter to uaccess_ttbr0_enable which
> > is about to be required for arm64 PAN support.
> >
> > This patch doesn't introduce any functional change but ensures that the
> > kernel compiles once the KVM/ARM tree is merged with the arm64 tree by
> > ensuring a trivially mergable conflict.
> >
> > Cc: Will Deacon <will.deacon@arm.com>
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Maybe worth adding that this matches
> 27a921e75711d924617269e0ba4adb8bae9fd0d1 ("arm64: mm: Fix and re-enable
> ARM64_SW_TTBR0_PAN")?
Yes, definitely.
>
> Otherwise:
>
> Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Thanks.
I've pushed this to next. If anyone finds that objectionable, we can
always drop that patch again and pursue another resolution.
Thanks,
-Christoffer
^ permalink raw reply
* soc: imx: gpcv2: removing and probing fails
From: Stefan Agner @ 2018-01-09 19:26 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <1515509738.12538.35.camel@pengutronix.de>
On 2018-01-09 15:55, Lucas Stach wrote:
> Am Dienstag, den 09.01.2018, 15:44 +0100 schrieb Stefan Agner:
>> On 2018-01-09 15:24, Lucas Stach wrote:
>> > Am Sonntag, den 07.01.2018, 11:48 +0100 schrieb Stefan Agner:
>> > > Hi Andrew,
>> > >
>> > > I noticed that the driver fails when removing and probing again.
>> > > As far
>> > > as I can see due to duplicate add of the platform devices.
>> > >
>> > > As far as I can tell the driver should register the remove
>> > > callback and
>> > > do a platform_device_unregister on the newly created platform
>> > > devices.
>> > > However, as far as I can tell we don't hold on to a reference to
>> > > them...
>> > > I guess we could keep references in imx_gpcv2_probe, but maybe
>> > > there is
>> > > an easier way?
>> >
>> > The GPC v1 driver adds the necessary device dependency between the
>> > power domain devices and the GPC parent device. See the
>> > device_link_add() in imx_pgc_power_domain_probe().
>>
>> Note that despite device_link_add, GPC v1 seems to cause issue with
>> CONFIG_DEBUG_TEST_DRIVER_REMOVE=y:
>> https://marc.info/?l=linux-arm-kernel&m=151544599904423&w=4
>>
>> (sorry, I made it confusing, by adding a stack trace when using GPC
>> v1
>> in the gpcv2 thread...)
>
> IMHO this is an issue with the?CONFIG_DEBUG_TEST_DRIVER_REMOVE option,
> as it just blindly calls the remove callback instead of doing a proper
> __device_release_driver(). All the regular driver/device unbind paths
> will properly unbind the consumer devices before removing the driver.
>
There still seem to be an isse (gpc v1). I even removed PD consumers in
the device tree....
root at colibri-imx6:~# echo 20dc000.gpc >
/sys/bus/platform/drivers/imx-gpc/unbind
[ 18.821940] imx-pgc-pd imx-pgc-power-domain.0: Dropping the link to
20dc000.gpc
[ 18.833249] imx-pgc-pd imx-pgc-power-domain.1: Dropping the link to
20dc000.gpc
root at colibri-imx6:~# echo 20dc000.gpc >
/sys/bus/platform/drivers/imx-gpc/bind
[ 23.031945] ------------[ cut here ]------------
[ 23.036600] WARNING: CPU: 0 PID: 432 at fs/sysfs/dir.c:31
sysfs_warn_dup+0x64/0x74
[ 23.044251] sysfs: cannot create duplicate filename
'/devices/soc0/soc/2000000.aips-bus/20dc000.gpc/imx-pgc-power-domain.0'
[ 23.055434] Modules linked in:
[ 23.058505] CPU: 0 PID: 432 Comm: sh Not tainted
4.15.0-rc3-00061-g38f99f72e8f3-dirty #247
[ 23.066778] Hardware name: Freescale i.MX6 Quad/DualLite (Device
Tree)
[ 23.073332] [<8010f61c>] (unwind_backtrace) from [<8010b984>]
(show_stack+0x10/0x14)
[ 23.081094] [<8010b984>] (show_stack) from [<80831f84>]
(dump_stack+0x88/0x9c)
[ 23.088336] [<80831f84>] (dump_stack) from [<8011e7ac>]
(__warn+0xdc/0xf4)
[ 23.095226] [<8011e7ac>] (__warn) from [<8011e7fc>]
(warn_slowpath_fmt+0x38/0x48)
[ 23.102726] [<8011e7fc>] (warn_slowpath_fmt) from [<802768b0>]
(sysfs_warn_dup+0x64/0x74)
[ 23.110923] [<802768b0>] (sysfs_warn_dup) from [<80276988>]
(sysfs_create_dir_ns+0x84/0x90)
[ 23.119291] [<80276988>] (sysfs_create_dir_ns) from [<808364c0>]
(kobject_add_internal+0xb4/0x30c)
[ 23.128264] [<808364c0>] (kobject_add_internal) from [<80836764>]
(kobject_add+0x4c/0x9c)
[ 23.136469] [<80836764>] (kobject_add) from [<8050c030>]
(device_add+0xe0/0x594)
[ 23.143892] [<8050c030>] (device_add) from [<8051033c>]
(platform_device_add+0x110/0x224)
[ 23.152096] [<8051033c>] (platform_device_add) from [<8049bb94>]
(imx_gpc_probe+0x184/0x380)
[ 23.160556] [<8049bb94>] (imx_gpc_probe) from [<80510528>]
(platform_drv_probe+0x50/0xac)
[ 23.168758] [<80510528>] (platform_drv_probe) from [<8050ecac>]
(driver_probe_device+0x254/0x32c)
[ 23.177652] [<8050ecac>] (driver_probe_device) from [<8050d3f4>]
(bind_store+0xac/0x140)
[ 23.185765] [<8050d3f4>] (bind_store) from [<802757a4>]
(kernfs_fop_write+0xec/0x1f0)
[ 23.193623] [<802757a4>] (kernfs_fop_write) from [<80206064>]
(__vfs_write+0x1c/0x120)
[ 23.201562] [<80206064>] (__vfs_write) from [<80206310>]
(vfs_write+0xa4/0x1b4)
[ 23.208890] [<80206310>] (vfs_write) from [<80206520>]
(SyS_write+0x3c/0x90)
[ 23.215963] [<80206520>] (SyS_write) from [<80107940>]
(ret_fast_syscall+0x0/0x54)
[ 23.223619] ---[ end trace a5f9524661a4dba7 ]---
[ 23.228254] ------------[ cut here ]------------
[ 23.232920] WARNING: CPU: 0 PID: 432 at lib/kobject.c:240
kobject_add_internal+0x278/0x30c
[ 23.241205] kobject_add_internal failed for imx-pgc-power-domain.0
with -EEXIST, don't try to register things with the same name in the
same directory.
[ 23.254808] Modules linked in:
[ 23.257880] CPU: 0 PID: 432 Comm: sh Tainted: G W
4.15.0-rc3-00061-g38f99f72e8f3-dirty #247
[ 23.267459] Hardware name: Freescale i.MX6 Quad/DualLite (Device
Tree)
[ 23.274009] [<8010f61c>] (unwind_backtrace) from [<8010b984>]
(show_stack+0x10/0x14)
[ 23.281776] [<8010b984>] (show_stack) from [<80831f84>]
(dump_stack+0x88/0x9c)
[ 23.289022] [<80831f84>] (dump_stack) from [<8011e7ac>]
(__warn+0xdc/0xf4)
[ 23.295914] [<8011e7ac>] (__warn) from [<8011e7fc>]
(warn_slowpath_fmt+0x38/0x48)
[ 23.303417] [<8011e7fc>] (warn_slowpath_fmt) from [<80836684>]
(kobject_add_internal+0x278/0x30c)
[ 23.312308] [<80836684>] (kobject_add_internal) from [<80836764>]
(kobject_add+0x4c/0x9c)
[ 23.320506] [<80836764>] (kobject_add) from [<8050c030>]
(device_add+0xe0/0x594)
[ 23.327922] [<8050c030>] (device_add) from [<8051033c>]
(platform_device_add+0x110/0x224)
[ 23.336125] [<8051033c>] (platform_device_add) from [<8049bb94>]
(imx_gpc_probe+0x184/0x380)
[ 23.344586] [<8049bb94>] (imx_gpc_probe) from [<80510528>]
(platform_drv_probe+0x50/0xac)
[ 23.352783] [<80510528>] (platform_drv_probe) from [<8050ecac>]
(driver_probe_device+0x254/0x32c)
[ 23.361677] [<8050ecac>] (driver_probe_device) from [<8050d3f4>]
(bind_store+0xac/0x140)
[ 23.369788] [<8050d3f4>] (bind_store) from [<802757a4>]
(kernfs_fop_write+0xec/0x1f0)
[ 23.377644] [<802757a4>] (kernfs_fop_write) from [<80206064>]
(__vfs_write+0x1c/0x120)
[ 23.385583] [<80206064>] (__vfs_write) from [<80206310>]
(vfs_write+0xa4/0x1b4)
[ 23.392910] [<80206310>] (vfs_write) from [<80206520>]
(SyS_write+0x3c/0x90)
[ 23.399979] [<80206520>] (SyS_write) from [<80107940>]
(ret_fast_syscall+0x0/0x54)
[ 23.407603] ---[ end trace a5f9524661a4dba8 ]---
[ 23.412263] ------------[ cut here ]------------
[ 23.416893] Kernel BUG at 9e1dfcde [verbose debug info unavailable]
[ 23.423173] Internal error: Oops - BUG: 0 [#1] SMP ARM
[ 23.428322] Modules linked in:
[ 23.431391] CPU: 0 PID: 432 Comm: sh Tainted: G W
4.15.0-rc3-00061-g38f99f72e8f3-dirty #247
[ 23.440972] Hardware name: Freescale i.MX6 Quad/DualLite (Device
Tree)
[ 23.447514] PC is at kfree+0xfc/0x140
[ 23.451189] LR is at platform_device_release+0x10/0x34
[ 23.456339] pc : [<801fd744>] lr : [<80510114>] psr: 40010013
[ 23.462615] sp : 84cfdde8 ip : 00000000 fp : 84b34e00
[ 23.467849] r10: 84b35a00 r9 : 80d23968 r8 : 80aa8a40
[ 23.473084] r7 : 80952bac r6 : 00000000 r5 : 84b35a10 r4 :
84b35a10
[ 23.479622] r3 : 87dd2474 r2 : 87dd2460 r1 : a0010013 r0 :
80d23a20
[ 23.486162] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM
Segment none
[ 23.493308] Control: 10c5387d Table: 14bd0059 DAC: 00000051
[ 23.499065] Process sh (pid: 432, stack limit = 0xd85916d3)
[ 23.504647] Stack: (0x84cfdde8 to 0x84cfe000)
[ 23.509017] dde0: 84b35a10 84b35a18 84b35a10
84b35a10 84b35a10 00000000
[ 23.517212] de00: 80952bac 80510114 84b35a18 80509eac 84b35a18
80d2a784 84355780 80835e98
[ 23.525408] de20: 87da9920 80952bac 87da983c ffffffef 87da9920
8049bd2c 00000000 00000000
[ 23.533602] de40: 84151a10 00000042 84c28840 00000000 80b410d8
84151a10 fffffffe 80d239d8
[ 23.541797] de60: fffffdfb 80d239d8 00000000 00000003 84c2b610
80510528 84151a10 80daefe4
[ 23.549991] de80: 80daefe8 00000000 80d239d8 8050ecac 84151a10
80d2ab60 80d239d8 84151a44
[ 23.558185] dea0: 0000000c 84cfdf80 00000000 8050d3f4 0000000c
84c2b600 00000000 00000000
[ 23.566379] dec0: 84355a40 802757a4 00000000 00000000 84563910
802756b8 84efbcc0 01a661a8
[ 23.574574] dee0: 84cfdf80 00000000 0000000c 00000000 000a5608
80206064 0000000a 00000100
[ 23.582767] df00: 84dfd100 84e797c0 0000000a 8022401c 84cfb600
ffffe000 0000000a 00000001
[ 23.590961] df20: 84cfb600 84cfc000 0000000a 80224800 00000000
80845644 00000000 8457bd80
[ 23.599157] df40: 0000000c 84efbcc0 01a661a8 84cfdf80 00000000
80206310 84cfb600 00000002
[ 23.607352] df60: 0000000a 84efbcc0 84efbcc0 00000000 00000000
01a661a8 0000000c 80206520
[ 23.615548] df80: 00000000 00000000 00000000 0000000c 01a661a8
76f6ed60 00000004 80107b24
[ 23.623743] dfa0: 84cfc000 80107940 0000000c 01a661a8 00000001
01a661a8 0000000c 00000000
[ 23.631938] dfc0: 0000000c 01a661a8 76f6ed60 00000004 0000000c
0000000c 00086920 000a5608
[ 23.640133] dfe0: 00000000 7ed2395c 76e9cffc 76ef57e0 60010010
00000001 00000000 00000000
[ 23.648334] [<801fd744>] (kfree) from [<80510114>]
(platform_device_release+0x10/0x34)
[ 23.656276] [<80510114>] (platform_device_release) from [<80509eac>]
(device_release+0x2c/0x90)
[ 23.664999] [<80509eac>] (device_release) from [<80835e98>]
(kobject_put+0x94/0xe4)
[ 23.672679] [<80835e98>] (kobject_put) from [<8049bd2c>]
(imx_gpc_probe+0x31c/0x380)
[ 23.680443] [<8049bd2c>] (imx_gpc_probe) from [<80510528>]
(platform_drv_probe+0x50/0xac)
[ 23.688640] [<80510528>] (platform_drv_probe) from [<8050ecac>]
(driver_probe_device+0x254/0x32c)
[ 23.697531] [<8050ecac>] (driver_probe_device) from [<8050d3f4>]
(bind_store+0xac/0x140)
[ 23.705644] [<8050d3f4>] (bind_store) from [<802757a4>]
(kernfs_fop_write+0xec/0x1f0)
[ 23.713499] [<802757a4>] (kernfs_fop_write) from [<80206064>]
(__vfs_write+0x1c/0x120)
[ 23.721436] [<80206064>] (__vfs_write) from [<80206310>]
(vfs_write+0xa4/0x1b4)
[ 23.728764] [<80206310>] (vfs_write) from [<80206520>]
(SyS_write+0x3c/0x90)
[ 23.735832] [<80206520>] (SyS_write) from [<80107940>]
(ret_fast_syscall+0x0/0x54)
[ 23.743420] Code: 1a000003 e5923014 e3130001 1a000000 (e7f001f2)
[ 23.749526] ---[ end trace a5f9524661a4dba9 ]---
--
Stefan
^ permalink raw reply
* [kvmtool hack 3/3] kvmtool: arm64: Switch memory layout
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
If the guest wants to use a larger physical address space place
the RAM at upper half of the address space. Otherwise, it uses the
default layout.
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arm/aarch32/include/kvm/kvm-arch.h | 1 +
arm/aarch64/include/kvm/kvm-arch.h | 15 ++++++++++++---
arm/include/arm-common/kvm-arch.h | 11 ++++++-----
arm/kvm.c | 2 +-
4 files changed, 20 insertions(+), 9 deletions(-)
diff --git a/arm/aarch32/include/kvm/kvm-arch.h b/arm/aarch32/include/kvm/kvm-arch.h
index cd31e72..2d62aab 100644
--- a/arm/aarch32/include/kvm/kvm-arch.h
+++ b/arm/aarch32/include/kvm/kvm-arch.h
@@ -4,6 +4,7 @@
#define ARM_KERN_OFFSET(...) 0x8000
#define ARM_MAX_MEMORY(...) ARM_LOMAP_MAX_MEMORY
+#define ARM_MEMORY_AREA(...) ARM32_MEMORY_AREA
#include "arm-common/kvm-arch.h"
diff --git a/arm/aarch64/include/kvm/kvm-arch.h b/arm/aarch64/include/kvm/kvm-arch.h
index 9de623a..bad35b9 100644
--- a/arm/aarch64/include/kvm/kvm-arch.h
+++ b/arm/aarch64/include/kvm/kvm-arch.h
@@ -1,14 +1,23 @@
#ifndef KVM__KVM_ARCH_H
#define KVM__KVM_ARCH_H
+#include "arm-common/kvm-arch.h"
+
+#define ARM64_MEMORY_AREA(phys_shift) (1UL << (phys_shift - 1))
+#define ARM64_MAX_MEMORY(phys_shift) \
+ ((1ULL << (phys_shift)) - ARM64_MEMORY_AREA(phys_shift))
+
+#define ARM_MEMORY_AREA(kvm) ((kvm)->cfg.arch.aarch32_guest ? \
+ ARM32_MEMORY_AREA : \
+ ARM64_MEMORY_AREA(kvm->cfg.arch.phys_shift))
+
#define ARM_KERN_OFFSET(kvm) ((kvm)->cfg.arch.aarch32_guest ? \
0x8000 : \
0x80000)
#define ARM_MAX_MEMORY(kvm) ((kvm)->cfg.arch.aarch32_guest ? \
- ARM_LOMAP_MAX_MEMORY : \
- ARM_HIMAP_MAX_MEMORY)
+ ARM32_MAX_MEMORY : \
+ ARM64_MAX_MEMORY(kvm->cfg.arch.phys_shift))
-#include "arm-common/kvm-arch.h"
#endif /* KVM__KVM_ARCH_H */
diff --git a/arm/include/arm-common/kvm-arch.h b/arm/include/arm-common/kvm-arch.h
index c83c45f..ca7ab0f 100644
--- a/arm/include/arm-common/kvm-arch.h
+++ b/arm/include/arm-common/kvm-arch.h
@@ -6,14 +6,15 @@
#include <linux/types.h>
#include "arm-common/gic.h"
-
#define ARM_IOPORT_AREA _AC(0x0000000000000000, UL)
#define ARM_MMIO_AREA _AC(0x0000000000010000, UL)
#define ARM_AXI_AREA _AC(0x0000000040000000, UL)
-#define ARM_MEMORY_AREA _AC(0x0000000080000000, UL)
-#define ARM_LOMAP_MAX_MEMORY ((1ULL << 32) - ARM_MEMORY_AREA)
-#define ARM_HIMAP_MAX_MEMORY ((1ULL << 40) - ARM_MEMORY_AREA)
+#define ARM32_MEMORY_AREA _AC(0x0000000080000000, UL)
+#define ARM32_MAX_MEMORY ((1ULL << 32) - ARM32_MEMORY_AREA)
+
+#define ARM_IOMEM_AREA_END ARM32_MEMORY_AREA
+
#define ARM_GIC_DIST_BASE (ARM_AXI_AREA - ARM_GIC_DIST_SIZE)
#define ARM_GIC_CPUI_BASE (ARM_GIC_DIST_BASE - ARM_GIC_CPUI_SIZE)
@@ -24,7 +25,7 @@
#define ARM_IOPORT_SIZE (ARM_MMIO_AREA - ARM_IOPORT_AREA)
#define ARM_VIRTIO_MMIO_SIZE (ARM_AXI_AREA - (ARM_MMIO_AREA + ARM_GIC_SIZE))
#define ARM_PCI_CFG_SIZE (1ULL << 24)
-#define ARM_PCI_MMIO_SIZE (ARM_MEMORY_AREA - \
+#define ARM_PCI_MMIO_SIZE (ARM_IOMEM_AREA_END - \
(ARM_AXI_AREA + ARM_PCI_CFG_SIZE))
#define KVM_IOPORT_AREA ARM_IOPORT_AREA
diff --git a/arm/kvm.c b/arm/kvm.c
index 7573391..0f155c6 100644
--- a/arm/kvm.c
+++ b/arm/kvm.c
@@ -38,7 +38,7 @@ void kvm__init_ram(struct kvm *kvm)
u64 phys_start, phys_size;
void *host_mem;
- phys_start = ARM_MEMORY_AREA;
+ phys_start = ARM_MEMORY_AREA(kvm);
phys_size = kvm->ram_size;
host_mem = kvm->ram_start;
--
2.13.6
^ permalink raw reply related
* [kvmtool hack 2/3] kvmtool: arm64: Add support for guest physical address size
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
Add an option to specify the physical address size used by this
VM.
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arm/aarch64/include/kvm/kvm-config-arch.h | 5 ++++-
arm/include/arm-common/kvm-config-arch.h | 1 +
arm/kvm.c | 30 ++++++++++++++++++++++++++++++
3 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/arm/aarch64/include/kvm/kvm-config-arch.h b/arm/aarch64/include/kvm/kvm-config-arch.h
index 04be43d..c4bb207 100644
--- a/arm/aarch64/include/kvm/kvm-config-arch.h
+++ b/arm/aarch64/include/kvm/kvm-config-arch.h
@@ -8,7 +8,10 @@
"Create PMUv3 device"), \
OPT_U64('\0', "kaslr-seed", &(cfg)->kaslr_seed, \
"Specify random seed for Kernel Address Space " \
- "Layout Randomization (KASLR)"),
+ "Layout Randomization (KASLR)"), \
+ OPT_UINTEGER('\0', "phys-shift", &(cfg)->phys_shift, \
+ "Specify maximum physical address size (not " \
+ "the amount of memory)"),
#include "arm-common/kvm-config-arch.h"
diff --git a/arm/include/arm-common/kvm-config-arch.h b/arm/include/arm-common/kvm-config-arch.h
index 6a196f1..d841b0b 100644
--- a/arm/include/arm-common/kvm-config-arch.h
+++ b/arm/include/arm-common/kvm-config-arch.h
@@ -11,6 +11,7 @@ struct kvm_config_arch {
bool has_pmuv3;
u64 kaslr_seed;
enum irqchip_type irqchip;
+ unsigned int phys_shift;
};
int irqchip_parser(const struct option *opt, const char *arg, int unset);
diff --git a/arm/kvm.c b/arm/kvm.c
index 2ab436e..7573391 100644
--- a/arm/kvm.c
+++ b/arm/kvm.c
@@ -18,6 +18,14 @@ struct kvm_ext kvm_req_ext[] = {
{ 0, 0 },
};
+#ifndef KVM_CAP_ARM_CONFIG_PHYS_SHIFT
+#define KVM_CAP_ARM_CONFIG_PHYS_SHIFT 151
+#endif
+
+#ifndef KVM_ARM_SET_PHYS_SIZE
+#define KVM_ARM_SET_PHYS_SIZE _IOW(KVMIO, 0xb2, __u32)
+#endif
+
bool kvm__arch_cpu_supports_vm(void)
{
/* The KVM capability check is enough. */
@@ -57,8 +65,30 @@ void kvm__arch_set_cmdline(char *cmdline, bool video)
{
}
+static void kvm__init_phys_size(struct kvm *kvm)
+{
+ if (!kvm->cfg.arch.phys_shift)
+ goto default_phys_size;
+ if (kvm->cfg.arch.phys_shift > 48)
+ die("Physical memory size is limited to 48bits, %d\n",
+ kvm->cfg.arch.phys_shift);
+
+ if (!kvm__supports_extension(kvm, KVM_CAP_ARM_CONFIG_PHYS_SHIFT)) {
+ pr_warning("System doesn't support phys size configuration\n");
+ goto default_phys_size;
+ }
+ if (ioctl(kvm->vm_fd, KVM_ARM_SET_PHYS_SIZE, &kvm->cfg.arch.phys_shift))
+ die("Failed to set physical memory size to %dbits\n",
+ kvm->cfg.arch.phys_shift);
+ return;
+default_phys_size:
+ kvm->cfg.arch.phys_shift = 40;
+ return;
+}
+
void kvm__arch_init(struct kvm *kvm, const char *hugetlbfs_path, u64 ram_size)
{
+ kvm__init_phys_size(kvm);
/*
* Allocate guest memory. We must align our buffer to 64K to
* correlate with the maximum guest page size for virtio-mmio.
--
2.13.6
^ permalink raw reply related
* [kvmtool hack 1/3] virtio: Handle aborts using invalid PFN
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
When the host fails to complete the shake hand due to various
reasons. e.g, for PCI and MMIO, if 0 is written as the PFN,
it implies the host has given up and simply don't take any
action.
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
virtio/mmio.c | 14 ++++++++------
virtio/pci.c | 10 ++++++----
2 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/virtio/mmio.c b/virtio/mmio.c
index f0af4bd..ba02358 100644
--- a/virtio/mmio.c
+++ b/virtio/mmio.c
@@ -188,12 +188,14 @@ static void virtio_mmio_config_out(struct kvm_cpu *vcpu,
break;
case VIRTIO_MMIO_QUEUE_PFN:
val = ioport__read32(data);
- virtio_mmio_init_ioeventfd(vmmio->kvm, vdev, vmmio->hdr.queue_sel);
- vdev->ops->init_vq(vmmio->kvm, vmmio->dev,
- vmmio->hdr.queue_sel,
- vmmio->hdr.guest_page_size,
- vmmio->hdr.queue_align,
- val);
+ if (val) {
+ virtio_mmio_init_ioeventfd(vmmio->kvm, vdev, vmmio->hdr.queue_sel);
+ vdev->ops->init_vq(vmmio->kvm, vmmio->dev,
+ vmmio->hdr.queue_sel,
+ vmmio->hdr.guest_page_size,
+ vmmio->hdr.queue_align,
+ val);
+ }
break;
case VIRTIO_MMIO_QUEUE_NOTIFY:
val = ioport__read32(data);
diff --git a/virtio/pci.c b/virtio/pci.c
index 4ce1111..3c694c2 100644
--- a/virtio/pci.c
+++ b/virtio/pci.c
@@ -271,10 +271,12 @@ static bool virtio_pci__io_out(struct ioport *ioport, struct kvm_cpu *vcpu, u16
break;
case VIRTIO_PCI_QUEUE_PFN:
val = ioport__read32(data);
- virtio_pci__init_ioeventfd(kvm, vdev, vpci->queue_selector);
- vdev->ops->init_vq(kvm, vpci->dev, vpci->queue_selector,
- 1 << VIRTIO_PCI_QUEUE_ADDR_SHIFT,
- VIRTIO_PCI_VRING_ALIGN, val);
+ if (val) {
+ virtio_pci__init_ioeventfd(kvm, vdev, vpci->queue_selector);
+ vdev->ops->init_vq(kvm, vpci->dev, vpci->queue_selector,
+ 1 << VIRTIO_PCI_QUEUE_ADDR_SHIFT,
+ VIRTIO_PCI_VRING_ALIGN, val);
+ }
break;
case VIRTIO_PCI_QUEUE_SEL:
vpci->queue_selector = ioport__read16(data);
--
2.13.6
^ permalink raw reply related
* [PATCH v1 16/16] vgic: its: Add support for 52bit guest physical address
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
From: Kristina Martsenko <kristina.martsenko@arm.com>
We only support 64K for the VGIC, which makes it easier to
support the 52bit guest PA by simply removing the restriction
that we put in to limit the bits to 48.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Kristina Martsenko <kristina.martsenko@arm.com>
[ Clean up macro usages, Add fixes for propbaser handling ]
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
virt/kvm/arm/vgic/vgic-its.c | 36 ++++++++++--------------------------
virt/kvm/arm/vgic/vgic-mmio-v3.c | 1 -
2 files changed, 10 insertions(+), 27 deletions(-)
diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c
index 8e633bd9cc1e..60ab293ec542 100644
--- a/virt/kvm/arm/vgic/vgic-its.c
+++ b/virt/kvm/arm/vgic/vgic-its.c
@@ -233,13 +233,6 @@ static struct its_ite *find_ite(struct vgic_its *its, u32 device_id,
list_for_each_entry(dev, &(its)->device_list, dev_list) \
list_for_each_entry(ite, &(dev)->itt_head, ite_list)
-/*
- * We only implement 48 bits of PA at the moment, although the ITS
- * supports more. Let's be restrictive here.
- */
-#define BASER_ADDRESS(x) ((x) & GENMASK_ULL(47, 16))
-#define CBASER_ADDRESS(x) ((x) & GENMASK_ULL(47, 12))
-
#define GIC_LPI_OFFSET 8192
#define VITS_TYPER_IDBITS 16
@@ -769,7 +762,7 @@ static bool vgic_its_check_id(struct vgic_its *its, u64 baser, u32 id,
if (id >= (l1_tbl_size / esz))
return false;
- addr = BASER_ADDRESS(baser) + id * esz;
+ addr = GITS_BASER_ADDR64K_TO_PHYS(baser) + id * esz;
gfn = addr >> PAGE_SHIFT;
if (eaddr)
@@ -784,7 +777,8 @@ static bool vgic_its_check_id(struct vgic_its *its, u64 baser, u32 id,
/* Each 1st level entry is represented by a 64-bit value. */
if (kvm_read_guest(its->dev->kvm,
- BASER_ADDRESS(baser) + index * sizeof(indirect_ptr),
+ GITS_BASER_ADDR64K_TO_PHYS(baser) +
+ index * sizeof(indirect_ptr),
&indirect_ptr, sizeof(indirect_ptr)))
return false;
@@ -794,11 +788,7 @@ static bool vgic_its_check_id(struct vgic_its *its, u64 baser, u32 id,
if (!(indirect_ptr & BIT_ULL(63)))
return false;
- /*
- * Mask the guest physical address and calculate the frame number.
- * Any address beyond our supported 48 bits of PA will be caught
- * by the actual check in the final step.
- */
+ /* Mask the guest physical address and calculate the frame number. */
indirect_ptr &= GENMASK_ULL(51, 16);
/* Find the address of the actual entry */
@@ -1292,9 +1282,6 @@ static u64 vgic_sanitise_its_baser(u64 reg)
GITS_BASER_OUTER_CACHEABILITY_SHIFT,
vgic_sanitise_outer_cacheability);
- /* Bits 15:12 contain bits 51:48 of the PA, which we don't support. */
- reg &= ~GENMASK_ULL(15, 12);
-
/* We support only one (ITS) page size: 64K */
reg = (reg & ~GITS_BASER_PAGE_SIZE_MASK) | GITS_BASER_PAGE_SIZE_64K;
@@ -1313,11 +1300,8 @@ static u64 vgic_sanitise_its_cbaser(u64 reg)
GITS_CBASER_OUTER_CACHEABILITY_SHIFT,
vgic_sanitise_outer_cacheability);
- /*
- * Sanitise the physical address to be 64k aligned.
- * Also limit the physical addresses to 48 bits.
- */
- reg &= ~(GENMASK_ULL(51, 48) | GENMASK_ULL(15, 12));
+ /* Sanitise the physical address to be 64k aligned. */
+ reg &= ~GENMASK_ULL(15, 12);
return reg;
}
@@ -1363,7 +1347,7 @@ static void vgic_its_process_commands(struct kvm *kvm, struct vgic_its *its)
if (!its->enabled)
return;
- cbaser = CBASER_ADDRESS(its->cbaser);
+ cbaser = GITS_CBASER_ADDRESS(its->cbaser);
while (its->cwriter != its->creadr) {
int ret = kvm_read_guest(kvm, cbaser + its->creadr,
@@ -2221,7 +2205,7 @@ static int vgic_its_restore_device_tables(struct vgic_its *its)
if (!(baser & GITS_BASER_VALID))
return 0;
- l1_gpa = BASER_ADDRESS(baser);
+ l1_gpa = GITS_BASER_ADDR64K_TO_PHYS(baser);
if (baser & GITS_BASER_INDIRECT) {
l1_esz = GITS_LVL1_ENTRY_SIZE;
@@ -2293,7 +2277,7 @@ static int vgic_its_save_collection_table(struct vgic_its *its)
{
const struct vgic_its_abi *abi = vgic_its_get_abi(its);
u64 baser = its->baser_coll_table;
- gpa_t gpa = BASER_ADDRESS(baser);
+ gpa_t gpa = GITS_BASER_ADDR64K_TO_PHYS(baser);
struct its_collection *collection;
u64 val;
size_t max_size, filled = 0;
@@ -2342,7 +2326,7 @@ static int vgic_its_restore_collection_table(struct vgic_its *its)
if (!(baser & GITS_BASER_VALID))
return 0;
- gpa = BASER_ADDRESS(baser);
+ gpa = GITS_BASER_ADDR64K_TO_PHYS(baser);
max_size = GITS_BASER_NR_PAGES(baser) * SZ_64K;
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 671fe81f8e1d..90f36d9c946b 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -351,7 +351,6 @@ static u64 vgic_sanitise_propbaser(u64 reg)
vgic_sanitise_outer_cacheability);
reg &= ~PROPBASER_RES0_MASK;
- reg &= ~GENMASK_ULL(51, 48);
return reg;
}
--
2.13.6
^ permalink raw reply related
* [PATCH v1 15/16] kvm: arm64: Allow configuring physical address space size
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
Allow the guests to choose a larger physical address space size.
The default and minimum size is 40bits. A guest can change this
right after the VM creation, but before the stage2 entry page
tables are allocated (i.e, before it registers a memory range
or maps a device address). The size is restricted to the maximum
supported by the host. Also, the guest can only increase the PA size,
from the existing value, as reducing it could break the devices which
may have verified their physical address for validity and may do a
lazy mapping(e.g, VGIC).
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Documentation/virtual/kvm/api.txt | 27 ++++++++++++++++++++++++++
arch/arm/include/asm/kvm_host.h | 7 +++++++
arch/arm64/include/asm/kvm_host.h | 1 +
arch/arm64/include/asm/kvm_mmu.h | 41 ++++++++++++++++++++++++++++++---------
arch/arm64/kvm/reset.c | 28 ++++++++++++++++++++++++++
include/uapi/linux/kvm.h | 4 ++++
virt/kvm/arm/arm.c | 2 +-
7 files changed, 100 insertions(+), 10 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 57d3ee9e4bde..a203faf768c4 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3403,6 +3403,33 @@ invalid, if invalid pages are written to (e.g. after the end of memory)
or if no page table is present for the addresses (e.g. when using
hugepages).
+4.109 KVM_ARM_GET_PHYS_SHIFT
+
+Capability: KVM_CAP_ARM_CONFIG_PHYS_SHIFT
+Architectures: arm64
+Type: vm ioctl
+Parameters: __u32 (out)
+Returns: 0 on success, a negative value on error
+
+This ioctl is used to get the current maximum physical address size for
+the VM. The value is Log2(Maximum_Physical_Address). This is neither the
+ amount of physical memory assigned to the VM nor the maximum physical address
+that a real CPU on the host can handle. Rather, this is the upper limit of the
+guest physical address that can be used by the VM.
+
+4.109 KVM_ARM_SET_PHYS_SHIFT
+
+Capability: KVM_CAP_ARM_CONFIG_PHYS_SHIFT
+Architectures: arm64
+Type: vm ioctl
+Parameters: __u32 (in)
+Returns: 0 on success, a negative value on error
+
+This ioctl is used to set the maximum physical address size for
+the VM. The value is Log2(Maximum_Physical_Address). The value can only
+be increased from the existing setting. The value cannot be changed
+after the stage-2 page tables are allocated and will return an error.
+
5. The kvm_run structure
------------------------
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index a9f7d3f47134..fa8e68a4f692 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -268,6 +268,13 @@ static inline int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
return 0;
}
+static inline long kvm_arch_dev_vm_ioctl(struct kvm *kvm,
+ unsigned int ioctl,
+ unsigned long arg)
+{
+ return -EINVAL;
+}
+
int kvm_perf_init(void);
int kvm_perf_teardown(void);
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 1e66e5ab3dde..2895c2cda8fc 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -50,6 +50,7 @@
int __attribute_const__ kvm_target_cpu(void);
int kvm_reset_vcpu(struct kvm_vcpu *vcpu);
int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext);
+long kvm_arch_dev_vm_ioctl(struct kvm *kvm, unsigned int ioctl, unsigned long arg);
void __extended_idmap_trampoline(phys_addr_t boot_pgd, phys_addr_t idmap_start);
struct kvm_arch {
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index ab6a8b905065..ab7f50f20bcd 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -347,21 +347,44 @@ static inline u64 kvm_vttbr_baddr_mask(struct kvm *kvm)
return GENMASK_ULL(PHYS_MASK_SHIFT - 1, x);
}
+static inline int kvm_reconfig_stage2(struct kvm *kvm, u32 phys_shift)
+{
+ int rc = 0;
+ unsigned int pa_max, parange;
+
+ parange = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1) & 7;
+ pa_max = id_aa64mmfr0_parange_to_phys_shift(parange);
+ /* Raise it to 40bits for backward compatibility */
+ pa_max = (pa_max < 40) ? 40 : pa_max;
+ /* Make sure the size is supported/available */
+ if (phys_shift > PHYS_MASK_SHIFT || phys_shift > pa_max)
+ return -EINVAL;
+ /*
+ * The stage2 PGD is dependent on the settings we initialise here
+ * and should be allocated only after this step. We cannot allow
+ * down sizing the IPA size as there could be devices or memory
+ * regions, that depend on the previous size.
+ */
+ mutex_lock(&kvm->slots_lock);
+ if (kvm->arch.pgd || phys_shift < kvm->arch.phys_shift) {
+ rc = -EPERM;
+ } else if (phys_shift > kvm->arch.phys_shift) {
+ kvm->arch.phys_shift = phys_shift;
+ kvm->arch.s2_levels = stage2_pt_levels(kvm->arch.phys_shift);
+ kvm->arch.vtcr_private = VTCR_EL2_SL0(kvm->arch.s2_levels) |
+ TCR_T0SZ(kvm->arch.phys_shift);
+ }
+ mutex_unlock(&kvm->slots_lock);
+ return rc;
+}
+
/*
* kvm_init_stage2_config: Initialise the VM specific stage2 page table
* details to default IPA size.
*/
static inline void kvm_init_stage2_config(struct kvm *kvm)
{
- /*
- * The stage2 PGD is dependent on the settings we initialise here
- * and should be allocated only after this step.
- */
- VM_BUG_ON(kvm->arch.pgd != NULL);
- kvm->arch.phys_shift = KVM_PHYS_SHIFT_DEFAULT;
- kvm->arch.s2_levels = stage2_pt_levels(kvm->arch.phys_shift);
- kvm->arch.vtcr_private = VTCR_EL2_SL0(kvm->arch.s2_levels) |
- TCR_T0SZ(kvm->arch.phys_shift);
+ kvm_reconfig_stage2(kvm, KVM_PHYS_SHIFT_DEFAULT);
}
#endif /* __ASSEMBLY__ */
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index 3256b9228e75..90ceca823aca 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -23,6 +23,7 @@
#include <linux/kvm_host.h>
#include <linux/kvm.h>
#include <linux/hw_breakpoint.h>
+#include <linux/uaccess.h>
#include <kvm/arm_arch_timer.h>
@@ -81,6 +82,9 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_VCPU_ATTRIBUTES:
r = 1;
break;
+ case KVM_CAP_ARM_CONFIG_PHYS_SHIFT:
+ r = 1;
+ break;
default:
r = 0;
}
@@ -88,6 +92,30 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
return r;
}
+long kvm_arch_dev_vm_ioctl(struct kvm *kvm,
+ unsigned int ioctl, unsigned long arg)
+{
+ void __user *argp = (void __user *)arg;
+ u32 phys_shift;
+ long r = -EFAULT;
+
+ switch (ioctl) {
+ case KVM_ARM_GET_PHYS_SHIFT:
+ phys_shift = kvm_phys_shift(kvm);
+ if (!put_user(phys_shift, (u32 __user *)argp))
+ r = 0;
+ break;
+ case KVM_ARM_SET_PHYS_SHIFT:
+ if (!get_user(phys_shift, (u32 __user*)argp))
+ r = kvm_reconfig_stage2(kvm, phys_shift);
+ break;
+ default:
+ r = -EINVAL;
+ }
+ return r;
+}
+
+
/**
* kvm_reset_vcpu - sets core registers and sys_regs to reset value
* @vcpu: The VCPU pointer
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 496e59a2738b..66bfbe19b434 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -932,6 +932,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_HYPERV_SYNIC2 148
#define KVM_CAP_HYPERV_VP_INDEX 149
#define KVM_CAP_S390_AIS_MIGRATION 150
+#define KVM_CAP_ARM_CONFIG_PHYS_SHIFT 151
#ifdef KVM_CAP_IRQ_ROUTING
@@ -1261,6 +1262,9 @@ struct kvm_s390_ucas_mapping {
#define KVM_PPC_CONFIGURE_V3_MMU _IOW(KVMIO, 0xaf, struct kvm_ppc_mmuv3_cfg)
/* Available with KVM_CAP_PPC_RADIX_MMU */
#define KVM_PPC_GET_RMMU_INFO _IOW(KVMIO, 0xb0, struct kvm_ppc_rmmu_info)
+/* Available with KVM_CAP_ARM_CONFIG_PHYS_SHIFT */
+#define KVM_ARM_GET_PHYS_SHIFT _IOR(KVMIO, 0xb1, __u32)
+#define KVM_ARM_SET_PHYS_SHIFT _IOW(KVMIO, 0xb2, __u32)
/* ioctl for vm fd */
#define KVM_CREATE_DEVICE _IOWR(KVMIO, 0xe0, struct kvm_create_device)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index e0bf8d19fcfe..05fc49304722 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -1136,7 +1136,7 @@ long kvm_arch_vm_ioctl(struct file *filp,
return 0;
}
default:
- return -EINVAL;
+ return kvm_arch_dev_vm_ioctl(kvm, ioctl, arg);
}
}
--
2.13.6
^ permalink raw reply related
* [PATCH v1 14/16] kvm: arm64: Switch to per VM IPA
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
Now that we can manage the stage2 page table per VM, switch the
configuration details to per VM instance. We keep track of the
IPA bits, number of page table levels and the VTCR bits (which
depends on the IPA and the number of levels).
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm/include/asm/kvm_mmu.h | 1 +
arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++
arch/arm64/include/asm/kvm_mmu.h | 22 ++++++++++++++++++++--
arch/arm64/include/asm/stage2_pgtable.h | 1 -
arch/arm64/kvm/hyp/switch.c | 3 +--
virt/kvm/arm/arm.c | 2 +-
6 files changed, 35 insertions(+), 6 deletions(-)
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 440c80589453..dd592fe45660 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -48,6 +48,7 @@
#define kvm_vttbr_baddr_mask(kvm) VTTBR_BADDR_MASK
#define stage2_pgd_size(kvm) (PTRS_PER_S2_PGD * sizeof(pgd_t))
+#define kvm_init_stage2_config(kvm) do { } while (0)
int create_hyp_mappings(void *from, void *to, pgprot_t prot);
int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
void free_hyp_pgds(void);
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 9a9ddeb33c84..1e66e5ab3dde 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -64,6 +64,18 @@ struct kvm_arch {
/* VTTBR value associated with above pgd and vmid */
u64 vttbr;
+ /* Private bits of VTCR_EL2 for this VM */
+ u64 vtcr_private;
+ /* Size of the PA size for this guest */
+ u8 phys_shift;
+ /*
+ * Number of levels in page table. We could always calculate
+ * it from phys_shift above. We cache it for faster switches
+ * in stage2 page table helpers.
+ */
+ u8 s2_levels;
+
+
/* The last vcpu id that ran on each physical CPU */
int __percpu *last_vcpu_ran;
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 483185ed2ecd..ab6a8b905065 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -134,11 +134,12 @@ static inline unsigned long __kern_hyp_va(unsigned long v)
/*
* We currently only support a 40bit IPA.
*/
-#define KVM_PHYS_SHIFT (40)
+#define KVM_PHYS_SHIFT_DEFAULT (40)
-#define kvm_phys_shift(kvm) KVM_PHYS_SHIFT
+#define kvm_phys_shift(kvm) (kvm->arch.phys_shift)
#define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm))
#define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL))
+#define kvm_stage2_levels(kvm) (kvm->arch.s2_levels)
static inline bool kvm_page_empty(void *ptr)
{
@@ -346,5 +347,22 @@ static inline u64 kvm_vttbr_baddr_mask(struct kvm *kvm)
return GENMASK_ULL(PHYS_MASK_SHIFT - 1, x);
}
+/*
+ * kvm_init_stage2_config: Initialise the VM specific stage2 page table
+ * details to default IPA size.
+ */
+static inline void kvm_init_stage2_config(struct kvm *kvm)
+{
+ /*
+ * The stage2 PGD is dependent on the settings we initialise here
+ * and should be allocated only after this step.
+ */
+ VM_BUG_ON(kvm->arch.pgd != NULL);
+ kvm->arch.phys_shift = KVM_PHYS_SHIFT_DEFAULT;
+ kvm->arch.s2_levels = stage2_pt_levels(kvm->arch.phys_shift);
+ kvm->arch.vtcr_private = VTCR_EL2_SL0(kvm->arch.s2_levels) |
+ TCR_T0SZ(kvm->arch.phys_shift);
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h
index 33e8ebb25037..9b75b83da643 100644
--- a/arch/arm64/include/asm/stage2_pgtable.h
+++ b/arch/arm64/include/asm/stage2_pgtable.h
@@ -44,7 +44,6 @@
*/
#define __s2_pgd_ptrs(pa, lvls) (1 << ((pa) - pt_levels_pgdir_shift((lvls))))
-#define kvm_stage2_levels(kvm) stage2_pt_levels(kvm_phys_shift(kvm))
#define stage2_pgdir_shift(kvm) \
pt_levels_pgdir_shift(kvm_stage2_levels(kvm))
#define stage2_pgdir_size(kvm) (_AC(1, UL) << stage2_pgdir_shift((kvm)))
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 523471f0af7b..d0725562ee3f 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -160,8 +160,7 @@ static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
u64 vtcr = read_sysreg(vtcr_el2);
vtcr &= ~VTCR_EL2_PRIVATE_MASK;
- vtcr |= VTCR_EL2_SL0(stage2_pt_levels(kvm)) |
- VTCR_EL2_T0SZ(kvm_phys_shift(kvm));
+ vtcr |= kvm->arch.vtcr_private;
write_sysreg(vtcr, vtcr_el2);
write_sysreg(kvm->arch.vttbr, vttbr_el2);
}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 8564ed907b18..e0bf8d19fcfe 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -143,7 +143,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
/* The maximum number of VCPUs is limited by the host's GIC model */
kvm->arch.max_vcpus = vgic_present ?
kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
-
+ kvm_init_stage2_config(kvm);
return ret;
}
--
2.13.6
^ permalink raw reply related
* [PATCH v1 13/16] kvm: arm64: Configure VTCR per VM
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
We set VTCR_EL2 very early during the stage2 init and don't
touch it ever. This is fine as we had a fixed IPA size. This
patch changes the behavior to set the VTCR for a given VM,
depending on its stage2 table. The common configuration for
VTCR is still performed during the early init. But the SL0
and T0SZ are programmed for each VM and is cleared once we
exit the VM.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/kvm_arm.h | 16 ++++++----------
arch/arm64/include/asm/kvm_asm.h | 2 +-
arch/arm64/include/asm/kvm_host.h | 8 +++++---
arch/arm64/kvm/hyp/s2-setup.c | 16 +---------------
arch/arm64/kvm/hyp/switch.c | 9 +++++++++
5 files changed, 22 insertions(+), 29 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index eb90d349e55f..d5c40816f073 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -115,9 +115,7 @@
#define VTCR_EL2_IRGN0_WBWA TCR_IRGN0_WBWA
#define VTCR_EL2_SL0_SHIFT 6
#define VTCR_EL2_SL0_MASK (3 << VTCR_EL2_SL0_SHIFT)
-#define VTCR_EL2_SL0_LVL1 (1 << VTCR_EL2_SL0_SHIFT)
#define VTCR_EL2_T0SZ_MASK 0x3f
-#define VTCR_EL2_T0SZ_40B 24
#define VTCR_EL2_VS_SHIFT 19
#define VTCR_EL2_VS_8BIT (0 << VTCR_EL2_VS_SHIFT)
#define VTCR_EL2_VS_16BIT (1 << VTCR_EL2_VS_SHIFT)
@@ -139,38 +137,36 @@
* D4-23 and D4-25 in ARM DDI 0487A.b.
*/
-#define VTCR_EL2_T0SZ_IPA VTCR_EL2_T0SZ_40B
#define VTCR_EL2_COMMON_BITS (VTCR_EL2_SH0_INNER | VTCR_EL2_ORGN0_WBWA | \
VTCR_EL2_IRGN0_WBWA | VTCR_EL2_RES1)
+#define VTCR_EL2_PRIVATE_MASK (VTCR_EL2_SL0_MASK | VTCR_EL2_T0SZ_MASK)
#ifdef CONFIG_ARM64_64K_PAGES
/*
* Stage2 translation configuration:
* 64kB pages (TG0 = 1)
- * 2 level page tables (SL = 1)
*/
-#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1)
+#define VTCR_EL2_TGRAN VTCR_EL2_TG0_64K
#define VTCR_EL2_TGRAN_SL0_BASE 3UL
#elif defined(CONFIG_ARM64_16K_PAGES)
/*
* Stage2 translation configuration:
* 16kB pages (TG0 = 2)
- * 2 level page tables (SL = 1)
*/
-#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1)
+#define VTCR_EL2_TGRAN VTCR_EL2_TG0_16K
#define VTCR_EL2_TGRAN_SL0_BASE 3UL
#else /* 4K */
/*
* Stage2 translation configuration:
* 4kB pages (TG0 = 0)
- * 3 level page tables (SL = 1)
*/
-#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1)
+#define VTCR_EL2_TGRAN VTCR_EL2_TG0_4K
#define VTCR_EL2_TGRAN_SL0_BASE 2UL
#endif
-#define VTCR_EL2_FLAGS (VTCR_EL2_COMMON_BITS | VTCR_EL2_TGRAN_FLAGS)
+#define VTCR_EL2_FLAGS (VTCR_EL2_COMMON_BITS | VTCR_EL2_TGRAN)
+
/*
* VTCR_EL2:SL0 indicates the entry level for Stage2 translation.
* Interestingly, it depends on the page size.
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index ab4d0a926043..21cfd1fe692c 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -66,7 +66,7 @@ extern void __vgic_v3_init_lrs(void);
extern u32 __kvm_get_mdcr_el2(void);
-extern u32 __init_stage2_translation(void);
+extern void __init_stage2_translation(void);
#endif
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index ea6cb5b24258..9a9ddeb33c84 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -380,10 +380,12 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
static inline void __cpu_init_stage2(void)
{
- u32 parange = kvm_call_hyp(__init_stage2_translation);
+ u32 ps;
- WARN_ONCE(parange < 40,
- "PARange is %d bits, unsupported configuration!", parange);
+ kvm_call_hyp(__init_stage2_translation);
+ ps = id_aa64mmfr0_parange_to_phys_shift(read_sysreg(id_aa64mmfr0_el1));
+ WARN_ONCE(ps < 40,
+ "PARange is %d bits, unsupported configuration!", ps);
}
/*
diff --git a/arch/arm64/kvm/hyp/s2-setup.c b/arch/arm64/kvm/hyp/s2-setup.c
index b1129c83c531..5c26ad4b8ac9 100644
--- a/arch/arm64/kvm/hyp/s2-setup.c
+++ b/arch/arm64/kvm/hyp/s2-setup.c
@@ -19,13 +19,11 @@
#include <asm/kvm_arm.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_hyp.h>
-#include <asm/cpufeature.h>
-u32 __hyp_text __init_stage2_translation(void)
+void __hyp_text __init_stage2_translation(void)
{
u64 val = VTCR_EL2_FLAGS;
u64 parange;
- u32 phys_shift;
u64 tmp;
/*
@@ -38,16 +36,6 @@ u32 __hyp_text __init_stage2_translation(void)
parange = ID_AA64MMFR0_PARANGE_MAX;
val |= parange << 16;
- /* Compute the actual PARange... */
- phys_shift = id_aa64mmfr0_parange_to_phys_shift(parange);
-
- /*
- * ... and clamp it to 40 bits, unless we have some braindead
- * HW that implements less than that. In all cases, we'll
- * return that value for the rest of the kernel to decide what
- * to do.
- */
- val |= 64 - (phys_shift > 40 ? 40 : phys_shift);
/*
* Check the availability of Hardware Access Flag / Dirty Bit
@@ -67,6 +55,4 @@ u32 __hyp_text __init_stage2_translation(void)
VTCR_EL2_VS_8BIT;
write_sysreg(val, vtcr_el2);
-
- return phys_shift;
}
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index f7c651f3a8c0..523471f0af7b 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -157,11 +157,20 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+ u64 vtcr = read_sysreg(vtcr_el2);
+
+ vtcr &= ~VTCR_EL2_PRIVATE_MASK;
+ vtcr |= VTCR_EL2_SL0(stage2_pt_levels(kvm)) |
+ VTCR_EL2_T0SZ(kvm_phys_shift(kvm));
+ write_sysreg(vtcr, vtcr_el2);
write_sysreg(kvm->arch.vttbr, vttbr_el2);
}
static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
{
+ u64 vtcr = read_sysreg(vtcr_el2) & ~VTCR_EL2_PRIVATE_MASK;
+
+ write_sysreg(vtcr, vtcr_el2);
write_sysreg(0, vttbr_el2);
}
--
2.13.6
^ permalink raw reply related
* [PATCH v1 12/16] kvm: arm64: Dynamic configuration of VTCR and VTTBR mask
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
VTCR_EL2 holds the following key stage2 translation table
parameters:
SL0 - Entry level in the page table lookup.
T0SZ - Denotes the size of the memory addressed by the table.
We have been using fixed values for the SL0 depending on the
page size as we have a fixed IPA size. But since we are about
to make it dynamic, we need to calculate the SL0 at runtime
per VM.
Also the VTTBR:BADDR holds the base address for the stage2
translation table and the ARM ARM mandates that bits
BADDR[x-1:0] should be 0, where 'x' defined using some
magical constant, which depends on the page size, T0SZ
and the entry level of lookup (Since the entry level page
tables can be concatenated at stage2, a given T0SZ could
possibly start at 2 different levels). We need a way to
calculate this magical value per VM, depending on the
IPA size. Luckily there is a magic formula for finding
the "magic" number to find "x". See the patch for more
details.
This patch adds helpers to figure out the VTCR_SL0 and
the magic "X" for a configuration of stage2.
The other advantage we have with this change is switching
the entry level for a given IPA size, depending on if we
are able to get contiguous block of memory for the entry
level page table. (e.g, With 64KB page size and 46bit IPA
starting at level 2, finding 16 * 64KB contiguous block on a
loaded system could be tricky. So we could decide to rather
enter at level 1, with a single page).
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/kvm_arm.h | 96 +++++++++++++++++++++++++++++++++++++---
arch/arm64/include/asm/kvm_mmu.h | 20 ++++++++-
2 files changed, 110 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 715d395ef45b..eb90d349e55f 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -122,6 +122,8 @@
#define VTCR_EL2_VS_8BIT (0 << VTCR_EL2_VS_SHIFT)
#define VTCR_EL2_VS_16BIT (1 << VTCR_EL2_VS_SHIFT)
+#define VTCR_EL2_T0SZ(x) TCR_T0SZ((x))
+
/*
* We configure the Stage-2 page tables to always restrict the IPA space to be
* 40 bits wide (T0SZ = 24). Systems with a PARange smaller than 40 bits are
@@ -148,7 +150,8 @@
* 2 level page tables (SL = 1)
*/
#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1)
-#define VTTBR_X_TGRAN_MAGIC 38
+#define VTCR_EL2_TGRAN_SL0_BASE 3UL
+
#elif defined(CONFIG_ARM64_16K_PAGES)
/*
* Stage2 translation configuration:
@@ -156,7 +159,7 @@
* 2 level page tables (SL = 1)
*/
#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1)
-#define VTTBR_X_TGRAN_MAGIC 42
+#define VTCR_EL2_TGRAN_SL0_BASE 3UL
#else /* 4K */
/*
* Stage2 translation configuration:
@@ -164,13 +167,96 @@
* 3 level page tables (SL = 1)
*/
#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1)
-#define VTTBR_X_TGRAN_MAGIC 37
+#define VTCR_EL2_TGRAN_SL0_BASE 2UL
#endif
#define VTCR_EL2_FLAGS (VTCR_EL2_COMMON_BITS | VTCR_EL2_TGRAN_FLAGS)
-#define VTTBR_X (VTTBR_X_TGRAN_MAGIC - VTCR_EL2_T0SZ_IPA)
+/*
+ * VTCR_EL2:SL0 indicates the entry level for Stage2 translation.
+ * Interestingly, it depends on the page size.
+ * See D.10.2.110, VTCR_EL2, in ARM DDI 0487B.b
+ *
+ * -----------------------------------------
+ * | Entry level | 4K | 16K/64K |
+ * ------------------------------------------
+ * | Level: 0 | 2 | - |
+ * ------------------------------------------
+ * | Level: 1 | 1 | 2 |
+ * ------------------------------------------
+ * | Level: 2 | 0 | 1 |
+ * ------------------------------------------
+ * | Level: 3 | - | 0 |
+ * ------------------------------------------
+ *
+ * That table roughly translates to :
+ *
+ * SL0(PAGE_SIZE, Entry_level) = SL0_BASE(PAGE_SIZE) - Entry_Level
+ *
+ * Where SL0_BASE(4K) = 2 and SL0_BASE(16K) = 3, SL0_BASE(64K) = 3, provided
+ * we take care of ruling out the unsupported cases and
+ * Entry_Level = 4 - Number_of_levels.
+ *
+ */
+#define VTCR_EL2_SL0(levels) \
+ ((VTCR_EL2_TGRAN_SL0_BASE - (4 - (levels))) << VTCR_EL2_SL0_SHIFT)
+/*
+ * ARM VMSAv8-64 defines an algorithm for finding the translation table
+ * descriptors in section D4.2.8 in ARM DDI 0487B.b.
+ *
+ * The algorithm defines the expectaions on the BaseAddress (for the page
+ * table) bits resolved at each level based on the page size, entry level
+ * and T0SZ. The variable "x" in the algorithm also affects the VTTBR:BADDR
+ * for stage2 page table.
+ *
+ * The value of "x" is calculated as :
+ * x = Magic_N - T0SZ
+ *
+ * where Magic_N is an integer depending on the page size and the entry
+ * level of the page table as below:
+ *
+ * --------------------------------------------
+ * | Entry level | 4K 16K 64K |
+ * --------------------------------------------
+ * | Level: 0 (4 levels) | 28 | - | - |
+ * --------------------------------------------
+ * | Level: 1 (3 levels) | 37 | 31 | 25 |
+ * --------------------------------------------
+ * | Level: 2 (2 levels) | 46 | 42 | 38 |
+ * --------------------------------------------
+ * | Level: 3 (1 level) | - | 53 | 51 |
+ * --------------------------------------------
+ *
+ * We have a magic formula for the Magic_N below.
+ * Which can also be expressed as:
+ *
+ * Magic_N(PAGE_SIZE, Entry_Level) = 64 - ((PAGE_SHIFT - 3) * Number of levels)
+ *
+ * where number of levels = (4 - Entry_Level).
+ *
+ * So, given that T0SZ = (64 - PA_SHIFT), we can compute 'x' as follows:
+ *
+ * x = (64 - ((PAGE_SHIFT - 3) * Number_of_levels)) - (64 - PA_SHIFT)
+ * = PA_SHIFT - ((PAGE_SHIFT - 3) * Number of levels)
+ *
+ * Here is one way to explain the Magic Formula:
+ *
+ * x = log2(Size_of_Entry_Level_Table)
+ *
+ * Since, we can resolve (PAGE_SHIFT - 3) bits@each level, and another
+ * PAGE_SHIFT bits in the PTE, we have :
+ *
+ * Bits_Entry_level = PA_SHIFT - ((PAGE_SHIFT - 3) * (n - 1) + PAGE_SHIFT)
+ * = PA_SHIFT - (PAGE_SHIFT - 3) * n - 3
+ * where n = number of levels, and since each pointer is 8bytes, we have:
+ *
+ * x = Bits_Entry_Level + 3
+ * = PA_SHIFT - (PAGE_SHIFT - 3) * n
+ *
+ * The only constraint here is that, we have to find the number of page table
+ * levels for a given IPA size (which we do, see STAGE2_PGTABLE_LEVELS).
+ */
+#define ARM64_VTTBR_X(ipa, levels) ((ipa) - ((levels) * (PAGE_SHIFT - 3)))
-#define VTTBR_BADDR_MASK (((UL(1) << (PHYS_MASK_SHIFT - VTTBR_X)) - 1) << VTTBR_X)
#define VTTBR_VMID_SHIFT (UL(48))
#define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT)
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index df2ee97f4428..483185ed2ecd 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -139,7 +139,6 @@ static inline unsigned long __kern_hyp_va(unsigned long v)
#define kvm_phys_shift(kvm) KVM_PHYS_SHIFT
#define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm))
#define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL))
-#define kvm_vttbr_baddr_mask(kvm) VTTBR_BADDR_MASK
static inline bool kvm_page_empty(void *ptr)
{
@@ -328,5 +327,24 @@ static inline unsigned int kvm_get_vmid_bits(void)
#define kvm_phys_to_vttbr(addr) phys_to_ttbr(addr)
+/*
+ * Get the magic number 'x' for VTTBR:BADDR of this KVM instance.
+ * With v8.2 LVA extensions, 'x' (rather 'z') should be a minimum
+ * of 6 with 52bit IPS.
+ */
+static inline int kvm_vttbr_x(struct kvm *kvm)
+{
+ int x = ARM64_VTTBR_X(kvm_phys_shift(kvm), kvm_stage2_levels(kvm));
+
+ return (IS_ENABLED(CONFIG_ARM64_PA_BITS_52) && x < 6) ? 6 : x;
+}
+
+static inline u64 kvm_vttbr_baddr_mask(struct kvm *kvm)
+{
+ unsigned int x = kvm_vttbr_x(kvm);
+
+ return GENMASK_ULL(PHYS_MASK_SHIFT - 1, x);
+}
+
#endif /* __ASSEMBLY__ */
#endif /* __ARM64_KVM_MMU_H__ */
--
2.13.6
^ permalink raw reply related
* [PATCH v1 11/16] kvm: arm64: Make stage2 page table layout dynamic
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
So far we had a static stage2 page table handling code, based on a
fixed IPA of 40bits. As we prepare for a configurable IPA size per
VM, make the our stage2 page table code dynamic to do the right thing
for a given VM.
Support for the IPA size configuration needs other changes in the way
we configure the EL2 registers (VTTBR and VTCR). So, the IPA is still
fixed to 40bits.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm/include/asm/kvm_mmu.h | 1 +
arch/arm64/include/asm/kvm_mmu.h | 16 +-
arch/arm64/include/asm/stage2_pgtable-nopmd.h | 42 ------
arch/arm64/include/asm/stage2_pgtable-nopud.h | 39 -----
arch/arm64/include/asm/stage2_pgtable.h | 203 +++++++++++++++++---------
virt/kvm/arm/arm.c | 2 +-
6 files changed, 147 insertions(+), 156 deletions(-)
delete mode 100644 arch/arm64/include/asm/stage2_pgtable-nopmd.h
delete mode 100644 arch/arm64/include/asm/stage2_pgtable-nopud.h
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index a3312f87a6e0..440c80589453 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -45,6 +45,7 @@
#define kvm_phys_shift(kvm) KVM_PHYS_SHIFT
#define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm))
#define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL))
+#define kvm_vttbr_baddr_mask(kvm) VTTBR_BADDR_MASK
#define stage2_pgd_size(kvm) (PTRS_PER_S2_PGD * sizeof(pgd_t))
int create_hyp_mappings(void *from, void *to, pgprot_t prot);
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index de542aa72d80..df2ee97f4428 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -18,9 +18,10 @@
#ifndef __ARM64_KVM_MMU_H__
#define __ARM64_KVM_MMU_H__
+#include <asm/cpufeature.h>
#include <asm/page.h>
#include <asm/memory.h>
-#include <asm/cpufeature.h>
+#include <asm/kvm_arm.h>
/*
* As ARMv8.0 only has the TTBR0_EL2 register, we cannot express
@@ -138,6 +139,13 @@ static inline unsigned long __kern_hyp_va(unsigned long v)
#define kvm_phys_shift(kvm) KVM_PHYS_SHIFT
#define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm))
#define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL))
+#define kvm_vttbr_baddr_mask(kvm) VTTBR_BADDR_MASK
+
+static inline bool kvm_page_empty(void *ptr)
+{
+ struct page *ptr_page = virt_to_page(ptr);
+ return page_count(ptr_page) == 1;
+}
#include <asm/stage2_pgtable.h>
@@ -203,12 +211,6 @@ static inline bool kvm_s2pmd_readonly(pmd_t *pmd)
return kvm_s2pte_readonly((pte_t *)pmd);
}
-static inline bool kvm_page_empty(void *ptr)
-{
- struct page *ptr_page = virt_to_page(ptr);
- return page_count(ptr_page) == 1;
-}
-
#define hyp_pte_table_empty(ptep) kvm_page_empty(ptep)
#ifdef __PAGETABLE_PMD_FOLDED
diff --git a/arch/arm64/include/asm/stage2_pgtable-nopmd.h b/arch/arm64/include/asm/stage2_pgtable-nopmd.h
deleted file mode 100644
index 0280dedbf75f..000000000000
--- a/arch/arm64/include/asm/stage2_pgtable-nopmd.h
+++ /dev/null
@@ -1,42 +0,0 @@
-/*
- * Copyright (C) 2016 - ARM Ltd
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-
-#ifndef __ARM64_S2_PGTABLE_NOPMD_H_
-#define __ARM64_S2_PGTABLE_NOPMD_H_
-
-#include <asm/stage2_pgtable-nopud.h>
-
-#define __S2_PGTABLE_PMD_FOLDED
-
-#define S2_PMD_SHIFT S2_PUD_SHIFT
-#define S2_PTRS_PER_PMD 1
-#define S2_PMD_SIZE (1UL << S2_PMD_SHIFT)
-#define S2_PMD_MASK (~(S2_PMD_SIZE-1))
-
-#define stage2_pud_none(kvm, pud) (0)
-#define stage2_pud_present(kvm, pud) (1)
-#define stage2_pud_clear(kvm, pud) do { } while (0)
-#define stage2_pud_populate(kvm, pud, pmd) do { } while (0)
-#define stage2_pmd_offset(kvm, pud, address) ((pmd_t *)(pud))
-
-#define stage2_pmd_free(kvm, pmd) do { } while (0)
-
-#define stage2_pmd_addr_end(kvm, addr, end) (end)
-
-#define stage2_pud_huge(kvm, pud) (0)
-#define stage2_pmd_table_empty(kvm, pmdp) (0)
-
-#endif
diff --git a/arch/arm64/include/asm/stage2_pgtable-nopud.h b/arch/arm64/include/asm/stage2_pgtable-nopud.h
deleted file mode 100644
index cd6304e203be..000000000000
--- a/arch/arm64/include/asm/stage2_pgtable-nopud.h
+++ /dev/null
@@ -1,39 +0,0 @@
-/*
- * Copyright (C) 2016 - ARM Ltd
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-
-#ifndef __ARM64_S2_PGTABLE_NOPUD_H_
-#define __ARM64_S2_PGTABLE_NOPUD_H_
-
-#define __S2_PGTABLE_PUD_FOLDED
-
-#define S2_PUD_SHIFT S2_PGDIR_SHIFT
-#define S2_PTRS_PER_PUD 1
-#define S2_PUD_SIZE (_AC(1, UL) << S2_PUD_SHIFT)
-#define S2_PUD_MASK (~(S2_PUD_SIZE-1))
-
-#define stage2_pgd_none(kvm, pgd) (0)
-#define stage2_pgd_present(kvm, pgd) (1)
-#define stage2_pgd_clear(kvm, pgd) do { } while (0)
-#define stage2_pgd_populate(kvm, pgd, pud) do { } while (0)
-
-#define stage2_pud_offset(kvm, pgd, address) ((pud_t *)(pgd))
-
-#define stage2_pud_free(kvm, x) do { } while (0)
-
-#define stage2_pud_addr_end(kvm, addr, end) (end)
-#define stage2_pud_table_empty(kvm, pmdp) (0)
-
-#endif
diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h
index 057a405fa727..33e8ebb25037 100644
--- a/arch/arm64/include/asm/stage2_pgtable.h
+++ b/arch/arm64/include/asm/stage2_pgtable.h
@@ -21,6 +21,9 @@
#include <asm/pgtable.h>
+/* The PGDIR shift for a given page table with "n" levels. */
+#define pt_levels_pgdir_shift(n) ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - (n))
+
/*
* The hardware supports concatenation of up to 16 tables at stage2 entry level
* and we use the feature whenever possible.
@@ -29,118 +32,184 @@
* On arm64, the smallest PAGE_SIZE supported is 4k, which means
* (PAGE_SHIFT - 3) > 4 holds for all page sizes.
* This implies, the total number of page table levels at stage2 expected
- * by the hardware is actually the number of levels required for (KVM_PHYS_SHIFT - 4)
+ * by the hardware is actually the number of levels required for (IPA_SHIFT - 4)
* in normal translations(e.g, stage1), since we cannot have another level in
- * the range (KVM_PHYS_SHIFT, KVM_PHYS_SHIFT - 4).
- */
-#define STAGE2_PGTABLE_LEVELS ARM64_HW_PGTABLE_LEVELS(KVM_PHYS_SHIFT - 4)
-
-/*
- * With all the supported VA_BITs and 40bit guest IPA, the following condition
- * is always true:
- *
- * STAGE2_PGTABLE_LEVELS <= CONFIG_PGTABLE_LEVELS
- *
- * We base our stage-2 page table walker helpers on this assumption and
- * fall back to using the host version of the helper wherever possible.
- * i.e, if a particular level is not folded (e.g, PUD) at stage2, we fall back
- * to using the host version, since it is guaranteed it is not folded@host.
- *
- * If the condition breaks in the future, we can rearrange the host level
- * definitions and reuse them for stage2. Till then...
+ * the range (IPA_SHIFT, IPA_SHIFT - 4).
*/
-#if STAGE2_PGTABLE_LEVELS > CONFIG_PGTABLE_LEVELS
-#error "Unsupported combination of guest IPA and host VA_BITS."
-#endif
-
-/* S2_PGDIR_SHIFT is the size mapped by top-level stage2 entry */
-#define S2_PGDIR_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - STAGE2_PGTABLE_LEVELS)
-#define S2_PGDIR_SIZE (_AC(1, UL) << S2_PGDIR_SHIFT)
-#define S2_PGDIR_MASK (~(S2_PGDIR_SIZE - 1))
+#define stage2_pt_levels(ipa_shift) ARM64_HW_PGTABLE_LEVELS((ipa_shift) - 4)
/*
* The number of PTRS across all concatenated stage2 tables given by the
* number of bits resolved@the initial level.
*/
-#define PTRS_PER_S2_PGD (1 << (KVM_PHYS_SHIFT - S2_PGDIR_SHIFT))
+#define __s2_pgd_ptrs(pa, lvls) (1 << ((pa) - pt_levels_pgdir_shift((lvls))))
+
+#define kvm_stage2_levels(kvm) stage2_pt_levels(kvm_phys_shift(kvm))
+#define stage2_pgdir_shift(kvm) \
+ pt_levels_pgdir_shift(kvm_stage2_levels(kvm))
+#define stage2_pgdir_size(kvm) (_AC(1, UL) << stage2_pgdir_shift((kvm)))
+#define stage2_pgdir_mask(kvm) (~(stage2_pgdir_size((kvm)) - 1))
+#define stage2_pgd_ptrs(kvm) \
+ __s2_pgd_ptrs(kvm_phys_shift(kvm), kvm_stage2_levels(kvm))
+
/*
* kvm_mmmu_cache_min_pages is the number of stage2 page table translation
* levels in addition to the PGD.
*/
-#define kvm_mmu_cache_min_pages(kvm) (STAGE2_PGTABLE_LEVELS - 1)
+#define kvm_mmu_cache_min_pages(kvm) (kvm_stage2_levels(kvm) - 1)
+
+/* PUD/PMD definitions if present */
+#define __S2_PUD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(1)
+#define __S2_PUD_SIZE (_AC(1, UL) << __S2_PUD_SHIFT)
+#define __S2_PUD_MASK (~(__S2_PUD_SIZE - 1))
-#if STAGE2_PGTABLE_LEVELS > 3
+#define __S2_PMD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
+#define __S2_PMD_SIZE (_AC(1, UL) << __S2_PMD_SHIFT)
+#define __S2_PMD_MASK (~(__S2_PMD_SIZE - 1))
-#define S2_PUD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(1)
-#define S2_PUD_SIZE (_AC(1, UL) << S2_PUD_SHIFT)
-#define S2_PUD_MASK (~(S2_PUD_SIZE - 1))
+#define __s2_pud_index(addr) \
+ (((addr) >> __S2_PUD_SHIFT) & (PTRS_PER_PTE - 1))
+#define __s2_pmd_index(addr) \
+ (((addr) >> __S2_PMD_SHIFT) & (PTRS_PER_PTE - 1))
+
+static inline int stage2_pgd_none(struct kvm *kvm, pgd_t pgd)
+{
+ return (kvm_stage2_levels(kvm) > 3) ? __raw_pgd_none(pgd) : 0;
+}
+
+static inline void stage2_pgd_clear(struct kvm *kvm, pgd_t *pgdp)
+{
+ if (kvm_stage2_levels(kvm) > 3)
+ __raw_pgd_clear(pgdp);
+}
+
+static inline int stage2_pgd_present(struct kvm *kvm, pgd_t pgd)
+{
+ return kvm_stage2_levels(kvm) > 3 ? __raw_pgd_present(pgd) : 1;
+}
+
+static inline void stage2_pgd_populate(struct kvm *kvm, pgd_t *pgdp, pud_t *pud)
+{
+ if (kvm_stage2_levels(kvm) > 3)
+ __raw_pgd_populate(pgdp, __pa(pud), PUD_TYPE_TABLE);
+ else
+ BUG();
+}
+
+static inline pud_t *stage2_pud_offset(struct kvm *kvm,
+ pgd_t *pgd, unsigned long address)
+{
+ if (kvm_stage2_levels(kvm) > 3) {
+ phys_addr_t pud_phys = __raw_pgd_page_paddr(*pgd);
+
+ pud_phys += __s2_pud_index(address) * sizeof(pud_t);
+ return __va(pud_phys);
+ }
+ return (pud_t *)pgd;
+}
-#define stage2_pgd_none(kvm, pgd) pgd_none(pgd)
-#define stage2_pgd_clear(kvm, pgd) pgd_clear(pgd)
-#define stage2_pgd_present(kvm, pgd) pgd_present(pgd)
-#define stage2_pgd_populate(kvm, pgd, pud) pgd_populate(NULL, pgd, pud)
-#define stage2_pud_offset(kvm, pgd, address) pud_offset(pgd, address)
-#define stage2_pud_free(kvm, pud) pud_free(NULL, pud)
+static inline void stage2_pud_free(struct kvm *kvm, pud_t *pud)
+{
+ if (kvm_stage2_levels(kvm) > 3)
+ __raw_pud_free(pud);
+}
-#define stage2_pud_table_empty(kvm, pudp) kvm_page_empty(pudp)
+static inline int stage2_pud_table_empty(struct kvm *kvm, pud_t *pudp)
+{
+ return kvm_stage2_levels(kvm) > 3 && kvm_page_empty(pudp);
+}
static inline phys_addr_t
stage2_pud_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
- phys_addr_t boundary = (addr + S2_PUD_SIZE) & S2_PUD_MASK;
+ if (kvm_stage2_levels(kvm) > 3) {
+ phys_addr_t boundary = (addr + __S2_PUD_SIZE) & __S2_PUD_MASK;
- return (boundary - 1 < end - 1) ? boundary : end;
+ return (boundary - 1 < end - 1) ? boundary : end;
+ }
+ return end;
}
-#endif /* STAGE2_PGTABLE_LEVELS > 3 */
+static inline int stage2_pud_none(struct kvm *kvm, pud_t pud)
+{
+ return kvm_stage2_levels(kvm) > 2 ? __raw_pud_none(pud) : 0;
+}
+static inline void stage2_pud_clear(struct kvm *kvm, pud_t *pudp)
+{
+ if (kvm_stage2_levels(kvm) > 2)
+ __raw_pud_clear(pudp);
+}
-#if STAGE2_PGTABLE_LEVELS > 2
+static inline int stage2_pud_present(struct kvm *kvm, pud_t pud)
+{
+ return kvm_stage2_levels(kvm) > 2 ? __raw_pud_present(pud) : 1;
+}
-#define S2_PMD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
-#define S2_PMD_SIZE (_AC(1, UL) << S2_PMD_SHIFT)
-#define S2_PMD_MASK (~(S2_PMD_SIZE - 1))
+static inline void stage2_pud_populate(struct kvm *kvm, pud_t *pudp, pmd_t *pmd)
+{
+ if (kvm_stage2_levels(kvm) > 2)
+ __raw_pud_populate(pudp, __pa(pmd), PMD_TYPE_TABLE);
+ else
+ BUG();
+}
-#define stage2_pud_none(kvm, pud) pud_none(pud)
-#define stage2_pud_clear(kvm, pud) pud_clear(pud)
-#define stage2_pud_present(kvm, pud) pud_present(pud)
-#define stage2_pud_populate(kvm, pud, pmd) pud_populate(NULL, pud, pmd)
-#define stage2_pmd_offset(kvm, pud, address) pmd_offset(pud, address)
-#define stage2_pmd_free(kvm, pmd) pmd_free(NULL, pmd)
+static inline pmd_t *stage2_pmd_offset(struct kvm *kvm,
+ pud_t *pud, unsigned long address)
+{
+ if (kvm_stage2_levels(kvm) > 2) {
+ phys_addr_t pmd_phys = __raw_pud_page_paddr(*pud);
-#define stage2_pud_huge(kvm, pud) pud_huge(pud)
-#define stage2_pmd_table_empty(kvm, pmdp) kvm_page_empty(pmdp)
+ pmd_phys += __s2_pmd_index(address) * sizeof(pmd_t);
+ return __va(pmd_phys);
+ }
+ return (pmd_t *)pud;
+}
+
+static inline void stage2_pmd_free(struct kvm *kvm, pmd_t *pmd)
+{
+ if (kvm_stage2_levels(kvm) > 2)
+ __raw_pmd_free(pmd);
+}
+
+static inline int stage2_pmd_table_empty(struct kvm *kvm, pmd_t *pmdp)
+{
+ return kvm_stage2_levels(kvm) > 2 && kvm_page_empty(pmdp);
+}
static inline phys_addr_t
stage2_pmd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
- phys_addr_t boundary = (addr + S2_PMD_SIZE) & S2_PMD_MASK;
+ if (kvm_stage2_levels(kvm) > 2) {
+ phys_addr_t boundary = (addr + __S2_PMD_SIZE) & __S2_PMD_MASK;
- return (boundary - 1 < end - 1) ? boundary : end;
+ return (boundary - 1 < end - 1) ? boundary : end;
+ }
+ return end;
}
-#endif /* STAGE2_PGTABLE_LEVELS > 2 */
+static inline int stage2_pud_huge(struct kvm *kvm, pud_t pud)
+{
+ return kvm_stage2_levels(kvm) > 2 ? __raw_pud_huge(pud) : 0;
+}
#define stage2_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
-#if STAGE2_PGTABLE_LEVELS == 2
-#include <asm/stage2_pgtable-nopmd.h>
-#elif STAGE2_PGTABLE_LEVELS == 3
-#include <asm/stage2_pgtable-nopud.h>
-#endif
+#define stage2_pgd_size(kvm) (stage2_pgd_ptrs(kvm) * sizeof(pgd_t))
-#define stage2_pgd_size(kvm) (PTRS_PER_S2_PGD * sizeof(pgd_t))
-
-#define stage2_pgd_index(kvm, addr) \
- (((addr) >> S2_PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
+static inline unsigned long stage2_pgd_index(struct kvm *kvm, phys_addr_t addr)
+{
+ return (addr >> stage2_pgdir_shift(kvm)) & (stage2_pgd_ptrs(kvm) - 1);
+}
static inline phys_addr_t
stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
- phys_addr_t boundary = (addr + S2_PGDIR_SIZE) & S2_PGDIR_MASK;
+ phys_addr_t boundary;
+ boundary = (addr + stage2_pgdir_size(kvm)) & stage2_pgdir_mask(kvm);
return (boundary - 1 < end - 1) ? boundary : end;
}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index d06f00566664..8564ed907b18 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -502,7 +502,7 @@ static void update_vttbr(struct kvm *kvm)
/* update vttbr to be used with the new vmid */
pgd_phys = virt_to_phys(kvm->arch.pgd);
- BUG_ON(pgd_phys & ~VTTBR_BADDR_MASK);
+ BUG_ON(pgd_phys & ~kvm_vttbr_baddr_mask(kvm));
vmid = ((u64)(kvm->arch.vmid) << VTTBR_VMID_SHIFT) & VTTBR_VMID_MASK(kvm_vmid_bits);
kvm->arch.vttbr = kvm_phys_to_vttbr(pgd_phys) | vmid;
--
2.13.6
^ permalink raw reply related
* [PATCH v1 10/16] kvm: arm/arm64: Prepare for VM specific stage2 translations
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
Right now the stage2 page table for a VM is hard coded, assuming
an IPA of 40bits. As we are about to add support for per VM IPA,
prepare the stage2 page table helpers to accept the kvm instance
to make the right decision. No functional changes.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm/include/asm/kvm_arm.h | 2 -
arch/arm/include/asm/kvm_mmu.h | 11 ++-
arch/arm/include/asm/stage2_pgtable.h | 46 ++++++-----
arch/arm64/include/asm/kvm_mmu.h | 6 +-
arch/arm64/include/asm/pgtable.h | 2 +-
arch/arm64/include/asm/stage2_pgtable-nopmd.h | 18 ++--
arch/arm64/include/asm/stage2_pgtable-nopud.h | 16 ++--
arch/arm64/include/asm/stage2_pgtable.h | 49 ++++++-----
virt/kvm/arm/mmu.c | 114 +++++++++++++-------------
virt/kvm/arm/vgic/vgic-kvm-device.c | 2 +-
10 files changed, 140 insertions(+), 126 deletions(-)
diff --git a/arch/arm/include/asm/kvm_arm.h b/arch/arm/include/asm/kvm_arm.h
index 3ab8b3781bfe..4ebaf0c29723 100644
--- a/arch/arm/include/asm/kvm_arm.h
+++ b/arch/arm/include/asm/kvm_arm.h
@@ -133,8 +133,6 @@
* space.
*/
#define KVM_PHYS_SHIFT (40)
-#define KVM_PHYS_SIZE (_AC(1, ULL) << KVM_PHYS_SHIFT)
-#define KVM_PHYS_MASK (KVM_PHYS_SIZE - _AC(1, ULL))
#define PTRS_PER_S2_PGD (_AC(1, ULL) << (KVM_PHYS_SHIFT - 30))
/* Virtualization Translation Control Register (VTCR) bits */
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 8c5643e2eea4..a3312f87a6e0 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -29,17 +29,24 @@
#define kern_hyp_va(kva) (kva)
/*
- * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation levels.
+ * kvm_mmu_cache_min_pages is the number of stage2 page table
+ * translation levels excluding the top level.
*/
-#define KVM_MMU_CACHE_MIN_PAGES 2
+#define kvm_mmu_cache_min_pages(kvm) 2
#ifndef __ASSEMBLY__
#include <linux/highmem.h>
#include <asm/cacheflush.h>
+#include <asm/kvm_arm.h>
#include <asm/pgalloc.h>
#include <asm/stage2_pgtable.h>
+#define kvm_phys_shift(kvm) KVM_PHYS_SHIFT
+#define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm))
+#define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL))
+
+#define stage2_pgd_size(kvm) (PTRS_PER_S2_PGD * sizeof(pgd_t))
int create_hyp_mappings(void *from, void *to, pgprot_t prot);
int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
void free_hyp_pgds(void);
diff --git a/arch/arm/include/asm/stage2_pgtable.h b/arch/arm/include/asm/stage2_pgtable.h
index 460d616bb2d6..e22ae94f0bf9 100644
--- a/arch/arm/include/asm/stage2_pgtable.h
+++ b/arch/arm/include/asm/stage2_pgtable.h
@@ -19,43 +19,45 @@
#ifndef __ARM_S2_PGTABLE_H_
#define __ARM_S2_PGTABLE_H_
-#define stage2_pgd_none(pgd) pgd_none(pgd)
-#define stage2_pgd_clear(pgd) pgd_clear(pgd)
-#define stage2_pgd_present(pgd) pgd_present(pgd)
-#define stage2_pgd_populate(pgd, pud) pgd_populate(NULL, pgd, pud)
-#define stage2_pud_offset(pgd, address) pud_offset(pgd, address)
-#define stage2_pud_free(pud) pud_free(NULL, pud)
-
-#define stage2_pud_none(pud) pud_none(pud)
-#define stage2_pud_clear(pud) pud_clear(pud)
-#define stage2_pud_present(pud) pud_present(pud)
-#define stage2_pud_populate(pud, pmd) pud_populate(NULL, pud, pmd)
-#define stage2_pmd_offset(pud, address) pmd_offset(pud, address)
-#define stage2_pmd_free(pmd) pmd_free(NULL, pmd)
-
-#define stage2_pud_huge(pud) pud_huge(pud)
+#define stage2_pgd_none(kvm, pgd) pgd_none(pgd)
+#define stage2_pgd_clear(kvm, pgd) pgd_clear(pgd)
+#define stage2_pgd_present(kvm, pgd) pgd_present(pgd)
+#define stage2_pgd_populate(kvm, pgd, pud) pgd_populate(NULL, pgd, pud)
+#define stage2_pud_offset(kvm, pgd, address) pud_offset(pgd, address)
+#define stage2_pud_free(kvm, pud) pud_free(NULL, pud)
+
+#define stage2_pud_none(kvm, pud) pud_none(pud)
+#define stage2_pud_clear(kvm, pud) pud_clear(pud)
+#define stage2_pud_present(kvm, pud) pud_present(pud)
+#define stage2_pud_populate(kvm, pud, pmd) pud_populate(NULL, pud, pmd)
+#define stage2_pmd_offset(kvm, pud, address) pmd_offset(pud, address)
+#define stage2_pmd_free(kvm, pmd) pmd_free(NULL, pmd)
+
+#define stage2_pud_huge(kvm, pud) pud_huge(pud)
/* Open coded p*d_addr_end that can deal with 64bit addresses */
-static inline phys_addr_t stage2_pgd_addr_end(phys_addr_t addr, phys_addr_t end)
+static inline phys_addr_t
+stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
phys_addr_t boundary = (addr + PGDIR_SIZE) & PGDIR_MASK;
return (boundary - 1 < end - 1) ? boundary : end;
}
-#define stage2_pud_addr_end(addr, end) (end)
+#define stage2_pud_addr_end(kvm, addr, end) (end)
-static inline phys_addr_t stage2_pmd_addr_end(phys_addr_t addr, phys_addr_t end)
+static inline phys_addr_t
+stage2_pmd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
phys_addr_t boundary = (addr + PMD_SIZE) & PMD_MASK;
return (boundary - 1 < end - 1) ? boundary : end;
}
-#define stage2_pgd_index(addr) pgd_index(addr)
+#define stage2_pgd_index(kvm, addr) pgd_index(addr)
-#define stage2_pte_table_empty(ptep) kvm_page_empty(ptep)
-#define stage2_pmd_table_empty(pmdp) kvm_page_empty(pmdp)
-#define stage2_pud_table_empty(pudp) false
+#define stage2_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
+#define stage2_pmd_table_empty(kvm, pmdp) kvm_page_empty(pmdp)
+#define stage2_pud_table_empty(kvm, pudp) false
#endif /* __ARM_S2_PGTABLE_H_ */
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index b33bdb5eeb3d..de542aa72d80 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -134,8 +134,10 @@ static inline unsigned long __kern_hyp_va(unsigned long v)
* We currently only support a 40bit IPA.
*/
#define KVM_PHYS_SHIFT (40)
-#define KVM_PHYS_SIZE (1UL << KVM_PHYS_SHIFT)
-#define KVM_PHYS_MASK (KVM_PHYS_SIZE - 1UL)
+
+#define kvm_phys_shift(kvm) KVM_PHYS_SHIFT
+#define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm))
+#define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL))
#include <asm/stage2_pgtable.h>
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index a1c6e93a1a11..50d72be469ca 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -467,7 +467,7 @@ static inline phys_addr_t pmd_page_paddr(pmd_t pmd)
#define __raw_pud_none(pud) (!pud_val((pud)))
#define __raw_pud_bad(pud) (!(pud_val((pud)) & PUD_TABLE_BIT))
-#define __raw_pud_present(pud) pte_present(pud_pte((pud)))
+#define __raw_pud_present(pud) pte_present(pud_pte((pud)))
static inline int __raw_pud_huge(pud_t pud)
{
diff --git a/arch/arm64/include/asm/stage2_pgtable-nopmd.h b/arch/arm64/include/asm/stage2_pgtable-nopmd.h
index 2656a0fd05a6..0280dedbf75f 100644
--- a/arch/arm64/include/asm/stage2_pgtable-nopmd.h
+++ b/arch/arm64/include/asm/stage2_pgtable-nopmd.h
@@ -26,17 +26,17 @@
#define S2_PMD_SIZE (1UL << S2_PMD_SHIFT)
#define S2_PMD_MASK (~(S2_PMD_SIZE-1))
-#define stage2_pud_none(pud) (0)
-#define stage2_pud_present(pud) (1)
-#define stage2_pud_clear(pud) do { } while (0)
-#define stage2_pud_populate(pud, pmd) do { } while (0)
-#define stage2_pmd_offset(pud, address) ((pmd_t *)(pud))
+#define stage2_pud_none(kvm, pud) (0)
+#define stage2_pud_present(kvm, pud) (1)
+#define stage2_pud_clear(kvm, pud) do { } while (0)
+#define stage2_pud_populate(kvm, pud, pmd) do { } while (0)
+#define stage2_pmd_offset(kvm, pud, address) ((pmd_t *)(pud))
-#define stage2_pmd_free(pmd) do { } while (0)
+#define stage2_pmd_free(kvm, pmd) do { } while (0)
-#define stage2_pmd_addr_end(addr, end) (end)
+#define stage2_pmd_addr_end(kvm, addr, end) (end)
-#define stage2_pud_huge(pud) (0)
-#define stage2_pmd_table_empty(pmdp) (0)
+#define stage2_pud_huge(kvm, pud) (0)
+#define stage2_pmd_table_empty(kvm, pmdp) (0)
#endif
diff --git a/arch/arm64/include/asm/stage2_pgtable-nopud.h b/arch/arm64/include/asm/stage2_pgtable-nopud.h
index 5ee87b54ebf3..cd6304e203be 100644
--- a/arch/arm64/include/asm/stage2_pgtable-nopud.h
+++ b/arch/arm64/include/asm/stage2_pgtable-nopud.h
@@ -24,16 +24,16 @@
#define S2_PUD_SIZE (_AC(1, UL) << S2_PUD_SHIFT)
#define S2_PUD_MASK (~(S2_PUD_SIZE-1))
-#define stage2_pgd_none(pgd) (0)
-#define stage2_pgd_present(pgd) (1)
-#define stage2_pgd_clear(pgd) do { } while (0)
-#define stage2_pgd_populate(pgd, pud) do { } while (0)
+#define stage2_pgd_none(kvm, pgd) (0)
+#define stage2_pgd_present(kvm, pgd) (1)
+#define stage2_pgd_clear(kvm, pgd) do { } while (0)
+#define stage2_pgd_populate(kvm, pgd, pud) do { } while (0)
-#define stage2_pud_offset(pgd, address) ((pud_t *)(pgd))
+#define stage2_pud_offset(kvm, pgd, address) ((pud_t *)(pgd))
-#define stage2_pud_free(x) do { } while (0)
+#define stage2_pud_free(kvm, x) do { } while (0)
-#define stage2_pud_addr_end(addr, end) (end)
-#define stage2_pud_table_empty(pmdp) (0)
+#define stage2_pud_addr_end(kvm, addr, end) (end)
+#define stage2_pud_table_empty(kvm, pmdp) (0)
#endif
diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h
index 8b68099348e5..057a405fa727 100644
--- a/arch/arm64/include/asm/stage2_pgtable.h
+++ b/arch/arm64/include/asm/stage2_pgtable.h
@@ -65,10 +65,10 @@
#define PTRS_PER_S2_PGD (1 << (KVM_PHYS_SHIFT - S2_PGDIR_SHIFT))
/*
- * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
+ * kvm_mmmu_cache_min_pages is the number of stage2 page table translation
* levels in addition to the PGD.
*/
-#define KVM_MMU_CACHE_MIN_PAGES (STAGE2_PGTABLE_LEVELS - 1)
+#define kvm_mmu_cache_min_pages(kvm) (STAGE2_PGTABLE_LEVELS - 1)
#if STAGE2_PGTABLE_LEVELS > 3
@@ -77,16 +77,17 @@
#define S2_PUD_SIZE (_AC(1, UL) << S2_PUD_SHIFT)
#define S2_PUD_MASK (~(S2_PUD_SIZE - 1))
-#define stage2_pgd_none(pgd) pgd_none(pgd)
-#define stage2_pgd_clear(pgd) pgd_clear(pgd)
-#define stage2_pgd_present(pgd) pgd_present(pgd)
-#define stage2_pgd_populate(pgd, pud) pgd_populate(NULL, pgd, pud)
-#define stage2_pud_offset(pgd, address) pud_offset(pgd, address)
-#define stage2_pud_free(pud) pud_free(NULL, pud)
+#define stage2_pgd_none(kvm, pgd) pgd_none(pgd)
+#define stage2_pgd_clear(kvm, pgd) pgd_clear(pgd)
+#define stage2_pgd_present(kvm, pgd) pgd_present(pgd)
+#define stage2_pgd_populate(kvm, pgd, pud) pgd_populate(NULL, pgd, pud)
+#define stage2_pud_offset(kvm, pgd, address) pud_offset(pgd, address)
+#define stage2_pud_free(kvm, pud) pud_free(NULL, pud)
-#define stage2_pud_table_empty(pudp) kvm_page_empty(pudp)
+#define stage2_pud_table_empty(kvm, pudp) kvm_page_empty(pudp)
-static inline phys_addr_t stage2_pud_addr_end(phys_addr_t addr, phys_addr_t end)
+static inline phys_addr_t
+stage2_pud_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
phys_addr_t boundary = (addr + S2_PUD_SIZE) & S2_PUD_MASK;
@@ -102,17 +103,18 @@ static inline phys_addr_t stage2_pud_addr_end(phys_addr_t addr, phys_addr_t end)
#define S2_PMD_SIZE (_AC(1, UL) << S2_PMD_SHIFT)
#define S2_PMD_MASK (~(S2_PMD_SIZE - 1))
-#define stage2_pud_none(pud) pud_none(pud)
-#define stage2_pud_clear(pud) pud_clear(pud)
-#define stage2_pud_present(pud) pud_present(pud)
-#define stage2_pud_populate(pud, pmd) pud_populate(NULL, pud, pmd)
-#define stage2_pmd_offset(pud, address) pmd_offset(pud, address)
-#define stage2_pmd_free(pmd) pmd_free(NULL, pmd)
+#define stage2_pud_none(kvm, pud) pud_none(pud)
+#define stage2_pud_clear(kvm, pud) pud_clear(pud)
+#define stage2_pud_present(kvm, pud) pud_present(pud)
+#define stage2_pud_populate(kvm, pud, pmd) pud_populate(NULL, pud, pmd)
+#define stage2_pmd_offset(kvm, pud, address) pmd_offset(pud, address)
+#define stage2_pmd_free(kvm, pmd) pmd_free(NULL, pmd)
-#define stage2_pud_huge(pud) pud_huge(pud)
-#define stage2_pmd_table_empty(pmdp) kvm_page_empty(pmdp)
+#define stage2_pud_huge(kvm, pud) pud_huge(pud)
+#define stage2_pmd_table_empty(kvm, pmdp) kvm_page_empty(pmdp)
-static inline phys_addr_t stage2_pmd_addr_end(phys_addr_t addr, phys_addr_t end)
+static inline phys_addr_t
+stage2_pmd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
phys_addr_t boundary = (addr + S2_PMD_SIZE) & S2_PMD_MASK;
@@ -121,7 +123,7 @@ static inline phys_addr_t stage2_pmd_addr_end(phys_addr_t addr, phys_addr_t end)
#endif /* STAGE2_PGTABLE_LEVELS > 2 */
-#define stage2_pte_table_empty(ptep) kvm_page_empty(ptep)
+#define stage2_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
#if STAGE2_PGTABLE_LEVELS == 2
#include <asm/stage2_pgtable-nopmd.h>
@@ -129,10 +131,13 @@ static inline phys_addr_t stage2_pmd_addr_end(phys_addr_t addr, phys_addr_t end)
#include <asm/stage2_pgtable-nopud.h>
#endif
+#define stage2_pgd_size(kvm) (PTRS_PER_S2_PGD * sizeof(pgd_t))
-#define stage2_pgd_index(addr) (((addr) >> S2_PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
+#define stage2_pgd_index(kvm, addr) \
+ (((addr) >> S2_PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
-static inline phys_addr_t stage2_pgd_addr_end(phys_addr_t addr, phys_addr_t end)
+static inline phys_addr_t
+stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
{
phys_addr_t boundary = (addr + S2_PGDIR_SIZE) & S2_PGDIR_MASK;
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index 257f2a8ccfc7..cd355aa70c61 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -43,7 +43,6 @@ static unsigned long hyp_idmap_start;
static unsigned long hyp_idmap_end;
static phys_addr_t hyp_idmap_vector;
-#define S2_PGD_SIZE (PTRS_PER_S2_PGD * sizeof(pgd_t))
#define hyp_pgd_order get_order(PTRS_PER_PGD * sizeof(pgd_t))
#define KVM_S2PTE_FLAG_IS_IOMAP (1UL << 0)
@@ -148,20 +147,20 @@ static void *mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc)
static void clear_stage2_pgd_entry(struct kvm *kvm, pgd_t *pgd, phys_addr_t addr)
{
- pud_t *pud_table __maybe_unused = stage2_pud_offset(pgd, 0UL);
- stage2_pgd_clear(pgd);
+ pud_t *pud_table __maybe_unused = stage2_pud_offset(kvm, pgd, 0UL);
+ stage2_pgd_clear(kvm, pgd);
kvm_tlb_flush_vmid_ipa(kvm, addr);
- stage2_pud_free(pud_table);
+ stage2_pud_free(kvm, pud_table);
put_page(virt_to_page(pgd));
}
static void clear_stage2_pud_entry(struct kvm *kvm, pud_t *pud, phys_addr_t addr)
{
- pmd_t *pmd_table __maybe_unused = stage2_pmd_offset(pud, 0);
- VM_BUG_ON(stage2_pud_huge(*pud));
- stage2_pud_clear(pud);
+ pmd_t *pmd_table __maybe_unused = stage2_pmd_offset(kvm, pud, 0);
+ VM_BUG_ON(stage2_pud_huge(kvm, *pud));
+ stage2_pud_clear(kvm, pud);
kvm_tlb_flush_vmid_ipa(kvm, addr);
- stage2_pmd_free(pmd_table);
+ stage2_pmd_free(kvm, pmd_table);
put_page(virt_to_page(pud));
}
@@ -217,7 +216,7 @@ static void unmap_stage2_ptes(struct kvm *kvm, pmd_t *pmd,
}
} while (pte++, addr += PAGE_SIZE, addr != end);
- if (stage2_pte_table_empty(start_pte))
+ if (stage2_pte_table_empty(kvm, start_pte))
clear_stage2_pmd_entry(kvm, pmd, start_addr);
}
@@ -227,9 +226,9 @@ static void unmap_stage2_pmds(struct kvm *kvm, pud_t *pud,
phys_addr_t next, start_addr = addr;
pmd_t *pmd, *start_pmd;
- start_pmd = pmd = stage2_pmd_offset(pud, addr);
+ start_pmd = pmd = stage2_pmd_offset(kvm, pud, addr);
do {
- next = stage2_pmd_addr_end(addr, end);
+ next = stage2_pmd_addr_end(kvm, addr, end);
if (!pmd_none(*pmd)) {
if (pmd_thp_or_huge(*pmd)) {
pmd_t old_pmd = *pmd;
@@ -246,7 +245,7 @@ static void unmap_stage2_pmds(struct kvm *kvm, pud_t *pud,
}
} while (pmd++, addr = next, addr != end);
- if (stage2_pmd_table_empty(start_pmd))
+ if (stage2_pmd_table_empty(kvm, start_pmd))
clear_stage2_pud_entry(kvm, pud, start_addr);
}
@@ -256,14 +255,14 @@ static void unmap_stage2_puds(struct kvm *kvm, pgd_t *pgd,
phys_addr_t next, start_addr = addr;
pud_t *pud, *start_pud;
- start_pud = pud = stage2_pud_offset(pgd, addr);
+ start_pud = pud = stage2_pud_offset(kvm, pgd, addr);
do {
- next = stage2_pud_addr_end(addr, end);
- if (!stage2_pud_none(*pud)) {
- if (stage2_pud_huge(*pud)) {
+ next = stage2_pud_addr_end(kvm, addr, end);
+ if (!stage2_pud_none(kvm, *pud)) {
+ if (stage2_pud_huge(kvm, *pud)) {
pud_t old_pud = *pud;
- stage2_pud_clear(pud);
+ stage2_pud_clear(kvm, pud);
kvm_tlb_flush_vmid_ipa(kvm, addr);
kvm_flush_dcache_pud(old_pud);
put_page(virt_to_page(pud));
@@ -273,7 +272,7 @@ static void unmap_stage2_puds(struct kvm *kvm, pgd_t *pgd,
}
} while (pud++, addr = next, addr != end);
- if (stage2_pud_table_empty(start_pud))
+ if (stage2_pud_table_empty(kvm, start_pud))
clear_stage2_pgd_entry(kvm, pgd, start_addr);
}
@@ -295,7 +294,7 @@ static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
phys_addr_t next;
assert_spin_locked(&kvm->mmu_lock);
- pgd = kvm->arch.pgd + stage2_pgd_index(addr);
+ pgd = kvm->arch.pgd + stage2_pgd_index(kvm, addr);
do {
/*
* The page table shouldn't be free'd as we still hold a reference
@@ -303,8 +302,8 @@ static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
*/
if (WARN_ON(!READ_ONCE(kvm->arch.pgd)))
break;
- next = stage2_pgd_addr_end(addr, end);
- if (!stage2_pgd_none(*pgd))
+ next = stage2_pgd_addr_end(kvm, addr, end);
+ if (!stage2_pgd_none(kvm, *pgd))
unmap_stage2_puds(kvm, pgd, addr, next);
/*
* If the range is too large, release the kvm->mmu_lock
@@ -333,9 +332,9 @@ static void stage2_flush_pmds(struct kvm *kvm, pud_t *pud,
pmd_t *pmd;
phys_addr_t next;
- pmd = stage2_pmd_offset(pud, addr);
+ pmd = stage2_pmd_offset(kvm, pud, addr);
do {
- next = stage2_pmd_addr_end(addr, end);
+ next = stage2_pmd_addr_end(kvm, addr, end);
if (!pmd_none(*pmd)) {
if (pmd_thp_or_huge(*pmd))
kvm_flush_dcache_pmd(*pmd);
@@ -351,11 +350,11 @@ static void stage2_flush_puds(struct kvm *kvm, pgd_t *pgd,
pud_t *pud;
phys_addr_t next;
- pud = stage2_pud_offset(pgd, addr);
+ pud = stage2_pud_offset(kvm, pgd, addr);
do {
- next = stage2_pud_addr_end(addr, end);
- if (!stage2_pud_none(*pud)) {
- if (stage2_pud_huge(*pud))
+ next = stage2_pud_addr_end(kvm, addr, end);
+ if (!stage2_pud_none(kvm, *pud)) {
+ if (stage2_pud_huge(kvm, *pud))
kvm_flush_dcache_pud(*pud);
else
stage2_flush_pmds(kvm, pud, addr, next);
@@ -371,10 +370,10 @@ static void stage2_flush_memslot(struct kvm *kvm,
phys_addr_t next;
pgd_t *pgd;
- pgd = kvm->arch.pgd + stage2_pgd_index(addr);
+ pgd = kvm->arch.pgd + stage2_pgd_index(kvm, addr);
do {
- next = stage2_pgd_addr_end(addr, end);
- if (!stage2_pgd_none(*pgd))
+ next = stage2_pgd_addr_end(kvm, addr, end);
+ if (!stage2_pgd_none(kvm, *pgd))
stage2_flush_puds(kvm, pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
@@ -762,7 +761,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
}
/* Allocate the HW PGD, making sure that each page gets its own refcount */
- pgd = alloc_pages_exact(S2_PGD_SIZE, GFP_KERNEL | __GFP_ZERO);
+ pgd = alloc_pages_exact(stage2_pgd_size(kvm), GFP_KERNEL | __GFP_ZERO);
if (!pgd)
return -ENOMEM;
@@ -844,7 +843,7 @@ static void kvm_flush_stage2_all(struct kvm *kvm)
{
spin_lock(&kvm->mmu_lock);
if (kvm->arch.pgd)
- unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
+ unmap_stage2_range(kvm, 0, kvm_phys_size(kvm));
spin_unlock(&kvm->mmu_lock);
}
@@ -861,7 +860,7 @@ static void kvm_flush_stage2_all(struct kvm *kvm)
void kvm_free_stage2_pgd(struct kvm *kvm)
{
kvm_flush_stage2_all(kvm);
- free_pages_exact(kvm->arch.pgd, S2_PGD_SIZE);
+ free_pages_exact(kvm->arch.pgd, stage2_pgd_size(kvm));
kvm->arch.pgd = NULL;
}
@@ -871,16 +870,16 @@ static pud_t *stage2_get_pud(struct kvm *kvm, struct kvm_mmu_memory_cache *cache
pgd_t *pgd;
pud_t *pud;
- pgd = kvm->arch.pgd + stage2_pgd_index(addr);
- if (stage2_pgd_none(*pgd)) {
+ pgd = kvm->arch.pgd + stage2_pgd_index(kvm, addr);
+ if (stage2_pgd_none(kvm, *pgd)) {
if (!cache)
return NULL;
pud = mmu_memory_cache_alloc(cache);
- stage2_pgd_populate(pgd, pud);
+ stage2_pgd_populate(kvm, pgd, pud);
get_page(virt_to_page(pgd));
}
- return stage2_pud_offset(pgd, addr);
+ return stage2_pud_offset(kvm, pgd, addr);
}
static pmd_t *stage2_get_pmd(struct kvm *kvm, struct kvm_mmu_memory_cache *cache,
@@ -893,15 +892,15 @@ static pmd_t *stage2_get_pmd(struct kvm *kvm, struct kvm_mmu_memory_cache *cache
if (!pud)
return NULL;
- if (stage2_pud_none(*pud)) {
+ if (stage2_pud_none(kvm, *pud)) {
if (!cache)
return NULL;
pmd = mmu_memory_cache_alloc(cache);
- stage2_pud_populate(pud, pmd);
+ stage2_pud_populate(kvm, pud, pmd);
get_page(virt_to_page(pud));
}
- return stage2_pmd_offset(pud, addr);
+ return stage2_pmd_offset(kvm, pud, addr);
}
static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache
@@ -1060,7 +1059,7 @@ static int __kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
if (writable)
pte = kvm_s2pte_mkwrite(pte);
- ret = mmu_topup_memory_cache(&cache, KVM_MMU_CACHE_MIN_PAGES,
+ ret = mmu_topup_memory_cache(&cache, kvm_mmu_cache_min_pages(kvm),
KVM_NR_MEM_OBJS);
if (ret)
goto out;
@@ -1166,19 +1165,20 @@ static void stage2_wp_ptes(pmd_t *pmd, phys_addr_t addr, phys_addr_t end)
/**
* stage2_wp_pmds - write protect PUD range
+ * kvm: kvm instance for the VM
* @pud: pointer to pud entry
* @addr: range start address
* @end: range end address
*/
-static void stage2_wp_pmds(pud_t *pud, phys_addr_t addr, phys_addr_t end)
+static void stage2_wp_pmds(struct kvm *kvm, pud_t *pud, phys_addr_t addr, phys_addr_t end)
{
pmd_t *pmd;
phys_addr_t next;
- pmd = stage2_pmd_offset(pud, addr);
+ pmd = stage2_pmd_offset(kvm, pud, addr);
do {
- next = stage2_pmd_addr_end(addr, end);
+ next = stage2_pmd_addr_end(kvm, addr, end);
if (!pmd_none(*pmd)) {
if (pmd_thp_or_huge(*pmd)) {
if (!kvm_s2pmd_readonly(pmd))
@@ -1198,18 +1198,18 @@ static void stage2_wp_pmds(pud_t *pud, phys_addr_t addr, phys_addr_t end)
*
* Process PUD entries, for a huge PUD we cause a panic.
*/
-static void stage2_wp_puds(pgd_t *pgd, phys_addr_t addr, phys_addr_t end)
+static void stage2_wp_puds(struct kvm *kvm, pgd_t *pgd, phys_addr_t addr, phys_addr_t end)
{
pud_t *pud;
phys_addr_t next;
- pud = stage2_pud_offset(pgd, addr);
+ pud = stage2_pud_offset(kvm, pgd, addr);
do {
- next = stage2_pud_addr_end(addr, end);
- if (!stage2_pud_none(*pud)) {
+ next = stage2_pud_addr_end(kvm, addr, end);
+ if (!stage2_pud_none(kvm, *pud)) {
/* TODO:PUD not supported, revisit later if supported */
- BUG_ON(stage2_pud_huge(*pud));
- stage2_wp_pmds(pud, addr, next);
+ BUG_ON(stage2_pud_huge(kvm, *pud));
+ stage2_wp_pmds(kvm, pud, addr, next);
}
} while (pud++, addr = next, addr != end);
}
@@ -1225,7 +1225,7 @@ static void stage2_wp_range(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
pgd_t *pgd;
phys_addr_t next;
- pgd = kvm->arch.pgd + stage2_pgd_index(addr);
+ pgd = kvm->arch.pgd + stage2_pgd_index(kvm, addr);
do {
/*
* Release kvm_mmu_lock periodically if the memory region is
@@ -1239,9 +1239,9 @@ static void stage2_wp_range(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
cond_resched_lock(&kvm->mmu_lock);
if (WARN_ON(!READ_ONCE(kvm->arch.pgd)))
break;
- next = stage2_pgd_addr_end(addr, end);
- if (stage2_pgd_present(*pgd))
- stage2_wp_puds(pgd, addr, next);
+ next = stage2_pgd_addr_end(kvm, addr, end);
+ if (stage2_pgd_present(kvm, *pgd))
+ stage2_wp_puds(kvm, pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
@@ -1382,7 +1382,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
up_read(¤t->mm->mmap_sem);
/* We need minimum second+third level pages */
- ret = mmu_topup_memory_cache(memcache, KVM_MMU_CACHE_MIN_PAGES,
+ ret = mmu_topup_memory_cache(memcache, kvm_mmu_cache_min_pages(kvm),
KVM_NR_MEM_OBJS);
if (ret)
return ret;
@@ -1601,7 +1601,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run)
}
/* Userspace should not be able to register out-of-bounds IPAs */
- VM_BUG_ON(fault_ipa >= KVM_PHYS_SIZE);
+ VM_BUG_ON(fault_ipa >= kvm_phys_size(vcpu->kvm));
if (fault_status == FSC_ACCESS) {
handle_access_fault(vcpu, fault_ipa);
@@ -1901,7 +1901,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
* space addressable by the KVM guest IPA space.
*/
if (memslot->base_gfn + memslot->npages >=
- (KVM_PHYS_SIZE >> PAGE_SHIFT))
+ (kvm_phys_size(kvm) >> PAGE_SHIFT))
return -EFAULT;
down_read(¤t->mm->mmap_sem);
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 10ae6f394b71..613ff4abcad5 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -25,7 +25,7 @@
int vgic_check_ioaddr(struct kvm *kvm, phys_addr_t *ioaddr,
phys_addr_t addr, phys_addr_t alignment)
{
- if (addr & ~KVM_PHYS_MASK)
+ if (addr & ~kvm_phys_mask(kvm))
return -E2BIG;
if (!IS_ALIGNED(addr, alignment))
--
2.13.6
^ permalink raw reply related
* [PATCH v1 09/16] kvm: arm/arm64: Delay stage2 page table allocation
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
We allocate the entry level page tables for stage2 when the
VM is created. This doesn't give us the flexibility of configuring
the physical address space size for a VM. In order to allow
the VM to choose the required size, we delay the allocation of
stage2 entry level tables until we really try to map something.
This could be either when the VM creates a memory range or when
it tries to map a device memory. So we add in a hook to these
two places to make sure the tables are allocated. We use
kvm->slots_lock to serialize the allocation entry point, since
we add hooks to the arch specific call back with the mutex held.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
virt/kvm/arm/arm.c | 18 ++++++----------
virt/kvm/arm/mmu.c | 61 +++++++++++++++++++++++++++++++++++++++++++++---------
2 files changed, 57 insertions(+), 22 deletions(-)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 19b720ddedce..d06f00566664 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -127,13 +127,13 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
for_each_possible_cpu(cpu)
*per_cpu_ptr(kvm->arch.last_vcpu_ran, cpu) = -1;
- ret = kvm_alloc_stage2_pgd(kvm);
- if (ret)
- goto out_fail_alloc;
-
ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP);
- if (ret)
- goto out_free_stage2_pgd;
+ if (ret) {
+ free_percpu(kvm->arch.last_vcpu_ran);
+ kvm->arch.last_vcpu_ran = NULL;
+ return ret;
+ }
+
kvm_vgic_early_init(kvm);
@@ -145,12 +145,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
return ret;
-out_free_stage2_pgd:
- kvm_free_stage2_pgd(kvm);
-out_fail_alloc:
- free_percpu(kvm->arch.last_vcpu_ran);
- kvm->arch.last_vcpu_ran = NULL;
- return ret;
}
bool kvm_arch_has_vcpu_debugfs(void)
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index c94c61ac38b9..257f2a8ccfc7 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -1011,15 +1011,39 @@ static int stage2_pmdp_test_and_clear_young(pmd_t *pmd)
return stage2_ptep_test_and_clear_young((pte_t *)pmd);
}
-/**
- * kvm_phys_addr_ioremap - map a device range to guest IPA
- *
- * @kvm: The KVM pointer
- * @guest_ipa: The IPA at which to insert the mapping
- * @pa: The physical address of the device
- * @size: The size of the mapping
+/*
+ * Finalise the stage2 page table layout. Must be called with kvm->slots_lock
+ * held.
*/
-int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
+static int __kvm_init_stage2_table(struct kvm *kvm)
+{
+ /* Double check if somebody has already allocated it */
+ if (likely(kvm->arch.pgd))
+ return 0;
+ return kvm_alloc_stage2_pgd(kvm);
+}
+
+static int kvm_init_stage2_table(struct kvm *kvm)
+{
+ int rc;
+
+ /*
+ * Once allocated, the stage2 entry level tables are only
+ * freed when the KVM instance is destroyed. So, if we see
+ * something valid here, that guarantees that we have
+ * done the one time allocation and it is something valid
+ * and won't go away until the last reference to the KVM
+ * is gone.
+ */
+ if (likely(kvm->arch.pgd))
+ return 0;
+ mutex_lock(&kvm->slots_lock);
+ rc = __kvm_init_stage2_table(kvm);
+ mutex_unlock(&kvm->slots_lock);
+ return rc;
+}
+
+static int __kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
phys_addr_t pa, unsigned long size, bool writable)
{
phys_addr_t addr, end;
@@ -1055,6 +1079,23 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
return ret;
}
+/**
+ * kvm_phys_addr_ioremap - map a device range to guest IPA.
+ * Acquires kvm->slots_lock for making sure that the stage2 is initialized.
+ *
+ * @kvm: The KVM pointer
+ * @guest_ipa: The IPA at which to insert the mapping
+ * @pa: The physical address of the device
+ * @size: The size of the mapping
+ */
+int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
+ phys_addr_t pa, unsigned long size, bool writable)
+{
+ if (unlikely(kvm_init_stage2_table(kvm)))
+ return -ENOMEM;
+ return __kvm_phys_addr_ioremap(kvm, guest_ipa, pa, size, writable);
+}
+
static bool transparent_hugepage_adjust(kvm_pfn_t *pfnp, phys_addr_t *ipap)
{
kvm_pfn_t pfn = *pfnp;
@@ -1912,7 +1953,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
goto out;
}
- ret = kvm_phys_addr_ioremap(kvm, gpa, pa,
+ ret = __kvm_phys_addr_ioremap(kvm, gpa, pa,
vm_end - vm_start,
writable);
if (ret)
@@ -1943,7 +1984,7 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
unsigned long npages)
{
- return 0;
+ return __kvm_init_stage2_table(kvm);
}
void kvm_arch_memslots_updated(struct kvm *kvm, struct kvm_memslots *slots)
--
2.13.6
^ permalink raw reply related
* [PATCH v1 08/16] kvm: arm/arm64: Clean up stage2 pgd life time
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
On arm/arm64 we pre-allocate the entry level page tables when
a VM is created and is free'd when either all the mm users are
gone or the KVM is about to get destroyed. i.e, kvm_free_stage2_pgd
is triggered via kvm_arch_flush_shadow_all() which can be invoked
from two different paths :
1) do_exit()-> .-> mmu_notifier->release()-> ..-> kvm_arch_flush_shadow_all()
OR
2) kvm_destroy_vm()-> mmu_notifier_unregister-> kvm_arch_flush_shadow_all()
This has created lot of race conditions in the past as some of
the VCPUs could be active when we free the stage2 via path (1).
On a closer look, all we need to do with kvm_arch_flush_shadow_all() is,
to ensure that the stage2 mappings are cleared. This doesn't mean we
have to free up the stage2 entry level page tables yet, which could
be delayed until the kvm is destroyed. This would avoid issues
of use-after-free, as we don't free the page tables and anyone who
tries to access the page table would find them in the appropriate
state (mapped vs unmapped), as the page table modifications are
serialised via kvm->mmu_lock. This will be later used for delaying
the allocation of the stage2 entry level page tables until we really
need to do something with it.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
virt/kvm/arm/arm.c | 1 +
virt/kvm/arm/mmu.c | 56 ++++++++++++++++++++++++++++--------------------------
2 files changed, 30 insertions(+), 27 deletions(-)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index c8d49879307f..19b720ddedce 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -189,6 +189,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
}
}
atomic_set(&kvm->online_vcpus, 0);
+ kvm_free_stage2_pgd(kvm);
}
int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index 78253fe00fc4..c94c61ac38b9 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -298,11 +298,10 @@ static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
pgd = kvm->arch.pgd + stage2_pgd_index(addr);
do {
/*
- * Make sure the page table is still active, as another thread
- * could have possibly freed the page table, while we released
- * the lock.
+ * The page table shouldn't be free'd as we still hold a reference
+ * to the KVM.
*/
- if (!READ_ONCE(kvm->arch.pgd))
+ if (WARN_ON(!READ_ONCE(kvm->arch.pgd)))
break;
next = stage2_pgd_addr_end(addr, end);
if (!stage2_pgd_none(*pgd))
@@ -837,30 +836,33 @@ void stage2_unmap_vm(struct kvm *kvm)
up_read(¤t->mm->mmap_sem);
srcu_read_unlock(&kvm->srcu, idx);
}
-
-/**
- * kvm_free_stage2_pgd - free all stage-2 tables
- * @kvm: The KVM struct pointer for the VM.
- *
- * Walks the level-1 page table pointed to by kvm->arch.pgd and frees all
- * underlying level-2 and level-3 tables before freeing the actual level-1 table
- * and setting the struct pointer to NULL.
+/*
+ * kvm_flush_stage2_all: Unmap the entire stage2 mappings including
+ * device and regular RAM backing memory.
*/
-void kvm_free_stage2_pgd(struct kvm *kvm)
+static void kvm_flush_stage2_all(struct kvm *kvm)
{
- void *pgd = NULL;
-
spin_lock(&kvm->mmu_lock);
- if (kvm->arch.pgd) {
+ if (kvm->arch.pgd)
unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
- pgd = READ_ONCE(kvm->arch.pgd);
- kvm->arch.pgd = NULL;
- }
spin_unlock(&kvm->mmu_lock);
+}
- /* Free the HW pgd, one page at a time */
- if (pgd)
- free_pages_exact(pgd, S2_PGD_SIZE);
+/**
+ * kvm_free_stage2_pgd - Free the entry level page tables in stage-2.
+ * This is called when all reference to the KVM has gone away and we
+ * really don't need any protection in resetting the PGD. This also
+ * means that nobody should be touching stage2 at this point, as we
+ * have unmapped the entire stage2 already and all dynamic entities,
+ * (VCPUs and devices) are no longer active.
+ *
+ * @kvm: The KVM struct pointer for the VM.
+ */
+void kvm_free_stage2_pgd(struct kvm *kvm)
+{
+ kvm_flush_stage2_all(kvm);
+ free_pages_exact(kvm->arch.pgd, S2_PGD_SIZE);
+ kvm->arch.pgd = NULL;
}
static pud_t *stage2_get_pud(struct kvm *kvm, struct kvm_mmu_memory_cache *cache,
@@ -1189,12 +1191,12 @@ static void stage2_wp_range(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
* large. Otherwise, we may see kernel panics with
* CONFIG_DETECT_HUNG_TASK, CONFIG_LOCKUP_DETECTOR,
* CONFIG_LOCKDEP. Additionally, holding the lock too long
- * will also starve other vCPUs. We have to also make sure
- * that the page tables are not freed while we released
- * the lock.
+ * will also starve other vCPUs.
+ * The page tables shouldn't be free'd while we released the
+ * lock, since we hold a reference to the KVM.
*/
cond_resched_lock(&kvm->mmu_lock);
- if (!READ_ONCE(kvm->arch.pgd))
+ if (WARN_ON(!READ_ONCE(kvm->arch.pgd)))
break;
next = stage2_pgd_addr_end(addr, end);
if (stage2_pgd_present(*pgd))
@@ -1950,7 +1952,7 @@ void kvm_arch_memslots_updated(struct kvm *kvm, struct kvm_memslots *slots)
void kvm_arch_flush_shadow_all(struct kvm *kvm)
{
- kvm_free_stage2_pgd(kvm);
+ kvm_flush_stage2_all(kvm);
}
void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
--
2.13.6
^ permalink raw reply related
* [PATCH v1 07/16] kvm: arm/arm64: Remove spurious WARN_ON
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
On a 4-level page table pgd entry can be empty, unlike a 3-level
page table. Remove the spurious WARN_ON() in stage_get_pud().
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
virt/kvm/arm/mmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index e6548c85c495..78253fe00fc4 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -870,7 +870,7 @@ static pud_t *stage2_get_pud(struct kvm *kvm, struct kvm_mmu_memory_cache *cache
pud_t *pud;
pgd = kvm->arch.pgd + stage2_pgd_index(addr);
- if (WARN_ON(stage2_pgd_none(*pgd))) {
+ if (stage2_pgd_none(*pgd)) {
if (!cache)
return NULL;
pud = mmu_memory_cache_alloc(cache);
--
2.13.6
^ permalink raw reply related
* [PATCH v1 06/16] kvm: arm/arm64: Fix stage2_flush_memslot for 4 level page table
From: Suzuki K Poulose @ 2018-01-09 19:04 UTC (permalink / raw)
To: linux-arm-kernel
In-Reply-To: <20180109190414.4017-1-suzuki.poulose@arm.com>
So far we have only supported 3 level page table with fixed IPA of 40bits.
Fix stage2_flush_memslot() to accommodate for 4 level tables.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
virt/kvm/arm/mmu.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index 761787befd3b..e6548c85c495 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -375,7 +375,8 @@ static void stage2_flush_memslot(struct kvm *kvm,
pgd = kvm->arch.pgd + stage2_pgd_index(addr);
do {
next = stage2_pgd_addr_end(addr, end);
- stage2_flush_puds(kvm, pgd, addr, next);
+ if (!stage2_pgd_none(*pgd))
+ stage2_flush_puds(kvm, pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
--
2.13.6
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox