* [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA
@ 2019-12-27 11:47 Russell King
2019-12-27 11:54 ` Russell King - ARM Linux admin
2019-12-28 11:33 ` Marc Zyngier
0 siblings, 2 replies; 3+ messages in thread
From: Russell King @ 2019-12-27 11:47 UTC (permalink / raw)
To: Marc Zyngier
Cc: Suzuki K Poulose, Catalin Marinas, James Morse, Julien Thierry,
Will Deacon, kvmarm, linux-arm-kernel
Booting 5.4 on LX2160A reveals that KVM is non-functional:
kvm: Limiting the IPA size due to kernel Virtual Address limit
kvm [1]: IPA Size Limit: 43bits
kvm [1]: IDMAP intersecting with HYP VA, unable to continue
kvm [1]: error initializing Hyp mode: -22
Debugging shows:
kvm [1]: IDMAP page: 81a26000
kvm [1]: HYP VA range: 0:22ffffffff
as RAM is located at:
80000000-fbdfffff : System RAM
2080000000-237fffffff : System RAM
Comparing this with the same kernel on Armada 8040 shows:
kvm: Limiting the IPA size due to kernel Virtual Address limit
kvm [1]: IPA Size Limit: 43bits
kvm [1]: IDMAP page: 2a26000
kvm [1]: HYP VA range: 4800000000:493fffffff
...
kvm [1]: Hyp mode initialized successfully
which indicates that hyp_va_msb is set, and is always set to the
opposite value of the idmap page to avoid the overlap. This does not
happen with the LX2160A.
Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
there is still one bit available for hyp_va_msb. Set this bit
appropriately. This allows kvm to be functional on the LX2160A, but
without any HYP VA randomisation:
kvm: Limiting the IPA size due to kernel Virtual Address limit
kvm [1]: IPA Size Limit: 43bits
kvm [1]: IDMAP page: 81a24000
kvm [1]: HYP VA range: 4000000000:62ffffffff
...
kvm [1]: Hyp mode initialized successfully
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
arch/arm64/kvm/va_layout.c | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c
index 2cf7d4b606c3..83f8b3f51cf4 100644
--- a/arch/arm64/kvm/va_layout.c
+++ b/arch/arm64/kvm/va_layout.c
@@ -22,6 +22,17 @@ static u8 tag_lsb;
static u64 tag_val;
static u64 va_mask;
+/*
+ * We want to generate a hyp VA with the following format:
+ *
+ * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
+ * ---------------------------------------------------------
+ * | 0000000 | hyp_va_msb | random tag | kern linear VA |
+ *
+ * which does not conflict with the idmap regions. This means that hyp_va_msb
+ * must always be present. Luckily, when kva_msb == (vabits_actual - 1) we
+ * still have one bit for this, but no bits for the random tag.
+ */
static void compute_layout(void)
{
phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start);
@@ -39,19 +50,16 @@ static void compute_layout(void)
/*
* No space in the address, let's compute the mask so
* that it covers (vabits_actual - 1) bits, and the region
- * bit. The tag stays set to zero.
+ * bit.
*/
- va_mask = BIT(vabits_actual - 1) - 1;
- va_mask |= hyp_va_msb;
+ tag_lsb = kva_msb;
+ va_mask = BIT(vabits_actual - 1) - 1;
+ tag_val = hyp_va_msb >> tag_lsb;
} else {
/*
* We do have some free bits to insert a random tag.
* Hyp VAs are now created from kernel linear map VAs
* using the following formula (with V == vabits_actual):
- *
- * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
- * ---------------------------------------------------------
- * | 0000000 | hyp_va_msb | random tag | kern linear VA |
*/
tag_lsb = kva_msb;
va_mask = GENMASK_ULL(tag_lsb - 1, 0);
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA
2019-12-27 11:47 [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA Russell King
@ 2019-12-27 11:54 ` Russell King - ARM Linux admin
2019-12-28 11:33 ` Marc Zyngier
1 sibling, 0 replies; 3+ messages in thread
From: Russell King - ARM Linux admin @ 2019-12-27 11:54 UTC (permalink / raw)
To: Marc Zyngier
Cc: Suzuki K Poulose, Catalin Marinas, James Morse, Julien Thierry,
Will Deacon, kvmarm, linux-arm-kernel
On Fri, Dec 27, 2019 at 11:47:35AM +0000, Russell King wrote:
> Booting 5.4 on LX2160A reveals that KVM is non-functional:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> kvm [1]: error initializing Hyp mode: -22
>
> Debugging shows:
>
> kvm [1]: IDMAP page: 81a26000
> kvm [1]: HYP VA range: 0:22ffffffff
>
> as RAM is located at:
>
> 80000000-fbdfffff : System RAM
> 2080000000-237fffffff : System RAM
>
> Comparing this with the same kernel on Armada 8040 shows:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 2a26000
> kvm [1]: HYP VA range: 4800000000:493fffffff
> ...
> kvm [1]: Hyp mode initialized successfully
>
> which indicates that hyp_va_msb is set, and is always set to the
> opposite value of the idmap page to avoid the overlap. This does not
> happen with the LX2160A.
>
> Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> there is still one bit available for hyp_va_msb. Set this bit
> appropriately. This allows kvm to be functional on the LX2160A, but
> without any HYP VA randomisation:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 81a24000
> kvm [1]: HYP VA range: 4000000000:62ffffffff
> ...
> kvm [1]: Hyp mode initialized successfully
>
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> ---
I assume it's not possible for kva_msb >= vabits_actual ?
> arch/arm64/kvm/va_layout.c | 22 +++++++++++++++-------
> 1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c
> index 2cf7d4b606c3..83f8b3f51cf4 100644
> --- a/arch/arm64/kvm/va_layout.c
> +++ b/arch/arm64/kvm/va_layout.c
> @@ -22,6 +22,17 @@ static u8 tag_lsb;
> static u64 tag_val;
> static u64 va_mask;
>
> +/*
> + * We want to generate a hyp VA with the following format:
> + *
> + * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> + * ---------------------------------------------------------
> + * | 0000000 | hyp_va_msb | random tag | kern linear VA |
> + *
> + * which does not conflict with the idmap regions. This means that hyp_va_msb
> + * must always be present. Luckily, when kva_msb == (vabits_actual - 1) we
> + * still have one bit for this, but no bits for the random tag.
> + */
> static void compute_layout(void)
> {
> phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start);
> @@ -39,19 +50,16 @@ static void compute_layout(void)
> /*
> * No space in the address, let's compute the mask so
> * that it covers (vabits_actual - 1) bits, and the region
> - * bit. The tag stays set to zero.
> + * bit.
> */
> - va_mask = BIT(vabits_actual - 1) - 1;
> - va_mask |= hyp_va_msb;
> + tag_lsb = kva_msb;
> + va_mask = BIT(vabits_actual - 1) - 1;
> + tag_val = hyp_va_msb >> tag_lsb;
> } else {
> /*
> * We do have some free bits to insert a random tag.
> * Hyp VAs are now created from kernel linear map VAs
> * using the following formula (with V == vabits_actual):
> - *
> - * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> - * ---------------------------------------------------------
> - * | 0000000 | hyp_va_msb | random tag | kern linear VA |
> */
> tag_lsb = kva_msb;
> va_mask = GENMASK_ULL(tag_lsb - 1, 0);
> --
> 2.20.1
>
>
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA
2019-12-27 11:47 [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA Russell King
2019-12-27 11:54 ` Russell King - ARM Linux admin
@ 2019-12-28 11:33 ` Marc Zyngier
1 sibling, 0 replies; 3+ messages in thread
From: Marc Zyngier @ 2019-12-28 11:33 UTC (permalink / raw)
To: Russell King
Cc: Suzuki K Poulose, Catalin Marinas, James Morse, Julien Thierry,
Will Deacon, kvmarm, linux-arm-kernel
Hi Russell,
Thanks for this.
On Fri, 27 Dec 2019 11:47:35 +0000,
Russell King <rmk+kernel@armlinux.org.uk> wrote:
>
> Booting 5.4 on LX2160A reveals that KVM is non-functional:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> kvm [1]: error initializing Hyp mode: -22
>
> Debugging shows:
>
> kvm [1]: IDMAP page: 81a26000
> kvm [1]: HYP VA range: 0:22ffffffff
>
> as RAM is located at:
>
> 80000000-fbdfffff : System RAM
> 2080000000-237fffffff : System RAM
Ouch. This looks like a terrible choice for a memory map.
>
> Comparing this with the same kernel on Armada 8040 shows:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 2a26000
> kvm [1]: HYP VA range: 4800000000:493fffffff
> ...
> kvm [1]: Hyp mode initialized successfully
>
> which indicates that hyp_va_msb is set, and is always set to the
> opposite value of the idmap page to avoid the overlap. This does not
> happen with the LX2160A.
>
> Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> there is still one bit available for hyp_va_msb. Set this bit
> appropriately. This allows kvm to be functional on the LX2160A, but
> without any HYP VA randomisation:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 81a24000
> kvm [1]: HYP VA range: 4000000000:62ffffffff
> ...
> kvm [1]: Hyp mode initialized successfully
Nice bit of debugging. I guess part of the confusion is due to the
fact that the hyp_va_msb is part of the tag (as you found out), but
that the documentation doesn't really make that clear at all (and only
mentions the random part of the tag).
>
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> ---
> arch/arm64/kvm/va_layout.c | 22 +++++++++++++++-------
> 1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c
> index 2cf7d4b606c3..83f8b3f51cf4 100644
> --- a/arch/arm64/kvm/va_layout.c
> +++ b/arch/arm64/kvm/va_layout.c
> @@ -22,6 +22,17 @@ static u8 tag_lsb;
> static u64 tag_val;
> static u64 va_mask;
>
> +/*
> + * We want to generate a hyp VA with the following format:
> + *
> + * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> + * ---------------------------------------------------------
> + * | 0000000 | hyp_va_msb | random tag | kern linear VA |
\---------- tag -------------/
How about the above, to make it clearer that the tag must include the
hyp_va_msb bit to avoid clashing with the IDMAP?
> + *
> + * which does not conflict with the idmap regions. This means that hyp_va_msb
> + * must always be present. Luckily, when kva_msb == (vabits_actual - 1) we
I'm not sure the "Luckily" part is appropriate here. Given the way
kva_msb is computed, I can't see how it can see how this can
fail.
This stems from the fact that the vabits space for kernel mappings is
split in two: vabits-1 for the linear map, and vabits-1 for the rest
(kernel text and co). Given that we define kva_msb as the highest
order bit that can change within the linear map, its value is at most
vabits-1.
> + * still have one bit for this, but no bits for the random tag.
> + */
> static void compute_layout(void)
> {
> phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start);
> @@ -39,19 +50,16 @@ static void compute_layout(void)
> /*
> * No space in the address, let's compute the mask so
> * that it covers (vabits_actual - 1) bits, and the region
> - * bit. The tag stays set to zero.
> + * bit.
> */
> - va_mask = BIT(vabits_actual - 1) - 1;
> - va_mask |= hyp_va_msb;
> + tag_lsb = kva_msb;
> + va_mask = BIT(vabits_actual - 1) - 1;
> + tag_val = hyp_va_msb >> tag_lsb;
> } else {
> /*
> * We do have some free bits to insert a random tag.
> * Hyp VAs are now created from kernel linear map VAs
> * using the following formula (with V == vabits_actual):
> - *
> - * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> - * ---------------------------------------------------------
> - * | 0000000 | hyp_va_msb | random tag | kern linear VA |
> */
> tag_lsb = kva_msb;
> va_mask = GENMASK_ULL(tag_lsb - 1, 0);
In the light of this, it'd be great to rework this code to simplify it
(getting rid of kva_msb, which really is tag_lsb) and make some of the
computing common to both branches.
Thanks,
M.
--
Jazz is not dead, it just smells funny.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-12-28 11:43 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-12-27 11:47 [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA Russell King
2019-12-27 11:54 ` Russell King - ARM Linux admin
2019-12-28 11:33 ` Marc Zyngier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).