[PATCH v2] arm64: kvm: fix IDMAP overlap with HYP VA

[PATCH v2] arm64: kvm: fix IDMAP overlap with HYP VA

[Russell King]

Booting 5.4 on LX2160A reveals that KVM is non-functional:

kvm: Limiting the IPA size due to kernel Virtual Address limit
kvm [1]: IPA Size Limit: 43bits
kvm [1]: IDMAP intersecting with HYP VA, unable to continue
kvm [1]: error initializing Hyp mode: -22

Debugging shows:

kvm [1]: IDMAP page: 81a26000
kvm [1]: HYP VA range: 0:22ffffffff

as RAM is located at:

80000000-fbdfffff : System RAM
2080000000-237fffffff : System RAM

Comparing this with the same kernel on Armada 8040 shows:

kvm: Limiting the IPA size due to kernel Virtual Address limit
kvm [1]: IPA Size Limit: 43bits
kvm [1]: IDMAP page: 2a26000
kvm [1]: HYP VA range: 4800000000:493fffffff
...
kvm [1]: Hyp mode initialized successfully

which indicates that hyp_va_msb is set, and is always set to the
opposite value of the idmap page to avoid the overlap. This does not
happen with the LX2160A.

Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
there is still one bit available for hyp_va_msb. Set this bit
appropriately. This allows kvm to be functional on the LX2160A, but
without any HYP VA randomisation:

kvm: Limiting the IPA size due to kernel Virtual Address limit
kvm [1]: IPA Size Limit: 43bits
kvm [1]: IDMAP page: 81a24000
kvm [1]: HYP VA range: 4000000000:62ffffffff
...
kvm [1]: Hyp mode initialized successfully

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
 arch/arm64/kvm/va_layout.c | 44 ++++++++++++++++----------------------
 1 file changed, 19 insertions(+), 25 deletions(-)

diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c
index 2cf7d4b606c3..1598909afbaf 100644
--- a/arch/arm64/kvm/va_layout.c
+++ b/arch/arm64/kvm/va_layout.c
@@ -22,43 +22,37 @@ static u8 tag_lsb;
 static u64 tag_val;
 static u64 va_mask;
 
+/*
+ * We want to generate a hyp VA with the following format (with V ==
+ * vabits_actual):
+ *
+ *  63 ... V |     V-1    | V-2 .. tag_lsb | tag_lsb - 1 .. 0
+ *  ---------------------------------------------------------
+ * | 0000000 | hyp_va_msb |   random tag   |  kern linear VA |
+ *           |--------- tag_val -----------|----- va_mask ---|
+ *
+ * which does not conflict with the idmap regions.
+ */
 static void compute_layout(void)
 {
 	phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start);
 	u64 hyp_va_msb;
-	int kva_msb;
 
 	/* Where is my RAM region? */
 	hyp_va_msb  = idmap_addr & BIT(vabits_actual - 1);
 	hyp_va_msb ^= BIT(vabits_actual - 1);
 
-	kva_msb = fls64((u64)phys_to_virt(memblock_start_of_DRAM()) ^
+	tag_lsb = fls64((u64)phys_to_virt(memblock_start_of_DRAM()) ^
 			(u64)(high_memory - 1));
 
-	if (kva_msb == (vabits_actual - 1)) {
-		/*
-		 * No space in the address, let's compute the mask so
-		 * that it covers (vabits_actual - 1) bits, and the region
-		 * bit. The tag stays set to zero.
-		 */
-		va_mask  = BIT(vabits_actual - 1) - 1;
-		va_mask |= hyp_va_msb;
-	} else {
-		/*
-		 * We do have some free bits to insert a random tag.
-		 * Hyp VAs are now created from kernel linear map VAs
-		 * using the following formula (with V == vabits_actual):
-		 *
-		 *  63 ... V |     V-1    | V-2 .. tag_lsb | tag_lsb - 1 .. 0
-		 *  ---------------------------------------------------------
-		 * | 0000000 | hyp_va_msb |    random tag  |  kern linear VA |
-		 */
-		tag_lsb = kva_msb;
-		va_mask = GENMASK_ULL(tag_lsb - 1, 0);
-		tag_val = get_random_long() & GENMASK_ULL(vabits_actual - 2, tag_lsb);
-		tag_val |= hyp_va_msb;
-		tag_val >>= tag_lsb;
+	va_mask = GENMASK_ULL(tag_lsb - 1, 0);
+	tag_val = hyp_va_msb;
+
+	if (tag_lsb != (vabits_actual - 1)) {
+		/* We have some free bits to insert a random tag. */
+		tag_val |= get_random_long() & GENMASK_ULL(vabits_actual - 2, tag_lsb);
 	}
+	tag_val >>= tag_lsb;
 }
 
 static u32 compute_instruction(int n, u32 rd, u32 rn)
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] arm64: kvm: fix IDMAP overlap with HYP VA

[Marc Zyngier]

On Sat, 28 Dec 2019 11:57:14 +0000
Russell King <rmk+kernel@armlinux.org.uk> wrote:

> Booting 5.4 on LX2160A reveals that KVM is non-functional:
> 
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> kvm [1]: error initializing Hyp mode: -22
> 
> Debugging shows:
> 
> kvm [1]: IDMAP page: 81a26000
> kvm [1]: HYP VA range: 0:22ffffffff
> 
> as RAM is located at:
> 
> 80000000-fbdfffff : System RAM
> 2080000000-237fffffff : System RAM
> 
> Comparing this with the same kernel on Armada 8040 shows:
> 
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 2a26000
> kvm [1]: HYP VA range: 4800000000:493fffffff
> ...
> kvm [1]: Hyp mode initialized successfully
> 
> which indicates that hyp_va_msb is set, and is always set to the
> opposite value of the idmap page to avoid the overlap. This does not
> happen with the LX2160A.
> 
> Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> there is still one bit available for hyp_va_msb. Set this bit
> appropriately. This allows kvm to be functional on the LX2160A, but
> without any HYP VA randomisation:
> 
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 81a24000
> kvm [1]: HYP VA range: 4000000000:62ffffffff
> ...
> kvm [1]: Hyp mode initialized successfully
> 
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

I've applied this to kvmarm-next with a couple of cleanups, and
preserving the fallback when the tag is zero (only the mask gets
applied, without any ROR or ADD).

	M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] arm64: kvm: fix IDMAP overlap with HYP VA

[Russell King - ARM Linux admin]

On Sun, Jan 19, 2020 at 05:43:27PM +0000, Marc Zyngier wrote:
> On Sat, 28 Dec 2019 11:57:14 +0000
> Russell King <rmk+kernel@armlinux.org.uk> wrote:
> 
> > Booting 5.4 on LX2160A reveals that KVM is non-functional:
> > 
> > kvm: Limiting the IPA size due to kernel Virtual Address limit
> > kvm [1]: IPA Size Limit: 43bits
> > kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> > kvm [1]: error initializing Hyp mode: -22
> > 
> > Debugging shows:
> > 
> > kvm [1]: IDMAP page: 81a26000
> > kvm [1]: HYP VA range: 0:22ffffffff
> > 
> > as RAM is located at:
> > 
> > 80000000-fbdfffff : System RAM
> > 2080000000-237fffffff : System RAM
> > 
> > Comparing this with the same kernel on Armada 8040 shows:
> > 
> > kvm: Limiting the IPA size due to kernel Virtual Address limit
> > kvm [1]: IPA Size Limit: 43bits
> > kvm [1]: IDMAP page: 2a26000
> > kvm [1]: HYP VA range: 4800000000:493fffffff
> > ...
> > kvm [1]: Hyp mode initialized successfully
> > 
> > which indicates that hyp_va_msb is set, and is always set to the
> > opposite value of the idmap page to avoid the overlap. This does not
> > happen with the LX2160A.
> > 
> > Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> > kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> > there is still one bit available for hyp_va_msb. Set this bit
> > appropriately. This allows kvm to be functional on the LX2160A, but
> > without any HYP VA randomisation:
> > 
> > kvm: Limiting the IPA size due to kernel Virtual Address limit
> > kvm [1]: IPA Size Limit: 43bits
> > kvm [1]: IDMAP page: 81a24000
> > kvm [1]: HYP VA range: 4000000000:62ffffffff
> > ...
> > kvm [1]: Hyp mode initialized successfully
> > 
> > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> 
> I've applied this to kvmarm-next with a couple of cleanups, and
> preserving the fallback when the tag is zero (only the mask gets
> applied, without any ROR or ADD).

If only the mask is applied, then it will overlap with the IDMAP
region, and KVM will fail - so I think it would be a good idea in
that case to print something a little more useful, rather than
attributing the KVM failure to an overlap of IDMAP and the KVM
range.

The real problem is there aren't enough VA bits to allow the KVM
range to be adequately placed, rather than the overlap itself.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2] arm64: kvm: fix IDMAP overlap with HYP VA

[Marc Zyngier]

On Sun, 19 Jan 2020 19:43:40 +0000
Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote:

> On Sun, Jan 19, 2020 at 05:43:27PM +0000, Marc Zyngier wrote:
> > On Sat, 28 Dec 2019 11:57:14 +0000
> > Russell King <rmk+kernel@armlinux.org.uk> wrote:
> >   
> > > Booting 5.4 on LX2160A reveals that KVM is non-functional:
> > > 
> > > kvm: Limiting the IPA size due to kernel Virtual Address limit
> > > kvm [1]: IPA Size Limit: 43bits
> > > kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> > > kvm [1]: error initializing Hyp mode: -22
> > > 
> > > Debugging shows:
> > > 
> > > kvm [1]: IDMAP page: 81a26000
> > > kvm [1]: HYP VA range: 0:22ffffffff
> > > 
> > > as RAM is located at:
> > > 
> > > 80000000-fbdfffff : System RAM
> > > 2080000000-237fffffff : System RAM
> > > 
> > > Comparing this with the same kernel on Armada 8040 shows:
> > > 
> > > kvm: Limiting the IPA size due to kernel Virtual Address limit
> > > kvm [1]: IPA Size Limit: 43bits
> > > kvm [1]: IDMAP page: 2a26000
> > > kvm [1]: HYP VA range: 4800000000:493fffffff
> > > ...
> > > kvm [1]: Hyp mode initialized successfully
> > > 
> > > which indicates that hyp_va_msb is set, and is always set to the
> > > opposite value of the idmap page to avoid the overlap. This does not
> > > happen with the LX2160A.
> > > 
> > > Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> > > kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> > > there is still one bit available for hyp_va_msb. Set this bit
> > > appropriately. This allows kvm to be functional on the LX2160A, but
> > > without any HYP VA randomisation:
> > > 
> > > kvm: Limiting the IPA size due to kernel Virtual Address limit
> > > kvm [1]: IPA Size Limit: 43bits
> > > kvm [1]: IDMAP page: 81a24000
> > > kvm [1]: HYP VA range: 4000000000:62ffffffff
> > > ...
> > > kvm [1]: Hyp mode initialized successfully
> > > 
> > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>  
> > 
> > I've applied this to kvmarm-next with a couple of cleanups, and
> > preserving the fallback when the tag is zero (only the mask gets
> > applied, without any ROR or ADD).  
> 
> If only the mask is applied, then it will overlap with the IDMAP
> region, and KVM will fail 

If the tag (which includes the V-1 bit) is *zero*, what else would you
add?

> - so I think it would be a good idea in
> that case to print something a little more useful, rather than
> attributing the KVM failure to an overlap of IDMAP and the KVM
> range.

What other failure mode do you anticipate?

> The real problem is there aren't enough VA bits to allow the KVM
> range to be adequately placed, rather than the overlap itself.

I don't get your point. By construction, there *are* enough VA bits,
since EL2 is only concerned with the linear mapping which only occupies
(at most) half of that VA space. If we can't do that at EL2, then we
can't do it at EL1 either.

	M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel