Re: [PATCH] arm64: kvm: fix IDMAP overlap with HYP VA

[Marc Zyngier <maz@kernel.org>]

Hi Russell,

Thanks for this.

On Fri, 27 Dec 2019 11:47:35 +0000,
Russell King <rmk+kernel@armlinux.org.uk> wrote:
> 
> Booting 5.4 on LX2160A reveals that KVM is non-functional:
> 
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> kvm [1]: error initializing Hyp mode: -22
> 
> Debugging shows:
> 
> kvm [1]: IDMAP page: 81a26000
> kvm [1]: HYP VA range: 0:22ffffffff
> 
> as RAM is located at:
> 
> 80000000-fbdfffff : System RAM
> 2080000000-237fffffff : System RAM

Ouch. This looks like a terrible choice for a memory map.

> 
> Comparing this with the same kernel on Armada 8040 shows:
> 
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 2a26000
> kvm [1]: HYP VA range: 4800000000:493fffffff
> ...
> kvm [1]: Hyp mode initialized successfully
> 
> which indicates that hyp_va_msb is set, and is always set to the
> opposite value of the idmap page to avoid the overlap. This does not
> happen with the LX2160A.
> 
> Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> there is still one bit available for hyp_va_msb. Set this bit
> appropriately. This allows kvm to be functional on the LX2160A, but
> without any HYP VA randomisation:
> 
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 81a24000
> kvm [1]: HYP VA range: 4000000000:62ffffffff
> ...
> kvm [1]: Hyp mode initialized successfully

Nice bit of debugging. I guess part of the confusion is due to the
fact that the hyp_va_msb is part of the tag (as you found out), but
that the documentation doesn't really make that clear at all (and only
mentions the random part of the tag).

> 
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> ---
>  arch/arm64/kvm/va_layout.c | 22 +++++++++++++++-------
>  1 file changed, 15 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c
> index 2cf7d4b606c3..83f8b3f51cf4 100644
> --- a/arch/arm64/kvm/va_layout.c
> +++ b/arch/arm64/kvm/va_layout.c
> @@ -22,6 +22,17 @@ static u8 tag_lsb;
>  static u64 tag_val;
>  static u64 va_mask;
>  
> +/*
> + * We want to generate a hyp VA with the following format:
> + *
> + *  63 ... V |     V-1    | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> + *  ---------------------------------------------------------
> + * | 0000000 | hyp_va_msb |    random tag  |  kern linear VA |
                \---------- tag -------------/

How about the above, to make it clearer that the tag must include the
hyp_va_msb bit to avoid clashing with the IDMAP?

> + *
> + * which does not conflict with the idmap regions. This means that hyp_va_msb
> + * must always be present. Luckily, when kva_msb == (vabits_actual - 1) we

I'm not sure the "Luckily" part is appropriate here. Given the way
kva_msb is computed, I can't see how it can see how this can
fail.

This stems from the fact that the vabits space for kernel mappings is
split in two: vabits-1 for the linear map, and vabits-1 for the rest
(kernel text and co). Given that we define kva_msb as the highest
order bit that can change within the linear map, its value is at most
vabits-1.

> + * still have one bit for this, but no bits for the random tag.
> + */
>  static void compute_layout(void)
>  {
>  	phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start);
> @@ -39,19 +50,16 @@ static void compute_layout(void)
>  		/*
>  		 * No space in the address, let's compute the mask so
>  		 * that it covers (vabits_actual - 1) bits, and the region
> -		 * bit. The tag stays set to zero.
> +		 * bit.
>  		 */
> -		va_mask  = BIT(vabits_actual - 1) - 1;
> -		va_mask |= hyp_va_msb;
> +		tag_lsb = kva_msb;
> +		va_mask = BIT(vabits_actual - 1) - 1;
> +		tag_val = hyp_va_msb >> tag_lsb;
>  	} else {
>  		/*
>  		 * We do have some free bits to insert a random tag.
>  		 * Hyp VAs are now created from kernel linear map VAs
>  		 * using the following formula (with V == vabits_actual):
> -		 *
> -		 *  63 ... V |     V-1    | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> -		 *  ---------------------------------------------------------
> -		 * | 0000000 | hyp_va_msb |    random tag  |  kern linear VA |
>  		 */
>  		tag_lsb = kva_msb;
>  		va_mask = GENMASK_ULL(tag_lsb - 1, 0);

In the light of this, it'd be great to rework this code to simplify it
(getting rid of kva_msb, which really is tag_lsb) and make some of the
computing common to both branches.

Thanks,

	M.

-- 
Jazz is not dead, it just smells funny.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm