[PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

[PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

[Eric Auger]

Currently when a KVM region is removed using
kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
corresponding intermediate physical memory is not unmapped.

This patch unmaps the region's IPA range in
kvm_arch_commit_memory_region using unmap_stage2_range.

The patch was tested on QEMU VFIO based use case where RAM memory region
creation/deletion frequently happens for IRQ handling.

Notes:
- the KVM_MR_MOVE case shall request some similar addition but I cannot test
  this currently

Signed-off-by: Eric Auger <eric.auger@linaro.org>
---
 arch/arm/include/asm/kvm_mmu.h | 2 ++
 arch/arm/kvm/arm.c             | 8 ++++++++
 arch/arm/kvm/mmu.c             | 2 +-
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 2d122ad..a91c863 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -52,6 +52,8 @@ void kvm_free_stage2_pgd(struct kvm *kvm);
 int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
 			  phys_addr_t pa, unsigned long size);
 
+void unmap_stage2_range(struct kvm *kvm, phys_addr_t guest_ipa, u64 size);
+
 int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run);
 
 void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index bd18bb8..9a4bc10 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -241,6 +241,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 				   const struct kvm_memory_slot *old,
 				   enum kvm_mr_change change)
 {
+	if (change == KVM_MR_DELETE) {
+		gpa_t gpa = old->base_gfn << PAGE_SHIFT;
+		u64 size = old->npages << PAGE_SHIFT;
+
+		spin_lock(&kvm->mmu_lock);
+		unmap_stage2_range(kvm, gpa, size);
+		spin_unlock(&kvm->mmu_lock);
+	}
 }
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 7789857..e8580e2 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -443,7 +443,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
  * destroying the VM), otherwise another faulting VCPU may come in and mess
  * with things behind our backs.
  */
-static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
+void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
 {
 	unmap_range(kvm, kvm->arch.pgd, start, size);
 }
-- 
1.9.1

[PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

[Marc Zyngier]

Hi Eric,

On 03/04/14 16:17, Eric Auger wrote:
> Currently when a KVM region is removed using
> kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
> corresponding intermediate physical memory is not unmapped.
> 
> This patch unmaps the region's IPA range in
> kvm_arch_commit_memory_region using unmap_stage2_range.
> 
> The patch was tested on QEMU VFIO based use case where RAM memory region
> creation/deletion frequently happens for IRQ handling.
> 
> Notes:
> - the KVM_MR_MOVE case shall request some similar addition but I cannot test
>   this currently

I think you should try to handle it anyway. I'm sure you could hack QEMU
to do this test it, but even if you don't, better plug that hole right now.

> Signed-off-by: Eric Auger <eric.auger@linaro.org>
> ---
>  arch/arm/include/asm/kvm_mmu.h | 2 ++
>  arch/arm/kvm/arm.c             | 8 ++++++++
>  arch/arm/kvm/mmu.c             | 2 +-
>  3 files changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
> index 2d122ad..a91c863 100644
> --- a/arch/arm/include/asm/kvm_mmu.h
> +++ b/arch/arm/include/asm/kvm_mmu.h
> @@ -52,6 +52,8 @@ void kvm_free_stage2_pgd(struct kvm *kvm);
>  int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
>  			  phys_addr_t pa, unsigned long size);
>  
> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t guest_ipa, u64 size);
> +
>  int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run);
>  
>  void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index bd18bb8..9a4bc10 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -241,6 +241,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>  				   const struct kvm_memory_slot *old,
>  				   enum kvm_mr_change change)
>  {
> +	if (change == KVM_MR_DELETE) {
> +		gpa_t gpa = old->base_gfn << PAGE_SHIFT;
> +		u64 size = old->npages << PAGE_SHIFT;
> +
> +		spin_lock(&kvm->mmu_lock);
> +		unmap_stage2_range(kvm, gpa, size);
> +		spin_unlock(&kvm->mmu_lock);
> +	}
>  }

Just move the whole function to mmu.c, as it makes more sense to have it
there. And while you're at it, how about moving the other
memslot/memory_region stubs?

>  void kvm_arch_flush_shadow_all(struct kvm *kvm)
> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
> index 7789857..e8580e2 100644
> --- a/arch/arm/kvm/mmu.c
> +++ b/arch/arm/kvm/mmu.c
> @@ -443,7 +443,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
>   * destroying the VM), otherwise another faulting VCPU may come in and mess
>   * with things behind our backs.
>   */
> -static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
>  {
>  	unmap_range(kvm, kvm->arch.pgd, start, size);
>  }
> 

Looks sensible otherwise.

Thanks!

	M.
-- 
Jazz is not dead. It just smells funny...

[PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

[Eric Auger]

On 04/03/2014 05:26 PM, Marc Zyngier wrote:
> Hi Eric,
> 
> On 03/04/14 16:17, Eric Auger wrote:
>> Currently when a KVM region is removed using
>> kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
>> corresponding intermediate physical memory is not unmapped.
>>
>> This patch unmaps the region's IPA range in
>> kvm_arch_commit_memory_region using unmap_stage2_range.
>>
>> The patch was tested on QEMU VFIO based use case where RAM memory region
>> creation/deletion frequently happens for IRQ handling.
>>
>> Notes:
>> - the KVM_MR_MOVE case shall request some similar addition but I cannot test
>>   this currently
> 
> I think you should try to handle it anyway. I'm sure you could hack QEMU
> to do this test it, but even if you don't, better plug that hole right now.

Hi Marc,

OK I will proceed

> 
>> Signed-off-by: Eric Auger <eric.auger@linaro.org>
>> ---
>>  arch/arm/include/asm/kvm_mmu.h | 2 ++
>>  arch/arm/kvm/arm.c             | 8 ++++++++
>>  arch/arm/kvm/mmu.c             | 2 +-
>>  3 files changed, 11 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
>> index 2d122ad..a91c863 100644
>> --- a/arch/arm/include/asm/kvm_mmu.h
>> +++ b/arch/arm/include/asm/kvm_mmu.h
>> @@ -52,6 +52,8 @@ void kvm_free_stage2_pgd(struct kvm *kvm);
>>  int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa,
>>  			  phys_addr_t pa, unsigned long size);
>>  
>> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t guest_ipa, u64 size);
>> +
>>  int kvm_handle_guest_abort(struct kvm_vcpu *vcpu, struct kvm_run *run);
>>  
>>  void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu);
>> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
>> index bd18bb8..9a4bc10 100644
>> --- a/arch/arm/kvm/arm.c
>> +++ b/arch/arm/kvm/arm.c
>> @@ -241,6 +241,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>>  				   const struct kvm_memory_slot *old,
>>  				   enum kvm_mr_change change)
>>  {
>> +	if (change == KVM_MR_DELETE) {
>> +		gpa_t gpa = old->base_gfn << PAGE_SHIFT;
>> +		u64 size = old->npages << PAGE_SHIFT;
>> +
>> +		spin_lock(&kvm->mmu_lock);
>> +		unmap_stage2_range(kvm, gpa, size);
>> +		spin_unlock(&kvm->mmu_lock);
>> +	}
>>  }
> 
> Just move the whole function to mmu.c, as it makes more sense to have it
> there. And while you're at it, how about moving the other
> memslot/memory_region stubs?

OK I will move:
kvm_arch_free_memslot,
kvm_arch_create_memslot,
kvm_arch_prepare_memory_region,
kvm_arch_commit_memory_region,
kvm_arch_shadow_all and
kvm_arch_shadow_memslot then.

Best Regards

Eric

> 
>>  void kvm_arch_flush_shadow_all(struct kvm *kvm)
>> diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
>> index 7789857..e8580e2 100644
>> --- a/arch/arm/kvm/mmu.c
>> +++ b/arch/arm/kvm/mmu.c
>> @@ -443,7 +443,7 @@ int kvm_alloc_stage2_pgd(struct kvm *kvm)
>>   * destroying the VM), otherwise another faulting VCPU may come in and mess
>>   * with things behind our backs.
>>   */
>> -static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
>> +void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
>>  {
>>  	unmap_range(kvm, kvm->arch.pgd, start, size);
>>  }
>>
> 
> Looks sensible otherwise.
> 
> Thanks!
> 
> 	M.
> 

[PATCH] ARM: KVM: Handle IPA unmapping on memory region deletion

[Christoffer Dall]

On 3 April 2014 08:26, Marc Zyngier <marc.zyngier@arm.com> wrote:
> Hi Eric,
>
> On 03/04/14 16:17, Eric Auger wrote:
>> Currently when a KVM region is removed using
>> kvm_vm_ioctl_set_memory_region (with memory region size equal to 0), the
>> corresponding intermediate physical memory is not unmapped.
>>
>> This patch unmaps the region's IPA range in
>> kvm_arch_commit_memory_region using unmap_stage2_range.
>>
>> The patch was tested on QEMU VFIO based use case where RAM memory region
>> creation/deletion frequently happens for IRQ handling.
>>
>> Notes:
>> - the KVM_MR_MOVE case shall request some similar addition but I cannot test
>>   this currently
>
> I think you should try to handle it anyway. I'm sure you could hack QEMU
> to do this test it, but even if you don't, better plug that hole right now.
>
I don't think we want untested code in the kernel if we can avoid it,
so I would say, either test it somehow (kvm-unit-tests ?) or print a
warning...?

-Christoffer