[PATCH v3 0/2] kvm: arm/arm64: Fixes for race conditions

[PATCH v3 0/2] kvm: arm/arm64: Fixes for race conditions

[Suzuki K Poulose]

The patches fixes race conditions in stage2 pgd accesses.

Patch 1 is a fix up for the patch which has already been pushed to
kvmarm/master.

Patch 2 fixes a case where stage2 PGD could be free'd when we release
the kvm->mmu_lock to prevent VCPU starvation.

Applies on kvmarm/master tree

Changes since V2:
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-May/504401.html

 - Patch 1 in V2 has been pushed to kvmarm/master. Added a fixup to the
   original patch to make sure we don't use cached stage2 pgd.
 - Patch 2: Handle cases where unmap_stage2_range may be called without
   checking the validity of stage2 pgd.


Changes since v1:
 http://lists.infradead.org/pipermail/linux-arm-kernel/2017-April/502867.html

 - Dropped patch for fixing mmu_notifier race condition, which couldn't be
   reproduced.
 - Added reviewed-by from Christoffer
 - Added new patch to fix another race condition

Suzuki K Poulose (2):
  kvm: arm/arm64: Force reading uncached stage2 PGD
  kvm: arm/arm64: Fix use after free of stage2 page table

 virt/kvm/arm/mmu.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

-- 
2.7.4

[PATCH v3 1/2] kvm: arm/arm64: Force reading uncached stage2 PGD

[Suzuki K Poulose]

Make sure we don't use a cached value of the KVM stage2 PGD while
resetting the PGD.

Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Christoffer Dall <cdall@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 virt/kvm/arm/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index 909a1a7..704e35f 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -837,7 +837,7 @@ void kvm_free_stage2_pgd(struct kvm *kvm)
 	spin_lock(&kvm->mmu_lock);
 	if (kvm->arch.pgd) {
 		unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
-		pgd = kvm->arch.pgd;
+		pgd = READ_ONCE(kvm->arch.pgd);
 		kvm->arch.pgd = NULL;
 	}
 	spin_unlock(&kvm->mmu_lock);
-- 
2.7.4

[PATCH v3 2/2] kvm: arm/arm64: Fix use after free of stage2 page table

[Suzuki K Poulose]

We yield the kvm->mmu_lock occassionaly while performing an operation
(e.g, unmap or permission changes) on a large area of stage2 mappings.
However this could possibly cause another thread to clear and free up
the stage2 page tables while we were waiting for regaining the lock and
thus the original thread could end up in accessing memory that was
freed. This patch fixes the problem by making sure that the stage2
pagetable is still valid after we regain the lock. The fact that
mmu_notifer->release() could be called twice (via __mmu_notifier_release
and mmu_notifier_unregsister) enhances the possibility of hitting
this race where there are two threads trying to unmap the entire guest
shadow pages.

While at it, cleanup the redudant checks around cond_resched_lock in
stage2_wp_range(), as cond_resched_lock already does the same checks.

Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Radim Kr?m?? <rkrcmar@redhat.com>
Cc: andreyknvl at google.com
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 virt/kvm/arm/mmu.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c
index 704e35f..a2d6324 100644
--- a/virt/kvm/arm/mmu.c
+++ b/virt/kvm/arm/mmu.c
@@ -295,6 +295,13 @@ static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
 	assert_spin_locked(&kvm->mmu_lock);
 	pgd = kvm->arch.pgd + stage2_pgd_index(addr);
 	do {
+		/*
+		 * Make sure the page table is still active, as another thread
+		 * could have possibly freed the page table, while we released
+		 * the lock.
+		 */
+		if (!READ_ONCE(kvm->arch.pgd))
+			break;
 		next = stage2_pgd_addr_end(addr, end);
 		if (!stage2_pgd_none(*pgd))
 			unmap_stage2_puds(kvm, pgd, addr, next);
@@ -1170,11 +1177,13 @@ static void stage2_wp_range(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
 		 * large. Otherwise, we may see kernel panics with
 		 * CONFIG_DETECT_HUNG_TASK, CONFIG_LOCKUP_DETECTOR,
 		 * CONFIG_LOCKDEP. Additionally, holding the lock too long
-		 * will also starve other vCPUs.
+		 * will also starve other vCPUs. We have to also make sure
+		 * that the page tables are not freed while we released
+		 * the lock.
 		 */
-		if (need_resched() || spin_needbreak(&kvm->mmu_lock))
-			cond_resched_lock(&kvm->mmu_lock);
-
+		cond_resched_lock(&kvm->mmu_lock);
+		if (!READ_ONCE(kvm->arch.pgd))
+			break;
 		next = stage2_pgd_addr_end(addr, end);
 		if (stage2_pgd_present(*pgd))
 			stage2_wp_puds(pgd, addr, next);
-- 
2.7.4

[PATCH v3 0/2] kvm: arm/arm64: Fixes for race conditions

[Christoffer Dall]

On Tue, May 16, 2017 at 10:34:53AM +0100, Suzuki K Poulose wrote:
> The patches fixes race conditions in stage2 pgd accesses.
> 
> Patch 1 is a fix up for the patch which has already been pushed to
> kvmarm/master.
> 
> Patch 2 fixes a case where stage2 PGD could be free'd when we release
> the kvm->mmu_lock to prevent VCPU starvation.
> 
> Applies on kvmarm/master tree

Thanks for the rebase and fix.

Applied to master, and for both patches::

Reviewed-by: Christoffer Dall <cdall@linaro.org>

[PATCH v3 0/2] kvm: arm/arm64: Fixes for race conditions

[Suzuki K Poulose]

On 16/05/17 10:53, Christoffer Dall wrote:
> On Tue, May 16, 2017 at 10:34:53AM +0100, Suzuki K Poulose wrote:
>> The patches fixes race conditions in stage2 pgd accesses.
>>
>> Patch 1 is a fix up for the patch which has already been pushed to
>> kvmarm/master.
>>
>> Patch 2 fixes a case where stage2 PGD could be free'd when we release
>> the kvm->mmu_lock to prevent VCPU starvation.
>>
>> Applies on kvmarm/master tree
>
> Thanks for the rebase and fix.
>
> Applied to master, and for both patches::
>
> Reviewed-by: Christoffer Dall <cdall@linaro.org>
>

Thanks, I think these fixes needs to go to stable as well. I will wait for
these to hit mainline and then send a back port.

Suzuki

[PATCH v3 0/2] kvm: arm/arm64: Fixes for race conditions

[Christoffer Dall]

On Tue, May 16, 2017 at 11:01:54AM +0100, Suzuki K Poulose wrote:
> On 16/05/17 10:53, Christoffer Dall wrote:
> >On Tue, May 16, 2017 at 10:34:53AM +0100, Suzuki K Poulose wrote:
> >>The patches fixes race conditions in stage2 pgd accesses.
> >>
> >>Patch 1 is a fix up for the patch which has already been pushed to
> >>kvmarm/master.
> >>
> >>Patch 2 fixes a case where stage2 PGD could be free'd when we release
> >>the kvm->mmu_lock to prevent VCPU starvation.
> >>
> >>Applies on kvmarm/master tree
> >
> >Thanks for the rebase and fix.
> >
> >Applied to master, and for both patches::
> >
> >Reviewed-by: Christoffer Dall <cdall@linaro.org>
> >
> 
> Thanks, I think these fixes needs to go to stable as well. I will wait for
> these to hit mainline and then send a back port.

Agreed, I already added the cc to stable when applying them, so I think
you'll see the report from the stable folks for the parts that don't
apply and can do the backport then.

Thanks,
-Christoffer