[PATCH v8 0/3] KVM: Dirty quota-based throttling

[PATCH v8 0/3] KVM: Dirty quota-based throttling

[Shivam Kumar]

This is v8 of the dirty quota series, with the following changes over
v7:

1. Removed pages_dirtied stat. Now, a single variable
dirty_quota_bytes is being used for throttling.
2. IOCTL to enable/disable dirty quota throttling. Enabling/disabling
can be done dynamically, e.g. we can enable dirty quota just before a
live migration and disable it just after the live migration.
3. Decoupled dirty quota from dirty logging. Introduced a new function
update_dirty_quota that decreases dirty_quota_bytes by by the
appropriate architecture-specific granule or page size. It also raises
a KVM request if dirty quota is exhausted.
4. Each arch that wants to use dirty quota throttling feature needs
to call update_dirty_quota at each time a page is dirtied. Also, it
needs to process the KVM request raised by update_dirty_quota and
facilitate exit to userspace. Added support for x86 and arm64.
5. Code refactoring and minor nits.

v1:
https://lore.kernel.org/kvm/20211114145721.209219-1-shivam.kumar1@nutanix.com/
v2: https://lore.kernel.org/kvm/Ydx2EW6U3fpJoJF0@google.com/T/
v3: https://lore.kernel.org/kvm/YkT1kzWidaRFdQQh@google.com/T/
v4:
https://lore.kernel.org/all/20220521202937.184189-1-shivam.kumar1@nutanix.com/
v5: https://lore.kernel.org/all/202209130532.2BJwW65L-lkp@intel.com/T/
v6:
https://lore.kernel.org/all/20220915101049.187325-1-shivam.kumar1@nutanix.com/
v7:
https://lore.kernel.org/all/a64d9818-c68d-1e33-5783-414e9a9bdbd1@nutanix.com/t/

Thanks,
Shivam

Shivam Kumar (3):
  KVM: Implement dirty quota-based throttling of vcpus
  KVM: x86: Dirty quota-based throttling of vcpus
  KVM: arm64: Dirty quota-based throttling of vcpus

 Documentation/virt/kvm/api.rst | 17 +++++++++++++++++
 arch/arm64/kvm/Kconfig         |  1 +
 arch/arm64/kvm/arm.c           |  7 +++++++
 arch/arm64/kvm/mmu.c           |  3 +++
 arch/x86/kvm/Kconfig           |  1 +
 arch/x86/kvm/mmu/mmu.c         |  8 +++++++-
 arch/x86/kvm/mmu/spte.c        |  3 +++
 arch/x86/kvm/mmu/tdp_mmu.c     |  3 +++
 arch/x86/kvm/vmx/vmx.c         |  5 +++++
 arch/x86/kvm/x86.c             | 16 ++++++++++++++++
 arch/x86/kvm/xen.c             | 12 +++++++++++-
 include/linux/kvm_host.h       |  5 +++++
 include/uapi/linux/kvm.h       |  8 ++++++++
 tools/include/uapi/linux/kvm.h |  1 +
 virt/kvm/Kconfig               |  3 +++
 virt/kvm/kvm_main.c            | 31 +++++++++++++++++++++++++++++++
 16 files changed, 122 insertions(+), 2 deletions(-)

-- 
2.22.3

[PATCH v8 1/3] KVM: Implement dirty quota-based throttling of vcpus

[Shivam Kumar]

Define dirty_quota_bytes variable to track and throttle memory
dirtying for every vcpu. This variable stores the number of bytes the
vcpu is allowed to dirty. To dirty more, the vcpu needs to request
more quota by exiting to userspace.

Implement update_dirty_quota function which

i) Decreases dirty_quota_bytes by arch-specific page size whenever a
page is dirtied.
ii) Raises a KVM request KVM_REQ_DIRTY_QUOTA_EXIT whenever the dirty
quota is exhausted (i.e. dirty_quota_bytes <= 0).

Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
---
 Documentation/virt/kvm/api.rst | 17 +++++++++++++++++
 include/linux/kvm_host.h       |  5 +++++
 include/uapi/linux/kvm.h       |  8 ++++++++
 tools/include/uapi/linux/kvm.h |  1 +
 virt/kvm/Kconfig               |  3 +++
 virt/kvm/kvm_main.c            | 31 +++++++++++++++++++++++++++++++
 6 files changed, 65 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 62de0768d6aa..3a283fe212d8 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6688,6 +6688,23 @@ Please note that the kernel is allowed to use the kvm_run structure as the
 primary storage for certain register types. Therefore, the kernel may use the
 values in kvm_run even if the corresponding bit in kvm_dirty_regs is not set.
 
+::
+
+	/*
+	 * Number of bytes the vCPU is allowed to dirty if KVM_CAP_DIRTY_QUOTA is
+	 * enabled. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if this quota
+	 * is exhausted, i.e. dirty_quota_bytes <= 0.
+	 */
+	long dirty_quota_bytes;
+
+Please note that enforcing the quota is best effort. Dirty quota is reduced by
+arch-specific page size when any guest page is dirtied. Also, the guest may dirty
+multiple pages before KVM can recheck the quota.
+
+::
+  };
+
+
 
 6. Capabilities that can be enabled on vCPUs
 ============================================
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 8ada23756b0e..f5ce343c64f2 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -167,6 +167,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_VM_DEAD			(1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
 #define KVM_REQ_UNBLOCK			2
 #define KVM_REQ_DIRTY_RING_SOFT_FULL	3
+#define KVM_REQ_DIRTY_QUOTA_EXIT	4
 #define KVM_REQUEST_ARCH_BASE		8
 
 /*
@@ -800,6 +801,9 @@ struct kvm {
 	bool dirty_ring_with_bitmap;
 	bool vm_bugged;
 	bool vm_dead;
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	bool dirty_quota_enabled;
+#endif
 
 #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
 	struct notifier_block pm_notifier;
@@ -1235,6 +1239,7 @@ struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn);
 bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn);
 bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn);
 unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn);
+void update_dirty_quota(struct kvm *kvm, unsigned long page_size_bytes);
 void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn);
 void mark_page_dirty(struct kvm *kvm, gfn_t gfn);
 
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index d77aef872a0a..ddb9d3d797c4 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -264,6 +264,7 @@ struct kvm_xen_exit {
 #define KVM_EXIT_RISCV_SBI        35
 #define KVM_EXIT_RISCV_CSR        36
 #define KVM_EXIT_NOTIFY           37
+#define KVM_EXIT_DIRTY_QUOTA_EXHAUSTED 38
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -526,6 +527,12 @@ struct kvm_run {
 		struct kvm_sync_regs regs;
 		char padding[SYNC_REGS_SIZE_BYTES];
 	} s;
+	/*
+	 * Number of bytes the vCPU is allowed to dirty if KVM_CAP_DIRTY_QUOTA is
+	 * enabled. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if this quota
+	 * is exhausted, i.e. dirty_quota_bytes <= 0.
+	 */
+	long dirty_quota_bytes;
 };
 
 /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */
@@ -1184,6 +1191,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
 #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
 #define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226
+#define KVM_CAP_DIRTY_QUOTA 227
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index 55155e262646..48f236e2b836 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -1175,6 +1175,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223
 #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
 #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
+#define KVM_CAP_DIRTY_QUOTA 227
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index b74916de5183..ccaa332d88f9 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -19,6 +19,9 @@ config HAVE_KVM_IRQ_ROUTING
 config HAVE_KVM_DIRTY_RING
        bool
 
+config HAVE_KVM_DIRTY_QUOTA
+       bool
+
 # Only strongly ordered architectures can select this, as it doesn't
 # put any explicit constraint on userspace ordering. They can also
 # select the _ACQ_REL version.
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index d255964ec331..744b955514ce 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -3096,6 +3096,9 @@ static int __kvm_write_guest_page(struct kvm *kvm,
 	r = __copy_to_user((void __user *)addr + offset, data, len);
 	if (r)
 		return -EFAULT;
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(kvm, memslot, gfn);
 	return 0;
 }
@@ -3234,6 +3237,9 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
 	r = __copy_to_user((void __user *)ghc->hva + offset, data, len);
 	if (r)
 		return -EFAULT;
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(kvm, ghc->memslot, gpa >> PAGE_SHIFT);
 
 	return 0;
@@ -3304,6 +3310,18 @@ int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len)
 }
 EXPORT_SYMBOL_GPL(kvm_clear_guest);
 
+void update_dirty_quota(struct kvm *kvm, unsigned long page_size_bytes)
+{
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+	if (!vcpu || (vcpu->kvm != kvm) || !READ_ONCE(kvm->dirty_quota_enabled))
+		return;
+
+	vcpu->run->dirty_quota_bytes -= page_size_bytes;
+	if (vcpu->run->dirty_quota_bytes <= 0)
+		kvm_make_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu);
+}
+
 void mark_page_dirty_in_slot(struct kvm *kvm,
 			     const struct kvm_memory_slot *memslot,
 		 	     gfn_t gfn)
@@ -3334,6 +3352,9 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn)
 	struct kvm_memory_slot *memslot;
 
 	memslot = gfn_to_memslot(kvm, gfn);
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(kvm, memslot, gfn);
 }
 EXPORT_SYMBOL_GPL(mark_page_dirty);
@@ -3343,6 +3364,9 @@ void kvm_vcpu_mark_page_dirty(struct kvm_vcpu *vcpu, gfn_t gfn)
 	struct kvm_memory_slot *memslot;
 
 	memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(vcpu->kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(vcpu->kvm, memslot, gfn);
 }
 EXPORT_SYMBOL_GPL(kvm_vcpu_mark_page_dirty);
@@ -4524,6 +4548,8 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
 	case KVM_CAP_BINARY_STATS_FD:
 	case KVM_CAP_SYSTEM_EVENT_DATA:
 		return 1;
+	case KVM_CAP_DIRTY_QUOTA:
+		return !!IS_ENABLED(CONFIG_HAVE_KVM_DIRTY_QUOTA);
 	default:
 		break;
 	}
@@ -4673,6 +4699,11 @@ static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm,
 
 		return r;
 	}
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	case KVM_CAP_DIRTY_QUOTA:
+		WRITE_ONCE(kvm->dirty_quota_enabled, cap->args[0]);
+		return 0;
+#endif
 	default:
 		return kvm_vm_ioctl_enable_cap(kvm, cap);
 	}
-- 
2.22.3

[PATCH v8 2/3] KVM: x86: Dirty quota-based throttling of vcpus

[Shivam Kumar]

Call update_dirty_quota whenever a page is marked dirty with
appropriate arch-specific page size. Process the KVM request
KVM_REQ_DIRTY_QUOTA_EXIT (raised by update_dirty_quota) to exit to
userspace with exit reason KVM_EXIT_DIRTY_QUOTA_EXHAUSTED.

Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
---
 arch/x86/kvm/Kconfig       |  1 +
 arch/x86/kvm/mmu/mmu.c     |  8 +++++++-
 arch/x86/kvm/mmu/spte.c    |  3 +++
 arch/x86/kvm/mmu/tdp_mmu.c |  3 +++
 arch/x86/kvm/vmx/vmx.c     |  5 +++++
 arch/x86/kvm/x86.c         | 16 ++++++++++++++++
 arch/x86/kvm/xen.c         | 12 +++++++++++-
 7 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index 8e578311ca9d..8621a9512572 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -48,6 +48,7 @@ config KVM
 	select KVM_VFIO
 	select SRCU
 	select INTERVAL_TREE
+	select HAVE_KVM_DIRTY_QUOTA
 	select HAVE_KVM_PM_NOTIFIER if PM
 	select KVM_GENERIC_HARDWARE_ENABLING
 	help
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index c8ebe542c565..e0c8348ecdf1 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -3323,8 +3323,14 @@ fast_pf_fix_direct_spte(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
 	if (!try_cmpxchg64(sptep, &old_spte, new_spte))
 		return false;
 
-	if (is_writable_pte(new_spte) && !is_writable_pte(old_spte))
+	if (is_writable_pte(new_spte) && !is_writable_pte(old_spte)) {
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		struct kvm_mmu_page *sp = sptep_to_sp(sptep);
+
+		update_dirty_quota(vcpu->kvm, (1L << SPTE_LEVEL_SHIFT(sp->role.level)));
+#endif
 		mark_page_dirty_in_slot(vcpu->kvm, fault->slot, fault->gfn);
+	}
 
 	return true;
 }
diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c
index c15bfca3ed15..15f4f1d97ce9 100644
--- a/arch/x86/kvm/mmu/spte.c
+++ b/arch/x86/kvm/mmu/spte.c
@@ -243,6 +243,9 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
 	if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) {
 		/* Enforced by kvm_mmu_hugepage_adjust. */
 		WARN_ON(level > PG_LEVEL_4K);
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		update_dirty_quota(vcpu->kvm, (1L << SPTE_LEVEL_SHIFT(level)));
+#endif
 		mark_page_dirty_in_slot(vcpu->kvm, slot, gfn);
 	}
 
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index 7c25dbf32ecc..4bf98e96343d 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -358,6 +358,9 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn,
 
 	if ((!is_writable_pte(old_spte) || pfn_changed) &&
 	    is_writable_pte(new_spte)) {
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		update_dirty_quota(kvm, (1L << SPTE_LEVEL_SHIFT(level)));
+#endif
 		slot = __gfn_to_memslot(__kvm_memslots(kvm, as_id), gfn);
 		mark_page_dirty_in_slot(kvm, slot, gfn);
 	}
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index bcac3efcde41..da4c6342a647 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -5861,6 +5861,11 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)
 		 */
 		if (__xfer_to_guest_mode_work_pending())
 			return 1;
+
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		if (kvm_test_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu))
+			return 1;
+#endif
 	}
 
 	return 1;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 7713420abab0..1733be829197 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3092,6 +3092,9 @@ static void kvm_setup_guest_pvclock(struct kvm_vcpu *v,
 
 	guest_hv_clock->version = ++vcpu->hv_clock.version;
 
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(v->kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(v->kvm, gpc->memslot, gpc->gpa >> PAGE_SHIFT);
 	read_unlock_irqrestore(&gpc->lock, flags);
 
@@ -3566,6 +3569,9 @@ static void record_steal_time(struct kvm_vcpu *vcpu)
  out:
 	user_access_end();
  dirty:
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(vcpu->kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(vcpu->kvm, ghc->memslot, gpa_to_gfn(ghc->gpa));
 }
 
@@ -4815,6 +4821,9 @@ static void kvm_steal_time_set_preempted(struct kvm_vcpu *vcpu)
 	if (!copy_to_user_nofault(&st->preempted, &preempted, sizeof(preempted)))
 		vcpu->arch.st.preempted = KVM_VCPU_PREEMPTED;
 
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(vcpu->kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(vcpu->kvm, ghc->memslot, gpa_to_gfn(ghc->gpa));
 }
 
@@ -10514,6 +10523,13 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 			r = 0;
 			goto out;
 		}
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) {
+			vcpu->run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED;
+			r = 0;
+			goto out;
+		}
+#endif
 
 		/*
 		 * KVM_REQ_HV_STIMER has to be processed after
diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c
index 40edf4d1974c..00a3ac438539 100644
--- a/arch/x86/kvm/xen.c
+++ b/arch/x86/kvm/xen.c
@@ -435,9 +435,16 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic)
 
 	read_unlock_irqrestore(&gpc1->lock, flags);
 
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(v->kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(v->kvm, gpc1->memslot, gpc1->gpa >> PAGE_SHIFT);
-	if (user_len2)
+	if (user_len2) {
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		update_dirty_quota(v->kvm, PAGE_SIZE);
+#endif
 		mark_page_dirty_in_slot(v->kvm, gpc2->memslot, gpc2->gpa >> PAGE_SHIFT);
+	}
 }
 
 void kvm_xen_update_runstate(struct kvm_vcpu *v, int state)
@@ -549,6 +556,9 @@ void kvm_xen_inject_pending_events(struct kvm_vcpu *v)
 	if (v->arch.xen.upcall_vector)
 		kvm_xen_inject_vcpu_vector(v);
 
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+	update_dirty_quota(v->kvm, PAGE_SIZE);
+#endif
 	mark_page_dirty_in_slot(v->kvm, gpc->memslot, gpc->gpa >> PAGE_SHIFT);
 }
 
-- 
2.22.3

[PATCH v8 3/3] KVM: arm64: Dirty quota-based throttling of vcpus

[Shivam Kumar]

Call update_dirty_quota whenever a page is marked dirty with
appropriate arch-specific page size. Process the KVM request
KVM_REQ_DIRTY_QUOTA_EXIT (raised by update_dirty_quota) to exit to
userspace with exit reason KVM_EXIT_DIRTY_QUOTA_EXHAUSTED.

Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
---
 arch/arm64/kvm/Kconfig | 1 +
 arch/arm64/kvm/arm.c   | 7 +++++++
 arch/arm64/kvm/mmu.c   | 3 +++
 3 files changed, 11 insertions(+)

diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index ca6eadeb7d1a..8e7dea2c3a9f 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -44,6 +44,7 @@ menuconfig KVM
 	select SCHED_INFO
 	select GUEST_PERF_EVENTS if PERF_EVENTS
 	select INTERVAL_TREE
+	select HAVE_KVM_DIRTY_QUOTA
 	help
 	  Support hosting virtualized guest machines.
 
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 3bd732eaf087..5162b2fc46a1 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -757,6 +757,13 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
 
 		if (kvm_dirty_ring_check_request(vcpu))
 			return 0;
+
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) {
+			vcpu->run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED;
+			return 0;
+		}
+#endif
 	}
 
 	return 1;
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 7113587222ff..baf416046f46 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1390,6 +1390,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	/* Mark the page dirty only if the fault is handled successfully */
 	if (writable && !ret) {
 		kvm_set_pfn_dirty(pfn);
+#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
+		update_dirty_quota(kvm, fault_granule);
+#endif
 		mark_page_dirty_in_slot(kvm, memslot, gfn);
 	}
 
-- 
2.22.3

Re: [PATCH v8 1/3] KVM: Implement dirty quota-based throttling of vcpus

[Marc Zyngier]

On Sat, 25 Feb 2023 20:47:57 +0000,
Shivam Kumar <shivam.kumar1@nutanix.com> wrote:
> 
> Define dirty_quota_bytes variable to track and throttle memory
> dirtying for every vcpu. This variable stores the number of bytes the
> vcpu is allowed to dirty. To dirty more, the vcpu needs to request
> more quota by exiting to userspace.
> 
> Implement update_dirty_quota function which
> 
> i) Decreases dirty_quota_bytes by arch-specific page size whenever a
> page is dirtied.
> ii) Raises a KVM request KVM_REQ_DIRTY_QUOTA_EXIT whenever the dirty
> quota is exhausted (i.e. dirty_quota_bytes <= 0).
> 
> Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
> Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
> Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
> Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
> ---
>  Documentation/virt/kvm/api.rst | 17 +++++++++++++++++
>  include/linux/kvm_host.h       |  5 +++++
>  include/uapi/linux/kvm.h       |  8 ++++++++
>  tools/include/uapi/linux/kvm.h |  1 +
>  virt/kvm/Kconfig               |  3 +++
>  virt/kvm/kvm_main.c            | 31 +++++++++++++++++++++++++++++++
>  6 files changed, 65 insertions(+)
> 
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index 62de0768d6aa..3a283fe212d8 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -6688,6 +6688,23 @@ Please note that the kernel is allowed to use the kvm_run structure as the
>  primary storage for certain register types. Therefore, the kernel may use the
>  values in kvm_run even if the corresponding bit in kvm_dirty_regs is not set.
>  
> +::
> +
> +	/*
> +	 * Number of bytes the vCPU is allowed to dirty if KVM_CAP_DIRTY_QUOTA is
> +	 * enabled. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if this quota
> +	 * is exhausted, i.e. dirty_quota_bytes <= 0.
> +	 */
> +	long dirty_quota_bytes;
> +
> +Please note that enforcing the quota is best effort. Dirty quota is reduced by
> +arch-specific page size when any guest page is dirtied. Also, the guest may dirty
> +multiple pages before KVM can recheck the quota.

What are the events that trigger such quota reduction?

> +
> +::
> +  };
> +
> +
>  
>  6. Capabilities that can be enabled on vCPUs
>  ============================================
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 8ada23756b0e..f5ce343c64f2 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -167,6 +167,7 @@ static inline bool is_error_page(struct page *page)
>  #define KVM_REQ_VM_DEAD			(1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
>  #define KVM_REQ_UNBLOCK			2
>  #define KVM_REQ_DIRTY_RING_SOFT_FULL	3
> +#define KVM_REQ_DIRTY_QUOTA_EXIT	4
>  #define KVM_REQUEST_ARCH_BASE		8
>  
>  /*
> @@ -800,6 +801,9 @@ struct kvm {
>  	bool dirty_ring_with_bitmap;
>  	bool vm_bugged;
>  	bool vm_dead;
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	bool dirty_quota_enabled;
> +#endif
>  
>  #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
>  	struct notifier_block pm_notifier;
> @@ -1235,6 +1239,7 @@ struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn);
>  bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn);
>  bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn);
>  unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn);
> +void update_dirty_quota(struct kvm *kvm, unsigned long page_size_bytes);
>  void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn);
>  void mark_page_dirty(struct kvm *kvm, gfn_t gfn);
>  
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index d77aef872a0a..ddb9d3d797c4 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -264,6 +264,7 @@ struct kvm_xen_exit {
>  #define KVM_EXIT_RISCV_SBI        35
>  #define KVM_EXIT_RISCV_CSR        36
>  #define KVM_EXIT_NOTIFY           37
> +#define KVM_EXIT_DIRTY_QUOTA_EXHAUSTED 38
>  
>  /* For KVM_EXIT_INTERNAL_ERROR */
>  /* Emulate instruction failed. */
> @@ -526,6 +527,12 @@ struct kvm_run {
>  		struct kvm_sync_regs regs;
>  		char padding[SYNC_REGS_SIZE_BYTES];
>  	} s;
> +	/*
> +	 * Number of bytes the vCPU is allowed to dirty if KVM_CAP_DIRTY_QUOTA is
> +	 * enabled. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if this quota
> +	 * is exhausted, i.e. dirty_quota_bytes <= 0.
> +	 */
> +	long dirty_quota_bytes;
>  };
>  
>  /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */
> @@ -1184,6 +1191,7 @@ struct kvm_ppc_resize_hpt {
>  #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
>  #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
>  #define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226
> +#define KVM_CAP_DIRTY_QUOTA 227
>  
>  #ifdef KVM_CAP_IRQ_ROUTING
>  
> diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
> index 55155e262646..48f236e2b836 100644
> --- a/tools/include/uapi/linux/kvm.h
> +++ b/tools/include/uapi/linux/kvm.h
> @@ -1175,6 +1175,7 @@ struct kvm_ppc_resize_hpt {
>  #define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223
>  #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
>  #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
> +#define KVM_CAP_DIRTY_QUOTA 227
>  
>  #ifdef KVM_CAP_IRQ_ROUTING
>  
> diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
> index b74916de5183..ccaa332d88f9 100644
> --- a/virt/kvm/Kconfig
> +++ b/virt/kvm/Kconfig
> @@ -19,6 +19,9 @@ config HAVE_KVM_IRQ_ROUTING
>  config HAVE_KVM_DIRTY_RING
>         bool
>  
> +config HAVE_KVM_DIRTY_QUOTA
> +       bool
> +
>  # Only strongly ordered architectures can select this, as it doesn't
>  # put any explicit constraint on userspace ordering. They can also
>  # select the _ACQ_REL version.
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index d255964ec331..744b955514ce 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -3096,6 +3096,9 @@ static int __kvm_write_guest_page(struct kvm *kvm,
>  	r = __copy_to_user((void __user *)addr + offset, data, len);
>  	if (r)
>  		return -EFAULT;
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	update_dirty_quota(kvm, PAGE_SIZE);
> +#endif

Why PAGE_SIZE? Why not 'len'? Why if the page was already dirtied? Why
should it be accounted for multiple times? In most cases, this is the
*hypervisor* writing to the guest, not the vcpu. Why should this be
accounted to the vcpu quota?

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v8 3/3] KVM: arm64: Dirty quota-based throttling of vcpus

[Marc Zyngier]

On Sat, 25 Feb 2023 20:48:01 +0000,
Shivam Kumar <shivam.kumar1@nutanix.com> wrote:
> 
> Call update_dirty_quota whenever a page is marked dirty with
> appropriate arch-specific page size. Process the KVM request
> KVM_REQ_DIRTY_QUOTA_EXIT (raised by update_dirty_quota) to exit to
> userspace with exit reason KVM_EXIT_DIRTY_QUOTA_EXHAUSTED.
> 
> Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
> Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
> Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
> Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
> ---
>  arch/arm64/kvm/Kconfig | 1 +
>  arch/arm64/kvm/arm.c   | 7 +++++++
>  arch/arm64/kvm/mmu.c   | 3 +++
>  3 files changed, 11 insertions(+)
> 
> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
> index ca6eadeb7d1a..8e7dea2c3a9f 100644
> --- a/arch/arm64/kvm/Kconfig
> +++ b/arch/arm64/kvm/Kconfig
> @@ -44,6 +44,7 @@ menuconfig KVM
>  	select SCHED_INFO
>  	select GUEST_PERF_EVENTS if PERF_EVENTS
>  	select INTERVAL_TREE
> +	select HAVE_KVM_DIRTY_QUOTA

So this is selected unconditionally...

>  	help
>  	  Support hosting virtualized guest machines.
>  
> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index 3bd732eaf087..5162b2fc46a1 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -757,6 +757,13 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
>  
>  		if (kvm_dirty_ring_check_request(vcpu))
>  			return 0;
> +
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA

... and yet you litter the arch code with #ifdefs...

> +		if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) {
> +			vcpu->run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED;
> +			return 0;

What rechecks the quota on entry?

> +		}
> +#endif
>  	}
>  
>  	return 1;
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 7113587222ff..baf416046f46 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1390,6 +1390,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>  	/* Mark the page dirty only if the fault is handled successfully */
>  	if (writable && !ret) {
>  		kvm_set_pfn_dirty(pfn);
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		update_dirty_quota(kvm, fault_granule);

fault_granule isn't necessarily the amount that gets dirtied.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v8 2/3] KVM: x86: Dirty quota-based throttling of vcpus

[Yuan Yao]

On Sat, Feb 25, 2023 at 08:47:59PM +0000, Shivam Kumar wrote:
> Call update_dirty_quota whenever a page is marked dirty with
> appropriate arch-specific page size. Process the KVM request
> KVM_REQ_DIRTY_QUOTA_EXIT (raised by update_dirty_quota) to exit to
> userspace with exit reason KVM_EXIT_DIRTY_QUOTA_EXHAUSTED.
>
> Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
> Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
> Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
> Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
> ---
>  arch/x86/kvm/Kconfig       |  1 +
>  arch/x86/kvm/mmu/mmu.c     |  8 +++++++-
>  arch/x86/kvm/mmu/spte.c    |  3 +++
>  arch/x86/kvm/mmu/tdp_mmu.c |  3 +++
>  arch/x86/kvm/vmx/vmx.c     |  5 +++++
>  arch/x86/kvm/x86.c         | 16 ++++++++++++++++
>  arch/x86/kvm/xen.c         | 12 +++++++++++-
>  7 files changed, 46 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
> index 8e578311ca9d..8621a9512572 100644
> --- a/arch/x86/kvm/Kconfig
> +++ b/arch/x86/kvm/Kconfig
> @@ -48,6 +48,7 @@ config KVM
>  	select KVM_VFIO
>  	select SRCU
>  	select INTERVAL_TREE
> +	select HAVE_KVM_DIRTY_QUOTA
>  	select HAVE_KVM_PM_NOTIFIER if PM
>  	select KVM_GENERIC_HARDWARE_ENABLING
>  	help
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index c8ebe542c565..e0c8348ecdf1 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -3323,8 +3323,14 @@ fast_pf_fix_direct_spte(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
>  	if (!try_cmpxchg64(sptep, &old_spte, new_spte))
>  		return false;
>
> -	if (is_writable_pte(new_spte) && !is_writable_pte(old_spte))
> +	if (is_writable_pte(new_spte) && !is_writable_pte(old_spte)) {
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		struct kvm_mmu_page *sp = sptep_to_sp(sptep);
> +
> +		update_dirty_quota(vcpu->kvm, (1L << SPTE_LEVEL_SHIFT(sp->role.level)));
> +#endif
>  		mark_page_dirty_in_slot(vcpu->kvm, fault->slot, fault->gfn);

Possible to call update_dirty_quota() from mark_page_dirty_in_slot() ?
Then other Architectures can be covered yet.

> +	}
>
>  	return true;
>  }
> diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c
> index c15bfca3ed15..15f4f1d97ce9 100644
> --- a/arch/x86/kvm/mmu/spte.c
> +++ b/arch/x86/kvm/mmu/spte.c
> @@ -243,6 +243,9 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
>  	if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) {
>  		/* Enforced by kvm_mmu_hugepage_adjust. */
>  		WARN_ON(level > PG_LEVEL_4K);
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		update_dirty_quota(vcpu->kvm, (1L << SPTE_LEVEL_SHIFT(level)));
> +#endif
>  		mark_page_dirty_in_slot(vcpu->kvm, slot, gfn);
>  	}
>
> diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
> index 7c25dbf32ecc..4bf98e96343d 100644
> --- a/arch/x86/kvm/mmu/tdp_mmu.c
> +++ b/arch/x86/kvm/mmu/tdp_mmu.c
> @@ -358,6 +358,9 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn,
>
>  	if ((!is_writable_pte(old_spte) || pfn_changed) &&
>  	    is_writable_pte(new_spte)) {
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		update_dirty_quota(kvm, (1L << SPTE_LEVEL_SHIFT(level)));
> +#endif
>  		slot = __gfn_to_memslot(__kvm_memslots(kvm, as_id), gfn);
>  		mark_page_dirty_in_slot(kvm, slot, gfn);
>  	}
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index bcac3efcde41..da4c6342a647 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -5861,6 +5861,11 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)
>  		 */
>  		if (__xfer_to_guest_mode_work_pending())
>  			return 1;
> +
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		if (kvm_test_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu))
> +			return 1;
> +#endif
>  	}
>
>  	return 1;
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 7713420abab0..1733be829197 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -3092,6 +3092,9 @@ static void kvm_setup_guest_pvclock(struct kvm_vcpu *v,
>
>  	guest_hv_clock->version = ++vcpu->hv_clock.version;
>
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	update_dirty_quota(v->kvm, PAGE_SIZE);
> +#endif
>  	mark_page_dirty_in_slot(v->kvm, gpc->memslot, gpc->gpa >> PAGE_SHIFT);
>  	read_unlock_irqrestore(&gpc->lock, flags);
>
> @@ -3566,6 +3569,9 @@ static void record_steal_time(struct kvm_vcpu *vcpu)
>   out:
>  	user_access_end();
>   dirty:
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	update_dirty_quota(vcpu->kvm, PAGE_SIZE);
> +#endif
>  	mark_page_dirty_in_slot(vcpu->kvm, ghc->memslot, gpa_to_gfn(ghc->gpa));
>  }
>
> @@ -4815,6 +4821,9 @@ static void kvm_steal_time_set_preempted(struct kvm_vcpu *vcpu)
>  	if (!copy_to_user_nofault(&st->preempted, &preempted, sizeof(preempted)))
>  		vcpu->arch.st.preempted = KVM_VCPU_PREEMPTED;
>
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	update_dirty_quota(vcpu->kvm, PAGE_SIZE);
> +#endif
>  	mark_page_dirty_in_slot(vcpu->kvm, ghc->memslot, gpa_to_gfn(ghc->gpa));
>  }
>
> @@ -10514,6 +10523,13 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>  			r = 0;
>  			goto out;
>  		}
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) {
> +			vcpu->run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED;
> +			r = 0;
> +			goto out;
> +		}
> +#endif
>
>  		/*
>  		 * KVM_REQ_HV_STIMER has to be processed after
> diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c
> index 40edf4d1974c..00a3ac438539 100644
> --- a/arch/x86/kvm/xen.c
> +++ b/arch/x86/kvm/xen.c
> @@ -435,9 +435,16 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic)
>
>  	read_unlock_irqrestore(&gpc1->lock, flags);
>
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	update_dirty_quota(v->kvm, PAGE_SIZE);
> +#endif
>  	mark_page_dirty_in_slot(v->kvm, gpc1->memslot, gpc1->gpa >> PAGE_SHIFT);
> -	if (user_len2)
> +	if (user_len2) {
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +		update_dirty_quota(v->kvm, PAGE_SIZE);
> +#endif
>  		mark_page_dirty_in_slot(v->kvm, gpc2->memslot, gpc2->gpa >> PAGE_SHIFT);
> +	}
>  }
>
>  void kvm_xen_update_runstate(struct kvm_vcpu *v, int state)
> @@ -549,6 +556,9 @@ void kvm_xen_inject_pending_events(struct kvm_vcpu *v)
>  	if (v->arch.xen.upcall_vector)
>  		kvm_xen_inject_vcpu_vector(v);
>
> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> +	update_dirty_quota(v->kvm, PAGE_SIZE);
> +#endif
>  	mark_page_dirty_in_slot(v->kvm, gpc->memslot, gpc->gpa >> PAGE_SHIFT);
>  }
>
> --
> 2.22.3
>

Re: [PATCH v8 1/3] KVM: Implement dirty quota-based throttling of vcpus

[Shivam Kumar]

On 27/02/23 7:19 am, Marc Zyngier wrote:
> On Sat, 25 Feb 2023 20:47:57 +0000,
> Shivam Kumar <shivam.kumar1@nutanix.com> wrote:
>>
>> Define dirty_quota_bytes variable to track and throttle memory
>> dirtying for every vcpu. This variable stores the number of bytes the
>> vcpu is allowed to dirty. To dirty more, the vcpu needs to request
>> more quota by exiting to userspace.
>>
>> Implement update_dirty_quota function which
>>
>> i) Decreases dirty_quota_bytes by arch-specific page size whenever a
>> page is dirtied.
>> ii) Raises a KVM request KVM_REQ_DIRTY_QUOTA_EXIT whenever the dirty
>> quota is exhausted (i.e. dirty_quota_bytes <= 0).
>>
>> Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
>> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
>> Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
>> Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
>> Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
>> ---
>>   Documentation/virt/kvm/api.rst | 17 +++++++++++++++++
>>   include/linux/kvm_host.h       |  5 +++++
>>   include/uapi/linux/kvm.h       |  8 ++++++++
>>   tools/include/uapi/linux/kvm.h |  1 +
>>   virt/kvm/Kconfig               |  3 +++
>>   virt/kvm/kvm_main.c            | 31 +++++++++++++++++++++++++++++++
>>   6 files changed, 65 insertions(+)
>>
>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
>> index 62de0768d6aa..3a283fe212d8 100644
>> --- a/Documentation/virt/kvm/api.rst
>> +++ b/Documentation/virt/kvm/api.rst
>> @@ -6688,6 +6688,23 @@ Please note that the kernel is allowed to use the kvm_run structure as the
>>   primary storage for certain register types. Therefore, the kernel may use the
>>   values in kvm_run even if the corresponding bit in kvm_dirty_regs is not set.
>>   
>> +::
>> +
>> +	/*
>> +	 * Number of bytes the vCPU is allowed to dirty if KVM_CAP_DIRTY_QUOTA is
>> +	 * enabled. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if this quota
>> +	 * is exhausted, i.e. dirty_quota_bytes <= 0.
>> +	 */
>> +	long dirty_quota_bytes;
>> +
>> +Please note that enforcing the quota is best effort. Dirty quota is reduced by
>> +arch-specific page size when any guest page is dirtied. Also, the guest may dirty
>> +multiple pages before KVM can recheck the quota.
> 
> What are the events that trigger such quota reduction?

If PML is enabled or when functions like nested_mark_vmcs12_pages_dirty 
get called that can mark multiple pages dirtied in a single exit.

Thanks.

> 
>> +
>> +::
>> +  };
>> +
>> +
>>   
>>   6. Capabilities that can be enabled on vCPUs
>>   ============================================
>> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
>> index 8ada23756b0e..f5ce343c64f2 100644
>> --- a/include/linux/kvm_host.h
>> +++ b/include/linux/kvm_host.h
>> @@ -167,6 +167,7 @@ static inline bool is_error_page(struct page *page)
>>   #define KVM_REQ_VM_DEAD			(1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
>>   #define KVM_REQ_UNBLOCK			2
>>   #define KVM_REQ_DIRTY_RING_SOFT_FULL	3
>> +#define KVM_REQ_DIRTY_QUOTA_EXIT	4
>>   #define KVM_REQUEST_ARCH_BASE		8
>>   
>>   /*
>> @@ -800,6 +801,9 @@ struct kvm {
>>   	bool dirty_ring_with_bitmap;
>>   	bool vm_bugged;
>>   	bool vm_dead;
>> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
>> +	bool dirty_quota_enabled;
>> +#endif
>>   
>>   #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER
>>   	struct notifier_block pm_notifier;
>> @@ -1235,6 +1239,7 @@ struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn);
>>   bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn);
>>   bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn);
>>   unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn);
>> +void update_dirty_quota(struct kvm *kvm, unsigned long page_size_bytes);
>>   void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn);
>>   void mark_page_dirty(struct kvm *kvm, gfn_t gfn);
>>   
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index d77aef872a0a..ddb9d3d797c4 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -264,6 +264,7 @@ struct kvm_xen_exit {
>>   #define KVM_EXIT_RISCV_SBI        35
>>   #define KVM_EXIT_RISCV_CSR        36
>>   #define KVM_EXIT_NOTIFY           37
>> +#define KVM_EXIT_DIRTY_QUOTA_EXHAUSTED 38
>>   
>>   /* For KVM_EXIT_INTERNAL_ERROR */
>>   /* Emulate instruction failed. */
>> @@ -526,6 +527,12 @@ struct kvm_run {
>>   		struct kvm_sync_regs regs;
>>   		char padding[SYNC_REGS_SIZE_BYTES];
>>   	} s;
>> +	/*
>> +	 * Number of bytes the vCPU is allowed to dirty if KVM_CAP_DIRTY_QUOTA is
>> +	 * enabled. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if this quota
>> +	 * is exhausted, i.e. dirty_quota_bytes <= 0.
>> +	 */
>> +	long dirty_quota_bytes;
>>   };
>>   
>>   /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */
>> @@ -1184,6 +1191,7 @@ struct kvm_ppc_resize_hpt {
>>   #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
>>   #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
>>   #define KVM_CAP_PMU_EVENT_MASKED_EVENTS 226
>> +#define KVM_CAP_DIRTY_QUOTA 227
>>   
>>   #ifdef KVM_CAP_IRQ_ROUTING
>>   
>> diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
>> index 55155e262646..48f236e2b836 100644
>> --- a/tools/include/uapi/linux/kvm.h
>> +++ b/tools/include/uapi/linux/kvm.h
>> @@ -1175,6 +1175,7 @@ struct kvm_ppc_resize_hpt {
>>   #define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223
>>   #define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224
>>   #define KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP 225
>> +#define KVM_CAP_DIRTY_QUOTA 227
>>   
>>   #ifdef KVM_CAP_IRQ_ROUTING
>>   
>> diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
>> index b74916de5183..ccaa332d88f9 100644
>> --- a/virt/kvm/Kconfig
>> +++ b/virt/kvm/Kconfig
>> @@ -19,6 +19,9 @@ config HAVE_KVM_IRQ_ROUTING
>>   config HAVE_KVM_DIRTY_RING
>>          bool
>>   
>> +config HAVE_KVM_DIRTY_QUOTA
>> +       bool
>> +
>>   # Only strongly ordered architectures can select this, as it doesn't
>>   # put any explicit constraint on userspace ordering. They can also
>>   # select the _ACQ_REL version.
>> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
>> index d255964ec331..744b955514ce 100644
>> --- a/virt/kvm/kvm_main.c
>> +++ b/virt/kvm/kvm_main.c
>> @@ -3096,6 +3096,9 @@ static int __kvm_write_guest_page(struct kvm *kvm,
>>   	r = __copy_to_user((void __user *)addr + offset, data, len);
>>   	if (r)
>>   		return -EFAULT;
>> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
>> +	update_dirty_quota(kvm, PAGE_SIZE);
>> +#endif
> 
> Why PAGE_SIZE? Why not 'len'? Why if the page was already dirtied? Why
> should it be accounted for multiple times? In most cases, this is the
> *hypervisor* writing to the guest, not the vcpu. Why should this be
> accounted to the vcpu quota?

Agreed, update doesn't make much sense here. Thanks.

Thanks,
Shivam

Re: [PATCH v8 3/3] KVM: arm64: Dirty quota-based throttling of vcpus

[Shivam Kumar]

On 27/02/23 7:19 am, Marc Zyngier wrote:
> On Sat, 25 Feb 2023 20:48:01 +0000,
> Shivam Kumar <shivam.kumar1@nutanix.com> wrote:
>>
>> Call update_dirty_quota whenever a page is marked dirty with
>> appropriate arch-specific page size. Process the KVM request
>> KVM_REQ_DIRTY_QUOTA_EXIT (raised by update_dirty_quota) to exit to
>> userspace with exit reason KVM_EXIT_DIRTY_QUOTA_EXHAUSTED.
>>
>> Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
>> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
>> Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
>> Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
>> Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
>> ---
>>   arch/arm64/kvm/Kconfig | 1 +
>>   arch/arm64/kvm/arm.c   | 7 +++++++
>>   arch/arm64/kvm/mmu.c   | 3 +++
>>   3 files changed, 11 insertions(+)
>>
>> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
>> index ca6eadeb7d1a..8e7dea2c3a9f 100644
>> --- a/arch/arm64/kvm/Kconfig
>> +++ b/arch/arm64/kvm/Kconfig
>> @@ -44,6 +44,7 @@ menuconfig KVM
>>   	select SCHED_INFO
>>   	select GUEST_PERF_EVENTS if PERF_EVENTS
>>   	select INTERVAL_TREE
>> +	select HAVE_KVM_DIRTY_QUOTA
> 
> So this is selected unconditionally...
> 
>>   	help
>>   	  Support hosting virtualized guest machines.
>>   
>> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
>> index 3bd732eaf087..5162b2fc46a1 100644
>> --- a/arch/arm64/kvm/arm.c
>> +++ b/arch/arm64/kvm/arm.c
>> @@ -757,6 +757,13 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
>>   
>>   		if (kvm_dirty_ring_check_request(vcpu))
>>   			return 0;
>> +
>> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
> 
> ... and yet you litter the arch code with #ifdefs...

Sorry about that. #ifdefs are not required here.

> 
>> +		if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) {
>> +			vcpu->run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED;
>> +			return 0;
> 
> What rechecks the quota on entry?

Right now, we are not rechecking the quota after entry. So, if the 
userspace doesn't update the quota, then we let the vcpu run until it 
tries to dirty again.

I think it's a good idea to check the quota on entry and keep exiting to 
userspace until the quota is a positive value. Can add this in the next 
patchset.

Thanks.

> 
>> +		}
>> +#endif
>>   	}
>>   
>>   	return 1;
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 7113587222ff..baf416046f46 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -1390,6 +1390,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>>   	/* Mark the page dirty only if the fault is handled successfully */
>>   	if (writable && !ret) {
>>   		kvm_set_pfn_dirty(pfn);
>> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
>> +		update_dirty_quota(kvm, fault_granule);
> 
> fault_granule isn't necessarily the amount that gets dirtied.
> 
> 	M.
> 

For most of the paths where we are updating the quota, we cannot track 
(or precisely account for) dirtying at a granularity less than the 
minimum page size. Looking forward to your thoughts on what we can do 
better here. Thanks.


Thanks,
Shivam

Re: [PATCH v8 2/3] KVM: x86: Dirty quota-based throttling of vcpus

[Shivam Kumar]

On 28/02/23 7:01 am, Yuan Yao wrote:
> On Sat, Feb 25, 2023 at 08:47:59PM +0000, Shivam Kumar wrote:
>> Call update_dirty_quota whenever a page is marked dirty with
>> appropriate arch-specific page size. Process the KVM request
>> KVM_REQ_DIRTY_QUOTA_EXIT (raised by update_dirty_quota) to exit to
>> userspace with exit reason KVM_EXIT_DIRTY_QUOTA_EXHAUSTED.
>>
>> Suggested-by: Shaju Abraham <shaju.abraham@nutanix.com>
>> Suggested-by: Manish Mishra <manish.mishra@nutanix.com>
>> Co-developed-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
>> Signed-off-by: Anurag Madnawat <anurag.madnawat@nutanix.com>
>> Signed-off-by: Shivam Kumar <shivam.kumar1@nutanix.com>
>> ---
>>   arch/x86/kvm/Kconfig       |  1 +
>>   arch/x86/kvm/mmu/mmu.c     |  8 +++++++-
>>   arch/x86/kvm/mmu/spte.c    |  3 +++
>>   arch/x86/kvm/mmu/tdp_mmu.c |  3 +++
>>   arch/x86/kvm/vmx/vmx.c     |  5 +++++
>>   arch/x86/kvm/x86.c         | 16 ++++++++++++++++
>>   arch/x86/kvm/xen.c         | 12 +++++++++++-
>>   7 files changed, 46 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
>> index 8e578311ca9d..8621a9512572 100644
>> --- a/arch/x86/kvm/Kconfig
>> +++ b/arch/x86/kvm/Kconfig
>> @@ -48,6 +48,7 @@ config KVM
>>   	select KVM_VFIO
>>   	select SRCU
>>   	select INTERVAL_TREE
>> +	select HAVE_KVM_DIRTY_QUOTA
>>   	select HAVE_KVM_PM_NOTIFIER if PM
>>   	select KVM_GENERIC_HARDWARE_ENABLING
>>   	help
>> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
>> index c8ebe542c565..e0c8348ecdf1 100644
>> --- a/arch/x86/kvm/mmu/mmu.c
>> +++ b/arch/x86/kvm/mmu/mmu.c
>> @@ -3323,8 +3323,14 @@ fast_pf_fix_direct_spte(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
>>   	if (!try_cmpxchg64(sptep, &old_spte, new_spte))
>>   		return false;
>>
>> -	if (is_writable_pte(new_spte) && !is_writable_pte(old_spte))
>> +	if (is_writable_pte(new_spte) && !is_writable_pte(old_spte)) {
>> +#ifdef CONFIG_HAVE_KVM_DIRTY_QUOTA
>> +		struct kvm_mmu_page *sp = sptep_to_sp(sptep);
>> +
>> +		update_dirty_quota(vcpu->kvm, (1L << SPTE_LEVEL_SHIFT(sp->role.level)));
>> +#endif
>>   		mark_page_dirty_in_slot(vcpu->kvm, fault->slot, fault->gfn);
> 
> Possible to call update_dirty_quota() from mark_page_dirty_in_slot() ?
> Then other Architectures can be covered yet.

As Marc commented on the first patch of this patchset, 
mark_page_dirty_in_slot can be called multiple times for the same page, 
e.g. in the case of PML for nested guests. If bitmap-based dirty 
tracking is not enabled, we might not be able to handle those cases 
without adding an extra param (which can tell us whether a dirty quota 
update is required or not) in mark_page_dirty_in_slot. Thanks.

Thanks,
Shivam