Re: [PATCH v4 1/4] KVM: Implement dirty quota-based throttling of vcpus

[Sean Christopherson <seanjc@google.com>]

On Thu, May 26, 2022, Marc Zyngier wrote:
> > >> +{
> > >> +	struct kvm_run *run = vcpu->run;
> > >> +	u64 dirty_quota = READ_ONCE(run->dirty_quota);
> > >> +	u64 pages_dirtied = vcpu->stat.generic.pages_dirtied;
> > >> +
> > >> +	if (!dirty_quota || (pages_dirtied < dirty_quota))
> > >> +		return 1;
> > > What happens when page_dirtied becomes large and dirty_quota has to
> > > wrap to allow further progress?
> > Every time the quota is exhausted, userspace is expected to set it to
> > pages_dirtied + new quota. So, pages_dirtied will always follow dirty
> > quota. I'll be sending the qemu patches soon. Thanks.
> 
> Right, so let's assume that page_dirtied=0xffffffffffffffff (yes, I
> have dirtied that many pages).

Really?  Written that many bytes from a guest?  Maybe.  But actually marked that
many pages dirty in hardware, let alone in KVM?  And on a single CPU?

By my back of the napkin math, a 4096 CPU system running at 16ghz with each CPU
able to access one page of memory per cycle would take ~3 days to access 2^64
pages.

Assuming a ridiculously optimistic ~20 cycles to walk page tables, fetch the cache
line from memory, insert into the TLB, and mark the PTE dirty, that's still ~60
days to actually dirty that many pages in hardware.

Let's again be comically optimistic and assume KVM can somehow propagate a dirty
bit from hardware PTEs to the dirty bitmap/ring in another ~20 cycles.  That brings
us to ~1200 days.

But the stat is per vCPU, so that actually means it would take ~13.8k years for a
single vCPU/CPU to dirty 2^64 pages... running at a ludicrous 16ghz on a CPU with
latencies that are a likely an order of magnitude faster than anything that exists
today.