Re: [PATCH v4 1/4] KVM: Implement dirty quota-based throttling of vcpus

[Marc Zyngier <maz@kernel.org>]

On Thu, 26 May 2022 16:44:13 +0100,
Sean Christopherson <seanjc@google.com> wrote:
> 
> On Thu, May 26, 2022, Marc Zyngier wrote:
> > > >> +{
> > > >> +	struct kvm_run *run = vcpu->run;
> > > >> +	u64 dirty_quota = READ_ONCE(run->dirty_quota);
> > > >> +	u64 pages_dirtied = vcpu->stat.generic.pages_dirtied;
> > > >> +
> > > >> +	if (!dirty_quota || (pages_dirtied < dirty_quota))
> > > >> +		return 1;
> > > > What happens when page_dirtied becomes large and dirty_quota has to
> > > > wrap to allow further progress?
> > > Every time the quota is exhausted, userspace is expected to set it to
> > > pages_dirtied + new quota. So, pages_dirtied will always follow dirty
> > > quota. I'll be sending the qemu patches soon. Thanks.
> > 
> > Right, so let's assume that page_dirtied=0xffffffffffffffff (yes, I
> > have dirtied that many pages).
> 
> Really?  Written that many bytes from a guest?  Maybe.  But actually
> marked that many pages dirty in hardware, let alone in KVM?  And on
> a single CPU?
>
> By my back of the napkin math, a 4096 CPU system running at 16ghz
> with each CPU able to access one page of memory per cycle would take
> ~3 days to access 2^64 pages.
> 
> Assuming a ridiculously optimistic ~20 cycles to walk page tables,
> fetch the cache line from memory, insert into the TLB, and mark the
> PTE dirty, that's still ~60 days to actually dirty that many pages
> in hardware.
> 
> Let's again be comically optimistic and assume KVM can somehow
> propagate a dirty bit from hardware PTEs to the dirty bitmap/ring in
> another ~20 cycles.  That brings us to ~1200 days.
> 
> But the stat is per vCPU, so that actually means it would take
> ~13.8k years for a single vCPU/CPU to dirty 2^64 pages... running at
> a ludicrous 16ghz on a CPU with latencies that are a likely an order
> of magnitude faster than anything that exists today.

Congratulations, you can multiply! ;-)

It just shows that the proposed API is pretty bad, because instead of
working as a credit, it works as a ceiling, based on a value that is
dependent on the vpcu previous state (forcing userspace to recompute
the next quota on each exit), and with undocumented, arbitrary limits
as a bonus.

I don't like it, and probably won't like it in 13.8k years either.

	M.

-- 
Without deviation from the norm, progress is not possible.