Re: [PATCH 3/6] KVM: Dirty memory tracking for performant checkpointing and improved live migration

["Radim Krčmář" <rkrcmar@redhat.com>]

2016-05-03 18:06+1200, Huang, Kai:
> Actually my concern is, with your new mechanism to track guest dirty pages,
> there will be two logdirty mechanisms (using bitmap and your per-vcpu list),
> which I think is not good as it's a little bit redundant, given both
> mechanisms are used for dirty page logging.
> 
> I think your main concern of current bitmap mechanism is scanning bitmap
> takes lots of time, especially when only few pages get dirty, you still have
> to scan the entire bitmap, which results in bad performance if you runs
> checkpoint very frequently. My suggestion is, instead of introducing two
> logdirty data structures, maybe you can try to use another more efficient
> data structure instead of bitmap for both current logdirty mechanism and
> your new interfaces. Maybe Xen's log-dirty tree is a good reference.

A sparse structure (buffer, tree, ...) also needs a mechanism to grow
(store new entries), so concurrent accesses become a problem, because
there has to be synchronization.  I think that per-vcpu structure
becomes mandatory when thousands VCPUs dirty memory at the same time.

>                      Maybe Xen's log-dirty tree is a good reference.

Is there some top-level overview?

>From a glance at the code, it looked like GPA bitmap sparsified with
radix tree in a manner similar to the page table hierarchy.

> Of course this is just my concern and I'll leave it to maintainers.

I too would prefer if both userspace interfaces used a common backend.
A possible backend for that is

  vcpu -> memslot -> sparse dirty log

We should have dynamic sparse dirty log, to avoid wasting memory when
there are many small memslots, but a linear structure is probably still
fine.

We don't care which vcpu dirtied the page, so it seems like a waste to
have them in the hierarchy, but I can't think of designs where the
sparse dirty log is rooted in memslot and its updates scale well.

'memslot -> sparse dirty log' usually evolve into buffering on the VCPU
side before writing to the memslot or aren't efficient for sparse
dataset.

Where do you think is the balance between 'memslot -> bitmap' and
'vcpu -> memslot -> dirty buffer'?

Thanks.