Re: [RFC] need to improve slot creation/destruction? -- Re: [RFC][PATCH] srcu: Implement call_srcu()

[Marcelo Tosatti <mtosatti@redhat.com>]

On Thu, Feb 09, 2012 at 12:43:20AM +0900, Takuya Yoshikawa wrote:
> [Dropped non-kvm members from cc]
> 
> Marcelo Tosatti <mtosatti@redhat.com> wrote:
> 
> > VGABIOS mode constantly destroys and creates 0xa0000 slot, so
> > performance is required for KVM_SET_MEM too (it can probably be fixed in
> > qemu, but older qemu's must be supported).
> 
> Apart from srcu, I see some problems concerning slot creation/destruction:
> heavy shadow flush (and extra write protection).
> 
> 
> Look at __kvm_set_memory_region().
> 
> 1. When we invalidate a memory slot, we call kvm_arch_flush_shadow() and
> zap all shadow pages, not restricted to that slot.
> 
> 	/* From this point no new shadow pages pointing to a deleted
> 	 * memslot will be created.
> 	 *
> 	 * validation of sp->gfn happens in:
> 	 *	- gfn_to_hva (kvm_read_guest, gfn_to_pfn)
> 	 *	- kvm_is_visible_gfn (mmu_check_roots)
> 	 */
> 	kvm_arch_flush_shadow(kvm);
> 
> 2. When we create(and shift?) a memory slot, we call kvm_arch_flush_shadow()
> to clear all mmio sptes, again not restricted to that slot.
> 
> 	/*
> 	 * If the new memory slot is created, we need to clear all
> 	 * mmio sptes.
> 	 */
> 	if (npages && old.base_gfn != mem->guest_phys_addr >> PAGE_SHIFT)
> 		kvm_arch_flush_shadow(kvm);
> 
> 3. In addition to these, we do write-protect all pages in that slot in
> kvm_arch_commit_memory_region() every time.
> 
> 
> For 1:  I made a patch to restrict the flush to that slot by using sp->slot_bitmap.
> (seems working here)
> 
> For 2:  I think we can do the same:  not 100% sure yet because I am still
> struggling with the "mmio spte optimization" code.  (really hacky ...)

Yes, you can. The flush is there because a new slot invalidates any mmio
sptes in that region, but in the region covered by the new slot only.

> For 3:  I think doing both "write protection" and "shadow flush" is unnecessary.

If you enable dirty logging on a slot, certainly you have to write
protect?

> BTW do we really need fast slot creation/destruction?

At the moment yes. Boot a RHEL/Fedora installation disk (or any other
guest which uses SYSLINUX splash screen) and you will see. That
particular case is a limitation of cirrus in QEMU, ideally it should be
optimized there.