Re: x86 MMU: RMap Interface

[Sean Christopherson <sean.j.christopherson@intel.com>]

On Sun, Jul 19, 2020 at 06:32:22PM -0400, contact@kevinloughlin.org wrote:
> Hi,
> 
> I'm a bit confused by the interface for interacting with the page rmap. For
> context, on a TDP-enabled x86-64 host, I'm logging each time a GFN->PFN
> mapping is created/modified/removed for a non-MMIO page (kernel version
> 5.4).
> 
> First, my understanding is that the page rmap is a mapping of non-MMIO PFNs
> back to the GFNs that use them. The interface for creating an rmap entry
> (and thus, a new GFN->PFN mapping) appears to be rmap_add() and is quite
> straightforward. However, rmap_remove() does not appear to be the (only)
> function for removing an entry from the page rmap. For instance,
> kvm_zap_rmapp()---used by the mmu_notifier for invalidations---jumps
> straight to pte_list_remove(), while drop_spte() uses rmap_remove().

The rmaps are associated with the memslot, the drop_spte() path allows KVM
to clean up SPTEs without having to guarantee the validity of the memslot
that was used to create the SPTE.

> Would it be fair to say that mmu_spte_clear_track_bits() is found on all
> paths for removing an entry from the page rmap?

Yes, that should hold true.
 
> Second, for updates to the frame numbers in an existing SPTE, there are both
> mmu_set_spte() and mmu_spte_set(). Could someone please clarify the
> difference between these functions?

mmu_set_spte() is the higher level helper that is used during a page fault
or prefetch to convert a host PFN and basic access permissions into a SPTE
value, handle large/huge page interactions and accounting, add the rmap,
etc..., and of course eventually update the SPTE.

mmu_spte_set() is a low level helper that does nothing more than write a
SPTE.  It's just a wrapper to __set_spte() that also WARNs if the old SPTE
is present.

> Finally, much of the logic between the page rmap and parent PTE rmaps
> (understandably) overlaps. However, with TDP-enabled, I'm not entirely sure
> what the role of the parent PTE rmaps is relative to the page rmap. Could
> someone possibly clarify?

KVM needs the backpointers to remove the SPTE for a shadow page, which
exists in the parent shadow page, when the child is zapped, e.g. if a L2 SP
is removed, its SPTE in a L3 SP needs to be updated.