From: David Vrabel <david.vrabel@citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Andrew Cooper <andrew.cooper3@citrix.com>,
Kevin Tian <kevin.tian@intel.com>,
Jan Beulich <jbeulich@suse.com>,
David Vrabel <david.vrabel@citrix.com>,
Jun Nakajima <jun.nakajima@intel.com>
Subject: [RFC PATCHv1 0/3]: x86/ept: reduce translation invalidation impact
Date: Fri, 6 Nov 2015 17:37:14 +0000 [thread overview]
Message-ID: <1446831437-5897-1-git-send-email-david.vrabel@citrix.com> (raw)
This RFC series improves the performance of EPT by reducing the impact
of the translation invalidations (ept_sync_domain()). Two approaches
are used:
a) Removing unnecessary invalidations after fixing misconfigured
entries (after a type change).
b) Deferring invalidations until the p2m write lock is released.
Prior to this change a 16 VCPU guest could not be successfully
migrated on an (admittedly slow) 160 PCPU box because the p2m write
lock was held for such extended periods of time. This starved the
read lock needed (by the toolstack) to map the domain's memory,
triggering the watchdog.
After this change a 64 VCPU guest could be successfully migrated.
ept_sync_domain() is very expensive because:
a) it uses on_selected_cpus() and the IPI cost can be particularly
high for a multi-socket machine.
b) on_selected_cpus() is serialized by its own spin lock.
On this particular box, ept_sync_domain() could take ~3-5 ms.
Simply using a fair rw lock was not sufficient to resolve this (but it
was an improvement) as the cost of the ept_sync_domain calls() was
still delaying the read locks enough for the watchdog to trigger (the
toolstack maps a batch of 1024 GFNs at a time, which means trying to
acquire the p2m read lock 1024 times).
David
next reply other threads:[~2015-11-06 17:37 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-06 17:37 David Vrabel [this message]
2015-11-06 17:37 ` [PATCHv1 1/3] x86/ept: remove unnecessary sync after resolving misconfigured entries David Vrabel
2015-11-06 18:29 ` Andrew Cooper
2015-11-10 12:22 ` Jan Beulich
2015-11-12 16:18 ` David Vrabel
2015-11-12 16:30 ` Jan Beulich
2015-11-06 17:37 ` [PATCHv1 2/3] mm: don't free pages until mm locks are released David Vrabel
2015-11-06 17:37 ` [PATCHv1 3/3] x86/ept: defer the invalidation until the p2m lock is released David Vrabel
2015-11-06 18:39 ` Andrew Cooper
2015-11-09 14:13 ` Jan Beulich
2015-11-10 13:35 ` Tim Deegan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1446831437-5897-1-git-send-email-david.vrabel@citrix.com \
--to=david.vrabel@citrix.com \
--cc=andrew.cooper3@citrix.com \
--cc=jbeulich@suse.com \
--cc=jun.nakajima@intel.com \
--cc=kevin.tian@intel.com \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.