Re: [PATCH v2] x86/HVM: Avoid cache flush operations during hvm_load

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 02/06/15 13:47, Ross Lagerwall wrote:
> An MTRR record is processed for each vCPU during hvm_load. Each MTRR
> record sets several mtrrs, each of which flushes the cache on all pCPUs.
> This can take some time and trip the watchdog for HVM guests with many
> CPUs.
>
> To fix this, introduce a flag which prevents flushing the cache on x86
> while loading the restore records and instead does a single cache flush
> at the end of hvm_load.
>
> This reduces the time to restore an HVM guest with 32 vCPUs by about 5
> seconds on an Intel Xeon CPU E7-2870.
>
> Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
> ---
>
> In v2: Code moved into arch hooks since it's x86 specific.
>
>  xen/arch/x86/hvm/mtrr.c    |  5 +++++
>  xen/arch/x86/hvm/save.c    | 10 ++++++++++
>  xen/common/hvm/save.c      | 15 ++++++++++-----
>  xen/include/asm-x86/mtrr.h |  9 +++++++++
>  xen/include/xen/hvm/save.h |  1 +
>  5 files changed, 35 insertions(+), 5 deletions(-)
>
> diff --git a/xen/arch/x86/hvm/mtrr.c b/xen/arch/x86/hvm/mtrr.c
> index a69ee62..f21b367 100644
> --- a/xen/arch/x86/hvm/mtrr.c
> +++ b/xen/arch/x86/hvm/mtrr.c
> @@ -65,6 +65,8 @@ static const uint8_t mm_type_tbl[MTRR_NUM_TYPES][PAT_TYPE_NUMS] = {
>  #undef RS
>  };
>  
> +DEFINE_PER_CPU(bool_t, memory_type_changed_ignore);

Thinking about this, this should be "memory_type_changed_defer" and
there should be a second bool "memory_type_changed_wanted".

This way, we don't suffer a memory_type_changed() in
arch_hvm_load_post() if there were no MTRR records loaded; we support
partial loads of state via this path as well.

~Andrew