[PATCH v5 00/15] x86/mtrr: fix handling with PAT but without MTRR

[PATCH v5 00/15] x86/mtrr: fix handling with PAT but without MTRR

[Juergen Gross]

This series tries to fix the rather special case of PAT being available
without having MTRRs (either due to CONFIG_MTRR being not set, or
because the feature has been disabled e.g. by a hypervisor).

The main use cases are Xen PV guests and SEV-SNP guests running under
Hyper-V.

Instead of trying to work around all the issues by adding if statements
here and there, just try to use the complete available infrastructure
by setting up a read-only MTRR state when needed.

In the Xen PV case the current MTRR MSR values can be read from the
hypervisor, while for the SEV-SNP case all needed is to set the
default caching mode to "WB".

I have added more cleanup which has been discussed when looking into
the most recent failures.

Note that I couldn't test the Hyper-V related change (patch 3).

Running on bare metal and with Xen didn't show any problems with the
series applied.

It should be noted that patches 9+10 are replacing today's way to
lookup the MTRR cache type for a memory region from looking at the
MTRR register values to building a memory map with the cache types.
This should make the lookup much faster and much easier to understand.

Changes in V2:
- replaced former patches 1+2 with new patches 1-4, avoiding especially
  the rather hacky approach of V1, while making all the MTRR type
  conflict tests available for the Xen PV case
- updated patch 6 (was patch 4 in V1)

Changes in V3:
- dropped patch 5 of V2, as already applied
- split patch 1 of V2 into 2 patches
- new patches 6-10
- addressed comments

Changes in V4:
- addressed comments

Changes in V5
- addressed comments
- some other small fixes
- new patches 3, 8 and 15

clone of "mtrr"

Juergen Gross (15):
  x86/mtrr: split off physical address size calculation
  x86/mtrr: optimize mtrr_calc_physbits()
  x86/mtrr: replace some constants with defines
  x86/mtrr: support setting MTRR state for software defined MTRRs
  x86/hyperv: set MTRR state when running as SEV-SNP Hyper-V guest
  x86/xen: set MTRR state when running as Xen PV initial domain
  x86/mtrr: replace vendor tests in MTRR code
  x86/mtrr: have only one set_mtrr() variant
  x86/mtrr: allocate mtrr_value array dynamically
  x86/mtrr: add get_effective_type() service function
  x86/mtrr: construct a memory map with cache modes
  x86/mtrr: use new cache_map in mtrr_type_lookup()
  x86/mtrr: don't let mtrr_type_lookup() return MTRR_TYPE_INVALID
  x86/mm: only check uniform after calling mtrr_type_lookup()
  x86/mtrr: remove unused code

 arch/x86/include/asm/mtrr.h        |  44 ++-
 arch/x86/include/uapi/asm/mtrr.h   |   6 +-
 arch/x86/kernel/cpu/mshyperv.c     |   4 +
 arch/x86/kernel/cpu/mtrr/amd.c     |   2 +-
 arch/x86/kernel/cpu/mtrr/centaur.c |  11 +-
 arch/x86/kernel/cpu/mtrr/cleanup.c |   6 +-
 arch/x86/kernel/cpu/mtrr/cyrix.c   |   2 +-
 arch/x86/kernel/cpu/mtrr/generic.c | 578 +++++++++++++++++++----------
 arch/x86/kernel/cpu/mtrr/mtrr.c    | 146 ++++----
 arch/x86/kernel/cpu/mtrr/mtrr.h    |   7 +-
 arch/x86/kernel/setup.c            |   2 +
 arch/x86/mm/pgtable.c              |  24 +-
 arch/x86/xen/enlighten_pv.c        |  52 +++
 13 files changed, 573 insertions(+), 311 deletions(-)

-- 
2.35.3

[PATCH v5 05/15] x86/hyperv: set MTRR state when running as SEV-SNP Hyper-V guest

[Juergen Gross]

In order to avoid mappings using the UC- cache attribute, set the
MTRR state to use WB caching as the default.

This is needed in order to cope with the fact that PAT is enabled,
while MTRRs are not supported by the hypervisor.

Fixes: 90b926e68f50 ("x86/pat: Fix pat_x_mtrr_type() for MTRR disabled case")
Signed-off-by: Juergen Gross <jgross@suse.com>
Tested-by: Michael Kelley <mikelley@microsoft.com>
---
V2:
- new patch
---
 arch/x86/kernel/cpu/mshyperv.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index f36dc2f796c5..0a6cc3cf8919 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -34,6 +34,7 @@
 #include <clocksource/hyperv_timer.h>
 #include <asm/numa.h>
 #include <asm/coco.h>
+#include <asm/mtrr.h>
 
 /* Is Linux running as the root partition? */
 bool hv_root_partition;
@@ -408,6 +409,9 @@ static void __init ms_hyperv_init_platform(void)
 #ifdef CONFIG_SWIOTLB
 			swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary;
 #endif
+
+			/* Set WB as the default cache mode. */
+			mtrr_overwrite_state(NULL, 0, MTRR_TYPE_WRBACK);
 		}
 		/* Isolation VMs are unenlightened SEV-based VMs, thus this check: */
 		if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) {
-- 
2.35.3

RE: [PATCH v5 05/15] x86/hyperv: set MTRR state when running as SEV-SNP Hyper-V guest

[Michael Kelley (LINUX)]

From: Juergen Gross <jgross@suse.com> Sent: Friday, March 31, 2023 11:37 PM
> 
> In order to avoid mappings using the UC- cache attribute, set the
> MTRR state to use WB caching as the default.
> 
> This is needed in order to cope with the fact that PAT is enabled,
> while MTRRs are not supported by the hypervisor.
> 
> Fixes: 90b926e68f50 ("x86/pat: Fix pat_x_mtrr_type() for MTRR disabled case")
> Signed-off-by: Juergen Gross <jgross@suse.com>
> Tested-by: Michael Kelley <mikelley@microsoft.com>

I've tested the full v5 series in a Hyper-V SEV-SNP guest with vTOM.  All looks
good.  The following output appears in dmesg, which I think matches expectations:

[    0.004000] MTRR map: 0 entries (0 fixed + 0 variable; max 0), built from 0 variable MTRRs
[    0.004000] MTRRs set to read-only
[    0.004000] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT

The page attributes listed in /sys/kernel/debug/x86/pat_memtype_list are
"write-back" in the expected cases, and the "mtrr" x86 feature no longer appears
in the "flags" output of "lscpu" or /proc/cpuinfo.  /proc/mtrr does not exist, again
as expected.

> ---
> V2:
> - new patch
> ---
>  arch/x86/kernel/cpu/mshyperv.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index f36dc2f796c5..0a6cc3cf8919 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -34,6 +34,7 @@
>  #include <clocksource/hyperv_timer.h>
>  #include <asm/numa.h>
>  #include <asm/coco.h>
> +#include <asm/mtrr.h>
> 
>  /* Is Linux running as the root partition? */
>  bool hv_root_partition;
> @@ -408,6 +409,9 @@ static void __init ms_hyperv_init_platform(void)
>  #ifdef CONFIG_SWIOTLB
>  			swiotlb_unencrypted_base =
> ms_hyperv.shared_gpa_boundary;
>  #endif
> +
> +			/* Set WB as the default cache mode. */
> +			mtrr_overwrite_state(NULL, 0, MTRR_TYPE_WRBACK);

The placement of this call needs to change due to a patch set I submitted and
that Boris has accepted into the 'tip' tree for the 6.4 merge window (so it's now
in linux-next).  The call should be placed in the function hv_vtom_init() in the
source file arch/x86/hyperv/ivm.c.  Anywhere in hv_vtom_init() is fine -- I would
suggest at the end.

Minor request: In a v6 of this patch set, please include me on all the patches in
the series so I can easily keep track of how it is progressing.

Michael

>  		}
>  		/* Isolation VMs are unenlightened SEV-based VMs, thus this check: */
>  		if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) {
> --
> 2.35.3