[PATCH v4 0/3] KVM: x86: Update HWCR virtualization

[PATCH v4 0/3] KVM: x86: Update HWCR virtualization

[Jim Mattson]

Allow HWCR.McStatusWrEn to be cleared once set, and allow
HWCR.TscFreqSel to be set as well.

v1 -> v2: KVM no longer sets HWCR.TscFreqSel
          HWCR.TscFreqSel can be cleared from userspace
          Selftest modified accordingly
v2 -> v3: kvm_set_msr_common() changes simplified
v3 -> v4: kvm_set_msr_common() changes further simplified
          HWCR.TscFreqSel can be modified from the guest
	  Targets reordered in selftest Makefile

Jim Mattson (3):
  KVM: x86: Allow HWCR.McStatusWrEn to be cleared once set
  KVM: x86: Virtualize HWCR.TscFreqSel[bit 24]
  KVM: selftests: Test behavior of HWCR

 arch/x86/kvm/x86.c                            | 11 ++--
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../selftests/kvm/x86_64/hwcr_msr_test.c      | 52 +++++++++++++++++++
 3 files changed, 60 insertions(+), 4 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c

-- 
2.42.0.582.g8ccd20d70d-goog

[PATCH v4 1/3] KVM: x86: Allow HWCR.McStatusWrEn to be cleared once set

[Jim Mattson]

When HWCR is set to 0, store 0 in vcpu->arch.msr_hwcr.

Fixes: 191c8137a939 ("x86/kvm: Implement HWCR support")
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/x86.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 9f18b06bbda6..1a323cae219c 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3701,12 +3701,11 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		data &= ~(u64)0x8;	/* ignore TLB cache disable */
 
 		/* Handle McStatusWrEn */
-		if (data == BIT_ULL(18)) {
-			vcpu->arch.msr_hwcr = data;
-		} else if (data != 0) {
+		if (data & ~BIT_ULL(18)) {
 			kvm_pr_unimpl_wrmsr(vcpu, msr, data);
 			return 1;
 		}
+		vcpu->arch.msr_hwcr = data;
 		break;
 	case MSR_FAM10H_MMIO_CONF_BASE:
 		if (data != 0) {
-- 
2.42.0.582.g8ccd20d70d-goog

[PATCH v4 2/3] KVM: x86: Virtualize HWCR.TscFreqSel[bit 24]

[Jim Mattson]

On certain CPUs, Linux guests expect HWCR.TscFreqSel[bit 24] to be
set. If it isn't set, they complain:
	[Firmware Bug]: TSC doesn't count with P0 frequency!

Allow userspace to set this bit in the virtual HWCR to eliminate the
above complaint.

Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/x86.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1a323cae219c..86a1bb0e6227 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3700,8 +3700,12 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		data &= ~(u64)0x100;	/* ignore ignne emulation enable */
 		data &= ~(u64)0x8;	/* ignore TLB cache disable */
 
-		/* Handle McStatusWrEn */
-		if (data & ~BIT_ULL(18)) {
+		/*
+		 * Allow McStatusWrEn and TscFreqSel. (Linux guests from v3.2
+		 * through at least v6.6 whine if TscFreqSel is clear,
+		 * depending on F/M/S.
+		 */
+		if (data & ~(BIT_ULL(18) | BIT_ULL(24))) {
 			kvm_pr_unimpl_wrmsr(vcpu, msr, data);
 			return 1;
 		}
-- 
2.42.0.582.g8ccd20d70d-goog

[PATCH v4 3/3] KVM: selftests: Test behavior of HWCR

[Jim Mattson]

Verify the following:
* Attempts to set bits 3, 6, or 8 are ignored
* Bits 18 and 24 are the only bits that can be set
* Any bit that can be set can also be cleared

Signed-off-by: Jim Mattson <jmattson@google.com>
---
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../selftests/kvm/x86_64/hwcr_msr_test.c      | 52 +++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c

diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index a3bb36fb3cfc..3b82c583c68d 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -119,6 +119,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/amx_test
 TEST_GEN_PROGS_x86_64 += x86_64/max_vcpuid_cap_test
 TEST_GEN_PROGS_x86_64 += x86_64/triple_fault_event_test
 TEST_GEN_PROGS_x86_64 += x86_64/recalc_apic_map_test
+TEST_GEN_PROGS_x86_64 += x86_64/hwcr_msr_test
 TEST_GEN_PROGS_x86_64 += access_tracking_perf_test
 TEST_GEN_PROGS_x86_64 += demand_paging_test
 TEST_GEN_PROGS_x86_64 += dirty_log_test
diff --git a/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c b/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c
new file mode 100644
index 000000000000..1a6a09791ac3
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c
@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023, Google LLC.
+ *
+ * Tests for the K7_HWCR MSR.
+ */
+
+#define _GNU_SOURCE /* for program_invocation_short_name */
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "vmx.h"
+
+void test_hwcr_bit(struct kvm_vcpu *vcpu, unsigned int bit)
+{
+	const unsigned long long ignored = BIT_ULL(3) | BIT_ULL(6) | BIT_ULL(8);
+	const unsigned long long valid = BIT_ULL(18) | BIT_ULL(24);
+	const unsigned long long legal = ignored | valid;
+	uint64_t val = BIT_ULL(bit);
+	uint64_t check;
+	int r;
+
+	r = _vcpu_set_msr(vcpu, MSR_K7_HWCR, val);
+	TEST_ASSERT((r == 1 && (val & legal)) || (r == 0 && !(val & legal)),
+		    "Unexpected result (%d) when setting HWCR[bit %u]", r, bit);
+	check =	vcpu_get_msr(vcpu, MSR_K7_HWCR);
+	if (val & valid) {
+		TEST_ASSERT(check == val,
+			    "Bit %u: unexpected HWCR %lx; expected %lx", bit,
+			    check, val);
+		vcpu_set_msr(vcpu, MSR_K7_HWCR, 0);
+	} else {
+		TEST_ASSERT(!check,
+			    "Bit %u: unexpected HWCR %lx; expected 0", bit,
+			    check);
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	struct kvm_vm *vm;
+	struct kvm_vcpu *vcpu;
+	unsigned int bit;
+
+	vm = vm_create_with_one_vcpu(&vcpu, NULL);
+
+	for (bit = 0; bit < BITS_PER_LONG; bit++)
+		test_hwcr_bit(vcpu, bit);
+
+	kvm_vm_free(vm);
+}
-- 
2.42.0.582.g8ccd20d70d-goog

Re: [PATCH v4 3/3] KVM: selftests: Test behavior of HWCR

[Sean Christopherson]

On Fri, Sep 29, 2023, Jim Mattson wrote:
> diff --git a/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c b/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c
> new file mode 100644
> index 000000000000..1a6a09791ac3
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c
> @@ -0,0 +1,52 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023, Google LLC.
> + *
> + * Tests for the K7_HWCR MSR.

My vote is to drop this.  I have a hard time believing anyone ever gets value
out of these, and they have a bad habit of becoming stale.

> + */
> +
> +#define _GNU_SOURCE /* for program_invocation_short_name */
> +#include <sys/ioctl.h>
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "vmx.h"
> +
> +void test_hwcr_bit(struct kvm_vcpu *vcpu, unsigned int bit)
> +{
> +	const unsigned long long ignored = BIT_ULL(3) | BIT_ULL(6) | BIT_ULL(8);

uint64_t instead of "unsigned long long"

> +	const unsigned long long valid = BIT_ULL(18) | BIT_ULL(24);
> +	const unsigned long long legal = ignored | valid;
> +	uint64_t val = BIT_ULL(bit);
> +	uint64_t check;
> +	int r;
> +
> +	r = _vcpu_set_msr(vcpu, MSR_K7_HWCR, val);
> +	TEST_ASSERT((r == 1 && (val & legal)) || (r == 0 && !(val & legal)),

Hrm.  The "val & legal" is weird.  It works, but only because "val" is guaranteed
to be a single bit.  I much prefer to check for illegal bits, not if there is at
least one legal bit.  It's kinda ugly, but IMO using a ternary operator is more
intuitive than checking the same thing twice.

> +		    "Unexpected result (%d) when setting HWCR[bit %u]", r, bit);

And rather than just say "Unexpected result", explicitly say "fail" or "succeed",
I always forget that KVM_SET_MSRS has odd return codes.

	r = _vcpu_set_msr(vcpu, MSR_K7_HWCR, val);
	TEST_ASSERT(val & ~legal ? !r : r == 1,
		    "Expected KVM_SET_MSRS(MSR_K7_HWCR) = 0x%lx to %s",
		    val, val & ~legal ? "succeed" : "fail");


> +	check =	vcpu_get_msr(vcpu, MSR_K7_HWCR);

s/check/actual?

> +	if (val & valid) {
> +		TEST_ASSERT(check == val,

There's no reason to force this code to work with single bits.  I don't mind
applying the test with single bit testing, but making it play nice with more
complex values is trivial, and IMO much easier on the eyes.

	TEST_ASSERT(actual == (val & valid),
		    "Bit %u: unexpected HWCR 0x%lx; expected 0x%lx",
		    bit, actual, (val & valid));

> +			    "Bit %u: unexpected HWCR %lx; expected %lx", bit,

0x for the value to avoid decimal vs. hex confusion.

> +			    check, val);
> +		vcpu_set_msr(vcpu, MSR_K7_HWCR, 0);
> +	} else {
> +		TEST_ASSERT(!check,
> +			    "Bit %u: unexpected HWCR %lx; expected 0", bit,
> +			    check);
> +	}
> +}

If you've no objections, I'll apply the changes as below:

---
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../selftests/kvm/x86_64/hwcr_msr_test.c      | 47 +++++++++++++++++++
 2 files changed, 48 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c

diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index a3bb36fb3cfc..3b82c583c68d 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -119,6 +119,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/amx_test
 TEST_GEN_PROGS_x86_64 += x86_64/max_vcpuid_cap_test
 TEST_GEN_PROGS_x86_64 += x86_64/triple_fault_event_test
 TEST_GEN_PROGS_x86_64 += x86_64/recalc_apic_map_test
+TEST_GEN_PROGS_x86_64 += x86_64/hwcr_msr_test
 TEST_GEN_PROGS_x86_64 += access_tracking_perf_test
 TEST_GEN_PROGS_x86_64 += demand_paging_test
 TEST_GEN_PROGS_x86_64 += dirty_log_test
diff --git a/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c b/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c
new file mode 100644
index 000000000000..df351ae17029
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/hwcr_msr_test.c
@@ -0,0 +1,47 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023, Google LLC.
+ */
+
+#define _GNU_SOURCE /* for program_invocation_short_name */
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "vmx.h"
+
+void test_hwcr_bit(struct kvm_vcpu *vcpu, unsigned int bit)
+{
+	const uint64_t ignored = BIT_ULL(3) | BIT_ULL(6) | BIT_ULL(8);
+	const uint64_t valid = BIT_ULL(18) | BIT_ULL(24);
+	const uint64_t legal = ignored | valid;
+	uint64_t val = BIT_ULL(bit);
+	uint64_t actual;
+	int r;
+
+	r = _vcpu_set_msr(vcpu, MSR_K7_HWCR, val);
+	TEST_ASSERT(val & ~legal ? !r : r == 1,
+		    "Expected KVM_SET_MSRS(MSR_K7_HWCR) = 0x%lx to %s",
+		    val, val & ~legal ? "fail" : "succeed");
+
+	actual = vcpu_get_msr(vcpu, MSR_K7_HWCR);
+	TEST_ASSERT(actual == (val & valid),
+		    "Bit %u: unexpected HWCR 0x%lx; expected 0x%lx",
+		    bit, actual, (val & valid));
+
+	vcpu_set_msr(vcpu, MSR_K7_HWCR, 0);
+}
+
+int main(int argc, char *argv[])
+{
+	struct kvm_vm *vm;
+	struct kvm_vcpu *vcpu;
+	unsigned int bit;
+
+	vm = vm_create_with_one_vcpu(&vcpu, NULL);
+
+	for (bit = 0; bit < BITS_PER_LONG; bit++)
+		test_hwcr_bit(vcpu, bit);
+
+	kvm_vm_free(vm);
+}

base-commit: 9a90e7aad79e519f7e4afd9d98778192c3b44b6e
-- 

Re: [PATCH v4 3/3] KVM: selftests: Test behavior of HWCR

[Jim Mattson]

On Thu, Oct 5, 2023 at 8:44 PM Sean Christopherson <seanjc@google.com> wrote:

> If you've no objections, I'll apply the changes as below:
Go for it.

Re: [PATCH v4 0/3] KVM: x86: Update HWCR virtualization

[Sean Christopherson]

On Fri, 29 Sep 2023 16:02:43 -0700, Jim Mattson wrote:
> Allow HWCR.McStatusWrEn to be cleared once set, and allow
> HWCR.TscFreqSel to be set as well.
> 
> v1 -> v2: KVM no longer sets HWCR.TscFreqSel
>           HWCR.TscFreqSel can be cleared from userspace
>           Selftest modified accordingly
> v2 -> v3: kvm_set_msr_common() changes simplified
> v3 -> v4: kvm_set_msr_common() changes further simplified
>           HWCR.TscFreqSel can be modified from the guest
> 	  Targets reordered in selftest Makefile
> 
> [...]

Applied to kvm-x86 misc, thanks!

[1/3] KVM: x86: Allow HWCR.McStatusWrEn to be cleared once set
      https://github.com/kvm-x86/linux/commit/598a790fc20f
[2/3] KVM: x86: Virtualize HWCR.TscFreqSel[bit 24]
      https://github.com/kvm-x86/linux/commit/8b0e00fba934
[3/3] KVM: selftests: Test behavior of HWCR
      https://github.com/kvm-x86/linux/commit/591455325a79

--
https://github.com/kvm-x86/linux/tree/next