[PATCH v5 0/2] Enable legacy LBR support for guest

[PATCH v5 0/2] Enable legacy LBR support for guest

[Yang Weijiang]

KVM legacy LBR patches have been merged in kernel 5.12, this patchset
is to expose the feature to guest from the perf capability MSR. Qemu can
add LBR format in cpu option to achieve it, e.g., -cpu host,lbr-fmt=0x5,
the format should match host value in IA32_PERF_CAPABILITIES.

Note, KVM legacy LBR solution accelerates guest perf performace by LBR MSR
passthrough so it requires guest cpu model matches that of host's, i.e.,
only -cpu host is supported.

Change in v5:
	1. This patchset is rebased on tip : 6621441db5
	2. No functional change since v4.


Yang Weijiang (2):
  qdev-properties: Add a new macro with bitmask check for uint64_t
    property
  target/i386: Add lbr-fmt vPMU option to support guest LBR

 hw/core/qdev-properties.c    | 19 +++++++++++++++++
 include/hw/qdev-properties.h | 12 +++++++++++
 target/i386/cpu.c            | 40 ++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h            | 10 +++++++++
 4 files changed, 81 insertions(+)

-- 
2.27.0

[PATCH v5 1/2] qdev-properties: Add a new macro with bitmask check for uint64_t property

[Yang Weijiang]

The DEFINE_PROP_UINT64_CHECKMASK maro applies certain mask check agaist
user-supplied property value, reject the value if it violates the bitmask.

Co-developed-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
---
 hw/core/qdev-properties.c    | 19 +++++++++++++++++++
 include/hw/qdev-properties.h | 12 ++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c
index c34aac6ebc..27566e5ef7 100644
--- a/hw/core/qdev-properties.c
+++ b/hw/core/qdev-properties.c
@@ -428,6 +428,25 @@ const PropertyInfo qdev_prop_int64 = {
     .set_default_value = qdev_propinfo_set_default_value_int,
 };
 
+static void set_uint64_checkmask(Object *obj, Visitor *v, const char *name,
+                      void *opaque, Error **errp)
+{
+    Property *prop = opaque;
+    uint64_t *ptr = object_field_prop_ptr(obj, prop);
+
+    visit_type_uint64(v, name, ptr, errp);
+    if (*ptr & ~prop->bitmask) {
+        error_setg(errp, "Property value for '%s' violates bitmask '0x%lx'",
+                   name, prop->bitmask);
+    }
+}
+
+const PropertyInfo qdev_prop_uint64_checkmask = {
+    .name  = "uint64",
+    .get   = get_uint64,
+    .set   = set_uint64_checkmask,
+};
+
 /* --- string --- */
 
 static void release_string(Object *obj, const char *name, void *opaque)
diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
index f7925f67d0..e1df08876c 100644
--- a/include/hw/qdev-properties.h
+++ b/include/hw/qdev-properties.h
@@ -17,6 +17,7 @@ struct Property {
     const PropertyInfo *info;
     ptrdiff_t    offset;
     uint8_t      bitnr;
+    uint64_t     bitmask;
     bool         set_default;
     union {
         int64_t i;
@@ -54,6 +55,7 @@ extern const PropertyInfo qdev_prop_uint16;
 extern const PropertyInfo qdev_prop_uint32;
 extern const PropertyInfo qdev_prop_int32;
 extern const PropertyInfo qdev_prop_uint64;
+extern const PropertyInfo qdev_prop_uint64_checkmask;
 extern const PropertyInfo qdev_prop_int64;
 extern const PropertyInfo qdev_prop_size;
 extern const PropertyInfo qdev_prop_string;
@@ -103,6 +105,16 @@ extern const PropertyInfo qdev_prop_link;
                 .set_default = true,                         \
                 .defval.u    = (bool)_defval)
 
+/**
+ * The DEFINE_PROP_UINT64_CHECKMASK macro checks a user-supplied value
+ * against corresponding bitmask, rejects the value if it violates.
+ * The default value is set in instance_init().
+ */
+#define DEFINE_PROP_UINT64_CHECKMASK(_name, _state, _field, _bitmask)   \
+    DEFINE_PROP(_name, _state, _field, qdev_prop_uint64_checkmask, uint64_t, \
+                .bitmask    = (_bitmask),                     \
+                .set_default = false)
+
 #define PROP_ARRAY_LEN_PREFIX "len-"
 
 /**
-- 
2.27.0

[PATCH v5 2/2] target/i386: Add lbr-fmt vPMU option to support guest LBR

[Yang Weijiang]

The Last Branch Recording (LBR) is a performance monitor unit (PMU)
feature on Intel processors which records a running trace of the most
recent branches taken by the processor in the LBR stack. This option
indicates the LBR format to enable for guest perf.

The LBR feature is enabled if below conditions are met:
1) KVM is enabled and the PMU is enabled.
2) msr-based-feature IA32_PERF_CAPABILITIES is supporterd on KVM.
3) Supported returned value for lbr_fmt from above msr is non-zero.
4) Guest vcpu model does support FEAT_1_ECX.CPUID_EXT_PDCM.
5) User-provided lbr-fmt value doesn't violate its bitmask (0x3f).
6) Target guest LBR format matches that of host.

Co-developed-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Like Xu <like.xu@linux.intel.com>
Signed-off-by: Yang Weijiang <weijiang.yang@intel.com>
---
 target/i386/cpu.c | 40 ++++++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h | 10 ++++++++++
 2 files changed, 50 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index aa9e636800..55eb519214 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6280,6 +6280,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     CPUX86State *env = &cpu->env;
     Error *local_err = NULL;
     static bool ht_warned;
+    uint64_t requested_lbr_fmt;
 
     if (cpu->apic_id == UNASSIGNED_APIC_ID) {
         error_setg(errp, "apic-id property was not initialized properly");
@@ -6297,6 +6298,42 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         goto out;
     }
 
+    /*
+     * Override env->features[FEAT_PERF_CAPABILITIES].LBR_FMT
+     * with user-provided setting.
+     */
+    if (cpu->lbr_fmt != ~PERF_CAP_LBR_FMT) {
+        if ((cpu->lbr_fmt & PERF_CAP_LBR_FMT) != cpu->lbr_fmt) {
+            error_setg(errp, "invalid lbr-fmt");
+            return;
+        }
+        env->features[FEAT_PERF_CAPABILITIES] &= ~PERF_CAP_LBR_FMT;
+        env->features[FEAT_PERF_CAPABILITIES] |= cpu->lbr_fmt;
+    }
+
+    /*
+     * vPMU LBR is supported when 1) KVM is enabled 2) Option pmu=on and
+     * 3)vPMU LBR format matches that of host setting.
+     */
+    requested_lbr_fmt =
+        env->features[FEAT_PERF_CAPABILITIES] & PERF_CAP_LBR_FMT;
+    if (requested_lbr_fmt && kvm_enabled()) {
+        uint64_t host_perf_cap =
+            x86_cpu_get_supported_feature_word(FEAT_PERF_CAPABILITIES, false);
+        uint64_t host_lbr_fmt = host_perf_cap & PERF_CAP_LBR_FMT;
+
+        if (!cpu->enable_pmu) {
+            error_setg(errp, "vPMU: LBR is unsupported without pmu=on");
+            return;
+        }
+        if (requested_lbr_fmt != host_lbr_fmt) {
+            error_setg(errp, "vPMU: the lbr-fmt value (0x%lx) mismatches "
+                        "the host supported value (0x%lx).",
+                        requested_lbr_fmt, host_lbr_fmt);
+            return;
+        }
+    }
+
     x86_cpu_filter_features(cpu, cpu->check_cpuid || cpu->enforce_cpuid);
 
     if (cpu->enforce_cpuid && x86_cpu_have_filtered_features(cpu)) {
@@ -6649,6 +6686,8 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add_alias(obj, "sse4_2", obj, "sse4.2");
 
     object_property_add_alias(obj, "hv-apicv", obj, "hv-avic");
+    cpu->lbr_fmt = ~PERF_CAP_LBR_FMT;
+    object_property_add_alias(obj, "lbr_fmt", obj, "lbr-fmt");
 
     if (xcc->model) {
         x86_cpu_load_model(cpu, xcc->model);
@@ -6804,6 +6843,7 @@ static Property x86_cpu_properties[] = {
 #endif
     DEFINE_PROP_INT32("node-id", X86CPU, node_id, CPU_UNSET_NUMA_NODE_ID),
     DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
+    DEFINE_PROP_UINT64_CHECKMASK("lbr-fmt", X86CPU, lbr_fmt, PERF_CAP_LBR_FMT),
 
     DEFINE_PROP_UINT32("hv-spinlocks", X86CPU, hyperv_spinlock_attempts,
                        HYPERV_SPINLOCK_NEVER_NOTIFY),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 9911d7c871..ee8974abc9 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -383,6 +383,7 @@ typedef enum X86Seg {
 #define ARCH_CAP_TSX_CTRL_MSR		(1<<7)
 
 #define MSR_IA32_PERF_CAPABILITIES      0x345
+#define PERF_CAP_LBR_FMT                0x3f
 
 #define MSR_IA32_TSX_CTRL		0x122
 #define MSR_IA32_TSCDEADLINE            0x6e0
@@ -1775,6 +1776,15 @@ struct X86CPU {
      */
     bool enable_pmu;
 
+    /*
+     * Enable LBR_FMT bits of IA32_PERF_CAPABILITIES MSR.
+     * This can't be initialized with a default because it doesn't have
+     * stable ABI support yet. It is only allowed to pass all LBR_FMT bits
+     * returned by kvm_arch_get_supported_msr_feature()(which depends on both
+     * host CPU and kernel capabilities) to the guest.
+     */
+    uint64_t lbr_fmt;
+
     /* LMCE support can be enabled/disabled via cpu option 'lmce=on/off'. It is
      * disabled by default to avoid breaking migration between QEMU with
      * different LMCE configurations.
-- 
2.27.0

Re: [PATCH v5 0/2] Enable legacy LBR support for guest

[Like Xu]

Hi Weijiang,

On 23/1/2022 12:11 am, Yang Weijiang wrote:
> KVM legacy LBR patches have been merged in kernel 5.12, this patchset
> is to expose the feature to guest from the perf capability MSR. Qemu can
> add LBR format in cpu option to achieve it, e.g., -cpu host,lbr-fmt=0x5,

Some older Intel CPUs may have lbr-fmt=LBR_FORMAT_32 (which is 0), would
you help verify that KVM is supported on these platforms ? If so, how do we enable
guest LBR form the QEMU side, w/ -cpu host,lbr-fmt=0x0 ?

> the format should match host value in IA32_PERF_CAPABILITIES.
> 
> Note, KVM legacy LBR solution accelerates guest perf performace by LBR MSR
> passthrough so it requires guest cpu model matches that of host's, i.e.,

Would you help add live migration support across host/guest CPU models when
hosts at both ends have the same number of LBR entries and the same lbr-fmt ?

Thanks,
Like Xu

> only -cpu host is supported.
> 
> Change in v5:
> 	1. This patchset is rebased on tip : 6621441db5
> 	2. No functional change since v4.

Re: [PATCH v5 0/2] Enable legacy LBR support for guest

[Yang, Weijiang]

On 2/9/2022 5:14 PM, Like Xu wrote:
> Hi Weijiang,
>
> On 23/1/2022 12:11 am, Yang Weijiang wrote:
>> KVM legacy LBR patches have been merged in kernel 5.12, this patchset
>> is to expose the feature to guest from the perf capability MSR. Qemu can
>> add LBR format in cpu option to achieve it, e.g., -cpu host,lbr-fmt=0x5,
>
> Some older Intel CPUs may have lbr-fmt=LBR_FORMAT_32 (which is 0), would
> you help verify that KVM is supported on these platforms ? If so, how 
> do we enable
> guest LBR form the QEMU side, w/ -cpu host,lbr-fmt=0x0 ?

Hi, Like, do you know which cpu model or platform so that I can have a 
test on?

>
>> the format should match host value in IA32_PERF_CAPABILITIES.
>>
>> Note, KVM legacy LBR solution accelerates guest perf performace by 
>> LBR MSR
>> passthrough so it requires guest cpu model matches that of host's, i.e.,
>
> Would you help add live migration support across host/guest CPU models 
> when
> hosts at both ends have the same number of LBR entries and the same 
> lbr-fmt ?
Yes, I'm working on this part for Arch LBR, then enable it for legacy 
LBR as well.
>
> Thanks,
> Like Xu
>
>> only -cpu host is supported.
>>
>> Change in v5:
>>     1. This patchset is rebased on tip : 6621441db5
>>     2. No functional change since v4.