[PATCH RFC 00/22] i386: KVM: expand Hyper-V features early

[PATCH RFC 00/22] i386: KVM: expand Hyper-V features early

[Vitaly Kuznetsov]

This pairs with https://lore.kernel.org/kvm/20200807083946.377654-1-vkuznets@redhat.com/
and as KVM part is not yet accepted, sending this as RFC.

Upper layer tools like libvirt want to figure out which Hyper-V features are
supported by the underlying stack (QEMU/KVM) but currently they are unable to
do so. We have a nice 'hv_passthrough' CPU flag supported by QEMU but it has
no effect on e.g. QMP's 

query-cpu-model-expansion type=full model={"name":"host","props":{"hv-passthrough":true}}

command as we parse Hyper-V features after creating KVM vCPUs and not at
feature expansion time. To support the use-case we first need to make 
KVM_GET_SUPPORTED_HV_CPUID ioctl a system-wide ioctl as the existing
vCPU version can't be used that early. This is what KVM part does. With
that done, we can make early Hyper-V feature expansion (this series).

Vitaly Kuznetsov (22):
  WIP: update linux/headers
  i386: drop x86_cpu_get_supported_feature_word() forward declaration
  i386: move hyperv_vendor_id initialization to x86_cpu_realizefn()
  i386: move hyperv_interface_id initialization to x86_cpu_realizefn()
  i386: move hyperv_version_id initialization to x86_cpu_realizefn()
  i386: move hyperv_limits initialization to x86_cpu_realizefn()
  i386: fill in FEAT_HYPERV_EDX from edx instead of eax
  i386: invert hyperv_spinlock_attempts setting logic with
    hv_passthrough
  i386: add reserved FEAT_HYPERV_ECX CPUID leaf
  i386: add reserved FEAT_HV_RECOMM_ECX/FEAT_HV_RECOMM_EDX CPUID leaves
  i386: add reserved FEAT_HV_NESTED_EBX/ECX/EDX CPUID leaves
  i386: always fill Hyper-V CPUID feature leaves from X86CPU data
  i386: split hyperv_handle_properties() into
    hyperv_expand_features()/hyperv_fill_cpuids()
  i386: move eVMCS enablement to hyperv_init_vcpu()
  i386: switch hyperv_expand_features() to using error_setg()
  i386: make hyperv_expand_features() return void
  i386: adjust the expected KVM_GET_SUPPORTED_HV_CPUID array size
  i386: prefer system KVM_GET_SUPPORTED_HV_CPUID ioctl over vCPU's one
  i386: prepare hyperv_expand_features() to be called at CPU feature
    expansion time
  i386: use global kvm_state in hyperv_enabled() check
  i386: record if Hyper-V features were already expanded
  i386: expand Hyper-V features early

 include/standard-headers/drm/drm_fourcc.h     | 140 +++++++
 include/standard-headers/linux/ethtool.h      |  87 +++++
 .../linux/input-event-codes.h                 |   3 +-
 linux-headers/asm-generic/unistd.h            |   6 +-
 linux-headers/asm-mips/unistd_n32.h           |   1 +
 linux-headers/asm-mips/unistd_n64.h           |   1 +
 linux-headers/asm-mips/unistd_o32.h           |   1 +
 linux-headers/asm-powerpc/kvm.h               |   5 +
 linux-headers/asm-powerpc/mman.h              |   2 +-
 linux-headers/asm-powerpc/unistd_32.h         |   1 +
 linux-headers/asm-powerpc/unistd_64.h         |   1 +
 linux-headers/asm-s390/kvm.h                  |   7 +-
 linux-headers/asm-s390/unistd_32.h            |   1 +
 linux-headers/asm-s390/unistd_64.h            |   1 +
 linux-headers/asm-x86/unistd_32.h             |   1 +
 linux-headers/asm-x86/unistd_64.h             |   1 +
 linux-headers/asm-x86/unistd_x32.h            |   1 +
 linux-headers/linux/kvm.h                     |   7 +-
 linux-headers/linux/vfio.h                    |   2 +-
 target/i386/cpu.c                             |  78 +++-
 target/i386/cpu.h                             |  13 +-
 target/i386/kvm.c                             | 347 ++++++++++++------
 target/i386/kvm_i386.h                        |   7 +
 23 files changed, 579 insertions(+), 135 deletions(-)

-- 
2.25.4

[PATCH RFC 01/22] WIP: update linux/headers

[Vitaly Kuznetsov]

KVM_CAP_SYS_HYPERV_CPUID definition is needed for this series.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 include/standard-headers/drm/drm_fourcc.h     | 140 ++++++++++++++++++
 include/standard-headers/linux/ethtool.h      |  87 +++++++++++
 .../linux/input-event-codes.h                 |   3 +-
 linux-headers/asm-generic/unistd.h            |   6 +-
 linux-headers/asm-mips/unistd_n32.h           |   1 +
 linux-headers/asm-mips/unistd_n64.h           |   1 +
 linux-headers/asm-mips/unistd_o32.h           |   1 +
 linux-headers/asm-powerpc/kvm.h               |   5 +
 linux-headers/asm-powerpc/mman.h              |   2 +-
 linux-headers/asm-powerpc/unistd_32.h         |   1 +
 linux-headers/asm-powerpc/unistd_64.h         |   1 +
 linux-headers/asm-s390/kvm.h                  |   7 +-
 linux-headers/asm-s390/unistd_32.h            |   1 +
 linux-headers/asm-s390/unistd_64.h            |   1 +
 linux-headers/asm-x86/unistd_32.h             |   1 +
 linux-headers/asm-x86/unistd_64.h             |   1 +
 linux-headers/asm-x86/unistd_x32.h            |   1 +
 linux-headers/linux/kvm.h                     |   7 +-
 linux-headers/linux/vfio.h                    |   2 +-
 19 files changed, 261 insertions(+), 8 deletions(-)

diff --git a/include/standard-headers/drm/drm_fourcc.h b/include/standard-headers/drm/drm_fourcc.h
index 909a66753c03..2757bdf6fa70 100644
--- a/include/standard-headers/drm/drm_fourcc.h
+++ b/include/standard-headers/drm/drm_fourcc.h
@@ -235,6 +235,12 @@ extern "C" {
 #define DRM_FORMAT_NV61		fourcc_code('N', 'V', '6', '1') /* 2x1 subsampled Cb:Cr plane */
 #define DRM_FORMAT_NV24		fourcc_code('N', 'V', '2', '4') /* non-subsampled Cr:Cb plane */
 #define DRM_FORMAT_NV42		fourcc_code('N', 'V', '4', '2') /* non-subsampled Cb:Cr plane */
+/*
+ * 2 plane YCbCr
+ * index 0 = Y plane, [39:0] Y3:Y2:Y1:Y0 little endian
+ * index 1 = Cr:Cb plane, [39:0] Cr1:Cb1:Cr0:Cb0 little endian
+ */
+#define DRM_FORMAT_NV15		fourcc_code('N', 'V', '1', '5') /* 2x2 subsampled Cr:Cb plane */
 
 /*
  * 2 plane YCbCr MSB aligned
@@ -264,6 +270,22 @@ extern "C" {
  */
 #define DRM_FORMAT_P016		fourcc_code('P', '0', '1', '6') /* 2x2 subsampled Cr:Cb plane 16 bits per channel */
 
+/* 3 plane non-subsampled (444) YCbCr
+ * 16 bits per component, but only 10 bits are used and 6 bits are padded
+ * index 0: Y plane, [15:0] Y:x [10:6] little endian
+ * index 1: Cb plane, [15:0] Cb:x [10:6] little endian
+ * index 2: Cr plane, [15:0] Cr:x [10:6] little endian
+ */
+#define DRM_FORMAT_Q410		fourcc_code('Q', '4', '1', '0')
+
+/* 3 plane non-subsampled (444) YCrCb
+ * 16 bits per component, but only 10 bits are used and 6 bits are padded
+ * index 0: Y plane, [15:0] Y:x [10:6] little endian
+ * index 1: Cr plane, [15:0] Cr:x [10:6] little endian
+ * index 2: Cb plane, [15:0] Cb:x [10:6] little endian
+ */
+#define DRM_FORMAT_Q401		fourcc_code('Q', '4', '0', '1')
+
 /*
  * 3 plane YCbCr
  * index 0: Y plane, [7:0] Y
@@ -308,6 +330,7 @@ extern "C" {
 #define DRM_FORMAT_MOD_VENDOR_BROADCOM 0x07
 #define DRM_FORMAT_MOD_VENDOR_ARM     0x08
 #define DRM_FORMAT_MOD_VENDOR_ALLWINNER 0x09
+#define DRM_FORMAT_MOD_VENDOR_AMLOGIC 0x0a
 
 /* add more to the end as needed */
 
@@ -322,8 +345,33 @@ extern "C" {
  * When adding a new token please document the layout with a code comment,
  * similar to the fourcc codes above. drm_fourcc.h is considered the
  * authoritative source for all of these.
+ *
+ * Generic modifier names:
+ *
+ * DRM_FORMAT_MOD_GENERIC_* definitions are used to provide vendor-neutral names
+ * for layouts which are common across multiple vendors. To preserve
+ * compatibility, in cases where a vendor-specific definition already exists and
+ * a generic name for it is desired, the common name is a purely symbolic alias
+ * and must use the same numerical value as the original definition.
+ *
+ * Note that generic names should only be used for modifiers which describe
+ * generic layouts (such as pixel re-ordering), which may have
+ * independently-developed support across multiple vendors.
+ *
+ * In future cases where a generic layout is identified before merging with a
+ * vendor-specific modifier, a new 'GENERIC' vendor or modifier using vendor
+ * 'NONE' could be considered. This should only be for obvious, exceptional
+ * cases to avoid polluting the 'GENERIC' namespace with modifiers which only
+ * apply to a single vendor.
+ *
+ * Generic names should not be used for cases where multiple hardware vendors
+ * have implementations of the same standardised compression scheme (such as
+ * AFBC). In those cases, all implementations should use the same format
+ * modifier(s), reflecting the vendor of the standard.
  */
 
+#define DRM_FORMAT_MOD_GENERIC_16_16_TILE DRM_FORMAT_MOD_SAMSUNG_16_16_TILE
+
 /*
  * Invalid Modifier
  *
@@ -891,6 +939,18 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)
  */
 #define AFBC_FORMAT_MOD_BCH     (1ULL << 11)
 
+/* AFBC uncompressed storage mode
+ *
+ * Indicates that the buffer is using AFBC uncompressed storage mode.
+ * In this mode all superblock payloads in the buffer use the uncompressed
+ * storage mode, which is usually only used for data which cannot be compressed.
+ * The buffer layout is the same as for AFBC buffers without USM set, this only
+ * affects the storage mode of the individual superblocks. Note that even a
+ * buffer without USM set may use uncompressed storage mode for some or all
+ * superblocks, USM just guarantees it for all.
+ */
+#define AFBC_FORMAT_MOD_USM	(1ULL << 12)
+
 /*
  * Arm 16x16 Block U-Interleaved modifier
  *
@@ -915,6 +975,86 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)
  */
 #define DRM_FORMAT_MOD_ALLWINNER_TILED fourcc_mod_code(ALLWINNER, 1)
 
+/*
+ * Amlogic Video Framebuffer Compression modifiers
+ *
+ * Amlogic uses a proprietary lossless image compression protocol and format
+ * for their hardware video codec accelerators, either video decoders or
+ * video input encoders.
+ *
+ * It considerably reduces memory bandwidth while writing and reading
+ * frames in memory.
+ *
+ * The underlying storage is considered to be 3 components, 8bit or 10-bit
+ * per component YCbCr 420, single plane :
+ * - DRM_FORMAT_YUV420_8BIT
+ * - DRM_FORMAT_YUV420_10BIT
+ *
+ * The first 8 bits of the mode defines the layout, then the following 8 bits
+ * defines the options changing the layout.
+ *
+ * Not all combinations are valid, and different SoCs may support different
+ * combinations of layout and options.
+ */
+#define __fourcc_mod_amlogic_layout_mask 0xf
+#define __fourcc_mod_amlogic_options_shift 8
+#define __fourcc_mod_amlogic_options_mask 0xf
+
+#define DRM_FORMAT_MOD_AMLOGIC_FBC(__layout, __options) \
+	fourcc_mod_code(AMLOGIC, \
+			((__layout) & __fourcc_mod_amlogic_layout_mask) | \
+			((__options) & __fourcc_mod_amlogic_options_mask \
+			 << __fourcc_mod_amlogic_options_shift))
+
+/* Amlogic FBC Layouts */
+
+/*
+ * Amlogic FBC Basic Layout
+ *
+ * The basic layout is composed of:
+ * - a body content organized in 64x32 superblocks with 4096 bytes per
+ *   superblock in default mode.
+ * - a 32 bytes per 128x64 header block
+ *
+ * This layout is transferrable between Amlogic SoCs supporting this modifier.
+ */
+#define AMLOGIC_FBC_LAYOUT_BASIC		(1ULL)
+
+/*
+ * Amlogic FBC Scatter Memory layout
+ *
+ * Indicates the header contains IOMMU references to the compressed
+ * frames content to optimize memory access and layout.
+ *
+ * In this mode, only the header memory address is needed, thus the
+ * content memory organization is tied to the current producer
+ * execution and cannot be saved/dumped neither transferrable between
+ * Amlogic SoCs supporting this modifier.
+ *
+ * Due to the nature of the layout, these buffers are not expected to
+ * be accessible by the user-space clients, but only accessible by the
+ * hardware producers and consumers.
+ *
+ * The user-space clients should expect a failure while trying to mmap
+ * the DMA-BUF handle returned by the producer.
+ */
+#define AMLOGIC_FBC_LAYOUT_SCATTER		(2ULL)
+
+/* Amlogic FBC Layout Options Bit Mask */
+
+/*
+ * Amlogic FBC Memory Saving mode
+ *
+ * Indicates the storage is packed when pixel size is multiple of word
+ * boudaries, i.e. 8bit should be stored in this mode to save allocation
+ * memory.
+ *
+ * This mode reduces body layout to 3072 bytes per 64x32 superblock with
+ * the basic layout and 3200 bytes per 64x32 superblock combined with
+ * the scatter layout.
+ */
+#define AMLOGIC_FBC_OPTION_MEM_SAVING		(1ULL << 0)
+
 #if defined(__cplusplus)
 }
 #endif
diff --git a/include/standard-headers/linux/ethtool.h b/include/standard-headers/linux/ethtool.h
index fd8d2cccfe89..e13eff44882d 100644
--- a/include/standard-headers/linux/ethtool.h
+++ b/include/standard-headers/linux/ethtool.h
@@ -579,6 +579,76 @@ struct ethtool_pauseparam {
 	uint32_t	tx_pause;
 };
 
+/**
+ * enum ethtool_link_ext_state - link extended state
+ */
+enum ethtool_link_ext_state {
+	ETHTOOL_LINK_EXT_STATE_AUTONEG,
+	ETHTOOL_LINK_EXT_STATE_LINK_TRAINING_FAILURE,
+	ETHTOOL_LINK_EXT_STATE_LINK_LOGICAL_MISMATCH,
+	ETHTOOL_LINK_EXT_STATE_BAD_SIGNAL_INTEGRITY,
+	ETHTOOL_LINK_EXT_STATE_NO_CABLE,
+	ETHTOOL_LINK_EXT_STATE_CABLE_ISSUE,
+	ETHTOOL_LINK_EXT_STATE_EEPROM_ISSUE,
+	ETHTOOL_LINK_EXT_STATE_CALIBRATION_FAILURE,
+	ETHTOOL_LINK_EXT_STATE_POWER_BUDGET_EXCEEDED,
+	ETHTOOL_LINK_EXT_STATE_OVERHEAT,
+};
+
+/**
+ * enum ethtool_link_ext_substate_autoneg - more information in addition to
+ * ETHTOOL_LINK_EXT_STATE_AUTONEG.
+ */
+enum ethtool_link_ext_substate_autoneg {
+	ETHTOOL_LINK_EXT_SUBSTATE_AN_NO_PARTNER_DETECTED = 1,
+	ETHTOOL_LINK_EXT_SUBSTATE_AN_ACK_NOT_RECEIVED,
+	ETHTOOL_LINK_EXT_SUBSTATE_AN_NEXT_PAGE_EXCHANGE_FAILED,
+	ETHTOOL_LINK_EXT_SUBSTATE_AN_NO_PARTNER_DETECTED_FORCE_MODE,
+	ETHTOOL_LINK_EXT_SUBSTATE_AN_FEC_MISMATCH_DURING_OVERRIDE,
+	ETHTOOL_LINK_EXT_SUBSTATE_AN_NO_HCD,
+};
+
+/**
+ * enum ethtool_link_ext_substate_link_training - more information in addition to
+ * ETHTOOL_LINK_EXT_STATE_LINK_TRAINING_FAILURE.
+ */
+enum ethtool_link_ext_substate_link_training {
+	ETHTOOL_LINK_EXT_SUBSTATE_LT_KR_FRAME_LOCK_NOT_ACQUIRED = 1,
+	ETHTOOL_LINK_EXT_SUBSTATE_LT_KR_LINK_INHIBIT_TIMEOUT,
+	ETHTOOL_LINK_EXT_SUBSTATE_LT_KR_LINK_PARTNER_DID_NOT_SET_RECEIVER_READY,
+	ETHTOOL_LINK_EXT_SUBSTATE_LT_REMOTE_FAULT,
+};
+
+/**
+ * enum ethtool_link_ext_substate_logical_mismatch - more information in addition
+ * to ETHTOOL_LINK_EXT_STATE_LINK_LOGICAL_MISMATCH.
+ */
+enum ethtool_link_ext_substate_link_logical_mismatch {
+	ETHTOOL_LINK_EXT_SUBSTATE_LLM_PCS_DID_NOT_ACQUIRE_BLOCK_LOCK = 1,
+	ETHTOOL_LINK_EXT_SUBSTATE_LLM_PCS_DID_NOT_ACQUIRE_AM_LOCK,
+	ETHTOOL_LINK_EXT_SUBSTATE_LLM_PCS_DID_NOT_GET_ALIGN_STATUS,
+	ETHTOOL_LINK_EXT_SUBSTATE_LLM_FC_FEC_IS_NOT_LOCKED,
+	ETHTOOL_LINK_EXT_SUBSTATE_LLM_RS_FEC_IS_NOT_LOCKED,
+};
+
+/**
+ * enum ethtool_link_ext_substate_bad_signal_integrity - more information in
+ * addition to ETHTOOL_LINK_EXT_STATE_BAD_SIGNAL_INTEGRITY.
+ */
+enum ethtool_link_ext_substate_bad_signal_integrity {
+	ETHTOOL_LINK_EXT_SUBSTATE_BSI_LARGE_NUMBER_OF_PHYSICAL_ERRORS = 1,
+	ETHTOOL_LINK_EXT_SUBSTATE_BSI_UNSUPPORTED_RATE,
+};
+
+/**
+ * enum ethtool_link_ext_substate_cable_issue - more information in
+ * addition to ETHTOOL_LINK_EXT_STATE_CABLE_ISSUE.
+ */
+enum ethtool_link_ext_substate_cable_issue {
+	ETHTOOL_LINK_EXT_SUBSTATE_CI_UNSUPPORTED_CABLE = 1,
+	ETHTOOL_LINK_EXT_SUBSTATE_CI_CABLE_TEST_FAILURE,
+};
+
 #define ETH_GSTRING_LEN		32
 
 /**
@@ -599,6 +669,7 @@ struct ethtool_pauseparam {
  * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
  * @ETH_SS_TS_TX_TYPES: timestamping Tx types
  * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
+ * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types
  */
 enum ethtool_stringset {
 	ETH_SS_TEST		= 0,
@@ -616,6 +687,7 @@ enum ethtool_stringset {
 	ETH_SS_SOF_TIMESTAMPING,
 	ETH_SS_TS_TX_TYPES,
 	ETH_SS_TS_RX_FILTERS,
+	ETH_SS_UDP_TUNNEL_TYPES,
 
 	/* add new constants above here */
 	ETH_SS_COUNT
@@ -1530,6 +1602,21 @@ enum ethtool_link_mode_bit_indices {
 	ETHTOOL_LINK_MODE_400000baseDR8_Full_BIT	 = 72,
 	ETHTOOL_LINK_MODE_400000baseCR8_Full_BIT	 = 73,
 	ETHTOOL_LINK_MODE_FEC_LLRS_BIT			 = 74,
+	ETHTOOL_LINK_MODE_100000baseKR_Full_BIT		 = 75,
+	ETHTOOL_LINK_MODE_100000baseSR_Full_BIT		 = 76,
+	ETHTOOL_LINK_MODE_100000baseLR_ER_FR_Full_BIT	 = 77,
+	ETHTOOL_LINK_MODE_100000baseCR_Full_BIT		 = 78,
+	ETHTOOL_LINK_MODE_100000baseDR_Full_BIT		 = 79,
+	ETHTOOL_LINK_MODE_200000baseKR2_Full_BIT	 = 80,
+	ETHTOOL_LINK_MODE_200000baseSR2_Full_BIT	 = 81,
+	ETHTOOL_LINK_MODE_200000baseLR2_ER2_FR2_Full_BIT = 82,
+	ETHTOOL_LINK_MODE_200000baseDR2_Full_BIT	 = 83,
+	ETHTOOL_LINK_MODE_200000baseCR2_Full_BIT	 = 84,
+	ETHTOOL_LINK_MODE_400000baseKR4_Full_BIT	 = 85,
+	ETHTOOL_LINK_MODE_400000baseSR4_Full_BIT	 = 86,
+	ETHTOOL_LINK_MODE_400000baseLR4_ER4_FR4_Full_BIT = 87,
+	ETHTOOL_LINK_MODE_400000baseDR4_Full_BIT	 = 88,
+	ETHTOOL_LINK_MODE_400000baseCR4_Full_BIT	 = 89,
 	/* must be last entry */
 	__ETHTOOL_LINK_MODE_MASK_NBITS
 };
diff --git a/include/standard-headers/linux/input-event-codes.h b/include/standard-headers/linux/input-event-codes.h
index ebf72c10317b..e740ad9f2e01 100644
--- a/include/standard-headers/linux/input-event-codes.h
+++ b/include/standard-headers/linux/input-event-codes.h
@@ -888,7 +888,8 @@
 #define SW_LINEIN_INSERT	0x0d  /* set = inserted */
 #define SW_MUTE_DEVICE		0x0e  /* set = device disabled */
 #define SW_PEN_INSERTED		0x0f  /* set = pen inserted */
-#define SW_MAX_			0x0f
+#define SW_MACHINE_COVER	0x10  /* set = cover closed */
+#define SW_MAX_			0x10
 #define SW_CNT			(SW_MAX_+1)
 
 /*
diff --git a/linux-headers/asm-generic/unistd.h b/linux-headers/asm-generic/unistd.h
index f4a01305d9a6..995b36c2ea7d 100644
--- a/linux-headers/asm-generic/unistd.h
+++ b/linux-headers/asm-generic/unistd.h
@@ -606,9 +606,9 @@ __SYSCALL(__NR_sendto, sys_sendto)
 #define __NR_recvfrom 207
 __SC_COMP(__NR_recvfrom, sys_recvfrom, compat_sys_recvfrom)
 #define __NR_setsockopt 208
-__SC_COMP(__NR_setsockopt, sys_setsockopt, compat_sys_setsockopt)
+__SC_COMP(__NR_setsockopt, sys_setsockopt, sys_setsockopt)
 #define __NR_getsockopt 209
-__SC_COMP(__NR_getsockopt, sys_getsockopt, compat_sys_getsockopt)
+__SC_COMP(__NR_getsockopt, sys_getsockopt, sys_getsockopt)
 #define __NR_shutdown 210
 __SYSCALL(__NR_shutdown, sys_shutdown)
 #define __NR_sendmsg 211
@@ -850,6 +850,8 @@ __SYSCALL(__NR_pidfd_open, sys_pidfd_open)
 #define __NR_clone3 435
 __SYSCALL(__NR_clone3, sys_clone3)
 #endif
+#define __NR_close_range 436
+__SYSCALL(__NR_close_range, sys_close_range)
 
 #define __NR_openat2 437
 __SYSCALL(__NR_openat2, sys_openat2)
diff --git a/linux-headers/asm-mips/unistd_n32.h b/linux-headers/asm-mips/unistd_n32.h
index 3b9eda7e7d8f..246fbb6a7885 100644
--- a/linux-headers/asm-mips/unistd_n32.h
+++ b/linux-headers/asm-mips/unistd_n32.h
@@ -365,6 +365,7 @@
 #define __NR_fspick	(__NR_Linux + 433)
 #define __NR_pidfd_open	(__NR_Linux + 434)
 #define __NR_clone3	(__NR_Linux + 435)
+#define __NR_close_range	(__NR_Linux + 436)
 #define __NR_openat2	(__NR_Linux + 437)
 #define __NR_pidfd_getfd	(__NR_Linux + 438)
 #define __NR_faccessat2	(__NR_Linux + 439)
diff --git a/linux-headers/asm-mips/unistd_n64.h b/linux-headers/asm-mips/unistd_n64.h
index 9cdf9b6c60df..194d777dfd42 100644
--- a/linux-headers/asm-mips/unistd_n64.h
+++ b/linux-headers/asm-mips/unistd_n64.h
@@ -341,6 +341,7 @@
 #define __NR_fspick	(__NR_Linux + 433)
 #define __NR_pidfd_open	(__NR_Linux + 434)
 #define __NR_clone3	(__NR_Linux + 435)
+#define __NR_close_range	(__NR_Linux + 436)
 #define __NR_openat2	(__NR_Linux + 437)
 #define __NR_pidfd_getfd	(__NR_Linux + 438)
 #define __NR_faccessat2	(__NR_Linux + 439)
diff --git a/linux-headers/asm-mips/unistd_o32.h b/linux-headers/asm-mips/unistd_o32.h
index e3e5e238f026..3e093dd9134d 100644
--- a/linux-headers/asm-mips/unistd_o32.h
+++ b/linux-headers/asm-mips/unistd_o32.h
@@ -411,6 +411,7 @@
 #define __NR_fspick	(__NR_Linux + 433)
 #define __NR_pidfd_open	(__NR_Linux + 434)
 #define __NR_clone3	(__NR_Linux + 435)
+#define __NR_close_range	(__NR_Linux + 436)
 #define __NR_openat2	(__NR_Linux + 437)
 #define __NR_pidfd_getfd	(__NR_Linux + 438)
 #define __NR_faccessat2	(__NR_Linux + 439)
diff --git a/linux-headers/asm-powerpc/kvm.h b/linux-headers/asm-powerpc/kvm.h
index 264e266a85bf..c3af3f324c5a 100644
--- a/linux-headers/asm-powerpc/kvm.h
+++ b/linux-headers/asm-powerpc/kvm.h
@@ -640,6 +640,11 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_ONLINE	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0xbf)
 #define KVM_REG_PPC_PTCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc0)
 
+/* POWER10 registers */
+#define KVM_REG_PPC_MMCR3	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc1)
+#define KVM_REG_PPC_SIER2	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc2)
+#define KVM_REG_PPC_SIER3	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc3)
+
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
  */
diff --git a/linux-headers/asm-powerpc/mman.h b/linux-headers/asm-powerpc/mman.h
index 8db7c2a3be30..c16840d33bfd 100644
--- a/linux-headers/asm-powerpc/mman.h
+++ b/linux-headers/asm-powerpc/mman.h
@@ -11,7 +11,7 @@
 #include <asm-generic/mman-common.h>
 
 
-#define PROT_SAO	0x10		/* Strong Access Ordering */
+#define PROT_SAO	0x10		/* Unsupported since v5.9 */
 
 #define MAP_RENAME      MAP_ANONYMOUS   /* In SunOS terminology */
 #define MAP_NORESERVE   0x40            /* don't reserve swap pages */
diff --git a/linux-headers/asm-powerpc/unistd_32.h b/linux-headers/asm-powerpc/unistd_32.h
index 862edb7448c5..0db9481d4962 100644
--- a/linux-headers/asm-powerpc/unistd_32.h
+++ b/linux-headers/asm-powerpc/unistd_32.h
@@ -418,6 +418,7 @@
 #define __NR_fspick	433
 #define __NR_pidfd_open	434
 #define __NR_clone3	435
+#define __NR_close_range	436
 #define __NR_openat2	437
 #define __NR_pidfd_getfd	438
 #define __NR_faccessat2	439
diff --git a/linux-headers/asm-powerpc/unistd_64.h b/linux-headers/asm-powerpc/unistd_64.h
index f553224ce408..9f74310988e1 100644
--- a/linux-headers/asm-powerpc/unistd_64.h
+++ b/linux-headers/asm-powerpc/unistd_64.h
@@ -390,6 +390,7 @@
 #define __NR_fspick	433
 #define __NR_pidfd_open	434
 #define __NR_clone3	435
+#define __NR_close_range	436
 #define __NR_openat2	437
 #define __NR_pidfd_getfd	438
 #define __NR_faccessat2	439
diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h
index 0138ccb0d892..f053b8304a85 100644
--- a/linux-headers/asm-s390/kvm.h
+++ b/linux-headers/asm-s390/kvm.h
@@ -231,11 +231,13 @@ struct kvm_guest_debug_arch {
 #define KVM_SYNC_GSCB   (1UL << 9)
 #define KVM_SYNC_BPBC   (1UL << 10)
 #define KVM_SYNC_ETOKEN (1UL << 11)
+#define KVM_SYNC_DIAG318 (1UL << 12)
 
 #define KVM_SYNC_S390_VALID_FIELDS \
 	(KVM_SYNC_PREFIX | KVM_SYNC_GPRS | KVM_SYNC_ACRS | KVM_SYNC_CRS | \
 	 KVM_SYNC_ARCH0 | KVM_SYNC_PFAULT | KVM_SYNC_VRS | KVM_SYNC_RICCB | \
-	 KVM_SYNC_FPRS | KVM_SYNC_GSCB | KVM_SYNC_BPBC | KVM_SYNC_ETOKEN)
+	 KVM_SYNC_FPRS | KVM_SYNC_GSCB | KVM_SYNC_BPBC | KVM_SYNC_ETOKEN | \
+	 KVM_SYNC_DIAG318)
 
 /* length and alignment of the sdnx as a power of two */
 #define SDNXC 8
@@ -264,7 +266,8 @@ struct kvm_sync_regs {
 	__u8 reserved2 : 7;
 	__u8 padding1[51];	/* riccb needs to be 64byte aligned */
 	__u8 riccb[64];		/* runtime instrumentation controls block */
-	__u8 padding2[192];	/* sdnx needs to be 256byte aligned */
+	__u64 diag318;		/* diagnose 0x318 info */
+	__u8 padding2[184];	/* sdnx needs to be 256byte aligned */
 	union {
 		__u8 sdnx[SDNXL];  /* state description annex */
 		struct {
diff --git a/linux-headers/asm-s390/unistd_32.h b/linux-headers/asm-s390/unistd_32.h
index e08233c0c377..1803cd0c3ba6 100644
--- a/linux-headers/asm-s390/unistd_32.h
+++ b/linux-headers/asm-s390/unistd_32.h
@@ -408,6 +408,7 @@
 #define __NR_fspick 433
 #define __NR_pidfd_open 434
 #define __NR_clone3 435
+#define __NR_close_range 436
 #define __NR_openat2 437
 #define __NR_pidfd_getfd 438
 #define __NR_faccessat2 439
diff --git a/linux-headers/asm-s390/unistd_64.h b/linux-headers/asm-s390/unistd_64.h
index 560e19ae2bb4..228d5004e5a8 100644
--- a/linux-headers/asm-s390/unistd_64.h
+++ b/linux-headers/asm-s390/unistd_64.h
@@ -356,6 +356,7 @@
 #define __NR_fspick 433
 #define __NR_pidfd_open 434
 #define __NR_clone3 435
+#define __NR_close_range 436
 #define __NR_openat2 437
 #define __NR_pidfd_getfd 438
 #define __NR_faccessat2 439
diff --git a/linux-headers/asm-x86/unistd_32.h b/linux-headers/asm-x86/unistd_32.h
index c727981d4a3a..356c12c2dbce 100644
--- a/linux-headers/asm-x86/unistd_32.h
+++ b/linux-headers/asm-x86/unistd_32.h
@@ -426,6 +426,7 @@
 #define __NR_fspick 433
 #define __NR_pidfd_open 434
 #define __NR_clone3 435
+#define __NR_close_range 436
 #define __NR_openat2 437
 #define __NR_pidfd_getfd 438
 #define __NR_faccessat2 439
diff --git a/linux-headers/asm-x86/unistd_64.h b/linux-headers/asm-x86/unistd_64.h
index 843fa6274584..ef70e1c7c93f 100644
--- a/linux-headers/asm-x86/unistd_64.h
+++ b/linux-headers/asm-x86/unistd_64.h
@@ -348,6 +348,7 @@
 #define __NR_fspick 433
 #define __NR_pidfd_open 434
 #define __NR_clone3 435
+#define __NR_close_range 436
 #define __NR_openat2 437
 #define __NR_pidfd_getfd 438
 #define __NR_faccessat2 439
diff --git a/linux-headers/asm-x86/unistd_x32.h b/linux-headers/asm-x86/unistd_x32.h
index 7d63d703cab4..84ae8e9f5fca 100644
--- a/linux-headers/asm-x86/unistd_x32.h
+++ b/linux-headers/asm-x86/unistd_x32.h
@@ -301,6 +301,7 @@
 #define __NR_fspick (__X32_SYSCALL_BIT + 433)
 #define __NR_pidfd_open (__X32_SYSCALL_BIT + 434)
 #define __NR_clone3 (__X32_SYSCALL_BIT + 435)
+#define __NR_close_range (__X32_SYSCALL_BIT + 436)
 #define __NR_openat2 (__X32_SYSCALL_BIT + 437)
 #define __NR_pidfd_getfd (__X32_SYSCALL_BIT + 438)
 #define __NR_faccessat2 (__X32_SYSCALL_BIT + 439)
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index a28c3667370b..b4bd05d09135 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -289,6 +289,7 @@ struct kvm_run {
 		/* KVM_EXIT_FAIL_ENTRY */
 		struct {
 			__u64 hardware_entry_failure_reason;
+			__u32 cpu;
 		} fail_entry;
 		/* KVM_EXIT_EXCEPTION */
 		struct {
@@ -1031,6 +1032,10 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_PPC_SECURE_GUEST 181
 #define KVM_CAP_HALT_POLL 182
 #define KVM_CAP_ASYNC_PF_INT 183
+#define KVM_CAP_LAST_CPU 184
+#define KVM_CAP_SMALLER_MAXPHYADDR 185
+#define KVM_CAP_S390_DIAG318 186
+#define KVM_CAP_SYS_HYPERV_CPUID 187
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -1489,7 +1494,7 @@ struct kvm_enc_region {
 /* Available with KVM_CAP_MANUAL_DIRTY_LOG_PROTECT_2 */
 #define KVM_CLEAR_DIRTY_LOG          _IOWR(KVMIO, 0xc0, struct kvm_clear_dirty_log)
 
-/* Available with KVM_CAP_HYPERV_CPUID */
+/* Available with KVM_CAP_HYPERV_CPUID (vcpu) / KVM_CAP_SYS_HYPERV_CPUID (system) */
 #define KVM_GET_SUPPORTED_HV_CPUID _IOWR(KVMIO, 0xc1, struct kvm_cpuid2)
 
 /* Available with KVM_CAP_ARM_SVE */
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index f09df262c4b5..a90672494dc5 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -1030,7 +1030,7 @@ struct vfio_iommu_type1_info_cap_iova_range {
  * size in bytes that can be used by user applications when getting the dirty
  * bitmap.
  */
-#define VFIO_IOMMU_TYPE1_INFO_CAP_MIGRATION  1
+#define VFIO_IOMMU_TYPE1_INFO_CAP_MIGRATION  2
 
 struct vfio_iommu_type1_info_cap_migration {
 	struct	vfio_info_cap_header header;
-- 
2.25.4

[PATCH RFC 02/22] i386: drop x86_cpu_get_supported_feature_word() forward declaration

[Vitaly Kuznetsov]

We only use x86_cpu_get_supported_feature_word() after its implementation,
no forward declaration needed.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 49d89585288d..14489def2177 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4283,9 +4283,6 @@ void x86_cpu_change_kvm_default(const char *prop, const char *value)
     assert(pv->prop);
 }
 
-static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
-                                                   bool migratable_only);
-
 static bool lmce_supported(void)
 {
     uint64_t mce_cap = 0;
-- 
2.25.4

[PATCH RFC 03/22] i386: move hyperv_vendor_id initialization to x86_cpu_realizefn()

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, move
hyperv_vendor_id initialization to x86_cpu_realizefn().

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c | 15 ++++++++++++++-
 target/i386/cpu.h |  3 ++-
 target/i386/kvm.c | 25 ++++++++++---------------
 3 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 14489def2177..07e9da9e567e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6625,6 +6625,19 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         }
     }
 
+    if (!cpu->hyperv_vendor) {
+        memcpy(cpu->hyperv_vendor_id, "Microsoft Hv", 12);
+    } else {
+        size_t len = strlen(cpu->hyperv_vendor);
+
+        if (len > 12) {
+            warn_report("hv-vendor-id truncated to 12 characters");
+            len = 12;
+        }
+        memset(cpu->hyperv_vendor_id, 0, 12);
+        memcpy(cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);
+    }
+
     if (cpu->ucode_rev == 0) {
         /* The default is the same as KVM's.  */
         if (IS_AMD_CPU(env)) {
@@ -7313,7 +7326,7 @@ static Property x86_cpu_properties[] = {
     DEFINE_PROP_UINT32("min-xlevel2", X86CPU, env.cpuid_min_xlevel2, 0),
     DEFINE_PROP_UINT64("ucode-rev", X86CPU, ucode_rev, 0),
     DEFINE_PROP_BOOL("full-cpuid-auto-level", X86CPU, full_cpuid_auto_level, true),
-    DEFINE_PROP_STRING("hv-vendor-id", X86CPU, hyperv_vendor_id),
+    DEFINE_PROP_STRING("hv-vendor-id", X86CPU, hyperv_vendor),
     DEFINE_PROP_BOOL("cpuid-0xb", X86CPU, enable_cpuid_0xb, true),
     DEFINE_PROP_BOOL("lmce", X86CPU, enable_lmce, false),
     DEFINE_PROP_BOOL("l3-cache", X86CPU, enable_l3_cache, true),
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index d3097be6a50a..903994818093 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1654,11 +1654,12 @@ struct X86CPU {
     uint64_t ucode_rev;
 
     uint32_t hyperv_spinlock_attempts;
-    char *hyperv_vendor_id;
+    char *hyperv_vendor;
     bool hyperv_synic_kvm_only;
     uint64_t hyperv_features;
     bool hyperv_passthrough;
     OnOffAuto hyperv_no_nonarch_cs;
+    uint32_t hyperv_vendor_id[3];
 
     bool check_cpuid;
     bool enforce_cpuid;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 205b68bc0ce8..47779c5e1efd 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1225,6 +1225,13 @@ static int hyperv_handle_properties(CPUState *cs,
         memcpy(cpuid_ent, &cpuid->entries[0],
                cpuid->nent * sizeof(cpuid->entries[0]));
 
+        c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
+        if (c) {
+            cpu->hyperv_vendor_id[0] = c->ebx;
+            cpu->hyperv_vendor_id[1] = c->ecx;
+            cpu->hyperv_vendor_id[2] = c->edx;
+        }
+
         c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
         if (c) {
             env->features[FEAT_HYPERV_EAX] = c->eax;
@@ -1299,23 +1306,11 @@ static int hyperv_handle_properties(CPUState *cs,
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
-    if (!cpu->hyperv_vendor_id) {
-        memcpy(signature, "Microsoft Hv", 12);
-    } else {
-        size_t len = strlen(cpu->hyperv_vendor_id);
-
-        if (len > 12) {
-            error_report("hv-vendor-id truncated to 12 characters");
-            len = 12;
-        }
-        memset(signature, 0, 12);
-        memcpy(signature, cpu->hyperv_vendor_id, len);
-    }
     c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
         HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
-    c->ebx = signature[0];
-    c->ecx = signature[1];
-    c->edx = signature[2];
+    c->ebx = cpu->hyperv_vendor_id[0];
+    c->ecx = cpu->hyperv_vendor_id[1];
+    c->edx = cpu->hyperv_vendor_id[2];
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_INTERFACE;
-- 
2.25.4

[PATCH RFC 04/22] i386: move hyperv_interface_id initialization to x86_cpu_realizefn()

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, move
hyperv_interface_id initialization to x86_cpu_realizefn().

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c |  6 ++++++
 target/i386/cpu.h |  1 +
 target/i386/kvm.c | 18 ++++++++++++------
 3 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 07e9da9e567e..16888125a30a 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6638,6 +6638,12 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         memcpy(cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);
     }
 
+    /* 'Hv#1' interface identification*/
+    cpu->hyperv_interface_id[0] = 0x31237648;
+    cpu->hyperv_interface_id[1] = 0;
+    cpu->hyperv_interface_id[2] = 0;
+    cpu->hyperv_interface_id[3] = 0;
+
     if (cpu->ucode_rev == 0) {
         /* The default is the same as KVM's.  */
         if (IS_AMD_CPU(env)) {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 903994818093..91edc54a268c 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1660,6 +1660,7 @@ struct X86CPU {
     bool hyperv_passthrough;
     OnOffAuto hyperv_no_nonarch_cs;
     uint32_t hyperv_vendor_id[3];
+    uint32_t hyperv_interface_id[4];
 
     bool check_cpuid;
     bool enforce_cpuid;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 47779c5e1efd..a36c65100cd0 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1188,7 +1188,6 @@ static int hyperv_handle_properties(CPUState *cs,
     CPUX86State *env = &cpu->env;
     struct kvm_cpuid2 *cpuid;
     struct kvm_cpuid_entry2 *c;
-    uint32_t signature[3];
     uint32_t cpuid_i = 0;
     int r;
 
@@ -1232,6 +1231,14 @@ static int hyperv_handle_properties(CPUState *cs,
             cpu->hyperv_vendor_id[2] = c->edx;
         }
 
+        c = cpuid_find_entry(cpuid, HV_CPUID_INTERFACE, 0);
+        if (c) {
+            cpu->hyperv_interface_id[0] = c->eax;
+            cpu->hyperv_interface_id[1] = c->ebx;
+            cpu->hyperv_interface_id[2] = c->ecx;
+            cpu->hyperv_interface_id[3] = c->edx;
+        }
+
         c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
         if (c) {
             env->features[FEAT_HYPERV_EAX] = c->eax;
@@ -1314,11 +1321,10 @@ static int hyperv_handle_properties(CPUState *cs,
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_INTERFACE;
-    memcpy(signature, "Hv#1\0\0\0\0\0\0\0\0", 12);
-    c->eax = signature[0];
-    c->ebx = 0;
-    c->ecx = 0;
-    c->edx = 0;
+    c->eax = cpu->hyperv_interface_id[0];
+    c->ebx = cpu->hyperv_interface_id[1];
+    c->ecx = cpu->hyperv_interface_id[2];
+    c->edx = cpu->hyperv_interface_id[3];
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_VERSION;
-- 
2.25.4

[PATCH RFC 05/22] i386: move hyperv_version_id initialization to x86_cpu_realizefn()

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, move
hyperv_version_id initialization to x86_cpu_realizefn().

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c |  4 ++++
 target/i386/cpu.h |  1 +
 target/i386/kvm.c | 14 ++++++++++++--
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 16888125a30a..e605399eb8c0 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6644,6 +6644,10 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     cpu->hyperv_interface_id[2] = 0;
     cpu->hyperv_interface_id[3] = 0;
 
+    /* Hypervisor system identity */
+    cpu->hyperv_version_id[0] = 0x00001bbc;
+    cpu->hyperv_version_id[1] = 0x00060001;
+
     if (cpu->ucode_rev == 0) {
         /* The default is the same as KVM's.  */
         if (IS_AMD_CPU(env)) {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 91edc54a268c..2630ffd2d4b2 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1661,6 +1661,7 @@ struct X86CPU {
     OnOffAuto hyperv_no_nonarch_cs;
     uint32_t hyperv_vendor_id[3];
     uint32_t hyperv_interface_id[4];
+    uint32_t hyperv_version_id[4];
 
     bool check_cpuid;
     bool enforce_cpuid;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index a36c65100cd0..169bae2779a4 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1239,6 +1239,14 @@ static int hyperv_handle_properties(CPUState *cs,
             cpu->hyperv_interface_id[3] = c->edx;
         }
 
+        c = cpuid_find_entry(cpuid, HV_CPUID_VERSION, 0);
+        if (c) {
+            cpu->hyperv_version_id[0] = c->eax;
+            cpu->hyperv_version_id[1] = c->ebx;
+            cpu->hyperv_version_id[2] = c->ecx;
+            cpu->hyperv_version_id[3] = c->edx;
+        }
+
         c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
         if (c) {
             env->features[FEAT_HYPERV_EAX] = c->eax;
@@ -1328,8 +1336,10 @@ static int hyperv_handle_properties(CPUState *cs,
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_VERSION;
-    c->eax = 0x00001bbc;
-    c->ebx = 0x00060001;
+    c->eax = cpu->hyperv_version_id[0];
+    c->ebx = cpu->hyperv_version_id[1];
+    c->ecx = cpu->hyperv_version_id[2];
+    c->edx = cpu->hyperv_version_id[3];
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_FEATURES;
-- 
2.25.4

[PATCH RFC 06/22] i386: move hyperv_limits initialization to x86_cpu_realizefn()

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, move
hyperv_limits initialization to x86_cpu_realizefn().

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c |  5 +++++
 target/i386/cpu.h |  1 +
 target/i386/kvm.c | 13 ++++++++++++-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index e605399eb8c0..ef3c672cf415 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6648,6 +6648,11 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     cpu->hyperv_version_id[0] = 0x00001bbc;
     cpu->hyperv_version_id[1] = 0x00060001;
 
+    /* Hypervisor implementation limits */
+    cpu->hyperv_limits[0] = 64;
+    cpu->hyperv_limits[1] = 0;
+    cpu->hyperv_limits[2] = 0;
+
     if (cpu->ucode_rev == 0) {
         /* The default is the same as KVM's.  */
         if (IS_AMD_CPU(env)) {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 2630ffd2d4b2..095d0bf75493 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1662,6 +1662,7 @@ struct X86CPU {
     uint32_t hyperv_vendor_id[3];
     uint32_t hyperv_interface_id[4];
     uint32_t hyperv_version_id[4];
+    uint32_t hyperv_limits[3];
 
     bool check_cpuid;
     bool enforce_cpuid;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 169bae2779a4..720c30e9df17 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1253,6 +1253,15 @@ static int hyperv_handle_properties(CPUState *cs,
             env->features[FEAT_HYPERV_EBX] = c->ebx;
             env->features[FEAT_HYPERV_EDX] = c->eax;
         }
+
+        c = cpuid_find_entry(cpuid, HV_CPUID_IMPLEMENT_LIMITS, 0);
+        if (c) {
+            cpu->hv_max_vps = c->eax;
+            cpu->hyperv_limits[0] = c->ebx;
+            cpu->hyperv_limits[1] = c->ecx;
+            cpu->hyperv_limits[2] = c->edx;
+        }
+
         c = cpuid_find_entry(cpuid, HV_CPUID_ENLIGHTMENT_INFO, 0);
         if (c) {
             env->features[FEAT_HV_RECOMM_EAX] = c->eax;
@@ -1355,7 +1364,9 @@ static int hyperv_handle_properties(CPUState *cs,
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_IMPLEMENT_LIMITS;
     c->eax = cpu->hv_max_vps;
-    c->ebx = 0x40;
+    c->ebx = cpu->hyperv_limits[0];
+    c->ecx = cpu->hyperv_limits[1];
+    c->edx = cpu->hyperv_limits[2];
 
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
         __u32 function;
-- 
2.25.4

[PATCH RFC 07/22] i386: fill in FEAT_HYPERV_EDX from edx instead of eax

[Vitaly Kuznetsov]

There was a typo which went unnoticed.

Fixes: e48ddcc6ce13 ("i386/kvm: implement 'hv-passthrough' mode")
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 720c30e9df17..1cb5592d4f7d 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1251,7 +1251,7 @@ static int hyperv_handle_properties(CPUState *cs,
         if (c) {
             env->features[FEAT_HYPERV_EAX] = c->eax;
             env->features[FEAT_HYPERV_EBX] = c->ebx;
-            env->features[FEAT_HYPERV_EDX] = c->eax;
+            env->features[FEAT_HYPERV_EDX] = c->edx;
         }
 
         c = cpuid_find_entry(cpuid, HV_CPUID_IMPLEMENT_LIMITS, 0);
-- 
2.25.4

[PATCH RFC 08/22] i386: invert hyperv_spinlock_attempts setting logic with hv_passthrough

[Vitaly Kuznetsov]

There is no need to have this special case: like all other Hyper-V
enlightenments we can just use kernel's supplied value in hv_passthrough
mode.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 1cb5592d4f7d..96ac719adca1 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1265,11 +1265,7 @@ static int hyperv_handle_properties(CPUState *cs,
         c = cpuid_find_entry(cpuid, HV_CPUID_ENLIGHTMENT_INFO, 0);
         if (c) {
             env->features[FEAT_HV_RECOMM_EAX] = c->eax;
-
-            /* hv-spinlocks may have been overriden */
-            if (cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY) {
-                c->ebx = cpu->hyperv_spinlock_attempts;
-            }
+            cpu->hyperv_spinlock_attempts = c->ebx;
         }
         c = cpuid_find_entry(cpuid, HV_CPUID_NESTED_FEATURES, 0);
         if (c) {
-- 
2.25.4

[PATCH RFC 09/22] i386: add reserved FEAT_HYPERV_ECX CPUID leaf

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, add
reserved FEAT_HYPERV_ECX CPUID leaf.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c | 5 +++++
 target/i386/cpu.h | 1 +
 target/i386/kvm.c | 2 ++
 3 files changed, 8 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index ef3c672cf415..70588571ccb1 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -974,6 +974,11 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         },
         .cpuid = { .eax = 0x40000003, .reg = R_EBX, },
     },
+    [FEAT_HYPERV_ECX] = {
+        .type = CPUID_FEATURE_WORD,
+        /* reserved */
+        .cpuid = { .eax = 0x40000003, .reg = R_ECX, },
+    },
     [FEAT_HYPERV_EDX] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 095d0bf75493..39e0e89aa41f 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -521,6 +521,7 @@ typedef enum FeatureWord {
     FEAT_KVM_HINTS,     /* CPUID[4000_0001].EDX */
     FEAT_HYPERV_EAX,    /* CPUID[4000_0003].EAX */
     FEAT_HYPERV_EBX,    /* CPUID[4000_0003].EBX */
+    FEAT_HYPERV_ECX,    /* CPUID[4000_0003].ECX */
     FEAT_HYPERV_EDX,    /* CPUID[4000_0003].EDX */
     FEAT_HV_RECOMM_EAX, /* CPUID[4000_0004].EAX */
     FEAT_HV_NESTED_EAX, /* CPUID[4000_000A].EAX */
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 96ac719adca1..259be2d514dd 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1251,6 +1251,7 @@ static int hyperv_handle_properties(CPUState *cs,
         if (c) {
             env->features[FEAT_HYPERV_EAX] = c->eax;
             env->features[FEAT_HYPERV_EBX] = c->ebx;
+            env->features[FEAT_HYPERV_ECX] = c->ecx;
             env->features[FEAT_HYPERV_EDX] = c->edx;
         }
 
@@ -1350,6 +1351,7 @@ static int hyperv_handle_properties(CPUState *cs,
     c->function = HV_CPUID_FEATURES;
     c->eax = env->features[FEAT_HYPERV_EAX];
     c->ebx = env->features[FEAT_HYPERV_EBX];
+    c->ecx = env->features[FEAT_HYPERV_ECX];
     c->edx = env->features[FEAT_HYPERV_EDX];
 
     c = &cpuid_ent[cpuid_i++];
-- 
2.25.4

[PATCH RFC 10/22] i386: add reserved FEAT_HV_RECOMM_ECX/FEAT_HV_RECOMM_EDX CPUID leaves

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, add reserved
FEAT_HV_RECOMM_ECX/FEAT_HV_RECOMM_EDX CPUID leaves.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c | 10 ++++++++++
 target/i386/cpu.h |  2 ++
 target/i386/kvm.c |  5 +++++
 3 files changed, 17 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 70588571ccb1..3665f22093e9 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1021,6 +1021,16 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         },
         .cpuid = { .eax = 0x40000004, .reg = R_EAX, },
     },
+    [FEAT_HV_RECOMM_ECX] = {
+        .type = CPUID_FEATURE_WORD,
+        /* reserved */
+        .cpuid = { .eax = 0x40000004, .reg = R_ECX, },
+    },
+    [FEAT_HV_RECOMM_EDX] = {
+        .type = CPUID_FEATURE_WORD,
+        /* reserved */
+        .cpuid = { .eax = 0x40000004, .reg = R_EDX, },
+    },
     [FEAT_HV_NESTED_EAX] = {
         .type = CPUID_FEATURE_WORD,
         .cpuid = { .eax = 0x4000000A, .reg = R_EAX, },
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 39e0e89aa41f..0668fe74df4f 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -524,6 +524,8 @@ typedef enum FeatureWord {
     FEAT_HYPERV_ECX,    /* CPUID[4000_0003].ECX */
     FEAT_HYPERV_EDX,    /* CPUID[4000_0003].EDX */
     FEAT_HV_RECOMM_EAX, /* CPUID[4000_0004].EAX */
+    FEAT_HV_RECOMM_ECX, /* CPUID[4000_0004].ECX */
+    FEAT_HV_RECOMM_EDX, /* CPUID[4000_0004].EDX */
     FEAT_HV_NESTED_EAX, /* CPUID[4000_000A].EAX */
     FEAT_SVM,           /* CPUID[8000_000A].EDX */
     FEAT_XSAVE,         /* CPUID[EAX=0xd,ECX=1].EAX */
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 259be2d514dd..c18f221cd301 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1267,7 +1267,10 @@ static int hyperv_handle_properties(CPUState *cs,
         if (c) {
             env->features[FEAT_HV_RECOMM_EAX] = c->eax;
             cpu->hyperv_spinlock_attempts = c->ebx;
+            env->features[FEAT_HV_RECOMM_ECX] = c->ecx;
+            env->features[FEAT_HV_RECOMM_EDX] = c->edx;
         }
+
         c = cpuid_find_entry(cpuid, HV_CPUID_NESTED_FEATURES, 0);
         if (c) {
             env->features[FEAT_HV_NESTED_EAX] = c->eax;
@@ -1358,6 +1361,8 @@ static int hyperv_handle_properties(CPUState *cs,
     c->function = HV_CPUID_ENLIGHTMENT_INFO;
     c->eax = env->features[FEAT_HV_RECOMM_EAX];
     c->ebx = cpu->hyperv_spinlock_attempts;
+    c->ecx = env->features[FEAT_HV_RECOMM_ECX];
+    c->edx = env->features[FEAT_HV_RECOMM_EDX];
 
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_IMPLEMENT_LIMITS;
-- 
2.25.4

[PATCH RFC 11/22] i386: add reserved FEAT_HV_NESTED_EBX/ECX/EDX CPUID leaves

[Vitaly Kuznetsov]

As a preparation to expanding Hyper-V CPU features early, add reserved
FEAT_HV_NESTED_EBX/ECX/EDX CPUID leaves.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c | 15 +++++++++++++++
 target/i386/cpu.h |  3 +++
 target/i386/kvm.c |  6 ++++++
 3 files changed, 24 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 3665f22093e9..479c4bbbf459 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1035,6 +1035,21 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
         .type = CPUID_FEATURE_WORD,
         .cpuid = { .eax = 0x4000000A, .reg = R_EAX, },
     },
+    [FEAT_HV_NESTED_EBX] = {
+        .type = CPUID_FEATURE_WORD,
+        /* reserved */
+        .cpuid = { .eax = 0x4000000A, .reg = R_EBX, },
+    },
+    [FEAT_HV_NESTED_ECX] = {
+        .type = CPUID_FEATURE_WORD,
+        /* reserved */
+        .cpuid = { .eax = 0x4000000A, .reg = R_ECX, },
+    },
+    [FEAT_HV_NESTED_EDX] = {
+        .type = CPUID_FEATURE_WORD,
+        /* reserved */
+        .cpuid = { .eax = 0x4000000A, .reg = R_EDX, },
+    },
     [FEAT_SVM] = {
         .type = CPUID_FEATURE_WORD,
         .feat_names = {
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0668fe74df4f..0aad60e0c707 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -527,6 +527,9 @@ typedef enum FeatureWord {
     FEAT_HV_RECOMM_ECX, /* CPUID[4000_0004].ECX */
     FEAT_HV_RECOMM_EDX, /* CPUID[4000_0004].EDX */
     FEAT_HV_NESTED_EAX, /* CPUID[4000_000A].EAX */
+    FEAT_HV_NESTED_EBX, /* CPUID[4000_000A].EBX */
+    FEAT_HV_NESTED_ECX, /* CPUID[4000_000A].ECX */
+    FEAT_HV_NESTED_EDX, /* CPUID[4000_000A].EDX */
     FEAT_SVM,           /* CPUID[8000_000A].EDX */
     FEAT_XSAVE,         /* CPUID[EAX=0xd,ECX=1].EAX */
     FEAT_6_EAX,         /* CPUID[6].EAX */
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index c18f221cd301..87b83a2aa2cb 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1274,6 +1274,9 @@ static int hyperv_handle_properties(CPUState *cs,
         c = cpuid_find_entry(cpuid, HV_CPUID_NESTED_FEATURES, 0);
         if (c) {
             env->features[FEAT_HV_NESTED_EAX] = c->eax;
+            env->features[FEAT_HV_NESTED_EBX] = c->ebx;
+            env->features[FEAT_HV_NESTED_ECX] = c->ecx;
+            env->features[FEAT_HV_NESTED_EDX] = c->edx;
         }
     }
 
@@ -1384,6 +1387,9 @@ static int hyperv_handle_properties(CPUState *cs,
         c = &cpuid_ent[cpuid_i++];
         c->function = HV_CPUID_NESTED_FEATURES;
         c->eax = env->features[FEAT_HV_NESTED_EAX];
+        c->ebx = env->features[FEAT_HV_NESTED_EBX];
+        c->ecx = env->features[FEAT_HV_NESTED_ECX];
+        c->edx = env->features[FEAT_HV_NESTED_EDX];
     }
     r = cpuid_i;
 
-- 
2.25.4

[PATCH RFC 12/22] i386: always fill Hyper-V CPUID feature leaves from X86CPU data

[Vitaly Kuznetsov]

We have all the required data in X86CPU already and as we are about to
split hyperv_handle_properties() into hyperv_expand_features()/
hyperv_fill_cpuids() we can remove the blind copy. The functional change
is that QEMU won't pass CPUID leaves it doesn't currently know about
to the guest but arguably this is a good change.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 87b83a2aa2cb..b7f0e200a75f 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1221,9 +1221,6 @@ static int hyperv_handle_properties(CPUState *cs,
     }
 
     if (cpu->hyperv_passthrough) {
-        memcpy(cpuid_ent, &cpuid->entries[0],
-               cpuid->nent * sizeof(cpuid->entries[0]));
-
         c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
         if (c) {
             cpu->hyperv_vendor_id[0] = c->ebx;
@@ -1325,12 +1322,6 @@ static int hyperv_handle_properties(CPUState *cs,
         goto free;
     }
 
-    if (cpu->hyperv_passthrough) {
-        /* We already copied all feature words from KVM as is */
-        r = cpuid->nent;
-        goto free;
-    }
-
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
     c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
-- 
2.25.4

[PATCH RFC 13/22] i386: split hyperv_handle_properties() into hyperv_expand_features()/hyperv_fill_cpuids()

[Vitaly Kuznetsov]

The intention is to call hyperv_expand_features() early, before vCPUs
are created and use the acquired data later when we set guest visible
CPUID data.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 45 +++++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 16 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b7f0e200a75f..c4bb147090a9 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1177,18 +1177,18 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
 }
 
 /*
- * Fill in Hyper-V CPUIDs. Returns the number of entries filled in cpuid_ent in
- * case of success, errno < 0 in case of failure and 0 when no Hyper-V
- * extentions are enabled.
+ * Expand Hyper-V CPU features. In partucular, check that all the requested
+ * features are supported by the host and the sanity of the configuration
+ * (that all the required dependencies are included). Also, this takes care
+ * of 'hv_passthrough' mode and fills the environment with all supported
+ * Hyper-V features.
  */
-static int hyperv_handle_properties(CPUState *cs,
-                                    struct kvm_cpuid_entry2 *cpuid_ent)
+static int hyperv_expand_features(CPUState *cs)
 {
     X86CPU *cpu = X86_CPU(cs);
     CPUX86State *env = &cpu->env;
     struct kvm_cpuid2 *cpuid;
     struct kvm_cpuid_entry2 *c;
-    uint32_t cpuid_i = 0;
     int r;
 
     if (!hyperv_enabled(cpu))
@@ -1317,11 +1317,26 @@ static int hyperv_handle_properties(CPUState *cs,
     /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
     env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
 
+    g_free(cpuid);
+
     if (r) {
-        r = -ENOSYS;
-        goto free;
+        return -ENOSYS;
     }
 
+    return 0;
+}
+
+/*
+ * Fill in Hyper-V CPUIDs. Returns the number of entries filled in cpuid_ent.
+ */
+static int hyperv_fill_cpuids(CPUState *cs,
+                              struct kvm_cpuid_entry2 *cpuid_ent)
+{
+    X86CPU *cpu = X86_CPU(cs);
+    CPUX86State *env = &cpu->env;
+    struct kvm_cpuid_entry2 *c;
+    uint32_t cpuid_i = 0;
+
     c = &cpuid_ent[cpuid_i++];
     c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
     c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
@@ -1382,12 +1397,8 @@ static int hyperv_handle_properties(CPUState *cs,
         c->ecx = env->features[FEAT_HV_NESTED_ECX];
         c->edx = env->features[FEAT_HV_NESTED_EDX];
     }
-    r = cpuid_i;
 
-free:
-    g_free(cpuid);
-
-    return r;
+    return cpuid_i;
 }
 
 static Error *hv_passthrough_mig_blocker;
@@ -1529,11 +1540,13 @@ int kvm_arch_init_vcpu(CPUState *cs)
     env->apic_bus_freq = KVM_APIC_BUS_FREQUENCY;
 
     /* Paravirtualization CPUIDs */
-    r = hyperv_handle_properties(cs, cpuid_data.entries);
+    r = hyperv_expand_features(cs);
     if (r < 0) {
         return r;
-    } else if (r > 0) {
-        cpuid_i = r;
+    }
+
+    if (hyperv_enabled(cpu)) {
+        cpuid_i = hyperv_fill_cpuids(cs, cpuid_data.entries);
         kvm_base = KVM_CPUID_SIGNATURE_NEXT;
         has_msr_hv_hypercall = true;
     }
-- 
2.25.4

[PATCH RFC 14/22] i386: move eVMCS enablement to hyperv_init_vcpu()

[Vitaly Kuznetsov]

hyperv_expand_features() will be called before we create vCPU so
evmcs enablement should go away. hyperv_init_vcpu() looks like the
right place.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 63 ++++++++++++++++++++++++++++-------------------
 1 file changed, 38 insertions(+), 25 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index c4bb147090a9..2e9ceafa2421 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -974,6 +974,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
 {
     struct kvm_cpuid2 *cpuid;
     int max = 7; /* 0x40000000..0x40000005, 0x4000000A */
+    int i;
 
     /*
      * When the buffer is too small, KVM_GET_SUPPORTED_HV_CPUID fails with
@@ -983,6 +984,22 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
     while ((cpuid = try_get_hv_cpuid(cs, max)) == NULL) {
         max++;
     }
+
+    /*
+     * KVM_GET_SUPPORTED_HV_CPUID does not set EVMCS CPUID bit before
+     * KVM_CAP_HYPERV_ENLIGHTENED_VMCS is enabled but we want to get the
+     * information early, just check for the capability and set the bit
+     * manually.
+     */
+    if (kvm_check_extension(cs->kvm_state,
+                            KVM_CAP_HYPERV_ENLIGHTENED_VMCS) > 0) {
+        for (i = 0; i < cpuid->nent; i++) {
+            if (cpuid->entries[i].function == HV_CPUID_ENLIGHTMENT_INFO) {
+                cpuid->entries[i].eax |= HV_ENLIGHTENED_VMCS_RECOMMENDED;
+            }
+        }
+    }
+
     return cpuid;
 }
 
@@ -1194,26 +1211,6 @@ static int hyperv_expand_features(CPUState *cs)
     if (!hyperv_enabled(cpu))
         return 0;
 
-    if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ||
-        cpu->hyperv_passthrough) {
-        uint16_t evmcs_version;
-
-        r = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
-                                (uintptr_t)&evmcs_version);
-
-        if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) && r) {
-            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
-                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
-            return -ENOSYS;
-        }
-
-        if (!r) {
-            env->features[FEAT_HV_RECOMM_EAX] |=
-                HV_ENLIGHTENED_VMCS_RECOMMENDED;
-            env->features[FEAT_HV_NESTED_EAX] = evmcs_version;
-        }
-    }
-
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
         cpuid = get_supported_hv_cpuid(cs);
     } else {
@@ -1407,6 +1404,7 @@ static Error *hv_no_nonarch_cs_mig_blocker;
 static int hyperv_init_vcpu(X86CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
+    CPUX86State *env = &cpu->env;
     Error *local_err = NULL;
     int ret;
 
@@ -1482,6 +1480,21 @@ static int hyperv_init_vcpu(X86CPU *cpu)
         }
     }
 
+    if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
+        uint16_t evmcs_version;
+
+        ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
+                                  (uintptr_t)&evmcs_version);
+
+        if (ret < 0) {
+            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
+                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
+            return ret;
+        }
+
+        env->features[FEAT_HV_NESTED_EAX] = evmcs_version;
+    }
+
     return 0;
 }
 
@@ -1546,6 +1559,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
     }
 
     if (hyperv_enabled(cpu)) {
+        r = hyperv_init_vcpu(cpu);
+        if (r) {
+            return r;
+        }
+
         cpuid_i = hyperv_fill_cpuids(cs, cpuid_data.entries);
         kvm_base = KVM_CPUID_SIGNATURE_NEXT;
         has_msr_hv_hypercall = true;
@@ -1892,11 +1910,6 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     kvm_init_msrs(cpu);
 
-    r = hyperv_init_vcpu(cpu);
-    if (r) {
-        goto fail;
-    }
-
     return 0;
 
  fail:
-- 
2.25.4

[PATCH RFC 15/22] i386: switch hyperv_expand_features() to using error_setg()

[Vitaly Kuznetsov]

Use standard error_setg() mechanism in hyperv_expand_features().

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 96 +++++++++++++++++++++++++++++++----------------
 1 file changed, 64 insertions(+), 32 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 2e9ceafa2421..760b93091fea 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1139,7 +1139,7 @@ static int hv_cpuid_get_fw(struct kvm_cpuid2 *cpuid, int fw, uint32_t *r)
 }
 
 static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
-                                  int feature)
+                                  int feature, Error **errp)
 {
     X86CPU *cpu = X86_CPU(cs);
     CPUX86State *env = &cpu->env;
@@ -1155,11 +1155,10 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
     while (deps) {
         dep_feat = ctz64(deps);
         if (!(hyperv_feat_enabled(cpu, dep_feat))) {
-                fprintf(stderr,
-                        "Hyper-V %s requires Hyper-V %s\n",
-                        kvm_hyperv_properties[feature].desc,
-                        kvm_hyperv_properties[dep_feat].desc);
-                return 1;
+            error_setg(errp, "Hyper-V %s requires Hyper-V %s",
+                       kvm_hyperv_properties[feature].desc,
+                       kvm_hyperv_properties[dep_feat].desc);
+            return 1;
         }
         deps &= ~(1ull << dep_feat);
     }
@@ -1174,9 +1173,8 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
 
         if (hv_cpuid_get_fw(cpuid, fw, &r) || (r & bits) != bits) {
             if (hyperv_feat_enabled(cpu, feature)) {
-                fprintf(stderr,
-                        "Hyper-V %s is not supported by kernel\n",
-                        kvm_hyperv_properties[feature].desc);
+                error_setg(errp, "Hyper-V %s is not supported by kernel",
+                           kvm_hyperv_properties[feature].desc);
                 return 1;
             } else {
                 return 0;
@@ -1200,13 +1198,13 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
  * of 'hv_passthrough' mode and fills the environment with all supported
  * Hyper-V features.
  */
-static int hyperv_expand_features(CPUState *cs)
+static int hyperv_expand_features(CPUState *cs, Error **errp)
 {
     X86CPU *cpu = X86_CPU(cs);
     CPUX86State *env = &cpu->env;
     struct kvm_cpuid2 *cpuid;
     struct kvm_cpuid_entry2 *c;
-    int r;
+    int r = 1;
 
     if (!hyperv_enabled(cpu))
         return 0;
@@ -1285,34 +1283,67 @@ static int hyperv_expand_features(CPUState *cs)
     }
 
     /* Features */
-    r = hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RELAXED);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VAPIC);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TIME);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_CRASH);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RESET);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VPINDEX);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RUNTIME);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_SYNIC);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_STIMER);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_FREQUENCIES);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_REENLIGHTENMENT);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TLBFLUSH);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_EVMCS);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_IPI);
-    r |= hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_STIMER_DIRECT);
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RELAXED, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VAPIC, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TIME, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_CRASH, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RESET, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_VPINDEX, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_RUNTIME, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_SYNIC, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_STIMER, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_FREQUENCIES, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_REENLIGHTENMENT, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_TLBFLUSH, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_EVMCS, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_IPI, errp)) {
+        goto out;
+    }
+    if (hv_cpuid_check_and_set(cs, cpuid, HYPERV_FEAT_STIMER_DIRECT, errp)) {
+        goto out;
+    }
 
     /* Additional dependencies not covered by kvm_hyperv_properties[] */
     if (hyperv_feat_enabled(cpu, HYPERV_FEAT_SYNIC) &&
         !cpu->hyperv_synic_kvm_only &&
         !hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX)) {
-        fprintf(stderr, "Hyper-V %s requires Hyper-V %s\n",
-                kvm_hyperv_properties[HYPERV_FEAT_SYNIC].desc,
-                kvm_hyperv_properties[HYPERV_FEAT_VPINDEX].desc);
-        r |= 1;
+        error_setg(errp, "Hyper-V %s requires Hyper-V %s",
+                   kvm_hyperv_properties[HYPERV_FEAT_SYNIC].desc,
+                   kvm_hyperv_properties[HYPERV_FEAT_VPINDEX].desc);
+        goto out;
     }
 
     /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
     env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
+    r = 0;
+
+out:
 
     g_free(cpuid);
 
@@ -1553,8 +1584,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
     env->apic_bus_freq = KVM_APIC_BUS_FREQUENCY;
 
     /* Paravirtualization CPUIDs */
-    r = hyperv_expand_features(cs);
-    if (r < 0) {
+    r = hyperv_expand_features(cs, &local_err);
+    if (local_err) {
+        error_report_err(local_err);
         return r;
     }
 
-- 
2.25.4

[PATCH RFC 16/22] i386: make hyperv_expand_features() return void

[Vitaly Kuznetsov]

As hyperv_expand_features() now uses standard error_setg() mechanism
we can make it return void.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 16 +++++-----------
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 760b93091fea..cb35f759acaa 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1198,16 +1198,15 @@ static int hv_cpuid_check_and_set(CPUState *cs, struct kvm_cpuid2 *cpuid,
  * of 'hv_passthrough' mode and fills the environment with all supported
  * Hyper-V features.
  */
-static int hyperv_expand_features(CPUState *cs, Error **errp)
+static void hyperv_expand_features(CPUState *cs, Error **errp)
 {
     X86CPU *cpu = X86_CPU(cs);
     CPUX86State *env = &cpu->env;
     struct kvm_cpuid2 *cpuid;
     struct kvm_cpuid_entry2 *c;
-    int r = 1;
 
     if (!hyperv_enabled(cpu))
-        return 0;
+        return;
 
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
         cpuid = get_supported_hv_cpuid(cs);
@@ -1341,17 +1340,12 @@ static int hyperv_expand_features(CPUState *cs, Error **errp)
 
     /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
     env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
-    r = 0;
 
 out:
 
     g_free(cpuid);
 
-    if (r) {
-        return -ENOSYS;
-    }
-
-    return 0;
+    return;
 }
 
 /*
@@ -1584,10 +1578,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
     env->apic_bus_freq = KVM_APIC_BUS_FREQUENCY;
 
     /* Paravirtualization CPUIDs */
-    r = hyperv_expand_features(cs, &local_err);
+    hyperv_expand_features(cs, &local_err);
     if (local_err) {
         error_report_err(local_err);
-        return r;
+        return -ENOSYS;
     }
 
     if (hyperv_enabled(cpu)) {
-- 
2.25.4

[PATCH RFC 17/22] i386: adjust the expected KVM_GET_SUPPORTED_HV_CPUID array size

[Vitaly Kuznetsov]

SYNDBG leaves were recently (Linux-5.8) added to KVM but we haven't
updated the expected size of KVM_GET_SUPPORTED_HV_CPUID output in
KVM so we now make serveral tries before succeeding. Update the
default.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index cb35f759acaa..70db2ca26313 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -973,7 +973,8 @@ static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max)
 static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
 {
     struct kvm_cpuid2 *cpuid;
-    int max = 7; /* 0x40000000..0x40000005, 0x4000000A */
+    /* 0x40000000..0x40000005, 0x4000000A, 0x40000080..0x40000080 leaves */
+    int max = 10;
     int i;
 
     /*
-- 
2.25.4

[PATCH RFC 18/22] i386: prefer system KVM_GET_SUPPORTED_HV_CPUID ioctl over vCPU's one

[Vitaly Kuznetsov]

KVM_GET_SUPPORTED_HV_CPUID was made a system wide ioctl which can be called
prior to creating vCPUs and we are going to use that to expand Hyper-V cpu
features early. Use it when it is supported by KVM.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 70db2ca26313..b76d4a5a147b 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -940,7 +940,8 @@ static struct {
     },
 };
 
-static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max)
+static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max,
+                                           bool do_sys_ioctl)
 {
     struct kvm_cpuid2 *cpuid;
     int r, size;
@@ -949,7 +950,11 @@ static struct kvm_cpuid2 *try_get_hv_cpuid(CPUState *cs, int max)
     cpuid = g_malloc0(size);
     cpuid->nent = max;
 
-    r = kvm_vcpu_ioctl(cs, KVM_GET_SUPPORTED_HV_CPUID, cpuid);
+    if (do_sys_ioctl) {
+        r = kvm_ioctl(kvm_state, KVM_GET_SUPPORTED_HV_CPUID, cpuid);
+    } else {
+        r = kvm_vcpu_ioctl(cs, KVM_GET_SUPPORTED_HV_CPUID, cpuid);
+    }
     if (r == 0 && cpuid->nent >= max) {
         r = -E2BIG;
     }
@@ -976,13 +981,17 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
     /* 0x40000000..0x40000005, 0x4000000A, 0x40000080..0x40000080 leaves */
     int max = 10;
     int i;
+    bool do_sys_ioctl;
+
+    do_sys_ioctl =
+        kvm_check_extension(kvm_state, KVM_CAP_SYS_HYPERV_CPUID) > 0;
 
     /*
      * When the buffer is too small, KVM_GET_SUPPORTED_HV_CPUID fails with
      * -E2BIG, however, it doesn't report back the right size. Keep increasing
      * it and re-trying until we succeed.
      */
-    while ((cpuid = try_get_hv_cpuid(cs, max)) == NULL) {
+    while ((cpuid = try_get_hv_cpuid(cs, max, do_sys_ioctl)) == NULL) {
         max++;
     }
 
@@ -992,7 +1001,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
      * information early, just check for the capability and set the bit
      * manually.
      */
-    if (kvm_check_extension(cs->kvm_state,
+    if (!do_sys_ioctl && kvm_check_extension(cs->kvm_state,
                             KVM_CAP_HYPERV_ENLIGHTENED_VMCS) > 0) {
         for (i = 0; i < cpuid->nent; i++) {
             if (cpuid->entries[i].function == HV_CPUID_ENLIGHTMENT_INFO) {
-- 
2.25.4

[PATCH RFC 19/22] i386: prepare hyperv_expand_features() to be called at CPU feature expansion time

[Vitaly Kuznetsov]

If hyperv_expand_features() is called before vCPU is created, cs->kvm_state
is NULL. We can only do the expantion if system wide KVM_GET_SUPPORTED_HV_CPUID
is supported.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b76d4a5a147b..c17e7db5e627 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -986,6 +986,10 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid(CPUState *cs)
     do_sys_ioctl =
         kvm_check_extension(kvm_state, KVM_CAP_SYS_HYPERV_CPUID) > 0;
 
+    if (!do_sys_ioctl && !cs->kvm_state) {
+        return NULL;
+    }
+
     /*
      * When the buffer is too small, KVM_GET_SUPPORTED_HV_CPUID fails with
      * -E2BIG, however, it doesn't report back the right size. Keep increasing
@@ -1023,6 +1027,10 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)
     struct kvm_cpuid2 *cpuid;
     struct kvm_cpuid_entry2 *entry_feat, *entry_recomm;
 
+    if (!cs->kvm_state) {
+        return NULL;
+    }
+
     /* HV_CPUID_FEATURES, HV_CPUID_ENLIGHTMENT_INFO */
     cpuid = g_malloc0(sizeof(*cpuid) + 2 * sizeof(*cpuid->entries));
     cpuid->nent = 2;
@@ -1218,12 +1226,15 @@ static void hyperv_expand_features(CPUState *cs, Error **errp)
     if (!hyperv_enabled(cpu))
         return;
 
-    if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
+    if (kvm_check_extension(kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
         cpuid = get_supported_hv_cpuid(cs);
     } else {
         cpuid = get_supported_hv_cpuid_legacy(cs);
     }
 
+    if (!cpuid)
+        return;
+
     if (cpu->hyperv_passthrough) {
         c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
         if (c) {
-- 
2.25.4

[PATCH RFC 20/22] i386: use global kvm_state in hyperv_enabled() check

[Vitaly Kuznetsov]

There is no need to use vCPU-specific kvm state in hyperv_enabled() check
and we need to do that when feature expansion happens early, before vCPU
specific KVM state is created.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/kvm.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index c17e7db5e627..0945983498b2 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -728,8 +728,7 @@ unsigned long kvm_arch_vcpu_id(CPUState *cs)
 
 static bool hyperv_enabled(X86CPU *cpu)
 {
-    CPUState *cs = CPU(cpu);
-    return kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0 &&
+    return kvm_check_extension(kvm_state, KVM_CAP_HYPERV) > 0 &&
         ((cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY) ||
          cpu->hyperv_features || cpu->hyperv_passthrough);
 }
-- 
2.25.4

[PATCH RFC 21/22] i386: record if Hyper-V features were already expanded

[Vitaly Kuznetsov]

Introduce a per cpu hyperv_features_expaned flag indicating that Hyper-V
related CPUIDs were already acquired from KVM. We are going to start
doing the expansion prior to creating KVM vCPU but in case KVM doesn't
support sustem wide KVM_GET_SUPPORTED_HV_CPUID ioctl this can't happen.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.h | 1 +
 target/i386/kvm.c | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0aad60e0c707..c72755139047 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1669,6 +1669,7 @@ struct X86CPU {
     uint32_t hyperv_interface_id[4];
     uint32_t hyperv_version_id[4];
     uint32_t hyperv_limits[3];
+    bool hyperv_features_expaned;
 
     bool check_cpuid;
     bool enforce_cpuid;
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 0945983498b2..961241528a5c 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1225,6 +1225,10 @@ static void hyperv_expand_features(CPUState *cs, Error **errp)
     if (!hyperv_enabled(cpu))
         return;
 
+    if (cpu->hyperv_features_expaned) {
+        return;
+    }
+
     if (kvm_check_extension(kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
         cpuid = get_supported_hv_cpuid(cs);
     } else {
@@ -1361,6 +1365,8 @@ static void hyperv_expand_features(CPUState *cs, Error **errp)
     /* Not exposed by KVM but needed to make CPU hotplug in Windows work */
     env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
 
+    cpu->hyperv_features_expaned = true;
+
 out:
 
     g_free(cpuid);
-- 
2.25.4

[PATCH RFC 22/22] i386: expand Hyper-V features early

[Vitaly Kuznetsov]

To make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we
need to expand and set the corresponding CPUID leaves early. Modify
x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V
specific kvm_hv_get_supported_cpuid() instead of
kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid()
as Hyper-V specific CPUID leaves intersect with KVM's.

Note, early expansion will only happen when KVM supports system wide
KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 target/i386/cpu.c      | 15 +++++++++------
 target/i386/kvm.c      | 15 +++++++++++++++
 target/i386/kvm_i386.h |  7 +++++++
 3 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 479c4bbbf459..d3c4ecb3535c 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5147,7 +5147,7 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
     return cpu_list;
 }
 
-static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
+static uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w,
                                                    bool migratable_only)
 {
     FeatureWordInfo *wi = &feature_word_info[w];
@@ -5156,9 +5156,12 @@ static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
     if (kvm_enabled()) {
         switch (wi->type) {
         case CPUID_FEATURE_WORD:
-            r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
-                                                        wi->cpuid.ecx,
-                                                        wi->cpuid.reg);
+            if (hyperv_feature_word(w))
+                r = kvm_hv_get_supported_cpuid(cpu, w);
+            else
+                r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
+                                                 wi->cpuid.ecx,
+                                                 wi->cpuid.reg);
             break;
         case MSR_FEATURE_WORD:
             r = kvm_arch_get_supported_msr_feature(kvm_state,
@@ -6485,7 +6488,7 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
              * by the user.
              */
             env->features[w] |=
-                x86_cpu_get_supported_feature_word(w, cpu->migratable) &
+                x86_cpu_get_supported_feature_word(cpu, w, cpu->migratable) &
                 ~env->user_features[w] &
                 ~feature_word_info[w].no_autoenable_flags;
         }
@@ -6589,7 +6592,7 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool verbose)
 
     for (w = 0; w < FEATURE_WORDS; w++) {
         uint64_t host_feat =
-            x86_cpu_get_supported_feature_word(w, false);
+            x86_cpu_get_supported_feature_word(cpu, w, false);
         uint64_t requested_features = env->features[w];
         uint64_t unavailable_features = requested_features & ~host_feat;
         mark_unavailable_features(cpu, w, unavailable_features, prefix);
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 961241528a5c..764b96fbbb7d 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1449,6 +1449,21 @@ static int hyperv_fill_cpuids(CPUState *cs,
     return cpuid_i;
 }
 
+uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w)
+{
+    CPUState *cs = CPU(cpu);
+    CPUX86State *env = &cpu->env;
+    Error *local_err = NULL;
+
+    hyperv_expand_features(cs, &local_err);
+
+    if (local_err) {
+            error_report_err(local_err);
+    }
+
+    return env->features[w];
+}
+
 static Error *hv_passthrough_mig_blocker;
 static Error *hv_no_nonarch_cs_mig_blocker;
 
diff --git a/target/i386/kvm_i386.h b/target/i386/kvm_i386.h
index 064b8798a26c..2e7da4f39668 100644
--- a/target/i386/kvm_i386.h
+++ b/target/i386/kvm_i386.h
@@ -48,4 +48,11 @@ bool kvm_has_waitpkg(void);
 
 bool kvm_hv_vpindex_settable(void);
 
+static inline bool hyperv_feature_word(enum FeatureWord w)
+{
+    return w >= FEAT_HYPERV_EAX && w <= FEAT_HV_NESTED_EDX;
+}
+
+uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w);
+
 #endif
-- 
2.25.4

Re: [PATCH RFC 03/22] i386: move hyperv_vendor_id initialization to x86_cpu_realizefn()

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:12PM +0200, Vitaly Kuznetsov wrote:
> As a preparation to expanding Hyper-V CPU features early, move
> hyperv_vendor_id initialization to x86_cpu_realizefn().
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/cpu.c | 15 ++++++++++++++-
>  target/i386/cpu.h |  3 ++-
>  target/i386/kvm.c | 25 ++++++++++---------------
>  3 files changed, 26 insertions(+), 17 deletions(-)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 14489def2177..07e9da9e567e 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -6625,6 +6625,19 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>          }
>      }
>  
> +    if (!cpu->hyperv_vendor) {
> +        memcpy(cpu->hyperv_vendor_id, "Microsoft Hv", 12);
> +    } else {
> +        size_t len = strlen(cpu->hyperv_vendor);
> +
> +        if (len > 12) {
> +            warn_report("hv-vendor-id truncated to 12 characters");
> +            len = 12;
> +        }
> +        memset(cpu->hyperv_vendor_id, 0, 12);
> +        memcpy(cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);
> +    }
> +

The change makes sense, but considering that we'll have a lot of
new code added to x86_cpu_realizefn(), I would prefer to create a
separate x86_cpu_hyperv_realize() function to make
x86_cpu_realizefn() a bit more readable.


>      if (cpu->ucode_rev == 0) {
>          /* The default is the same as KVM's.  */
>          if (IS_AMD_CPU(env)) {
> @@ -7313,7 +7326,7 @@ static Property x86_cpu_properties[] = {
>      DEFINE_PROP_UINT32("min-xlevel2", X86CPU, env.cpuid_min_xlevel2, 0),
>      DEFINE_PROP_UINT64("ucode-rev", X86CPU, ucode_rev, 0),
>      DEFINE_PROP_BOOL("full-cpuid-auto-level", X86CPU, full_cpuid_auto_level, true),
> -    DEFINE_PROP_STRING("hv-vendor-id", X86CPU, hyperv_vendor_id),
> +    DEFINE_PROP_STRING("hv-vendor-id", X86CPU, hyperv_vendor),
>      DEFINE_PROP_BOOL("cpuid-0xb", X86CPU, enable_cpuid_0xb, true),
>      DEFINE_PROP_BOOL("lmce", X86CPU, enable_lmce, false),
>      DEFINE_PROP_BOOL("l3-cache", X86CPU, enable_l3_cache, true),
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index d3097be6a50a..903994818093 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -1654,11 +1654,12 @@ struct X86CPU {
>      uint64_t ucode_rev;
>  
>      uint32_t hyperv_spinlock_attempts;
> -    char *hyperv_vendor_id;
> +    char *hyperv_vendor;
>      bool hyperv_synic_kvm_only;
>      uint64_t hyperv_features;
>      bool hyperv_passthrough;
>      OnOffAuto hyperv_no_nonarch_cs;
> +    uint32_t hyperv_vendor_id[3];
>  
>      bool check_cpuid;
>      bool enforce_cpuid;
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 205b68bc0ce8..47779c5e1efd 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1225,6 +1225,13 @@ static int hyperv_handle_properties(CPUState *cs,
>          memcpy(cpuid_ent, &cpuid->entries[0],
>                 cpuid->nent * sizeof(cpuid->entries[0]));
>  
> +        c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
> +        if (c) {
> +            cpu->hyperv_vendor_id[0] = c->ebx;
> +            cpu->hyperv_vendor_id[1] = c->ecx;
> +            cpu->hyperv_vendor_id[2] = c->edx;
> +        }
> +

I can't find the equivalent of this code in the current tree?  Is
hyperv vendor ID broken when using hv-passthrough today?

Maybe this could be done as a separate patch, as it changes
behavior of hv-passthrough?

>          c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
>          if (c) {
>              env->features[FEAT_HYPERV_EAX] = c->eax;
> @@ -1299,23 +1306,11 @@ static int hyperv_handle_properties(CPUState *cs,
>  
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
> -    if (!cpu->hyperv_vendor_id) {
> -        memcpy(signature, "Microsoft Hv", 12);
> -    } else {
> -        size_t len = strlen(cpu->hyperv_vendor_id);
> -
> -        if (len > 12) {
> -            error_report("hv-vendor-id truncated to 12 characters");
> -            len = 12;
> -        }
> -        memset(signature, 0, 12);
> -        memcpy(signature, cpu->hyperv_vendor_id, len);
> -    }
>      c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
>          HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
> -    c->ebx = signature[0];
> -    c->ecx = signature[1];
> -    c->edx = signature[2];
> +    c->ebx = cpu->hyperv_vendor_id[0];
> +    c->ecx = cpu->hyperv_vendor_id[1];
> +    c->edx = cpu->hyperv_vendor_id[2];
>  
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_INTERFACE;
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 05/22] i386: move hyperv_version_id initialization to x86_cpu_realizefn()

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:14PM +0200, Vitaly Kuznetsov wrote:
> As a preparation to expanding Hyper-V CPU features early, move
> hyperv_version_id initialization to x86_cpu_realizefn().
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/cpu.c |  4 ++++
>  target/i386/cpu.h |  1 +
>  target/i386/kvm.c | 14 ++++++++++++--
>  3 files changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 16888125a30a..e605399eb8c0 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -6644,6 +6644,10 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>      cpu->hyperv_interface_id[2] = 0;
>      cpu->hyperv_interface_id[3] = 0;
>  
> +    /* Hypervisor system identity */
> +    cpu->hyperv_version_id[0] = 0x00001bbc;
> +    cpu->hyperv_version_id[1] = 0x00060001;
> +
>      if (cpu->ucode_rev == 0) {
>          /* The default is the same as KVM's.  */
>          if (IS_AMD_CPU(env)) {
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 91edc54a268c..2630ffd2d4b2 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -1661,6 +1661,7 @@ struct X86CPU {
>      OnOffAuto hyperv_no_nonarch_cs;
>      uint32_t hyperv_vendor_id[3];
>      uint32_t hyperv_interface_id[4];
> +    uint32_t hyperv_version_id[4];
>  
>      bool check_cpuid;
>      bool enforce_cpuid;
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index a36c65100cd0..169bae2779a4 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1239,6 +1239,14 @@ static int hyperv_handle_properties(CPUState *cs,
>              cpu->hyperv_interface_id[3] = c->edx;
>          }
>  
> +        c = cpuid_find_entry(cpuid, HV_CPUID_VERSION, 0);
> +        if (c) {
> +            cpu->hyperv_version_id[0] = c->eax;
> +            cpu->hyperv_version_id[1] = c->ebx;
> +            cpu->hyperv_version_id[2] = c->ecx;
> +            cpu->hyperv_version_id[3] = c->edx;
> +        }
> +

Same questions I had for vendor id: is hyperv_version_id broken
when using hv-passthrough today?  Should this go to a separate
patch?

>          c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
>          if (c) {
>              env->features[FEAT_HYPERV_EAX] = c->eax;
> @@ -1328,8 +1336,10 @@ static int hyperv_handle_properties(CPUState *cs,
>  
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_VERSION;
> -    c->eax = 0x00001bbc;
> -    c->ebx = 0x00060001;
> +    c->eax = cpu->hyperv_version_id[0];
> +    c->ebx = cpu->hyperv_version_id[1];
> +    c->ecx = cpu->hyperv_version_id[2];
> +    c->edx = cpu->hyperv_version_id[3];
>  
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_FEATURES;
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 06/22] i386: move hyperv_limits initialization to x86_cpu_realizefn()

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:15PM +0200, Vitaly Kuznetsov wrote:
> As a preparation to expanding Hyper-V CPU features early, move
> hyperv_limits initialization to x86_cpu_realizefn().
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/cpu.c |  5 +++++
>  target/i386/cpu.h |  1 +
>  target/i386/kvm.c | 13 ++++++++++++-
>  3 files changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index e605399eb8c0..ef3c672cf415 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -6648,6 +6648,11 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>      cpu->hyperv_version_id[0] = 0x00001bbc;
>      cpu->hyperv_version_id[1] = 0x00060001;
>  
> +    /* Hypervisor implementation limits */
> +    cpu->hyperv_limits[0] = 64;
> +    cpu->hyperv_limits[1] = 0;
> +    cpu->hyperv_limits[2] = 0;
> +
>      if (cpu->ucode_rev == 0) {
>          /* The default is the same as KVM's.  */
>          if (IS_AMD_CPU(env)) {
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 2630ffd2d4b2..095d0bf75493 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -1662,6 +1662,7 @@ struct X86CPU {
>      uint32_t hyperv_vendor_id[3];
>      uint32_t hyperv_interface_id[4];
>      uint32_t hyperv_version_id[4];
> +    uint32_t hyperv_limits[3];
>  
>      bool check_cpuid;
>      bool enforce_cpuid;
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 169bae2779a4..720c30e9df17 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1253,6 +1253,15 @@ static int hyperv_handle_properties(CPUState *cs,
>              env->features[FEAT_HYPERV_EBX] = c->ebx;
>              env->features[FEAT_HYPERV_EDX] = c->eax;
>          }
> +
> +        c = cpuid_find_entry(cpuid, HV_CPUID_IMPLEMENT_LIMITS, 0);
> +        if (c) {
> +            cpu->hv_max_vps = c->eax;
> +            cpu->hyperv_limits[0] = c->ebx;
> +            cpu->hyperv_limits[1] = c->ecx;
> +            cpu->hyperv_limits[2] = c->edx;
> +        }
> +

Sam question I had for vendor id and version id: should it go to
a separate patch as it changes behavior of hv-passthrough?

>          c = cpuid_find_entry(cpuid, HV_CPUID_ENLIGHTMENT_INFO, 0);
>          if (c) {
>              env->features[FEAT_HV_RECOMM_EAX] = c->eax;
> @@ -1355,7 +1364,9 @@ static int hyperv_handle_properties(CPUState *cs,
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_IMPLEMENT_LIMITS;
>      c->eax = cpu->hv_max_vps;
> -    c->ebx = 0x40;
> +    c->ebx = cpu->hyperv_limits[0];
> +    c->ecx = cpu->hyperv_limits[1];
> +    c->edx = cpu->hyperv_limits[2];
>  
>      if (hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS)) {
>          __u32 function;
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 07/22] i386: fill in FEAT_HYPERV_EDX from edx instead of eax

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:16PM +0200, Vitaly Kuznetsov wrote:
> There was a typo which went unnoticed.
> 
> Fixes: e48ddcc6ce13 ("i386/kvm: implement 'hv-passthrough' mode")
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>

Similar fix had been submitted at:
https://lore.kernel.org/qemu-devel/20190820103030.12515-1-zhenyuw@linux.intel.com/

> ---
>  target/i386/kvm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 720c30e9df17..1cb5592d4f7d 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1251,7 +1251,7 @@ static int hyperv_handle_properties(CPUState *cs,
>          if (c) {
>              env->features[FEAT_HYPERV_EAX] = c->eax;
>              env->features[FEAT_HYPERV_EBX] = c->ebx;
> -            env->features[FEAT_HYPERV_EDX] = c->eax;
> +            env->features[FEAT_HYPERV_EDX] = c->edx;
>          }
>  
>          c = cpuid_find_entry(cpuid, HV_CPUID_IMPLEMENT_LIMITS, 0);
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 04/22] i386: move hyperv_interface_id initialization to x86_cpu_realizefn()

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:13PM +0200, Vitaly Kuznetsov wrote:
> As a preparation to expanding Hyper-V CPU features early, move
> hyperv_interface_id initialization to x86_cpu_realizefn().
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/cpu.c |  6 ++++++
>  target/i386/cpu.h |  1 +
>  target/i386/kvm.c | 18 ++++++++++++------
>  3 files changed, 19 insertions(+), 6 deletions(-)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 07e9da9e567e..16888125a30a 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -6638,6 +6638,12 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>          memcpy(cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);
>      }
>  
> +    /* 'Hv#1' interface identification*/
> +    cpu->hyperv_interface_id[0] = 0x31237648;
> +    cpu->hyperv_interface_id[1] = 0;
> +    cpu->hyperv_interface_id[2] = 0;
> +    cpu->hyperv_interface_id[3] = 0;
> +
>      if (cpu->ucode_rev == 0) {
>          /* The default is the same as KVM's.  */
>          if (IS_AMD_CPU(env)) {
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 903994818093..91edc54a268c 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -1660,6 +1660,7 @@ struct X86CPU {
>      bool hyperv_passthrough;
>      OnOffAuto hyperv_no_nonarch_cs;
>      uint32_t hyperv_vendor_id[3];
> +    uint32_t hyperv_interface_id[4];
>  
>      bool check_cpuid;
>      bool enforce_cpuid;
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 47779c5e1efd..a36c65100cd0 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1188,7 +1188,6 @@ static int hyperv_handle_properties(CPUState *cs,
>      CPUX86State *env = &cpu->env;
>      struct kvm_cpuid2 *cpuid;
>      struct kvm_cpuid_entry2 *c;
> -    uint32_t signature[3];
>      uint32_t cpuid_i = 0;
>      int r;
>  
> @@ -1232,6 +1231,14 @@ static int hyperv_handle_properties(CPUState *cs,
>              cpu->hyperv_vendor_id[2] = c->edx;
>          }
>  
> +        c = cpuid_find_entry(cpuid, HV_CPUID_INTERFACE, 0);
> +        if (c) {
> +            cpu->hyperv_interface_id[0] = c->eax;
> +            cpu->hyperv_interface_id[1] = c->ebx;
> +            cpu->hyperv_interface_id[2] = c->ecx;
> +            cpu->hyperv_interface_id[3] = c->edx;
> +        }
> +

Same question I sent for vendor id, version id, hyperv_limits:
should this go to a separate patch, as it changes behavior of
hv-passthrough?

>          c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
>          if (c) {
>              env->features[FEAT_HYPERV_EAX] = c->eax;
> @@ -1314,11 +1321,10 @@ static int hyperv_handle_properties(CPUState *cs,
>  
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_INTERFACE;
> -    memcpy(signature, "Hv#1\0\0\0\0\0\0\0\0", 12);
> -    c->eax = signature[0];
> -    c->ebx = 0;
> -    c->ecx = 0;
> -    c->edx = 0;
> +    c->eax = cpu->hyperv_interface_id[0];
> +    c->ebx = cpu->hyperv_interface_id[1];
> +    c->ecx = cpu->hyperv_interface_id[2];
> +    c->edx = cpu->hyperv_interface_id[3];
>  
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_VERSION;
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 09/22] i386: add reserved FEAT_HYPERV_ECX CPUID leaf

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:18PM +0200, Vitaly Kuznetsov wrote:
> As a preparation to expanding Hyper-V CPU features early, add
> reserved FEAT_HYPERV_ECX CPUID leaf.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/cpu.c | 5 +++++
>  target/i386/cpu.h | 1 +
>  target/i386/kvm.c | 2 ++
>  3 files changed, 8 insertions(+)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index ef3c672cf415..70588571ccb1 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -974,6 +974,11 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
>          },
>          .cpuid = { .eax = 0x40000003, .reg = R_EBX, },
>      },
> +    [FEAT_HYPERV_ECX] = {
> +        .type = CPUID_FEATURE_WORD,
> +        /* reserved */

What does "reserved" mean here?

> +        .cpuid = { .eax = 0x40000003, .reg = R_ECX, },
> +    },
>      [FEAT_HYPERV_EDX] = {
>          .type = CPUID_FEATURE_WORD,
>          .feat_names = {
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 095d0bf75493..39e0e89aa41f 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -521,6 +521,7 @@ typedef enum FeatureWord {
>      FEAT_KVM_HINTS,     /* CPUID[4000_0001].EDX */
>      FEAT_HYPERV_EAX,    /* CPUID[4000_0003].EAX */
>      FEAT_HYPERV_EBX,    /* CPUID[4000_0003].EBX */
> +    FEAT_HYPERV_ECX,    /* CPUID[4000_0003].ECX */
>      FEAT_HYPERV_EDX,    /* CPUID[4000_0003].EDX */
>      FEAT_HV_RECOMM_EAX, /* CPUID[4000_0004].EAX */
>      FEAT_HV_NESTED_EAX, /* CPUID[4000_000A].EAX */
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 96ac719adca1..259be2d514dd 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1251,6 +1251,7 @@ static int hyperv_handle_properties(CPUState *cs,
>          if (c) {
>              env->features[FEAT_HYPERV_EAX] = c->eax;
>              env->features[FEAT_HYPERV_EBX] = c->ebx;
> +            env->features[FEAT_HYPERV_ECX] = c->ecx;
>              env->features[FEAT_HYPERV_EDX] = c->edx;
>          }
>  
> @@ -1350,6 +1351,7 @@ static int hyperv_handle_properties(CPUState *cs,
>      c->function = HV_CPUID_FEATURES;
>      c->eax = env->features[FEAT_HYPERV_EAX];
>      c->ebx = env->features[FEAT_HYPERV_EBX];
> +    c->ecx = env->features[FEAT_HYPERV_ECX];
>      c->edx = env->features[FEAT_HYPERV_EDX];
>  
>      c = &cpuid_ent[cpuid_i++];
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 12/22] i386: always fill Hyper-V CPUID feature leaves from X86CPU data

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:21PM +0200, Vitaly Kuznetsov wrote:
> We have all the required data in X86CPU already and as we are about to
> split hyperv_handle_properties() into hyperv_expand_features()/
> hyperv_fill_cpuids() we can remove the blind copy. The functional change
> is that QEMU won't pass CPUID leaves it doesn't currently know about
> to the guest but arguably this is a good change.

Is it, though?  Maybe we don't want it to be the default, but a
blind passthrough mode that copies all CPUID leaves would be
useful for testing new features.

> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/kvm.c | 9 ---------
>  1 file changed, 9 deletions(-)
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 87b83a2aa2cb..b7f0e200a75f 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1221,9 +1221,6 @@ static int hyperv_handle_properties(CPUState *cs,
>      }
>  
>      if (cpu->hyperv_passthrough) {
> -        memcpy(cpuid_ent, &cpuid->entries[0],
> -               cpuid->nent * sizeof(cpuid->entries[0]));
> -
>          c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
>          if (c) {
>              cpu->hyperv_vendor_id[0] = c->ebx;
> @@ -1325,12 +1322,6 @@ static int hyperv_handle_properties(CPUState *cs,
>          goto free;
>      }
>  
> -    if (cpu->hyperv_passthrough) {
> -        /* We already copied all feature words from KVM as is */
> -        r = cpuid->nent;
> -        goto free;
> -    }
> -
>      c = &cpuid_ent[cpuid_i++];
>      c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
>      c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 22/22] i386: expand Hyper-V features early

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:31PM +0200, Vitaly Kuznetsov wrote:
> To make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we
> need to expand and set the corresponding CPUID leaves early. Modify
> x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V
> specific kvm_hv_get_supported_cpuid() instead of
> kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid()
> as Hyper-V specific CPUID leaves intersect with KVM's.
> 
> Note, early expansion will only happen when KVM supports system wide
> KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).

Will this implicitly enable hyperv CPUID passthrough when using
"-cpu host"?  Do we want it to?

> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  target/i386/cpu.c      | 15 +++++++++------
>  target/i386/kvm.c      | 15 +++++++++++++++
>  target/i386/kvm_i386.h |  7 +++++++
>  3 files changed, 31 insertions(+), 6 deletions(-)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 479c4bbbf459..d3c4ecb3535c 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -5147,7 +5147,7 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
>      return cpu_list;
>  }
>  
> -static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
> +static uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w,
>                                                     bool migratable_only)
>  {
>      FeatureWordInfo *wi = &feature_word_info[w];
> @@ -5156,9 +5156,12 @@ static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
>      if (kvm_enabled()) {
>          switch (wi->type) {
>          case CPUID_FEATURE_WORD:
> -            r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
> -                                                        wi->cpuid.ecx,
> -                                                        wi->cpuid.reg);
> +            if (hyperv_feature_word(w))
> +                r = kvm_hv_get_supported_cpuid(cpu, w);
> +            else
> +                r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
> +                                                 wi->cpuid.ecx,
> +                                                 wi->cpuid.reg);
>              break;
>          case MSR_FEATURE_WORD:
>              r = kvm_arch_get_supported_msr_feature(kvm_state,
> @@ -6485,7 +6488,7 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
>               * by the user.
>               */
>              env->features[w] |=
> -                x86_cpu_get_supported_feature_word(w, cpu->migratable) &
> +                x86_cpu_get_supported_feature_word(cpu, w, cpu->migratable) &
>                  ~env->user_features[w] &
>                  ~feature_word_info[w].no_autoenable_flags;
>          }
> @@ -6589,7 +6592,7 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool verbose)
>  
>      for (w = 0; w < FEATURE_WORDS; w++) {
>          uint64_t host_feat =
> -            x86_cpu_get_supported_feature_word(w, false);
> +            x86_cpu_get_supported_feature_word(cpu, w, false);
>          uint64_t requested_features = env->features[w];
>          uint64_t unavailable_features = requested_features & ~host_feat;
>          mark_unavailable_features(cpu, w, unavailable_features, prefix);
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 961241528a5c..764b96fbbb7d 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -1449,6 +1449,21 @@ static int hyperv_fill_cpuids(CPUState *cs,
>      return cpuid_i;
>  }
>  
> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w)
> +{
> +    CPUState *cs = CPU(cpu);
> +    CPUX86State *env = &cpu->env;
> +    Error *local_err = NULL;
> +
> +    hyperv_expand_features(cs, &local_err);
> +
> +    if (local_err) {
> +            error_report_err(local_err);
> +    }
> +
> +    return env->features[w];
> +}
> +
>  static Error *hv_passthrough_mig_blocker;
>  static Error *hv_no_nonarch_cs_mig_blocker;
>  
> diff --git a/target/i386/kvm_i386.h b/target/i386/kvm_i386.h
> index 064b8798a26c..2e7da4f39668 100644
> --- a/target/i386/kvm_i386.h
> +++ b/target/i386/kvm_i386.h
> @@ -48,4 +48,11 @@ bool kvm_has_waitpkg(void);
>  
>  bool kvm_hv_vpindex_settable(void);
>  
> +static inline bool hyperv_feature_word(enum FeatureWord w)
> +{
> +    return w >= FEAT_HYPERV_EAX && w <= FEAT_HV_NESTED_EDX;
> +}
> +
> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w);
> +
>  #endif
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 22/22] i386: expand Hyper-V features early

[Eduardo Habkost]

On Fri, Sep 04, 2020 at 04:54:31PM +0200, Vitaly Kuznetsov wrote:
> To make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we
> need to expand and set the corresponding CPUID leaves early. Modify
> x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V
> specific kvm_hv_get_supported_cpuid() instead of
> kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid()
> as Hyper-V specific CPUID leaves intersect with KVM's.
> 
> Note, early expansion will only happen when KVM supports system wide
> KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
[...]
> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w)
> +{
> +    CPUState *cs = CPU(cpu);
> +    CPUX86State *env = &cpu->env;
> +    Error *local_err = NULL;
> +
> +    hyperv_expand_features(cs, &local_err);

This makes a function that sounds like it doesn't have any side
effect ("get supported cpuid") have an unintended side effect
(hyperv_expand_features() will change all CPUID data inside the
cpu object).

What about making it more similar to
kvm_arch_get_supported_cpuid(), and be just a wrapper to
get_supported_hv_cpuid()?

I would also make sure get_supported_hv_cpuid() doesn't get
CPUState as argument, just to be sure it will never touch the CPU
object state.

> +
> +    if (local_err) {
> +            error_report_err(local_err);
> +    }
> +
> +    return env->features[w];
> +}
> +
>  static Error *hv_passthrough_mig_blocker;
>  static Error *hv_no_nonarch_cs_mig_blocker;
>  
> diff --git a/target/i386/kvm_i386.h b/target/i386/kvm_i386.h
> index 064b8798a26c..2e7da4f39668 100644
> --- a/target/i386/kvm_i386.h
> +++ b/target/i386/kvm_i386.h
> @@ -48,4 +48,11 @@ bool kvm_has_waitpkg(void);
>  
>  bool kvm_hv_vpindex_settable(void);
>  
> +static inline bool hyperv_feature_word(enum FeatureWord w)
> +{
> +    return w >= FEAT_HYPERV_EAX && w <= FEAT_HV_NESTED_EDX;
> +}
> +
> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w);
> +
>  #endif
> -- 
> 2.25.4
> 

-- 
Eduardo

Re: [PATCH RFC 03/22] i386: move hyperv_vendor_id initialization to x86_cpu_realizefn()

[Vitaly Kuznetsov]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Sep 04, 2020 at 04:54:12PM +0200, Vitaly Kuznetsov wrote:
>> As a preparation to expanding Hyper-V CPU features early, move
>> hyperv_vendor_id initialization to x86_cpu_realizefn().
>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  target/i386/cpu.c | 15 ++++++++++++++-
>>  target/i386/cpu.h |  3 ++-
>>  target/i386/kvm.c | 25 ++++++++++---------------
>>  3 files changed, 26 insertions(+), 17 deletions(-)
>> 
>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>> index 14489def2177..07e9da9e567e 100644
>> --- a/target/i386/cpu.c
>> +++ b/target/i386/cpu.c
>> @@ -6625,6 +6625,19 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>>          }
>>      }
>>  
>> +    if (!cpu->hyperv_vendor) {
>> +        memcpy(cpu->hyperv_vendor_id, "Microsoft Hv", 12);
>> +    } else {
>> +        size_t len = strlen(cpu->hyperv_vendor);
>> +
>> +        if (len > 12) {
>> +            warn_report("hv-vendor-id truncated to 12 characters");
>> +            len = 12;
>> +        }
>> +        memset(cpu->hyperv_vendor_id, 0, 12);
>> +        memcpy(cpu->hyperv_vendor_id, cpu->hyperv_vendor, len);
>> +    }
>> +
>
> The change makes sense, but considering that we'll have a lot of
> new code added to x86_cpu_realizefn(), I would prefer to create a
> separate x86_cpu_hyperv_realize() function to make
> x86_cpu_realizefn() a bit more readable.
>

Agreed.

>
>>      if (cpu->ucode_rev == 0) {
>>          /* The default is the same as KVM's.  */
>>          if (IS_AMD_CPU(env)) {
>> @@ -7313,7 +7326,7 @@ static Property x86_cpu_properties[] = {
>>      DEFINE_PROP_UINT32("min-xlevel2", X86CPU, env.cpuid_min_xlevel2, 0),
>>      DEFINE_PROP_UINT64("ucode-rev", X86CPU, ucode_rev, 0),
>>      DEFINE_PROP_BOOL("full-cpuid-auto-level", X86CPU, full_cpuid_auto_level, true),
>> -    DEFINE_PROP_STRING("hv-vendor-id", X86CPU, hyperv_vendor_id),
>> +    DEFINE_PROP_STRING("hv-vendor-id", X86CPU, hyperv_vendor),
>>      DEFINE_PROP_BOOL("cpuid-0xb", X86CPU, enable_cpuid_0xb, true),
>>      DEFINE_PROP_BOOL("lmce", X86CPU, enable_lmce, false),
>>      DEFINE_PROP_BOOL("l3-cache", X86CPU, enable_l3_cache, true),
>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>> index d3097be6a50a..903994818093 100644
>> --- a/target/i386/cpu.h
>> +++ b/target/i386/cpu.h
>> @@ -1654,11 +1654,12 @@ struct X86CPU {
>>      uint64_t ucode_rev;
>>  
>>      uint32_t hyperv_spinlock_attempts;
>> -    char *hyperv_vendor_id;
>> +    char *hyperv_vendor;
>>      bool hyperv_synic_kvm_only;
>>      uint64_t hyperv_features;
>>      bool hyperv_passthrough;
>>      OnOffAuto hyperv_no_nonarch_cs;
>> +    uint32_t hyperv_vendor_id[3];
>>  
>>      bool check_cpuid;
>>      bool enforce_cpuid;
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index 205b68bc0ce8..47779c5e1efd 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -1225,6 +1225,13 @@ static int hyperv_handle_properties(CPUState *cs,
>>          memcpy(cpuid_ent, &cpuid->entries[0],
>>                 cpuid->nent * sizeof(cpuid->entries[0]));
>>  
>> +        c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
>> +        if (c) {
>> +            cpu->hyperv_vendor_id[0] = c->ebx;
>> +            cpu->hyperv_vendor_id[1] = c->ecx;
>> +            cpu->hyperv_vendor_id[2] = c->edx;
>> +        }
>> +
>
> I can't find the equivalent of this code in the current tree?  Is
> hyperv vendor ID broken when using hv-passthrough today?
>
> Maybe this could be done as a separate patch, as it changes
> behavior of hv-passthrough?

(this and similar changes in other patches) Actually we don't change
anything. Before this series and with hv-passthrough we just don't
reflect host's CPUIDs in our internal QEMU structures so
e.g. X86CPU->hyperv_vendor remains 'Microsoft Hv' while in reality
guest sees what kernel told us ("Linux KVM Hv" BTW). We just copy
everything we get from KVM_GET_SUPPORTED_HV_CPUID into guest's CPUIDs.
This is fine as we didn't actually need the information in QEMU but
to achieve the goal of the series we need to keep proper in-QEMU
representation.

The real change is that post-series QEMU is not enabling any Hyper-V
features which it doesn't know about while pre-series it was actually
doing this. This is arguably a good change: enabling new features may
require some additional work (e.g. enabling capabilities in KVM) and
without it just passing CPUID feature bits the guest may get confused.

>
>>          c = cpuid_find_entry(cpuid, HV_CPUID_FEATURES, 0);
>>          if (c) {
>>              env->features[FEAT_HYPERV_EAX] = c->eax;
>> @@ -1299,23 +1306,11 @@ static int hyperv_handle_properties(CPUState *cs,
>>  
>>      c = &cpuid_ent[cpuid_i++];
>>      c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
>> -    if (!cpu->hyperv_vendor_id) {
>> -        memcpy(signature, "Microsoft Hv", 12);
>> -    } else {
>> -        size_t len = strlen(cpu->hyperv_vendor_id);
>> -
>> -        if (len > 12) {
>> -            error_report("hv-vendor-id truncated to 12 characters");
>> -            len = 12;
>> -        }
>> -        memset(signature, 0, 12);
>> -        memcpy(signature, cpu->hyperv_vendor_id, len);
>> -    }
>>      c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
>>          HV_CPUID_NESTED_FEATURES : HV_CPUID_IMPLEMENT_LIMITS;
>> -    c->ebx = signature[0];
>> -    c->ecx = signature[1];
>> -    c->edx = signature[2];
>> +    c->ebx = cpu->hyperv_vendor_id[0];
>> +    c->ecx = cpu->hyperv_vendor_id[1];
>> +    c->edx = cpu->hyperv_vendor_id[2];
>>  
>>      c = &cpuid_ent[cpuid_i++];
>>      c->function = HV_CPUID_INTERFACE;
>> -- 
>> 2.25.4
>> 

-- 
Vitaly

Re: [PATCH RFC 09/22] i386: add reserved FEAT_HYPERV_ECX CPUID leaf

[Vitaly Kuznetsov]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Sep 04, 2020 at 04:54:18PM +0200, Vitaly Kuznetsov wrote:
>> As a preparation to expanding Hyper-V CPU features early, add
>> reserved FEAT_HYPERV_ECX CPUID leaf.
>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  target/i386/cpu.c | 5 +++++
>>  target/i386/cpu.h | 1 +
>>  target/i386/kvm.c | 2 ++
>>  3 files changed, 8 insertions(+)
>> 
>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>> index ef3c672cf415..70588571ccb1 100644
>> --- a/target/i386/cpu.c
>> +++ b/target/i386/cpu.c
>> @@ -974,6 +974,11 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
>>          },
>>          .cpuid = { .eax = 0x40000003, .reg = R_EBX, },
>>      },
>> +    [FEAT_HYPERV_ECX] = {
>> +        .type = CPUID_FEATURE_WORD,
>> +        /* reserved */
>
> What does "reserved" mean here?

This is actually from TLFS, these bits are currently reserved and there
are no features represented by them. We can, of course, short-circuit
this to '0' but I'd prefer to keep it for consistency. I can change to
something like 'Reserved as of TLFS v6.0b' to make it clear. 

>
>> +        .cpuid = { .eax = 0x40000003, .reg = R_ECX, },
>> +    },
>>      [FEAT_HYPERV_EDX] = {
>>          .type = CPUID_FEATURE_WORD,
>>          .feat_names = {
>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>> index 095d0bf75493..39e0e89aa41f 100644
>> --- a/target/i386/cpu.h
>> +++ b/target/i386/cpu.h
>> @@ -521,6 +521,7 @@ typedef enum FeatureWord {
>>      FEAT_KVM_HINTS,     /* CPUID[4000_0001].EDX */
>>      FEAT_HYPERV_EAX,    /* CPUID[4000_0003].EAX */
>>      FEAT_HYPERV_EBX,    /* CPUID[4000_0003].EBX */
>> +    FEAT_HYPERV_ECX,    /* CPUID[4000_0003].ECX */
>>      FEAT_HYPERV_EDX,    /* CPUID[4000_0003].EDX */
>>      FEAT_HV_RECOMM_EAX, /* CPUID[4000_0004].EAX */
>>      FEAT_HV_NESTED_EAX, /* CPUID[4000_000A].EAX */
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index 96ac719adca1..259be2d514dd 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -1251,6 +1251,7 @@ static int hyperv_handle_properties(CPUState *cs,
>>          if (c) {
>>              env->features[FEAT_HYPERV_EAX] = c->eax;
>>              env->features[FEAT_HYPERV_EBX] = c->ebx;
>> +            env->features[FEAT_HYPERV_ECX] = c->ecx;
>>              env->features[FEAT_HYPERV_EDX] = c->edx;
>>          }
>>  
>> @@ -1350,6 +1351,7 @@ static int hyperv_handle_properties(CPUState *cs,
>>      c->function = HV_CPUID_FEATURES;
>>      c->eax = env->features[FEAT_HYPERV_EAX];
>>      c->ebx = env->features[FEAT_HYPERV_EBX];
>> +    c->ecx = env->features[FEAT_HYPERV_ECX];
>>      c->edx = env->features[FEAT_HYPERV_EDX];
>>  
>>      c = &cpuid_ent[cpuid_i++];
>> -- 
>> 2.25.4
>> 

-- 
Vitaly

Re: [PATCH RFC 12/22] i386: always fill Hyper-V CPUID feature leaves from X86CPU data

[Vitaly Kuznetsov]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Sep 04, 2020 at 04:54:21PM +0200, Vitaly Kuznetsov wrote:
>> We have all the required data in X86CPU already and as we are about to
>> split hyperv_handle_properties() into hyperv_expand_features()/
>> hyperv_fill_cpuids() we can remove the blind copy. The functional change
>> is that QEMU won't pass CPUID leaves it doesn't currently know about
>> to the guest but arguably this is a good change.
>
> Is it, though?  Maybe we don't want it to be the default, but a
> blind passthrough mode that copies all CPUID leaves would be
> useful for testing new features.
>

We already have Hyper-V features which require extra work to get enabled
(SynIC, EVMCS). Passing corresponding feature bits to guests without
actually enabling these features in KVM will likely break them. So a pure
'blind passthrough' mode is not possible. 

>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  target/i386/kvm.c | 9 ---------
>>  1 file changed, 9 deletions(-)
>> 
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index 87b83a2aa2cb..b7f0e200a75f 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -1221,9 +1221,6 @@ static int hyperv_handle_properties(CPUState *cs,
>>      }
>>  
>>      if (cpu->hyperv_passthrough) {
>> -        memcpy(cpuid_ent, &cpuid->entries[0],
>> -               cpuid->nent * sizeof(cpuid->entries[0]));
>> -
>>          c = cpuid_find_entry(cpuid, HV_CPUID_VENDOR_AND_MAX_FUNCTIONS, 0);
>>          if (c) {
>>              cpu->hyperv_vendor_id[0] = c->ebx;
>> @@ -1325,12 +1322,6 @@ static int hyperv_handle_properties(CPUState *cs,
>>          goto free;
>>      }
>>  
>> -    if (cpu->hyperv_passthrough) {
>> -        /* We already copied all feature words from KVM as is */
>> -        r = cpuid->nent;
>> -        goto free;
>> -    }
>> -
>>      c = &cpuid_ent[cpuid_i++];
>>      c->function = HV_CPUID_VENDOR_AND_MAX_FUNCTIONS;
>>      c->eax = hyperv_feat_enabled(cpu, HYPERV_FEAT_EVMCS) ?
>> -- 
>> 2.25.4
>> 

-- 
Vitaly

Re: [PATCH RFC 22/22] i386: expand Hyper-V features early

[Vitaly Kuznetsov]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Sep 04, 2020 at 04:54:31PM +0200, Vitaly Kuznetsov wrote:
>> To make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we
>> need to expand and set the corresponding CPUID leaves early. Modify
>> x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V
>> specific kvm_hv_get_supported_cpuid() instead of
>> kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid()
>> as Hyper-V specific CPUID leaves intersect with KVM's.
>> 
>> Note, early expansion will only happen when KVM supports system wide
>> KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).
>
> Will this implicitly enable hyperv CPUID passthrough when using
> "-cpu host"?  Do we want it to?
>

I hope it won't. With 'hv_*' features set explicitly we will get just
them (if the host supports), with 'hv_passthrough' we will get
everything supported by the host. With no 'hv_*' parameters on the
command line we should get no Hyper-V features. I'll check this.

Personally, I don't think '-host' mode should enable any Hyper-V
features as these features have side-effects. E.g. if you enable SynIC
you won't be able to use vAPIC. Probably not a reasonable default for
the majority of the guests which (hopefully) are Linux.

>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  target/i386/cpu.c      | 15 +++++++++------
>>  target/i386/kvm.c      | 15 +++++++++++++++
>>  target/i386/kvm_i386.h |  7 +++++++
>>  3 files changed, 31 insertions(+), 6 deletions(-)
>> 
>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>> index 479c4bbbf459..d3c4ecb3535c 100644
>> --- a/target/i386/cpu.c
>> +++ b/target/i386/cpu.c
>> @@ -5147,7 +5147,7 @@ CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
>>      return cpu_list;
>>  }
>>  
>> -static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
>> +static uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w,
>>                                                     bool migratable_only)
>>  {
>>      FeatureWordInfo *wi = &feature_word_info[w];
>> @@ -5156,9 +5156,12 @@ static uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
>>      if (kvm_enabled()) {
>>          switch (wi->type) {
>>          case CPUID_FEATURE_WORD:
>> -            r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
>> -                                                        wi->cpuid.ecx,
>> -                                                        wi->cpuid.reg);
>> +            if (hyperv_feature_word(w))
>> +                r = kvm_hv_get_supported_cpuid(cpu, w);
>> +            else
>> +                r = kvm_arch_get_supported_cpuid(kvm_state, wi->cpuid.eax,
>> +                                                 wi->cpuid.ecx,
>> +                                                 wi->cpuid.reg);
>>              break;
>>          case MSR_FEATURE_WORD:
>>              r = kvm_arch_get_supported_msr_feature(kvm_state,
>> @@ -6485,7 +6488,7 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
>>               * by the user.
>>               */
>>              env->features[w] |=
>> -                x86_cpu_get_supported_feature_word(w, cpu->migratable) &
>> +                x86_cpu_get_supported_feature_word(cpu, w, cpu->migratable) &
>>                  ~env->user_features[w] &
>>                  ~feature_word_info[w].no_autoenable_flags;
>>          }
>> @@ -6589,7 +6592,7 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool verbose)
>>  
>>      for (w = 0; w < FEATURE_WORDS; w++) {
>>          uint64_t host_feat =
>> -            x86_cpu_get_supported_feature_word(w, false);
>> +            x86_cpu_get_supported_feature_word(cpu, w, false);
>>          uint64_t requested_features = env->features[w];
>>          uint64_t unavailable_features = requested_features & ~host_feat;
>>          mark_unavailable_features(cpu, w, unavailable_features, prefix);
>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>> index 961241528a5c..764b96fbbb7d 100644
>> --- a/target/i386/kvm.c
>> +++ b/target/i386/kvm.c
>> @@ -1449,6 +1449,21 @@ static int hyperv_fill_cpuids(CPUState *cs,
>>      return cpuid_i;
>>  }
>>  
>> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w)
>> +{
>> +    CPUState *cs = CPU(cpu);
>> +    CPUX86State *env = &cpu->env;
>> +    Error *local_err = NULL;
>> +
>> +    hyperv_expand_features(cs, &local_err);
>> +
>> +    if (local_err) {
>> +            error_report_err(local_err);
>> +    }
>> +
>> +    return env->features[w];
>> +}
>> +
>>  static Error *hv_passthrough_mig_blocker;
>>  static Error *hv_no_nonarch_cs_mig_blocker;
>>  
>> diff --git a/target/i386/kvm_i386.h b/target/i386/kvm_i386.h
>> index 064b8798a26c..2e7da4f39668 100644
>> --- a/target/i386/kvm_i386.h
>> +++ b/target/i386/kvm_i386.h
>> @@ -48,4 +48,11 @@ bool kvm_has_waitpkg(void);
>>  
>>  bool kvm_hv_vpindex_settable(void);
>>  
>> +static inline bool hyperv_feature_word(enum FeatureWord w)
>> +{
>> +    return w >= FEAT_HYPERV_EAX && w <= FEAT_HV_NESTED_EDX;
>> +}
>> +
>> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w);
>> +
>>  #endif
>> -- 
>> 2.25.4
>> 

-- 
Vitaly

Re: [PATCH RFC 22/22] i386: expand Hyper-V features early

[Vitaly Kuznetsov]

Eduardo Habkost <ehabkost@redhat.com> writes:

> On Fri, Sep 04, 2020 at 04:54:31PM +0200, Vitaly Kuznetsov wrote:
>> To make Hyper-V features appear in e.g. QMP query-cpu-model-expansion we
>> need to expand and set the corresponding CPUID leaves early. Modify
>> x86_cpu_get_supported_feature_word() to call newly intoduced Hyper-V
>> specific kvm_hv_get_supported_cpuid() instead of
>> kvm_arch_get_supported_cpuid(). We can't use kvm_arch_get_supported_cpuid()
>> as Hyper-V specific CPUID leaves intersect with KVM's.
>> 
>> Note, early expansion will only happen when KVM supports system wide
>> KVM_GET_SUPPORTED_HV_CPUID ioctl (KVM_CAP_SYS_HYPERV_CPUID).
>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
> [...]
>> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w)
>> +{
>> +    CPUState *cs = CPU(cpu);
>> +    CPUX86State *env = &cpu->env;
>> +    Error *local_err = NULL;
>> +
>> +    hyperv_expand_features(cs, &local_err);
>
> This makes a function that sounds like it doesn't have any side
> effect ("get supported cpuid") have an unintended side effect
> (hyperv_expand_features() will change all CPUID data inside the
> cpu object).
>
> What about making it more similar to
> kvm_arch_get_supported_cpuid(), and be just a wrapper to
> get_supported_hv_cpuid()?
>
> I would also make sure get_supported_hv_cpuid() doesn't get
> CPUState as argument, just to be sure it will never touch the CPU
> object state.

Sure, I can try. The difference with get_supported_cpuid() is that
get_supported_hv_cpuid() currently doesn't have cache and we don't
probably want to call KVM_GET_SUPPORTED_HV_CPUID ioctl for every CPUID
leaf. I don't see why it can't be introduced though.

>
>> +
>> +    if (local_err) {
>> +            error_report_err(local_err);
>> +    }
>> +
>> +    return env->features[w];
>> +}
>> +
>>  static Error *hv_passthrough_mig_blocker;
>>  static Error *hv_no_nonarch_cs_mig_blocker;
>>  
>> diff --git a/target/i386/kvm_i386.h b/target/i386/kvm_i386.h
>> index 064b8798a26c..2e7da4f39668 100644
>> --- a/target/i386/kvm_i386.h
>> +++ b/target/i386/kvm_i386.h
>> @@ -48,4 +48,11 @@ bool kvm_has_waitpkg(void);
>>  
>>  bool kvm_hv_vpindex_settable(void);
>>  
>> +static inline bool hyperv_feature_word(enum FeatureWord w)
>> +{
>> +    return w >= FEAT_HYPERV_EAX && w <= FEAT_HV_NESTED_EDX;
>> +}
>> +
>> +uint32_t kvm_hv_get_supported_cpuid(X86CPU *cpu, enum FeatureWord w);
>> +
>>  #endif
>> -- 
>> 2.25.4
>> 

-- 
Vitaly