[Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Andrey Smetanin]

Per Hyper-V specification (and as required by Hyper-V-aware guests),
SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
of MSRs, and signals expiration by delivering a special format message
to the configured SynIC message slot and triggering the corresponding
synthetic interrupt.

Note: as implemented by this patch, all periodic timers are "lazy"
(i.e. if the vCPU wasn't scheduled for more than the timer period the
timer events are lost), regardless of the corresponding configuration
MSR.  If deemed necessary, the "catch up" mode (the timer period is
shortened until the timer catches up) will be implemented later.

The Hyper-V SynIC timers support is required to load winhv.sys
inside Windows guest on which guest VMBus devices depends on.

This patches depends on Hyper-V SynIC patches previosly sent.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org

Andrey Smetanin (7):
  drivers/hv: Move HV_SYNIC_STIMER_COUNT into Hyper-V UAPI x86 header
  drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header
  kvm/x86: Rearrange func's declarations inside Hyper-V header
  kvm/x86: Added Hyper-V vcpu_to_hv_vcpu()/hv_vcpu_to_vcpu() helpers
  kvm/x86: Hyper-V internal helper to read MSR HV_X64_MSR_TIME_REF_COUNT
  kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack
  kvm/x86: Hyper-V SynIC timers

 arch/x86/include/asm/kvm_host.h    |  13 ++
 arch/x86/include/uapi/asm/hyperv.h |  99 ++++++++++
 arch/x86/kvm/hyperv.c              | 367 ++++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/hyperv.h              |  54 ++++--
 arch/x86/kvm/x86.c                 |   9 +
 drivers/hv/hyperv_vmbus.h          |  93 ----------
 include/linux/kvm_host.h           |   3 +
 7 files changed, 527 insertions(+), 111 deletions(-)

-- 
2.4.3

[Qemu-devel] [PATCH v1 1/7] drivers/hv: Move HV_SYNIC_STIMER_COUNT into Hyper-V UAPI x86 header

[Andrey Smetanin]

This constant is required for Hyper-V SynIC timers MSR's
support by userspace(QEMU).

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/include/uapi/asm/hyperv.h | 2 ++
 drivers/hv/hyperv_vmbus.h          | 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 040d408..07981f0 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -269,4 +269,6 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
 #define HV_SYNIC_SINT_AUTO_EOI		(1ULL << 17)
 #define HV_SYNIC_SINT_VECTOR_MASK	(0xFF)
 
+#define HV_SYNIC_STIMER_COUNT		(4)
+
 #endif
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 3782636..46e23d1 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -102,8 +102,6 @@ enum hv_message_type {
 	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
 };
 
-#define HV_SYNIC_STIMER_COUNT		(4)
-
 /* Define invalid partition identifier. */
 #define HV_PARTITION_ID_INVALID		((u64)0x0)
 
-- 
2.4.3

Re: [Qemu-devel] [PATCH v1 1/7] drivers/hv: Move HV_SYNIC_STIMER_COUNT into Hyper-V UAPI x86 header

[KY Srinivasan]

> -----Original Message-----
> From: Andrey Smetanin [mailto:asmetanin@virtuozzo.com]
> Sent: Wednesday, November 25, 2015 7:20 AM
> To: kvm@vger.kernel.org
> Cc: Gleb Natapov <gleb@kernel.org>; Paolo Bonzini
> <pbonzini@redhat.com>; KY Srinivasan <kys@microsoft.com>; Haiyang
> Zhang <haiyangz@microsoft.com>; Vitaly Kuznetsov
> <vkuznets@redhat.com>; Roman Kagan <rkagan@virtuozzo.com>; Denis V.
> Lunev <den@openvz.org>; qemu-devel@nongnu.org
> Subject: [PATCH v1 1/7] drivers/hv: Move HV_SYNIC_STIMER_COUNT into
> Hyper-V UAPI x86 header
> 
> This constant is required for Hyper-V SynIC timers MSR's
> support by userspace(QEMU).
> 
> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>

Acked-by:  K. Y. Srinivasan <kys@microsoft.com>
> CC: Gleb Natapov <gleb@kernel.org>
> CC: Paolo Bonzini <pbonzini@redhat.com>
> CC: "K. Y. Srinivasan" <kys@microsoft.com>
> CC: Haiyang Zhang <haiyangz@microsoft.com>
> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> CC: Roman Kagan <rkagan@virtuozzo.com>
> CC: Denis V. Lunev <den@openvz.org>
> CC: qemu-devel@nongnu.org
> ---
>  arch/x86/include/uapi/asm/hyperv.h | 2 ++
>  drivers/hv/hyperv_vmbus.h          | 2 --
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/include/uapi/asm/hyperv.h
> b/arch/x86/include/uapi/asm/hyperv.h
> index 040d408..07981f0 100644
> --- a/arch/x86/include/uapi/asm/hyperv.h
> +++ b/arch/x86/include/uapi/asm/hyperv.h
> @@ -269,4 +269,6 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
>  #define HV_SYNIC_SINT_AUTO_EOI		(1ULL << 17)
>  #define HV_SYNIC_SINT_VECTOR_MASK	(0xFF)
> 
> +#define HV_SYNIC_STIMER_COUNT		(4)
> +
>  #endif
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 3782636..46e23d1 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -102,8 +102,6 @@ enum hv_message_type {
>  	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
>  };
> 
> -#define HV_SYNIC_STIMER_COUNT		(4)
> -
>  /* Define invalid partition identifier. */
>  #define HV_PARTITION_ID_INVALID		((u64)0x0)
> 
> --
> 2.4.3

[Qemu-devel] [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[Andrey Smetanin]

This struct is required for Hyper-V SynIC timers implementation inside KVM
and for upcoming Hyper-V VMBus support by userspace(QEMU). So place it into
Hyper-V UAPI header.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/include/uapi/asm/hyperv.h | 91 ++++++++++++++++++++++++++++++++++++++
 drivers/hv/hyperv_vmbus.h          | 91 --------------------------------------
 2 files changed, 91 insertions(+), 91 deletions(-)

diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 07981f0..e86d77e 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
 
 #define HV_SYNIC_STIMER_COUNT		(4)
 
+/* Define synthetic interrupt controller message constants. */
+#define HV_MESSAGE_SIZE			(256)
+#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
+#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
+
+/* Define hypervisor message types. */
+enum hv_message_type {
+	HVMSG_NONE			= 0x00000000,
+
+	/* Memory access messages. */
+	HVMSG_UNMAPPED_GPA		= 0x80000000,
+	HVMSG_GPA_INTERCEPT		= 0x80000001,
+
+	/* Timer notification messages. */
+	HVMSG_TIMER_EXPIRED			= 0x80000010,
+
+	/* Error messages. */
+	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
+	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
+	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
+
+	/* Trace buffer complete messages. */
+	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
+
+	/* Platform-specific processor intercept messages. */
+	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
+	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
+	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
+	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
+	HVMSG_X64_APIC_EOI			= 0x80010004,
+	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
+};
+
+/* Define synthetic interrupt controller message flags. */
+union hv_message_flags {
+	__u8 asu8;
+	struct {
+		__u8 msg_pending:1;
+		__u8 reserved:7;
+	};
+};
+
+/* Define port identifier type. */
+union hv_port_id {
+	__u32 asu32;
+	struct {
+		__u32 id:24;
+		__u32 reserved:8;
+	} u;
+};
+
+/* Define port type. */
+enum hv_port_type {
+	HVPORT_MSG	= 1,
+	HVPORT_EVENT		= 2,
+	HVPORT_MONITOR	= 3
+};
+
+/* Define synthetic interrupt controller message header. */
+struct hv_message_header {
+	enum hv_message_type message_type;
+	__u8 payload_size;
+	union hv_message_flags message_flags;
+	__u8 reserved[2];
+	union {
+		__u64 sender;
+		union hv_port_id port;
+	};
+};
+
+/* Define timer message payload structure. */
+struct hv_timer_message_payload {
+	__u32 timer_index;
+	__u32 reserved;
+	__u64 expiration_time;	/* When the timer expired */
+	__u64 delivery_time;	/* When the message was delivered */
+};
+
+/* Define synthetic interrupt controller message format. */
+struct hv_message {
+	struct hv_message_header header;
+	union {
+		__u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
+	} u;
+};
+
+/* Define the synthetic interrupt message page layout. */
+struct hv_message_page {
+	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
+};
+
 #endif
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 46e23d1..d22230c 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -63,10 +63,6 @@ enum hv_cpuid_function {
 /* Define version of the synthetic interrupt controller. */
 #define HV_SYNIC_VERSION		(1)
 
-/* Define synthetic interrupt controller message constants. */
-#define HV_MESSAGE_SIZE			(256)
-#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
-#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
 #define HV_ANY_VP			(0xFFFFFFFF)
 
 /* Define synthetic interrupt controller flag constants. */
@@ -74,53 +70,9 @@ enum hv_cpuid_function {
 #define HV_EVENT_FLAGS_BYTE_COUNT	(256)
 #define HV_EVENT_FLAGS_DWORD_COUNT	(256 / sizeof(u32))
 
-/* Define hypervisor message types. */
-enum hv_message_type {
-	HVMSG_NONE			= 0x00000000,
-
-	/* Memory access messages. */
-	HVMSG_UNMAPPED_GPA		= 0x80000000,
-	HVMSG_GPA_INTERCEPT		= 0x80000001,
-
-	/* Timer notification messages. */
-	HVMSG_TIMER_EXPIRED			= 0x80000010,
-
-	/* Error messages. */
-	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
-	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
-	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
-
-	/* Trace buffer complete messages. */
-	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
-
-	/* Platform-specific processor intercept messages. */
-	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
-	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
-	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
-	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
-	HVMSG_X64_APIC_EOI			= 0x80010004,
-	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
-};
-
 /* Define invalid partition identifier. */
 #define HV_PARTITION_ID_INVALID		((u64)0x0)
 
-/* Define port identifier type. */
-union hv_port_id {
-	u32 asu32;
-	struct {
-		u32 id:24;
-		u32 reserved:8;
-	} u ;
-};
-
-/* Define port type. */
-enum hv_port_type {
-	HVPORT_MSG	= 1,
-	HVPORT_EVENT		= 2,
-	HVPORT_MONITOR	= 3
-};
-
 /* Define port information structure. */
 struct hv_port_info {
 	enum hv_port_type port_type;
@@ -161,27 +113,6 @@ struct hv_connection_info {
 	};
 };
 
-/* Define synthetic interrupt controller message flags. */
-union hv_message_flags {
-	u8 asu8;
-	struct {
-		u8 msg_pending:1;
-		u8 reserved:7;
-	};
-};
-
-/* Define synthetic interrupt controller message header. */
-struct hv_message_header {
-	enum hv_message_type message_type;
-	u8 payload_size;
-	union hv_message_flags message_flags;
-	u8 reserved[2];
-	union {
-		u64 sender;
-		union hv_port_id port;
-	};
-};
-
 /*
  * Timer configuration register.
  */
@@ -198,31 +129,9 @@ union hv_timer_config {
 	};
 };
 
-
-/* Define timer message payload structure. */
-struct hv_timer_message_payload {
-	u32 timer_index;
-	u32 reserved;
-	u64 expiration_time;	/* When the timer expired */
-	u64 delivery_time;	/* When the message was delivered */
-};
-
-/* Define synthetic interrupt controller message format. */
-struct hv_message {
-	struct hv_message_header header;
-	union {
-		u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
-	} u ;
-};
-
 /* Define the number of message buffers associated with each port. */
 #define HV_PORT_MESSAGE_BUFFER_COUNT	(16)
 
-/* Define the synthetic interrupt message page layout. */
-struct hv_message_page {
-	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
-};
-
 /* Define the synthetic interrupt controller event flags format. */
 union hv_synic_event_flags {
 	u8 flags8[HV_EVENT_FLAGS_BYTE_COUNT];
-- 
2.4.3

Re: [Qemu-devel] [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[Paolo Bonzini]

On 25/11/2015 16:20, Andrey Smetanin wrote:
> This struct is required for Hyper-V SynIC timers implementation inside KVM
> and for upcoming Hyper-V VMBus support by userspace(QEMU). So place it into
> Hyper-V UAPI header.
> 
> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
> CC: Gleb Natapov <gleb@kernel.org>
> CC: Paolo Bonzini <pbonzini@redhat.com>
> CC: "K. Y. Srinivasan" <kys@microsoft.com>
> CC: Haiyang Zhang <haiyangz@microsoft.com>
> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> CC: Roman Kagan <rkagan@virtuozzo.com>
> CC: Denis V. Lunev <den@openvz.org>
> CC: qemu-devel@nongnu.org
> ---
>  arch/x86/include/uapi/asm/hyperv.h | 91 ++++++++++++++++++++++++++++++++++++++
>  drivers/hv/hyperv_vmbus.h          | 91 --------------------------------------
>  2 files changed, 91 insertions(+), 91 deletions(-)
> 
> diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
> index 07981f0..e86d77e 100644
> --- a/arch/x86/include/uapi/asm/hyperv.h
> +++ b/arch/x86/include/uapi/asm/hyperv.h
> @@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
>  
>  #define HV_SYNIC_STIMER_COUNT		(4)
>  
> +/* Define synthetic interrupt controller message constants. */
> +#define HV_MESSAGE_SIZE			(256)
> +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
> +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
> +
> +/* Define hypervisor message types. */
> +enum hv_message_type {
> +	HVMSG_NONE			= 0x00000000,
> +
> +	/* Memory access messages. */
> +	HVMSG_UNMAPPED_GPA		= 0x80000000,
> +	HVMSG_GPA_INTERCEPT		= 0x80000001,
> +
> +	/* Timer notification messages. */
> +	HVMSG_TIMER_EXPIRED			= 0x80000010,
> +
> +	/* Error messages. */
> +	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
> +	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
> +	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
> +
> +	/* Trace buffer complete messages. */
> +	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
> +
> +	/* Platform-specific processor intercept messages. */
> +	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
> +	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
> +	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
> +	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
> +	HVMSG_X64_APIC_EOI			= 0x80010004,
> +	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
> +};
> +
> +/* Define synthetic interrupt controller message flags. */
> +union hv_message_flags {
> +	__u8 asu8;
> +	struct {
> +		__u8 msg_pending:1;
> +		__u8 reserved:7;
> +	};
> +};
> +
> +/* Define port identifier type. */
> +union hv_port_id {
> +	__u32 asu32;
> +	struct {
> +		__u32 id:24;
> +		__u32 reserved:8;
> +	} u;
> +};
> +
> +/* Define port type. */
> +enum hv_port_type {
> +	HVPORT_MSG	= 1,
> +	HVPORT_EVENT		= 2,
> +	HVPORT_MONITOR	= 3
> +};
> +
> +/* Define synthetic interrupt controller message header. */
> +struct hv_message_header {
> +	enum hv_message_type message_type;

Do not declare this as an enum, declare it as __u32 to make the size
portable.  It can be a patch on top.

KY, can you ack these two patches?

Paolo

> +	__u8 payload_size;
> +	union hv_message_flags message_flags;
> +	__u8 reserved[2];
> +	union {
> +		__u64 sender;
> +		union hv_port_id port;
> +	};
> +};
> +
> +/* Define timer message payload structure. */
> +struct hv_timer_message_payload {
> +	__u32 timer_index;
> +	__u32 reserved;
> +	__u64 expiration_time;	/* When the timer expired */
> +	__u64 delivery_time;	/* When the message was delivered */
> +};
> +
> +/* Define synthetic interrupt controller message format. */
> +struct hv_message {
> +	struct hv_message_header header;
> +	union {
> +		__u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
> +	} u;
> +};
> +
> +/* Define the synthetic interrupt message page layout. */
> +struct hv_message_page {
> +	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
> +};
> +
>  #endif
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index 46e23d1..d22230c 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -63,10 +63,6 @@ enum hv_cpuid_function {
>  /* Define version of the synthetic interrupt controller. */
>  #define HV_SYNIC_VERSION		(1)
>  
> -/* Define synthetic interrupt controller message constants. */
> -#define HV_MESSAGE_SIZE			(256)
> -#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
> -#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
>  #define HV_ANY_VP			(0xFFFFFFFF)
>  
>  /* Define synthetic interrupt controller flag constants. */
> @@ -74,53 +70,9 @@ enum hv_cpuid_function {
>  #define HV_EVENT_FLAGS_BYTE_COUNT	(256)
>  #define HV_EVENT_FLAGS_DWORD_COUNT	(256 / sizeof(u32))
>  
> -/* Define hypervisor message types. */
> -enum hv_message_type {
> -	HVMSG_NONE			= 0x00000000,
> -
> -	/* Memory access messages. */
> -	HVMSG_UNMAPPED_GPA		= 0x80000000,
> -	HVMSG_GPA_INTERCEPT		= 0x80000001,
> -
> -	/* Timer notification messages. */
> -	HVMSG_TIMER_EXPIRED			= 0x80000010,
> -
> -	/* Error messages. */
> -	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
> -	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
> -	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
> -
> -	/* Trace buffer complete messages. */
> -	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
> -
> -	/* Platform-specific processor intercept messages. */
> -	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
> -	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
> -	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
> -	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
> -	HVMSG_X64_APIC_EOI			= 0x80010004,
> -	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
> -};
> -
>  /* Define invalid partition identifier. */
>  #define HV_PARTITION_ID_INVALID		((u64)0x0)
>  
> -/* Define port identifier type. */
> -union hv_port_id {
> -	u32 asu32;
> -	struct {
> -		u32 id:24;
> -		u32 reserved:8;
> -	} u ;
> -};
> -
> -/* Define port type. */
> -enum hv_port_type {
> -	HVPORT_MSG	= 1,
> -	HVPORT_EVENT		= 2,
> -	HVPORT_MONITOR	= 3
> -};
> -
>  /* Define port information structure. */
>  struct hv_port_info {
>  	enum hv_port_type port_type;
> @@ -161,27 +113,6 @@ struct hv_connection_info {
>  	};
>  };
>  
> -/* Define synthetic interrupt controller message flags. */
> -union hv_message_flags {
> -	u8 asu8;
> -	struct {
> -		u8 msg_pending:1;
> -		u8 reserved:7;
> -	};
> -};
> -
> -/* Define synthetic interrupt controller message header. */
> -struct hv_message_header {
> -	enum hv_message_type message_type;
> -	u8 payload_size;
> -	union hv_message_flags message_flags;
> -	u8 reserved[2];
> -	union {
> -		u64 sender;
> -		union hv_port_id port;
> -	};
> -};
> -
>  /*
>   * Timer configuration register.
>   */
> @@ -198,31 +129,9 @@ union hv_timer_config {
>  	};
>  };
>  
> -
> -/* Define timer message payload structure. */
> -struct hv_timer_message_payload {
> -	u32 timer_index;
> -	u32 reserved;
> -	u64 expiration_time;	/* When the timer expired */
> -	u64 delivery_time;	/* When the message was delivered */
> -};
> -
> -/* Define synthetic interrupt controller message format. */
> -struct hv_message {
> -	struct hv_message_header header;
> -	union {
> -		u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
> -	} u ;
> -};
> -
>  /* Define the number of message buffers associated with each port. */
>  #define HV_PORT_MESSAGE_BUFFER_COUNT	(16)
>  
> -/* Define the synthetic interrupt message page layout. */
> -struct hv_message_page {
> -	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
> -};
> -
>  /* Define the synthetic interrupt controller event flags format. */
>  union hv_synic_event_flags {
>  	u8 flags8[HV_EVENT_FLAGS_BYTE_COUNT];
> 

Re: [Qemu-devel] [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[Andrey Smetanin]

On 11/27/2015 12:34 PM, Paolo Bonzini wrote:
>
>
> On 25/11/2015 16:20, Andrey Smetanin wrote:
>> This struct is required for Hyper-V SynIC timers implementation inside KVM
>> and for upcoming Hyper-V VMBus support by userspace(QEMU). So place it into
>> Hyper-V UAPI header.
>>
>> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
>> Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
>> CC: Gleb Natapov <gleb@kernel.org>
>> CC: Paolo Bonzini <pbonzini@redhat.com>
>> CC: "K. Y. Srinivasan" <kys@microsoft.com>
>> CC: Haiyang Zhang <haiyangz@microsoft.com>
>> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
>> CC: Roman Kagan <rkagan@virtuozzo.com>
>> CC: Denis V. Lunev <den@openvz.org>
>> CC: qemu-devel@nongnu.org
>> ---
>>   arch/x86/include/uapi/asm/hyperv.h | 91 ++++++++++++++++++++++++++++++++++++++
>>   drivers/hv/hyperv_vmbus.h          | 91 --------------------------------------
>>   2 files changed, 91 insertions(+), 91 deletions(-)
>>
>> diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
>> index 07981f0..e86d77e 100644
>> --- a/arch/x86/include/uapi/asm/hyperv.h
>> +++ b/arch/x86/include/uapi/asm/hyperv.h
>> @@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
>>
>>   #define HV_SYNIC_STIMER_COUNT		(4)
>>
>> +/* Define synthetic interrupt controller message constants. */
>> +#define HV_MESSAGE_SIZE			(256)
>> +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
>> +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
>> +
>> +/* Define hypervisor message types. */
>> +enum hv_message_type {
>> +	HVMSG_NONE			= 0x00000000,
>> +
>> +	/* Memory access messages. */
>> +	HVMSG_UNMAPPED_GPA		= 0x80000000,
>> +	HVMSG_GPA_INTERCEPT		= 0x80000001,
>> +
>> +	/* Timer notification messages. */
>> +	HVMSG_TIMER_EXPIRED			= 0x80000010,
>> +
>> +	/* Error messages. */
>> +	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
>> +	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
>> +	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
>> +
>> +	/* Trace buffer complete messages. */
>> +	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
>> +
>> +	/* Platform-specific processor intercept messages. */
>> +	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
>> +	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
>> +	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
>> +	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
>> +	HVMSG_X64_APIC_EOI			= 0x80010004,
>> +	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
>> +};
>> +
>> +/* Define synthetic interrupt controller message flags. */
>> +union hv_message_flags {
>> +	__u8 asu8;
>> +	struct {
>> +		__u8 msg_pending:1;
>> +		__u8 reserved:7;
>> +	};
>> +};
>> +
>> +/* Define port identifier type. */
>> +union hv_port_id {
>> +	__u32 asu32;
>> +	struct {
>> +		__u32 id:24;
>> +		__u32 reserved:8;
>> +	} u;
>> +};
>> +
>> +/* Define port type. */
>> +enum hv_port_type {
>> +	HVPORT_MSG	= 1,
>> +	HVPORT_EVENT		= 2,
>> +	HVPORT_MONITOR	= 3
>> +};
>> +
>> +/* Define synthetic interrupt controller message header. */
>> +struct hv_message_header {
>> +	enum hv_message_type message_type;
>
> Do not declare this as an enum, declare it as __u32 to make the size
> portable.  It can be a patch on top.
Ok, I'll prepare such patch.
>
> KY, can you ack these two patches?
>
> Paolo
>
>> +	__u8 payload_size;
>> +	union hv_message_flags message_flags;
>> +	__u8 reserved[2];
>> +	union {
>> +		__u64 sender;
>> +		union hv_port_id port;
>> +	};
>> +};
>> +
>> +/* Define timer message payload structure. */
>> +struct hv_timer_message_payload {
>> +	__u32 timer_index;
>> +	__u32 reserved;
>> +	__u64 expiration_time;	/* When the timer expired */
>> +	__u64 delivery_time;	/* When the message was delivered */
>> +};
>> +
>> +/* Define synthetic interrupt controller message format. */
>> +struct hv_message {
>> +	struct hv_message_header header;
>> +	union {
>> +		__u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
>> +	} u;
>> +};
>> +
>> +/* Define the synthetic interrupt message page layout. */
>> +struct hv_message_page {
>> +	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
>> +};
>> +
>>   #endif
>> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
>> index 46e23d1..d22230c 100644
>> --- a/drivers/hv/hyperv_vmbus.h
>> +++ b/drivers/hv/hyperv_vmbus.h
>> @@ -63,10 +63,6 @@ enum hv_cpuid_function {
>>   /* Define version of the synthetic interrupt controller. */
>>   #define HV_SYNIC_VERSION		(1)
>>
>> -/* Define synthetic interrupt controller message constants. */
>> -#define HV_MESSAGE_SIZE			(256)
>> -#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
>> -#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
>>   #define HV_ANY_VP			(0xFFFFFFFF)
>>
>>   /* Define synthetic interrupt controller flag constants. */
>> @@ -74,53 +70,9 @@ enum hv_cpuid_function {
>>   #define HV_EVENT_FLAGS_BYTE_COUNT	(256)
>>   #define HV_EVENT_FLAGS_DWORD_COUNT	(256 / sizeof(u32))
>>
>> -/* Define hypervisor message types. */
>> -enum hv_message_type {
>> -	HVMSG_NONE			= 0x00000000,
>> -
>> -	/* Memory access messages. */
>> -	HVMSG_UNMAPPED_GPA		= 0x80000000,
>> -	HVMSG_GPA_INTERCEPT		= 0x80000001,
>> -
>> -	/* Timer notification messages. */
>> -	HVMSG_TIMER_EXPIRED			= 0x80000010,
>> -
>> -	/* Error messages. */
>> -	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
>> -	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
>> -	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
>> -
>> -	/* Trace buffer complete messages. */
>> -	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
>> -
>> -	/* Platform-specific processor intercept messages. */
>> -	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
>> -	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
>> -	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
>> -	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
>> -	HVMSG_X64_APIC_EOI			= 0x80010004,
>> -	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
>> -};
>> -
>>   /* Define invalid partition identifier. */
>>   #define HV_PARTITION_ID_INVALID		((u64)0x0)
>>
>> -/* Define port identifier type. */
>> -union hv_port_id {
>> -	u32 asu32;
>> -	struct {
>> -		u32 id:24;
>> -		u32 reserved:8;
>> -	} u ;
>> -};
>> -
>> -/* Define port type. */
>> -enum hv_port_type {
>> -	HVPORT_MSG	= 1,
>> -	HVPORT_EVENT		= 2,
>> -	HVPORT_MONITOR	= 3
>> -};
>> -
>>   /* Define port information structure. */
>>   struct hv_port_info {
>>   	enum hv_port_type port_type;
>> @@ -161,27 +113,6 @@ struct hv_connection_info {
>>   	};
>>   };
>>
>> -/* Define synthetic interrupt controller message flags. */
>> -union hv_message_flags {
>> -	u8 asu8;
>> -	struct {
>> -		u8 msg_pending:1;
>> -		u8 reserved:7;
>> -	};
>> -};
>> -
>> -/* Define synthetic interrupt controller message header. */
>> -struct hv_message_header {
>> -	enum hv_message_type message_type;
>> -	u8 payload_size;
>> -	union hv_message_flags message_flags;
>> -	u8 reserved[2];
>> -	union {
>> -		u64 sender;
>> -		union hv_port_id port;
>> -	};
>> -};
>> -
>>   /*
>>    * Timer configuration register.
>>    */
>> @@ -198,31 +129,9 @@ union hv_timer_config {
>>   	};
>>   };
>>
>> -
>> -/* Define timer message payload structure. */
>> -struct hv_timer_message_payload {
>> -	u32 timer_index;
>> -	u32 reserved;
>> -	u64 expiration_time;	/* When the timer expired */
>> -	u64 delivery_time;	/* When the message was delivered */
>> -};
>> -
>> -/* Define synthetic interrupt controller message format. */
>> -struct hv_message {
>> -	struct hv_message_header header;
>> -	union {
>> -		u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
>> -	} u ;
>> -};
>> -
>>   /* Define the number of message buffers associated with each port. */
>>   #define HV_PORT_MESSAGE_BUFFER_COUNT	(16)
>>
>> -/* Define the synthetic interrupt message page layout. */
>> -struct hv_message_page {
>> -	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
>> -};
>> -
>>   /* Define the synthetic interrupt controller event flags format. */
>>   union hv_synic_event_flags {
>>   	u8 flags8[HV_EVENT_FLAGS_BYTE_COUNT];
>>

Re: [Qemu-devel] [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI Hyper-V x86 header

[KY Srinivasan]

> -----Original Message-----
> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo
> Bonzini
> Sent: Friday, November 27, 2015 1:35 AM
> To: Andrey Smetanin <asmetanin@virtuozzo.com>; kvm@vger.kernel.org
> Cc: Gleb Natapov <gleb@kernel.org>; KY Srinivasan <kys@microsoft.com>;
> Haiyang Zhang <haiyangz@microsoft.com>; Vitaly Kuznetsov
> <vkuznets@redhat.com>; Roman Kagan <rkagan@virtuozzo.com>; Denis V.
> Lunev <den@openvz.org>; qemu-devel@nongnu.org
> Subject: Re: [PATCH v1 2/7] drivers/hv: Move struct hv_message into UAPI
> Hyper-V x86 header
> 
> 
> 
> On 25/11/2015 16:20, Andrey Smetanin wrote:
> > This struct is required for Hyper-V SynIC timers implementation inside KVM
> > and for upcoming Hyper-V VMBus support by userspace(QEMU). So place
> it into
> > Hyper-V UAPI header.
> >
> > Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> > Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Acked-by: K. Y. Srinivasan <kys@microsoft.com>

> > CC: Gleb Natapov <gleb@kernel.org>
> > CC: Paolo Bonzini <pbonzini@redhat.com>
> > CC: "K. Y. Srinivasan" <kys@microsoft.com>
> > CC: Haiyang Zhang <haiyangz@microsoft.com>
> > CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> > CC: Roman Kagan <rkagan@virtuozzo.com>
> > CC: Denis V. Lunev <den@openvz.org>
> > CC: qemu-devel@nongnu.org
> > ---
> >  arch/x86/include/uapi/asm/hyperv.h | 91
> ++++++++++++++++++++++++++++++++++++++
> >  drivers/hv/hyperv_vmbus.h          | 91 --------------------------------------
> >  2 files changed, 91 insertions(+), 91 deletions(-)
> >
> > diff --git a/arch/x86/include/uapi/asm/hyperv.h
> b/arch/x86/include/uapi/asm/hyperv.h
> > index 07981f0..e86d77e 100644
> > --- a/arch/x86/include/uapi/asm/hyperv.h
> > +++ b/arch/x86/include/uapi/asm/hyperv.h
> > @@ -271,4 +271,95 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
> >
> >  #define HV_SYNIC_STIMER_COUNT		(4)
> >
> > +/* Define synthetic interrupt controller message constants. */
> > +#define HV_MESSAGE_SIZE			(256)
> > +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
> > +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
> > +
> > +/* Define hypervisor message types. */
> > +enum hv_message_type {
> > +	HVMSG_NONE			= 0x00000000,
> > +
> > +	/* Memory access messages. */
> > +	HVMSG_UNMAPPED_GPA		= 0x80000000,
> > +	HVMSG_GPA_INTERCEPT		= 0x80000001,
> > +
> > +	/* Timer notification messages. */
> > +	HVMSG_TIMER_EXPIRED			= 0x80000010,
> > +
> > +	/* Error messages. */
> > +	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
> > +	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
> > +	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
> > +
> > +	/* Trace buffer complete messages. */
> > +	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
> > +
> > +	/* Platform-specific processor intercept messages. */
> > +	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
> > +	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
> > +	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
> > +	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
> > +	HVMSG_X64_APIC_EOI			= 0x80010004,
> > +	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
> > +};
> > +
> > +/* Define synthetic interrupt controller message flags. */
> > +union hv_message_flags {
> > +	__u8 asu8;
> > +	struct {
> > +		__u8 msg_pending:1;
> > +		__u8 reserved:7;
> > +	};
> > +};
> > +
> > +/* Define port identifier type. */
> > +union hv_port_id {
> > +	__u32 asu32;
> > +	struct {
> > +		__u32 id:24;
> > +		__u32 reserved:8;
> > +	} u;
> > +};
> > +
> > +/* Define port type. */
> > +enum hv_port_type {
> > +	HVPORT_MSG	= 1,
> > +	HVPORT_EVENT		= 2,
> > +	HVPORT_MONITOR	= 3
> > +};
> > +
> > +/* Define synthetic interrupt controller message header. */
> > +struct hv_message_header {
> > +	enum hv_message_type message_type;
> 
> Do not declare this as an enum, declare it as __u32 to make the size
> portable.  It can be a patch on top.
> 
> KY, can you ack these two patches?
> 
> Paolo
> 
> > +	__u8 payload_size;
> > +	union hv_message_flags message_flags;
> > +	__u8 reserved[2];
> > +	union {
> > +		__u64 sender;
> > +		union hv_port_id port;
> > +	};
> > +};
> > +
> > +/* Define timer message payload structure. */
> > +struct hv_timer_message_payload {
> > +	__u32 timer_index;
> > +	__u32 reserved;
> > +	__u64 expiration_time;	/* When the timer expired */
> > +	__u64 delivery_time;	/* When the message was delivered */
> > +};
> > +
> > +/* Define synthetic interrupt controller message format. */
> > +struct hv_message {
> > +	struct hv_message_header header;
> > +	union {
> > +		__u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
> > +	} u;
> > +};
> > +
> > +/* Define the synthetic interrupt message page layout. */
> > +struct hv_message_page {
> > +	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
> > +};
> > +
> >  #endif
> > diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> > index 46e23d1..d22230c 100644
> > --- a/drivers/hv/hyperv_vmbus.h
> > +++ b/drivers/hv/hyperv_vmbus.h
> > @@ -63,10 +63,6 @@ enum hv_cpuid_function {
> >  /* Define version of the synthetic interrupt controller. */
> >  #define HV_SYNIC_VERSION		(1)
> >
> > -/* Define synthetic interrupt controller message constants. */
> > -#define HV_MESSAGE_SIZE			(256)
> > -#define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
> > -#define HV_MESSAGE_PAYLOAD_QWORD_COUNT	(30)
> >  #define HV_ANY_VP			(0xFFFFFFFF)
> >
> >  /* Define synthetic interrupt controller flag constants. */
> > @@ -74,53 +70,9 @@ enum hv_cpuid_function {
> >  #define HV_EVENT_FLAGS_BYTE_COUNT	(256)
> >  #define HV_EVENT_FLAGS_DWORD_COUNT	(256 / sizeof(u32))
> >
> > -/* Define hypervisor message types. */
> > -enum hv_message_type {
> > -	HVMSG_NONE			= 0x00000000,
> > -
> > -	/* Memory access messages. */
> > -	HVMSG_UNMAPPED_GPA		= 0x80000000,
> > -	HVMSG_GPA_INTERCEPT		= 0x80000001,
> > -
> > -	/* Timer notification messages. */
> > -	HVMSG_TIMER_EXPIRED			= 0x80000010,
> > -
> > -	/* Error messages. */
> > -	HVMSG_INVALID_VP_REGISTER_VALUE	= 0x80000020,
> > -	HVMSG_UNRECOVERABLE_EXCEPTION	= 0x80000021,
> > -	HVMSG_UNSUPPORTED_FEATURE		= 0x80000022,
> > -
> > -	/* Trace buffer complete messages. */
> > -	HVMSG_EVENTLOG_BUFFERCOMPLETE	= 0x80000040,
> > -
> > -	/* Platform-specific processor intercept messages. */
> > -	HVMSG_X64_IOPORT_INTERCEPT		= 0x80010000,
> > -	HVMSG_X64_MSR_INTERCEPT		= 0x80010001,
> > -	HVMSG_X64_CPUID_INTERCEPT		= 0x80010002,
> > -	HVMSG_X64_EXCEPTION_INTERCEPT	= 0x80010003,
> > -	HVMSG_X64_APIC_EOI			= 0x80010004,
> > -	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
> > -};
> > -
> >  /* Define invalid partition identifier. */
> >  #define HV_PARTITION_ID_INVALID		((u64)0x0)
> >
> > -/* Define port identifier type. */
> > -union hv_port_id {
> > -	u32 asu32;
> > -	struct {
> > -		u32 id:24;
> > -		u32 reserved:8;
> > -	} u ;
> > -};
> > -
> > -/* Define port type. */
> > -enum hv_port_type {
> > -	HVPORT_MSG	= 1,
> > -	HVPORT_EVENT		= 2,
> > -	HVPORT_MONITOR	= 3
> > -};
> > -
> >  /* Define port information structure. */
> >  struct hv_port_info {
> >  	enum hv_port_type port_type;
> > @@ -161,27 +113,6 @@ struct hv_connection_info {
> >  	};
> >  };
> >
> > -/* Define synthetic interrupt controller message flags. */
> > -union hv_message_flags {
> > -	u8 asu8;
> > -	struct {
> > -		u8 msg_pending:1;
> > -		u8 reserved:7;
> > -	};
> > -};
> > -
> > -/* Define synthetic interrupt controller message header. */
> > -struct hv_message_header {
> > -	enum hv_message_type message_type;
> > -	u8 payload_size;
> > -	union hv_message_flags message_flags;
> > -	u8 reserved[2];
> > -	union {
> > -		u64 sender;
> > -		union hv_port_id port;
> > -	};
> > -};
> > -
> >  /*
> >   * Timer configuration register.
> >   */
> > @@ -198,31 +129,9 @@ union hv_timer_config {
> >  	};
> >  };
> >
> > -
> > -/* Define timer message payload structure. */
> > -struct hv_timer_message_payload {
> > -	u32 timer_index;
> > -	u32 reserved;
> > -	u64 expiration_time;	/* When the timer expired */
> > -	u64 delivery_time;	/* When the message was delivered */
> > -};
> > -
> > -/* Define synthetic interrupt controller message format. */
> > -struct hv_message {
> > -	struct hv_message_header header;
> > -	union {
> > -		u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
> > -	} u ;
> > -};
> > -
> >  /* Define the number of message buffers associated with each port. */
> >  #define HV_PORT_MESSAGE_BUFFER_COUNT	(16)
> >
> > -/* Define the synthetic interrupt message page layout. */
> > -struct hv_message_page {
> > -	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
> > -};
> > -
> >  /* Define the synthetic interrupt controller event flags format. */
> >  union hv_synic_event_flags {
> >  	u8 flags8[HV_EVENT_FLAGS_BYTE_COUNT];
> >

[Qemu-devel] [PATCH v1 3/7] kvm/x86: Rearrange func's declarations inside Hyper-V header

[Andrey Smetanin]

This rearrangement places functions declarations together
according to their functionality, so future additions
will be simplier.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/kvm/hyperv.h | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index 315af4b..9483d49 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -24,14 +24,6 @@
 #ifndef __ARCH_X86_KVM_HYPERV_H__
 #define __ARCH_X86_KVM_HYPERV_H__
 
-int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host);
-int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
-bool kvm_hv_hypercall_enabled(struct kvm *kvm);
-int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
-
-int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
-void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
-
 static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
 {
 	return &vcpu->arch.hyperv.synic;
@@ -46,10 +38,18 @@ static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
 	arch = container_of(hv, struct kvm_vcpu_arch, hyperv);
 	return container_of(arch, struct kvm_vcpu, arch);
 }
-void kvm_hv_irq_routing_update(struct kvm *kvm);
 
-void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
+int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host);
+int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
+
+bool kvm_hv_hypercall_enabled(struct kvm *kvm);
+int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
 
+void kvm_hv_irq_routing_update(struct kvm *kvm);
+int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
+void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
 int kvm_hv_activate_synic(struct kvm_vcpu *vcpu);
 
+void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
+
 #endif
-- 
2.4.3

[Qemu-devel] [PATCH v1 4/7] kvm/x86: Added Hyper-V vcpu_to_hv_vcpu()/hv_vcpu_to_vcpu() helpers

[Andrey Smetanin]

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/kvm/hyperv.h | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index 9483d49..d5d8217 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -24,21 +24,29 @@
 #ifndef __ARCH_X86_KVM_HYPERV_H__
 #define __ARCH_X86_KVM_HYPERV_H__
 
-static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
+static inline struct kvm_vcpu_hv *vcpu_to_hv_vcpu(struct kvm_vcpu *vcpu)
 {
-	return &vcpu->arch.hyperv.synic;
+	return &vcpu->arch.hyperv;
 }
 
-static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
+static inline struct kvm_vcpu *hv_vcpu_to_vcpu(struct kvm_vcpu_hv *hv_vcpu)
 {
-	struct kvm_vcpu_hv *hv;
 	struct kvm_vcpu_arch *arch;
 
-	hv = container_of(synic, struct kvm_vcpu_hv, synic);
-	arch = container_of(hv, struct kvm_vcpu_arch, hyperv);
+	arch = container_of(hv_vcpu, struct kvm_vcpu_arch, hyperv);
 	return container_of(arch, struct kvm_vcpu, arch);
 }
 
+static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
+{
+	return &vcpu->arch.hyperv.synic;
+}
+
+static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
+{
+	return hv_vcpu_to_vcpu(container_of(synic, struct kvm_vcpu_hv, synic));
+}
+
 int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host);
 int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
 
-- 
2.4.3

[Qemu-devel] [PATCH v1 5/7] kvm/x86: Hyper-V internal helper to read MSR HV_X64_MSR_TIME_REF_COUNT

[Andrey Smetanin]

This helper will be used also in Hyper-V SynIC timers implementation.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/kvm/hyperv.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 41869a9..9958926 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -335,6 +335,11 @@ static void synic_init(struct kvm_vcpu_hv_synic *synic)
 	}
 }
 
+static u64 get_time_ref_counter(struct kvm *kvm)
+{
+	return div_u64(get_kernel_ns() + kvm->arch.kvmclock_offset, 100);
+}
+
 void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu)
 {
 	synic_init(vcpu_to_synic(vcpu));
@@ -576,11 +581,9 @@ static int kvm_hv_get_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 	case HV_X64_MSR_HYPERCALL:
 		data = hv->hv_hypercall;
 		break;
-	case HV_X64_MSR_TIME_REF_COUNT: {
-		data =
-		     div_u64(get_kernel_ns() + kvm->arch.kvmclock_offset, 100);
+	case HV_X64_MSR_TIME_REF_COUNT:
+		data = get_time_ref_counter(kvm);
 		break;
-	}
 	case HV_X64_MSR_REFERENCE_TSC:
 		data = hv->hv_tsc_page;
 		break;
-- 
2.4.3

[Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Andrey Smetanin]

The SynIC message protocol mandates that the message slot is claimed
by atomically setting message type to something other than HVMSG_NONE.
If another message is to be delivered while the slot is still busy,
message pending flag is asserted to indicate to the guest that the
hypervisor wants to be notified when the slot is released.

To make sure the protocol works regardless of where the message
sources are (kernel or userspace), clear the pending flag on SINT ACK
notification, and let the message sources compete for the slot again.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/kvm/hyperv.c    | 31 +++++++++++++++++++++++++++++++
 include/linux/kvm_host.h |  2 ++
 2 files changed, 33 insertions(+)

diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 9958926..6412b6b 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -27,6 +27,7 @@
 #include "hyperv.h"
 
 #include <linux/kvm_host.h>
+#include <linux/highmem.h>
 #include <asm/apicdef.h>
 #include <trace/events/kvm.h>
 
@@ -116,13 +117,43 @@ static struct kvm_vcpu_hv_synic *synic_get(struct kvm *kvm, u32 vcpu_id)
 	return (synic->active) ? synic : NULL;
 }
 
+static void synic_clear_sint_msg_pending(struct kvm_vcpu_hv_synic *synic,
+					u32 sint)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	struct page *page;
+	gpa_t gpa;
+	struct hv_message *msg;
+	struct hv_message_page *msg_page;
+
+	gpa = synic->msg_page & PAGE_MASK;
+	page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
+	if (is_error_page(page)) {
+		vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa 0x%llx\n",
+			 gpa);
+		return;
+	}
+	msg_page = kmap_atomic(page);
+
+	msg = &msg_page->sint_message[sint];
+	msg->header.message_flags.msg_pending = 0;
+
+	kunmap_atomic(msg_page);
+	kvm_release_page_dirty(page);
+	kvm_vcpu_mark_page_dirty(vcpu, gpa >> PAGE_SHIFT);
+}
+
 static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
 {
 	struct kvm *kvm = vcpu->kvm;
+	struct kvm_vcpu_hv_synic *synic = vcpu_to_synic(vcpu);
 	int gsi, idx;
 
 	vcpu_debug(vcpu, "Hyper-V SynIC acked sint %d\n", sint);
 
+	if (synic->msg_page & HV_SYNIC_SIMP_ENABLE)
+		synic_clear_sint_msg_pending(synic, sint);
+
 	idx = srcu_read_lock(&kvm->irq_srcu);
 	gsi = atomic_read(&vcpu_to_synic(vcpu)->sint_to_gsi[sint]);
 	if (gsi != -1)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 2911919..9b64c8c 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -450,6 +450,8 @@ struct kvm {
 
 #define vcpu_debug(vcpu, fmt, ...)					\
 	kvm_debug("vcpu%i " fmt, (vcpu)->vcpu_id, ## __VA_ARGS__)
+#define vcpu_err(vcpu, fmt, ...)					\
+	kvm_err("vcpu%i " fmt, (vcpu)->vcpu_id, ## __VA_ARGS__)
 
 static inline struct kvm_vcpu *kvm_get_vcpu(struct kvm *kvm, int i)
 {
-- 
2.4.3

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Paolo Bonzini]

On 25/11/2015 16:20, Andrey Smetanin wrote:
> +static void synic_clear_sint_msg_pending(struct kvm_vcpu_hv_synic *synic,
> +					u32 sint)
> +{
> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
> +	struct page *page;
> +	gpa_t gpa;
> +	struct hv_message *msg;
> +	struct hv_message_page *msg_page;
> +
> +	gpa = synic->msg_page & PAGE_MASK;
> +	page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
> +	if (is_error_page(page)) {
> +		vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa 0x%llx\n",
> +			 gpa);
> +		return;
> +	}
> +	msg_page = kmap_atomic(page);

But the message page is not being pinned, is it?

Paolo

> +	msg = &msg_page->sint_message[sint];
> +	msg->header.message_flags.msg_pending = 0;
> +
> +	kunmap_atomic(msg_page);
> +	kvm_release_page_dirty(page);
> +	kvm_vcpu_mark_page_dirty(vcpu, gpa >> PAGE_SHIFT);
> +}
> +

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Andrey Smetanin]

On 11/25/2015 07:52 PM, Paolo Bonzini wrote:
>
>
> On 25/11/2015 16:20, Andrey Smetanin wrote:
>> +static void synic_clear_sint_msg_pending(struct kvm_vcpu_hv_synic *synic,
>> +					u32 sint)
>> +{
>> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
>> +	struct page *page;
>> +	gpa_t gpa;
>> +	struct hv_message *msg;
>> +	struct hv_message_page *msg_page;
>> +
>> +	gpa = synic->msg_page & PAGE_MASK;
>> +	page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
>> +	if (is_error_page(page)) {
>> +		vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa 0x%llx\n",
>> +			 gpa);
>> +		return;
>> +	}
>> +	msg_page = kmap_atomic(page);
>
> But the message page is not being pinned, is it?
>
Actually I don't know anything about pinning.
Is it pinning against page swapping ?
Could you please clarify and provide an API to use in this case ?
> Paolo
>
>> +	msg = &msg_page->sint_message[sint];
>> +	msg->header.message_flags.msg_pending = 0;
>> +
>> +	kunmap_atomic(msg_page);
>> +	kvm_release_page_dirty(page);
>> +	kvm_vcpu_mark_page_dirty(vcpu, gpa >> PAGE_SHIFT);
>> +}
>> +

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Paolo Bonzini]

On 25/11/2015 17:55, Andrey Smetanin wrote:
>>
>> +    gpa = synic->msg_page & PAGE_MASK;
>> +    page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
>> +    if (is_error_page(page)) {
>> +        vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa 0x%llx\n",
>> +             gpa);
>> +        return;
>> +    }
>> +    msg_page = kmap_atomic(page);
>
> But the message page is not being pinned, is it?
>
> Actually I don't know anything about pinning.
> Is it pinning against page swapping ?

Yes.  Unless the page is pinned, kmap_atomic can fail.

However, I don't think that kvm_hv_notify_acked_sint is called from
atomic context.  It is only called from apic_set_eoi.  Could you just
use kvm_vcpu_write_guest_page?

By the way, do you need to do this also in kvm_get_apic_interrupt, for
auto EOI interrupts?

Thanks,

Paolo

> Could you please clarify and provide an API to use in this case ?

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Andrey Smetanin]

On 11/25/2015 08:14 PM, Paolo Bonzini wrote:
>
>
> On 25/11/2015 17:55, Andrey Smetanin wrote:
>>>
>>> +    gpa = synic->msg_page & PAGE_MASK;
>>> +    page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
>>> +    if (is_error_page(page)) {
>>> +        vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa 0x%llx\n",
>>> +             gpa);
>>> +        return;
>>> +    }
>>> +    msg_page = kmap_atomic(page);
>>
>> But the message page is not being pinned, is it?
>>
>> Actually I don't know anything about pinning.
>> Is it pinning against page swapping ?
>
> Yes.  Unless the page is pinned, kmap_atomic can fail.
kmap_atomic() can't fail for a valid page struct. Does
kvm_vcpu_gfn_to_page() can provide invalid page(swapped page) struct 
which may pass is_error_page(page) check but can leads to incorrect
behavior inside kmap_atomic()?
>
> However, I don't think that kvm_hv_notify_acked_sint is called from
> atomic context.  It is only called from apic_set_eoi.  Could you just
> use kvm_vcpu_write_guest_page?
In this case I can use kvm_vcpu_write_guest_page(), but in the 'PATCH v1 
7/7' I do the same page mapping method to sync_cmpxchg() at guest 
message page address to exclusively acquire message page slot(see 
synic_deliver_msg()). So we need some method to map and access 
atomically memory of guest page in KVM. Does any method to pin and map 
guest page in kernel exists? Or should we use mlock() for this page in 
QEMU part ?
>
> By the way, do you need to do this also in kvm_get_apic_interrupt, for
> auto EOI interrupts?
No we don't need this because in case of auto EOI interrupts, if 
->msg_pending was set, host will receive HV_X64_MSR_EOM write request 
which calls kvm_hv_notify_acked_sint().
>
> Thanks,
>
> Paolo
>
>> Could you please clarify and provide an API to use in this case ?

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Paolo Bonzini]

On 26/11/2015 10:06, Andrey Smetanin wrote:
> 
> 
> On 11/25/2015 08:14 PM, Paolo Bonzini wrote:
>>
>>
>> On 25/11/2015 17:55, Andrey Smetanin wrote:
>>>>
>>>> +    gpa = synic->msg_page & PAGE_MASK;
>>>> +    page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
>>>> +    if (is_error_page(page)) {
>>>> +        vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa
>>>> 0x%llx\n",
>>>> +             gpa);
>>>> +        return;
>>>> +    }
>>>> +    msg_page = kmap_atomic(page);
>>>
>>> But the message page is not being pinned, is it?
>>>
>>> Actually I don't know anything about pinning.
>>> Is it pinning against page swapping ?
>>
>> Yes.  Unless the page is pinned, kmap_atomic can fail.
> kmap_atomic() can't fail for a valid page struct. Does
> kvm_vcpu_gfn_to_page() can provide invalid page(swapped page) struct
> which may pass is_error_page(page) check but can leads to incorrect
> behavior inside kmap_atomic()?

No, you're right.  Nevermind, I was confused because I thought you
needed kmap_atomic rather than kmap.  Here using kmap_atomic is just an
optimization, so it's okay.  (If you needed kmap_atomic, the problem
would have been that kvm_vcpu_gfn_to_page() can sleep).

In patch 7/7 you're also not in atomic context, so kvm_vcpu_gfn_to_page
is okay.

Shouldn't have reviewed the patch when tired. :)

Then the patches look good, I think.  With a testcase I can try them out
and hopefully merge them for Linux 4.5 / QEMU 2.6.

Paolo

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Andrey Smetanin]

On 11/26/2015 05:43 PM, Paolo Bonzini wrote:
>
>
> On 26/11/2015 10:06, Andrey Smetanin wrote:
>>
>>
>> On 11/25/2015 08:14 PM, Paolo Bonzini wrote:
>>>
>>>
>>> On 25/11/2015 17:55, Andrey Smetanin wrote:
>>>>>
>>>>> +    gpa = synic->msg_page & PAGE_MASK;
>>>>> +    page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
>>>>> +    if (is_error_page(page)) {
>>>>> +        vcpu_err(vcpu, "Hyper-V SynIC can't get msg page, gpa
>>>>> 0x%llx\n",
>>>>> +             gpa);
>>>>> +        return;
>>>>> +    }
>>>>> +    msg_page = kmap_atomic(page);
>>>>
>>>> But the message page is not being pinned, is it?
>>>>
>>>> Actually I don't know anything about pinning.
>>>> Is it pinning against page swapping ?
>>>
>>> Yes.  Unless the page is pinned, kmap_atomic can fail.
>> kmap_atomic() can't fail for a valid page struct. Does
>> kvm_vcpu_gfn_to_page() can provide invalid page(swapped page) struct
>> which may pass is_error_page(page) check but can leads to incorrect
>> behavior inside kmap_atomic()?
>
> No, you're right.  Nevermind, I was confused because I thought you
> needed kmap_atomic rather than kmap.  Here using kmap_atomic is just an
> optimization, so it's okay.  (If you needed kmap_atomic, the problem
> would have been that kvm_vcpu_gfn_to_page() can sleep).
>
> In patch 7/7 you're also not in atomic context, so kvm_vcpu_gfn_to_page
> is okay.
>
> Shouldn't have reviewed the patch when tired. :)
>
> Then the patches look good, I think.  With a testcase I can try them out
> and hopefully merge them for Linux 4.5 / QEMU 2.6.
Thank you!

We already have a working Hyper-V SynIC timers kvm-unit-tests test case.
We are going to send appropriate patches seria into kvm-unit-tests git.

But  kvm-unit-tests master now broken:
 > make
objcopy -O elf32-i386 x86/memory.elf x86/memory.flat
make: *** No rule to make target 'x86/pku.o', needed by 'x86/pku.elf'. 
Stop.

The problem is in latest commit 3da70799dd3cf1169c4668b4a3fd6f598528b8b9.

The commit adds 'pku' test case building, but not added any pku.c 
implementation file.

[root@asm-pc kvm-unit-tests]# ls -al x86/pku.c
ls: cannot access x86/pku.c: No such file or directory


Could you please fix it ?
>
> Paolo
>

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Paolo Bonzini]

On 26/11/2015 16:53, Andrey Smetanin wrote:
>> Then the patches look good, I think.  With a testcase I can try them out
>> and hopefully merge them for Linux 4.5 / QEMU 2.6.
> 
> Thank you!
> 
> We already have a working Hyper-V SynIC timers kvm-unit-tests test case.
> We are going to send appropriate patches seria into kvm-unit-tests git.
> 
> But  kvm-unit-tests master now broken:
>> make
> objcopy -O elf32-i386 x86/memory.elf x86/memory.flat
> make: *** No rule to make target 'x86/pku.o', needed by 'x86/pku.elf'.
> Stop.
> 
> The problem is in latest commit 3da70799dd3cf1169c4668b4a3fd6f598528b8b9.
> 
> The commit adds 'pku' test case building, but not added any pku.c
> implementation file.
> 
> [root@asm-pc kvm-unit-tests]# ls -al x86/pku.c
> ls: cannot access x86/pku.c: No such file or directory
> 
> 
> Could you please fix it ?

Done, sorry.

Paolo

Re: [Qemu-devel] [PATCH v1 6/7] kvm/x86: Hyper-V SynIC message slot pending clearing at SINT ack

[Roman Kagan]

On Wed, Nov 25, 2015 at 06:20:20PM +0300, Andrey Smetanin wrote:
> The SynIC message protocol mandates that the message slot is claimed
> by atomically setting message type to something other than HVMSG_NONE.
> If another message is to be delivered while the slot is still busy,
> message pending flag is asserted to indicate to the guest that the
> hypervisor wants to be notified when the slot is released.
> 
> To make sure the protocol works regardless of where the message
> sources are (kernel or userspace), clear the pending flag on SINT ACK
> notification, and let the message sources compete for the slot again.
> 
> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> CC: Gleb Natapov <gleb@kernel.org>
> CC: Paolo Bonzini <pbonzini@redhat.com>
> CC: "K. Y. Srinivasan" <kys@microsoft.com>
> CC: Haiyang Zhang <haiyangz@microsoft.com>
> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> CC: Roman Kagan <rkagan@virtuozzo.com>
> CC: Denis V. Lunev <den@openvz.org>
> CC: qemu-devel@nongnu.org

Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>

Roman.

[Qemu-devel] [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Andrey Smetanin]

Per Hyper-V specification (and as required by Hyper-V-aware guests),
SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
of MSRs, and signals expiration by delivering a special format message
to the configured SynIC message slot and triggering the corresponding
synthetic interrupt.

Note: as implemented by this patch, all periodic timers are "lazy"
(i.e. if the vCPU wasn't scheduled for more than the timer period the
timer events are lost), regardless of the corresponding configuration
MSR.  If deemed necessary, the "catch up" mode (the timer period is
shortened until the timer catches up) will be implemented later.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Haiyang Zhang <haiyangz@microsoft.com>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>
CC: Denis V. Lunev <den@openvz.org>
CC: qemu-devel@nongnu.org
---
 arch/x86/include/asm/kvm_host.h    |  13 ++
 arch/x86/include/uapi/asm/hyperv.h |   6 +
 arch/x86/kvm/hyperv.c              | 325 ++++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/hyperv.h              |  24 +++
 arch/x86/kvm/x86.c                 |   9 +
 include/linux/kvm_host.h           |   1 +
 6 files changed, 375 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index f608e17..e35c5ca 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -375,6 +375,17 @@ struct kvm_mtrr {
 	struct list_head head;
 };
 
+/* Hyper-V SynIC timer */
+struct kvm_vcpu_hv_stimer {
+	struct hrtimer timer;
+	int index;
+	u64 config;
+	u64 count;
+	u64 exp_time;
+	struct hv_message msg;
+	bool msg_pending;
+};
+
 /* Hyper-V synthetic interrupt controller (SynIC)*/
 struct kvm_vcpu_hv_synic {
 	u64 version;
@@ -394,6 +405,8 @@ struct kvm_vcpu_hv {
 	s64 runtime_offset;
 	struct kvm_vcpu_hv_synic synic;
 	struct kvm_hyperv_exit exit;
+	struct kvm_vcpu_hv_stimer stimer[HV_SYNIC_STIMER_COUNT];
+	DECLARE_BITMAP(stimer_pending_bitmap, HV_SYNIC_STIMER_COUNT);
 };
 
 struct kvm_vcpu_arch {
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index e86d77e..f9d3349 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -362,4 +362,10 @@ struct hv_message_page {
 	struct hv_message sint_message[HV_SYNIC_SINT_COUNT];
 };
 
+#define HV_STIMER_ENABLE		(1ULL << 0)
+#define HV_STIMER_PERIODIC		(1ULL << 1)
+#define HV_STIMER_LAZY			(1ULL << 2)
+#define HV_STIMER_AUTOENABLE		(1ULL << 3)
+#define HV_STIMER_SINT(config)		(__u8)(((config) >> 16) & 0x0F)
+
 #endif
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 6412b6b..9f8eb82 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -147,15 +147,32 @@ static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
 {
 	struct kvm *kvm = vcpu->kvm;
 	struct kvm_vcpu_hv_synic *synic = vcpu_to_synic(vcpu);
-	int gsi, idx;
+	struct kvm_vcpu_hv *hv_vcpu = vcpu_to_hv_vcpu(vcpu);
+	struct kvm_vcpu_hv_stimer *stimer;
+	int gsi, idx, stimers_pending;
 
 	vcpu_debug(vcpu, "Hyper-V SynIC acked sint %d\n", sint);
 
 	if (synic->msg_page & HV_SYNIC_SIMP_ENABLE)
 		synic_clear_sint_msg_pending(synic, sint);
 
+	/* Try to deliver pending Hyper-V SynIC timers messages */
+	stimers_pending = 0;
+	for (idx = 0; idx < ARRAY_SIZE(hv_vcpu->stimer); idx++) {
+		stimer = &hv_vcpu->stimer[idx];
+		if (stimer->msg_pending &&
+		    (stimer->config & HV_STIMER_ENABLE) &&
+		    HV_STIMER_SINT(stimer->config) == sint) {
+			set_bit(stimer->index,
+				hv_vcpu->stimer_pending_bitmap);
+			stimers_pending++;
+		}
+	}
+	if (stimers_pending)
+		kvm_make_request(KVM_REQ_HV_STIMER, vcpu);
+
 	idx = srcu_read_lock(&kvm->irq_srcu);
-	gsi = atomic_read(&vcpu_to_synic(vcpu)->sint_to_gsi[sint]);
+	gsi = atomic_read(&synic->sint_to_gsi[sint]);
 	if (gsi != -1)
 		kvm_notify_acked_gsi(kvm, gsi);
 	srcu_read_unlock(&kvm->irq_srcu, idx);
@@ -371,9 +388,275 @@ static u64 get_time_ref_counter(struct kvm *kvm)
 	return div_u64(get_kernel_ns() + kvm->arch.kvmclock_offset, 100);
 }
 
+static void stimer_mark_expired(struct kvm_vcpu_hv_stimer *stimer,
+				bool vcpu_kick)
+{
+	struct kvm_vcpu *vcpu = stimer_to_vcpu(stimer);
+
+	set_bit(stimer->index,
+		vcpu_to_hv_vcpu(vcpu)->stimer_pending_bitmap);
+	kvm_make_request(KVM_REQ_HV_STIMER, vcpu);
+	if (vcpu_kick)
+		kvm_vcpu_kick(vcpu);
+}
+
+static void stimer_stop(struct kvm_vcpu_hv_stimer *stimer)
+{
+	hrtimer_cancel(&stimer->timer);
+}
+
+static void stimer_cleanup(struct kvm_vcpu_hv_stimer *stimer)
+{
+	struct kvm_vcpu *vcpu = stimer_to_vcpu(stimer);
+
+	stimer_stop(stimer);
+	clear_bit(stimer->index,
+		  vcpu_to_hv_vcpu(vcpu)->stimer_pending_bitmap);
+	stimer->msg_pending = false;
+}
+
+static enum hrtimer_restart stimer_timer_callback(struct hrtimer *timer)
+{
+	struct kvm_vcpu_hv_stimer *stimer;
+
+	stimer = container_of(timer, struct kvm_vcpu_hv_stimer, timer);
+	stimer_mark_expired(stimer, true);
+
+	return HRTIMER_NORESTART;
+}
+
+static void stimer_restart(struct kvm_vcpu_hv_stimer *stimer)
+{
+	u64 time_now;
+	ktime_t ktime_now;
+	u64 n;
+
+	time_now = get_time_ref_counter(stimer_to_vcpu(stimer)->kvm);
+	ktime_now = ktime_get();
+
+	/*
+	 * Calculate positive integer n for which condtion -
+	 * (stimer->exp_time + n * stimer->count) > time_now
+	 * is true. We will use (stimer->exp_time + n * stimer->count)
+	 * as new stimer->exp_time.
+	 */
+
+	n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
+	stimer->exp_time += n * stimer->count;
+
+	hrtimer_start(&stimer->timer,
+		      ktime_add_ns(ktime_now,
+				   100 * (stimer->exp_time - time_now)),
+		      HRTIMER_MODE_ABS);
+}
+
+static int stimer_start(struct kvm_vcpu_hv_stimer *stimer)
+{
+	u64 time_now;
+	ktime_t ktime_now;
+
+	time_now = get_time_ref_counter(stimer_to_vcpu(stimer)->kvm);
+	ktime_now = ktime_get();
+
+	if (stimer->config & HV_STIMER_PERIODIC) {
+		if (stimer->count == 0)
+			return -EINVAL;
+
+		stimer->exp_time = time_now + stimer->count;
+		hrtimer_start(&stimer->timer,
+			      ktime_add_ns(ktime_now, 100 * stimer->count),
+			      HRTIMER_MODE_ABS);
+		return 0;
+	}
+	stimer->exp_time = stimer->count;
+	if (time_now >= stimer->count) {
+		/*
+		 * Expire timer according to Hypervisor Top-Level Functional
+		 * specification v4(15.3.1):
+		 * "If a one shot is enabled and the specified count is in
+		 * the past, it will expire immediately."
+		 */
+		stimer_mark_expired(stimer, false);
+		return 0;
+	}
+
+	hrtimer_start(&stimer->timer,
+		      ktime_add_ns(ktime_now, 100 * (stimer->count - time_now)),
+		      HRTIMER_MODE_ABS);
+	return 0;
+}
+
+static int stimer_set_config(struct kvm_vcpu_hv_stimer *stimer, u64 config,
+			     bool host)
+{
+	if (stimer->count == 0 || HV_STIMER_SINT(config) == 0)
+		config &= ~HV_STIMER_ENABLE;
+	stimer->config = config;
+	stimer_cleanup(stimer);
+	if (stimer->config & HV_STIMER_ENABLE)
+		if (stimer_start(stimer))
+			return 1;
+	return 0;
+}
+
+static int stimer_set_count(struct kvm_vcpu_hv_stimer *stimer, u64 count,
+			    bool host)
+{
+	stimer->count = count;
+
+	stimer_cleanup(stimer);
+	if (stimer->count == 0)
+		stimer->config &= ~HV_STIMER_ENABLE;
+	else if (stimer->config & HV_STIMER_AUTOENABLE) {
+		stimer->config |= HV_STIMER_ENABLE;
+		if (stimer_start(stimer))
+			return 1;
+	}
+
+	return 0;
+}
+
+static int stimer_get_config(struct kvm_vcpu_hv_stimer *stimer, u64 *pconfig)
+{
+	*pconfig = stimer->config;
+	return 0;
+}
+
+static int stimer_get_count(struct kvm_vcpu_hv_stimer *stimer, u64 *pcount)
+{
+	*pcount = stimer->count;
+	return 0;
+}
+
+static int synic_deliver_msg(struct kvm_vcpu_hv_synic *synic, u32 sint,
+			     struct hv_message *src_msg)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	struct page *page;
+	gpa_t gpa;
+	struct hv_message *dst_msg;
+	int r;
+	struct hv_message_page *msg_page;
+
+	if (!(synic->msg_page & HV_SYNIC_SIMP_ENABLE))
+		return -ENOENT;
+
+	gpa = synic->msg_page & PAGE_MASK;
+	page = kvm_vcpu_gfn_to_page(vcpu, gpa >> PAGE_SHIFT);
+	if (is_error_page(page))
+		return -EFAULT;
+
+	msg_page = kmap_atomic(page);
+	dst_msg = &msg_page->sint_message[sint];
+	if (sync_cmpxchg(&dst_msg->header.message_type, HVMSG_NONE,
+			 src_msg->header.message_type) != HVMSG_NONE) {
+		dst_msg->header.message_flags.msg_pending = 1;
+		r = -EAGAIN;
+	} else {
+		memcpy(&dst_msg->u.payload, &src_msg->u.payload,
+		       src_msg->header.payload_size);
+		dst_msg->header.message_type = src_msg->header.message_type;
+		dst_msg->header.payload_size = src_msg->header.payload_size;
+		r = synic_set_irq(synic, sint);
+		if (r >= 1)
+			r = 0;
+		else if (r == 0)
+			r = -EFAULT;
+	}
+	kunmap_atomic(msg_page);
+	kvm_release_page_dirty(page);
+	kvm_vcpu_mark_page_dirty(vcpu, gpa >> PAGE_SHIFT);
+	return r;
+}
+
+static void stimer_send_msg(struct kvm_vcpu_hv_stimer *stimer)
+{
+	struct kvm_vcpu *vcpu = stimer_to_vcpu(stimer);
+	struct hv_message *msg = &stimer->msg;
+	struct hv_timer_message_payload *payload =
+			(struct hv_timer_message_payload *)&msg->u.payload;
+	int r;
+
+	stimer->msg_pending = true;
+	payload->expiration_time = stimer->exp_time;
+	payload->delivery_time = get_time_ref_counter(vcpu->kvm);
+	r = synic_deliver_msg(vcpu_to_synic(vcpu),
+			      HV_STIMER_SINT(stimer->config), msg);
+	if (!r)
+		stimer->msg_pending = false;
+}
+
+static void stimer_expiration(struct kvm_vcpu_hv_stimer *stimer)
+{
+	stimer_send_msg(stimer);
+	if (!(stimer->config & HV_STIMER_PERIODIC))
+		stimer->config |= ~HV_STIMER_ENABLE;
+	else
+		stimer_restart(stimer);
+}
+
+void kvm_hv_process_stimers(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vcpu_hv *hv_vcpu = vcpu_to_hv_vcpu(vcpu);
+	struct kvm_vcpu_hv_stimer *stimer;
+	u64 time_now;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(hv_vcpu->stimer); i++)
+		if (test_and_clear_bit(i, hv_vcpu->stimer_pending_bitmap)) {
+			stimer = &hv_vcpu->stimer[i];
+			stimer_stop(stimer);
+			if (stimer->config & HV_STIMER_ENABLE) {
+				time_now = get_time_ref_counter(vcpu->kvm);
+				if (time_now >= stimer->exp_time)
+					stimer_expiration(stimer);
+			}
+		}
+}
+
+void kvm_hv_vcpu_uninit(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vcpu_hv *hv_vcpu = vcpu_to_hv_vcpu(vcpu);
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(hv_vcpu->stimer); i++)
+		stimer_cleanup(&hv_vcpu->stimer[i]);
+}
+
+static void stimer_prepare_msg(struct kvm_vcpu_hv_stimer *stimer)
+{
+	struct hv_message *msg = &stimer->msg;
+	struct hv_timer_message_payload *payload =
+			(struct hv_timer_message_payload *)&msg->u.payload;
+
+	memset(&msg->header, 0, sizeof(msg->header));
+	msg->header.message_type = HVMSG_TIMER_EXPIRED;
+	msg->header.payload_size = sizeof(*payload);
+
+	payload->timer_index = stimer->index;
+	payload->expiration_time = 0;
+	payload->delivery_time = 0;
+}
+
+static void stimer_init(struct kvm_vcpu_hv_stimer *stimer, int timer_index)
+{
+	memset(stimer, 0, sizeof(*stimer));
+	stimer->index = timer_index;
+	hrtimer_init(&stimer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
+	stimer->timer.function = stimer_timer_callback;
+	stimer_prepare_msg(stimer);
+}
+
 void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu)
 {
-	synic_init(vcpu_to_synic(vcpu));
+	struct kvm_vcpu_hv *hv_vcpu = vcpu_to_hv_vcpu(vcpu);
+	int i;
+
+	synic_init(&hv_vcpu->synic);
+
+	bitmap_zero(hv_vcpu->stimer_pending_bitmap, HV_SYNIC_STIMER_COUNT);
+	for (i = 0; i < ARRAY_SIZE(hv_vcpu->stimer); i++)
+		stimer_init(&hv_vcpu->stimer[i], i);
 }
 
 int kvm_hv_activate_synic(struct kvm_vcpu *vcpu)
@@ -590,6 +873,24 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)
 	case HV_X64_MSR_EOM:
 	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
 		return synic_set_msr(vcpu_to_synic(vcpu), msr, data, host);
+	case HV_X64_MSR_STIMER0_CONFIG:
+	case HV_X64_MSR_STIMER1_CONFIG:
+	case HV_X64_MSR_STIMER2_CONFIG:
+	case HV_X64_MSR_STIMER3_CONFIG: {
+		int timer_index = (msr - HV_X64_MSR_STIMER0_CONFIG)/2;
+
+		return stimer_set_config(vcpu_to_stimer(vcpu, timer_index),
+					 data, host);
+	}
+	case HV_X64_MSR_STIMER0_COUNT:
+	case HV_X64_MSR_STIMER1_COUNT:
+	case HV_X64_MSR_STIMER2_COUNT:
+	case HV_X64_MSR_STIMER3_COUNT: {
+		int timer_index = (msr - HV_X64_MSR_STIMER0_COUNT)/2;
+
+		return stimer_set_count(vcpu_to_stimer(vcpu, timer_index),
+					data, host);
+	}
 	default:
 		vcpu_unimpl(vcpu, "Hyper-V uhandled wrmsr: 0x%x data 0x%llx\n",
 			    msr, data);
@@ -673,6 +974,24 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 	case HV_X64_MSR_EOM:
 	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
 		return synic_get_msr(vcpu_to_synic(vcpu), msr, pdata);
+	case HV_X64_MSR_STIMER0_CONFIG:
+	case HV_X64_MSR_STIMER1_CONFIG:
+	case HV_X64_MSR_STIMER2_CONFIG:
+	case HV_X64_MSR_STIMER3_CONFIG: {
+		int timer_index = (msr - HV_X64_MSR_STIMER0_CONFIG)/2;
+
+		return stimer_get_config(vcpu_to_stimer(vcpu, timer_index),
+					 pdata);
+	}
+	case HV_X64_MSR_STIMER0_COUNT:
+	case HV_X64_MSR_STIMER1_COUNT:
+	case HV_X64_MSR_STIMER2_COUNT:
+	case HV_X64_MSR_STIMER3_COUNT: {
+		int timer_index = (msr - HV_X64_MSR_STIMER0_COUNT)/2;
+
+		return stimer_get_count(vcpu_to_stimer(vcpu, timer_index),
+					pdata);
+	}
 	default:
 		vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
 		return 1;
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index d5d8217..60eccd4 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -59,5 +59,29 @@ void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
 int kvm_hv_activate_synic(struct kvm_vcpu *vcpu);
 
 void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
+void kvm_hv_vcpu_uninit(struct kvm_vcpu *vcpu);
+
+static inline struct kvm_vcpu_hv_stimer *vcpu_to_stimer(struct kvm_vcpu *vcpu,
+							int timer_index)
+{
+	return &vcpu_to_hv_vcpu(vcpu)->stimer[timer_index];
+}
+
+static inline struct kvm_vcpu *stimer_to_vcpu(struct kvm_vcpu_hv_stimer *stimer)
+{
+	struct kvm_vcpu_hv *hv_vcpu;
+
+	hv_vcpu = container_of(stimer - stimer->index, struct kvm_vcpu_hv,
+			       stimer[0]);
+	return hv_vcpu_to_vcpu(hv_vcpu);
+}
+
+static inline bool kvm_hv_has_stimer_pending(struct kvm_vcpu *vcpu)
+{
+	return !bitmap_empty(vcpu->arch.hyperv.stimer_pending_bitmap,
+			     HV_SYNIC_STIMER_COUNT);
+}
+
+void kvm_hv_process_stimers(struct kvm_vcpu *vcpu);
 
 #endif
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f1d6501..b6102c1 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -967,6 +967,7 @@ static u32 emulated_msrs[] = {
 	HV_X64_MSR_VP_INDEX,
 	HV_X64_MSR_VP_RUNTIME,
 	HV_X64_MSR_SCONTROL,
+	HV_X64_MSR_STIMER0_CONFIG,
 	HV_X64_MSR_APIC_ASSIST_PAGE, MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
 	MSR_KVM_PV_EOI_EN,
 
@@ -2199,6 +2200,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
 	case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 	case HV_X64_MSR_CRASH_CTL:
+	case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:
 		return kvm_hv_set_msr_common(vcpu, msr, data,
 					     msr_info->host_initiated);
 	case MSR_IA32_BBL_CR_CTL3:
@@ -2403,6 +2405,7 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
 	case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 	case HV_X64_MSR_CRASH_CTL:
+	case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:
 		return kvm_hv_get_msr_common(vcpu,
 					     msr_info->index, &msr_info->data);
 		break;
@@ -6489,6 +6492,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 			r = 0;
 			goto out;
 		}
+		if (kvm_check_request(KVM_REQ_HV_STIMER, vcpu))
+			kvm_hv_process_stimers(vcpu);
 	}
 
 	/*
@@ -7649,6 +7654,7 @@ void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu)
 {
 	int idx;
 
+	kvm_hv_vcpu_uninit(vcpu);
 	kvm_pmu_destroy(vcpu);
 	kfree(vcpu->arch.mce_banks);
 	kvm_free_lapic(vcpu);
@@ -8043,6 +8049,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)
 	    kvm_cpu_has_interrupt(vcpu))
 		return true;
 
+	if (kvm_hv_has_stimer_pending(vcpu))
+		return true;
+
 	return false;
 }
 
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9b64c8c..57a7ce8 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -144,6 +144,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_IOAPIC_EOI_EXIT   28
 #define KVM_REQ_HV_RESET          29
 #define KVM_REQ_HV_EXIT           30
+#define KVM_REQ_HV_STIMER         31
 
 #define KVM_USERSPACE_IRQ_SOURCE_ID		0
 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
-- 
2.4.3

Re: [Qemu-devel] [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Roman Kagan]

On Wed, Nov 25, 2015 at 06:20:21PM +0300, Andrey Smetanin wrote:
> Per Hyper-V specification (and as required by Hyper-V-aware guests),
> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
> of MSRs, and signals expiration by delivering a special format message
> to the configured SynIC message slot and triggering the corresponding
> synthetic interrupt.
> 
> Note: as implemented by this patch, all periodic timers are "lazy"
> (i.e. if the vCPU wasn't scheduled for more than the timer period the
> timer events are lost), regardless of the corresponding configuration
> MSR.  If deemed necessary, the "catch up" mode (the timer period is
> shortened until the timer catches up) will be implemented later.
> 
> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> CC: Gleb Natapov <gleb@kernel.org>
> CC: Paolo Bonzini <pbonzini@redhat.com>
> CC: "K. Y. Srinivasan" <kys@microsoft.com>
> CC: Haiyang Zhang <haiyangz@microsoft.com>
> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> CC: Roman Kagan <rkagan@virtuozzo.com>
> CC: Denis V. Lunev <den@openvz.org>
> CC: qemu-devel@nongnu.org
> ---
>  arch/x86/include/asm/kvm_host.h    |  13 ++
>  arch/x86/include/uapi/asm/hyperv.h |   6 +
>  arch/x86/kvm/hyperv.c              | 325 ++++++++++++++++++++++++++++++++++++-
>  arch/x86/kvm/hyperv.h              |  24 +++
>  arch/x86/kvm/x86.c                 |   9 +
>  include/linux/kvm_host.h           |   1 +
>  6 files changed, 375 insertions(+), 3 deletions(-)

A couple of nitpicks:

> +static void stimer_restart(struct kvm_vcpu_hv_stimer *stimer)
> +{
> +	u64 time_now;
> +	ktime_t ktime_now;
> +	u64 n;
> +
> +	time_now = get_time_ref_counter(stimer_to_vcpu(stimer)->kvm);
> +	ktime_now = ktime_get();
> +
> +	/*
> +	 * Calculate positive integer n for which condtion -
> +	 * (stimer->exp_time + n * stimer->count) > time_now
> +	 * is true. We will use (stimer->exp_time + n * stimer->count)
> +	 * as new stimer->exp_time.
> +	 */
> +
> +	n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
> +	stimer->exp_time += n * stimer->count;

This is actually just a reminder calculation so I'd rather do it
directly with div64_u64_rem().

> +void kvm_hv_process_stimers(struct kvm_vcpu *vcpu)
> +{
> +	struct kvm_vcpu_hv *hv_vcpu = vcpu_to_hv_vcpu(vcpu);
> +	struct kvm_vcpu_hv_stimer *stimer;
> +	u64 time_now;
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(hv_vcpu->stimer); i++)
> +		if (test_and_clear_bit(i, hv_vcpu->stimer_pending_bitmap)) {
> +			stimer = &hv_vcpu->stimer[i];
> +			stimer_stop(stimer);

I think there's no need in this explicit stop: I see no way to arrive
here with a running timer, and even if there were, it would be safe as
your timer callback only manipulates the bitmaps atomically.

Neither comment is critical so

Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>

Roman.

Re: [Qemu-devel] [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Paolo Bonzini]

On 27/11/2015 09:12, Roman Kagan wrote:
>> > +	n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
>> > +	stimer->exp_time += n * stimer->count;
> This is actually just a reminder calculation so I'd rather do it
> directly with div64_u64_rem().

It took me a while to understand why it was a remained. :)  Expanding
Andrey's formula you get

	exp_time = exp_time + count + (time_now - exp_time) / count * count

the remainder, (time_now - exp_time) % count would be

	time_now - exp_time - (time_now - exp_time) / count * count

so -((time_now - exp_time) % count) is

	exp_time - time_now + (time_now - exp_time) / count * count

so Andrey's expression is

	exp_time = time_now + (count - (time_now - exp_time) % count)

Yeah, that looks nice.

Paolo

Re: [Qemu-devel] [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Andrey Smetanin]

On 11/27/2015 01:49 PM, Paolo Bonzini wrote:
>
>
> On 27/11/2015 09:12, Roman Kagan wrote:
>>>> +	n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
>>>> +	stimer->exp_time += n * stimer->count;
>> This is actually just a reminder calculation so I'd rather do it
>> directly with div64_u64_rem().
>
> It took me a while to understand why it was a remained. :)  Expanding
> Andrey's formula you get
>
> 	exp_time = exp_time + count + (time_now - exp_time) / count * count
>
> the remainder, (time_now - exp_time) % count would be
>
> 	time_now - exp_time - (time_now - exp_time) / count * count
>
> so -((time_now - exp_time) % count) is
>
> 	exp_time - time_now + (time_now - exp_time) / count * count
>
> so Andrey's expression is
>
> 	exp_time = time_now + (count - (time_now - exp_time) % count)
>
> Yeah, that looks nice.
I'll redo this way.
>
> Paolo
>

Re: [Qemu-devel] [PATCH v1 7/7] kvm/x86: Hyper-V SynIC timers

[Roman Kagan]

On Fri, Nov 27, 2015 at 11:49:40AM +0100, Paolo Bonzini wrote:

[ sorry missed your message on Friday, replying now ]

> On 27/11/2015 09:12, Roman Kagan wrote:
> >> > +	n = div64_u64(time_now - stimer->exp_time, stimer->count) + 1;
> >> > +	stimer->exp_time += n * stimer->count;
> > This is actually just a reminder calculation so I'd rather do it
> > directly with div64_u64_rem().
> 
> It took me a while to understand why it was a remained. :)

It gets easier if you think of it this way: we've slipped a few whole
periods and the remainder of the slack into the current period, so
the time left till the next tick is ("count" is the timer period here)

  delta = count - slack % count
where
  slack = time_now - exp_time

This gives you immediately your

> 	exp_time = time_now + (count - (time_now - exp_time) % count)

Roman.

Re: [Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Wanpeng Li]

2015-11-25 23:20 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
> Per Hyper-V specification (and as required by Hyper-V-aware guests),
> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
> of MSRs, and signals expiration by delivering a special format message
> to the configured SynIC message slot and triggering the corresponding
> synthetic interrupt.

Could you post a link for this specification?

Regards,
Wanpeng Li

Re: [Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Andrey Smetanin]

On 11/26/2015 08:28 AM, Wanpeng Li wrote:
> 2015-11-25 23:20 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>> Per Hyper-V specification (and as required by Hyper-V-aware guests),
>> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
>> of MSRs, and signals expiration by delivering a special format message
>> to the configured SynIC message slot and triggering the corresponding
>> synthetic interrupt.
>
> Could you post a link for this specification?

Official link:

http://download.microsoft.com/download/A/B/4/AB43A34E-BDD0-4FA6-BDEF-79EEF16E880B/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.docx

and there is a pdf variant(my own docx -> pdf conversion):

https://www.dropbox.com/s/ehxictr5wgnedq7/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.pdf?dl=0

>
> Regards,
> Wanpeng Li
>

Re: [Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Wanpeng Li]

2015-11-26 16:34 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>
>
> On 11/26/2015 08:28 AM, Wanpeng Li wrote:
>>
>> 2015-11-25 23:20 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>>>
>>> Per Hyper-V specification (and as required by Hyper-V-aware guests),
>>> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
>>> of MSRs, and signals expiration by delivering a special format message
>>> to the configured SynIC message slot and triggering the corresponding
>>> synthetic interrupt.
>>
>>
>> Could you post a link for this specification?
>
>
> Official link:
>
> http://download.microsoft.com/download/A/B/4/AB43A34E-BDD0-4FA6-BDEF-79EEF16E880B/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.docx
>
> and there is a pdf variant(my own docx -> pdf conversion):
>
> https://www.dropbox.com/s/ehxictr5wgnedq7/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.pdf?dl=0

Cool, thanks.

Regards,
Wanpeng Li

Re: [Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Wanpeng Li]

2015-11-26 16:34 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>
>
> On 11/26/2015 08:28 AM, Wanpeng Li wrote:
>>
>> 2015-11-25 23:20 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>>>
>>> Per Hyper-V specification (and as required by Hyper-V-aware guests),
>>> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
>>> of MSRs, and signals expiration by delivering a special format message
>>> to the configured SynIC message slot and triggering the corresponding
>>> synthetic interrupt.
>>
>>
>> Could you post a link for this specification?
>
>
> Official link:
>
> http://download.microsoft.com/download/A/B/4/AB43A34E-BDD0-4FA6-BDEF-79EEF16E880B/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.docx
>
> and there is a pdf variant(my own docx -> pdf conversion):
>
> https://www.dropbox.com/s/ehxictr5wgnedq7/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.pdf?dl=0

Btw, is there performance data for such feature?

Regards,
Wanpeng Li

Re: [Qemu-devel] [PATCH v1 0/7] KVM: Hyper-V SynIC timers

[Denis V. Lunev]

On 12/01/2015 01:12 PM, Wanpeng Li wrote:
> 2015-11-26 16:34 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>>
>> On 11/26/2015 08:28 AM, Wanpeng Li wrote:
>>> 2015-11-25 23:20 GMT+08:00 Andrey Smetanin <asmetanin@virtuozzo.com>:
>>>> Per Hyper-V specification (and as required by Hyper-V-aware guests),
>>>> SynIC provides 4 per-vCPU timers.  Each timer is programmed via a pair
>>>> of MSRs, and signals expiration by delivering a special format message
>>>> to the configured SynIC message slot and triggering the corresponding
>>>> synthetic interrupt.
>>>
>>> Could you post a link for this specification?
>>
>> Official link:
>>
>> http://download.microsoft.com/download/A/B/4/AB43A34E-BDD0-4FA6-BDEF-79EEF16E880B/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.docx
>>
>> and there is a pdf variant(my own docx -> pdf conversion):
>>
>> https://www.dropbox.com/s/ehxictr5wgnedq7/Hypervisor%20Top%20Level%20Functional%20Specification%20v4.0.pdf?dl=0
> Btw, is there performance data for such feature?
>
> Regards,
> Wanpeng Li
not yet.

This is a requirement for any Hyper-V device emulation
to be activated by Windows. We are going to have
basic infrastructure merged and run performance tests
with all that stuff done.

Den