[Qemu-devel] [PATCH v2 0/9] Hyper-V synthetic interrupt controller

[Qemu-devel] [PATCH v2 0/9] Hyper-V synthetic interrupt controller

[Denis V. Lunev]

This patchset implements the KVM part of the synthetic interrupt
controller (SynIC) which is a building block of the Hyper-V
paravirtualized device bus (vmbus).

SynIC is a lapic extension, which is controlled via MSRs and maintains
for each vCPU
 - 16 synthetic interrupt "lines" (SINT's); each can be configured to
   trigger a specific interrupt vector optionally with auto-EOI
   semantics
 - a message page in the guest memory with 16 256-byte per-SINT message
   slots
 - an event flag page in the guest memory with 16 2048-bit per-SINT
   event flag areas

The host triggers a SINT whenever it delivers a new message to the
corresponding slot or flips an event flag bit in the corresponding area.
The guest informs the host that it can try delivering a message by
explicitly asserting EOI in lapic or writing to End-Of-Message (EOM)
MSR.

The userspace (qemu) triggers interrupts and receives EOM notifications
via irqfd with resampler; for that, a GSI is allocated for each
configured SINT, and irq_routing api is extended to support GSI-SINT
mapping.

Besides, a new vcpu exit is introduced to notify the userspace of the
changes in SynIC configuraion triggered by guest writing to the
corresponding MSRs.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>

Changes v2:
* irqchip/eventfd preparation improvements to support
  arch specific routing entries like Hyper-V SynIC ones.
* add Hyper-V SynIC vectors into EOI exit bitmap.
* do not use posted interrupts in case of Hyper-V SynIC
  AutoEOI vectors

[Qemu-devel] [PATCH 1/9] kvm/eventfd: avoid loop inside irqfd_update()

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

The loop(for) inside irqfd_update() is unnecessary
because any other value for irq_entry.type will just trigger
schedule_work(&irqfd->inject).

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 virt/kvm/eventfd.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index b637965..518421e 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -238,20 +238,17 @@ static void irqfd_update(struct kvm *kvm, struct kvm_kernel_irqfd *irqfd)
 {
 	struct kvm_kernel_irq_routing_entry *e;
 	struct kvm_kernel_irq_routing_entry entries[KVM_NR_IRQCHIPS];
-	int i, n_entries;
+	int n_entries;
 
 	n_entries = kvm_irq_map_gsi(kvm, entries, irqfd->gsi);
 
 	write_seqcount_begin(&irqfd->irq_entry_sc);
 
-	irqfd->irq_entry.type = 0;
-
 	e = entries;
-	for (i = 0; i < n_entries; ++i, ++e) {
-		/* Only fast-path MSI. */
-		if (e->type == KVM_IRQ_ROUTING_MSI)
-			irqfd->irq_entry = *e;
-	}
+	if (n_entries == 1)
+		irqfd->irq_entry = *e;
+	else
+		irqfd->irq_entry.type = 0;
 
 	write_seqcount_end(&irqfd->irq_entry_sc);
 }
-- 
2.1.4

[Qemu-devel] [PATCH 2/9] kvm/eventfd: factor out kvm_notify_acked_gsi()

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

Factor out kvm_notify_acked_gsi() helper to iterate over EOI listeners
and notify those matching the given gsi.

It will be reused in the upcoming Hyper-V SynIC implementation.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 include/linux/kvm_host.h |  1 +
 virt/kvm/eventfd.c       | 16 +++++++++++-----
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9596a2f..b66861c 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -829,6 +829,7 @@ int kvm_set_irq_inatomic(struct kvm *kvm, int irq_source_id, u32 irq, int level)
 int kvm_set_msi(struct kvm_kernel_irq_routing_entry *irq_entry, struct kvm *kvm,
 		int irq_source_id, int level, bool line_status);
 bool kvm_irq_has_notifier(struct kvm *kvm, unsigned irqchip, unsigned pin);
+void kvm_notify_acked_gsi(struct kvm *kvm, int gsi);
 void kvm_notify_acked_irq(struct kvm *kvm, unsigned irqchip, unsigned pin);
 void kvm_register_irq_ack_notifier(struct kvm *kvm,
 				   struct kvm_irq_ack_notifier *kian);
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index 518421e..f6b986a 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -451,9 +451,18 @@ bool kvm_irq_has_notifier(struct kvm *kvm, unsigned irqchip, unsigned pin)
 }
 EXPORT_SYMBOL_GPL(kvm_irq_has_notifier);
 
-void kvm_notify_acked_irq(struct kvm *kvm, unsigned irqchip, unsigned pin)
+void kvm_notify_acked_gsi(struct kvm *kvm, int gsi)
 {
 	struct kvm_irq_ack_notifier *kian;
+
+	hlist_for_each_entry_rcu(kian, &kvm->irq_ack_notifier_list,
+				 link)
+		if (kian->gsi == gsi)
+			kian->irq_acked(kian);
+}
+
+void kvm_notify_acked_irq(struct kvm *kvm, unsigned irqchip, unsigned pin)
+{
 	int gsi, idx;
 
 	trace_kvm_ack_irq(irqchip, pin);
@@ -461,10 +470,7 @@ void kvm_notify_acked_irq(struct kvm *kvm, unsigned irqchip, unsigned pin)
 	idx = srcu_read_lock(&kvm->irq_srcu);
 	gsi = kvm_irq_map_chip_pin(kvm, irqchip, pin);
 	if (gsi != -1)
-		hlist_for_each_entry_rcu(kian, &kvm->irq_ack_notifier_list,
-					 link)
-			if (kian->gsi == gsi)
-				kian->irq_acked(kian);
+		kvm_notify_acked_gsi(kvm, gsi);
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
 
-- 
2.1.4

[Qemu-devel] [PATCH 3/9] kvm/eventfd: add arch-specific set_irq

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

Allow for arch-specific interrupt types to be set.  For that, add
kvm_arch_set_irq() which takes interrupt type-specific action if it
recognizes the interrupt type given, and -EWOULDBLOCK otherwise.

The default implementation always returns -EWOULDBLOCK.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 include/linux/kvm_host.h |  4 ++++
 virt/kvm/eventfd.c       | 13 ++++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index b66861c..eba9cae 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -828,6 +828,10 @@ int kvm_set_irq(struct kvm *kvm, int irq_source_id, u32 irq, int level,
 int kvm_set_irq_inatomic(struct kvm *kvm, int irq_source_id, u32 irq, int level);
 int kvm_set_msi(struct kvm_kernel_irq_routing_entry *irq_entry, struct kvm *kvm,
 		int irq_source_id, int level, bool line_status);
+
+int kvm_arch_set_irq(struct kvm_kernel_irq_routing_entry *irq, struct kvm *kvm,
+		     int irq_source_id, int level, bool line_status);
+
 bool kvm_irq_has_notifier(struct kvm *kvm, unsigned irqchip, unsigned pin);
 void kvm_notify_acked_gsi(struct kvm *kvm, int gsi);
 void kvm_notify_acked_irq(struct kvm *kvm, unsigned irqchip, unsigned pin);
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index f6b986a..e29fd26 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -171,6 +171,15 @@ irqfd_deactivate(struct kvm_kernel_irqfd *irqfd)
 	queue_work(irqfd_cleanup_wq, &irqfd->shutdown);
 }
 
+int __attribute__((weak)) kvm_arch_set_irq(
+				struct kvm_kernel_irq_routing_entry *irq,
+				struct kvm *kvm, int irq_source_id,
+				int level,
+				bool line_status)
+{
+	return -EWOULDBLOCK;
+}
+
 /*
  * Called with wqh->lock held and interrupts disabled
  */
@@ -195,7 +204,9 @@ irqfd_wakeup(wait_queue_t *wait, unsigned mode, int sync, void *key)
 		if (irq.type == KVM_IRQ_ROUTING_MSI)
 			kvm_set_msi(&irq, kvm, KVM_USERSPACE_IRQ_SOURCE_ID, 1,
 					false);
-		else
+		else if (kvm_arch_set_irq(&irq, kvm,
+					  KVM_USERSPACE_IRQ_SOURCE_ID, 1,
+					  false) == -EWOULDBLOCK)
 			schedule_work(&irqfd->inject);
 		srcu_read_unlock(&kvm->irq_srcu, idx);
 	}
-- 
2.1.4

[Qemu-devel] [PATCH 4/9] kvm/irqchip: allow only multiple irqchip routes per GSI

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

Any other irq routing types (MSI, S390_ADAPTER, upcoming Hyper-V
SynIC) map one-to-one to GSI.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 virt/kvm/irqchip.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/virt/kvm/irqchip.c b/virt/kvm/irqchip.c
index 716a1c4..f0b08a2 100644
--- a/virt/kvm/irqchip.c
+++ b/virt/kvm/irqchip.c
@@ -144,11 +144,11 @@ static int setup_routing_entry(struct kvm_irq_routing_table *rt,
 
 	/*
 	 * Do not allow GSI to be mapped to the same irqchip more than once.
-	 * Allow only one to one mapping between GSI and MSI.
+	 * Allow only one to one mapping between GSI and non-irqchip routing.
 	 */
 	hlist_for_each_entry(ei, &rt->map[ue->gsi], link)
-		if (ei->type == KVM_IRQ_ROUTING_MSI ||
-		    ue->type == KVM_IRQ_ROUTING_MSI ||
+		if (ei->type != KVM_IRQ_ROUTING_IRQCHIP ||
+		    ue->type != KVM_IRQ_ROUTING_IRQCHIP ||
 		    ue->u.irqchip.irqchip == ei->irqchip.irqchip)
 			return r;
 
-- 
2.1.4

[Qemu-devel] [PATCH 5/9] kvm/irqchip: kvm_arch_irq_routing_update renaming split

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

Actually kvm_arch_irq_routing_update() should be
kvm_arch_post_irq_routing_update() as it's called at the end
of irq routing update.

This renaming frees kvm_arch_irq_routing_update function name.
kvm_arch_irq_routing_update() weak function which will be used
to update mappings for arch-specific irq routing entries
(in particular, the upcoming Hyper-V synthetic interrupts).

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/kvm/irq_comm.c  | 2 +-
 include/linux/kvm_host.h | 5 +++--
 virt/kvm/irqchip.c       | 7 ++++++-
 3 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
index c892289..6f922c2 100644
--- a/arch/x86/kvm/irq_comm.c
+++ b/arch/x86/kvm/irq_comm.c
@@ -364,7 +364,7 @@ int kvm_setup_empty_irq_routing(struct kvm *kvm)
 	return kvm_set_irq_routing(kvm, empty_routing, 0, 0);
 }
 
-void kvm_arch_irq_routing_update(struct kvm *kvm)
+void kvm_arch_post_irq_routing_update(struct kvm *kvm)
 {
 	if (ioapic_in_kernel(kvm) || !irqchip_in_kernel(kvm))
 		return;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index eba9cae..c742e79 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -473,12 +473,12 @@ void vcpu_put(struct kvm_vcpu *vcpu);
 
 #ifdef __KVM_HAVE_IOAPIC
 void kvm_vcpu_request_scan_ioapic(struct kvm *kvm);
-void kvm_arch_irq_routing_update(struct kvm *kvm);
+void kvm_arch_post_irq_routing_update(struct kvm *kvm);
 #else
 static inline void kvm_vcpu_request_scan_ioapic(struct kvm *kvm)
 {
 }
-static inline void kvm_arch_irq_routing_update(struct kvm *kvm)
+static inline void kvm_arch_post_irq_routing_update(struct kvm *kvm)
 {
 }
 #endif
@@ -1080,6 +1080,7 @@ static inline void kvm_irq_routing_update(struct kvm *kvm)
 {
 }
 #endif
+void kvm_arch_irq_routing_update(struct kvm *kvm);
 
 static inline int kvm_ioeventfd(struct kvm *kvm, struct kvm_ioeventfd *args)
 {
diff --git a/virt/kvm/irqchip.c b/virt/kvm/irqchip.c
index f0b08a2..fe84e1a 100644
--- a/virt/kvm/irqchip.c
+++ b/virt/kvm/irqchip.c
@@ -166,6 +166,10 @@ out:
 	return r;
 }
 
+void __attribute__((weak)) kvm_arch_irq_routing_update(struct kvm *kvm)
+{
+}
+
 int kvm_set_irq_routing(struct kvm *kvm,
 			const struct kvm_irq_routing_entry *ue,
 			unsigned nr,
@@ -219,9 +223,10 @@ int kvm_set_irq_routing(struct kvm *kvm,
 	old = kvm->irq_routing;
 	rcu_assign_pointer(kvm->irq_routing, new);
 	kvm_irq_routing_update(kvm);
+	kvm_arch_irq_routing_update(kvm);
 	mutex_unlock(&kvm->irq_lock);
 
-	kvm_arch_irq_routing_update(kvm);
+	kvm_arch_post_irq_routing_update(kvm);
 
 	synchronize_srcu_expedited(&kvm->irq_srcu);
 
-- 
2.1.4

[Qemu-devel] [PATCH 6/9] drivers/hv: share Hyper-V SynIC constants with userspace

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

Moved Hyper-V synic contants from guest Hyper-V drivers private
header into x86 arch uapi Hyper-V header.

Added Hyper-V synic msr's flags into x86 arch uapi Hyper-V header.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/include/uapi/asm/hyperv.h | 12 ++++++++++++
 drivers/hv/hyperv_vmbus.h          |  5 -----
 include/linux/hyperv.h             |  1 +
 3 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 2677a0a..040d408 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -257,4 +257,16 @@ typedef struct _HV_REFERENCE_TSC_PAGE {
 	__s64 tsc_offset;
 } HV_REFERENCE_TSC_PAGE, *PHV_REFERENCE_TSC_PAGE;
 
+/* Define the number of synthetic interrupt sources. */
+#define HV_SYNIC_SINT_COUNT		(16)
+/* Define the expected SynIC version. */
+#define HV_SYNIC_VERSION_1		(0x1)
+
+#define HV_SYNIC_CONTROL_ENABLE		(1ULL << 0)
+#define HV_SYNIC_SIMP_ENABLE		(1ULL << 0)
+#define HV_SYNIC_SIEFP_ENABLE		(1ULL << 0)
+#define HV_SYNIC_SINT_MASKED		(1ULL << 16)
+#define HV_SYNIC_SINT_AUTO_EOI		(1ULL << 17)
+#define HV_SYNIC_SINT_VECTOR_MASK	(0xFF)
+
 #endif
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 3d70e36..3782636 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -63,9 +63,6 @@ enum hv_cpuid_function {
 /* Define version of the synthetic interrupt controller. */
 #define HV_SYNIC_VERSION		(1)
 
-/* Define the expected SynIC version. */
-#define HV_SYNIC_VERSION_1		(0x1)
-
 /* Define synthetic interrupt controller message constants. */
 #define HV_MESSAGE_SIZE			(256)
 #define HV_MESSAGE_PAYLOAD_BYTE_COUNT	(240)
@@ -105,8 +102,6 @@ enum hv_message_type {
 	HVMSG_X64_LEGACY_FP_ERROR		= 0x80010005
 };
 
-/* Define the number of synthetic interrupt sources. */
-#define HV_SYNIC_SINT_COUNT		(16)
 #define HV_SYNIC_STIMER_COUNT		(4)
 
 /* Define invalid partition identifier. */
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 54733d5..8fdc17b 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -26,6 +26,7 @@
 #define _HYPERV_H
 
 #include <uapi/linux/hyperv.h>
+#include <uapi/asm/hyperv.h>
 
 #include <linux/types.h>
 #include <linux/scatterlist.h>
-- 
2.1.4

[Qemu-devel] [PATCH 7/9] kvm/x86: split ioapic-handled and EOI exit bitmaps

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

The function to determine if the vector is handled by ioapic used to
rely on the fact that only ioapic-handled vectors were set up to
cause vmexits when virtual apic was in use.

We're going to break this assumption when introducing Hyper-V
synthetic interrupts: they may need to cause vmexits too.

To achieve that, introduce a new bitmap dedicated specifically for
ioapic-handled vectors, and populate EOI exit bitmap from it for now.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/include/asm/kvm_host.h |  4 ++--
 arch/x86/kvm/ioapic.c           |  4 ++--
 arch/x86/kvm/ioapic.h           |  7 ++++---
 arch/x86/kvm/irq_comm.c         |  6 +++---
 arch/x86/kvm/lapic.c            |  2 +-
 arch/x86/kvm/svm.c              |  2 +-
 arch/x86/kvm/vmx.c              |  3 +--
 arch/x86/kvm/x86.c              | 12 +++++++-----
 8 files changed, 21 insertions(+), 19 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 53deb27..07f7cd7 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -400,7 +400,7 @@ struct kvm_vcpu_arch {
 	u64 efer;
 	u64 apic_base;
 	struct kvm_lapic *apic;    /* kernel irqchip context */
-	u64 eoi_exit_bitmap[4];
+	DECLARE_BITMAP(ioapic_handled_vectors, 256);
 	unsigned long apic_attention;
 	int32_t apic_arb_prio;
 	int mp_state;
@@ -833,7 +833,7 @@ struct kvm_x86_ops {
 	int (*cpu_uses_apicv)(struct kvm_vcpu *vcpu);
 	void (*hwapic_irr_update)(struct kvm_vcpu *vcpu, int max_irr);
 	void (*hwapic_isr_update)(struct kvm *kvm, int isr);
-	void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu);
+	void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap);
 	void (*set_virtual_x2apic_mode)(struct kvm_vcpu *vcpu, bool set);
 	void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu, hpa_t hpa);
 	void (*deliver_posted_interrupt)(struct kvm_vcpu *vcpu, int vector);
diff --git a/arch/x86/kvm/ioapic.c b/arch/x86/kvm/ioapic.c
index 2dcda0f..3cf7a9c 100644
--- a/arch/x86/kvm/ioapic.c
+++ b/arch/x86/kvm/ioapic.c
@@ -233,7 +233,7 @@ static void kvm_ioapic_inject_all(struct kvm_ioapic *ioapic, unsigned long irr)
 }
 
 
-void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
+void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu, ulong *ioapic_handled_vectors)
 {
 	struct kvm_ioapic *ioapic = vcpu->kvm->arch.vioapic;
 	union kvm_ioapic_redirect_entry *e;
@@ -248,7 +248,7 @@ void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
 			if (kvm_apic_match_dest(vcpu, NULL, 0,
 				e->fields.dest_id, e->fields.dest_mode))
 				__set_bit(e->fields.vector,
-					(unsigned long *)eoi_exit_bitmap);
+					  ioapic_handled_vectors);
 		}
 	}
 	spin_unlock(&ioapic->lock);
diff --git a/arch/x86/kvm/ioapic.h b/arch/x86/kvm/ioapic.h
index 084617d..2d16dc2 100644
--- a/arch/x86/kvm/ioapic.h
+++ b/arch/x86/kvm/ioapic.h
@@ -121,7 +121,8 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
 		struct kvm_lapic_irq *irq, unsigned long *dest_map);
 int kvm_get_ioapic(struct kvm *kvm, struct kvm_ioapic_state *state);
 int kvm_set_ioapic(struct kvm *kvm, struct kvm_ioapic_state *state);
-void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap);
-void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap);
-
+void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu,
+			   ulong *ioapic_handled_vectors);
+void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
+			    ulong *ioapic_handled_vectors);
 #endif
diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
index 6f922c2..fe91f72 100644
--- a/arch/x86/kvm/irq_comm.c
+++ b/arch/x86/kvm/irq_comm.c
@@ -371,7 +371,8 @@ void kvm_arch_post_irq_routing_update(struct kvm *kvm)
 	kvm_make_scan_ioapic_request(kvm);
 }
 
-void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
+void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
+			    ulong *ioapic_handled_vectors)
 {
 	struct kvm *kvm = vcpu->kvm;
 	struct kvm_kernel_irq_routing_entry *entry;
@@ -398,8 +399,7 @@ void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
 						dest_mode)) {
 				u32 vector = entry->msi.data & 0xff;
 
-				__set_bit(vector,
-					  (unsigned long *) eoi_exit_bitmap);
+				__set_bit(vector, ioapic_handled_vectors);
 			}
 		}
 	}
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 944b38a..dc03a01 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -930,7 +930,7 @@ int kvm_apic_compare_prio(struct kvm_vcpu *vcpu1, struct kvm_vcpu *vcpu2)
 
 static bool kvm_ioapic_handles_vector(struct kvm_lapic *apic, int vector)
 {
-	return test_bit(vector, (ulong *)apic->vcpu->arch.eoi_exit_bitmap);
+	return test_bit(vector, apic->vcpu->arch.ioapic_handled_vectors);
 }
 
 static void kvm_ioapic_send_eoi(struct kvm_lapic *apic, int vector)
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 54a8618..03ccb78 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -3651,7 +3651,7 @@ static int svm_cpu_uses_apicv(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-static void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu)
+static void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
 {
 	return;
 }
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 8eeba6a..fccf39c 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -8179,9 +8179,8 @@ static void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr)
 	}
 }
 
-static void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu)
+static void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
 {
-	u64 *eoi_exit_bitmap = vcpu->arch.eoi_exit_bitmap;
 	if (!vmx_cpu_uses_apicv(vcpu))
 		return;
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bcbc9234..668a1c4 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6198,13 +6198,15 @@ static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
 	if (!kvm_apic_hw_enabled(vcpu->arch.apic))
 		return;
 
-	memset(vcpu->arch.eoi_exit_bitmap, 0, 256 / 8);
+	bitmap_zero(vcpu->arch.ioapic_handled_vectors, 256);
 
 	if (irqchip_split(vcpu->kvm))
-		kvm_scan_ioapic_routes(vcpu, vcpu->arch.eoi_exit_bitmap);
+		kvm_scan_ioapic_routes(vcpu, vcpu->arch.ioapic_handled_vectors);
 	else
-		kvm_ioapic_scan_entry(vcpu, vcpu->arch.eoi_exit_bitmap);
-	kvm_x86_ops->load_eoi_exitmap(vcpu);
+		kvm_ioapic_scan_entry(vcpu, vcpu->arch.ioapic_handled_vectors);
+
+	kvm_x86_ops->load_eoi_exitmap(vcpu,
+				      (u64 *)vcpu->arch.ioapic_handled_vectors);
 }
 
 static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
@@ -6310,7 +6312,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 		if (kvm_check_request(KVM_REQ_IOAPIC_EOI_EXIT, vcpu)) {
 			BUG_ON(vcpu->arch.pending_ioapic_eoi > 255);
 			if (test_bit(vcpu->arch.pending_ioapic_eoi,
-				     (void *) vcpu->arch.eoi_exit_bitmap)) {
+				     vcpu->arch.ioapic_handled_vectors)) {
 				vcpu->run->exit_reason = KVM_EXIT_IOAPIC_EOI;
 				vcpu->run->eoi.vector =
 						vcpu->arch.pending_ioapic_eoi;
-- 
2.1.4

[Qemu-devel] [PATCH 8/9] kvm/x86: Hyper-V synthetic interrupt controller

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

SynIC (synthetic interrupt controller) is a lapic extension,
which is controlled via MSRs and maintains for each vCPU
 - 16 synthetic interrupt "lines" (SINT's); each can be configured to
   trigger a specific interrupt vector optionally with auto-EOI
   semantics
 - a message page in the guest memory with 16 256-byte per-SINT message
   slots
 - an event flag page in the guest memory with 16 2048-bit per-SINT
   event flag areas

The host triggers a SINT whenever it delivers a new message to the
corresponding slot or flips an event flag bit in the corresponding area.
The guest informs the host that it can try delivering a message by
explicitly asserting EOI in lapic or writing to End-Of-Message (EOM)
MSR.

The userspace (qemu) triggers interrupts and receives EOM notifications
via irqfd with resampler; for that, a GSI is allocated for each
configured SINT, and irq_routing api is extended to support GSI-SINT
mapping.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>

Changes v2:
* do not use posted interrupts for Hyper-V SynIC AutoEOI vectors
* add Hyper-V SynIC vectors into EOI exit bitmap
* Hyper-V SyniIC SINT msr write logic simplified
---
 arch/x86/include/asm/kvm_host.h |  14 ++
 arch/x86/kvm/hyperv.c           | 297 ++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/hyperv.h           |  21 +++
 arch/x86/kvm/irq_comm.c         |  34 +++++
 arch/x86/kvm/lapic.c            |  18 ++-
 arch/x86/kvm/lapic.h            |   5 +
 arch/x86/kvm/x86.c              |  12 +-
 include/linux/kvm_host.h        |   6 +
 include/uapi/linux/kvm.h        |   8 ++
 9 files changed, 407 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 07f7cd7..dfdaf0f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -25,6 +25,7 @@
 #include <linux/pvclock_gtod.h>
 #include <linux/clocksource.h>
 #include <linux/irqbypass.h>
+#include <linux/hyperv.h>
 
 #include <asm/pvclock-abi.h>
 #include <asm/desc.h>
@@ -374,10 +375,23 @@ struct kvm_mtrr {
 	struct list_head head;
 };
 
+/* Hyper-V synthetic interrupt controller (SynIC)*/
+struct kvm_vcpu_hv_synic {
+	u64 version;
+	u64 control;
+	u64 msg_page;
+	u64 evt_page;
+	atomic64_t sint[HV_SYNIC_SINT_COUNT];
+	atomic_t sint_to_gsi[HV_SYNIC_SINT_COUNT];
+	DECLARE_BITMAP(auto_eoi_bitmap, 256);
+	DECLARE_BITMAP(vec_bitmap, 256);
+};
+
 /* Hyper-V per vcpu emulation context */
 struct kvm_vcpu_hv {
 	u64 hv_vapic;
 	s64 runtime_offset;
+	struct kvm_vcpu_hv_synic synic;
 };
 
 struct kvm_vcpu_arch {
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 62cf8c9..8ff71f3 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -23,13 +23,296 @@
 
 #include "x86.h"
 #include "lapic.h"
+#include "ioapic.h"
 #include "hyperv.h"
 
 #include <linux/kvm_host.h>
+#include <asm/apicdef.h>
 #include <trace/events/kvm.h>
 
 #include "trace.h"
 
+static inline u64 synic_read_sint(struct kvm_vcpu_hv_synic *synic, int sint)
+{
+	return atomic64_read(&synic->sint[sint]);
+}
+
+static inline int synic_get_sint_vector(u64 sint_value)
+{
+	if (sint_value & HV_SYNIC_SINT_MASKED)
+		return -1;
+	return sint_value & HV_SYNIC_SINT_VECTOR_MASK;
+}
+
+static bool synic_has_vector_connected(struct kvm_vcpu_hv_synic *synic,
+				      int vector)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
+		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
+			return true;
+	}
+	return false;
+}
+
+static bool synic_has_vector_auto_eoi(struct kvm_vcpu_hv_synic *synic,
+				     int vector)
+{
+	int i;
+	u64 sint_value;
+
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
+		sint_value = synic_read_sint(synic, i);
+		if (synic_get_sint_vector(sint_value) == vector &&
+		    sint_value & HV_SYNIC_SINT_AUTO_EOI)
+			return true;
+	}
+	return false;
+}
+
+static int synic_set_sint(struct kvm_vcpu_hv_synic *synic, int sint, u64 data)
+{
+	int vector;
+
+	vector = data & HV_SYNIC_SINT_VECTOR_MASK;
+	if (vector < 16)
+		return 1;
+	/*
+	 * Guest may configure multiple SINTs to use the same vector, so
+	 * we maintain a bitmap of vectors handled by synic, and a
+	 * bitmap of vectors with auto-eoi behavior.  The bitmaps are
+	 * updated here, and atomically queried on fast paths.
+	 */
+
+	atomic64_set(&synic->sint[sint], data);
+
+	if (synic_has_vector_connected(synic, vector))
+		__set_bit(vector, synic->vec_bitmap);
+	else
+		__clear_bit(vector, synic->vec_bitmap);
+
+	if (synic_has_vector_auto_eoi(synic, vector))
+		__set_bit(vector, synic->auto_eoi_bitmap);
+	else
+		__clear_bit(vector, synic->auto_eoi_bitmap);
+
+	/* Load SynIC vectors into EOI exit bitmap */
+	kvm_make_request(KVM_REQ_SCAN_IOAPIC, synic_to_vcpu(synic));
+	return 0;
+}
+
+static struct kvm_vcpu_hv_synic *synic_get(struct kvm *kvm, u32 vcpu_id)
+{
+	struct kvm_vcpu *vcpu;
+
+	if (vcpu_id >= atomic_read(&kvm->online_vcpus))
+		return NULL;
+	vcpu = kvm_get_vcpu(kvm, vcpu_id);
+	if (!vcpu)
+		return NULL;
+
+	return vcpu_to_synic(vcpu);
+}
+
+static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
+{
+	struct kvm *kvm = vcpu->kvm;
+	int gsi, idx;
+
+	vcpu_debug(vcpu, "Hyper-V SynIC acked sint %d\n", sint);
+
+	idx = srcu_read_lock(&kvm->irq_srcu);
+	gsi = atomic_read(&vcpu_to_synic(vcpu)->sint_to_gsi[sint]);
+	if (gsi != -1)
+		kvm_notify_acked_gsi(kvm, gsi);
+	srcu_read_unlock(&kvm->irq_srcu, idx);
+}
+
+static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
+			 u32 msr, u64 data, bool host)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	int ret;
+
+	vcpu_debug(vcpu, "Hyper-V SynIC set msr 0x%x 0x%llx host %d\n",
+		   msr, data, host);
+	ret = 0;
+	switch (msr) {
+	case HV_X64_MSR_SCONTROL:
+		synic->control = data;
+		break;
+	case HV_X64_MSR_SVERSION:
+		if (!host) {
+			ret = 1;
+			break;
+		}
+		synic->version = data;
+		break;
+	case HV_X64_MSR_SIEFP:
+		if (data & HV_SYNIC_SIEFP_ENABLE)
+			if (kvm_clear_guest(vcpu->kvm,
+					    data & PAGE_MASK, PAGE_SIZE)) {
+				ret = 1;
+				break;
+			}
+		synic->evt_page = data;
+		break;
+	case HV_X64_MSR_SIMP:
+		if (data & HV_SYNIC_SIMP_ENABLE)
+			if (kvm_clear_guest(vcpu->kvm,
+					    data & PAGE_MASK, PAGE_SIZE)) {
+				ret = 1;
+				break;
+			}
+		synic->msg_page = data;
+		break;
+	case HV_X64_MSR_EOM: {
+		int i;
+
+		for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
+			kvm_hv_notify_acked_sint(vcpu, i);
+		break;
+	}
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		ret = synic_set_sint(synic, msr - HV_X64_MSR_SINT0, data);
+		break;
+	default:
+		ret = 1;
+		break;
+	}
+	return ret;
+}
+
+static int synic_get_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 *pdata)
+{
+	int ret;
+
+	ret = 0;
+	switch (msr) {
+	case HV_X64_MSR_SCONTROL:
+		*pdata = synic->control;
+		break;
+	case HV_X64_MSR_SVERSION:
+		*pdata = synic->version;
+		break;
+	case HV_X64_MSR_SIEFP:
+		*pdata = synic->evt_page;
+		break;
+	case HV_X64_MSR_SIMP:
+		*pdata = synic->msg_page;
+		break;
+	case HV_X64_MSR_EOM:
+		*pdata = 0;
+		break;
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		*pdata = atomic64_read(&synic->sint[msr - HV_X64_MSR_SINT0]);
+		break;
+	default:
+		ret = 1;
+		break;
+	}
+	return ret;
+}
+
+int synic_set_irq(struct kvm_vcpu_hv_synic *synic, u32 sint)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	struct kvm_lapic_irq irq;
+	int ret, vector;
+
+	if (sint >= ARRAY_SIZE(synic->sint))
+		return -EINVAL;
+
+	vector = synic_get_sint_vector(synic_read_sint(synic, sint));
+	if (vector < 0)
+		return -ENOENT;
+
+	memset(&irq, 0, sizeof(irq));
+	irq.dest_id = kvm_apic_id(vcpu->arch.apic);
+	irq.dest_mode = APIC_DEST_PHYSICAL;
+	irq.delivery_mode = APIC_DM_FIXED;
+	irq.vector = vector;
+	irq.level = 1;
+
+	ret = kvm_irq_delivery_to_apic(vcpu->kvm, NULL, &irq, NULL);
+	vcpu_debug(vcpu, "Hyper-V SynIC set irq ret %d\n", ret);
+	return ret;
+}
+
+int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint)
+{
+	struct kvm_vcpu_hv_synic *synic;
+
+	synic = synic_get(kvm, vcpu_id);
+	if (!synic)
+		return -EINVAL;
+
+	return synic_set_irq(synic, sint);
+}
+
+void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector)
+{
+	struct kvm_vcpu_hv_synic *synic = vcpu_to_synic(vcpu);
+	int i;
+
+	vcpu_debug(vcpu, "Hyper-V SynIC send eoi vec %d\n", vector);
+
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
+		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
+			kvm_hv_notify_acked_sint(vcpu, i);
+}
+
+static int kvm_hv_set_sint_gsi(struct kvm *kvm, u32 vcpu_id, u32 sint, int gsi)
+{
+	struct kvm_vcpu_hv_synic *synic;
+
+	synic = synic_get(kvm, vcpu_id);
+	if (!synic)
+		return -EINVAL;
+
+	if (sint >= ARRAY_SIZE(synic->sint_to_gsi))
+		return -EINVAL;
+
+	atomic_set(&synic->sint_to_gsi[sint], gsi);
+	return 0;
+}
+
+void kvm_hv_irq_routing_update(struct kvm *kvm)
+{
+	struct kvm_irq_routing_table *irq_rt;
+	struct kvm_kernel_irq_routing_entry *e;
+	u32 gsi;
+
+	irq_rt = srcu_dereference_check(kvm->irq_routing, &kvm->irq_srcu,
+					lockdep_is_held(&kvm->irq_lock));
+
+	for (gsi = 0; gsi < irq_rt->nr_rt_entries; gsi++) {
+		hlist_for_each_entry(e, &irq_rt->map[gsi], link) {
+			if (e->type == KVM_IRQ_ROUTING_HV_SINT)
+				kvm_hv_set_sint_gsi(kvm, e->hv_sint.vcpu,
+						    e->hv_sint.sint, gsi);
+		}
+	}
+}
+
+static void synic_init(struct kvm_vcpu_hv_synic *synic)
+{
+	int i;
+
+	memset(synic, 0, sizeof(*synic));
+	synic->version = HV_SYNIC_VERSION_1;
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
+		atomic64_set(&synic->sint[i], HV_SYNIC_SINT_MASKED);
+		atomic_set(&synic->sint_to_gsi[i], -1);
+	}
+}
+
+void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu)
+{
+	synic_init(vcpu_to_synic(vcpu));
+}
+
 static bool kvm_hv_msr_partition_wide(u32 msr)
 {
 	bool r = false;
@@ -226,6 +509,13 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)
 			return 1;
 		hv->runtime_offset = data - current_task_runtime_100ns();
 		break;
+	case HV_X64_MSR_SCONTROL:
+	case HV_X64_MSR_SVERSION:
+	case HV_X64_MSR_SIEFP:
+	case HV_X64_MSR_SIMP:
+	case HV_X64_MSR_EOM:
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		return synic_set_msr(vcpu_to_synic(vcpu), msr, data, host);
 	default:
 		vcpu_unimpl(vcpu, "Hyper-V uhandled wrmsr: 0x%x data 0x%llx\n",
 			    msr, data);
@@ -304,6 +594,13 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 	case HV_X64_MSR_VP_RUNTIME:
 		data = current_task_runtime_100ns() + hv->runtime_offset;
 		break;
+	case HV_X64_MSR_SCONTROL:
+	case HV_X64_MSR_SVERSION:
+	case HV_X64_MSR_SIEFP:
+	case HV_X64_MSR_SIMP:
+	case HV_X64_MSR_EOM:
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		return synic_get_msr(vcpu_to_synic(vcpu), msr, pdata);
 	default:
 		vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
 		return 1;
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index c7bce55..8668612 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -29,4 +29,25 @@ int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
 bool kvm_hv_hypercall_enabled(struct kvm *kvm);
 int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
 
+int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
+void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
+
+static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
+{
+	return &vcpu->arch.hyperv.synic;
+}
+
+static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
+{
+	struct kvm_vcpu_hv *hv;
+	struct kvm_vcpu_arch *arch;
+
+	hv = container_of(synic, struct kvm_vcpu_hv, synic);
+	arch = container_of(hv, struct kvm_vcpu_arch, hyperv);
+	return container_of(arch, struct kvm_vcpu, arch);
+}
+void kvm_hv_irq_routing_update(struct kvm *kvm);
+
+void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
+
 #endif
diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
index fe91f72..5e195b9 100644
--- a/arch/x86/kvm/irq_comm.c
+++ b/arch/x86/kvm/irq_comm.c
@@ -33,6 +33,8 @@
 
 #include "lapic.h"
 
+#include "hyperv.h"
+
 static int kvm_set_pic_irq(struct kvm_kernel_irq_routing_entry *e,
 			   struct kvm *kvm, int irq_source_id, int level,
 			   bool line_status)
@@ -251,6 +253,16 @@ void kvm_fire_mask_notifiers(struct kvm *kvm, unsigned irqchip, unsigned pin,
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
 
+static int kvm_hv_set_sint(struct kvm_kernel_irq_routing_entry *e,
+		    struct kvm *kvm, int irq_source_id, int level,
+		    bool line_status)
+{
+	if (!level)
+		return -1;
+
+	return kvm_hv_synic_set_irq(kvm, e->hv_sint.vcpu, e->hv_sint.sint);
+}
+
 int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
 			  const struct kvm_irq_routing_entry *ue)
 {
@@ -289,6 +301,11 @@ int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
 		e->msi.address_hi = ue->u.msi.address_hi;
 		e->msi.data = ue->u.msi.data;
 		break;
+	case KVM_IRQ_ROUTING_HV_SINT:
+		e->set = kvm_hv_set_sint;
+		e->hv_sint.vcpu = ue->u.hv_sint.vcpu;
+		e->hv_sint.sint = ue->u.hv_sint.sint;
+		break;
 	default:
 		goto out;
 	}
@@ -405,3 +422,20 @@ void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
 	}
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
+
+int kvm_arch_set_irq(struct kvm_kernel_irq_routing_entry *irq, struct kvm *kvm,
+		     int irq_source_id, int level, bool line_status)
+{
+	switch (irq->type) {
+	case KVM_IRQ_ROUTING_HV_SINT:
+		return kvm_hv_set_sint(irq, kvm, irq_source_id, level,
+				       line_status);
+	default:
+		return -EWOULDBLOCK;
+	}
+}
+
+void kvm_arch_irq_routing_update(struct kvm *kvm)
+{
+	kvm_hv_irq_routing_update(kvm);
+}
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index dc03a01..3132478 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -41,6 +41,7 @@
 #include "trace.h"
 #include "x86.h"
 #include "cpuid.h"
+#include "hyperv.h"
 
 #ifndef CONFIG_X86_64
 #define mod_64(x, y) ((x) - (y) * div64_u64(x, y))
@@ -128,11 +129,6 @@ static inline int apic_enabled(struct kvm_lapic *apic)
 	(LVT_MASK | APIC_MODE_MASK | APIC_INPUT_POLARITY | \
 	 APIC_LVT_REMOTE_IRR | APIC_LVT_LEVEL_TRIGGER)
 
-static inline int kvm_apic_id(struct kvm_lapic *apic)
-{
-	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
-}
-
 /* The logical map is definitely wrong if we have multiple
  * modes at the same time.  (Physical map is always right.)
  */
@@ -850,7 +846,8 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
 				apic_clear_vector(vector, apic->regs + APIC_TMR);
 		}
 
-		if (kvm_x86_ops->deliver_posted_interrupt)
+		if (kvm_x86_ops->deliver_posted_interrupt &&
+		    !test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap))
 			kvm_x86_ops->deliver_posted_interrupt(vcpu, vector);
 		else {
 			apic_set_irr(vector, apic);
@@ -972,6 +969,9 @@ static int apic_set_eoi(struct kvm_lapic *apic)
 	apic_clear_isr(vector, apic);
 	apic_update_ppr(apic);
 
+	if (test_bit(vector, vcpu_to_synic(apic->vcpu)->vec_bitmap))
+		kvm_hv_synic_send_eoi(apic->vcpu, vector);
+
 	kvm_ioapic_send_eoi(apic, vector);
 	kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
 	return vector;
@@ -1881,6 +1881,12 @@ int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu)
 	apic_set_isr(vector, apic);
 	apic_update_ppr(apic);
 	apic_clear_irr(vector, apic);
+
+	if (test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap)) {
+		apic_clear_isr(vector, apic);
+		apic_update_ppr(apic);
+	}
+
 	return vector;
 }
 
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index fde8e35d..6c64090 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -164,6 +164,11 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)
 	return kvm_vcpu_has_lapic(vcpu) && test_bit(KVM_APIC_INIT, &vcpu->arch.apic->pending_events);
 }
 
+static inline int kvm_apic_id(struct kvm_lapic *apic)
+{
+	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
+}
+
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
 
 void wait_lapic_expire(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 668a1c4..807d124 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -959,6 +959,7 @@ static u32 emulated_msrs[] = {
 	HV_X64_MSR_RESET,
 	HV_X64_MSR_VP_INDEX,
 	HV_X64_MSR_VP_RUNTIME,
+	HV_X64_MSR_SCONTROL,
 	HV_X64_MSR_APIC_ASSIST_PAGE, MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
 	MSR_KVM_PV_EOI_EN,
 
@@ -2441,6 +2442,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_HYPERV:
 	case KVM_CAP_HYPERV_VAPIC:
 	case KVM_CAP_HYPERV_SPIN:
+	case KVM_CAP_HYPERV_SYNIC:
 	case KVM_CAP_PCI_SEGMENT:
 	case KVM_CAP_DEBUGREGS:
 	case KVM_CAP_X86_ROBUST_SINGLESTEP:
@@ -6195,6 +6197,8 @@ static void process_smi(struct kvm_vcpu *vcpu)
 
 static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
 {
+	u64 eoi_exit_bitmap[4];
+
 	if (!kvm_apic_hw_enabled(vcpu->arch.apic))
 		return;
 
@@ -6205,8 +6209,10 @@ static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
 	else
 		kvm_ioapic_scan_entry(vcpu, vcpu->arch.ioapic_handled_vectors);
 
-	kvm_x86_ops->load_eoi_exitmap(vcpu,
-				      (u64 *)vcpu->arch.ioapic_handled_vectors);
+	bitmap_or((ulong *)eoi_exit_bitmap, vcpu->arch.ioapic_handled_vectors,
+		  vcpu_to_synic(vcpu)->vec_bitmap, 256);
+
+	kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap);
 }
 
 static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
@@ -7461,6 +7467,8 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
 
 	vcpu->arch.pending_external_vector = -1;
 
+	kvm_hv_vcpu_init(vcpu);
+
 	return 0;
 
 fail_free_mce_banks:
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index c742e79..43b0141 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -318,6 +318,11 @@ struct kvm_s390_adapter_int {
 	u32 adapter_id;
 };
 
+struct kvm_hv_sint {
+	u32 vcpu;
+	u32 sint;
+};
+
 struct kvm_kernel_irq_routing_entry {
 	u32 gsi;
 	u32 type;
@@ -331,6 +336,7 @@ struct kvm_kernel_irq_routing_entry {
 		} irqchip;
 		struct msi_msg msi;
 		struct kvm_s390_adapter_int adapter;
+		struct kvm_hv_sint hv_sint;
 	};
 	struct hlist_node link;
 };
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 03f3618..27ce460 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -831,6 +831,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_GUEST_DEBUG_HW_WPS 120
 #define KVM_CAP_SPLIT_IRQCHIP 121
 #define KVM_CAP_IOEVENTFD_ANY_LENGTH 122
+#define KVM_CAP_HYPERV_SYNIC 123
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -854,10 +855,16 @@ struct kvm_irq_routing_s390_adapter {
 	__u32 adapter_id;
 };
 
+struct kvm_irq_routing_hv_sint {
+	__u32 vcpu;
+	__u32 sint;
+};
+
 /* gsi routing entry types */
 #define KVM_IRQ_ROUTING_IRQCHIP 1
 #define KVM_IRQ_ROUTING_MSI 2
 #define KVM_IRQ_ROUTING_S390_ADAPTER 3
+#define KVM_IRQ_ROUTING_HV_SINT 4
 
 struct kvm_irq_routing_entry {
 	__u32 gsi;
@@ -868,6 +875,7 @@ struct kvm_irq_routing_entry {
 		struct kvm_irq_routing_irqchip irqchip;
 		struct kvm_irq_routing_msi msi;
 		struct kvm_irq_routing_s390_adapter adapter;
+		struct kvm_irq_routing_hv_sint hv_sint;
 		__u32 pad[8];
 	} u;
 };
-- 
2.1.4

[Qemu-devel] [PATCH v3 8/9] kvm/x86: Hyper-V synthetic interrupt controller

[Andrey Smetanin]

SynIC (synthetic interrupt controller) is a lapic extension,
which is controlled via MSRs and maintains for each vCPU
 - 16 synthetic interrupt "lines" (SINT's); each can be configured to
   trigger a specific interrupt vector optionally with auto-EOI
   semantics
 - a message page in the guest memory with 16 256-byte per-SINT message
   slots
 - an event flag page in the guest memory with 16 2048-bit per-SINT
   event flag areas

The host triggers a SINT whenever it delivers a new message to the
corresponding slot or flips an event flag bit in the corresponding area.
The guest informs the host that it can try delivering a message by
explicitly asserting EOI in lapic or writing to End-Of-Message (EOM)
MSR.

The userspace (qemu) triggers interrupts and receives EOM notifications
via irqfd with resampler; for that, a GSI is allocated for each
configured SINT, and irq_routing api is extended to support GSI-SINT
mapping.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: Roman Kagan <rkagan@virtuozzo.com>

Changes v3:
* added KVM_CAP_HYPERV_SYNIC and KVM_IRQ_ROUTING_HV_SINT notes into
docs

Changes v2:
* do not use posted interrupts for Hyper-V SynIC AutoEOI vectors
* add Hyper-V SynIC vectors into EOI exit bitmap
* Hyper-V SyniIC SINT msr write logic simplified
---
 Documentation/virtual/kvm/api.txt |  14 ++
 arch/x86/include/asm/kvm_host.h   |  14 ++
 arch/x86/kvm/hyperv.c             | 297 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/hyperv.h             |  21 +++
 arch/x86/kvm/irq_comm.c           |  34 +++++
 arch/x86/kvm/lapic.c              |  18 ++-
 arch/x86/kvm/lapic.h              |   5 +
 arch/x86/kvm/x86.c                |  12 +-
 include/linux/kvm_host.h          |   6 +
 include/uapi/linux/kvm.h          |   8 +
 10 files changed, 421 insertions(+), 8 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 092ee9f..8710418 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -1451,6 +1451,7 @@ struct kvm_irq_routing_entry {
 		struct kvm_irq_routing_irqchip irqchip;
 		struct kvm_irq_routing_msi msi;
 		struct kvm_irq_routing_s390_adapter adapter;
+		struct kvm_irq_routing_hv_sint hv_sint;
 		__u32 pad[8];
 	} u;
 };
@@ -1459,6 +1460,7 @@ struct kvm_irq_routing_entry {
 #define KVM_IRQ_ROUTING_IRQCHIP 1
 #define KVM_IRQ_ROUTING_MSI 2
 #define KVM_IRQ_ROUTING_S390_ADAPTER 3
+#define KVM_IRQ_ROUTING_HV_SINT 4
 
 No flags are specified so far, the corresponding field must be set to zero.
 
@@ -1482,6 +1484,10 @@ struct kvm_irq_routing_s390_adapter {
 	__u32 adapter_id;
 };
 
+struct kvm_irq_routing_hv_sint {
+	__u32 vcpu;
+	__u32 sint;
+};
 
 4.53 KVM_ASSIGN_SET_MSIX_NR (deprecated)
 
@@ -3685,3 +3691,11 @@ available, means that that the kernel has an implementation of the
 H_RANDOM hypercall backed by a hardware random-number generator.
 If present, the kernel H_RANDOM handler can be enabled for guest use
 with the KVM_CAP_PPC_ENABLE_HCALL capability.
+
+8.2 KVM_CAP_HYPERV_SYNIC
+
+Architectures: x86
+This capability, if KVM_CHECK_EXTENSION indicates that it is
+available, means that that the kernel has an implementation of the
+Hyper-V Synthetic interrupt controller(SynIC). SynIC is used to
+support Windows Hyper-V based guest paravirt drivers(VMBus).
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3c6327d..8434f88 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -25,6 +25,7 @@
 #include <linux/pvclock_gtod.h>
 #include <linux/clocksource.h>
 #include <linux/irqbypass.h>
+#include <linux/hyperv.h>
 
 #include <asm/pvclock-abi.h>
 #include <asm/desc.h>
@@ -374,10 +375,23 @@ struct kvm_mtrr {
 	struct list_head head;
 };
 
+/* Hyper-V synthetic interrupt controller (SynIC)*/
+struct kvm_vcpu_hv_synic {
+	u64 version;
+	u64 control;
+	u64 msg_page;
+	u64 evt_page;
+	atomic64_t sint[HV_SYNIC_SINT_COUNT];
+	atomic_t sint_to_gsi[HV_SYNIC_SINT_COUNT];
+	DECLARE_BITMAP(auto_eoi_bitmap, 256);
+	DECLARE_BITMAP(vec_bitmap, 256);
+};
+
 /* Hyper-V per vcpu emulation context */
 struct kvm_vcpu_hv {
 	u64 hv_vapic;
 	s64 runtime_offset;
+	struct kvm_vcpu_hv_synic synic;
 };
 
 struct kvm_vcpu_arch {
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 62cf8c9..8ff71f3 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -23,13 +23,296 @@
 
 #include "x86.h"
 #include "lapic.h"
+#include "ioapic.h"
 #include "hyperv.h"
 
 #include <linux/kvm_host.h>
+#include <asm/apicdef.h>
 #include <trace/events/kvm.h>
 
 #include "trace.h"
 
+static inline u64 synic_read_sint(struct kvm_vcpu_hv_synic *synic, int sint)
+{
+	return atomic64_read(&synic->sint[sint]);
+}
+
+static inline int synic_get_sint_vector(u64 sint_value)
+{
+	if (sint_value & HV_SYNIC_SINT_MASKED)
+		return -1;
+	return sint_value & HV_SYNIC_SINT_VECTOR_MASK;
+}
+
+static bool synic_has_vector_connected(struct kvm_vcpu_hv_synic *synic,
+				      int vector)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
+		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
+			return true;
+	}
+	return false;
+}
+
+static bool synic_has_vector_auto_eoi(struct kvm_vcpu_hv_synic *synic,
+				     int vector)
+{
+	int i;
+	u64 sint_value;
+
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
+		sint_value = synic_read_sint(synic, i);
+		if (synic_get_sint_vector(sint_value) == vector &&
+		    sint_value & HV_SYNIC_SINT_AUTO_EOI)
+			return true;
+	}
+	return false;
+}
+
+static int synic_set_sint(struct kvm_vcpu_hv_synic *synic, int sint, u64 data)
+{
+	int vector;
+
+	vector = data & HV_SYNIC_SINT_VECTOR_MASK;
+	if (vector < 16)
+		return 1;
+	/*
+	 * Guest may configure multiple SINTs to use the same vector, so
+	 * we maintain a bitmap of vectors handled by synic, and a
+	 * bitmap of vectors with auto-eoi behavior.  The bitmaps are
+	 * updated here, and atomically queried on fast paths.
+	 */
+
+	atomic64_set(&synic->sint[sint], data);
+
+	if (synic_has_vector_connected(synic, vector))
+		__set_bit(vector, synic->vec_bitmap);
+	else
+		__clear_bit(vector, synic->vec_bitmap);
+
+	if (synic_has_vector_auto_eoi(synic, vector))
+		__set_bit(vector, synic->auto_eoi_bitmap);
+	else
+		__clear_bit(vector, synic->auto_eoi_bitmap);
+
+	/* Load SynIC vectors into EOI exit bitmap */
+	kvm_make_request(KVM_REQ_SCAN_IOAPIC, synic_to_vcpu(synic));
+	return 0;
+}
+
+static struct kvm_vcpu_hv_synic *synic_get(struct kvm *kvm, u32 vcpu_id)
+{
+	struct kvm_vcpu *vcpu;
+
+	if (vcpu_id >= atomic_read(&kvm->online_vcpus))
+		return NULL;
+	vcpu = kvm_get_vcpu(kvm, vcpu_id);
+	if (!vcpu)
+		return NULL;
+
+	return vcpu_to_synic(vcpu);
+}
+
+static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
+{
+	struct kvm *kvm = vcpu->kvm;
+	int gsi, idx;
+
+	vcpu_debug(vcpu, "Hyper-V SynIC acked sint %d\n", sint);
+
+	idx = srcu_read_lock(&kvm->irq_srcu);
+	gsi = atomic_read(&vcpu_to_synic(vcpu)->sint_to_gsi[sint]);
+	if (gsi != -1)
+		kvm_notify_acked_gsi(kvm, gsi);
+	srcu_read_unlock(&kvm->irq_srcu, idx);
+}
+
+static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
+			 u32 msr, u64 data, bool host)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	int ret;
+
+	vcpu_debug(vcpu, "Hyper-V SynIC set msr 0x%x 0x%llx host %d\n",
+		   msr, data, host);
+	ret = 0;
+	switch (msr) {
+	case HV_X64_MSR_SCONTROL:
+		synic->control = data;
+		break;
+	case HV_X64_MSR_SVERSION:
+		if (!host) {
+			ret = 1;
+			break;
+		}
+		synic->version = data;
+		break;
+	case HV_X64_MSR_SIEFP:
+		if (data & HV_SYNIC_SIEFP_ENABLE)
+			if (kvm_clear_guest(vcpu->kvm,
+					    data & PAGE_MASK, PAGE_SIZE)) {
+				ret = 1;
+				break;
+			}
+		synic->evt_page = data;
+		break;
+	case HV_X64_MSR_SIMP:
+		if (data & HV_SYNIC_SIMP_ENABLE)
+			if (kvm_clear_guest(vcpu->kvm,
+					    data & PAGE_MASK, PAGE_SIZE)) {
+				ret = 1;
+				break;
+			}
+		synic->msg_page = data;
+		break;
+	case HV_X64_MSR_EOM: {
+		int i;
+
+		for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
+			kvm_hv_notify_acked_sint(vcpu, i);
+		break;
+	}
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		ret = synic_set_sint(synic, msr - HV_X64_MSR_SINT0, data);
+		break;
+	default:
+		ret = 1;
+		break;
+	}
+	return ret;
+}
+
+static int synic_get_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 *pdata)
+{
+	int ret;
+
+	ret = 0;
+	switch (msr) {
+	case HV_X64_MSR_SCONTROL:
+		*pdata = synic->control;
+		break;
+	case HV_X64_MSR_SVERSION:
+		*pdata = synic->version;
+		break;
+	case HV_X64_MSR_SIEFP:
+		*pdata = synic->evt_page;
+		break;
+	case HV_X64_MSR_SIMP:
+		*pdata = synic->msg_page;
+		break;
+	case HV_X64_MSR_EOM:
+		*pdata = 0;
+		break;
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		*pdata = atomic64_read(&synic->sint[msr - HV_X64_MSR_SINT0]);
+		break;
+	default:
+		ret = 1;
+		break;
+	}
+	return ret;
+}
+
+int synic_set_irq(struct kvm_vcpu_hv_synic *synic, u32 sint)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	struct kvm_lapic_irq irq;
+	int ret, vector;
+
+	if (sint >= ARRAY_SIZE(synic->sint))
+		return -EINVAL;
+
+	vector = synic_get_sint_vector(synic_read_sint(synic, sint));
+	if (vector < 0)
+		return -ENOENT;
+
+	memset(&irq, 0, sizeof(irq));
+	irq.dest_id = kvm_apic_id(vcpu->arch.apic);
+	irq.dest_mode = APIC_DEST_PHYSICAL;
+	irq.delivery_mode = APIC_DM_FIXED;
+	irq.vector = vector;
+	irq.level = 1;
+
+	ret = kvm_irq_delivery_to_apic(vcpu->kvm, NULL, &irq, NULL);
+	vcpu_debug(vcpu, "Hyper-V SynIC set irq ret %d\n", ret);
+	return ret;
+}
+
+int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint)
+{
+	struct kvm_vcpu_hv_synic *synic;
+
+	synic = synic_get(kvm, vcpu_id);
+	if (!synic)
+		return -EINVAL;
+
+	return synic_set_irq(synic, sint);
+}
+
+void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector)
+{
+	struct kvm_vcpu_hv_synic *synic = vcpu_to_synic(vcpu);
+	int i;
+
+	vcpu_debug(vcpu, "Hyper-V SynIC send eoi vec %d\n", vector);
+
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
+		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
+			kvm_hv_notify_acked_sint(vcpu, i);
+}
+
+static int kvm_hv_set_sint_gsi(struct kvm *kvm, u32 vcpu_id, u32 sint, int gsi)
+{
+	struct kvm_vcpu_hv_synic *synic;
+
+	synic = synic_get(kvm, vcpu_id);
+	if (!synic)
+		return -EINVAL;
+
+	if (sint >= ARRAY_SIZE(synic->sint_to_gsi))
+		return -EINVAL;
+
+	atomic_set(&synic->sint_to_gsi[sint], gsi);
+	return 0;
+}
+
+void kvm_hv_irq_routing_update(struct kvm *kvm)
+{
+	struct kvm_irq_routing_table *irq_rt;
+	struct kvm_kernel_irq_routing_entry *e;
+	u32 gsi;
+
+	irq_rt = srcu_dereference_check(kvm->irq_routing, &kvm->irq_srcu,
+					lockdep_is_held(&kvm->irq_lock));
+
+	for (gsi = 0; gsi < irq_rt->nr_rt_entries; gsi++) {
+		hlist_for_each_entry(e, &irq_rt->map[gsi], link) {
+			if (e->type == KVM_IRQ_ROUTING_HV_SINT)
+				kvm_hv_set_sint_gsi(kvm, e->hv_sint.vcpu,
+						    e->hv_sint.sint, gsi);
+		}
+	}
+}
+
+static void synic_init(struct kvm_vcpu_hv_synic *synic)
+{
+	int i;
+
+	memset(synic, 0, sizeof(*synic));
+	synic->version = HV_SYNIC_VERSION_1;
+	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
+		atomic64_set(&synic->sint[i], HV_SYNIC_SINT_MASKED);
+		atomic_set(&synic->sint_to_gsi[i], -1);
+	}
+}
+
+void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu)
+{
+	synic_init(vcpu_to_synic(vcpu));
+}
+
 static bool kvm_hv_msr_partition_wide(u32 msr)
 {
 	bool r = false;
@@ -226,6 +509,13 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)
 			return 1;
 		hv->runtime_offset = data - current_task_runtime_100ns();
 		break;
+	case HV_X64_MSR_SCONTROL:
+	case HV_X64_MSR_SVERSION:
+	case HV_X64_MSR_SIEFP:
+	case HV_X64_MSR_SIMP:
+	case HV_X64_MSR_EOM:
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		return synic_set_msr(vcpu_to_synic(vcpu), msr, data, host);
 	default:
 		vcpu_unimpl(vcpu, "Hyper-V uhandled wrmsr: 0x%x data 0x%llx\n",
 			    msr, data);
@@ -304,6 +594,13 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 	case HV_X64_MSR_VP_RUNTIME:
 		data = current_task_runtime_100ns() + hv->runtime_offset;
 		break;
+	case HV_X64_MSR_SCONTROL:
+	case HV_X64_MSR_SVERSION:
+	case HV_X64_MSR_SIEFP:
+	case HV_X64_MSR_SIMP:
+	case HV_X64_MSR_EOM:
+	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
+		return synic_get_msr(vcpu_to_synic(vcpu), msr, pdata);
 	default:
 		vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
 		return 1;
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index c7bce55..8668612 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -29,4 +29,25 @@ int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
 bool kvm_hv_hypercall_enabled(struct kvm *kvm);
 int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
 
+int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
+void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
+
+static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
+{
+	return &vcpu->arch.hyperv.synic;
+}
+
+static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
+{
+	struct kvm_vcpu_hv *hv;
+	struct kvm_vcpu_arch *arch;
+
+	hv = container_of(synic, struct kvm_vcpu_hv, synic);
+	arch = container_of(hv, struct kvm_vcpu_arch, hyperv);
+	return container_of(arch, struct kvm_vcpu, arch);
+}
+void kvm_hv_irq_routing_update(struct kvm *kvm);
+
+void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
+
 #endif
diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
index fe91f72..5e195b9 100644
--- a/arch/x86/kvm/irq_comm.c
+++ b/arch/x86/kvm/irq_comm.c
@@ -33,6 +33,8 @@
 
 #include "lapic.h"
 
+#include "hyperv.h"
+
 static int kvm_set_pic_irq(struct kvm_kernel_irq_routing_entry *e,
 			   struct kvm *kvm, int irq_source_id, int level,
 			   bool line_status)
@@ -251,6 +253,16 @@ void kvm_fire_mask_notifiers(struct kvm *kvm, unsigned irqchip, unsigned pin,
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
 
+static int kvm_hv_set_sint(struct kvm_kernel_irq_routing_entry *e,
+		    struct kvm *kvm, int irq_source_id, int level,
+		    bool line_status)
+{
+	if (!level)
+		return -1;
+
+	return kvm_hv_synic_set_irq(kvm, e->hv_sint.vcpu, e->hv_sint.sint);
+}
+
 int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
 			  const struct kvm_irq_routing_entry *ue)
 {
@@ -289,6 +301,11 @@ int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
 		e->msi.address_hi = ue->u.msi.address_hi;
 		e->msi.data = ue->u.msi.data;
 		break;
+	case KVM_IRQ_ROUTING_HV_SINT:
+		e->set = kvm_hv_set_sint;
+		e->hv_sint.vcpu = ue->u.hv_sint.vcpu;
+		e->hv_sint.sint = ue->u.hv_sint.sint;
+		break;
 	default:
 		goto out;
 	}
@@ -405,3 +422,20 @@ void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
 	}
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
+
+int kvm_arch_set_irq(struct kvm_kernel_irq_routing_entry *irq, struct kvm *kvm,
+		     int irq_source_id, int level, bool line_status)
+{
+	switch (irq->type) {
+	case KVM_IRQ_ROUTING_HV_SINT:
+		return kvm_hv_set_sint(irq, kvm, irq_source_id, level,
+				       line_status);
+	default:
+		return -EWOULDBLOCK;
+	}
+}
+
+void kvm_arch_irq_routing_update(struct kvm *kvm)
+{
+	kvm_hv_irq_routing_update(kvm);
+}
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index dc03a01..3132478 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -41,6 +41,7 @@
 #include "trace.h"
 #include "x86.h"
 #include "cpuid.h"
+#include "hyperv.h"
 
 #ifndef CONFIG_X86_64
 #define mod_64(x, y) ((x) - (y) * div64_u64(x, y))
@@ -128,11 +129,6 @@ static inline int apic_enabled(struct kvm_lapic *apic)
 	(LVT_MASK | APIC_MODE_MASK | APIC_INPUT_POLARITY | \
 	 APIC_LVT_REMOTE_IRR | APIC_LVT_LEVEL_TRIGGER)
 
-static inline int kvm_apic_id(struct kvm_lapic *apic)
-{
-	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
-}
-
 /* The logical map is definitely wrong if we have multiple
  * modes at the same time.  (Physical map is always right.)
  */
@@ -850,7 +846,8 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
 				apic_clear_vector(vector, apic->regs + APIC_TMR);
 		}
 
-		if (kvm_x86_ops->deliver_posted_interrupt)
+		if (kvm_x86_ops->deliver_posted_interrupt &&
+		    !test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap))
 			kvm_x86_ops->deliver_posted_interrupt(vcpu, vector);
 		else {
 			apic_set_irr(vector, apic);
@@ -972,6 +969,9 @@ static int apic_set_eoi(struct kvm_lapic *apic)
 	apic_clear_isr(vector, apic);
 	apic_update_ppr(apic);
 
+	if (test_bit(vector, vcpu_to_synic(apic->vcpu)->vec_bitmap))
+		kvm_hv_synic_send_eoi(apic->vcpu, vector);
+
 	kvm_ioapic_send_eoi(apic, vector);
 	kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
 	return vector;
@@ -1881,6 +1881,12 @@ int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu)
 	apic_set_isr(vector, apic);
 	apic_update_ppr(apic);
 	apic_clear_irr(vector, apic);
+
+	if (test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap)) {
+		apic_clear_isr(vector, apic);
+		apic_update_ppr(apic);
+	}
+
 	return vector;
 }
 
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index fde8e35d..6c64090 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -164,6 +164,11 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)
 	return kvm_vcpu_has_lapic(vcpu) && test_bit(KVM_APIC_INIT, &vcpu->arch.apic->pending_events);
 }
 
+static inline int kvm_apic_id(struct kvm_lapic *apic)
+{
+	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
+}
+
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
 
 void wait_lapic_expire(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index fb6cfbf..b853b2df 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -959,6 +959,7 @@ static u32 emulated_msrs[] = {
 	HV_X64_MSR_RESET,
 	HV_X64_MSR_VP_INDEX,
 	HV_X64_MSR_VP_RUNTIME,
+	HV_X64_MSR_SCONTROL,
 	HV_X64_MSR_APIC_ASSIST_PAGE, MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
 	MSR_KVM_PV_EOI_EN,
 
@@ -2441,6 +2442,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_HYPERV:
 	case KVM_CAP_HYPERV_VAPIC:
 	case KVM_CAP_HYPERV_SPIN:
+	case KVM_CAP_HYPERV_SYNIC:
 	case KVM_CAP_PCI_SEGMENT:
 	case KVM_CAP_DEBUGREGS:
 	case KVM_CAP_X86_ROBUST_SINGLESTEP:
@@ -6195,6 +6197,8 @@ static void process_smi(struct kvm_vcpu *vcpu)
 
 static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
 {
+	u64 eoi_exit_bitmap[4];
+
 	if (!kvm_apic_hw_enabled(vcpu->arch.apic))
 		return;
 
@@ -6205,8 +6209,10 @@ static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
 	else
 		kvm_ioapic_scan_entry(vcpu, vcpu->arch.ioapic_handled_vectors);
 
-	kvm_x86_ops->load_eoi_exitmap(vcpu,
-				      (u64 *)vcpu->arch.ioapic_handled_vectors);
+	bitmap_or((ulong *)eoi_exit_bitmap, vcpu->arch.ioapic_handled_vectors,
+		  vcpu_to_synic(vcpu)->vec_bitmap, 256);
+
+	kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap);
 }
 
 static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
@@ -7456,6 +7462,8 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
 
 	vcpu->arch.pending_external_vector = -1;
 
+	kvm_hv_vcpu_init(vcpu);
+
 	return 0;
 
 fail_free_mce_banks:
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index c742e79..43b0141 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -318,6 +318,11 @@ struct kvm_s390_adapter_int {
 	u32 adapter_id;
 };
 
+struct kvm_hv_sint {
+	u32 vcpu;
+	u32 sint;
+};
+
 struct kvm_kernel_irq_routing_entry {
 	u32 gsi;
 	u32 type;
@@ -331,6 +336,7 @@ struct kvm_kernel_irq_routing_entry {
 		} irqchip;
 		struct msi_msg msi;
 		struct kvm_s390_adapter_int adapter;
+		struct kvm_hv_sint hv_sint;
 	};
 	struct hlist_node link;
 };
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 03f3618..27ce460 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -831,6 +831,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_GUEST_DEBUG_HW_WPS 120
 #define KVM_CAP_SPLIT_IRQCHIP 121
 #define KVM_CAP_IOEVENTFD_ANY_LENGTH 122
+#define KVM_CAP_HYPERV_SYNIC 123
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -854,10 +855,16 @@ struct kvm_irq_routing_s390_adapter {
 	__u32 adapter_id;
 };
 
+struct kvm_irq_routing_hv_sint {
+	__u32 vcpu;
+	__u32 sint;
+};
+
 /* gsi routing entry types */
 #define KVM_IRQ_ROUTING_IRQCHIP 1
 #define KVM_IRQ_ROUTING_MSI 2
 #define KVM_IRQ_ROUTING_S390_ADAPTER 3
+#define KVM_IRQ_ROUTING_HV_SINT 4
 
 struct kvm_irq_routing_entry {
 	__u32 gsi;
@@ -868,6 +875,7 @@ struct kvm_irq_routing_entry {
 		struct kvm_irq_routing_irqchip irqchip;
 		struct kvm_irq_routing_msi msi;
 		struct kvm_irq_routing_s390_adapter adapter;
+		struct kvm_irq_routing_hv_sint hv_sint;
 		__u32 pad[8];
 	} u;
 };
-- 
2.4.3

Re: [Qemu-devel] [PATCH v3 8/9] kvm/x86: Hyper-V synthetic interrupt controller

[Paolo Bonzini]

Hi Andrey,

just one question.  Is kvm_arch_set_irq actually needed?  I think
everything should work fine without it.  Can you check?  If so, I can
remove it myself and revert the patch that introduced the hook.

Paolo

On 22/10/2015 18:09, Andrey Smetanin wrote:
> SynIC (synthetic interrupt controller) is a lapic extension,
> which is controlled via MSRs and maintains for each vCPU
>  - 16 synthetic interrupt "lines" (SINT's); each can be configured to
>    trigger a specific interrupt vector optionally with auto-EOI
>    semantics
>  - a message page in the guest memory with 16 256-byte per-SINT message
>    slots
>  - an event flag page in the guest memory with 16 2048-bit per-SINT
>    event flag areas
> 
> The host triggers a SINT whenever it delivers a new message to the
> corresponding slot or flips an event flag bit in the corresponding area.
> The guest informs the host that it can try delivering a message by
> explicitly asserting EOI in lapic or writing to End-Of-Message (EOM)
> MSR.
> 
> The userspace (qemu) triggers interrupts and receives EOM notifications
> via irqfd with resampler; for that, a GSI is allocated for each
> configured SINT, and irq_routing api is extended to support GSI-SINT
> mapping.
> 
> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> CC: "K. Y. Srinivasan" <kys@microsoft.com>
> CC: Gleb Natapov <gleb@kernel.org>
> CC: Paolo Bonzini <pbonzini@redhat.com>
> CC: Roman Kagan <rkagan@virtuozzo.com>
> 
> Changes v3:
> * added KVM_CAP_HYPERV_SYNIC and KVM_IRQ_ROUTING_HV_SINT notes into
> docs
> 
> Changes v2:
> * do not use posted interrupts for Hyper-V SynIC AutoEOI vectors
> * add Hyper-V SynIC vectors into EOI exit bitmap
> * Hyper-V SyniIC SINT msr write logic simplified
> ---
>  Documentation/virtual/kvm/api.txt |  14 ++
>  arch/x86/include/asm/kvm_host.h   |  14 ++
>  arch/x86/kvm/hyperv.c             | 297 ++++++++++++++++++++++++++++++++++++++
>  arch/x86/kvm/hyperv.h             |  21 +++
>  arch/x86/kvm/irq_comm.c           |  34 +++++
>  arch/x86/kvm/lapic.c              |  18 ++-
>  arch/x86/kvm/lapic.h              |   5 +
>  arch/x86/kvm/x86.c                |  12 +-
>  include/linux/kvm_host.h          |   6 +
>  include/uapi/linux/kvm.h          |   8 +
>  10 files changed, 421 insertions(+), 8 deletions(-)
> 
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index 092ee9f..8710418 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -1451,6 +1451,7 @@ struct kvm_irq_routing_entry {
>  		struct kvm_irq_routing_irqchip irqchip;
>  		struct kvm_irq_routing_msi msi;
>  		struct kvm_irq_routing_s390_adapter adapter;
> +		struct kvm_irq_routing_hv_sint hv_sint;
>  		__u32 pad[8];
>  	} u;
>  };
> @@ -1459,6 +1460,7 @@ struct kvm_irq_routing_entry {
>  #define KVM_IRQ_ROUTING_IRQCHIP 1
>  #define KVM_IRQ_ROUTING_MSI 2
>  #define KVM_IRQ_ROUTING_S390_ADAPTER 3
> +#define KVM_IRQ_ROUTING_HV_SINT 4
>  
>  No flags are specified so far, the corresponding field must be set to zero.
>  
> @@ -1482,6 +1484,10 @@ struct kvm_irq_routing_s390_adapter {
>  	__u32 adapter_id;
>  };
>  
> +struct kvm_irq_routing_hv_sint {
> +	__u32 vcpu;
> +	__u32 sint;
> +};
>  
>  4.53 KVM_ASSIGN_SET_MSIX_NR (deprecated)
>  
> @@ -3685,3 +3691,11 @@ available, means that that the kernel has an implementation of the
>  H_RANDOM hypercall backed by a hardware random-number generator.
>  If present, the kernel H_RANDOM handler can be enabled for guest use
>  with the KVM_CAP_PPC_ENABLE_HCALL capability.
> +
> +8.2 KVM_CAP_HYPERV_SYNIC
> +
> +Architectures: x86
> +This capability, if KVM_CHECK_EXTENSION indicates that it is
> +available, means that that the kernel has an implementation of the
> +Hyper-V Synthetic interrupt controller(SynIC). SynIC is used to
> +support Windows Hyper-V based guest paravirt drivers(VMBus).
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 3c6327d..8434f88 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -25,6 +25,7 @@
>  #include <linux/pvclock_gtod.h>
>  #include <linux/clocksource.h>
>  #include <linux/irqbypass.h>
> +#include <linux/hyperv.h>
>  
>  #include <asm/pvclock-abi.h>
>  #include <asm/desc.h>
> @@ -374,10 +375,23 @@ struct kvm_mtrr {
>  	struct list_head head;
>  };
>  
> +/* Hyper-V synthetic interrupt controller (SynIC)*/
> +struct kvm_vcpu_hv_synic {
> +	u64 version;
> +	u64 control;
> +	u64 msg_page;
> +	u64 evt_page;
> +	atomic64_t sint[HV_SYNIC_SINT_COUNT];
> +	atomic_t sint_to_gsi[HV_SYNIC_SINT_COUNT];
> +	DECLARE_BITMAP(auto_eoi_bitmap, 256);
> +	DECLARE_BITMAP(vec_bitmap, 256);
> +};
> +
>  /* Hyper-V per vcpu emulation context */
>  struct kvm_vcpu_hv {
>  	u64 hv_vapic;
>  	s64 runtime_offset;
> +	struct kvm_vcpu_hv_synic synic;
>  };
>  
>  struct kvm_vcpu_arch {
> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
> index 62cf8c9..8ff71f3 100644
> --- a/arch/x86/kvm/hyperv.c
> +++ b/arch/x86/kvm/hyperv.c
> @@ -23,13 +23,296 @@
>  
>  #include "x86.h"
>  #include "lapic.h"
> +#include "ioapic.h"
>  #include "hyperv.h"
>  
>  #include <linux/kvm_host.h>
> +#include <asm/apicdef.h>
>  #include <trace/events/kvm.h>
>  
>  #include "trace.h"
>  
> +static inline u64 synic_read_sint(struct kvm_vcpu_hv_synic *synic, int sint)
> +{
> +	return atomic64_read(&synic->sint[sint]);
> +}
> +
> +static inline int synic_get_sint_vector(u64 sint_value)
> +{
> +	if (sint_value & HV_SYNIC_SINT_MASKED)
> +		return -1;
> +	return sint_value & HV_SYNIC_SINT_VECTOR_MASK;
> +}
> +
> +static bool synic_has_vector_connected(struct kvm_vcpu_hv_synic *synic,
> +				      int vector)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
> +		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
> +			return true;
> +	}
> +	return false;
> +}
> +
> +static bool synic_has_vector_auto_eoi(struct kvm_vcpu_hv_synic *synic,
> +				     int vector)
> +{
> +	int i;
> +	u64 sint_value;
> +
> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
> +		sint_value = synic_read_sint(synic, i);
> +		if (synic_get_sint_vector(sint_value) == vector &&
> +		    sint_value & HV_SYNIC_SINT_AUTO_EOI)
> +			return true;
> +	}
> +	return false;
> +}
> +
> +static int synic_set_sint(struct kvm_vcpu_hv_synic *synic, int sint, u64 data)
> +{
> +	int vector;
> +
> +	vector = data & HV_SYNIC_SINT_VECTOR_MASK;
> +	if (vector < 16)
> +		return 1;
> +	/*
> +	 * Guest may configure multiple SINTs to use the same vector, so
> +	 * we maintain a bitmap of vectors handled by synic, and a
> +	 * bitmap of vectors with auto-eoi behavior.  The bitmaps are
> +	 * updated here, and atomically queried on fast paths.
> +	 */
> +
> +	atomic64_set(&synic->sint[sint], data);
> +
> +	if (synic_has_vector_connected(synic, vector))
> +		__set_bit(vector, synic->vec_bitmap);
> +	else
> +		__clear_bit(vector, synic->vec_bitmap);
> +
> +	if (synic_has_vector_auto_eoi(synic, vector))
> +		__set_bit(vector, synic->auto_eoi_bitmap);
> +	else
> +		__clear_bit(vector, synic->auto_eoi_bitmap);
> +
> +	/* Load SynIC vectors into EOI exit bitmap */
> +	kvm_make_request(KVM_REQ_SCAN_IOAPIC, synic_to_vcpu(synic));
> +	return 0;
> +}
> +
> +static struct kvm_vcpu_hv_synic *synic_get(struct kvm *kvm, u32 vcpu_id)
> +{
> +	struct kvm_vcpu *vcpu;
> +
> +	if (vcpu_id >= atomic_read(&kvm->online_vcpus))
> +		return NULL;
> +	vcpu = kvm_get_vcpu(kvm, vcpu_id);
> +	if (!vcpu)
> +		return NULL;
> +
> +	return vcpu_to_synic(vcpu);
> +}
> +
> +static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
> +{
> +	struct kvm *kvm = vcpu->kvm;
> +	int gsi, idx;
> +
> +	vcpu_debug(vcpu, "Hyper-V SynIC acked sint %d\n", sint);
> +
> +	idx = srcu_read_lock(&kvm->irq_srcu);
> +	gsi = atomic_read(&vcpu_to_synic(vcpu)->sint_to_gsi[sint]);
> +	if (gsi != -1)
> +		kvm_notify_acked_gsi(kvm, gsi);
> +	srcu_read_unlock(&kvm->irq_srcu, idx);
> +}
> +
> +static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
> +			 u32 msr, u64 data, bool host)
> +{
> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
> +	int ret;
> +
> +	vcpu_debug(vcpu, "Hyper-V SynIC set msr 0x%x 0x%llx host %d\n",
> +		   msr, data, host);
> +	ret = 0;
> +	switch (msr) {
> +	case HV_X64_MSR_SCONTROL:
> +		synic->control = data;
> +		break;
> +	case HV_X64_MSR_SVERSION:
> +		if (!host) {
> +			ret = 1;
> +			break;
> +		}
> +		synic->version = data;
> +		break;
> +	case HV_X64_MSR_SIEFP:
> +		if (data & HV_SYNIC_SIEFP_ENABLE)
> +			if (kvm_clear_guest(vcpu->kvm,
> +					    data & PAGE_MASK, PAGE_SIZE)) {
> +				ret = 1;
> +				break;
> +			}
> +		synic->evt_page = data;
> +		break;
> +	case HV_X64_MSR_SIMP:
> +		if (data & HV_SYNIC_SIMP_ENABLE)
> +			if (kvm_clear_guest(vcpu->kvm,
> +					    data & PAGE_MASK, PAGE_SIZE)) {
> +				ret = 1;
> +				break;
> +			}
> +		synic->msg_page = data;
> +		break;
> +	case HV_X64_MSR_EOM: {
> +		int i;
> +
> +		for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
> +			kvm_hv_notify_acked_sint(vcpu, i);
> +		break;
> +	}
> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
> +		ret = synic_set_sint(synic, msr - HV_X64_MSR_SINT0, data);
> +		break;
> +	default:
> +		ret = 1;
> +		break;
> +	}
> +	return ret;
> +}
> +
> +static int synic_get_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 *pdata)
> +{
> +	int ret;
> +
> +	ret = 0;
> +	switch (msr) {
> +	case HV_X64_MSR_SCONTROL:
> +		*pdata = synic->control;
> +		break;
> +	case HV_X64_MSR_SVERSION:
> +		*pdata = synic->version;
> +		break;
> +	case HV_X64_MSR_SIEFP:
> +		*pdata = synic->evt_page;
> +		break;
> +	case HV_X64_MSR_SIMP:
> +		*pdata = synic->msg_page;
> +		break;
> +	case HV_X64_MSR_EOM:
> +		*pdata = 0;
> +		break;
> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
> +		*pdata = atomic64_read(&synic->sint[msr - HV_X64_MSR_SINT0]);
> +		break;
> +	default:
> +		ret = 1;
> +		break;
> +	}
> +	return ret;
> +}
> +
> +int synic_set_irq(struct kvm_vcpu_hv_synic *synic, u32 sint)
> +{
> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
> +	struct kvm_lapic_irq irq;
> +	int ret, vector;
> +
> +	if (sint >= ARRAY_SIZE(synic->sint))
> +		return -EINVAL;
> +
> +	vector = synic_get_sint_vector(synic_read_sint(synic, sint));
> +	if (vector < 0)
> +		return -ENOENT;
> +
> +	memset(&irq, 0, sizeof(irq));
> +	irq.dest_id = kvm_apic_id(vcpu->arch.apic);
> +	irq.dest_mode = APIC_DEST_PHYSICAL;
> +	irq.delivery_mode = APIC_DM_FIXED;
> +	irq.vector = vector;
> +	irq.level = 1;
> +
> +	ret = kvm_irq_delivery_to_apic(vcpu->kvm, NULL, &irq, NULL);
> +	vcpu_debug(vcpu, "Hyper-V SynIC set irq ret %d\n", ret);
> +	return ret;
> +}
> +
> +int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint)
> +{
> +	struct kvm_vcpu_hv_synic *synic;
> +
> +	synic = synic_get(kvm, vcpu_id);
> +	if (!synic)
> +		return -EINVAL;
> +
> +	return synic_set_irq(synic, sint);
> +}
> +
> +void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector)
> +{
> +	struct kvm_vcpu_hv_synic *synic = vcpu_to_synic(vcpu);
> +	int i;
> +
> +	vcpu_debug(vcpu, "Hyper-V SynIC send eoi vec %d\n", vector);
> +
> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
> +		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
> +			kvm_hv_notify_acked_sint(vcpu, i);
> +}
> +
> +static int kvm_hv_set_sint_gsi(struct kvm *kvm, u32 vcpu_id, u32 sint, int gsi)
> +{
> +	struct kvm_vcpu_hv_synic *synic;
> +
> +	synic = synic_get(kvm, vcpu_id);
> +	if (!synic)
> +		return -EINVAL;
> +
> +	if (sint >= ARRAY_SIZE(synic->sint_to_gsi))
> +		return -EINVAL;
> +
> +	atomic_set(&synic->sint_to_gsi[sint], gsi);
> +	return 0;
> +}
> +
> +void kvm_hv_irq_routing_update(struct kvm *kvm)
> +{
> +	struct kvm_irq_routing_table *irq_rt;
> +	struct kvm_kernel_irq_routing_entry *e;
> +	u32 gsi;
> +
> +	irq_rt = srcu_dereference_check(kvm->irq_routing, &kvm->irq_srcu,
> +					lockdep_is_held(&kvm->irq_lock));
> +
> +	for (gsi = 0; gsi < irq_rt->nr_rt_entries; gsi++) {
> +		hlist_for_each_entry(e, &irq_rt->map[gsi], link) {
> +			if (e->type == KVM_IRQ_ROUTING_HV_SINT)
> +				kvm_hv_set_sint_gsi(kvm, e->hv_sint.vcpu,
> +						    e->hv_sint.sint, gsi);
> +		}
> +	}
> +}
> +
> +static void synic_init(struct kvm_vcpu_hv_synic *synic)
> +{
> +	int i;
> +
> +	memset(synic, 0, sizeof(*synic));
> +	synic->version = HV_SYNIC_VERSION_1;
> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
> +		atomic64_set(&synic->sint[i], HV_SYNIC_SINT_MASKED);
> +		atomic_set(&synic->sint_to_gsi[i], -1);
> +	}
> +}
> +
> +void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu)
> +{
> +	synic_init(vcpu_to_synic(vcpu));
> +}
> +
>  static bool kvm_hv_msr_partition_wide(u32 msr)
>  {
>  	bool r = false;
> @@ -226,6 +509,13 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)
>  			return 1;
>  		hv->runtime_offset = data - current_task_runtime_100ns();
>  		break;
> +	case HV_X64_MSR_SCONTROL:
> +	case HV_X64_MSR_SVERSION:
> +	case HV_X64_MSR_SIEFP:
> +	case HV_X64_MSR_SIMP:
> +	case HV_X64_MSR_EOM:
> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
> +		return synic_set_msr(vcpu_to_synic(vcpu), msr, data, host);
>  	default:
>  		vcpu_unimpl(vcpu, "Hyper-V uhandled wrmsr: 0x%x data 0x%llx\n",
>  			    msr, data);
> @@ -304,6 +594,13 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
>  	case HV_X64_MSR_VP_RUNTIME:
>  		data = current_task_runtime_100ns() + hv->runtime_offset;
>  		break;
> +	case HV_X64_MSR_SCONTROL:
> +	case HV_X64_MSR_SVERSION:
> +	case HV_X64_MSR_SIEFP:
> +	case HV_X64_MSR_SIMP:
> +	case HV_X64_MSR_EOM:
> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
> +		return synic_get_msr(vcpu_to_synic(vcpu), msr, pdata);
>  	default:
>  		vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
>  		return 1;
> diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
> index c7bce55..8668612 100644
> --- a/arch/x86/kvm/hyperv.h
> +++ b/arch/x86/kvm/hyperv.h
> @@ -29,4 +29,25 @@ int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
>  bool kvm_hv_hypercall_enabled(struct kvm *kvm);
>  int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
>  
> +int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
> +void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
> +
> +static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
> +{
> +	return &vcpu->arch.hyperv.synic;
> +}
> +
> +static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
> +{
> +	struct kvm_vcpu_hv *hv;
> +	struct kvm_vcpu_arch *arch;
> +
> +	hv = container_of(synic, struct kvm_vcpu_hv, synic);
> +	arch = container_of(hv, struct kvm_vcpu_arch, hyperv);
> +	return container_of(arch, struct kvm_vcpu, arch);
> +}
> +void kvm_hv_irq_routing_update(struct kvm *kvm);
> +
> +void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
> +
>  #endif
> diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
> index fe91f72..5e195b9 100644
> --- a/arch/x86/kvm/irq_comm.c
> +++ b/arch/x86/kvm/irq_comm.c
> @@ -33,6 +33,8 @@
>  
>  #include "lapic.h"
>  
> +#include "hyperv.h"
> +
>  static int kvm_set_pic_irq(struct kvm_kernel_irq_routing_entry *e,
>  			   struct kvm *kvm, int irq_source_id, int level,
>  			   bool line_status)
> @@ -251,6 +253,16 @@ void kvm_fire_mask_notifiers(struct kvm *kvm, unsigned irqchip, unsigned pin,
>  	srcu_read_unlock(&kvm->irq_srcu, idx);
>  }
>  
> +static int kvm_hv_set_sint(struct kvm_kernel_irq_routing_entry *e,
> +		    struct kvm *kvm, int irq_source_id, int level,
> +		    bool line_status)
> +{
> +	if (!level)
> +		return -1;
> +
> +	return kvm_hv_synic_set_irq(kvm, e->hv_sint.vcpu, e->hv_sint.sint);
> +}
> +
>  int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
>  			  const struct kvm_irq_routing_entry *ue)
>  {
> @@ -289,6 +301,11 @@ int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
>  		e->msi.address_hi = ue->u.msi.address_hi;
>  		e->msi.data = ue->u.msi.data;
>  		break;
> +	case KVM_IRQ_ROUTING_HV_SINT:
> +		e->set = kvm_hv_set_sint;
> +		e->hv_sint.vcpu = ue->u.hv_sint.vcpu;
> +		e->hv_sint.sint = ue->u.hv_sint.sint;
> +		break;
>  	default:
>  		goto out;
>  	}
> @@ -405,3 +422,20 @@ void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
>  	}
>  	srcu_read_unlock(&kvm->irq_srcu, idx);
>  }
> +
> +int kvm_arch_set_irq(struct kvm_kernel_irq_routing_entry *irq, struct kvm *kvm,
> +		     int irq_source_id, int level, bool line_status)
> +{
> +	switch (irq->type) {
> +	case KVM_IRQ_ROUTING_HV_SINT:
> +		return kvm_hv_set_sint(irq, kvm, irq_source_id, level,
> +				       line_status);
> +	default:
> +		return -EWOULDBLOCK;
> +	}
> +}
> +
> +void kvm_arch_irq_routing_update(struct kvm *kvm)
> +{
> +	kvm_hv_irq_routing_update(kvm);
> +}
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index dc03a01..3132478 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -41,6 +41,7 @@
>  #include "trace.h"
>  #include "x86.h"
>  #include "cpuid.h"
> +#include "hyperv.h"
>  
>  #ifndef CONFIG_X86_64
>  #define mod_64(x, y) ((x) - (y) * div64_u64(x, y))
> @@ -128,11 +129,6 @@ static inline int apic_enabled(struct kvm_lapic *apic)
>  	(LVT_MASK | APIC_MODE_MASK | APIC_INPUT_POLARITY | \
>  	 APIC_LVT_REMOTE_IRR | APIC_LVT_LEVEL_TRIGGER)
>  
> -static inline int kvm_apic_id(struct kvm_lapic *apic)
> -{
> -	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
> -}
> -
>  /* The logical map is definitely wrong if we have multiple
>   * modes at the same time.  (Physical map is always right.)
>   */
> @@ -850,7 +846,8 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
>  				apic_clear_vector(vector, apic->regs + APIC_TMR);
>  		}
>  
> -		if (kvm_x86_ops->deliver_posted_interrupt)
> +		if (kvm_x86_ops->deliver_posted_interrupt &&
> +		    !test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap))
>  			kvm_x86_ops->deliver_posted_interrupt(vcpu, vector);
>  		else {
>  			apic_set_irr(vector, apic);
> @@ -972,6 +969,9 @@ static int apic_set_eoi(struct kvm_lapic *apic)
>  	apic_clear_isr(vector, apic);
>  	apic_update_ppr(apic);
>  
> +	if (test_bit(vector, vcpu_to_synic(apic->vcpu)->vec_bitmap))
> +		kvm_hv_synic_send_eoi(apic->vcpu, vector);
> +
>  	kvm_ioapic_send_eoi(apic, vector);
>  	kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
>  	return vector;
> @@ -1881,6 +1881,12 @@ int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu)
>  	apic_set_isr(vector, apic);
>  	apic_update_ppr(apic);
>  	apic_clear_irr(vector, apic);
> +
> +	if (test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap)) {
> +		apic_clear_isr(vector, apic);
> +		apic_update_ppr(apic);
> +	}
> +
>  	return vector;
>  }
>  
> diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
> index fde8e35d..6c64090 100644
> --- a/arch/x86/kvm/lapic.h
> +++ b/arch/x86/kvm/lapic.h
> @@ -164,6 +164,11 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)
>  	return kvm_vcpu_has_lapic(vcpu) && test_bit(KVM_APIC_INIT, &vcpu->arch.apic->pending_events);
>  }
>  
> +static inline int kvm_apic_id(struct kvm_lapic *apic)
> +{
> +	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
> +}
> +
>  bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
>  
>  void wait_lapic_expire(struct kvm_vcpu *vcpu);
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index fb6cfbf..b853b2df 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -959,6 +959,7 @@ static u32 emulated_msrs[] = {
>  	HV_X64_MSR_RESET,
>  	HV_X64_MSR_VP_INDEX,
>  	HV_X64_MSR_VP_RUNTIME,
> +	HV_X64_MSR_SCONTROL,
>  	HV_X64_MSR_APIC_ASSIST_PAGE, MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
>  	MSR_KVM_PV_EOI_EN,
>  
> @@ -2441,6 +2442,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>  	case KVM_CAP_HYPERV:
>  	case KVM_CAP_HYPERV_VAPIC:
>  	case KVM_CAP_HYPERV_SPIN:
> +	case KVM_CAP_HYPERV_SYNIC:
>  	case KVM_CAP_PCI_SEGMENT:
>  	case KVM_CAP_DEBUGREGS:
>  	case KVM_CAP_X86_ROBUST_SINGLESTEP:
> @@ -6195,6 +6197,8 @@ static void process_smi(struct kvm_vcpu *vcpu)
>  
>  static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
>  {
> +	u64 eoi_exit_bitmap[4];
> +
>  	if (!kvm_apic_hw_enabled(vcpu->arch.apic))
>  		return;
>  
> @@ -6205,8 +6209,10 @@ static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
>  	else
>  		kvm_ioapic_scan_entry(vcpu, vcpu->arch.ioapic_handled_vectors);
>  
> -	kvm_x86_ops->load_eoi_exitmap(vcpu,
> -				      (u64 *)vcpu->arch.ioapic_handled_vectors);
> +	bitmap_or((ulong *)eoi_exit_bitmap, vcpu->arch.ioapic_handled_vectors,
> +		  vcpu_to_synic(vcpu)->vec_bitmap, 256);
> +
> +	kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap);
>  }
>  
>  static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
> @@ -7456,6 +7462,8 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
>  
>  	vcpu->arch.pending_external_vector = -1;
>  
> +	kvm_hv_vcpu_init(vcpu);
> +
>  	return 0;
>  
>  fail_free_mce_banks:
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index c742e79..43b0141 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -318,6 +318,11 @@ struct kvm_s390_adapter_int {
>  	u32 adapter_id;
>  };
>  
> +struct kvm_hv_sint {
> +	u32 vcpu;
> +	u32 sint;
> +};
> +
>  struct kvm_kernel_irq_routing_entry {
>  	u32 gsi;
>  	u32 type;
> @@ -331,6 +336,7 @@ struct kvm_kernel_irq_routing_entry {
>  		} irqchip;
>  		struct msi_msg msi;
>  		struct kvm_s390_adapter_int adapter;
> +		struct kvm_hv_sint hv_sint;
>  	};
>  	struct hlist_node link;
>  };
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 03f3618..27ce460 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -831,6 +831,7 @@ struct kvm_ppc_smmu_info {
>  #define KVM_CAP_GUEST_DEBUG_HW_WPS 120
>  #define KVM_CAP_SPLIT_IRQCHIP 121
>  #define KVM_CAP_IOEVENTFD_ANY_LENGTH 122
> +#define KVM_CAP_HYPERV_SYNIC 123
>  
>  #ifdef KVM_CAP_IRQ_ROUTING
>  
> @@ -854,10 +855,16 @@ struct kvm_irq_routing_s390_adapter {
>  	__u32 adapter_id;
>  };
>  
> +struct kvm_irq_routing_hv_sint {
> +	__u32 vcpu;
> +	__u32 sint;
> +};
> +
>  /* gsi routing entry types */
>  #define KVM_IRQ_ROUTING_IRQCHIP 1
>  #define KVM_IRQ_ROUTING_MSI 2
>  #define KVM_IRQ_ROUTING_S390_ADAPTER 3
> +#define KVM_IRQ_ROUTING_HV_SINT 4
>  
>  struct kvm_irq_routing_entry {
>  	__u32 gsi;
> @@ -868,6 +875,7 @@ struct kvm_irq_routing_entry {
>  		struct kvm_irq_routing_irqchip irqchip;
>  		struct kvm_irq_routing_msi msi;
>  		struct kvm_irq_routing_s390_adapter adapter;
> +		struct kvm_irq_routing_hv_sint hv_sint;
>  		__u32 pad[8];
>  	} u;
>  };
> 

Re: [Qemu-devel] [PATCH v3 8/9] kvm/x86: Hyper-V synthetic interrupt controller

[Roman Kagan]

On Wed, Oct 28, 2015 at 06:41:45PM +0100, Paolo Bonzini wrote:
> Hi Andrey,
> 
> just one question.  Is kvm_arch_set_irq actually needed?  I think
> everything should work fine without it.  Can you check?  If so, I can
> remove it myself and revert the patch that introduced the hook.

While Andrey is testing it, I'd like to ask similar question re. MSI:
why is there a "shortcut" for KVM_IRQ_ROUTING_MSI case (which we
basically modelled after) when it would probably get handled through
->set handler in irqfd_inject() too?

Roman.

Re: [Qemu-devel] [PATCH v3 8/9] kvm/x86: Hyper-V synthetic interrupt controller

[Paolo Bonzini]

On 29/10/2015 09:45, Roman Kagan wrote:
> While Andrey is testing it, I'd like to ask similar question re. MSI:
> why is there a "shortcut" for KVM_IRQ_ROUTING_MSI case (which we
> basically modelled after) when it would probably get handled through
> ->set handler in irqfd_inject() too?

Because it's a bit faster that way. :)  By avoiding the schedule_work,
you can improve latency by a few microseconds.  It's nice to have it for
the VFIO case especially, where everything you do takes you further from
hardware performance.

However, that shortcut is badly implemented because it lets you do a
walk over all CPUs while interrupts are disabled.  It should be modified
to use kvm_set_msi_inatomic instead of kvm_set_msi (more precisely, I
would like to remove kvm_set_msi and keep kvm_arch_irq_update; then
kvm_arch_irq_update will call kvm_set_msi_inatomic).

I'll post a patch next Monday.  You can then benchmark the addition of
synthetic interrupts to the atomic-context fast path, and see if it
makes a speed difference.

Paolo

Re: [Qemu-devel] [PATCH v3 8/9] kvm/x86: Hyper-V synthetic interrupt controller

[Andrey Smetanin]

On 10/28/2015 08:41 PM, Paolo Bonzini wrote:
> Hi Andrey,
>
> just one question.  Is kvm_arch_set_irq actually needed?  I think
> everything should work fine without it.  Can you check?  If so, I can
> remove it myself and revert the patch that introduced the hook.
>
Hi Paolo,

I have checked that Hyper-V SynIC unit test and some hand-made tests 
with Windows guest(with enabled SynIC) works fine without 
kvm_arch_set_irq. It will be nice
to remove this function.

Thanks
> Paolo
>
> On 22/10/2015 18:09, Andrey Smetanin wrote:
>> SynIC (synthetic interrupt controller) is a lapic extension,
>> which is controlled via MSRs and maintains for each vCPU
>>   - 16 synthetic interrupt "lines" (SINT's); each can be configured to
>>     trigger a specific interrupt vector optionally with auto-EOI
>>     semantics
>>   - a message page in the guest memory with 16 256-byte per-SINT message
>>     slots
>>   - an event flag page in the guest memory with 16 2048-bit per-SINT
>>     event flag areas
>>
>> The host triggers a SINT whenever it delivers a new message to the
>> corresponding slot or flips an event flag bit in the corresponding area.
>> The guest informs the host that it can try delivering a message by
>> explicitly asserting EOI in lapic or writing to End-Of-Message (EOM)
>> MSR.
>>
>> The userspace (qemu) triggers interrupts and receives EOM notifications
>> via irqfd with resampler; for that, a GSI is allocated for each
>> configured SINT, and irq_routing api is extended to support GSI-SINT
>> mapping.
>>
>> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
>> Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
>> Signed-off-by: Denis V. Lunev <den@openvz.org>
>> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
>> CC: "K. Y. Srinivasan" <kys@microsoft.com>
>> CC: Gleb Natapov <gleb@kernel.org>
>> CC: Paolo Bonzini <pbonzini@redhat.com>
>> CC: Roman Kagan <rkagan@virtuozzo.com>
>>
>> Changes v3:
>> * added KVM_CAP_HYPERV_SYNIC and KVM_IRQ_ROUTING_HV_SINT notes into
>> docs
>>
>> Changes v2:
>> * do not use posted interrupts for Hyper-V SynIC AutoEOI vectors
>> * add Hyper-V SynIC vectors into EOI exit bitmap
>> * Hyper-V SyniIC SINT msr write logic simplified
>> ---
>>   Documentation/virtual/kvm/api.txt |  14 ++
>>   arch/x86/include/asm/kvm_host.h   |  14 ++
>>   arch/x86/kvm/hyperv.c             | 297 ++++++++++++++++++++++++++++++++++++++
>>   arch/x86/kvm/hyperv.h             |  21 +++
>>   arch/x86/kvm/irq_comm.c           |  34 +++++
>>   arch/x86/kvm/lapic.c              |  18 ++-
>>   arch/x86/kvm/lapic.h              |   5 +
>>   arch/x86/kvm/x86.c                |  12 +-
>>   include/linux/kvm_host.h          |   6 +
>>   include/uapi/linux/kvm.h          |   8 +
>>   10 files changed, 421 insertions(+), 8 deletions(-)
>>
>> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
>> index 092ee9f..8710418 100644
>> --- a/Documentation/virtual/kvm/api.txt
>> +++ b/Documentation/virtual/kvm/api.txt
>> @@ -1451,6 +1451,7 @@ struct kvm_irq_routing_entry {
>>   		struct kvm_irq_routing_irqchip irqchip;
>>   		struct kvm_irq_routing_msi msi;
>>   		struct kvm_irq_routing_s390_adapter adapter;
>> +		struct kvm_irq_routing_hv_sint hv_sint;
>>   		__u32 pad[8];
>>   	} u;
>>   };
>> @@ -1459,6 +1460,7 @@ struct kvm_irq_routing_entry {
>>   #define KVM_IRQ_ROUTING_IRQCHIP 1
>>   #define KVM_IRQ_ROUTING_MSI 2
>>   #define KVM_IRQ_ROUTING_S390_ADAPTER 3
>> +#define KVM_IRQ_ROUTING_HV_SINT 4
>>
>>   No flags are specified so far, the corresponding field must be set to zero.
>>
>> @@ -1482,6 +1484,10 @@ struct kvm_irq_routing_s390_adapter {
>>   	__u32 adapter_id;
>>   };
>>
>> +struct kvm_irq_routing_hv_sint {
>> +	__u32 vcpu;
>> +	__u32 sint;
>> +};
>>
>>   4.53 KVM_ASSIGN_SET_MSIX_NR (deprecated)
>>
>> @@ -3685,3 +3691,11 @@ available, means that that the kernel has an implementation of the
>>   H_RANDOM hypercall backed by a hardware random-number generator.
>>   If present, the kernel H_RANDOM handler can be enabled for guest use
>>   with the KVM_CAP_PPC_ENABLE_HCALL capability.
>> +
>> +8.2 KVM_CAP_HYPERV_SYNIC
>> +
>> +Architectures: x86
>> +This capability, if KVM_CHECK_EXTENSION indicates that it is
>> +available, means that that the kernel has an implementation of the
>> +Hyper-V Synthetic interrupt controller(SynIC). SynIC is used to
>> +support Windows Hyper-V based guest paravirt drivers(VMBus).
>> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>> index 3c6327d..8434f88 100644
>> --- a/arch/x86/include/asm/kvm_host.h
>> +++ b/arch/x86/include/asm/kvm_host.h
>> @@ -25,6 +25,7 @@
>>   #include <linux/pvclock_gtod.h>
>>   #include <linux/clocksource.h>
>>   #include <linux/irqbypass.h>
>> +#include <linux/hyperv.h>
>>
>>   #include <asm/pvclock-abi.h>
>>   #include <asm/desc.h>
>> @@ -374,10 +375,23 @@ struct kvm_mtrr {
>>   	struct list_head head;
>>   };
>>
>> +/* Hyper-V synthetic interrupt controller (SynIC)*/
>> +struct kvm_vcpu_hv_synic {
>> +	u64 version;
>> +	u64 control;
>> +	u64 msg_page;
>> +	u64 evt_page;
>> +	atomic64_t sint[HV_SYNIC_SINT_COUNT];
>> +	atomic_t sint_to_gsi[HV_SYNIC_SINT_COUNT];
>> +	DECLARE_BITMAP(auto_eoi_bitmap, 256);
>> +	DECLARE_BITMAP(vec_bitmap, 256);
>> +};
>> +
>>   /* Hyper-V per vcpu emulation context */
>>   struct kvm_vcpu_hv {
>>   	u64 hv_vapic;
>>   	s64 runtime_offset;
>> +	struct kvm_vcpu_hv_synic synic;
>>   };
>>
>>   struct kvm_vcpu_arch {
>> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
>> index 62cf8c9..8ff71f3 100644
>> --- a/arch/x86/kvm/hyperv.c
>> +++ b/arch/x86/kvm/hyperv.c
>> @@ -23,13 +23,296 @@
>>
>>   #include "x86.h"
>>   #include "lapic.h"
>> +#include "ioapic.h"
>>   #include "hyperv.h"
>>
>>   #include <linux/kvm_host.h>
>> +#include <asm/apicdef.h>
>>   #include <trace/events/kvm.h>
>>
>>   #include "trace.h"
>>
>> +static inline u64 synic_read_sint(struct kvm_vcpu_hv_synic *synic, int sint)
>> +{
>> +	return atomic64_read(&synic->sint[sint]);
>> +}
>> +
>> +static inline int synic_get_sint_vector(u64 sint_value)
>> +{
>> +	if (sint_value & HV_SYNIC_SINT_MASKED)
>> +		return -1;
>> +	return sint_value & HV_SYNIC_SINT_VECTOR_MASK;
>> +}
>> +
>> +static bool synic_has_vector_connected(struct kvm_vcpu_hv_synic *synic,
>> +				      int vector)
>> +{
>> +	int i;
>> +
>> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
>> +		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
>> +			return true;
>> +	}
>> +	return false;
>> +}
>> +
>> +static bool synic_has_vector_auto_eoi(struct kvm_vcpu_hv_synic *synic,
>> +				     int vector)
>> +{
>> +	int i;
>> +	u64 sint_value;
>> +
>> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
>> +		sint_value = synic_read_sint(synic, i);
>> +		if (synic_get_sint_vector(sint_value) == vector &&
>> +		    sint_value & HV_SYNIC_SINT_AUTO_EOI)
>> +			return true;
>> +	}
>> +	return false;
>> +}
>> +
>> +static int synic_set_sint(struct kvm_vcpu_hv_synic *synic, int sint, u64 data)
>> +{
>> +	int vector;
>> +
>> +	vector = data & HV_SYNIC_SINT_VECTOR_MASK;
>> +	if (vector < 16)
>> +		return 1;
>> +	/*
>> +	 * Guest may configure multiple SINTs to use the same vector, so
>> +	 * we maintain a bitmap of vectors handled by synic, and a
>> +	 * bitmap of vectors with auto-eoi behavior.  The bitmaps are
>> +	 * updated here, and atomically queried on fast paths.
>> +	 */
>> +
>> +	atomic64_set(&synic->sint[sint], data);
>> +
>> +	if (synic_has_vector_connected(synic, vector))
>> +		__set_bit(vector, synic->vec_bitmap);
>> +	else
>> +		__clear_bit(vector, synic->vec_bitmap);
>> +
>> +	if (synic_has_vector_auto_eoi(synic, vector))
>> +		__set_bit(vector, synic->auto_eoi_bitmap);
>> +	else
>> +		__clear_bit(vector, synic->auto_eoi_bitmap);
>> +
>> +	/* Load SynIC vectors into EOI exit bitmap */
>> +	kvm_make_request(KVM_REQ_SCAN_IOAPIC, synic_to_vcpu(synic));
>> +	return 0;
>> +}
>> +
>> +static struct kvm_vcpu_hv_synic *synic_get(struct kvm *kvm, u32 vcpu_id)
>> +{
>> +	struct kvm_vcpu *vcpu;
>> +
>> +	if (vcpu_id >= atomic_read(&kvm->online_vcpus))
>> +		return NULL;
>> +	vcpu = kvm_get_vcpu(kvm, vcpu_id);
>> +	if (!vcpu)
>> +		return NULL;
>> +
>> +	return vcpu_to_synic(vcpu);
>> +}
>> +
>> +static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
>> +{
>> +	struct kvm *kvm = vcpu->kvm;
>> +	int gsi, idx;
>> +
>> +	vcpu_debug(vcpu, "Hyper-V SynIC acked sint %d\n", sint);
>> +
>> +	idx = srcu_read_lock(&kvm->irq_srcu);
>> +	gsi = atomic_read(&vcpu_to_synic(vcpu)->sint_to_gsi[sint]);
>> +	if (gsi != -1)
>> +		kvm_notify_acked_gsi(kvm, gsi);
>> +	srcu_read_unlock(&kvm->irq_srcu, idx);
>> +}
>> +
>> +static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>> +			 u32 msr, u64 data, bool host)
>> +{
>> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
>> +	int ret;
>> +
>> +	vcpu_debug(vcpu, "Hyper-V SynIC set msr 0x%x 0x%llx host %d\n",
>> +		   msr, data, host);
>> +	ret = 0;
>> +	switch (msr) {
>> +	case HV_X64_MSR_SCONTROL:
>> +		synic->control = data;
>> +		break;
>> +	case HV_X64_MSR_SVERSION:
>> +		if (!host) {
>> +			ret = 1;
>> +			break;
>> +		}
>> +		synic->version = data;
>> +		break;
>> +	case HV_X64_MSR_SIEFP:
>> +		if (data & HV_SYNIC_SIEFP_ENABLE)
>> +			if (kvm_clear_guest(vcpu->kvm,
>> +					    data & PAGE_MASK, PAGE_SIZE)) {
>> +				ret = 1;
>> +				break;
>> +			}
>> +		synic->evt_page = data;
>> +		break;
>> +	case HV_X64_MSR_SIMP:
>> +		if (data & HV_SYNIC_SIMP_ENABLE)
>> +			if (kvm_clear_guest(vcpu->kvm,
>> +					    data & PAGE_MASK, PAGE_SIZE)) {
>> +				ret = 1;
>> +				break;
>> +			}
>> +		synic->msg_page = data;
>> +		break;
>> +	case HV_X64_MSR_EOM: {
>> +		int i;
>> +
>> +		for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
>> +			kvm_hv_notify_acked_sint(vcpu, i);
>> +		break;
>> +	}
>> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
>> +		ret = synic_set_sint(synic, msr - HV_X64_MSR_SINT0, data);
>> +		break;
>> +	default:
>> +		ret = 1;
>> +		break;
>> +	}
>> +	return ret;
>> +}
>> +
>> +static int synic_get_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 *pdata)
>> +{
>> +	int ret;
>> +
>> +	ret = 0;
>> +	switch (msr) {
>> +	case HV_X64_MSR_SCONTROL:
>> +		*pdata = synic->control;
>> +		break;
>> +	case HV_X64_MSR_SVERSION:
>> +		*pdata = synic->version;
>> +		break;
>> +	case HV_X64_MSR_SIEFP:
>> +		*pdata = synic->evt_page;
>> +		break;
>> +	case HV_X64_MSR_SIMP:
>> +		*pdata = synic->msg_page;
>> +		break;
>> +	case HV_X64_MSR_EOM:
>> +		*pdata = 0;
>> +		break;
>> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
>> +		*pdata = atomic64_read(&synic->sint[msr - HV_X64_MSR_SINT0]);
>> +		break;
>> +	default:
>> +		ret = 1;
>> +		break;
>> +	}
>> +	return ret;
>> +}
>> +
>> +int synic_set_irq(struct kvm_vcpu_hv_synic *synic, u32 sint)
>> +{
>> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
>> +	struct kvm_lapic_irq irq;
>> +	int ret, vector;
>> +
>> +	if (sint >= ARRAY_SIZE(synic->sint))
>> +		return -EINVAL;
>> +
>> +	vector = synic_get_sint_vector(synic_read_sint(synic, sint));
>> +	if (vector < 0)
>> +		return -ENOENT;
>> +
>> +	memset(&irq, 0, sizeof(irq));
>> +	irq.dest_id = kvm_apic_id(vcpu->arch.apic);
>> +	irq.dest_mode = APIC_DEST_PHYSICAL;
>> +	irq.delivery_mode = APIC_DM_FIXED;
>> +	irq.vector = vector;
>> +	irq.level = 1;
>> +
>> +	ret = kvm_irq_delivery_to_apic(vcpu->kvm, NULL, &irq, NULL);
>> +	vcpu_debug(vcpu, "Hyper-V SynIC set irq ret %d\n", ret);
>> +	return ret;
>> +}
>> +
>> +int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint)
>> +{
>> +	struct kvm_vcpu_hv_synic *synic;
>> +
>> +	synic = synic_get(kvm, vcpu_id);
>> +	if (!synic)
>> +		return -EINVAL;
>> +
>> +	return synic_set_irq(synic, sint);
>> +}
>> +
>> +void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector)
>> +{
>> +	struct kvm_vcpu_hv_synic *synic = vcpu_to_synic(vcpu);
>> +	int i;
>> +
>> +	vcpu_debug(vcpu, "Hyper-V SynIC send eoi vec %d\n", vector);
>> +
>> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++)
>> +		if (synic_get_sint_vector(synic_read_sint(synic, i)) == vector)
>> +			kvm_hv_notify_acked_sint(vcpu, i);
>> +}
>> +
>> +static int kvm_hv_set_sint_gsi(struct kvm *kvm, u32 vcpu_id, u32 sint, int gsi)
>> +{
>> +	struct kvm_vcpu_hv_synic *synic;
>> +
>> +	synic = synic_get(kvm, vcpu_id);
>> +	if (!synic)
>> +		return -EINVAL;
>> +
>> +	if (sint >= ARRAY_SIZE(synic->sint_to_gsi))
>> +		return -EINVAL;
>> +
>> +	atomic_set(&synic->sint_to_gsi[sint], gsi);
>> +	return 0;
>> +}
>> +
>> +void kvm_hv_irq_routing_update(struct kvm *kvm)
>> +{
>> +	struct kvm_irq_routing_table *irq_rt;
>> +	struct kvm_kernel_irq_routing_entry *e;
>> +	u32 gsi;
>> +
>> +	irq_rt = srcu_dereference_check(kvm->irq_routing, &kvm->irq_srcu,
>> +					lockdep_is_held(&kvm->irq_lock));
>> +
>> +	for (gsi = 0; gsi < irq_rt->nr_rt_entries; gsi++) {
>> +		hlist_for_each_entry(e, &irq_rt->map[gsi], link) {
>> +			if (e->type == KVM_IRQ_ROUTING_HV_SINT)
>> +				kvm_hv_set_sint_gsi(kvm, e->hv_sint.vcpu,
>> +						    e->hv_sint.sint, gsi);
>> +		}
>> +	}
>> +}
>> +
>> +static void synic_init(struct kvm_vcpu_hv_synic *synic)
>> +{
>> +	int i;
>> +
>> +	memset(synic, 0, sizeof(*synic));
>> +	synic->version = HV_SYNIC_VERSION_1;
>> +	for (i = 0; i < ARRAY_SIZE(synic->sint); i++) {
>> +		atomic64_set(&synic->sint[i], HV_SYNIC_SINT_MASKED);
>> +		atomic_set(&synic->sint_to_gsi[i], -1);
>> +	}
>> +}
>> +
>> +void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu)
>> +{
>> +	synic_init(vcpu_to_synic(vcpu));
>> +}
>> +
>>   static bool kvm_hv_msr_partition_wide(u32 msr)
>>   {
>>   	bool r = false;
>> @@ -226,6 +509,13 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)
>>   			return 1;
>>   		hv->runtime_offset = data - current_task_runtime_100ns();
>>   		break;
>> +	case HV_X64_MSR_SCONTROL:
>> +	case HV_X64_MSR_SVERSION:
>> +	case HV_X64_MSR_SIEFP:
>> +	case HV_X64_MSR_SIMP:
>> +	case HV_X64_MSR_EOM:
>> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
>> +		return synic_set_msr(vcpu_to_synic(vcpu), msr, data, host);
>>   	default:
>>   		vcpu_unimpl(vcpu, "Hyper-V uhandled wrmsr: 0x%x data 0x%llx\n",
>>   			    msr, data);
>> @@ -304,6 +594,13 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
>>   	case HV_X64_MSR_VP_RUNTIME:
>>   		data = current_task_runtime_100ns() + hv->runtime_offset;
>>   		break;
>> +	case HV_X64_MSR_SCONTROL:
>> +	case HV_X64_MSR_SVERSION:
>> +	case HV_X64_MSR_SIEFP:
>> +	case HV_X64_MSR_SIMP:
>> +	case HV_X64_MSR_EOM:
>> +	case HV_X64_MSR_SINT0 ... HV_X64_MSR_SINT15:
>> +		return synic_get_msr(vcpu_to_synic(vcpu), msr, pdata);
>>   	default:
>>   		vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
>>   		return 1;
>> diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
>> index c7bce55..8668612 100644
>> --- a/arch/x86/kvm/hyperv.h
>> +++ b/arch/x86/kvm/hyperv.h
>> @@ -29,4 +29,25 @@ int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);
>>   bool kvm_hv_hypercall_enabled(struct kvm *kvm);
>>   int kvm_hv_hypercall(struct kvm_vcpu *vcpu);
>>
>> +int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
>> +void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
>> +
>> +static inline struct kvm_vcpu_hv_synic *vcpu_to_synic(struct kvm_vcpu *vcpu)
>> +{
>> +	return &vcpu->arch.hyperv.synic;
>> +}
>> +
>> +static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
>> +{
>> +	struct kvm_vcpu_hv *hv;
>> +	struct kvm_vcpu_arch *arch;
>> +
>> +	hv = container_of(synic, struct kvm_vcpu_hv, synic);
>> +	arch = container_of(hv, struct kvm_vcpu_arch, hyperv);
>> +	return container_of(arch, struct kvm_vcpu, arch);
>> +}
>> +void kvm_hv_irq_routing_update(struct kvm *kvm);
>> +
>> +void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
>> +
>>   #endif
>> diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
>> index fe91f72..5e195b9 100644
>> --- a/arch/x86/kvm/irq_comm.c
>> +++ b/arch/x86/kvm/irq_comm.c
>> @@ -33,6 +33,8 @@
>>
>>   #include "lapic.h"
>>
>> +#include "hyperv.h"
>> +
>>   static int kvm_set_pic_irq(struct kvm_kernel_irq_routing_entry *e,
>>   			   struct kvm *kvm, int irq_source_id, int level,
>>   			   bool line_status)
>> @@ -251,6 +253,16 @@ void kvm_fire_mask_notifiers(struct kvm *kvm, unsigned irqchip, unsigned pin,
>>   	srcu_read_unlock(&kvm->irq_srcu, idx);
>>   }
>>
>> +static int kvm_hv_set_sint(struct kvm_kernel_irq_routing_entry *e,
>> +		    struct kvm *kvm, int irq_source_id, int level,
>> +		    bool line_status)
>> +{
>> +	if (!level)
>> +		return -1;
>> +
>> +	return kvm_hv_synic_set_irq(kvm, e->hv_sint.vcpu, e->hv_sint.sint);
>> +}
>> +
>>   int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
>>   			  const struct kvm_irq_routing_entry *ue)
>>   {
>> @@ -289,6 +301,11 @@ int kvm_set_routing_entry(struct kvm_kernel_irq_routing_entry *e,
>>   		e->msi.address_hi = ue->u.msi.address_hi;
>>   		e->msi.data = ue->u.msi.data;
>>   		break;
>> +	case KVM_IRQ_ROUTING_HV_SINT:
>> +		e->set = kvm_hv_set_sint;
>> +		e->hv_sint.vcpu = ue->u.hv_sint.vcpu;
>> +		e->hv_sint.sint = ue->u.hv_sint.sint;
>> +		break;
>>   	default:
>>   		goto out;
>>   	}
>> @@ -405,3 +422,20 @@ void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
>>   	}
>>   	srcu_read_unlock(&kvm->irq_srcu, idx);
>>   }
>> +
>> +int kvm_arch_set_irq(struct kvm_kernel_irq_routing_entry *irq, struct kvm *kvm,
>> +		     int irq_source_id, int level, bool line_status)
>> +{
>> +	switch (irq->type) {
>> +	case KVM_IRQ_ROUTING_HV_SINT:
>> +		return kvm_hv_set_sint(irq, kvm, irq_source_id, level,
>> +				       line_status);
>> +	default:
>> +		return -EWOULDBLOCK;
>> +	}
>> +}
>> +
>> +void kvm_arch_irq_routing_update(struct kvm *kvm)
>> +{
>> +	kvm_hv_irq_routing_update(kvm);
>> +}
>> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
>> index dc03a01..3132478 100644
>> --- a/arch/x86/kvm/lapic.c
>> +++ b/arch/x86/kvm/lapic.c
>> @@ -41,6 +41,7 @@
>>   #include "trace.h"
>>   #include "x86.h"
>>   #include "cpuid.h"
>> +#include "hyperv.h"
>>
>>   #ifndef CONFIG_X86_64
>>   #define mod_64(x, y) ((x) - (y) * div64_u64(x, y))
>> @@ -128,11 +129,6 @@ static inline int apic_enabled(struct kvm_lapic *apic)
>>   	(LVT_MASK | APIC_MODE_MASK | APIC_INPUT_POLARITY | \
>>   	 APIC_LVT_REMOTE_IRR | APIC_LVT_LEVEL_TRIGGER)
>>
>> -static inline int kvm_apic_id(struct kvm_lapic *apic)
>> -{
>> -	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
>> -}
>> -
>>   /* The logical map is definitely wrong if we have multiple
>>    * modes at the same time.  (Physical map is always right.)
>>    */
>> @@ -850,7 +846,8 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
>>   				apic_clear_vector(vector, apic->regs + APIC_TMR);
>>   		}
>>
>> -		if (kvm_x86_ops->deliver_posted_interrupt)
>> +		if (kvm_x86_ops->deliver_posted_interrupt &&
>> +		    !test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap))
>>   			kvm_x86_ops->deliver_posted_interrupt(vcpu, vector);
>>   		else {
>>   			apic_set_irr(vector, apic);
>> @@ -972,6 +969,9 @@ static int apic_set_eoi(struct kvm_lapic *apic)
>>   	apic_clear_isr(vector, apic);
>>   	apic_update_ppr(apic);
>>
>> +	if (test_bit(vector, vcpu_to_synic(apic->vcpu)->vec_bitmap))
>> +		kvm_hv_synic_send_eoi(apic->vcpu, vector);
>> +
>>   	kvm_ioapic_send_eoi(apic, vector);
>>   	kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
>>   	return vector;
>> @@ -1881,6 +1881,12 @@ int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu)
>>   	apic_set_isr(vector, apic);
>>   	apic_update_ppr(apic);
>>   	apic_clear_irr(vector, apic);
>> +
>> +	if (test_bit(vector, vcpu_to_synic(vcpu)->auto_eoi_bitmap)) {
>> +		apic_clear_isr(vector, apic);
>> +		apic_update_ppr(apic);
>> +	}
>> +
>>   	return vector;
>>   }
>>
>> diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
>> index fde8e35d..6c64090 100644
>> --- a/arch/x86/kvm/lapic.h
>> +++ b/arch/x86/kvm/lapic.h
>> @@ -164,6 +164,11 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)
>>   	return kvm_vcpu_has_lapic(vcpu) && test_bit(KVM_APIC_INIT, &vcpu->arch.apic->pending_events);
>>   }
>>
>> +static inline int kvm_apic_id(struct kvm_lapic *apic)
>> +{
>> +	return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
>> +}
>> +
>>   bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
>>
>>   void wait_lapic_expire(struct kvm_vcpu *vcpu);
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index fb6cfbf..b853b2df 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -959,6 +959,7 @@ static u32 emulated_msrs[] = {
>>   	HV_X64_MSR_RESET,
>>   	HV_X64_MSR_VP_INDEX,
>>   	HV_X64_MSR_VP_RUNTIME,
>> +	HV_X64_MSR_SCONTROL,
>>   	HV_X64_MSR_APIC_ASSIST_PAGE, MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
>>   	MSR_KVM_PV_EOI_EN,
>>
>> @@ -2441,6 +2442,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>>   	case KVM_CAP_HYPERV:
>>   	case KVM_CAP_HYPERV_VAPIC:
>>   	case KVM_CAP_HYPERV_SPIN:
>> +	case KVM_CAP_HYPERV_SYNIC:
>>   	case KVM_CAP_PCI_SEGMENT:
>>   	case KVM_CAP_DEBUGREGS:
>>   	case KVM_CAP_X86_ROBUST_SINGLESTEP:
>> @@ -6195,6 +6197,8 @@ static void process_smi(struct kvm_vcpu *vcpu)
>>
>>   static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
>>   {
>> +	u64 eoi_exit_bitmap[4];
>> +
>>   	if (!kvm_apic_hw_enabled(vcpu->arch.apic))
>>   		return;
>>
>> @@ -6205,8 +6209,10 @@ static void vcpu_scan_ioapic(struct kvm_vcpu *vcpu)
>>   	else
>>   		kvm_ioapic_scan_entry(vcpu, vcpu->arch.ioapic_handled_vectors);
>>
>> -	kvm_x86_ops->load_eoi_exitmap(vcpu,
>> -				      (u64 *)vcpu->arch.ioapic_handled_vectors);
>> +	bitmap_or((ulong *)eoi_exit_bitmap, vcpu->arch.ioapic_handled_vectors,
>> +		  vcpu_to_synic(vcpu)->vec_bitmap, 256);
>> +
>> +	kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap);
>>   }
>>
>>   static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
>> @@ -7456,6 +7462,8 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
>>
>>   	vcpu->arch.pending_external_vector = -1;
>>
>> +	kvm_hv_vcpu_init(vcpu);
>> +
>>   	return 0;
>>
>>   fail_free_mce_banks:
>> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
>> index c742e79..43b0141 100644
>> --- a/include/linux/kvm_host.h
>> +++ b/include/linux/kvm_host.h
>> @@ -318,6 +318,11 @@ struct kvm_s390_adapter_int {
>>   	u32 adapter_id;
>>   };
>>
>> +struct kvm_hv_sint {
>> +	u32 vcpu;
>> +	u32 sint;
>> +};
>> +
>>   struct kvm_kernel_irq_routing_entry {
>>   	u32 gsi;
>>   	u32 type;
>> @@ -331,6 +336,7 @@ struct kvm_kernel_irq_routing_entry {
>>   		} irqchip;
>>   		struct msi_msg msi;
>>   		struct kvm_s390_adapter_int adapter;
>> +		struct kvm_hv_sint hv_sint;
>>   	};
>>   	struct hlist_node link;
>>   };
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index 03f3618..27ce460 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -831,6 +831,7 @@ struct kvm_ppc_smmu_info {
>>   #define KVM_CAP_GUEST_DEBUG_HW_WPS 120
>>   #define KVM_CAP_SPLIT_IRQCHIP 121
>>   #define KVM_CAP_IOEVENTFD_ANY_LENGTH 122
>> +#define KVM_CAP_HYPERV_SYNIC 123
>>
>>   #ifdef KVM_CAP_IRQ_ROUTING
>>
>> @@ -854,10 +855,16 @@ struct kvm_irq_routing_s390_adapter {
>>   	__u32 adapter_id;
>>   };
>>
>> +struct kvm_irq_routing_hv_sint {
>> +	__u32 vcpu;
>> +	__u32 sint;
>> +};
>> +
>>   /* gsi routing entry types */
>>   #define KVM_IRQ_ROUTING_IRQCHIP 1
>>   #define KVM_IRQ_ROUTING_MSI 2
>>   #define KVM_IRQ_ROUTING_S390_ADAPTER 3
>> +#define KVM_IRQ_ROUTING_HV_SINT 4
>>
>>   struct kvm_irq_routing_entry {
>>   	__u32 gsi;
>> @@ -868,6 +875,7 @@ struct kvm_irq_routing_entry {
>>   		struct kvm_irq_routing_irqchip irqchip;
>>   		struct kvm_irq_routing_msi msi;
>>   		struct kvm_irq_routing_s390_adapter adapter;
>> +		struct kvm_irq_routing_hv_sint hv_sint;
>>   		__u32 pad[8];
>>   	} u;
>>   };
>>

[Qemu-devel] [PATCH 9/9] kvm/x86: Hyper-V kvm exit

[Denis V. Lunev]

From: Andrey Smetanin <asmetanin@virtuozzo.com>

A new vcpu exit is introduced to notify the userspace of the
changes in Hyper-V SynIC configuration triggered by guest writing to the
corresponding MSRs.

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtiozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
---
 Documentation/virtual/kvm/api.txt |  6 ++++++
 arch/x86/include/asm/kvm_host.h   |  1 +
 arch/x86/kvm/hyperv.c             | 17 +++++++++++++++++
 arch/x86/kvm/x86.c                |  6 ++++++
 include/linux/kvm_host.h          |  1 +
 include/uapi/linux/kvm.h          | 17 +++++++++++++++++
 6 files changed, 48 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 092ee9f..86cae88 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3331,6 +3331,12 @@ the userspace IOAPIC should process the EOI and retrigger the interrupt if
 it is still asserted.  Vector is the LAPIC interrupt vector for which the
 EOI was received.
 
+		/* KVM_EXIT_HYPERV */
+                struct kvm_hyperv_exit hyperv;
+Indicates that the VCPU exits into userspace to process some tasks
+related with Hyper-V emulation. Currently used to synchronize modified
+Hyper-V SynIC state with userspace.
+
 		/* Fix the size of the union. */
 		char padding[256];
 	};
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index dfdaf0f..a41d7ed 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -392,6 +392,7 @@ struct kvm_vcpu_hv {
 	u64 hv_vapic;
 	s64 runtime_offset;
 	struct kvm_vcpu_hv_synic synic;
+	struct kvm_hyperv_exit exit;
 };
 
 struct kvm_vcpu_arch {
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 8ff71f3..9443920 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -129,6 +129,20 @@ static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
 
+static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
+
+	hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNIC;
+	hv_vcpu->exit.u.synic.msr = msr;
+	hv_vcpu->exit.u.synic.control = synic->control;
+	hv_vcpu->exit.u.synic.evt_page = synic->evt_page;
+	hv_vcpu->exit.u.synic.msg_page = synic->msg_page;
+
+	kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
+}
+
 static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 			 u32 msr, u64 data, bool host)
 {
@@ -141,6 +155,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 	switch (msr) {
 	case HV_X64_MSR_SCONTROL:
 		synic->control = data;
+		synic_exit(synic, msr);
 		break;
 	case HV_X64_MSR_SVERSION:
 		if (!host) {
@@ -157,6 +172,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 				break;
 			}
 		synic->evt_page = data;
+		synic_exit(synic, msr);
 		break;
 	case HV_X64_MSR_SIMP:
 		if (data & HV_SYNIC_SIMP_ENABLE)
@@ -166,6 +182,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 				break;
 			}
 		synic->msg_page = data;
+		synic_exit(synic, msr);
 		break;
 	case HV_X64_MSR_EOM: {
 		int i;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 807d124..9453207 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6342,6 +6342,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 			r = 0;
 			goto out;
 		}
+		if (kvm_check_request(KVM_REQ_HV_EXIT, vcpu)) {
+			vcpu->run->exit_reason = KVM_EXIT_HYPERV;
+			vcpu->run->hyperv = vcpu->arch.hyperv.exit;
+			r = 0;
+			goto out;
+		}
 	}
 
 	/*
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 43b0141..e38ac16 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -143,6 +143,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_HV_CRASH          27
 #define KVM_REQ_IOAPIC_EOI_EXIT   28
 #define KVM_REQ_HV_RESET          29
+#define KVM_REQ_HV_EXIT           30
 
 #define KVM_USERSPACE_IRQ_SOURCE_ID		0
 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 27ce460..6e32f75 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -154,6 +154,20 @@ struct kvm_s390_skeys {
 	__u32 flags;
 	__u32 reserved[9];
 };
+
+struct kvm_hyperv_exit {
+#define KVM_EXIT_HYPERV_SYNIC          1
+	__u32 type;
+	union {
+		struct {
+			__u32 msr;
+			__u64 control;
+			__u64 evt_page;
+			__u64 msg_page;
+		} synic;
+	} u;
+};
+
 #define KVM_S390_GET_SKEYS_NONE   1
 #define KVM_S390_SKEYS_MAX        1048576
 
@@ -184,6 +198,7 @@ struct kvm_s390_skeys {
 #define KVM_EXIT_SYSTEM_EVENT     24
 #define KVM_EXIT_S390_STSI        25
 #define KVM_EXIT_IOAPIC_EOI       26
+#define KVM_EXIT_HYPERV           27
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -338,6 +353,8 @@ struct kvm_run {
 		struct {
 			__u8 vector;
 		} eoi;
+		/* KVM_EXIT_HYPERV */
+		struct kvm_hyperv_exit hyperv;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
-- 
2.1.4

Re: [Qemu-devel] [PATCH 9/9] kvm/x86: Hyper-V kvm exit

[Paolo Bonzini]

On 16/10/2015 09:07, Denis V. Lunev wrote:
>  
> +		/* KVM_EXIT_HYPERV */
> +                struct kvm_hyperv_exit hyperv;
> +Indicates that the VCPU exits into userspace to process some tasks
> +related with Hyper-V emulation. Currently used to synchronize modified
> +Hyper-V SynIC state with userspace.
> +

The documentation should include the definition of the struct and the
definition of the subtypes (currently KVM_EXIT_HYPERV_SYNIC only).

Documentation for KVM_CAP_HYPERV_SINIC and KVM_IRQ_ROUTING_HV_SINT is
missing, too.

Finally, it would be better to have unit tests in kvm-unit-tests.
Either this or QEMU support is a requirement for merging, and the unit
tests are probably easier.

But apart from this, the series looks great and I'm already applying it
to kvm/queue so that it gets some more testing.

Paolo

Re: [Qemu-devel] [PATCH 9/9] kvm/x86: Hyper-V kvm exit

[Roman Kagan]

On Fri, Oct 16, 2015 at 09:51:58AM +0200, Paolo Bonzini wrote:
> The documentation should include the definition of the struct and the
> definition of the subtypes (currently KVM_EXIT_HYPERV_SYNIC only).
> 
> Documentation for KVM_CAP_HYPERV_SINIC and KVM_IRQ_ROUTING_HV_SINT is
> missing, too.
> 
> Finally, it would be better to have unit tests in kvm-unit-tests.
> Either this or QEMU support is a requirement for merging, and the unit
> tests are probably easier.

OK we'll try to get this done early next week.

Thanks,
Roman.

[Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Andrey Smetanin]

A new vcpu exit is introduced to notify the userspace of the
changes in Hyper-V SynIC configuration triggered by guest writing to the
corresponding MSRs.

Changes v3:
* added KVM_EXIT_HYPERV types and structs notes into docs

Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
Reviewed-by: Roman Kagan <rkagan@virtiozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Vitaly Kuznetsov <vkuznets@redhat.com>
CC: "K. Y. Srinivasan" <kys@microsoft.com>
CC: Gleb Natapov <gleb@kernel.org>
CC: Paolo Bonzini <pbonzini@redhat.com>
CC: Roman Kagan <rkagan@virtiozzo.com>

---
 Documentation/virtual/kvm/api.txt | 22 ++++++++++++++++++++++
 arch/x86/include/asm/kvm_host.h   |  1 +
 arch/x86/kvm/hyperv.c             | 17 +++++++++++++++++
 arch/x86/kvm/x86.c                |  6 ++++++
 include/linux/kvm_host.h          |  1 +
 include/uapi/linux/kvm.h          | 17 +++++++++++++++++
 6 files changed, 64 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 8710418..a6858eb 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3337,6 +3337,28 @@ the userspace IOAPIC should process the EOI and retrigger the interrupt if
 it is still asserted.  Vector is the LAPIC interrupt vector for which the
 EOI was received.
 
+		struct kvm_hyperv_exit {
+#define KVM_EXIT_HYPERV_SYNIC          1
+			__u32 type;
+			union {
+				struct {
+					__u32 msr;
+					__u64 control;
+					__u64 evt_page;
+					__u64 msg_page;
+				} synic;
+			} u;
+		};
+		/* KVM_EXIT_HYPERV */
+                struct kvm_hyperv_exit hyperv;
+Indicates that the VCPU exits into userspace to process some tasks
+related to Hyper-V emulation.
+Valid values for 'type' are:
+	KVM_EXIT_HYPERV_SYNIC -- synchronously notify user-space about
+Hyper-V SynIC state change. Notification is used to remap SynIC
+event/message pages and to enable/disable SynIC messages/events processing
+in userspace.
+
 		/* Fix the size of the union. */
 		char padding[256];
 	};
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 8434f88..54c90d3 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -392,6 +392,7 @@ struct kvm_vcpu_hv {
 	u64 hv_vapic;
 	s64 runtime_offset;
 	struct kvm_vcpu_hv_synic synic;
+	struct kvm_hyperv_exit exit;
 };
 
 struct kvm_vcpu_arch {
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 8ff71f3..9443920 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -129,6 +129,20 @@ static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
 	srcu_read_unlock(&kvm->irq_srcu, idx);
 }
 
+static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr)
+{
+	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
+	struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
+
+	hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNIC;
+	hv_vcpu->exit.u.synic.msr = msr;
+	hv_vcpu->exit.u.synic.control = synic->control;
+	hv_vcpu->exit.u.synic.evt_page = synic->evt_page;
+	hv_vcpu->exit.u.synic.msg_page = synic->msg_page;
+
+	kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
+}
+
 static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 			 u32 msr, u64 data, bool host)
 {
@@ -141,6 +155,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 	switch (msr) {
 	case HV_X64_MSR_SCONTROL:
 		synic->control = data;
+		synic_exit(synic, msr);
 		break;
 	case HV_X64_MSR_SVERSION:
 		if (!host) {
@@ -157,6 +172,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 				break;
 			}
 		synic->evt_page = data;
+		synic_exit(synic, msr);
 		break;
 	case HV_X64_MSR_SIMP:
 		if (data & HV_SYNIC_SIMP_ENABLE)
@@ -166,6 +182,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
 				break;
 			}
 		synic->msg_page = data;
+		synic_exit(synic, msr);
 		break;
 	case HV_X64_MSR_EOM: {
 		int i;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b853b2df..0704ee3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6342,6 +6342,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 			r = 0;
 			goto out;
 		}
+		if (kvm_check_request(KVM_REQ_HV_EXIT, vcpu)) {
+			vcpu->run->exit_reason = KVM_EXIT_HYPERV;
+			vcpu->run->hyperv = vcpu->arch.hyperv.exit;
+			r = 0;
+			goto out;
+		}
 	}
 
 	/*
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 43b0141..e38ac16 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -143,6 +143,7 @@ static inline bool is_error_page(struct page *page)
 #define KVM_REQ_HV_CRASH          27
 #define KVM_REQ_IOAPIC_EOI_EXIT   28
 #define KVM_REQ_HV_RESET          29
+#define KVM_REQ_HV_EXIT           30
 
 #define KVM_USERSPACE_IRQ_SOURCE_ID		0
 #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 27ce460..6e32f75 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -154,6 +154,20 @@ struct kvm_s390_skeys {
 	__u32 flags;
 	__u32 reserved[9];
 };
+
+struct kvm_hyperv_exit {
+#define KVM_EXIT_HYPERV_SYNIC          1
+	__u32 type;
+	union {
+		struct {
+			__u32 msr;
+			__u64 control;
+			__u64 evt_page;
+			__u64 msg_page;
+		} synic;
+	} u;
+};
+
 #define KVM_S390_GET_SKEYS_NONE   1
 #define KVM_S390_SKEYS_MAX        1048576
 
@@ -184,6 +198,7 @@ struct kvm_s390_skeys {
 #define KVM_EXIT_SYSTEM_EVENT     24
 #define KVM_EXIT_S390_STSI        25
 #define KVM_EXIT_IOAPIC_EOI       26
+#define KVM_EXIT_HYPERV           27
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -338,6 +353,8 @@ struct kvm_run {
 		struct {
 			__u8 vector;
 		} eoi;
+		/* KVM_EXIT_HYPERV */
+		struct kvm_hyperv_exit hyperv;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
-- 
2.4.3

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Paolo Bonzini]

On 22/10/2015 18:10, Andrey Smetanin wrote:
> A new vcpu exit is introduced to notify the userspace of the
> changes in Hyper-V SynIC configuration triggered by guest writing to the
> corresponding MSRs.
> 
> Changes v3:
> * added KVM_EXIT_HYPERV types and structs notes into docs

Thanks.  The changes look good.  I look forward to the unit tests so I
can merge it.

Paolo

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Denis V. Lunev]

On 10/22/2015 07:34 PM, Paolo Bonzini wrote:
>
> On 22/10/2015 18:10, Andrey Smetanin wrote:
>> A new vcpu exit is introduced to notify the userspace of the
>> changes in Hyper-V SynIC configuration triggered by guest writing to the
>> corresponding MSRs.
>>
>> Changes v3:
>> * added KVM_EXIT_HYPERV types and structs notes into docs
> Thanks.  The changes look good.  I look forward to the unit tests so I
> can merge it.
>
> Paolo

sent.

Den

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Paolo Bonzini]

On 22/10/2015 18:10, Andrey Smetanin wrote:
> A new vcpu exit is introduced to notify the userspace of the
> changes in Hyper-V SynIC configuration triggered by guest writing to the
> corresponding MSRs.
> 
> Changes v3:
> * added KVM_EXIT_HYPERV types and structs notes into docs
> 
> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
> Reviewed-by: Roman Kagan <rkagan@virtiozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
> CC: "K. Y. Srinivasan" <kys@microsoft.com>
> CC: Gleb Natapov <gleb@kernel.org>
> CC: Paolo Bonzini <pbonzini@redhat.com>
> CC: Roman Kagan <rkagan@virtiozzo.com>
> 
> ---
>  Documentation/virtual/kvm/api.txt | 22 ++++++++++++++++++++++
>  arch/x86/include/asm/kvm_host.h   |  1 +
>  arch/x86/kvm/hyperv.c             | 17 +++++++++++++++++
>  arch/x86/kvm/x86.c                |  6 ++++++
>  include/linux/kvm_host.h          |  1 +
>  include/uapi/linux/kvm.h          | 17 +++++++++++++++++
>  6 files changed, 64 insertions(+)
> 
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index 8710418..a6858eb 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -3337,6 +3337,28 @@ the userspace IOAPIC should process the EOI and retrigger the interrupt if
>  it is still asserted.  Vector is the LAPIC interrupt vector for which the
>  EOI was received.
>  
> +		struct kvm_hyperv_exit {
> +#define KVM_EXIT_HYPERV_SYNIC          1
> +			__u32 type;
> +			union {
> +				struct {
> +					__u32 msr;
> +					__u64 control;
> +					__u64 evt_page;
> +					__u64 msg_page;
> +				} synic;
> +			} u;
> +		};
> +		/* KVM_EXIT_HYPERV */
> +                struct kvm_hyperv_exit hyperv;
> +Indicates that the VCPU exits into userspace to process some tasks
> +related to Hyper-V emulation.
> +Valid values for 'type' are:
> +	KVM_EXIT_HYPERV_SYNIC -- synchronously notify user-space about
> +Hyper-V SynIC state change. Notification is used to remap SynIC
> +event/message pages and to enable/disable SynIC messages/events processing
> +in userspace.
> +
>  		/* Fix the size of the union. */
>  		char padding[256];
>  	};
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 8434f88..54c90d3 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -392,6 +392,7 @@ struct kvm_vcpu_hv {
>  	u64 hv_vapic;
>  	s64 runtime_offset;
>  	struct kvm_vcpu_hv_synic synic;
> +	struct kvm_hyperv_exit exit;
>  };
>  
>  struct kvm_vcpu_arch {
> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
> index 8ff71f3..9443920 100644
> --- a/arch/x86/kvm/hyperv.c
> +++ b/arch/x86/kvm/hyperv.c
> @@ -129,6 +129,20 @@ static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
>  	srcu_read_unlock(&kvm->irq_srcu, idx);
>  }
>  
> +static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr)
> +{
> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
> +	struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
> +
> +	hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNIC;
> +	hv_vcpu->exit.u.synic.msr = msr;
> +	hv_vcpu->exit.u.synic.control = synic->control;
> +	hv_vcpu->exit.u.synic.evt_page = synic->evt_page;
> +	hv_vcpu->exit.u.synic.msg_page = synic->msg_page;
> +
> +	kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
> +}
> +
>  static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>  			 u32 msr, u64 data, bool host)
>  {
> @@ -141,6 +155,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>  	switch (msr) {
>  	case HV_X64_MSR_SCONTROL:
>  		synic->control = data;
> +		synic_exit(synic, msr);

Another note.  I am getting:

EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000663
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=90 90 90 90 eb c3 90 90 90 90 90 90 00 00 00 00 56 54 46 00 <90> 90
eb ac 90 90 90 90 90 90 90 90 90 90 90 90 00 00 00 00 00 00 00 00 00 00
00 00 00 00

if I run a patched QEMU but I *do not* enable the synthetic interrupt
controller.  I can fix it by wrapping the calls to synic_exit with "if
(!host)", but I haven't checked yet the source---so that may not be the
proper fix.  Sorry for not having looked more in detail.

Paolo

>  		break;
>  	case HV_X64_MSR_SVERSION:
>  		if (!host) {
> @@ -157,6 +172,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>  				break;
>  			}
>  		synic->evt_page = data;
> +		synic_exit(synic, msr);
>  		break;
>  	case HV_X64_MSR_SIMP:
>  		if (data & HV_SYNIC_SIMP_ENABLE)
> @@ -166,6 +182,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>  				break;
>  			}
>  		synic->msg_page = data;
> +		synic_exit(synic, msr);
>  		break;
>  	case HV_X64_MSR_EOM: {
>  		int i;
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index b853b2df..0704ee3 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -6342,6 +6342,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>  			r = 0;
>  			goto out;
>  		}
> +		if (kvm_check_request(KVM_REQ_HV_EXIT, vcpu)) {
> +			vcpu->run->exit_reason = KVM_EXIT_HYPERV;
> +			vcpu->run->hyperv = vcpu->arch.hyperv.exit;
> +			r = 0;
> +			goto out;
> +		}
>  	}
>  
>  	/*
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 43b0141..e38ac16 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -143,6 +143,7 @@ static inline bool is_error_page(struct page *page)
>  #define KVM_REQ_HV_CRASH          27
>  #define KVM_REQ_IOAPIC_EOI_EXIT   28
>  #define KVM_REQ_HV_RESET          29
> +#define KVM_REQ_HV_EXIT           30
>  
>  #define KVM_USERSPACE_IRQ_SOURCE_ID		0
>  #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 27ce460..6e32f75 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -154,6 +154,20 @@ struct kvm_s390_skeys {
>  	__u32 flags;
>  	__u32 reserved[9];
>  };
> +
> +struct kvm_hyperv_exit {
> +#define KVM_EXIT_HYPERV_SYNIC          1
> +	__u32 type;
> +	union {
> +		struct {
> +			__u32 msr;
> +			__u64 control;
> +			__u64 evt_page;
> +			__u64 msg_page;
> +		} synic;
> +	} u;
> +};
> +
>  #define KVM_S390_GET_SKEYS_NONE   1
>  #define KVM_S390_SKEYS_MAX        1048576
>  
> @@ -184,6 +198,7 @@ struct kvm_s390_skeys {
>  #define KVM_EXIT_SYSTEM_EVENT     24
>  #define KVM_EXIT_S390_STSI        25
>  #define KVM_EXIT_IOAPIC_EOI       26
> +#define KVM_EXIT_HYPERV           27
>  
>  /* For KVM_EXIT_INTERNAL_ERROR */
>  /* Emulate instruction failed. */
> @@ -338,6 +353,8 @@ struct kvm_run {
>  		struct {
>  			__u8 vector;
>  		} eoi;
> +		/* KVM_EXIT_HYPERV */
> +		struct kvm_hyperv_exit hyperv;
>  		/* Fix the size of the union. */
>  		char padding[256];
>  	};
> 

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Andrey Smetanin]

On 11/03/2015 04:28 PM, Paolo Bonzini wrote:
>
>
> On 22/10/2015 18:10, Andrey Smetanin wrote:
>> A new vcpu exit is introduced to notify the userspace of the
>> changes in Hyper-V SynIC configuration triggered by guest writing to the
>> corresponding MSRs.
>>
>> Changes v3:
>> * added KVM_EXIT_HYPERV types and structs notes into docs
>>
>> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
>> Reviewed-by: Roman Kagan <rkagan@virtiozzo.com>
>> Signed-off-by: Denis V. Lunev <den@openvz.org>
>> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
>> CC: "K. Y. Srinivasan" <kys@microsoft.com>
>> CC: Gleb Natapov <gleb@kernel.org>
>> CC: Paolo Bonzini <pbonzini@redhat.com>
>> CC: Roman Kagan <rkagan@virtiozzo.com>
>>
>> ---
>>   Documentation/virtual/kvm/api.txt | 22 ++++++++++++++++++++++
>>   arch/x86/include/asm/kvm_host.h   |  1 +
>>   arch/x86/kvm/hyperv.c             | 17 +++++++++++++++++
>>   arch/x86/kvm/x86.c                |  6 ++++++
>>   include/linux/kvm_host.h          |  1 +
>>   include/uapi/linux/kvm.h          | 17 +++++++++++++++++
>>   6 files changed, 64 insertions(+)
>>
>> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
>> index 8710418..a6858eb 100644
>> --- a/Documentation/virtual/kvm/api.txt
>> +++ b/Documentation/virtual/kvm/api.txt
>> @@ -3337,6 +3337,28 @@ the userspace IOAPIC should process the EOI and retrigger the interrupt if
>>   it is still asserted.  Vector is the LAPIC interrupt vector for which the
>>   EOI was received.
>>
>> +		struct kvm_hyperv_exit {
>> +#define KVM_EXIT_HYPERV_SYNIC          1
>> +			__u32 type;
>> +			union {
>> +				struct {
>> +					__u32 msr;
>> +					__u64 control;
>> +					__u64 evt_page;
>> +					__u64 msg_page;
>> +				} synic;
>> +			} u;
>> +		};
>> +		/* KVM_EXIT_HYPERV */
>> +                struct kvm_hyperv_exit hyperv;
>> +Indicates that the VCPU exits into userspace to process some tasks
>> +related to Hyper-V emulation.
>> +Valid values for 'type' are:
>> +	KVM_EXIT_HYPERV_SYNIC -- synchronously notify user-space about
>> +Hyper-V SynIC state change. Notification is used to remap SynIC
>> +event/message pages and to enable/disable SynIC messages/events processing
>> +in userspace.
>> +
>>   		/* Fix the size of the union. */
>>   		char padding[256];
>>   	};
>> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>> index 8434f88..54c90d3 100644
>> --- a/arch/x86/include/asm/kvm_host.h
>> +++ b/arch/x86/include/asm/kvm_host.h
>> @@ -392,6 +392,7 @@ struct kvm_vcpu_hv {
>>   	u64 hv_vapic;
>>   	s64 runtime_offset;
>>   	struct kvm_vcpu_hv_synic synic;
>> +	struct kvm_hyperv_exit exit;
>>   };
>>
>>   struct kvm_vcpu_arch {
>> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
>> index 8ff71f3..9443920 100644
>> --- a/arch/x86/kvm/hyperv.c
>> +++ b/arch/x86/kvm/hyperv.c
>> @@ -129,6 +129,20 @@ static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
>>   	srcu_read_unlock(&kvm->irq_srcu, idx);
>>   }
>>
>> +static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr)
>> +{
>> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
>> +	struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
>> +
>> +	hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNIC;
>> +	hv_vcpu->exit.u.synic.msr = msr;
>> +	hv_vcpu->exit.u.synic.control = synic->control;
>> +	hv_vcpu->exit.u.synic.evt_page = synic->evt_page;
>> +	hv_vcpu->exit.u.synic.msg_page = synic->msg_page;
>> +
>> +	kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
>> +}
>> +
>>   static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   			 u32 msr, u64 data, bool host)
>>   {
>> @@ -141,6 +155,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   	switch (msr) {
>>   	case HV_X64_MSR_SCONTROL:
>>   		synic->control = data;
>> +		synic_exit(synic, msr);
>
> Another note.  I am getting:
>
> EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000663
> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
> EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0000 00000000 0000ffff 00009300
> CS =f000 ffff0000 0000ffff 00009b00
> SS =0000 00000000 0000ffff 00009300
> DS =0000 00000000 0000ffff 00009300
> FS =0000 00000000 0000ffff 00009300
> GS =0000 00000000 0000ffff 00009300
> LDT=0000 00000000 0000ffff 00008200
> TR =0000 00000000 0000ffff 00008b00
> GDT=     00000000 0000ffff
> IDT=     00000000 0000ffff
> CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> Code=90 90 90 90 eb c3 90 90 90 90 90 90 00 00 00 00 56 54 46 00 <90> 90
> eb ac 90 90 90 90 90 90 90 90 90 90 90 90 00 00 00 00 00 00 00 00 00 00
> 00 00 00 00
>
> if I run a patched QEMU but I *do not* enable the synthetic interrupt
> controller.  I can fix it by wrapping the calls to synic_exit with "if
> (!host)", but I haven't checked yet the source---so that may not be the
> proper fix.  Sorry for not having looked more in detail.
>
> Paolo
>
Thank you, we will test and resend correct version.
>>   		break;
>>   	case HV_X64_MSR_SVERSION:
>>   		if (!host) {
>> @@ -157,6 +172,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   				break;
>>   			}
>>   		synic->evt_page = data;
>> +		synic_exit(synic, msr);
>>   		break;
>>   	case HV_X64_MSR_SIMP:
>>   		if (data & HV_SYNIC_SIMP_ENABLE)
>> @@ -166,6 +182,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   				break;
>>   			}
>>   		synic->msg_page = data;
>> +		synic_exit(synic, msr);
>>   		break;
>>   	case HV_X64_MSR_EOM: {
>>   		int i;
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index b853b2df..0704ee3 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -6342,6 +6342,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>>   			r = 0;
>>   			goto out;
>>   		}
>> +		if (kvm_check_request(KVM_REQ_HV_EXIT, vcpu)) {
>> +			vcpu->run->exit_reason = KVM_EXIT_HYPERV;
>> +			vcpu->run->hyperv = vcpu->arch.hyperv.exit;
>> +			r = 0;
>> +			goto out;
>> +		}
>>   	}
>>
>>   	/*
>> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
>> index 43b0141..e38ac16 100644
>> --- a/include/linux/kvm_host.h
>> +++ b/include/linux/kvm_host.h
>> @@ -143,6 +143,7 @@ static inline bool is_error_page(struct page *page)
>>   #define KVM_REQ_HV_CRASH          27
>>   #define KVM_REQ_IOAPIC_EOI_EXIT   28
>>   #define KVM_REQ_HV_RESET          29
>> +#define KVM_REQ_HV_EXIT           30
>>
>>   #define KVM_USERSPACE_IRQ_SOURCE_ID		0
>>   #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index 27ce460..6e32f75 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -154,6 +154,20 @@ struct kvm_s390_skeys {
>>   	__u32 flags;
>>   	__u32 reserved[9];
>>   };
>> +
>> +struct kvm_hyperv_exit {
>> +#define KVM_EXIT_HYPERV_SYNIC          1
>> +	__u32 type;
>> +	union {
>> +		struct {
>> +			__u32 msr;
>> +			__u64 control;
>> +			__u64 evt_page;
>> +			__u64 msg_page;
>> +		} synic;
>> +	} u;
>> +};
>> +
>>   #define KVM_S390_GET_SKEYS_NONE   1
>>   #define KVM_S390_SKEYS_MAX        1048576
>>
>> @@ -184,6 +198,7 @@ struct kvm_s390_skeys {
>>   #define KVM_EXIT_SYSTEM_EVENT     24
>>   #define KVM_EXIT_S390_STSI        25
>>   #define KVM_EXIT_IOAPIC_EOI       26
>> +#define KVM_EXIT_HYPERV           27
>>
>>   /* For KVM_EXIT_INTERNAL_ERROR */
>>   /* Emulate instruction failed. */
>> @@ -338,6 +353,8 @@ struct kvm_run {
>>   		struct {
>>   			__u8 vector;
>>   		} eoi;
>> +		/* KVM_EXIT_HYPERV */
>> +		struct kvm_hyperv_exit hyperv;
>>   		/* Fix the size of the union. */
>>   		char padding[256];
>>   	};
>>

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Andrey Smetanin]

On 11/03/2015 04:28 PM, Paolo Bonzini wrote:
>
>
> On 22/10/2015 18:10, Andrey Smetanin wrote:
>> A new vcpu exit is introduced to notify the userspace of the
>> changes in Hyper-V SynIC configuration triggered by guest writing to the
>> corresponding MSRs.
>>
>> Changes v3:
>> * added KVM_EXIT_HYPERV types and structs notes into docs
>>
>> Signed-off-by: Andrey Smetanin <asmetanin@virtuozzo.com>
>> Reviewed-by: Roman Kagan <rkagan@virtiozzo.com>
>> Signed-off-by: Denis V. Lunev <den@openvz.org>
>> CC: Vitaly Kuznetsov <vkuznets@redhat.com>
>> CC: "K. Y. Srinivasan" <kys@microsoft.com>
>> CC: Gleb Natapov <gleb@kernel.org>
>> CC: Paolo Bonzini <pbonzini@redhat.com>
>> CC: Roman Kagan <rkagan@virtiozzo.com>
>>
>> ---
>>   Documentation/virtual/kvm/api.txt | 22 ++++++++++++++++++++++
>>   arch/x86/include/asm/kvm_host.h   |  1 +
>>   arch/x86/kvm/hyperv.c             | 17 +++++++++++++++++
>>   arch/x86/kvm/x86.c                |  6 ++++++
>>   include/linux/kvm_host.h          |  1 +
>>   include/uapi/linux/kvm.h          | 17 +++++++++++++++++
>>   6 files changed, 64 insertions(+)
>>
>> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
>> index 8710418..a6858eb 100644
>> --- a/Documentation/virtual/kvm/api.txt
>> +++ b/Documentation/virtual/kvm/api.txt
>> @@ -3337,6 +3337,28 @@ the userspace IOAPIC should process the EOI and retrigger the interrupt if
>>   it is still asserted.  Vector is the LAPIC interrupt vector for which the
>>   EOI was received.
>>
>> +		struct kvm_hyperv_exit {
>> +#define KVM_EXIT_HYPERV_SYNIC          1
>> +			__u32 type;
>> +			union {
>> +				struct {
>> +					__u32 msr;
>> +					__u64 control;
>> +					__u64 evt_page;
>> +					__u64 msg_page;
>> +				} synic;
>> +			} u;
>> +		};
>> +		/* KVM_EXIT_HYPERV */
>> +                struct kvm_hyperv_exit hyperv;
>> +Indicates that the VCPU exits into userspace to process some tasks
>> +related to Hyper-V emulation.
>> +Valid values for 'type' are:
>> +	KVM_EXIT_HYPERV_SYNIC -- synchronously notify user-space about
>> +Hyper-V SynIC state change. Notification is used to remap SynIC
>> +event/message pages and to enable/disable SynIC messages/events processing
>> +in userspace.
>> +
>>   		/* Fix the size of the union. */
>>   		char padding[256];
>>   	};
>> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>> index 8434f88..54c90d3 100644
>> --- a/arch/x86/include/asm/kvm_host.h
>> +++ b/arch/x86/include/asm/kvm_host.h
>> @@ -392,6 +392,7 @@ struct kvm_vcpu_hv {
>>   	u64 hv_vapic;
>>   	s64 runtime_offset;
>>   	struct kvm_vcpu_hv_synic synic;
>> +	struct kvm_hyperv_exit exit;
>>   };
>>
>>   struct kvm_vcpu_arch {
>> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
>> index 8ff71f3..9443920 100644
>> --- a/arch/x86/kvm/hyperv.c
>> +++ b/arch/x86/kvm/hyperv.c
>> @@ -129,6 +129,20 @@ static void kvm_hv_notify_acked_sint(struct kvm_vcpu *vcpu, u32 sint)
>>   	srcu_read_unlock(&kvm->irq_srcu, idx);
>>   }
>>
>> +static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr)
>> +{
>> +	struct kvm_vcpu *vcpu = synic_to_vcpu(synic);
>> +	struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
>> +
>> +	hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNIC;
>> +	hv_vcpu->exit.u.synic.msr = msr;
>> +	hv_vcpu->exit.u.synic.control = synic->control;
>> +	hv_vcpu->exit.u.synic.evt_page = synic->evt_page;
>> +	hv_vcpu->exit.u.synic.msg_page = synic->msg_page;
>> +
>> +	kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
>> +}
>> +
>>   static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   			 u32 msr, u64 data, bool host)
>>   {
>> @@ -141,6 +155,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   	switch (msr) {
>>   	case HV_X64_MSR_SCONTROL:
>>   		synic->control = data;
>> +		synic_exit(synic, msr);
>
> Another note.  I am getting:
>
> EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000663
> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
> EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0000 00000000 0000ffff 00009300
> CS =f000 ffff0000 0000ffff 00009b00
> SS =0000 00000000 0000ffff 00009300
> DS =0000 00000000 0000ffff 00009300
> FS =0000 00000000 0000ffff 00009300
> GS =0000 00000000 0000ffff 00009300
> LDT=0000 00000000 0000ffff 00008200
> TR =0000 00000000 0000ffff 00008b00
> GDT=     00000000 0000ffff
> IDT=     00000000 0000ffff
> CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> Code=90 90 90 90 eb c3 90 90 90 90 90 90 00 00 00 00 56 54 46 00 <90> 90
> eb ac 90 90 90 90 90 90 90 90 90 90 90 90 00 00 00 00 00 00 00 00 00 00
> 00 00 00 00
>
> if I run a patched QEMU but I *do not* enable the synthetic interrupt
> controller.  I can fix it by wrapping the calls to synic_exit with "if
> (!host)", but I haven't checked yet the source---so that may not be the
> proper fix.  Sorry for not having looked more in detail.
>
Could you please specify test case(kvm unit tests ?) and kernel/qemu(if 
it's not standard)? We can't catch this bug anymore.
> Paolo
>
>>   		break;
>>   	case HV_X64_MSR_SVERSION:
>>   		if (!host) {
>> @@ -157,6 +172,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   				break;
>>   			}
>>   		synic->evt_page = data;
>> +		synic_exit(synic, msr);
>>   		break;
>>   	case HV_X64_MSR_SIMP:
>>   		if (data & HV_SYNIC_SIMP_ENABLE)
>> @@ -166,6 +182,7 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
>>   				break;
>>   			}
>>   		synic->msg_page = data;
>> +		synic_exit(synic, msr);
>>   		break;
>>   	case HV_X64_MSR_EOM: {
>>   		int i;
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index b853b2df..0704ee3 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -6342,6 +6342,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>>   			r = 0;
>>   			goto out;
>>   		}
>> +		if (kvm_check_request(KVM_REQ_HV_EXIT, vcpu)) {
>> +			vcpu->run->exit_reason = KVM_EXIT_HYPERV;
>> +			vcpu->run->hyperv = vcpu->arch.hyperv.exit;
>> +			r = 0;
>> +			goto out;
>> +		}
>>   	}
>>
>>   	/*
>> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
>> index 43b0141..e38ac16 100644
>> --- a/include/linux/kvm_host.h
>> +++ b/include/linux/kvm_host.h
>> @@ -143,6 +143,7 @@ static inline bool is_error_page(struct page *page)
>>   #define KVM_REQ_HV_CRASH          27
>>   #define KVM_REQ_IOAPIC_EOI_EXIT   28
>>   #define KVM_REQ_HV_RESET          29
>> +#define KVM_REQ_HV_EXIT           30
>>
>>   #define KVM_USERSPACE_IRQ_SOURCE_ID		0
>>   #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index 27ce460..6e32f75 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -154,6 +154,20 @@ struct kvm_s390_skeys {
>>   	__u32 flags;
>>   	__u32 reserved[9];
>>   };
>> +
>> +struct kvm_hyperv_exit {
>> +#define KVM_EXIT_HYPERV_SYNIC          1
>> +	__u32 type;
>> +	union {
>> +		struct {
>> +			__u32 msr;
>> +			__u64 control;
>> +			__u64 evt_page;
>> +			__u64 msg_page;
>> +		} synic;
>> +	} u;
>> +};
>> +
>>   #define KVM_S390_GET_SKEYS_NONE   1
>>   #define KVM_S390_SKEYS_MAX        1048576
>>
>> @@ -184,6 +198,7 @@ struct kvm_s390_skeys {
>>   #define KVM_EXIT_SYSTEM_EVENT     24
>>   #define KVM_EXIT_S390_STSI        25
>>   #define KVM_EXIT_IOAPIC_EOI       26
>> +#define KVM_EXIT_HYPERV           27
>>
>>   /* For KVM_EXIT_INTERNAL_ERROR */
>>   /* Emulate instruction failed. */
>> @@ -338,6 +353,8 @@ struct kvm_run {
>>   		struct {
>>   			__u8 vector;
>>   		} eoi;
>> +		/* KVM_EXIT_HYPERV */
>> +		struct kvm_hyperv_exit hyperv;
>>   		/* Fix the size of the union. */
>>   		char padding[256];
>>   	};
>>

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Paolo Bonzini]

On 03/11/2015 15:36, Andrey Smetanin wrote:
>>
>>
>> if I run a patched QEMU but I *do not* enable the synthetic interrupt
>> controller.  I can fix it by wrapping the calls to synic_exit with "if
>> (!host)", but I haven't checked yet the source---so that may not be the
>> proper fix.  Sorry for not having looked more in detail.
>>
> Could you please specify test case(kvm unit tests ?) and kernel/qemu(if
> it's not standard)?

It happens just by starting QEMU.

Kernel: kvm/queue
+ kvm/irqchip: kvm_arch_irq_routing_update renaming split
+ kvm/x86: split ioapic-handled and EOI exit bitmaps
+ kvm/x86: Hyper-V synthetic interrupt controller
+ kvm/x86: Hyper-V kvm exit

QEMU: 3a958f559ecd
+ standard-headers/x86: add Hyper-V SynIC constants
+ target-i386/kvm: Hyper-V SynIC MSR's support
+ linux-headers/kvm: add Hyper-V SynIC irq routing type and struct
+ kvm: Hyper-V SynIC irq routing support
+ linux-headers/kvm: KVM_EXIT_HYPERV type and struct
+ target-i386/hyperv: Hyper-V SynIC SINT routing and vCPU exit
+ hw/misc: Hyper-V test device 'hyperv-testdev'

Can be reproduced just with
"../qemu/+build/x86_64-softmmu/qemu-system-x86_64 --enable-kvm -cpu
kvm64 -display none".

Paolo

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Andrey Smetanin]

On 11/03/2015 05:51 PM, Paolo Bonzini wrote:
>
>
> On 03/11/2015 15:36, Andrey Smetanin wrote:
>>>
>>>
>>> if I run a patched QEMU but I *do not* enable the synthetic interrupt
>>> controller.  I can fix it by wrapping the calls to synic_exit with "if
>>> (!host)", but I haven't checked yet the source---so that may not be the
>>> proper fix.  Sorry for not having looked more in detail.
>>>
>> Could you please specify test case(kvm unit tests ?) and kernel/qemu(if
>> it's not standard)?
>
> It happens just by starting QEMU.
>
> Kernel: kvm/queue
> + kvm/irqchip: kvm_arch_irq_routing_update renaming split
> + kvm/x86: split ioapic-handled and EOI exit bitmaps
> + kvm/x86: Hyper-V synthetic interrupt controller
> + kvm/x86: Hyper-V kvm exit
>
> QEMU: 3a958f559ecd
> + standard-headers/x86: add Hyper-V SynIC constants
> + target-i386/kvm: Hyper-V SynIC MSR's support
> + linux-headers/kvm: add Hyper-V SynIC irq routing type and struct
> + kvm: Hyper-V SynIC irq routing support
> + linux-headers/kvm: KVM_EXIT_HYPERV type and struct
> + target-i386/hyperv: Hyper-V SynIC SINT routing and vCPU exit
> + hw/misc: Hyper-V test device 'hyperv-testdev'
>
> Can be reproduced just with
> "../qemu/+build/x86_64-softmmu/qemu-system-x86_64 --enable-kvm -cpu
> kvm64 -display none".
>
Thanks!
We probably found root case -
qemu reads/writes Hyper-V SynIC msrs just by check SynIC MSR's support 
in kernel. So KVM synic exits into userspace(at SynIC MSR's writes), 
while userspace Hyper-V SynIC handler doesn't expect this exit(cpu 
'hv-synic' option is not set), so handler returns -1 and qemu exits.
> Paolo
>

Re: [Qemu-devel] [PATCH v3 9/9] kvm/x86: Hyper-V kvm exit

[Roman Kagan]

On Tue, Nov 03, 2015 at 03:51:16PM +0100, Paolo Bonzini wrote:
> 
> 
> On 03/11/2015 15:36, Andrey Smetanin wrote:
> >>
> >>
> >> if I run a patched QEMU but I *do not* enable the synthetic interrupt
> >> controller.  I can fix it by wrapping the calls to synic_exit with "if
> >> (!host)", but I haven't checked yet the source---so that may not be the
> >> proper fix.  Sorry for not having looked more in detail.
> >>
> > Could you please specify test case(kvm unit tests ?) and kernel/qemu(if
> > it's not standard)?
> 
> It happens just by starting QEMU.
> 
> Kernel: kvm/queue
> + kvm/irqchip: kvm_arch_irq_routing_update renaming split
> + kvm/x86: split ioapic-handled and EOI exit bitmaps
> + kvm/x86: Hyper-V synthetic interrupt controller
> + kvm/x86: Hyper-V kvm exit
> 
> QEMU: 3a958f559ecd
> + standard-headers/x86: add Hyper-V SynIC constants
> + target-i386/kvm: Hyper-V SynIC MSR's support
> + linux-headers/kvm: add Hyper-V SynIC irq routing type and struct
> + kvm: Hyper-V SynIC irq routing support
> + linux-headers/kvm: KVM_EXIT_HYPERV type and struct
> + target-i386/hyperv: Hyper-V SynIC SINT routing and vCPU exit
> + hw/misc: Hyper-V test device 'hyperv-testdev'
> 
> Can be reproduced just with
> "../qemu/+build/x86_64-softmmu/qemu-system-x86_64 --enable-kvm -cpu
> kvm64 -display none".

Thanks!  We've figured it out:

qemu initializes the MSRs if has_msr_hv_synic is set, which depends only
on whether the kernel supports the MSRs and ignores the cpu property.

OTOH setting those MSRs (on the host side) triggers a vcpu exit which
checks the cpu property and aborts if it's unset.  Voila.

This way we also discovered that no error was triggered when the cpu
property was set but the kernel didn't support it (and this problem was
also present in other hyperv-related features).

The solution appears to be to bail out when a hyperv property is
requested but the host doesn't support it, and then check for the
property only when deciding if the relevant actions need to be taken.

Protecting vcpu exits with !host in the kernel seems to make sense, too.

We're in progress of preparing the updated patches.

Thanks,
Roman.