[PATCH v3 0/7] x86/kvm/hyperv: stable clocksorce for L2 guests when running nested KVM on Hyper-V

[PATCH v3 0/7] x86/kvm/hyperv: stable clocksorce for L2 guests when running nested KVM on Hyper-V

[Vitaly Kuznetsov]

Changes since v2:
- Add Paolo's Acked-by to PATCH6-7
- Add Thomas' Reviewed-by to PATCH1
- Update the description of PATCH2 to match the reality [Thomas Gleixner]
- Add __visible and __irq_entry annotations to hyperv_reenlightenment_intr()
  [Thomas Gleixner]
- Drop spinlock protection and use cpumask_any_but() in PATCH4
  [Thomas Gleixner]

Original description:

Currently, KVM passes PVCLOCK_TSC_STABLE_BIT to its guests when running in
so called 'masterclock' mode and this is only possible when the clocksource
on the host is TSC. When running nested on Hyper-V we're using a different
clocksource in L1 (Hyper-V TSC Page) which can actually be used for
masterclock. This series brings the required support.

Making KVM work with TSC page clocksource is relatively easy, it is done in
PATCH 6 of the series. All the rest is required to support L1 migration
when TSC frequency changes, we use a special feature from Hyper-V to do
the job.

Vitaly Kuznetsov (7):
  x86/hyper-v: check for required priviliges in hyperv_init()
  x86/hyper-v: add a function to read both TSC and TSC page value
    simulateneously
  x86/hyper-v: reenlightenment notifications support
  x86/hyper-v: redirect reenlightment notifications on CPU offlining
  x86/irq: Count Hyper-V reenlightenment interrupts
  x86/kvm: pass stable clocksource to guests when running nested on
    Hyper-V
  x86/kvm: support Hyper-V reenlightenment

 arch/x86/entry/entry_32.S          |   3 +
 arch/x86/entry/entry_64.S          |   3 +
 arch/x86/hyperv/hv_init.c          | 123 ++++++++++++++++++++++++++++++++-
 arch/x86/include/asm/hardirq.h     |   3 +
 arch/x86/include/asm/irq_vectors.h |   7 +-
 arch/x86/include/asm/mshyperv.h    |  33 +++++++--
 arch/x86/include/uapi/asm/hyperv.h |  27 ++++++++
 arch/x86/kernel/cpu/mshyperv.c     |   6 ++
 arch/x86/kernel/irq.c              |   9 +++
 arch/x86/kvm/x86.c                 | 138 ++++++++++++++++++++++++++++++-------
 10 files changed, 320 insertions(+), 32 deletions(-)

-- 
2.14.3

[PATCH v3 1/7] x86/hyper-v: check for required priviliges in hyperv_init()

[Vitaly Kuznetsov]

In hyperv_init() we presume we always have access to VP index and hypercall
MSRs while according to the specification we should check if we're allowed
to access the corresponding MSRs before accessing them.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/hyperv/hv_init.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 189a398290db..21f9d53d9f00 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -110,12 +110,19 @@ static int hv_cpu_init(unsigned int cpu)
  */
 void hyperv_init(void)
 {
-	u64 guest_id;
+	u64 guest_id, required_msrs;
 	union hv_x64_msr_hypercall_contents hypercall_msr;
 
 	if (x86_hyper_type != X86_HYPER_MS_HYPERV)
 		return;
 
+	/* Absolutely required MSRs */
+	required_msrs = HV_X64_MSR_HYPERCALL_AVAILABLE |
+		HV_X64_MSR_VP_INDEX_AVAILABLE;
+
+	if ((ms_hyperv.features & required_msrs) != required_msrs)
+		return;
+
 	/* Allocate percpu VP index */
 	hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
 				    GFP_KERNEL);
-- 
2.14.3

[PATCH v3 2/7] x86/hyper-v: add a function to read both TSC and TSC page value simulateneously

[Vitaly Kuznetsov]

This is going to be used from KVM code where we need to get both
TSC and TSC page value.

Nobody is supposed to use the function when Hyper-V code is compiled out,
just BUG().

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/hyperv/hv_init.c       |  1 +
 arch/x86/include/asm/mshyperv.h | 23 +++++++++++++++++++----
 2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 21f9d53d9f00..1a6c63f721bc 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -37,6 +37,7 @@ struct ms_hyperv_tsc_page *hv_get_tsc_page(void)
 {
 	return tsc_pg;
 }
+EXPORT_SYMBOL_GPL(hv_get_tsc_page);
 
 static u64 read_hv_clock_tsc(struct clocksource *arg)
 {
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 8bf450b13d9f..6b1d4ea78270 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -325,9 +325,10 @@ static inline void hyperv_setup_mmu_ops(void) {}
 
 #ifdef CONFIG_HYPERV_TSCPAGE
 struct ms_hyperv_tsc_page *hv_get_tsc_page(void);
-static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
+static inline u64 hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
+				       u64 *cur_tsc)
 {
-	u64 scale, offset, cur_tsc;
+	u64 scale, offset;
 	u32 sequence;
 
 	/*
@@ -358,7 +359,7 @@ static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
 
 		scale = READ_ONCE(tsc_pg->tsc_scale);
 		offset = READ_ONCE(tsc_pg->tsc_offset);
-		cur_tsc = rdtsc_ordered();
+		*cur_tsc = rdtsc_ordered();
 
 		/*
 		 * Make sure we read sequence after we read all other values
@@ -368,7 +369,14 @@ static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
 
 	} while (READ_ONCE(tsc_pg->tsc_sequence) != sequence);
 
-	return mul_u64_u64_shr(cur_tsc, scale, 64) + offset;
+	return mul_u64_u64_shr(*cur_tsc, scale, 64) + offset;
+}
+
+static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg)
+{
+	u64 cur_tsc;
+
+	return hv_read_tsc_page_tsc(tsc_pg, &cur_tsc);
 }
 
 #else
@@ -376,5 +384,12 @@ static inline struct ms_hyperv_tsc_page *hv_get_tsc_page(void)
 {
 	return NULL;
 }
+
+static inline u64 hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg,
+				       u64 *cur_tsc)
+{
+	BUG();
+	return U64_MAX;
+}
 #endif
 #endif
-- 
2.14.3

[PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Vitaly Kuznetsov]

Hyper-V supports Live Migration notification. This is supposed to be used
in conjunction with TSC emulation: when we are migrated to a host with
different TSC frequency for some short period host emulates our accesses
to TSC and sends us an interrupt to notify about the event. When we're
done updating everything we can disable TSC emulation and everything will
start working fast again.

We didn't need these notifications before as Hyper-V guests are not
supposed to use TSC as a clocksource: in Linux we even mark it as unstable
on boot. Guests normally use 'tsc page' clocksouce and host updates its
values on migrations automatically.

Things change when we want to run nested virtualization: even when we pass
through PV clocksources (kvm-clock or tsc page) to our guests we need to
know TSC frequency and when it changes.

Hyper-V Top Level Functional Specification (as of v5.0b) wrongly specifies
EAX:BIT(12) of CPUID:0x40000009 as the feature identification bit. The
right one to check is EAX:BIT(13) of CPUID:0x40000003. I was assured that
the fix in on the way.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
v2 -> v3:
- add __visible and __irq_entry [Thomas Gleixner]
---
 arch/x86/entry/entry_32.S          |  3 ++
 arch/x86/entry/entry_64.S          |  3 ++
 arch/x86/hyperv/hv_init.c          | 89 ++++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/irq_vectors.h |  7 ++-
 arch/x86/include/asm/mshyperv.h    | 10 +++++
 arch/x86/include/uapi/asm/hyperv.h | 27 ++++++++++++
 arch/x86/kernel/cpu/mshyperv.c     |  6 +++
 7 files changed, 144 insertions(+), 1 deletion(-)

diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
index a1f28a54f23a..4041608cd208 100644
--- a/arch/x86/entry/entry_32.S
+++ b/arch/x86/entry/entry_32.S
@@ -883,6 +883,9 @@ BUILD_INTERRUPT3(xen_hvm_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
 BUILD_INTERRUPT3(hyperv_callback_vector, HYPERVISOR_CALLBACK_VECTOR,
 		 hyperv_vector_handler)
 
+BUILD_INTERRUPT3(hyperv_reenlightenment_vector, HYPERV_REENLIGHTENMENT_VECTOR,
+		 hyperv_reenlightenment_intr)
+
 #endif /* CONFIG_HYPERV */
 
 ENTRY(page_fault)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 4f8e1d35a97c..7da977b8d1dd 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -1233,6 +1233,9 @@ apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \
 #if IS_ENABLED(CONFIG_HYPERV)
 apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \
 	hyperv_callback_vector hyperv_vector_handler
+
+apicinterrupt3 HYPERV_REENLIGHTENMENT_VECTOR \
+	hyperv_reenlightenment_vector hyperv_reenlightenment_intr
 #endif /* CONFIG_HYPERV */
 
 idtentry debug			do_debug		has_error_code=0	paranoid=1 shift_ist=DEBUG_STACK
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 1a6c63f721bc..712ac40081f7 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -18,6 +18,8 @@
  */
 
 #include <linux/types.h>
+#include <asm/apic.h>
+#include <asm/desc.h>
 #include <asm/hypervisor.h>
 #include <asm/hyperv.h>
 #include <asm/mshyperv.h>
@@ -102,6 +104,93 @@ static int hv_cpu_init(unsigned int cpu)
 	return 0;
 }
 
+static void (*hv_reenlightenment_cb)(void);
+
+static void hv_reenlightenment_notify(struct work_struct *dummy)
+{
+	struct hv_tsc_emulation_status emu_status;
+
+	rdmsrl(HV_X64_MSR_TSC_EMULATION_STATUS, *(u64 *)&emu_status);
+
+	/* Don't issue the callback if TSC accesses are not emulated */
+	if (hv_reenlightenment_cb && emu_status.inprogress)
+		hv_reenlightenment_cb();
+}
+static DECLARE_DELAYED_WORK(hv_reenlightenment_work, hv_reenlightenment_notify);
+
+void hyperv_stop_tsc_emulation(void)
+{
+	u64 freq;
+	struct hv_tsc_emulation_status emu_status;
+
+	rdmsrl(HV_X64_MSR_TSC_EMULATION_STATUS, *(u64 *)&emu_status);
+	emu_status.inprogress = 0;
+	wrmsrl(HV_X64_MSR_TSC_EMULATION_STATUS, *(u64 *)&emu_status);
+
+	rdmsrl(HV_X64_MSR_TSC_FREQUENCY, freq);
+	tsc_khz = div64_u64(freq, 1000);
+}
+EXPORT_SYMBOL_GPL(hyperv_stop_tsc_emulation);
+
+static inline bool hv_reenlightenment_available(void)
+{
+	/*
+	 * Check for required features and priviliges to make TSC frequency
+	 * change notifications work.
+	 */
+	return ms_hyperv.features & HV_X64_ACCESS_FREQUENCY_MSRS &&
+		ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE &&
+		ms_hyperv.features & HV_X64_ACCESS_REENLIGHTENMENT;
+}
+
+__visible void __irq_entry hyperv_reenlightenment_intr(struct pt_regs *regs)
+{
+	entering_ack_irq();
+
+	schedule_delayed_work(&hv_reenlightenment_work, HZ/10);
+
+	exiting_irq();
+}
+
+void set_hv_tscchange_cb(void (*cb)(void))
+{
+	struct hv_reenlightenment_control re_ctrl = {
+		.vector = HYPERV_REENLIGHTENMENT_VECTOR,
+		.enabled = 1,
+		.target_vp = hv_vp_index[smp_processor_id()]
+	};
+	struct hv_tsc_emulation_control emu_ctrl = {.enabled = 1};
+
+	if (!hv_reenlightenment_available()) {
+		pr_warn("Hyper-V: reenlightenment support is unavailable\n");
+		return;
+	}
+
+	hv_reenlightenment_cb = cb;
+
+	/* Make sure callback is registered before we write to MSRs */
+	wmb();
+
+	wrmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *((u64 *)&re_ctrl));
+	wrmsrl(HV_X64_MSR_TSC_EMULATION_CONTROL, *((u64 *)&emu_ctrl));
+}
+EXPORT_SYMBOL_GPL(set_hv_tscchange_cb);
+
+void clear_hv_tscchange_cb(void)
+{
+	struct hv_reenlightenment_control re_ctrl;
+
+	if (!hv_reenlightenment_available())
+		return;
+
+	rdmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *(u64 *)&re_ctrl);
+	re_ctrl.enabled = 0;
+	wrmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *(u64 *)&re_ctrl);
+
+	hv_reenlightenment_cb = NULL;
+}
+EXPORT_SYMBOL_GPL(clear_hv_tscchange_cb);
+
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h
index 67421f649cfa..e71c1120426b 100644
--- a/arch/x86/include/asm/irq_vectors.h
+++ b/arch/x86/include/asm/irq_vectors.h
@@ -103,7 +103,12 @@
 #endif
 
 #define MANAGED_IRQ_SHUTDOWN_VECTOR	0xef
-#define LOCAL_TIMER_VECTOR		0xee
+
+#if IS_ENABLED(CONFIG_HYPERV)
+#define HYPERV_REENLIGHTENMENT_VECTOR	0xee
+#endif
+
+#define LOCAL_TIMER_VECTOR		0xed
 
 #define NR_VECTORS			 256
 
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 6b1d4ea78270..96c96f8e570c 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -5,6 +5,7 @@
 #include <linux/types.h>
 #include <linux/atomic.h>
 #include <linux/nmi.h>
+#include <linux/interrupt.h>
 #include <asm/io.h>
 #include <asm/hyperv.h>
 #include <asm/nospec-branch.h>
@@ -160,6 +161,7 @@ static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type)
 #define hv_set_synint_state(int_num, val) wrmsrl(int_num, val)
 
 void hyperv_callback_vector(void);
+void hyperv_reenlightenment_vector(void);
 #ifdef CONFIG_TRACING
 #define trace_hyperv_callback_vector hyperv_callback_vector
 #endif
@@ -316,11 +318,19 @@ void hyper_alloc_mmu(void);
 void hyperv_report_panic(struct pt_regs *regs, long err);
 bool hv_is_hypercall_page_setup(void);
 void hyperv_cleanup(void);
+
+__visible void __irq_entry hyperv_reenlightenment_intr(struct pt_regs *regs);
+void set_hv_tscchange_cb(void (*cb)(void));
+void clear_hv_tscchange_cb(void);
+void hyperv_stop_tsc_emulation(void);
 #else /* CONFIG_HYPERV */
 static inline void hyperv_init(void) {}
 static inline bool hv_is_hypercall_page_setup(void) { return false; }
 static inline void hyperv_cleanup(void) {}
 static inline void hyperv_setup_mmu_ops(void) {}
+static inline void set_hv_tscchange_cb(void (*cb)(void)) {}
+static inline void clear_hv_tscchange_cb(void) {}
+static inline void hyperv_stop_tsc_emulation(void) {};
 #endif /* CONFIG_HYPERV */
 
 #ifdef CONFIG_HYPERV_TSCPAGE
diff --git a/arch/x86/include/uapi/asm/hyperv.h b/arch/x86/include/uapi/asm/hyperv.h
index 1a5bfead93b4..197c2e6c7376 100644
--- a/arch/x86/include/uapi/asm/hyperv.h
+++ b/arch/x86/include/uapi/asm/hyperv.h
@@ -40,6 +40,9 @@
  */
 #define HV_X64_ACCESS_FREQUENCY_MSRS		(1 << 11)
 
+/* AccessReenlightenmentControls privilege */
+#define HV_X64_ACCESS_REENLIGHTENMENT		BIT(13)
+
 /*
  * Basic SynIC MSRs (HV_X64_MSR_SCONTROL through HV_X64_MSR_EOM
  * and HV_X64_MSR_SINT0 through HV_X64_MSR_SINT15) available
@@ -234,6 +237,30 @@
 #define HV_X64_MSR_CRASH_PARAMS		\
 		(1 + (HV_X64_MSR_CRASH_P4 - HV_X64_MSR_CRASH_P0))
 
+/* TSC emulation after migration */
+#define HV_X64_MSR_REENLIGHTENMENT_CONTROL	0x40000106
+
+struct hv_reenlightenment_control {
+	u64 vector:8;
+	u64 reserved1:8;
+	u64 enabled:1;
+	u64 reserved2:15;
+	u64 target_vp:32;
+};
+
+#define HV_X64_MSR_TSC_EMULATION_CONTROL	0x40000107
+#define HV_X64_MSR_TSC_EMULATION_STATUS		0x40000108
+
+struct hv_tsc_emulation_control {
+	u64 enabled:1;
+	u64 reserved:63;
+};
+
+struct hv_tsc_emulation_status {
+	u64 inprogress:1;
+	u64 reserved:63;
+};
+
 #define HV_X64_MSR_HYPERCALL_ENABLE		0x00000001
 #define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT	12
 #define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK	\
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 85eb5fc180c8..9340f41ce8d3 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -251,6 +251,12 @@ static void __init ms_hyperv_init_platform(void)
 	hyperv_setup_mmu_ops();
 	/* Setup the IDT for hypervisor callback */
 	alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, hyperv_callback_vector);
+
+	/* Setup the IDT for reenlightenment notifications */
+	if (ms_hyperv.features & HV_X64_ACCESS_REENLIGHTENMENT)
+		alloc_intr_gate(HYPERV_REENLIGHTENMENT_VECTOR,
+				hyperv_reenlightenment_vector);
+
 #endif
 }
 
-- 
2.14.3

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Thomas Gleixner]

On Tue, 16 Jan 2018, Vitaly Kuznetsov wrote:
> Hyper-V supports Live Migration notification. This is supposed to be used
> in conjunction with TSC emulation: when we are migrated to a host with
> different TSC frequency for some short period host emulates our accesses
> to TSC and sends us an interrupt to notify about the event. When we're
> done updating everything we can disable TSC emulation and everything will
> start working fast again.
> 
> We didn't need these notifications before as Hyper-V guests are not
> supposed to use TSC as a clocksource: in Linux we even mark it as unstable
> on boot. Guests normally use 'tsc page' clocksouce and host updates its
> values on migrations automatically.
> 
> Things change when we want to run nested virtualization: even when we pass
> through PV clocksources (kvm-clock or tsc page) to our guests we need to
> know TSC frequency and when it changes.
> 
> Hyper-V Top Level Functional Specification (as of v5.0b) wrongly specifies
> EAX:BIT(12) of CPUID:0x40000009 as the feature identification bit. The
> right one to check is EAX:BIT(13) of CPUID:0x40000003. I was assured that
> the fix in on the way.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 25949 bytes --]

Hi Vitaly,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on tip/auto-latest]
[also build test WARNING on v4.15-rc8 next-20180118]
[cannot apply to tip/x86/core kvm/linux-next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Vitaly-Kuznetsov/x86-kvm-hyperv-stable-clocksorce-for-L2-guests-when-running-nested-KVM-on-Hyper-V/20180119-160814
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All warnings (new ones prefixed by >>):

   In file included from include/linux/kasan.h:17:0,
                    from include/linux/slab.h:129,
                    from include/linux/irq.h:26,
                    from arch/x86/include/asm/hardirq.h:6,
                    from include/linux/hardirq.h:9,
                    from include/linux/interrupt.h:13,
                    from arch/x86/include/asm/mshyperv.h:8,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
   arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-declaration]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
            ^~~~~~~~~~~~~~~~~~~
            u64_to_user_ptr
   arch/x86/include/asm/pgtable.h:1129:9: warning: passing argument 1 of 'memcpy' makes pointer from integer without a cast [-Wint-conversion]
   In file included from arch/x86/include/asm/page_32.h:35:0,
                    from arch/x86/include/asm/page.h:14,
                    from arch/x86/include/asm/thread_info.h:12,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
   include/linux/string.h:332:24: note: expected 'void *' but argument is of type 'int'
    __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
                           ^~~~~~
   In file included from include/linux/kasan.h:17:0,
                    from include/linux/slab.h:129,
                    from include/linux/irq.h:26,
                    from arch/x86/include/asm/hardirq.h:6,
                    from include/linux/hardirq.h:9,
                    from include/linux/interrupt.h:13,
                    from arch/x86/include/asm/mshyperv.h:8,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h:1129:35: warning: passing argument 2 of 'memcpy' makes pointer from integer without a cast [-Wint-conversion]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
                                      ^~~~~~~~~~~~~~~~~~~
   In file included from arch/x86/include/asm/page_32.h:35:0,
                    from arch/x86/include/asm/page.h:14,
                    from arch/x86/include/asm/thread_info.h:12,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
   include/linux/string.h:332:24: note: expected 'const void *' but argument is of type 'int'
    __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
                           ^~~~~~
   In file included from arch/x86/include/asm/paravirt_types.h:45:0,
                    from arch/x86/include/asm/ptrace.h:92,
                    from arch/x86/include/asm/math_emu.h:5,
                    from arch/x86/include/asm/processor.h:12,
                    from arch/x86/include/asm/cpufeature.h:5,
                    from arch/x86/include/asm/thread_info.h:53,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'pte_flags_pkey':
>> arch/x86/include/asm/pgtable_types.h:58:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT0 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:70:26: note: in expansion of macro '_PAGE_PKEY_BIT0'
    #define _PAGE_PKEY_MASK (_PAGE_PKEY_BIT0 | \
                             ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:59:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT1 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:71:5: note: in expansion of macro '_PAGE_PKEY_BIT1'
        _PAGE_PKEY_BIT1 | \
        ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:60:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT2 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT2)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:72:5: note: in expansion of macro '_PAGE_PKEY_BIT2'
        _PAGE_PKEY_BIT2 | \
        ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:61:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT3 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT3)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:73:5: note: in expansion of macro '_PAGE_PKEY_BIT3'
        _PAGE_PKEY_BIT3)
        ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   In file included from include/linux/kasan.h:17:0,
                    from include/linux/slab.h:129,
                    from include/linux/irq.h:26,
                    from arch/x86/include/asm/hardirq.h:6,
                    from include/linux/hardirq.h:9,
                    from include/linux/interrupt.h:13,
                    from arch/x86/include/asm/mshyperv.h:8,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
>> arch/x86/include/asm/pgtable.h:1223:39: warning: right shift count >= width of type [-Wshift-count-overflow]
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                                          ^~
   cc1: some warnings being treated as errors
--
   include/linux/kasan.h:31:19: sparse: constant 0xdffffc0000000000UL is so big it is unsigned long long
   In file included from include/linux/kasan.h:17:0,
                    from include/linux/slab.h:129,
                    from include/linux/irq.h:26,
                    from arch/x86/include/asm/hardirq.h:6,
                    from include/linux/hardirq.h:9,
                    from include/linux/interrupt.h:13,
                    from arch/x86/include/asm/mshyperv.h:8,
                    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:20,
                    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
   arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-declaration]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
            ^~~~~~~~~~~~~~~~~~~
            u64_to_user_ptr
   arch/x86/include/asm/pgtable.h:1129:9: warning: passing argument 1 of 'memcpy' makes pointer from integer without a cast [-Wint-conversion]
   In file included from arch/x86/include/asm/page_32.h:35:0,
                    from arch/x86/include/asm/page.h:14,
                    from arch/x86/include/asm/thread_info.h:12,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   include/linux/string.h:332:24: note: expected 'void *' but argument is of type 'int'
    __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
                           ^~~~~~
   In file included from include/linux/kasan.h:17:0,
                    from include/linux/slab.h:129,
                    from include/linux/irq.h:26,
                    from arch/x86/include/asm/hardirq.h:6,
                    from include/linux/hardirq.h:9,
                    from include/linux/interrupt.h:13,
                    from arch/x86/include/asm/mshyperv.h:8,
                    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:20,
                    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h:1129:35: warning: passing argument 2 of 'memcpy' makes pointer from integer without a cast [-Wint-conversion]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
                                      ^~~~~~~~~~~~~~~~~~~
   In file included from arch/x86/include/asm/page_32.h:35:0,
                    from arch/x86/include/asm/page.h:14,
                    from arch/x86/include/asm/thread_info.h:12,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   include/linux/string.h:332:24: note: expected 'const void *' but argument is of type 'int'
    __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size)
                           ^~~~~~
   In file included from arch/x86/include/asm/paravirt_types.h:45:0,
                    from arch/x86/include/asm/ptrace.h:92,
                    from arch/x86/include/asm/math_emu.h:5,
                    from arch/x86/include/asm/processor.h:12,
                    from arch/x86/include/asm/cpufeature.h:5,
                    from arch/x86/include/asm/thread_info.h:53,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'pte_flags_pkey':
>> arch/x86/include/asm/pgtable_types.h:58:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT0 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:70:26: note: in expansion of macro '_PAGE_PKEY_BIT0'
    #define _PAGE_PKEY_MASK (_PAGE_PKEY_BIT0 | \
                             ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:59:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT1 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:71:5: note: in expansion of macro '_PAGE_PKEY_BIT1'
        _PAGE_PKEY_BIT1 | \
        ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:60:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT2 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT2)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:72:5: note: in expansion of macro '_PAGE_PKEY_BIT2'
        _PAGE_PKEY_BIT2 | \
        ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:61:43: warning: left shift count >= width of type [-Wshift-count-overflow]
    #define _PAGE_PKEY_BIT3 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT3)
                                              ^
>> arch/x86/include/asm/pgtable_types.h:73:5: note: in expansion of macro '_PAGE_PKEY_BIT3'
        _PAGE_PKEY_BIT3)
        ^~~~~~~~~~~~~~~
>> arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                         ^~~~~~~~~~~~~~~
   In file included from include/linux/kasan.h:17:0,
                    from include/linux/slab.h:129,
                    from include/linux/irq.h:26,
                    from arch/x86/include/asm/hardirq.h:6,
                    from include/linux/hardirq.h:9,
                    from include/linux/interrupt.h:13,
                    from arch/x86/include/asm/mshyperv.h:8,
                    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:20,
                    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
>> arch/x86/include/asm/pgtable.h:1223:39: warning: right shift count >= width of type [-Wshift-count-overflow]
     return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
                                          ^~
   cc1: some warnings being treated as errors

sparse warnings: (new ones prefixed by >>)

>> include/linux/kasan.h:31:19: sparse: constant 0xdffffc0000000000UL is so big it is unsigned long long
   In file included from include/linux/kasan.h:17:0,
    from include/linux/slab.h:129,
    from include/linux/irq.h:26,
    from arch/x86/include/asm/hardirq.h:6,
    from include/linux/hardirq.h:9,
    from include/linux/interrupt.h:13,
    from arch/x86/include/asm/mshyperv.h:8,
    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:20,
    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
   arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean
    memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
    ^~~~~~~~~~~~~~~~~~~
    u64_to_user_ptr
   arch/x86/include/asm/pgtable.h:1129:9: warning: passing argument 1 of 'memcpy' makes pointer from integer without a cast
   In file included from arch/x86/include/asm/page_32.h:35:0,
    from arch/x86/include/asm/page.h:14,
    from arch/x86/include/asm/thread_info.h:12,
    from include/linux/thread_info.h:38,
    from arch/x86/include/asm/preempt.h:7,
    from include/linux/preempt.h:81,
    from include/linux/spinlock.h:51,
    from include/linux/seqlock.h:36,
    from include/linux/time.h:6,
    from include/uapi/linux/timex.h:56,
    from include/linux/timex.h:56,
    from include/linux/clocksource.h:13,
    from arch/x86/include/asm/vgtod.h:6,
    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:15,
    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   include/linux/string.h:332:24: note: expected 'void but argument is of type 'int'
    __FORTIFY_INLINE void const void __kernel_size_t size)
    ^~~~~~
   In file included from include/linux/kasan.h:17:0,
    from include/linux/slab.h:129,
    from include/linux/irq.h:26,
    from arch/x86/include/asm/hardirq.h:6,
    from include/linux/hardirq.h:9,
    from include/linux/interrupt.h:13,
    from arch/x86/include/asm/mshyperv.h:8,
    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:20,
    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h:1129:35: warning: passing argument 2 of 'memcpy' makes pointer from integer without a cast
    memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
    ^~~~~~~~~~~~~~~~~~~
   In file included from arch/x86/include/asm/page_32.h:35:0,
    from arch/x86/include/asm/page.h:14,
    from arch/x86/include/asm/thread_info.h:12,
    from include/linux/thread_info.h:38,
    from arch/x86/include/asm/preempt.h:7,
    from include/linux/preempt.h:81,
    from include/linux/spinlock.h:51,
    from include/linux/seqlock.h:36,
    from include/linux/time.h:6,
    from include/uapi/linux/timex.h:56,
    from include/linux/timex.h:56,
    from include/linux/clocksource.h:13,
    from arch/x86/include/asm/vgtod.h:6,
    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:15,
    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   include/linux/string.h:332:24: note: expected 'const void but argument is of type 'int'
    __FORTIFY_INLINE void const void __kernel_size_t size)
    ^~~~~~
   In file included from arch/x86/include/asm/paravirt_types.h:45:0,
    from arch/x86/include/asm/ptrace.h:92,
    from arch/x86/include/asm/math_emu.h:5,
    from arch/x86/include/asm/processor.h:12,
    from arch/x86/include/asm/cpufeature.h:5,
    from arch/x86/include/asm/thread_info.h:53,
    from include/linux/thread_info.h:38,
    from arch/x86/include/asm/preempt.h:7,
    from include/linux/preempt.h:81,
    from include/linux/spinlock.h:51,
    from include/linux/seqlock.h:36,
    from include/linux/time.h:6,
    from include/uapi/linux/timex.h:56,
    from include/linux/timex.h:56,
    from include/linux/clocksource.h:13,
    from arch/x86/include/asm/vgtod.h:6,
    from arch/x86/entry/vdso/vdso32/../vclock_gettime.c:15,
    from arch/x86/entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'pte_flags_pkey':
   arch/x86/include/asm/pgtable_types.h:58:43: warning: left shift count >= width of type
    #define _PAGE_PKEY_BIT0 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0)
    ^
   arch/x86/include/asm/pgtable_types.h:70:26: note: in expansion of macro '_PAGE_PKEY_BIT0'
    #define _PAGE_PKEY_MASK (_PAGE_PKEY_BIT0 | 89- ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
    return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
    ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable_types.h:59:43: warning: left shift count >= width of type
    #define _PAGE_PKEY_BIT1 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1)
    ^
   arch/x86/include/asm/pgtable_types.h:71:5: note: in expansion of macro '_PAGE_PKEY_BIT1'
    _PAGE_PKEY_BIT1 | 98- ^~~~~~~~~~~~~~~
   arch/x86/include/asm/pgtable.h:1223:22: note: in expansion of macro '_PAGE_PKEY_MASK'
    return (pte_flags & _PAGE_PKEY_MASK) >> _PAGE_BIT_PKEY_BIT0;
    ^~~~~~~~~~~~~~~

vim +58 arch/x86/include/asm/pgtable_types.h

8d19c99fa Jeremy Fitzhardinge 2009-02-08  41  
8d19c99fa Jeremy Fitzhardinge 2009-02-08  42  #define _PAGE_PRESENT	(_AT(pteval_t, 1) << _PAGE_BIT_PRESENT)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  43  #define _PAGE_RW	(_AT(pteval_t, 1) << _PAGE_BIT_RW)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  44  #define _PAGE_USER	(_AT(pteval_t, 1) << _PAGE_BIT_USER)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  45  #define _PAGE_PWT	(_AT(pteval_t, 1) << _PAGE_BIT_PWT)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  46  #define _PAGE_PCD	(_AT(pteval_t, 1) << _PAGE_BIT_PCD)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  47  #define _PAGE_ACCESSED	(_AT(pteval_t, 1) << _PAGE_BIT_ACCESSED)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  48  #define _PAGE_DIRTY	(_AT(pteval_t, 1) << _PAGE_BIT_DIRTY)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  49  #define _PAGE_PSE	(_AT(pteval_t, 1) << _PAGE_BIT_PSE)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  50  #define _PAGE_GLOBAL	(_AT(pteval_t, 1) << _PAGE_BIT_GLOBAL)
c46a7c817 Mel Gorman          2014-06-04  51  #define _PAGE_SOFTW1	(_AT(pteval_t, 1) << _PAGE_BIT_SOFTW1)
f955371ca David Vrabel        2014-01-07  52  #define _PAGE_SOFTW2	(_AT(pteval_t, 1) << _PAGE_BIT_SOFTW2)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  53  #define _PAGE_PAT	(_AT(pteval_t, 1) << _PAGE_BIT_PAT)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  54  #define _PAGE_PAT_LARGE (_AT(pteval_t, 1) << _PAGE_BIT_PAT_LARGE)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  55  #define _PAGE_SPECIAL	(_AT(pteval_t, 1) << _PAGE_BIT_SPECIAL)
8d19c99fa Jeremy Fitzhardinge 2009-02-08  56  #define _PAGE_CPA_TEST	(_AT(pteval_t, 1) << _PAGE_BIT_CPA_TEST)
5c1d90f51 Dave Hansen         2016-02-12  57  #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
5c1d90f51 Dave Hansen         2016-02-12 @58  #define _PAGE_PKEY_BIT0	(_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0)
5c1d90f51 Dave Hansen         2016-02-12  59  #define _PAGE_PKEY_BIT1	(_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1)
5c1d90f51 Dave Hansen         2016-02-12  60  #define _PAGE_PKEY_BIT2	(_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT2)
5c1d90f51 Dave Hansen         2016-02-12  61  #define _PAGE_PKEY_BIT3	(_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT3)
5c1d90f51 Dave Hansen         2016-02-12  62  #else
5c1d90f51 Dave Hansen         2016-02-12  63  #define _PAGE_PKEY_BIT0	(_AT(pteval_t, 0))
5c1d90f51 Dave Hansen         2016-02-12  64  #define _PAGE_PKEY_BIT1	(_AT(pteval_t, 0))
5c1d90f51 Dave Hansen         2016-02-12  65  #define _PAGE_PKEY_BIT2	(_AT(pteval_t, 0))
5c1d90f51 Dave Hansen         2016-02-12  66  #define _PAGE_PKEY_BIT3	(_AT(pteval_t, 0))
5c1d90f51 Dave Hansen         2016-02-12  67  #endif
8d19c99fa Jeremy Fitzhardinge 2009-02-08  68  #define __HAVE_ARCH_PTE_SPECIAL
8d19c99fa Jeremy Fitzhardinge 2009-02-08  69  
019132ff3 Dave Hansen         2016-02-12 @70  #define _PAGE_PKEY_MASK (_PAGE_PKEY_BIT0 | \
019132ff3 Dave Hansen         2016-02-12 @71  			 _PAGE_PKEY_BIT1 | \
019132ff3 Dave Hansen         2016-02-12 @72  			 _PAGE_PKEY_BIT2 | \
019132ff3 Dave Hansen         2016-02-12 @73  			 _PAGE_PKEY_BIT3)
019132ff3 Dave Hansen         2016-02-12  74  

:::::: The code at line 58 was first introduced by commit
:::::: 5c1d90f51027e197e1299ab1235a2fed78910905 x86/mm/pkeys: Add PTE bits for storing protection key

:::::: TO: Dave Hansen <dave.hansen@linux.intel.com>
:::::: CC: Ingo Molnar <mingo@kernel.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 62394 bytes --]

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Vitaly Kuznetsov]

kbuild test robot <lkp@intel.com> writes:

> Hi Vitaly,
>
> Thank you for the patch! Perhaps something to improve:
>
> [auto build test WARNING on tip/auto-latest]
> [also build test WARNING on v4.15-rc8 next-20180118]
> [cannot apply to tip/x86/core kvm/linux-next]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url:    https://github.com/0day-ci/linux/commits/Vitaly-Kuznetsov/x86-kvm-hyperv-stable-clocksorce-for-L2-guests-when-running-nested-KVM-on-Hyper-V/20180119-160814
> config: x86_64-allmodconfig (attached as .config)
> compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
> reproduce:
>         # save the attached .config to linux build tree
>         make ARCH=x86_64 
>
> All warnings (new ones prefixed by >>):
>
>    In file included from include/linux/kasan.h:17:0,
>                     from include/linux/slab.h:129,
>                     from include/linux/irq.h:26,
>                     from arch/x86/include/asm/hardirq.h:6,
>                     from include/linux/hardirq.h:9,
>                     from include/linux/interrupt.h:13,
>                     from arch/x86/include/asm/mshyperv.h:8,
>                     from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
>                     from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
>    arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
>    arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-declaration]
>      memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
>             ^~~~~~~~~~~~~~~~~~~

Sorry but I'm failing to see how this (and all the rest) is related to
my patch ...

[snip]

-- 
  Vitaly

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Thomas Gleixner]

On Fri, 19 Jan 2018, Vitaly Kuznetsov wrote:
> kbuild test robot <lkp@intel.com> writes:
> 
> > Hi Vitaly,
> >
> > Thank you for the patch! Perhaps something to improve:
> >
> > [auto build test WARNING on tip/auto-latest]
> > [also build test WARNING on v4.15-rc8 next-20180118]
> > [cannot apply to tip/x86/core kvm/linux-next]
> > [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
> >
> > url:    https://github.com/0day-ci/linux/commits/Vitaly-Kuznetsov/x86-kvm-hyperv-stable-clocksorce-for-L2-guests-when-running-nested-KVM-on-Hyper-V/20180119-160814
> > config: x86_64-allmodconfig (attached as .config)
> > compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
> > reproduce:
> >         # save the attached .config to linux build tree
> >         make ARCH=x86_64 
> >
> > All warnings (new ones prefixed by >>):
> >
> >    In file included from include/linux/kasan.h:17:0,
> >                     from include/linux/slab.h:129,
> >                     from include/linux/irq.h:26,
> >                     from arch/x86/include/asm/hardirq.h:6,
> >                     from include/linux/hardirq.h:9,
> >                     from include/linux/interrupt.h:13,
> >                     from arch/x86/include/asm/mshyperv.h:8,
> >                     from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
> >                     from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
> >    arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
> >    arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-declaration]
> >      memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
> >             ^~~~~~~~~~~~~~~~~~~
> 
> Sorry but I'm failing to see how this (and all the rest) is related to
> my patch ...

You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
vclock_gettime.c and pulls in other stuff which fails to expand....

Thanks,

	tglx

RE: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Michael Kelley (EOSG)]

On Fri, 19 Jan 2018, Thomas Gleixner wrote:

> -----Original Message-----
> From: Thomas Gleixner [mailto:tglx@linutronix.de]
> Sent: Friday, January 19, 2018 11:48 PM
> To: Vitaly Kuznetsov <vkuznets@redhat.com>
> Cc: kbuild test robot <lkp@intel.com>; kbuild-all@01.org; kvm@vger.kernel.org;
> x86@kernel.org; Stephen Hemminger <sthemmin@microsoft.com>; Radim Krčmář
> <rkrcmar@redhat.com>; Haiyang Zhang <haiyangz@microsoft.com>; linux-
> kernel@vger.kernel.org; devel@linuxdriverproject.org; Michael Kelley (EOSG)
> <Michael.H.Kelley@microsoft.com>; Ingo Molnar <mingo@redhat.com>; Roman Kagan
> <rkagan@virtuozzo.com>; Andy Lutomirski <luto@kernel.org>; H. Peter Anvin
> <hpa@zytor.com>; Paolo Bonzini <pbonzini@redhat.com>; Mohammed Gamal
> <mmorsy@redhat.com>
> Subject: Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support
> 
> On Fri, 19 Jan 2018, Vitaly Kuznetsov wrote:
> > kbuild test robot <lkp@intel.com> writes:
> >
> > > Hi Vitaly,
> > >
> > > Thank you for the patch! Perhaps something to improve:
> > >
> > > [auto build test WARNING on tip/auto-latest]
> > > [also build test WARNING on v4.15-rc8 next-20180118]
> > > [cannot apply to tip/x86/core kvm/linux-next]
> > > [if your patch is applied to the wrong git tree, please drop us a note to help improve the
> system]
> > >
> > > url:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2F0day-
> ci%2Flinux%2Fcommits%2FVitaly-Kuznetsov%2Fx86-kvm-hyperv-stable-clocksorce-for-L2-
> guests-when-running-nested-KVM-on-Hyper-V%2F20180119-
> 160814&data=02%7C01%7CMichael.H.Kelley%40microsoft.com%7Ce95c66107da6446826830
> 8d55fda2c2b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636520313062777623&s
> data=kAXl3mLVUdJi%2BsB4Ub0fmUHQfl6NuUDjW%2FAY9%2BFLZE4%3D&reserved=0
> > > config: x86_64-allmodconfig (attached as .config)
> > > compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
> > > reproduce:
> > >         # save the attached .config to linux build tree
> > >         make ARCH=x86_64
> > >
> > > All warnings (new ones prefixed by >>):
> > >
> > >    In file included from include/linux/kasan.h:17:0,
> > >                     from include/linux/slab.h:129,
> > >                     from include/linux/irq.h:26,
> > >                     from arch/x86/include/asm/hardirq.h:6,
> > >                     from include/linux/hardirq.h:9,
> > >                     from include/linux/interrupt.h:13,
> > >                     from arch/x86/include/asm/mshyperv.h:8,
> > >                     from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
> > >                     from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
> > >    arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
> > >    arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function
> 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-
> declaration]
> > >      memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
> > >             ^~~~~~~~~~~~~~~~~~~
> >
> > Sorry but I'm failing to see how this (and all the rest) is related to
> > my patch ...
> 
> You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
> vclock_gettime.c and pulls in other stuff which fails to expand....

Is the declaration of hyperv_reenlightenment_intr() even needed in
mshyperv.h?  The '#include <linux/interrupt.h>' is there for the __irq_entry
annotation on that declaration.   There's a declaration of the parallel (and
unannotated) hyperv_vector_handler(), but that looks like a fossil that
isn't needed either.

Michael

> 
> Thanks,
> 
> 	tglx

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Vitaly Kuznetsov]

"Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com> writes:

> On Fri, 19 Jan 2018, Thomas Gleixner wrote:
>
>> -----Original Message-----
>> From: Thomas Gleixner [mailto:tglx@linutronix.de]
>> Sent: Friday, January 19, 2018 11:48 PM
>> To: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Cc: kbuild test robot <lkp@intel.com>; kbuild-all@01.org; kvm@vger.kernel.org;
>> x86@kernel.org; Stephen Hemminger <sthemmin@microsoft.com>; Radim Krčmář
>> <rkrcmar@redhat.com>; Haiyang Zhang <haiyangz@microsoft.com>; linux-
>> kernel@vger.kernel.org; devel@linuxdriverproject.org; Michael Kelley (EOSG)
>> <Michael.H.Kelley@microsoft.com>; Ingo Molnar <mingo@redhat.com>; Roman Kagan
>> <rkagan@virtuozzo.com>; Andy Lutomirski <luto@kernel.org>; H. Peter Anvin
>> <hpa@zytor.com>; Paolo Bonzini <pbonzini@redhat.com>; Mohammed Gamal
>> <mmorsy@redhat.com>
>> Subject: Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support
>> 
>> On Fri, 19 Jan 2018, Vitaly Kuznetsov wrote:
>> > kbuild test robot <lkp@intel.com> writes:
>> >
>> > > Hi Vitaly,
>> > >
>> > > Thank you for the patch! Perhaps something to improve:
>> > >
>> > > [auto build test WARNING on tip/auto-latest]
>> > > [also build test WARNING on v4.15-rc8 next-20180118]
>> > > [cannot apply to tip/x86/core kvm/linux-next]
>> > > [if your patch is applied to the wrong git tree, please drop us a note to help improve the
>> system]
>> > >
>> > > url:
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2F0day-
>> ci%2Flinux%2Fcommits%2FVitaly-Kuznetsov%2Fx86-kvm-hyperv-stable-clocksorce-for-L2-
>> guests-when-running-nested-KVM-on-Hyper-V%2F20180119-
>> 160814&data=02%7C01%7CMichael.H.Kelley%40microsoft.com%7Ce95c66107da6446826830
>> 8d55fda2c2b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636520313062777623&s
>> data=kAXl3mLVUdJi%2BsB4Ub0fmUHQfl6NuUDjW%2FAY9%2BFLZE4%3D&reserved=0
>> > > config: x86_64-allmodconfig (attached as .config)
>> > > compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
>> > > reproduce:
>> > >         # save the attached .config to linux build tree
>> > >         make ARCH=x86_64
>> > >
>> > > All warnings (new ones prefixed by >>):
>> > >
>> > >    In file included from include/linux/kasan.h:17:0,
>> > >                     from include/linux/slab.h:129,
>> > >                     from include/linux/irq.h:26,
>> > >                     from arch/x86/include/asm/hardirq.h:6,
>> > >                     from include/linux/hardirq.h:9,
>> > >                     from include/linux/interrupt.h:13,
>> > >                     from arch/x86/include/asm/mshyperv.h:8,
>> > >                     from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
>> > >                     from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
>> > >    arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
>> > >    arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function
>> 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-
>> declaration]
>> > >      memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
>> > >             ^~~~~~~~~~~~~~~~~~~
>> >
>> > Sorry but I'm failing to see how this (and all the rest) is related to
>> > my patch ...
>> 
>> You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
>> vclock_gettime.c and pulls in other stuff which fails to expand....
>
> Is the declaration of hyperv_reenlightenment_intr() even needed in
> mshyperv.h?  The '#include <linux/interrupt.h>' is there for the __irq_entry
> annotation on that declaration.   There's a declaration of the parallel (and
> unannotated) hyperv_vector_handler(), but that looks like a fossil that
> isn't needed either.
>

True,

the only need for the declaration in mshyperv.h is to silence "warning:
no previous prototype for ‘hyperv_reenlightenment_intr’"; I'm not sure
if this actually needs fixing.

-- 
  Vitaly

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Vitaly Kuznetsov]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:

> "Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com> writes:
>
>> On Fri, 19 Jan 2018, Thomas Gleixner wrote:
>>
>>> 
>>> You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
>>> vclock_gettime.c and pulls in other stuff which fails to expand....
>>
>> Is the declaration of hyperv_reenlightenment_intr() even needed in
>> mshyperv.h?  The '#include <linux/interrupt.h>' is there for the __irq_entry
>> annotation on that declaration.   There's a declaration of the parallel (and
>> unannotated) hyperv_vector_handler(), but that looks like a fossil that
>> isn't needed either.
>>
>
> True,
>
> the only need for the declaration in mshyperv.h is to silence "warning:
> no previous prototype for ‘hyperv_reenlightenment_intr’"; I'm not sure
> if this actually needs fixing.

It seems we can get away with dropping '__visible' and '__irq_entry'
qualifiers from 'hyperv_reenlightenment_intr' declaration in mshyperv.h
while keeping them in hv_init.c for the implementation. This way we can
avoid the nasty 'no previous prototype' warning and not have to add
'#include <linux/interrupt.h>' to mshyperv.h breaking vdso.

I'm inclined to go ahead with this in v4 if nobody objects.

-- 
  Vitaly
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

RE: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Michael Kelley (EOSG)]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:
> 
> > "Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com> writes:
> >
> >> On Fri, 19 Jan 2018, Thomas Gleixner wrote:
> >>
> >>>
> >>> You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
> >>> vclock_gettime.c and pulls in other stuff which fails to expand....
> >>
> >> Is the declaration of hyperv_reenlightenment_intr() even needed in
> >> mshyperv.h?  The '#include <linux/interrupt.h>' is there for the __irq_entry
> >> annotation on that declaration.   There's a declaration of the parallel (and
> >> unannotated) hyperv_vector_handler(), but that looks like a fossil that
> >> isn't needed either.
> >>
> >
> > True,
> >
> > the only need for the declaration in mshyperv.h is to silence "warning:
> > no previous prototype for ‘hyperv_reenlightenment_intr’"; I'm not sure
> > if this actually needs fixing.
> 
> It seems we can get away with dropping '__visible' and '__irq_entry'
> qualifiers from 'hyperv_reenlightenment_intr' declaration in mshyperv.h
> while keeping them in hv_init.c for the implementation. This way we can
> avoid the nasty 'no previous prototype' warning and not have to add
> '#include <linux/interrupt.h>' to mshyperv.h breaking vdso.

Where exactly is the 'no previous prototype' warning being generated?
I was thinking the only references to hyperv_reenlightenment_intr()
would be in entry_32.S and entry_64.S, but maybe there's another one
I missed.

Michael

> 
> I'm inclined to go ahead with this in v4 if nobody objects.
> 
> --
>   Vitaly

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Vitaly Kuznetsov]

"Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com> writes:

> Vitaly Kuznetsov <vkuznets@redhat.com> writes:
>> 
>> > "Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com> writes:
>> >
>> >> On Fri, 19 Jan 2018, Thomas Gleixner wrote:
>> >>
>> >>>
>> >>> You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
>> >>> vclock_gettime.c and pulls in other stuff which fails to expand....
>> >>
>> >> Is the declaration of hyperv_reenlightenment_intr() even needed in
>> >> mshyperv.h?  The '#include <linux/interrupt.h>' is there for the __irq_entry
>> >> annotation on that declaration.   There's a declaration of the parallel (and
>> >> unannotated) hyperv_vector_handler(), but that looks like a fossil that
>> >> isn't needed either.
>> >>
>> >
>> > True,
>> >
>> > the only need for the declaration in mshyperv.h is to silence "warning:
>> > no previous prototype for ‘hyperv_reenlightenment_intr’"; I'm not sure
>> > if this actually needs fixing.
>> 
>> It seems we can get away with dropping '__visible' and '__irq_entry'
>> qualifiers from 'hyperv_reenlightenment_intr' declaration in mshyperv.h
>> while keeping them in hv_init.c for the implementation. This way we can
>> avoid the nasty 'no previous prototype' warning and not have to add
>> '#include <linux/interrupt.h>' to mshyperv.h breaking vdso.
>
> Where exactly is the 'no previous prototype' warning being generated?

It is generated while compiling the implementation in hv_init.c, every
non-static function needs a previously defined prototype.

-- 
  Vitaly

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[Vitaly Kuznetsov]

Thomas Gleixner <tglx@linutronix.de> writes:

> On Fri, 19 Jan 2018, Vitaly Kuznetsov wrote:
>> kbuild test robot <lkp@intel.com> writes:
>> 
>> > Hi Vitaly,
>> >
>> > Thank you for the patch! Perhaps something to improve:
>> >
>> > [auto build test WARNING on tip/auto-latest]
>> > [also build test WARNING on v4.15-rc8 next-20180118]
>> > [cannot apply to tip/x86/core kvm/linux-next]
>> > [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>> >
>> > url:    https://github.com/0day-ci/linux/commits/Vitaly-Kuznetsov/x86-kvm-hyperv-stable-clocksorce-for-L2-guests-when-running-nested-KVM-on-Hyper-V/20180119-160814
>> > config: x86_64-allmodconfig (attached as .config)
>> > compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
>> > reproduce:
>> >         # save the attached .config to linux build tree
>> >         make ARCH=x86_64 
>> >
>> > All warnings (new ones prefixed by >>):
>> >
>> >    In file included from include/linux/kasan.h:17:0,
>> >                     from include/linux/slab.h:129,
>> >                     from include/linux/irq.h:26,
>> >                     from arch/x86/include/asm/hardirq.h:6,
>> >                     from include/linux/hardirq.h:9,
>> >                     from include/linux/interrupt.h:13,
>> >                     from arch/x86/include/asm/mshyperv.h:8,
>> >                     from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:20,
>> >                     from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
>> >    arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
>> >    arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-declaration]
>> >      memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
>> >             ^~~~~~~~~~~~~~~~~~~
>> 
>> Sorry but I'm failing to see how this (and all the rest) is related to
>> my patch ...
>
> You added '#include <linux/interrupt.h>' to mshyperv.h which is included in
> vclock_gettime.c and pulls in other stuff which fails to expand....
>

Oh, right, thanks! I'll see what can be done.

-- 
  Vitaly

Re: [PATCH v3 3/7] x86/hyper-v: reenlightenment notifications support

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 6746 bytes --]

Hi Vitaly,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on tip/auto-latest]
[also build test WARNING on v4.15-rc8 next-20180118]
[cannot apply to tip/x86/core kvm/linux-next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Vitaly-Kuznetsov/x86-kvm-hyperv-stable-clocksorce-for-L2-guests-when-running-nested-KVM-on-Hyper-V/20180119-160814
config: x86_64-randconfig-s5-01191733 (attached as .config)
compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All warnings (new ones prefixed by >>):

   In file included from arch/x86/include/asm/string.h:3:0,
                    from include/linux/string.h:19,
                    from arch/x86/include/asm/page_32.h:35,
                    from arch/x86/include/asm/page.h:14,
                    from arch/x86/include/asm/thread_info.h:12,
                    from include/linux/thread_info.h:38,
                    from arch/x86/include/asm/preempt.h:7,
                    from include/linux/preempt.h:81,
                    from include/linux/spinlock.h:51,
                    from include/linux/seqlock.h:36,
                    from include/linux/time.h:6,
                    from include/uapi/linux/timex.h:56,
                    from include/linux/timex.h:56,
                    from include/linux/clocksource.h:13,
                    from arch/x86/include/asm/vgtod.h:6,
                    from arch/x86//entry/vdso/vdso32/../vclock_gettime.c:15,
                    from arch/x86//entry/vdso/vdso32/vclock_gettime.c:33:
   arch/x86/include/asm/pgtable.h: In function 'clone_pgd_range':
   arch/x86/include/asm/pgtable.h:1129:9: error: implicit declaration of function 'kernel_to_user_pgdp'; did you mean 'u64_to_user_ptr'? [-Werror=implicit-function-declaration]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
            ^
   arch/x86/include/asm/string_32.h:183:42: note: in definition of macro 'memcpy'
    #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
                                             ^
>> arch/x86/include/asm/pgtable.h:1129:9: warning: passing argument 1 of '__builtin_memcpy' makes pointer from integer without a cast [-Wint-conversion]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
            ^
   arch/x86/include/asm/string_32.h:183:42: note: in definition of macro 'memcpy'
    #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
                                             ^
   arch/x86/include/asm/pgtable.h:1129:9: note: expected 'void *' but argument is of type 'int'
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
            ^
   arch/x86/include/asm/string_32.h:183:42: note: in definition of macro 'memcpy'
    #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
                                             ^
   arch/x86/include/asm/pgtable.h:1129:35: warning: passing argument 2 of '__builtin_memcpy' makes pointer from integer without a cast [-Wint-conversion]
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
                                      ^
   arch/x86/include/asm/string_32.h:183:45: note: in definition of macro 'memcpy'
    #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
                                                ^
   arch/x86/include/asm/pgtable.h:1129:35: note: expected 'const void *' but argument is of type 'int'
     memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
                                      ^
   arch/x86/include/asm/string_32.h:183:45: note: in definition of macro 'memcpy'
    #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
                                                ^
   cc1: some warnings being treated as errors

vim +/__builtin_memcpy +1129 arch/x86/include/asm/pgtable.h

1501899a arch/x86/include/asm/pgtable.h Dan Williams        2017-11-29  1111  
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1112  /*
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1113   * clone_pgd_range(pgd_t *dst, pgd_t *src, int count);
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1114   *
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1115   *  dst - pointer to pgd range anwhere on a pgd page
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1116   *  src - ""
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1117   *  count - the number of pgds to copy.
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1118   *
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1119   * dst and src can be on the same page, but the range must not overlap,
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1120   * and must not cross a page boundary.
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1121   */
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1122  static inline void clone_pgd_range(pgd_t *dst, pgd_t *src, int count)
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1123  {
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1124  	memcpy(dst, src, count * sizeof(pgd_t));
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04  1125  #ifdef CONFIG_PAGE_TABLE_ISOLATION
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04  1126  	if (!static_cpu_has(X86_FEATURE_PTI))
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04  1127  		return;
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04  1128  	/* Clone the user space pgd as well */
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04 @1129  	memcpy(kernel_to_user_pgdp(dst), kernel_to_user_pgdp(src),
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04  1130  	       count * sizeof(pgd_t));
fc2fbc85 arch/x86/include/asm/pgtable.h Dave Hansen         2017-12-04  1131  #endif
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1132  }
85958b46 include/asm-x86/pgtable.h      Jeremy Fitzhardinge 2008-03-17  1133  

:::::: The code at line 1129 was first introduced by commit
:::::: fc2fbc8512ed08d1de7720936fd7d2e4ce02c3a2 x86/mm/pti: Populate user PGD

:::::: TO: Dave Hansen <dave.hansen@linux.intel.com>
:::::: CC: Ingo Molnar <mingo@kernel.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31973 bytes --]

[PATCH v3 4/7] x86/hyper-v: redirect reenlightment notifications on CPU offlining

[Vitaly Kuznetsov]

It is very unlikely for CPUs to get offlined when we run on Hyper-V as
we have a protection in vmbus module which prevents it when we have any
VMBus devices assigned. This, however,  may change in future if an option
to reassign an already active channel will be added. It is also possible
to run without any Hyper-V devices of have a CPU with no assigned channels.

Reassign reenlightenment notifications to some other active CPU when
the CPU which is assigned to get them goes offline.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
v2-> v3:
- Drop spinlock protection from hv_cpu_die() as cpu hotplug is already
  serialized [Thomas Gleixner]
- Use cpumask_any_but() in hv_cpu_die() [Thomas Gleixner]
---
 arch/x86/hyperv/hv_init.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 712ac40081f7..e4377e2f2a10 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -191,6 +191,26 @@ void clear_hv_tscchange_cb(void)
 }
 EXPORT_SYMBOL_GPL(clear_hv_tscchange_cb);
 
+static int hv_cpu_die(unsigned int cpu)
+{
+	struct hv_reenlightenment_control re_ctrl;
+	unsigned int new_cpu;
+
+	if (hv_reenlightenment_cb == NULL)
+		return 0;
+
+	rdmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *((u64 *)&re_ctrl));
+	if (re_ctrl.target_vp == hv_vp_index[cpu]) {
+		/* Reassign to some other online CPU */
+		new_cpu = cpumask_any_but(cpu_online_mask, cpu);
+
+		re_ctrl.target_vp = hv_vp_index[new_cpu];
+		wrmsrl(HV_X64_MSR_REENLIGHTENMENT_CONTROL, *((u64 *)&re_ctrl));
+	}
+
+	return 0;
+}
+
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
@@ -220,7 +240,7 @@ void hyperv_init(void)
 		return;
 
 	if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/hyperv_init:online",
-			      hv_cpu_init, NULL) < 0)
+			      hv_cpu_init, hv_cpu_die) < 0)
 		goto free_vp_index;
 
 	/*
-- 
2.14.3

Re: [PATCH v3 4/7] x86/hyper-v: redirect reenlightment notifications on CPU offlining

[Thomas Gleixner]

On Tue, 16 Jan 2018, Vitaly Kuznetsov wrote:

> It is very unlikely for CPUs to get offlined when we run on Hyper-V as
> we have a protection in vmbus module which prevents it when we have any
> VMBus devices assigned. This, however,  may change in future if an option
> to reassign an already active channel will be added. It is also possible
> to run without any Hyper-V devices of have a CPU with no assigned channels.
> 
> Reassign reenlightenment notifications to some other active CPU when
> the CPU which is assigned to get them goes offline.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

[PATCH v3 5/7] x86/irq: Count Hyper-V reenlightenment interrupts

[Vitaly Kuznetsov]

Hyper-V reenlightenment interrupts arrive when the VM is migrated, we're
not supposed to see many of them. However, it may be important to know
that the event has happened in case we have L2 nested guests.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/hyperv/hv_init.c      | 2 ++
 arch/x86/include/asm/hardirq.h | 3 +++
 arch/x86/kernel/irq.c          | 9 +++++++++
 3 files changed, 14 insertions(+)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index e4377e2f2a10..a3adece392f1 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -147,6 +147,8 @@ __visible void __irq_entry hyperv_reenlightenment_intr(struct pt_regs *regs)
 {
 	entering_ack_irq();
 
+	inc_irq_stat(irq_hv_reenlightenment_count);
+
 	schedule_delayed_work(&hv_reenlightenment_work, HZ/10);
 
 	exiting_irq();
diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h
index 51cc979dd364..7c341a74ec8c 100644
--- a/arch/x86/include/asm/hardirq.h
+++ b/arch/x86/include/asm/hardirq.h
@@ -38,6 +38,9 @@ typedef struct {
 #if IS_ENABLED(CONFIG_HYPERV) || defined(CONFIG_XEN)
 	unsigned int irq_hv_callback_count;
 #endif
+#if IS_ENABLED(CONFIG_HYPERV)
+	unsigned int irq_hv_reenlightenment_count;
+#endif
 } ____cacheline_aligned irq_cpustat_t;
 
 DECLARE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat);
diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c
index 68e1867cca80..45fb4d2565f8 100644
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -141,6 +141,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 				   irq_stats(j)->irq_hv_callback_count);
 		seq_puts(p, "  Hypervisor callback interrupts\n");
 	}
+#endif
+#if IS_ENABLED(CONFIG_HYPERV)
+	if (test_bit(HYPERV_REENLIGHTENMENT_VECTOR, system_vectors)) {
+		seq_printf(p, "%*s: ", prec, "HRE");
+		for_each_online_cpu(j)
+			seq_printf(p, "%10u ",
+				   irq_stats(j)->irq_hv_reenlightenment_count);
+		seq_puts(p, "  Hyper-V reenlightenment interrupts\n");
+	}
 #endif
 	seq_printf(p, "%*s: %10u\n", prec, "ERR", atomic_read(&irq_err_count));
 #if defined(CONFIG_X86_IO_APIC)
-- 
2.14.3

Re: [PATCH v3 5/7] x86/irq: Count Hyper-V reenlightenment interrupts

[Thomas Gleixner]

On Tue, 16 Jan 2018, Vitaly Kuznetsov wrote:

> Hyper-V reenlightenment interrupts arrive when the VM is migrated, we're
> not supposed to see many of them. However, it may be important to know
> that the event has happened in case we have L2 nested guests.
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>

Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

[PATCH v3 6/7] x86/kvm: pass stable clocksource to guests when running nested on Hyper-V

[Vitaly Kuznetsov]

Currently, KVM is able to work in 'masterclock' mode passing
PVCLOCK_TSC_STABLE_BIT to guests when the clocksource we use on the host
is TSC. When running nested on Hyper-V we normally use a different one:
TSC page which is resistant to TSC frequency changes on event like L1
migration. Add support for it in KVM.

The only non-trivial change in the patch is in vgettsc(): when updating
our gtod copy we now need to get both the clockread and tsc value.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/kvm/x86.c | 93 +++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 68 insertions(+), 25 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1cec2c62a0b0..f14e0129c8f5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -67,6 +67,7 @@
 #include <asm/pvclock.h>
 #include <asm/div64.h>
 #include <asm/irq_remapping.h>
+#include <asm/mshyperv.h>
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
@@ -1377,6 +1378,11 @@ static u64 compute_guest_tsc(struct kvm_vcpu *vcpu, s64 kernel_ns)
 	return tsc;
 }
 
+static inline int gtod_is_based_on_tsc(int mode)
+{
+	return mode == VCLOCK_TSC || mode == VCLOCK_HVCLOCK;
+}
+
 static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu)
 {
 #ifdef CONFIG_X86_64
@@ -1396,7 +1402,7 @@ static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu)
 	 * perform request to enable masterclock.
 	 */
 	if (ka->use_master_clock ||
-	    (gtod->clock.vclock_mode == VCLOCK_TSC && vcpus_matched))
+	    (gtod_is_based_on_tsc(gtod->clock.vclock_mode) && vcpus_matched))
 		kvm_make_request(KVM_REQ_MASTERCLOCK_UPDATE, vcpu);
 
 	trace_kvm_track_tsc(vcpu->vcpu_id, ka->nr_vcpus_matched_tsc,
@@ -1459,6 +1465,19 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset)
 	vcpu->arch.tsc_offset = offset;
 }
 
+static inline bool kvm_check_tsc_unstable(void)
+{
+#ifdef CONFIG_X86_64
+	/*
+	 * TSC is marked unstable when we're running on Hyper-V,
+	 * 'TSC page' clocksource is good.
+	 */
+	if (pvclock_gtod_data.clock.vclock_mode == VCLOCK_HVCLOCK)
+		return false;
+#endif
+	return check_tsc_unstable();
+}
+
 void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr)
 {
 	struct kvm *kvm = vcpu->kvm;
@@ -1504,7 +1523,7 @@ void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr)
          */
 	if (synchronizing &&
 	    vcpu->arch.virtual_tsc_khz == kvm->arch.last_tsc_khz) {
-		if (!check_tsc_unstable()) {
+		if (!kvm_check_tsc_unstable()) {
 			offset = kvm->arch.cur_tsc_offset;
 			pr_debug("kvm: matched tsc offset for %llu\n", data);
 		} else {
@@ -1604,18 +1623,43 @@ static u64 read_tsc(void)
 	return last;
 }
 
-static inline u64 vgettsc(u64 *cycle_now)
+static inline u64 vgettsc(u64 *tsc_timestamp, int *mode)
 {
 	long v;
 	struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
+	u64 tsc_pg_val;
+
+	switch (gtod->clock.vclock_mode) {
+	case VCLOCK_HVCLOCK:
+		tsc_pg_val = hv_read_tsc_page_tsc(hv_get_tsc_page(),
+						  tsc_timestamp);
+		if (tsc_pg_val != U64_MAX) {
+			/* TSC page valid */
+			*mode = VCLOCK_HVCLOCK;
+			v = (tsc_pg_val - gtod->clock.cycle_last) &
+				gtod->clock.mask;
+		} else {
+			/* TSC page invalid */
+			*mode = VCLOCK_NONE;
+		}
+		break;
+	case VCLOCK_TSC:
+		*mode = VCLOCK_TSC;
+		*tsc_timestamp = read_tsc();
+		v = (*tsc_timestamp - gtod->clock.cycle_last) &
+			gtod->clock.mask;
+		break;
+	default:
+		*mode = VCLOCK_NONE;
+	}
 
-	*cycle_now = read_tsc();
+	if (*mode == VCLOCK_NONE)
+		*tsc_timestamp = v = 0;
 
-	v = (*cycle_now - gtod->clock.cycle_last) & gtod->clock.mask;
 	return v * gtod->clock.mult;
 }
 
-static int do_monotonic_boot(s64 *t, u64 *cycle_now)
+static int do_monotonic_boot(s64 *t, u64 *tsc_timestamp)
 {
 	struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
 	unsigned long seq;
@@ -1624,9 +1668,8 @@ static int do_monotonic_boot(s64 *t, u64 *cycle_now)
 
 	do {
 		seq = read_seqcount_begin(&gtod->seq);
-		mode = gtod->clock.vclock_mode;
 		ns = gtod->nsec_base;
-		ns += vgettsc(cycle_now);
+		ns += vgettsc(tsc_timestamp, &mode);
 		ns >>= gtod->clock.shift;
 		ns += gtod->boot_ns;
 	} while (unlikely(read_seqcount_retry(&gtod->seq, seq)));
@@ -1635,7 +1678,7 @@ static int do_monotonic_boot(s64 *t, u64 *cycle_now)
 	return mode;
 }
 
-static int do_realtime(struct timespec *ts, u64 *cycle_now)
+static int do_realtime(struct timespec *ts, u64 *tsc_timestamp)
 {
 	struct pvclock_gtod_data *gtod = &pvclock_gtod_data;
 	unsigned long seq;
@@ -1644,10 +1687,9 @@ static int do_realtime(struct timespec *ts, u64 *cycle_now)
 
 	do {
 		seq = read_seqcount_begin(&gtod->seq);
-		mode = gtod->clock.vclock_mode;
 		ts->tv_sec = gtod->wall_time_sec;
 		ns = gtod->nsec_base;
-		ns += vgettsc(cycle_now);
+		ns += vgettsc(tsc_timestamp, &mode);
 		ns >>= gtod->clock.shift;
 	} while (unlikely(read_seqcount_retry(&gtod->seq, seq)));
 
@@ -1657,25 +1699,26 @@ static int do_realtime(struct timespec *ts, u64 *cycle_now)
 	return mode;
 }
 
-/* returns true if host is using tsc clocksource */
-static bool kvm_get_time_and_clockread(s64 *kernel_ns, u64 *cycle_now)
+/* returns true if host is using TSC based clocksource */
+static bool kvm_get_time_and_clockread(s64 *kernel_ns, u64 *tsc_timestamp)
 {
 	/* checked again under seqlock below */
-	if (pvclock_gtod_data.clock.vclock_mode != VCLOCK_TSC)
+	if (!gtod_is_based_on_tsc(pvclock_gtod_data.clock.vclock_mode))
 		return false;
 
-	return do_monotonic_boot(kernel_ns, cycle_now) == VCLOCK_TSC;
+	return gtod_is_based_on_tsc(do_monotonic_boot(kernel_ns,
+						      tsc_timestamp));
 }
 
-/* returns true if host is using tsc clocksource */
+/* returns true if host is using TSC based clocksource */
 static bool kvm_get_walltime_and_clockread(struct timespec *ts,
-					   u64 *cycle_now)
+					   u64 *tsc_timestamp)
 {
 	/* checked again under seqlock below */
-	if (pvclock_gtod_data.clock.vclock_mode != VCLOCK_TSC)
+	if (!gtod_is_based_on_tsc(pvclock_gtod_data.clock.vclock_mode))
 		return false;
 
-	return do_realtime(ts, cycle_now) == VCLOCK_TSC;
+	return gtod_is_based_on_tsc(do_realtime(ts, tsc_timestamp));
 }
 #endif
 
@@ -2869,13 +2912,13 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
 	}
 
-	if (unlikely(vcpu->cpu != cpu) || check_tsc_unstable()) {
+	if (unlikely(vcpu->cpu != cpu) || kvm_check_tsc_unstable()) {
 		s64 tsc_delta = !vcpu->arch.last_host_tsc ? 0 :
 				rdtsc() - vcpu->arch.last_host_tsc;
 		if (tsc_delta < 0)
 			mark_tsc_unstable("KVM discovered backwards TSC");
 
-		if (check_tsc_unstable()) {
+		if (kvm_check_tsc_unstable()) {
 			u64 offset = kvm_compute_tsc_offset(vcpu,
 						vcpu->arch.last_guest_tsc);
 			kvm_vcpu_write_tsc_offset(vcpu, offset);
@@ -6110,9 +6153,9 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused,
 	update_pvclock_gtod(tk);
 
 	/* disable master clock if host does not trust, or does not
-	 * use, TSC clocksource
+	 * use, TSC based clocksource.
 	 */
-	if (gtod->clock.vclock_mode != VCLOCK_TSC &&
+	if (!gtod_is_based_on_tsc(gtod->clock.vclock_mode) &&
 	    atomic_read(&kvm_guest_has_master_clock) != 0)
 		queue_work(system_long_wq, &pvclock_gtod_work);
 
@@ -7767,7 +7810,7 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm,
 {
 	struct kvm_vcpu *vcpu;
 
-	if (check_tsc_unstable() && atomic_read(&kvm->online_vcpus) != 0)
+	if (kvm_check_tsc_unstable() && atomic_read(&kvm->online_vcpus) != 0)
 		printk_once(KERN_WARNING
 		"kvm: SMP vm created on host with unstable TSC; "
 		"guest TSC will not be reliable\n");
@@ -7924,7 +7967,7 @@ int kvm_arch_hardware_enable(void)
 		return ret;
 
 	local_tsc = rdtsc();
-	stable = !check_tsc_unstable();
+	stable = !kvm_check_tsc_unstable();
 	list_for_each_entry(kvm, &vm_list, vm_list) {
 		kvm_for_each_vcpu(i, vcpu, kvm) {
 			if (!stable && vcpu->cpu == smp_processor_id())
-- 
2.14.3

[PATCH v3 7/7] x86/kvm: support Hyper-V reenlightenment

[Vitaly Kuznetsov]

When we run nested KVM on Hyper-V guests we need to update masterclocks for
all guests when L1 migrates to a host with different TSC frequency.
Implement the procedure in the following way:
- Pause all guests.
- Tell our host (Hyper-V) to stop emulating TSC accesses.
- Update our gtod copy, recompute clocks.
- Unpause all guests.

This is somewhat similar to cpufreq but we have two important differences:
we can only disable TSC emulation globally (on all CPUs) and we don't know
the new TSC frequency until we turn the emulation off so we can't
'prepare' ourselves to the event.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/kvm/x86.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index f14e0129c8f5..94f28e6002b2 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -68,6 +68,7 @@
 #include <asm/div64.h>
 #include <asm/irq_remapping.h>
 #include <asm/mshyperv.h>
+#include <asm/hypervisor.h>
 
 #define CREATE_TRACE_POINTS
 #include "trace.h"
@@ -5932,6 +5933,43 @@ static void tsc_khz_changed(void *data)
 	__this_cpu_write(cpu_tsc_khz, khz);
 }
 
+void kvm_hyperv_tsc_notifier(void)
+{
+#ifdef CONFIG_X86_64
+	struct kvm *kvm;
+	struct kvm_vcpu *vcpu;
+	int cpu;
+
+	spin_lock(&kvm_lock);
+	list_for_each_entry(kvm, &vm_list, vm_list)
+		kvm_make_mclock_inprogress_request(kvm);
+
+	hyperv_stop_tsc_emulation();
+
+	/* TSC frequency always matches when on Hyper-V */
+	for_each_present_cpu(cpu)
+		per_cpu(cpu_tsc_khz, cpu) = tsc_khz;
+	kvm_max_guest_tsc_khz = tsc_khz;
+
+	list_for_each_entry(kvm, &vm_list, vm_list) {
+		struct kvm_arch *ka = &kvm->arch;
+
+		spin_lock(&ka->pvclock_gtod_sync_lock);
+
+		pvclock_update_vm_gtod_copy(kvm);
+
+		kvm_for_each_vcpu(cpu, vcpu, kvm)
+			kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+
+		kvm_for_each_vcpu(cpu, vcpu, kvm)
+			kvm_clear_request(KVM_REQ_MCLOCK_INPROGRESS, vcpu);
+
+		spin_unlock(&ka->pvclock_gtod_sync_lock);
+	}
+	spin_unlock(&kvm_lock);
+#endif
+}
+
 static int kvmclock_cpufreq_notifier(struct notifier_block *nb, unsigned long val,
 				     void *data)
 {
@@ -6217,6 +6255,9 @@ int kvm_arch_init(void *opaque)
 	kvm_lapic_init();
 #ifdef CONFIG_X86_64
 	pvclock_gtod_register_notifier(&pvclock_gtod_notifier);
+
+	if (x86_hyper_type == X86_HYPER_MS_HYPERV)
+		set_hv_tscchange_cb(kvm_hyperv_tsc_notifier);
 #endif
 
 	return 0;
@@ -6229,6 +6270,10 @@ int kvm_arch_init(void *opaque)
 
 void kvm_arch_exit(void)
 {
+#ifdef CONFIG_X86_64
+	if (x86_hyper_type == X86_HYPER_MS_HYPERV)
+		clear_hv_tscchange_cb();
+#endif
 	kvm_lapic_exit();
 	perf_unregister_guest_info_callbacks(&kvm_guest_cbs);
 
-- 
2.14.3

[RFC PATCH] x86/kvm: kvm_hyperv_tsc_notifier() can be static

[kbuild test robot]

Fixes: ce386b0e39a0 ("x86/kvm: support Hyper-V reenlightenment")
Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
---
 x86.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 94f28e6..27e590a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5933,7 +5933,7 @@ static void tsc_khz_changed(void *data)
 	__this_cpu_write(cpu_tsc_khz, khz);
 }
 
-void kvm_hyperv_tsc_notifier(void)
+static void kvm_hyperv_tsc_notifier(void)
 {
 #ifdef CONFIG_X86_64
 	struct kvm *kvm;

Re: [PATCH v3 7/7] x86/kvm: support Hyper-V reenlightenment

[kbuild test robot]

Hi Vitaly,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on tip/auto-latest]
[also build test WARNING on v4.15-rc8 next-20180118]
[cannot apply to tip/x86/core kvm/linux-next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Vitaly-Kuznetsov/x86-kvm-hyperv-stable-clocksorce-for-L2-guests-when-running-nested-KVM-on-Hyper-V/20180119-160814
reproduce:
        # apt-get install sparse
        make ARCH=x86_64 allmodconfig
        make C=1 CF=-D__CHECK_ENDIAN__


sparse warnings: (new ones prefixed by >>)

   arch/x86/kvm/x86.c:2120:38: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const @@ got d const @@
   arch/x86/kvm/x86.c:2120:38: expected void const
   arch/x86/kvm/x86.c:2120:38: got unsigned char COPYING CREDITS Documentation Kbuild Kconfig MAINTAINERS Makefile README arch block certs crypto drivers firmware fs include init ipc kernel lib mm net samples scripts security sound tools usr virt
>> arch/x86/kvm/x86.c:5936:6: sparse: symbol 'kvm_hyperv_tsc_notifier' was not declared. Should it be
   arch/x86/kvm/x86.c:7585:5: sparse: symbol 'kvm_valid_sregs' was not declared. Should it be
   arch/x86/kvm/x86.c:8401:16: sparse: incompatible types in comparison expression (different address spaces)
   arch/x86/include/asm/paravirt.h:782:9: sparse: context imbalance in 'vcpu_enter_guest' - unexpected unlock

Please review and possibly fold the followup patch.

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation