* Re: [PATCH] x86/hyper-v: micro-optimize send_ipi_one case
From: Vitaly Kuznetsov @ 2019-10-25 10:44 UTC (permalink / raw)
To: Roman Kagan
Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
x86@kernel.org, K. Y. Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Michael Kelley
In-Reply-To: <20191024163204.GA4673@rkaganb.sw.ru>
Roman Kagan <rkagan@virtuozzo.com> writes:
> On Thu, Oct 24, 2019 at 05:21:52PM +0200, Vitaly Kuznetsov wrote:
>> When sending an IPI to a single CPU there is no need to deal with cpumasks.
>> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
>> cycles) improvement with smp_call_function_single() loop benchmark. The
>> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
>> important for PV spinlock kick.
>>
>> I was also wondering if it would make sense to switch to using regular
>> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
>> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
>> vector)).
>
> Is it with APICv or emulated apic?
That's actually a good question. Yesterday I was testing this on WS2019
host with Xeon e5-2420 v2 (Ivy Bridge EN) which I *think* should already
support APICv - but I'm not sure and ark.intel.com is not
helpful. Today, I decided to re-test on something more modern and I got
WS2016 host with E5-2667 v4 (Broadwell) and the results are:
'Ex' hypercall: 18000 cycles
orig_apic.send_IPI(): 46000 cycles
I'm, however, just assuming that Hyper-V uses APICv when it's available
and have no idea how to check from within the guest. I'm also not sure
if WS2019 is so much faster or if there are other differences on these
hosts which matter.
>
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>> arch/x86/hyperv/hv_apic.c | 22 +++++++++++++++++++---
>> arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
>> 2 files changed, 34 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
>> index e01078e93dd3..847f9d0328fe 100644
>> --- a/arch/x86/hyperv/hv_apic.c
>> +++ b/arch/x86/hyperv/hv_apic.c
>> @@ -194,10 +194,26 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector)
>>
>> static bool __send_ipi_one(int cpu, int vector)
>> {
>> - struct cpumask mask = CPU_MASK_NONE;
>> + int ret;
>>
>> - cpumask_set_cpu(cpu, &mask);
>> - return __send_ipi_mask(&mask, vector);
>> + trace_hyperv_send_ipi_one(cpu, vector);
>> +
>> + if (unlikely(!hv_hypercall_pg))
>> + return false;
>> +
>> + if (unlikely((vector < HV_IPI_LOW_VECTOR) ||
>> + (vector > HV_IPI_HIGH_VECTOR)))
>> + return false;
>
> I guess 'ulikely' is unnecessary in these cases.
>
All I can say is that the resulting asm with my gcc is a bit different
:-)
>> +
>> + if (cpu >= 64)
>> + goto do_ex_hypercall;
>> +
>> + ret = hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector,
>> + BIT_ULL(hv_cpu_number_to_vp_number(cpu)));
>> + return ((ret == 0) ? true : false);
>
> D'oh. Isn't "return ret == 0;" or just "return ret;" good enough?
That's how we do stuff in __send_ipi_mask() :-) I'll send v2
implementing Joe's suggestion to drop 'ret' and just do
return !hv_do_fast_hypercall16().
>
> These tiny nitpicks are no reason to hold the patch though, so
>
> Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
Thanks!
--
Vitaly
^ permalink raw reply
* [PATCH v2] x86/hyper-v: micro-optimize send_ipi_one case
From: Vitaly Kuznetsov @ 2019-10-25 13:15 UTC (permalink / raw)
To: linux-hyperv
Cc: linux-kernel, x86, K. Y. Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Roman Kagan, Michael Kelley,
Joe Perches
When sending an IPI to a single CPU there is no need to deal with cpumasks.
With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
cycles) improvement with smp_call_function_single() loop benchmark. The
optimization, however, is tiny and straitforward. Also, send_ipi_one() is
important for PV spinlock kick.
I was also wondering if it would make sense to switch to using regular
APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
vector)).
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
Changes since v1:
- Style changes [Roman, Joe]
---
arch/x86/hyperv/hv_apic.c | 13 ++++++++++---
arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
2 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
index e01078e93dd3..fd17c6341737 100644
--- a/arch/x86/hyperv/hv_apic.c
+++ b/arch/x86/hyperv/hv_apic.c
@@ -194,10 +194,17 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector)
static bool __send_ipi_one(int cpu, int vector)
{
- struct cpumask mask = CPU_MASK_NONE;
+ trace_hyperv_send_ipi_one(cpu, vector);
- cpumask_set_cpu(cpu, &mask);
- return __send_ipi_mask(&mask, vector);
+ if (!hv_hypercall_pg || (vector < HV_IPI_LOW_VECTOR) ||
+ (vector > HV_IPI_HIGH_VECTOR))
+ return false;
+
+ if (cpu >= 64)
+ return __send_ipi_mask_ex(cpumask_of(cpu), vector);
+
+ return !hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector,
+ BIT_ULL(hv_cpu_number_to_vp_number(cpu)));
}
static void hv_send_ipi(int cpu, int vector)
diff --git a/arch/x86/include/asm/trace/hyperv.h b/arch/x86/include/asm/trace/hyperv.h
index ace464f09681..4d705cb4d63b 100644
--- a/arch/x86/include/asm/trace/hyperv.h
+++ b/arch/x86/include/asm/trace/hyperv.h
@@ -71,6 +71,21 @@ TRACE_EVENT(hyperv_send_ipi_mask,
__entry->ncpus, __entry->vector)
);
+TRACE_EVENT(hyperv_send_ipi_one,
+ TP_PROTO(int cpu,
+ int vector),
+ TP_ARGS(cpu, vector),
+ TP_STRUCT__entry(
+ __field(int, cpu)
+ __field(int, vector)
+ ),
+ TP_fast_assign(__entry->cpu = cpu;
+ __entry->vector = vector;
+ ),
+ TP_printk("cpu %d vector %x",
+ __entry->cpu, __entry->vector)
+ );
+
#endif /* CONFIG_HYPERV */
#undef TRACE_INCLUDE_PATH
--
2.20.1
^ permalink raw reply related
* Re: [PATCH v2] x86/hyper-v: micro-optimize send_ipi_one case
From: Roman Kagan @ 2019-10-25 14:05 UTC (permalink / raw)
To: Vitaly Kuznetsov
Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
x86@kernel.org, K. Y. Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Michael Kelley, Joe Perches
In-Reply-To: <20191025131546.18794-1-vkuznets@redhat.com>
On Fri, Oct 25, 2019 at 03:15:46PM +0200, Vitaly Kuznetsov wrote:
> When sending an IPI to a single CPU there is no need to deal with cpumasks.
> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
> cycles) improvement with smp_call_function_single() loop benchmark. The
> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
> important for PV spinlock kick.
>
> I was also wondering if it would make sense to switch to using regular
> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
> vector)).
>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> Changes since v1:
> - Style changes [Roman, Joe]
> ---
> arch/x86/hyperv/hv_apic.c | 13 ++++++++++---
> arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
> 2 files changed, 25 insertions(+), 3 deletions(-)
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
^ permalink raw reply
* RE: [PATCH] x86/hyper-v: micro-optimize send_ipi_one case
From: Michael Kelley @ 2019-10-25 16:23 UTC (permalink / raw)
To: vkuznets, Roman Kagan
Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
x86@kernel.org, KY Srinivasan, Haiyang Zhang, Stephen Hemminger,
Sasha Levin, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin
In-Reply-To: <87r231xfyg.fsf@vitty.brq.redhat.com>
From: Vitaly Kuznetsov <vkuznets@redhat.com> Sent: Friday, October 25, 2019 3:44 AM
>
> Roman Kagan <rkagan@virtuozzo.com> writes:
>
> > On Thu, Oct 24, 2019 at 05:21:52PM +0200, Vitaly Kuznetsov wrote:
> >> When sending an IPI to a single CPU there is no need to deal with cpumasks.
> >> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
> >> cycles) improvement with smp_call_function_single() loop benchmark. The
> >> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
> >> important for PV spinlock kick.
> >>
> >> I was also wondering if it would make sense to switch to using regular
> >> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
> >> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
> >> vector)).
> >
> > Is it with APICv or emulated apic?
>
> That's actually a good question. Yesterday I was testing this on WS2019
> host with Xeon e5-2420 v2 (Ivy Bridge EN) which I *think* should already
> support APICv - but I'm not sure and ark.intel.com is not
> helpful. Today, I decided to re-test on something more modern and I got
> WS2016 host with E5-2667 v4 (Broadwell) and the results are:
>
> 'Ex' hypercall: 18000 cycles
> orig_apic.send_IPI(): 46000 cycles
>
> I'm, however, just assuming that Hyper-V uses APICv when it's available
> and have no idea how to check from within the guest. I'm also not sure
> if WS2019 is so much faster or if there are other differences on these
> hosts which matter.
>
On Hyper-V 2016 and 2019 (not sure about 2012 R2), and when the guest is
using xAPIC (not x2APIC), you can tell within the guest whether Intel APICv is
enabled based on the setting of HV_X64_APIC_ACCESS_RECOMMENDED. If
this flag is set, then APICv is not present, because Hyper-V only recommends
using the synthetic MSRs when APICv is not present. Conversely, if the flag is
not set, then APICv is present.
FWIW, when APICv is present in the hardware, you can disable its use in a
particular VM by using Powershell in the host to set compatibility mode on
the VM:
Set-VMProcessor <vmname> -CompatibilityForMigrationEnabled $true
Then when the VM is booted, HV_X64_APIC_ACCESS_RECOMMENDED
will show as set even if the hardware has APICv. This is useful for testing
the older code paths on hardware that has APICv.
Michael
^ permalink raw reply
* RE: [PATCH v2] x86/hyper-v: micro-optimize send_ipi_one case
From: Michael Kelley @ 2019-10-25 17:16 UTC (permalink / raw)
To: vkuznets, linux-hyperv@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, x86@kernel.org, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Sasha Levin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, H. Peter Anvin, Roman Kagan,
Joe Perches
In-Reply-To: <20191025131546.18794-1-vkuznets@redhat.com>
From: Vitaly Kuznetsov <vkuznets@redhat.com>
>
> When sending an IPI to a single CPU there is no need to deal with cpumasks.
> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
> cycles) improvement with smp_call_function_single() loop benchmark. The
> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
> important for PV spinlock kick.
>
> I was also wondering if it would make sense to switch to using regular
> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
> vector)).
>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> Changes since v1:
> - Style changes [Roman, Joe]
> ---
> arch/x86/hyperv/hv_apic.c | 13 ++++++++++---
> arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
> 2 files changed, 25 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
> index e01078e93dd3..fd17c6341737 100644
> --- a/arch/x86/hyperv/hv_apic.c
> +++ b/arch/x86/hyperv/hv_apic.c
> @@ -194,10 +194,17 @@ static bool __send_ipi_mask(const struct cpumask *mask, int
> vector)
>
> static bool __send_ipi_one(int cpu, int vector)
> {
> - struct cpumask mask = CPU_MASK_NONE;
> + trace_hyperv_send_ipi_one(cpu, vector);
>
> - cpumask_set_cpu(cpu, &mask);
> - return __send_ipi_mask(&mask, vector);
> + if (!hv_hypercall_pg || (vector < HV_IPI_LOW_VECTOR) ||
> + (vector > HV_IPI_HIGH_VECTOR))
> + return false;
> +
> + if (cpu >= 64)
> + return __send_ipi_mask_ex(cpumask_of(cpu), vector);
The above test should be checking the VP number, not the CPU
number, since the VP number is used to form the bitmap argument
to the hypercall. In all current implementations of Hyper-V, the CPU number
and VP number are the same as far as I am aware, but that's not guaranteed in
the future.
Michael
> +
> + return !hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector,
> + BIT_ULL(hv_cpu_number_to_vp_number(cpu)));
> }
>
^ permalink raw reply
* RE: [PATCH v2] x86/hyper-v: micro-optimize send_ipi_one case
From: Vitaly Kuznetsov @ 2019-10-25 17:26 UTC (permalink / raw)
To: Michael Kelley, linux-hyperv@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, x86@kernel.org, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Sasha Levin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, H. Peter Anvin, Roman Kagan,
Joe Perches
In-Reply-To: <DM5PR21MB013707183D9E271E60FBD435D7650@DM5PR21MB0137.namprd21.prod.outlook.com>
Michael Kelley <mikelley@microsoft.com> writes:
> From: Vitaly Kuznetsov <vkuznets@redhat.com>
>>
>> When sending an IPI to a single CPU there is no need to deal with cpumasks.
>> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
>> cycles) improvement with smp_call_function_single() loop benchmark. The
>> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
>> important for PV spinlock kick.
>>
>> I was also wondering if it would make sense to switch to using regular
>> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
>> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
>> vector)).
>>
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>> Changes since v1:
>> - Style changes [Roman, Joe]
>> ---
>> arch/x86/hyperv/hv_apic.c | 13 ++++++++++---
>> arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
>> 2 files changed, 25 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
>> index e01078e93dd3..fd17c6341737 100644
>> --- a/arch/x86/hyperv/hv_apic.c
>> +++ b/arch/x86/hyperv/hv_apic.c
>> @@ -194,10 +194,17 @@ static bool __send_ipi_mask(const struct cpumask *mask, int
>> vector)
>>
>> static bool __send_ipi_one(int cpu, int vector)
>> {
>> - struct cpumask mask = CPU_MASK_NONE;
>> + trace_hyperv_send_ipi_one(cpu, vector);
>>
>> - cpumask_set_cpu(cpu, &mask);
>> - return __send_ipi_mask(&mask, vector);
>> + if (!hv_hypercall_pg || (vector < HV_IPI_LOW_VECTOR) ||
>> + (vector > HV_IPI_HIGH_VECTOR))
>> + return false;
>> +
>> + if (cpu >= 64)
>> + return __send_ipi_mask_ex(cpumask_of(cpu), vector);
>
> The above test should be checking the VP number, not the CPU
> number,
Oops, of course, thanks for catching this! v3 is coming!
> since the VP number is used to form the bitmap argument
> to the hypercall. In all current implementations of Hyper-V, the CPU number
> and VP number are the same as far as I am aware, but that's not guaranteed in
> the future.
>
> Michael
>
>> +
>> + return !hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector,
>> + BIT_ULL(hv_cpu_number_to_vp_number(cpu)));
>> }
>>
--
Vitaly
^ permalink raw reply
* [PATCH AUTOSEL 4.14 29/33] x86/hyperv: Set pv_info.name to "Hyper-V"
From: Sasha Levin @ 2019-10-26 13:21 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Andrea Parri, Michael Kelley, Thomas Gleixner, Wei Liu,
YueHaibing, Sasha Levin, linux-hyperv
In-Reply-To: <20191026132110.4026-1-sashal@kernel.org>
From: Andrea Parri <parri.andrea@gmail.com>
[ Upstream commit f7c0f50f1857c1cf013466fcea4dc98d116bf456 ]
Michael reported that the x86/hyperv initialization code prints the
following dmesg when running in a VM on Hyper-V:
[ 0.000738] Booting paravirtualized kernel on bare hardware
Let the x86/hyperv initialization code set pv_info.name to "Hyper-V" so
dmesg reports correctly:
[ 0.000172] Booting paravirtualized kernel on Hyper-V
[ tglx: Folded build fix provided by Yue ]
Reported-by: Michael Kelley <mikelley@microsoft.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Wei Liu <wei.liu@kernel.org>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Cc: YueHaibing <yuehaibing@huawei.com>
Link: https://lkml.kernel.org/r/20191015103502.13156-1-parri.andrea@gmail.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kernel/cpu/mshyperv.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index c0201b11e9e2a..4e7adccf812d4 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -171,6 +171,10 @@ static void __init ms_hyperv_init_platform(void)
int hv_host_info_ecx;
int hv_host_info_edx;
+#ifdef CONFIG_PARAVIRT
+ pv_info.name = "Hyper-V";
+#endif
+
/*
* Extract the features and hints
*/
--
2.20.1
^ permalink raw reply related
* [PATCH AUTOSEL 4.19 55/59] x86/hyperv: Set pv_info.name to "Hyper-V"
From: Sasha Levin @ 2019-10-26 13:19 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Andrea Parri, Michael Kelley, Thomas Gleixner, Wei Liu,
YueHaibing, Sasha Levin, linux-hyperv
In-Reply-To: <20191026131910.3435-1-sashal@kernel.org>
From: Andrea Parri <parri.andrea@gmail.com>
[ Upstream commit f7c0f50f1857c1cf013466fcea4dc98d116bf456 ]
Michael reported that the x86/hyperv initialization code prints the
following dmesg when running in a VM on Hyper-V:
[ 0.000738] Booting paravirtualized kernel on bare hardware
Let the x86/hyperv initialization code set pv_info.name to "Hyper-V" so
dmesg reports correctly:
[ 0.000172] Booting paravirtualized kernel on Hyper-V
[ tglx: Folded build fix provided by Yue ]
Reported-by: Michael Kelley <mikelley@microsoft.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Wei Liu <wei.liu@kernel.org>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Cc: YueHaibing <yuehaibing@huawei.com>
Link: https://lkml.kernel.org/r/20191015103502.13156-1-parri.andrea@gmail.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kernel/cpu/mshyperv.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 852e74e48890b..1bec5e4bb1fa9 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -207,6 +207,10 @@ static void __init ms_hyperv_init_platform(void)
int hv_host_info_ecx;
int hv_host_info_edx;
+#ifdef CONFIG_PARAVIRT
+ pv_info.name = "Hyper-V";
+#endif
+
/*
* Extract the features and hints
*/
--
2.20.1
^ permalink raw reply related
* [PATCH AUTOSEL 5.3 89/99] x86/hyperv: Set pv_info.name to "Hyper-V"
From: Sasha Levin @ 2019-10-26 13:15 UTC (permalink / raw)
To: linux-kernel, stable
Cc: Andrea Parri, Michael Kelley, Thomas Gleixner, Wei Liu,
YueHaibing, Sasha Levin, linux-hyperv
In-Reply-To: <20191026131600.2507-1-sashal@kernel.org>
From: Andrea Parri <parri.andrea@gmail.com>
[ Upstream commit f7c0f50f1857c1cf013466fcea4dc98d116bf456 ]
Michael reported that the x86/hyperv initialization code prints the
following dmesg when running in a VM on Hyper-V:
[ 0.000738] Booting paravirtualized kernel on bare hardware
Let the x86/hyperv initialization code set pv_info.name to "Hyper-V" so
dmesg reports correctly:
[ 0.000172] Booting paravirtualized kernel on Hyper-V
[ tglx: Folded build fix provided by Yue ]
Reported-by: Michael Kelley <mikelley@microsoft.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Wei Liu <wei.liu@kernel.org>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Cc: YueHaibing <yuehaibing@huawei.com>
Link: https://lkml.kernel.org/r/20191015103502.13156-1-parri.andrea@gmail.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
arch/x86/kernel/cpu/mshyperv.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 062f77279ce3b..b3f164bf8eefa 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -215,6 +215,10 @@ static void __init ms_hyperv_init_platform(void)
int hv_host_info_ecx;
int hv_host_info_edx;
+#ifdef CONFIG_PARAVIRT
+ pv_info.name = "Hyper-V";
+#endif
+
/*
* Extract the features and hints
*/
--
2.20.1
^ permalink raw reply related
* Re: [PATCH net-next 12/14] vsock/vmci: register vmci_transport only when VMCI guest/host are active
From: Stefan Hajnoczi @ 2019-10-27 8:17 UTC (permalink / raw)
To: Stefano Garzarella
Cc: netdev, Sasha Levin, linux-hyperv, Stephen Hemminger,
Arnd Bergmann, kvm, Michael S. Tsirkin, Greg Kroah-Hartman,
Dexuan Cui, linux-kernel, virtualization, Haiyang Zhang,
Stefan Hajnoczi, David S. Miller, Jorgen Hansen
In-Reply-To: <20191023095554.11340-13-sgarzare@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 194 bytes --]
On Wed, Oct 23, 2019 at 11:55:52AM +0200, Stefano Garzarella wrote:
> +static int __init vmci_transport_init(void)
> +{
> + int features = VSOCK_TRANSPORT_F_DGRAM;
Where is this variable used?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* Re: [PATCH net-next 08/14] vsock: add vsock_create_connected() called by transports
From: Stefan Hajnoczi @ 2019-10-27 8:12 UTC (permalink / raw)
To: Stefano Garzarella
Cc: netdev, Sasha Levin, linux-hyperv, Stephen Hemminger,
Arnd Bergmann, kvm, Michael S. Tsirkin, Greg Kroah-Hartman,
Dexuan Cui, linux-kernel, virtualization, Haiyang Zhang,
Stefan Hajnoczi, David S. Miller, Jorgen Hansen
In-Reply-To: <20191023095554.11340-9-sgarzare@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 954 bytes --]
On Wed, Oct 23, 2019 at 11:55:48AM +0200, Stefano Garzarella wrote:
> All transports call __vsock_create() with the same parameters,
> most of them depending on the parent socket. In order to simplify
> the VSOCK core APIs exposed to the transports, this patch adds
> the vsock_create_connected() callable from transports to create
> a new socket when a connection request is received.
> We also unexported the __vsock_create().
>
> Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
> ---
> include/net/af_vsock.h | 5 +----
> net/vmw_vsock/af_vsock.c | 20 +++++++++++++-------
> net/vmw_vsock/hyperv_transport.c | 3 +--
> net/vmw_vsock/virtio_transport_common.c | 3 +--
> net/vmw_vsock/vmci_transport.c | 3 +--
> 5 files changed, 17 insertions(+), 17 deletions(-)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* Re: [PATCH net-next 07/14] vsock: handle buffer_size sockopts in the core
From: Stefan Hajnoczi @ 2019-10-27 8:08 UTC (permalink / raw)
To: Stefano Garzarella
Cc: netdev, Sasha Levin, linux-hyperv, Stephen Hemminger,
Arnd Bergmann, kvm, Michael S. Tsirkin, Greg Kroah-Hartman,
Dexuan Cui, linux-kernel, virtualization, Haiyang Zhang,
Stefan Hajnoczi, David S. Miller, Jorgen Hansen
In-Reply-To: <20191023095554.11340-8-sgarzare@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1545 bytes --]
On Wed, Oct 23, 2019 at 11:55:47AM +0200, Stefano Garzarella wrote:
> virtio_transport and vmci_transport handle the buffer_size
> sockopts in a very similar way.
>
> In order to support multiple transports, this patch moves this
> handling in the core to allow the user to change the options
> also if the socket is not yet assigned to any transport.
>
> This patch also adds the '.notify_buffer_size' callback in the
> 'struct virtio_transport' in order to inform the transport,
> when the buffer_size is changed by the user. It is also useful
> to limit the 'buffer_size' requested (e.g. virtio transports).
>
> Acked-by: Dexuan Cui <decui@microsoft.com>
> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
> ---
> RFC -> v1:
> - changed .notify_buffer_size return to void (Stefan)
> - documented that .notify_buffer_size is called with sk_lock held (Stefan)
> ---
> drivers/vhost/vsock.c | 7 +-
> include/linux/virtio_vsock.h | 15 +----
> include/net/af_vsock.h | 15 ++---
> net/vmw_vsock/af_vsock.c | 43 ++++++++++---
> net/vmw_vsock/hyperv_transport.c | 36 -----------
> net/vmw_vsock/virtio_transport.c | 8 +--
> net/vmw_vsock/virtio_transport_common.c | 79 ++++-------------------
> net/vmw_vsock/vmci_transport.c | 86 +++----------------------
> net/vmw_vsock/vmci_transport.h | 3 -
> 9 files changed, 65 insertions(+), 227 deletions(-)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* Re: [PATCH net-next 00/14] vsock: add multi-transports support
From: Stefan Hajnoczi @ 2019-10-27 8:01 UTC (permalink / raw)
To: Stefano Garzarella
Cc: netdev, Sasha Levin, linux-hyperv, Stephen Hemminger,
Arnd Bergmann, kvm, Michael S. Tsirkin, Greg Kroah-Hartman,
Dexuan Cui, linux-kernel, virtualization, Haiyang Zhang,
Stefan Hajnoczi, David S. Miller, Jorgen Hansen
In-Reply-To: <20191023095554.11340-1-sgarzare@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]
On Wed, Oct 23, 2019 at 11:55:40AM +0200, Stefano Garzarella wrote:
> This series adds the multi-transports support to vsock, following
> this proposal: https://www.spinics.net/lists/netdev/msg575792.html
>
> With the multi-transports support, we can use VSOCK with nested VMs
> (using also different hypervisors) loading both guest->host and
> host->guest transports at the same time.
> Before this series, vmci-transport supported this behavior but only
> using VMware hypervisor on L0, L1, etc.
>
> RFC: https://patchwork.ozlabs.org/cover/1168442/
> RFC -> v1:
> - Added R-b/A-b from Dexuan and Stefan
> - Fixed comments and typos in several patches (Stefan)
> - Patch 7: changed .notify_buffer_size return to void (Stefan)
> - Added patch 8 to simplify the API exposed to the transports (Stefan)
> - Patch 11:
> + documented VSOCK_TRANSPORT_F_* flags (Stefan)
> + fixed vsock_assign_transport() when the socket is already assigned
> + moved features outside of struct vsock_transport, and used as
> parameter of vsock_core_register() as a preparation of Patch 12
> - Removed "vsock: add 'transport_hg' to handle g2h\h2g transports" patch
> - Added patch 12 to register vmci_transport only when VMCI guest/host
> are active
Has there been feedback from Jorgen or someone else from VMware? A
Reviewed-by or Acked-by would be nice since this patch series affects
VMCI AF_VSOCK.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* [PATCH v3] x86/hyper-v: micro-optimize send_ipi_one case
From: Vitaly Kuznetsov @ 2019-10-27 15:19 UTC (permalink / raw)
To: linux-hyperv
Cc: linux-kernel, x86, K. Y. Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Roman Kagan, Michael Kelley,
Joe Perches
When sending an IPI to a single CPU there is no need to deal with cpumasks.
With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
cycles) improvement with smp_call_function_single() loop benchmark. The
optimization, however, is tiny and straitforward. Also, send_ipi_one() is
important for PV spinlock kick.
I was also wondering if it would make sense to switch to using regular
APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
vector)).
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
Changes since v2:
- Check VP number instead of CPU number against >= 64 [Michael]
- Check for VP_INVAL
---
arch/x86/hyperv/hv_apic.c | 16 +++++++++++++---
arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
2 files changed, 28 insertions(+), 3 deletions(-)
diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
index e01078e93dd3..40e0e322161d 100644
--- a/arch/x86/hyperv/hv_apic.c
+++ b/arch/x86/hyperv/hv_apic.c
@@ -194,10 +194,20 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector)
static bool __send_ipi_one(int cpu, int vector)
{
- struct cpumask mask = CPU_MASK_NONE;
+ int vp = hv_cpu_number_to_vp_number(cpu);
- cpumask_set_cpu(cpu, &mask);
- return __send_ipi_mask(&mask, vector);
+ trace_hyperv_send_ipi_one(cpu, vector);
+
+ if (!hv_hypercall_pg || (vp == VP_INVAL))
+ return false;
+
+ if ((vector < HV_IPI_LOW_VECTOR) || (vector > HV_IPI_HIGH_VECTOR))
+ return false;
+
+ if (vp >= 64)
+ return __send_ipi_mask_ex(cpumask_of(cpu), vector);
+
+ return !hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector, BIT_ULL(vp));
}
static void hv_send_ipi(int cpu, int vector)
diff --git a/arch/x86/include/asm/trace/hyperv.h b/arch/x86/include/asm/trace/hyperv.h
index ace464f09681..4d705cb4d63b 100644
--- a/arch/x86/include/asm/trace/hyperv.h
+++ b/arch/x86/include/asm/trace/hyperv.h
@@ -71,6 +71,21 @@ TRACE_EVENT(hyperv_send_ipi_mask,
__entry->ncpus, __entry->vector)
);
+TRACE_EVENT(hyperv_send_ipi_one,
+ TP_PROTO(int cpu,
+ int vector),
+ TP_ARGS(cpu, vector),
+ TP_STRUCT__entry(
+ __field(int, cpu)
+ __field(int, vector)
+ ),
+ TP_fast_assign(__entry->cpu = cpu;
+ __entry->vector = vector;
+ ),
+ TP_printk("cpu %d vector %x",
+ __entry->cpu, __entry->vector)
+ );
+
#endif /* CONFIG_HYPERV */
#undef TRACE_INCLUDE_PATH
--
2.20.1
^ permalink raw reply related
* RE: [PATCH v3] x86/hyper-v: micro-optimize send_ipi_one case
From: Michael Kelley @ 2019-10-27 16:49 UTC (permalink / raw)
To: vkuznets, linux-hyperv@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, x86@kernel.org, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Sasha Levin, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, H. Peter Anvin, Roman Kagan,
Joe Perches
In-Reply-To: <20191027151938.7296-1-vkuznets@redhat.com>
From: Vitaly Kuznetsov <vkuznets@redhat.com> Sent: Sunday, October 27, 2019 8:20 AM
>
> When sending an IPI to a single CPU there is no need to deal with cpumasks.
> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
> cycles) improvement with smp_call_function_single() loop benchmark. The
> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
> important for PV spinlock kick.
>
> I was also wondering if it would make sense to switch to using regular
> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
> vector)).
>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> Changes since v2:
> - Check VP number instead of CPU number against >= 64 [Michael]
> - Check for VP_INVAL
>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
^ permalink raw reply
* [PATCH 1/1] x86/hyperv: Initialize clockevents earlier in CPU onlining
From: Michael Kelley @ 2019-10-28 2:09 UTC (permalink / raw)
To: linux-kernel@vger.kernel.org, tglx@linutronix.de,
daniel.lezcano@linaro.org, vkuznets, Dexuan Cui, KY Srinivasan,
Stephen Hemminger, sashal@kernel.org, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org, will@kernel.org,
Ganapatrao.Kulkarni@cavium.com, james.morse@arm.com,
steven.price@arm.com, josephl@nvidia.com,
m.szyprowski@samsumg.com, linux-hyperv@vger.kernel.org
Cc: Michael Kelley
Hyper-V has historically initialized stimer-based clockevents late
in the process of onlining a CPU because clockevents depend on
stimer interrupts. In the original Hyper-V design, stimer
interrupts generate a VMbus message, so the VMbus machinery must
be running first, and VMbus can't be initialized until relatively
late. On x86/64, LAPIC timer based clockevents are used during early
initialization before VMbus and stimer-based clockevents are ready,
and again during CPU offlining after the stimer clockevents have been
shut down.
Unfortunately, this design creates problems when offling CPUs for
hibernation or other purposes. stimer-based clockevents are shut
down relatively early in the offlining process, so
clockevents_unbind_device() must be used to fallback to the
LAPIC-based clockevents for the remainder of the offlining process.
Furthermore, the late initialization and early shutdown of
stimer-based clockevents doesn't work well on ARM64 since there
is no other timer like the LAPIC to fallback to. So CPU onlining and
offlining doesn't work properly.
Fix this by recognizing that stimer Direct Mode is the normal path
for newer versions of Hyper-V on x86/64, and the only path on other
architectures. With stimer Direct Mode, stimer interrupts don't
require any VMbus machinery. stimer clockevents can be initialized
and shut down consistent with how it is done for other clockevent
devices. While the old VMbus-based stimer interrupts must still
be supported for backward compatibility on x86/64, that mode of
operation can be treated as legacy.
So add a new Hyper-V stimer entry in the CPU hotplug state
list, and use that new state when in Direct Mode. Update the
Hyper-V clocksource driver to allocate and initialize stimer
clockevents earlier during boot. Update Hyper-V initialization
and the VMbus driver to use this new design. As a result,
the LAPIC timer is no longer used during boot or CPU
onlining/offlining and clockevents_unbind_device() is not
called. But retain the old design as a legacy implementation
for older versions of Hyper-V that don't support Direct Mode.
Signed-off-by: Michael Kelley <mikelley@microsoft.com>
---
arch/x86/hyperv/hv_init.c | 7 ++
drivers/clocksource/hyperv_timer.c | 127 ++++++++++++++++++++++++++++++-------
drivers/hv/hv.c | 4 +-
drivers/hv/vmbus_drv.c | 31 ++++-----
include/clocksource/hyperv_timer.h | 6 +-
include/linux/cpuhotplug.h | 1 +
6 files changed, 131 insertions(+), 45 deletions(-)
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index b2daf0e..453d5fa 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -323,8 +323,15 @@ void __init hyperv_init(void)
x86_init.pci.arch_init = hv_pci_init;
+ if (hv_stimer_alloc())
+ goto remove_hypercall_page;
+
return;
+remove_hypercall_page:
+ hypercall_msr.as_uint64 = 0;
+ wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+ hv_hypercall_pg = NULL;
remove_cpuhp_state:
cpuhp_remove_state(cpuhp);
free_vp_assist_page:
diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
index 2317d4e..00f6429 100644
--- a/drivers/clocksource/hyperv_timer.c
+++ b/drivers/clocksource/hyperv_timer.c
@@ -17,6 +17,7 @@
#include <linux/clocksource.h>
#include <linux/sched_clock.h>
#include <linux/mm.h>
+#include <linux/cpuhotplug.h>
#include <clocksource/hyperv_timer.h>
#include <asm/hyperv-tlfs.h>
#include <asm/mshyperv.h>
@@ -30,12 +31,22 @@
* mechanism is used when running on older versions of Hyper-V
* that don't support Direct Mode. While Hyper-V provides
* four stimer's per CPU, Linux uses only stimer0.
+ *
+ * Because Direct Mode does not require processing a VMbus
+ * message, stimer interrupts can be enabled earlier in the
+ * process of booting a CPU, and consistent with when timer
+ * interrupts are enabled for other clocksource drivers.
+ * However, for legacy versions of Hyper-V when Direct Mode
+ * is not enabled, setting up stimer interrupts must be
+ * delayed until VMbus is initialized and can process the
+ * interrupt message.
*/
static bool direct_mode_enabled;
static int stimer0_irq;
static int stimer0_vector;
static int stimer0_message_sint;
+static int stimer0_cpuhp;
/*
* ISR for when stimer0 is operating in Direct Mode. Direct Mode
@@ -102,7 +113,7 @@ static int hv_ce_set_oneshot(struct clock_event_device *evt)
/*
* hv_stimer_init - Per-cpu initialization of the clockevent
*/
-void hv_stimer_init(unsigned int cpu)
+static int hv_stimer_init(unsigned int cpu)
{
struct clock_event_device *ce;
@@ -112,7 +123,7 @@ void hv_stimer_init(unsigned int cpu)
* clocksource based on emulated PIT or LAPIC timer hardware.
*/
if (!(ms_hyperv.features & HV_MSR_SYNTIMER_AVAILABLE))
- return;
+ return 0;
ce = per_cpu_ptr(hv_clock_event, cpu);
ce->name = "Hyper-V clockevent";
@@ -127,28 +138,42 @@ void hv_stimer_init(unsigned int cpu)
HV_CLOCK_HZ,
HV_MIN_DELTA_TICKS,
HV_MAX_MAX_DELTA_TICKS);
+ return 0;
}
-EXPORT_SYMBOL_GPL(hv_stimer_init);
/*
* hv_stimer_cleanup - Per-cpu cleanup of the clockevent
*/
-void hv_stimer_cleanup(unsigned int cpu)
+static int hv_stimer_cleanup(unsigned int cpu)
{
struct clock_event_device *ce;
/* Turn off clockevent device */
if (ms_hyperv.features & HV_MSR_SYNTIMER_AVAILABLE) {
ce = per_cpu_ptr(hv_clock_event, cpu);
+
+ /*
+ * In the legacy case where Direct Mode is not enabled
+ * (which can only be on x86/64), stimer cleanup happens
+ * relatively early in the CPU offlining process. We
+ * must unbind the stimer-based clockevent device so
+ * that the LAPIC timer can take over until clockevents
+ * are no longer needed in the offlining process. The
+ * unbind should not be done when Direct Mode is enabled
+ * because we may be on an architecture where there are
+ * no other clockevents devices to fallback to.
+ */
+ if (!direct_mode_enabled)
+ clockevents_unbind_device(ce, cpu);
hv_ce_shutdown(ce);
}
+ return 0;
}
-EXPORT_SYMBOL_GPL(hv_stimer_cleanup);
/* hv_stimer_alloc - Global initialization of the clockevent and stimer0 */
-int hv_stimer_alloc(int sint)
+int hv_stimer_alloc(void)
{
- int ret;
+ int ret = 0;
hv_clock_event = alloc_percpu(struct clock_event_device);
if (!hv_clock_event)
@@ -159,24 +184,83 @@ int hv_stimer_alloc(int sint)
if (direct_mode_enabled) {
ret = hv_setup_stimer0_irq(&stimer0_irq, &stimer0_vector,
hv_stimer0_isr);
- if (ret) {
- free_percpu(hv_clock_event);
- hv_clock_event = NULL;
- return ret;
- }
+ if (ret)
+ goto free_percpu;
+
+ /*
+ * Since we are in Direct Mode, stimer initialization
+ * can be done now with a CPUHP value in the same range
+ * as other clockevent devices.
+ */
+ ret = cpuhp_setup_state(CPUHP_AP_HYPERV_TIMER_STARTING,
+ "clockevents/hyperv/stimer:starting",
+ hv_stimer_init, hv_stimer_cleanup);
+ if (ret < 0)
+ goto free_stimer0_irq;
+ stimer0_cpuhp = ret;
}
+ return ret;
- stimer0_message_sint = sint;
- return 0;
+free_stimer0_irq:
+ hv_remove_stimer0_irq(stimer0_irq);
+free_percpu:
+ free_percpu(hv_clock_event);
+ hv_clock_event = NULL;
+ return ret;
}
EXPORT_SYMBOL_GPL(hv_stimer_alloc);
+/*
+ * hv_stimer_legacy_init -- Called from the VMbus driver to handle
+ * the case when Direct Mode is not enabled, and the stimer
+ * must be initialized late in the CPU onlining process.
+ *
+ */
+void hv_stimer_legacy_init(unsigned int cpu, int sint)
+{
+ if (direct_mode_enabled)
+ return;
+
+ /*
+ * This function gets called by each vCPU, so setting the
+ * global stimer_message_sint value each time is conceptually
+ * not ideal, but the value passed in is always the same and
+ * it avoids introducing yet another interface into this
+ * clocksource driver just to set the sint in the legacy
+ * (i.e., no Direct Mode) case.
+ */
+ stimer0_message_sint = sint;
+ (void)hv_stimer_init(cpu);
+}
+EXPORT_SYMBOL_GPL(hv_stimer_legacy_init);
+
+/*
+ * hv_stimer_legacy_cleanup -- Called from the VMbus driver to
+ * handle the case when Direct Mode is not enabled, and the
+ * stimer must be cleaned up early in the CPU offlining
+ * process.
+ */
+void hv_stimer_legacy_cleanup(unsigned int cpu)
+{
+ if (direct_mode_enabled)
+ return;
+ (void)hv_stimer_cleanup(cpu);
+}
+EXPORT_SYMBOL_GPL(hv_stimer_legacy_cleanup);
+
+
/* hv_stimer_free - Free global resources allocated by hv_stimer_alloc() */
void hv_stimer_free(void)
{
- if (direct_mode_enabled && (stimer0_irq != 0)) {
- hv_remove_stimer0_irq(stimer0_irq);
- stimer0_irq = 0;
+ if (direct_mode_enabled) {
+ if (stimer0_cpuhp) {
+ cpuhp_remove_state(stimer0_cpuhp);
+ stimer0_cpuhp = 0;
+ }
+ if (stimer0_irq) {
+ hv_remove_stimer0_irq(stimer0_irq);
+ stimer0_irq = 0;
+ }
}
free_percpu(hv_clock_event);
hv_clock_event = NULL;
@@ -190,14 +274,11 @@ void hv_stimer_free(void)
void hv_stimer_global_cleanup(void)
{
int cpu;
- struct clock_event_device *ce;
- if (ms_hyperv.features & HV_MSR_SYNTIMER_AVAILABLE) {
- for_each_present_cpu(cpu) {
- ce = per_cpu_ptr(hv_clock_event, cpu);
- clockevents_unbind_device(ce, cpu);
- }
+ for_each_present_cpu(cpu) {
+ hv_stimer_cleanup(cpu);
}
+
hv_stimer_free();
}
EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup);
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index fcc5279..6098e0c 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -202,7 +202,7 @@ int hv_synic_init(unsigned int cpu)
{
hv_synic_enable_regs(cpu);
- hv_stimer_init(cpu);
+ hv_stimer_legacy_init(cpu, VMBUS_MESSAGE_SINT);
return 0;
}
@@ -277,7 +277,7 @@ int hv_synic_cleanup(unsigned int cpu)
if (channel_found && vmbus_connection.conn_state == CONNECTED)
return -EBUSY;
- hv_stimer_cleanup(cpu);
+ hv_stimer_legacy_cleanup(cpu);
hv_synic_disable_regs(cpu);
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 0ab126b..18fdddb 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -1342,10 +1342,6 @@ static int vmbus_bus_init(void)
if (ret)
goto err_alloc;
- ret = hv_stimer_alloc(VMBUS_MESSAGE_SINT);
- if (ret < 0)
- goto err_alloc;
-
/*
* Initialize the per-cpu interrupt state and stimer state.
* Then connect to the host.
@@ -1402,9 +1398,8 @@ static int vmbus_bus_init(void)
err_connect:
cpuhp_remove_state(hyperv_cpuhp_online);
err_cpuhp:
- hv_stimer_free();
-err_alloc:
hv_synic_free();
+err_alloc:
hv_remove_vmbus_irq();
bus_unregister(&hv_bus);
@@ -2310,7 +2305,6 @@ static void hv_crash_handler(struct pt_regs *regs)
*/
vmbus_connection.conn_state = DISCONNECTED;
cpu = smp_processor_id();
- hv_stimer_cleanup(cpu);
hv_synic_cleanup(cpu);
hyperv_cleanup();
};
@@ -2318,20 +2312,23 @@ static void hv_crash_handler(struct pt_regs *regs)
static int hv_synic_suspend(void)
{
/*
- * When we reach here, all the non-boot CPUs have been offlined, and
- * the stimers on them have been unbound in hv_synic_cleanup() ->
+ * When we reach here, all the non-boot CPUs have been offlined.
+ * If we're in a legacy configuration where stimer Direct Mode is
+ * not enabled, the stimers on the non-boot CPUs have been unbound
+ * in hv_synic_cleanup() -> hv_stimer_legacy_cleanup() ->
* hv_stimer_cleanup() -> clockevents_unbind_device().
*
- * hv_synic_suspend() only runs on CPU0 with interrupts disabled. Here
- * we do not unbind the stimer on CPU0 because: 1) it's unnecessary
- * because the interrupts remain disabled between syscore_suspend()
- * and syscore_resume(): see create_image() and resume_target_kernel();
+ * hv_synic_suspend() only runs on CPU0 with interrupts disabled.
+ * Here we do not call hv_stimer_legacy_cleanup() on CPU0 because:
+ * 1) it's unnecessary as interrupts remain disabled between
+ * syscore_suspend() and syscore_resume(): see create_image() and
+ * resume_target_kernel()
* 2) the stimer on CPU0 is automatically disabled later by
* syscore_suspend() -> timekeeping_suspend() -> tick_suspend() -> ...
- * -> clockevents_shutdown() -> ... -> hv_ce_shutdown(); 3) a warning
- * would be triggered if we call clockevents_unbind_device(), which
- * may sleep, in an interrupts-disabled context. So, we intentionally
- * don't call hv_stimer_cleanup(0) here.
+ * -> clockevents_shutdown() -> ... -> hv_ce_shutdown()
+ * 3) a warning would be triggered if we call
+ * clockevents_unbind_device(), which may sleep, in an
+ * interrupts-disabled context.
*/
hv_synic_disable_regs(0);
diff --git a/include/clocksource/hyperv_timer.h b/include/clocksource/hyperv_timer.h
index 422f5e5..9b477c4 100644
--- a/include/clocksource/hyperv_timer.h
+++ b/include/clocksource/hyperv_timer.h
@@ -21,10 +21,10 @@
#define HV_MIN_DELTA_TICKS 1
/* Routines called by the VMbus driver */
-extern int hv_stimer_alloc(int sint);
+extern int hv_stimer_alloc(void);
extern void hv_stimer_free(void);
-extern void hv_stimer_init(unsigned int cpu);
-extern void hv_stimer_cleanup(unsigned int cpu);
+extern void hv_stimer_legacy_init(unsigned int cpu, int sint);
+extern void hv_stimer_legacy_cleanup(unsigned int cpu);
extern void hv_stimer_global_cleanup(void);
extern void hv_stimer0_isr(void);
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 89d75ed..e51ee77 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -129,6 +129,7 @@ enum cpuhp_state {
CPUHP_AP_ARC_TIMER_STARTING,
CPUHP_AP_RISCV_TIMER_STARTING,
CPUHP_AP_CSKY_TIMER_STARTING,
+ CPUHP_AP_HYPERV_TIMER_STARTING,
CPUHP_AP_KVM_STARTING,
CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
CPUHP_AP_KVM_ARM_VGIC_STARTING,
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH v3] x86/hyper-v: micro-optimize send_ipi_one case
From: Roman Kagan @ 2019-10-28 9:35 UTC (permalink / raw)
To: Vitaly Kuznetsov
Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
x86@kernel.org, K. Y. Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H. Peter Anvin, Michael Kelley, Joe Perches
In-Reply-To: <20191027151938.7296-1-vkuznets@redhat.com>
On Sun, Oct 27, 2019 at 04:19:38PM +0100, Vitaly Kuznetsov wrote:
> When sending an IPI to a single CPU there is no need to deal with cpumasks.
> With 2 CPU guest on WS2019 I'm seeing a minor (like 3%, 8043 -> 7761 CPU
> cycles) improvement with smp_call_function_single() loop benchmark. The
> optimization, however, is tiny and straitforward. Also, send_ipi_one() is
> important for PV spinlock kick.
>
> I was also wondering if it would make sense to switch to using regular
> APIC IPI send for CPU > 64 case but no, it is twice as expesive (12650 CPU
> cycles for __send_ipi_mask_ex() call, 26000 for orig_apic.send_IPI(cpu,
> vector)).
>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> Changes since v2:
> - Check VP number instead of CPU number against >= 64 [Michael]
> - Check for VP_INVAL
> ---
> arch/x86/hyperv/hv_apic.c | 16 +++++++++++++---
> arch/x86/include/asm/trace/hyperv.h | 15 +++++++++++++++
> 2 files changed, 28 insertions(+), 3 deletions(-)
Reviewed-by: Roman Kagan <rkagan@virtuozzo.com>
^ permalink raw reply
* Re: [PATCH v1] tools/hv: async name resolution in kvp_daemon
From: Sasha Levin @ 2019-10-28 16:17 UTC (permalink / raw)
To: Olaf Hering
Cc: K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
open list:Hyper-V CORE AND DRIVERS, open list
In-Reply-To: <20191024144943.26199-1-olaf@aepfle.de>
On Thu, Oct 24, 2019 at 04:49:43PM +0200, Olaf Hering wrote:
>The hostname is resolved just once since commit 58125210ab3b ("Tools:
>hv: cache FQDN in kvp_daemon to avoid timeouts") to make sure the VM
>responds within the timeout limits to requests from the host.
>
>If for some reason getaddrinfo fails, the string returned by the
>"FullyQualifiedDomainName" request contains some error string, which is
>then used by tools on the host side.
>
>Adjust the code to resolve the current hostname in a separate thread.
>This thread loops until getaddrinfo returns success. During this time
>all "FullyQualifiedDomainName" requests will be answered with an empty
>string.
>
>Signed-off-by: Olaf Hering <olaf@aepfle.de>
Thank for the patch Olaf. However, it seems to break build for me:
/usr/bin/ld: hv_kvp_daemon-in.o: in function `kvp_obtain_domain_name':
/home/sasha/linux/tools/hv/hv_kvp_daemon.c:1372: undefined reference to `pthread_create'
/usr/bin/ld: /home/sasha/linux/tools/hv/hv_kvp_daemon.c:1377: undefined reference to `pthread_detach'
collect2: error: ld returned 1 exit status
--
Thanks,
Sasha
^ permalink raw reply
* Re: [PATCH] drivers: iommu: hyperv: Make HYPERV_IOMMU only available on x86
From: Sasha Levin @ 2019-10-28 16:20 UTC (permalink / raw)
To: Boqun Feng
Cc: iommu, linux-kernel, Lan Tianyu, Michael Kelley, linux-hyperv,
Joerg Roedel
In-Reply-To: <20191017005711.2013647-1-boqun.feng@gmail.com>
On Thu, Oct 17, 2019 at 08:57:03AM +0800, Boqun Feng wrote:
>Currently hyperv-iommu is implemented in a x86 specific way, for
>example, apic is used. So make the HYPERV_IOMMU Kconfig depend on X86
>as a preparation for enabling HyperV on architecture other than x86.
>
>Cc: Lan Tianyu <Tianyu.Lan@microsoft.com>
>Cc: Michael Kelley <mikelley@microsoft.com>
>Cc: linux-hyperv@vger.kernel.org
>Signed-off-by: Boqun Feng (Microsoft) <boqun.feng@gmail.com>
Queued up for hyperv-fixes, thanks!
--
Thanks,
Sasha
^ permalink raw reply
* Re: [PATCH v3 0/3] Drivers: hv: vmbus: Miscellaneous improvements
From: Sasha Levin @ 2019-10-28 16:26 UTC (permalink / raw)
To: Andrea Parri
Cc: linux-kernel, linux-hyperv, K . Y . Srinivasan, Haiyang Zhang,
Stephen Hemminger, Michael Kelley, Vitaly Kuznetsov, Dexuan Cui,
Wei Liu
In-Reply-To: <20191015114646.15354-1-parri.andrea@gmail.com>
On Tue, Oct 15, 2019 at 01:46:43PM +0200, Andrea Parri wrote:
>Hi all,
>
>The patchset:
>
> - refactors the VMBus negotiation code by introducing the table of
> VMBus protocol versions (patch 1/3),
>
> - enables VMBus protocol version 4.1, 5.1 and 5.2 (patch 2/3),
>
> - introduces a module parameter to cap the VMBus protocol versions
> which a guest can negotiate with the hypervisor (patch 3/3).
Queued up for hyperv-next, thanks!
--
Thanks,
Sasha
^ permalink raw reply
* Re: [PATCH v2] x86/hyperv: Set pv_info.name to "Hyper-V"
From: Sasha Levin @ 2019-10-28 16:27 UTC (permalink / raw)
To: Michael Kelley
Cc: Andrea Parri, linux-kernel@vger.kernel.org,
linux-hyperv@vger.kernel.org, x86@kernel.org, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H . Peter Anvin, vkuznets, Dexuan Cui, Wei Liu
In-Reply-To: <DM5PR21MB01377F713A553FCF721EF99DD7930@DM5PR21MB0137.namprd21.prod.outlook.com>
On Tue, Oct 15, 2019 at 01:06:33PM +0000, Michael Kelley wrote:
>From: Andrea Parri <parri.andrea@gmail.com> Sent: Tuesday, October 15, 2019 3:35 AM
>>
>> Michael reported that the x86/hyperv initialization code printed the
>> following dmesg when running in a VM on Hyper-V:
>>
>> [ 0.000738] Booting paravirtualized kernel on bare hardware
>>
>> Let the x86/hyperv initialization code set pv_info.name to "Hyper-V";
>> with this addition, the dmesg read:
>>
>> [ 0.000172] Booting paravirtualized kernel on Hyper-V
>>
>> Reported-by: Michael Kelley <mikelley@microsoft.com>
>> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
>
>Reviewed-by: Michael Kelley <mikelley@microsoft.com>
Thomas, will you be taking this? Would you rather have me deal with the
hyperv bits under arch/x86/?
--
Thanks,
Sasha
^ permalink raw reply
* RE: [PATCH v2] x86/hyperv: Set pv_info.name to "Hyper-V"
From: Michael Kelley @ 2019-10-28 17:13 UTC (permalink / raw)
To: Sasha Levin
Cc: Andrea Parri, linux-kernel@vger.kernel.org,
linux-hyperv@vger.kernel.org, x86@kernel.org, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, H . Peter Anvin, vkuznets, Dexuan Cui, Wei Liu
In-Reply-To: <20191028162734.GI1554@sasha-vm>
From: Sasha Levin <sashal@kernel.org> Sent: Monday, October 28, 2019 9:28 AM
> On Tue, Oct 15, 2019 at 01:06:33PM +0000, Michael Kelley wrote:
> >From: Andrea Parri <parri.andrea@gmail.com> Sent: Tuesday, October 15, 2019 3:35 AM
> >>
> >> Michael reported that the x86/hyperv initialization code printed the
> >> following dmesg when running in a VM on Hyper-V:
> >>
> >> [ 0.000738] Booting paravirtualized kernel on bare hardware
> >>
> >> Let the x86/hyperv initialization code set pv_info.name to "Hyper-V";
> >> with this addition, the dmesg read:
> >>
> >> [ 0.000172] Booting paravirtualized kernel on Hyper-V
> >>
> >> Reported-by: Michael Kelley <mikelley@microsoft.com>
> >> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> >
> >Reviewed-by: Michael Kelley <mikelley@microsoft.com>
>
> Thomas, will you be taking this? Would you rather have me deal with the
> hyperv bits under arch/x86/?
>
Thomas has already pulled this one. It's in Linus' tree.
Michael
^ permalink raw reply
* Re: [PATCH v1] tools/hv: async name resolution in kvp_daemon
From: Olaf Hering @ 2019-10-28 17:49 UTC (permalink / raw)
To: Sasha Levin
Cc: K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
open list:Hyper-V CORE AND DRIVERS, open list
In-Reply-To: <20191028161754.GF1554@sasha-vm>
[-- Attachment #1: Type: text/plain, Size: 197 bytes --]
Am Mon, 28 Oct 2019 12:17:54 -0400
schrieb Sasha Levin <sashal@kernel.org>:
> undefined reference to `pthread_create'
Does "make V=1 -C tools/hv" work for you?
This is what I use.
Olaf
[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* [PATCH net-next, 0/4] hv_netvsc: Add XDP support and some error handling fixes
From: Haiyang Zhang @ 2019-10-28 21:05 UTC (permalink / raw)
To: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
This patch set fixes some error handling issues in netvsc driver,
and add XDP support.
Haiyang Zhang (4):
hv_netvsc: Fix error handling in netvsc_set_features()
hv_netvsc: Fix error handling in netvsc_attach()
hv_netvsc: Add XDP support
hv_netvsc: Update document for XDP support
.../networking/device_drivers/microsoft/netvsc.txt | 14 ++
drivers/net/hyperv/Makefile | 2 +-
drivers/net/hyperv/hyperv_net.h | 15 ++
drivers/net/hyperv/netvsc.c | 8 +-
drivers/net/hyperv/netvsc_bpf.c | 211 +++++++++++++++++++++
drivers/net/hyperv/netvsc_drv.c | 150 ++++++++++++---
6 files changed, 374 insertions(+), 26 deletions(-)
create mode 100644 drivers/net/hyperv/netvsc_bpf.c
--
1.8.3.1
^ permalink raw reply
* [PATCH net-next, 2/4] hv_netvsc: Fix error handling in netvsc_attach()
From: Haiyang Zhang @ 2019-10-28 21:07 UTC (permalink / raw)
To: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <1572296801-4789-1-git-send-email-haiyangz@microsoft.com>
If rndis_filter_open() fails, we need to remove the rndis device created
in earlier steps, before returning an error code. Otherwise, the retry of
netvsc_attach() from its callers will fail and hang.
Fixes: 7b2ee50c0cd5 ("hv_netvsc: common detach logic")
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
drivers/net/hyperv/netvsc_drv.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 734e411..a14fc8e 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -982,7 +982,7 @@ static int netvsc_attach(struct net_device *ndev,
if (netif_running(ndev)) {
ret = rndis_filter_open(nvdev);
if (ret)
- return ret;
+ goto err;
rdev = nvdev->extension;
if (!rdev->link_state)
@@ -990,6 +990,13 @@ static int netvsc_attach(struct net_device *ndev,
}
return 0;
+
+err:
+ netif_device_detach(ndev);
+
+ rndis_filter_device_remove(hdev, nvdev);
+
+ return ret;
}
static int netvsc_set_channels(struct net_device *net,
--
1.8.3.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox