* Re: [PATCH v2 0/8] mm/memory_hotplug: allow to specify a default online_type
From: Baoquan He @ 2020-03-18 14:50 UTC (permalink / raw)
To: David Hildenbrand
Cc: linux-kernel, linux-mm, linuxppc-dev, linux-hyperv,
Vitaly Kuznetsov, Yumei Huang, Igor Mammedov, Eduardo Habkost,
Milan Zamazal, Andrew Morton, Benjamin Herrenschmidt,
Greg Kroah-Hartman, Haiyang Zhang, K. Y. Srinivasan,
Michael Ellerman, Michal Hocko, Michal Hocko, Oscar Salvador,
Paul Mackerras, Rafael J. Wysocki, Stephen Hemminger, Wei Liu,
Wei Yang
In-Reply-To: <67a054f6-df07-e4fb-dd4b-e503cb767276@redhat.com>
On 03/18/20 at 02:50pm, David Hildenbrand wrote:
> On 18.03.20 14:05, Baoquan He wrote:
> > On 03/17/20 at 11:49am, David Hildenbrand wrote:
> >> Distributions nowadays use udev rules ([1] [2]) to specify if and
> >> how to online hotplugged memory. The rules seem to get more complex with
> >> many special cases. Due to the various special cases,
> >> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE cannot be used. All memory hotplug
> >> is handled via udev rules.
> >>
> >> Everytime we hotplug memory, the udev rule will come to the same
> >> conclusion. Especially Hyper-V (but also soon virtio-mem) add a lot of
> >> memory in separate memory blocks and wait for memory to get onlined by user
> >> space before continuing to add more memory blocks (to not add memory faster
> >> than it is getting onlined). This of course slows down the whole memory
> >> hotplug process.
> >>
> >> To make the job of distributions easier and to avoid udev rules that get
> >> more and more complicated, let's extend the mechanism provided by
> >> - /sys/devices/system/memory/auto_online_blocks
> >> - "memhp_default_state=" on the kernel cmdline
> >> to be able to specify also "online_movable" as well as "online_kernel"
> >
> > This patch series looks good, thanks. Since Andrew has merged it to -mm again,
> > I won't add my Reviewed-by to bother.
> >
> > Hi David, Vitaly
> >
> > There are several things unclear to me.
> >
> > So, these improved interfaces are used to alleviate the burden of the
> > existing udev rules, or try to replace it? As you know, we have been
>
> At least in RHEL, my plan is to replace it / use a udev rules as a
> fallback on older kernels (see the example scripts below). But other
Ok, got it. Didn't notice the script and the systemd service are your
part of plan, thought you are demonstrating the status. Thanks.
> distribution can handle it as they want.
>
> > using udev rules to interact between kernel and user space on bare metal,
> > and guests who want to hot add/remove.>
> > And also the OOM issue in hyperV when onlining pages after adding memory
> > block. I am not a virt devel expert, could this happen on bare metal
> > system?
>
> Don't think it's relevant on bare metal. If you plug a big DIMM, all
> memory blocks will be added first in one shot and then all memory blocks
> will be onlined. So it doesn't matter "how fast" you online that memory.
>
> In contrast, Hyper-V (and virtio-mem) add one (or a limited number of)
> memory block at a time and wait for them to get onlined.
>
> --
> Thanks,
>
> David / dhildenb
^ permalink raw reply
* Re: [PATCH v2 0/8] mm/memory_hotplug: allow to specify a default online_type
From: Vitaly Kuznetsov @ 2020-03-18 15:00 UTC (permalink / raw)
To: Baoquan He
Cc: David Hildenbrand, linux-kernel, linux-mm, linuxppc-dev,
linux-hyperv, Yumei Huang, Igor Mammedov, Eduardo Habkost,
Milan Zamazal, Andrew Morton, Benjamin Herrenschmidt,
Greg Kroah-Hartman, Haiyang Zhang, K. Y. Srinivasan,
Michael Ellerman, Michal Hocko, Michal Hocko, Oscar Salvador,
Paul Mackerras, Rafael J. Wysocki, Stephen Hemminger, Wei Liu,
Wei Yang
In-Reply-To: <20200318144119.GD30899@MiWiFi-R3L-srv>
Baoquan He <bhe@redhat.com> writes:
> Is there a reason hyperV need boot with small memory, then enlarge it
> with huge memory? Since it's a real case in hyperV, I guess there must
> be reason, I am just curious.
>
It doesn't really *need* to but this can be utilized in e.g. 'hot
standby' schemes I believe. Also, it may be enough if the administrator
is just trying to e.g. double the size of RAM but the VM is already
under memory pressure. I wouldn't say that these cases are common but
afair bugs like 'I tried adding more memory to my VM and it just OOMed'
were reported in the past.
--
Vitaly
^ permalink raw reply
* Re: [PATCH 0/4] x86/Hyper-V: Unload vmbus channel in hv panic callback
From: Vitaly Kuznetsov @ 2020-03-18 15:58 UTC (permalink / raw)
To: ltykernel
Cc: Tianyu Lan, linux-hyperv, linux-kernel, kys, haiyangz, sthemmin,
liuwe, tglx, mingo, bp, hpa, x86, michael.h.kelley
In-Reply-To: <20200317132523.1508-2-Tianyu.Lan@microsoft.com>
ltykernel@gmail.com writes:
> From: Tianyu Lan <Tianyu.Lan@microsoft.com>
>
> Customer reported Hyper-V VM still responded network traffic
> ack packets after kernel panic with kernel parameter "panic=0”.
> This becauses vmbus driver interrupt handler still works
> on the panic cpu after kernel panic. Panic cpu falls into
> infinite loop of panic() with interrupt enabled at that point.
> Vmbus driver can still handle network traffic.
>
> This confuses remote service that the panic system is still
> alive when it gets ack packets. Unload vmbus channel in hv panic
> callback and fix it.
>
> vmbus_initiate_unload() maybe double called during panic process
> (e.g, hyperv_panic_event() and hv_crash_handler()). So check
> and set connection state in vmbus_initiate_unload() to resolve
> reenter issue.
>
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> drivers/hv/channel_mgmt.c | 5 +++++
> drivers/hv/vmbus_drv.c | 17 +++++++++--------
> 2 files changed, 14 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
> index 0370364169c4..893493f2b420 100644
> --- a/drivers/hv/channel_mgmt.c
> +++ b/drivers/hv/channel_mgmt.c
> @@ -839,6 +839,9 @@ void vmbus_initiate_unload(bool crash)
> {
> struct vmbus_channel_message_header hdr;
>
> + if (vmbus_connection.conn_state == DISCONNECTED)
> + return;
> +
To make this less racy, can we do something like
if (xchg(&vmbus_connection.conn_state, DISCONNECTED) == DISCONNECTED)
return;
?
> /* Pre-Win2012R2 hosts don't support reconnect */
> if (vmbus_proto_version < VERSION_WIN8_1)
> return;
> @@ -857,6 +860,8 @@ void vmbus_initiate_unload(bool crash)
> wait_for_completion(&vmbus_connection.unload_event);
> else
> vmbus_wait_for_unload();
> +
> + vmbus_connection.conn_state = DISCONNECTED;
> }
>
> static void check_ready_for_resume_event(void)
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index 029378c27421..b56b9fb9bd90 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -53,9 +53,12 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned long val,
> {
> struct pt_regs *regs;
>
> - regs = current_pt_regs();
> + vmbus_initiate_unload(true);
>
> - hyperv_report_panic(regs, val);
> + if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
With Michael's effors to make code in drivers/hv arch agnostic, I think
we need a better, arch-neutral way.
> + regs = current_pt_regs();
> + hyperv_report_panic(regs, val);
> + }
> return NOTIFY_DONE;
> }
>
> @@ -1391,10 +1394,12 @@ static int vmbus_bus_init(void)
> }
>
> register_die_notifier(&hyperv_die_block);
> - atomic_notifier_chain_register(&panic_notifier_list,
> - &hyperv_panic_block);
> }
>
> + /* Vmbus channel is unloaded in panic callback when panic happens.*/
> + atomic_notifier_chain_register(&panic_notifier_list,
> + &hyperv_panic_block);
> +
> vmbus_request_offers();
>
> return 0;
> @@ -2204,8 +2209,6 @@ static int vmbus_bus_suspend(struct device *dev)
>
> vmbus_initiate_unload(false);
>
> - vmbus_connection.conn_state = DISCONNECTED;
> -
> /* Reset the event for the next resume. */
> reinit_completion(&vmbus_connection.ready_for_resume_event);
>
> @@ -2289,7 +2292,6 @@ static void hv_kexec_handler(void)
> {
> hv_stimer_global_cleanup();
> vmbus_initiate_unload(false);
> - vmbus_connection.conn_state = DISCONNECTED;
> /* Make sure conn_state is set as hv_synic_cleanup checks for it */
> mb();
> cpuhp_remove_state(hyperv_cpuhp_online);
> @@ -2306,7 +2308,6 @@ static void hv_crash_handler(struct pt_regs *regs)
> * doing the cleanup for current CPU only. This should be sufficient
> * for kdump.
> */
> - vmbus_connection.conn_state = DISCONNECTED;
> cpu = smp_processor_id();
> hv_stimer_cleanup(cpu);
> hv_synic_disable_regs(cpu);
--
Vitaly
^ permalink raw reply
* RE: [PATCH 0/4] x86/Hyper-V: Unload vmbus channel in hv panic callback
From: Michael Kelley @ 2020-03-19 0:33 UTC (permalink / raw)
To: vkuznets, ltykernel@gmail.com
Cc: Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Wei Liu, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org
In-Reply-To: <871rpp3ba8.fsf@vitty.brq.redhat.com>
From: Vitaly Kuznetsov <vkuznets@redhat.com> Sent: Wednesday, March 18, 2020 8:58 AM
>
> ltykernel@gmail.com writes:
>
> > From: Tianyu Lan <Tianyu.Lan@microsoft.com>
> >
> > Customer reported Hyper-V VM still responded network traffic
> > ack packets after kernel panic with kernel parameter "panic=0”.
> > This becauses vmbus driver interrupt handler still works
> > on the panic cpu after kernel panic. Panic cpu falls into
> > infinite loop of panic() with interrupt enabled at that point.
> > Vmbus driver can still handle network traffic.
> >
> > This confuses remote service that the panic system is still
> > alive when it gets ack packets. Unload vmbus channel in hv panic
> > callback and fix it.
> >
> > vmbus_initiate_unload() maybe double called during panic process
> > (e.g, hyperv_panic_event() and hv_crash_handler()). So check
> > and set connection state in vmbus_initiate_unload() to resolve
> > reenter issue.
Let me suggest a revised version of the commit message:
When kdump is not configured, a Hyper-V VM might still respond to
network traffic after a kernel panic when kernel parameter panic=0.
The panic CPU goes into an infinite loop with interrupts enabled,
and the VMbus driver interrupt handler still works because the
VMbus connection is unloaded only in the kdump path. The network
responses make the other end of the connection think the VM is
still functional even though it has panic'ed, which could affect any
failover actions that should be taken.
Fix this by unloading the VMbus connection during the panic process.
vmbus_initiate_unload() could then be called twice (e.g., by
hyperv_panic_event() and hv_crash_handler(), so reset the connection
state in vmbus_initiate_unload() to ensure the unload is done only
once.
> >
> > Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> > ---
> > drivers/hv/channel_mgmt.c | 5 +++++
> > drivers/hv/vmbus_drv.c | 17 +++++++++--------
> > 2 files changed, 14 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
> > index 0370364169c4..893493f2b420 100644
> > --- a/drivers/hv/channel_mgmt.c
> > +++ b/drivers/hv/channel_mgmt.c
> > @@ -839,6 +839,9 @@ void vmbus_initiate_unload(bool crash)
> > {
> > struct vmbus_channel_message_header hdr;
> >
> > + if (vmbus_connection.conn_state == DISCONNECTED)
> > + return;
> > +
>
> To make this less racy, can we do something like
>
> if (xchg(&vmbus_connection.conn_state, DISCONNECTED) == DISCONNECTED)
> return;
>
> ?
I was trying to decide if there can actually be a race. The panic() and die()
functions both ensure that only a single CPU can execute in those paths at any
one time, though maybe panic() and die() could be running concurrently.
And vmbus_initiate_unload() can also be called in the hibernation path in
vmbus_bus_suspend(), so there could be a race. Doing the xchg() makes
sense.
>
> > /* Pre-Win2012R2 hosts don't support reconnect */
> > if (vmbus_proto_version < VERSION_WIN8_1)
> > return;
> > @@ -857,6 +860,8 @@ void vmbus_initiate_unload(bool crash)
> > wait_for_completion(&vmbus_connection.unload_event);
> > else
> > vmbus_wait_for_unload();
> > +
> > + vmbus_connection.conn_state = DISCONNECTED;
> > }
> >
> > static void check_ready_for_resume_event(void)
> > diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> > index 029378c27421..b56b9fb9bd90 100644
> > --- a/drivers/hv/vmbus_drv.c
> > +++ b/drivers/hv/vmbus_drv.c
> > @@ -53,9 +53,12 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned
> long val,
> > {
> > struct pt_regs *regs;
> >
> > - regs = current_pt_regs();
> > + vmbus_initiate_unload(true);
> >
> > - hyperv_report_panic(regs, val);
> > + if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
>
> With Michael's effors to make code in drivers/hv arch agnostic, I think
> we need a better, arch-neutral way.
Vitaly -- could you elaborate on what part is not arch-neutral? I don't see
a problem. ms_hyperv and the misc_features field exist for both the x86
and ARM64 code branches. It turns out the particular bit for
GUEST_CRASH_MSR_AVAILABLE is different on the two architectures, but
the compiler will do the right thing.
>
> > + regs = current_pt_regs();
> > + hyperv_report_panic(regs, val);
> > + }
> > return NOTIFY_DONE;
> > }
> >
> > @@ -1391,10 +1394,12 @@ static int vmbus_bus_init(void)
> > }
> >
> > register_die_notifier(&hyperv_die_block);
> > - atomic_notifier_chain_register(&panic_notifier_list,
> > - &hyperv_panic_block);
> > }
> >
> > + /* Vmbus channel is unloaded in panic callback when panic happens.*/
> > + atomic_notifier_chain_register(&panic_notifier_list,
> > + &hyperv_panic_block);
> > +
> > vmbus_request_offers();
> >
> > return 0;
> > @@ -2204,8 +2209,6 @@ static int vmbus_bus_suspend(struct device *dev)
> >
> > vmbus_initiate_unload(false);
> >
> > - vmbus_connection.conn_state = DISCONNECTED;
> > -
> > /* Reset the event for the next resume. */
> > reinit_completion(&vmbus_connection.ready_for_resume_event);
> >
> > @@ -2289,7 +2292,6 @@ static void hv_kexec_handler(void)
> > {
> > hv_stimer_global_cleanup();
> > vmbus_initiate_unload(false);
> > - vmbus_connection.conn_state = DISCONNECTED;
> > /* Make sure conn_state is set as hv_synic_cleanup checks for it */
> > mb();
> > cpuhp_remove_state(hyperv_cpuhp_online);
> > @@ -2306,7 +2308,6 @@ static void hv_crash_handler(struct pt_regs *regs)
> > * doing the cleanup for current CPU only. This should be sufficient
> > * for kdump.
> > */
> > - vmbus_connection.conn_state = DISCONNECTED;
> > cpu = smp_processor_id();
> > hv_stimer_cleanup(cpu);
> > hv_synic_disable_regs(cpu);
>
> --
> Vitaly
^ permalink raw reply
* RE: [PATCH 2/4] x86/Hyper-V: Free hv_panic_page when fail to register kmsg dump
From: Michael Kelley @ 2020-03-19 0:38 UTC (permalink / raw)
To: ltykernel@gmail.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Wei Liu, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org
Cc: Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, vkuznets
In-Reply-To: <20200317132523.1508-3-Tianyu.Lan@microsoft.com>
From: ltykernel@gmail.com <ltykernel@gmail.com> Sent: Tuesday, March 17, 2020 6:25 AM
>
> If fail to register kmsg dump on Hyper-V platform, hv_panic_page
> will not be used anywhere. So free and reset it.
Slight commit message wording cleanup:
If kmsg_dump_register() fails, hv_panic_page will not be used
anywhere. So free and reset it.
>
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> drivers/hv/vmbus_drv.c | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index b56b9fb9bd90..b043efea092a 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -1385,9 +1385,13 @@ static int vmbus_bus_init(void)
> hv_panic_page = (void *)hv_alloc_hyperv_zeroed_page();
> if (hv_panic_page) {
> ret = kmsg_dump_register(&hv_kmsg_dumper);
> - if (ret)
> + if (ret) {
> pr_err("Hyper-V: kmsg dump register "
> "error 0x%x\n", ret);
> + hv_free_hyperv_page(
> + (unsigned long)hv_panic_page);
> + hv_panic_page = NULL;
> + }
> } else
> pr_err("Hyper-V: panic message page memory "
> "allocation failed");
> --
> 2.14.5
^ permalink raw reply
* RE: [PATCH 3/4] x86/Hyper-V: Trigger crash enlightenment only once during system crash.
From: Michael Kelley @ 2020-03-19 0:45 UTC (permalink / raw)
To: ltykernel@gmail.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Wei Liu, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org
Cc: Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, vkuznets
In-Reply-To: <20200317132523.1508-4-Tianyu.Lan@microsoft.com>
From: ltykernel@gmail.com <ltykernel@gmail.com> Sent: Tuesday, March 17, 2020 6:25 AM
>
> Hyper-V expects guest only triggers crash enlightenment.
> The second crash notify will be ignored by Hyper-V.
>
> Current code may trigger crash enlightenment during system
> panic twice.
> 1) The enlightenment is triggered in hyperv_panic/die_event()
> via hyperv_report_panic().
> 2) hv_kmsg_dump() reports kmsg to host via hyperv_report_panic_msg().
>
> Fix it. If kmsg dump is registered successfully, just report
> kmsg via hyperv_report_panic_msg() and not report register values
> via hyperv_report_panic().
Suggested wording improvements:
When a guest VM panics, Hyper-V should be notified only once via the
crash synthetic MSRs. Current Linux code might write these crash MSRs
twice during a system panic:
1) hyperv_panic/die_event() calling hyperv_report_panic()
2) hv_kmsg_dump() calling hyperv_report_panic_msg()
Fix this by not calling hyperv_report_panic() if a kmsg dump has been
successfully registered. The notification will happen later via
hyperv_report_panic_msg().
>
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> drivers/hv/vmbus_drv.c | 14 ++++++++++++--
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
> index b043efea092a..1787d6246251 100644
> --- a/drivers/hv/vmbus_drv.c
> +++ b/drivers/hv/vmbus_drv.c
> @@ -55,7 +55,12 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned
> long val,
>
> vmbus_initiate_unload(true);
>
> - if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
> + /*
> + * Crash notify only can be triggered once. If crash notify
> + * message is available, just report kmsg to crash buffer.
> + */
> + if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE
> + && !hv_panic_page) {
> regs = current_pt_regs();
> hyperv_report_panic(regs, val);
> }
> @@ -68,7 +73,12 @@ static int hyperv_die_event(struct notifier_block *nb, unsigned long
> val,
> struct die_args *die = (struct die_args *)args;
> struct pt_regs *regs = die->regs;
>
> - hyperv_report_panic(regs, val);
> + /*
> + * Crash notify only can be triggered once. If crash notify
> + * message is available, just report kmsg to crash buffer.
> + */
> + if (!hv_panic_page)
> + hyperv_report_panic(regs, val);
> return NOTIFY_DONE;
> }
>
> --
> 2.14.5
^ permalink raw reply
* RE: [PATCH 4/4] x86/Hyper-V: Report crash register data or ksmg before running crash kernel
From: Michael Kelley @ 2020-03-19 0:51 UTC (permalink / raw)
To: ltykernel@gmail.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Wei Liu, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org
Cc: Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, vkuznets
In-Reply-To: <20200317132523.1508-5-Tianyu.Lan@microsoft.com>
From: ltykernel@gmail.com <ltykernel@gmail.com> Sent: Tuesday, March 17, 2020 6:25 AM
>
> Hyper-V expects to get crash register data or kmsg via crash
> enlightenment when guest crash happens. crash_kexec_post_notifiers
> is default to be false and crash kernel runs before calling hv
> panic callback and dumping kmsg. In this case, Hyper-V doesn't
> get crash register data or kmsg from guest. Set crash_kexec_post
> _notifiers to be true for Hyper-V VM and fix it.
Suggested wording tweaks:
We want to notify Hyper-V when a Linux guest VM crash occurs, so
there is a record of the crash even when kdump is enabled. But
crash_kexec_post_notifiers defaults to "false", so the kdump kernel
runs before the notifiers and Hyper-V never gets notified. Fix this by
always setting crash_kexec_post_notifiers to be true for Hyper-V VMs.
>
> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
> ---
> arch/x86/kernel/cpu/mshyperv.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index caa032ce3fe3..5e296a7e6036 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -263,6 +263,16 @@ static void __init ms_hyperv_init_platform(void)
> cpuid_eax(HYPERV_CPUID_NESTED_FEATURES);
> }
>
> + /*
> + * Hyper-V expects to get crash register data or kmsg when
> + * crash enlightment is available and system crashes. Set
> + * crash_kexec_post_notifiers to be true to make sure that
> + * calling crash enlightment interface before running kdump
> + * kernel.
> + */
> + if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
> + crash_kexec_post_notifiers = true;
> +
> #ifdef CONFIG_X86_LOCAL_APIC
> if (ms_hyperv.features & HV_X64_ACCESS_FREQUENCY_MSRS &&
> ms_hyperv.misc_features & HV_FEATURE_FREQUENCY_MSRS_AVAILABLE) {
> --
> 2.14.5
^ permalink raw reply
* RE: [PATCH 0/4] x86/Hyper-V: Panic code path fixes
From: Michael Kelley @ 2020-03-19 0:57 UTC (permalink / raw)
To: ltykernel@gmail.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Wei Liu, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org
Cc: Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, vkuznets
In-Reply-To: <20200317132523.1508-1-Tianyu.Lan@microsoft.com>
From: ltykernel@gmail.com <ltykernel@gmail.com> Sent: Tuesday, March 17, 2020 6:25 AM
>
> This patchset fixes some issues in the Hyper-V panic code path.
> Patch 1 resolves issue that panic system still responses network
> packets.
> Patch 2-3 resolves crash enlightenment issues.
> Patch 4 is to set crash_kexec_post_notifiers to true for Hyper-V
> VM in order to report crash data or kmsg to host before running
> kdump kernel.
I still see an issue that isn't addressed by these patches. The VMbus
driver registers a "die notifier" and a "panic notifier". But die() will
eventually call panic() if panic_on_oops is set (which I think it typically
is). If the CRASH_NOTIFY_MSG option is *not* enabled, then
hyperv_report_panic() could get called by the die notifier, and then
again by the panic notifier.
Do we even need the "die notifier"? If it was removed, there would
not be any notification to Hyper-V via the die() path unless panic_on_oops
is set, which I think is actually the correct behavior. I'm not
completely clear on what is supposed to happen in general to the
Linux kernel if panic_on_oops is not set. Does it try to continue to run?
If so, then we should not be notifying Hyper-V if panic_on_oops is not
set, and removing the die notifier is the right thing to do.
Michael
>
> Tianyu Lan (4):
> x86/Hyper-V: Unload vmbus channel in hv panic callback
> x86/Hyper-V: Free hv_panic_page when fail to register kmsg dump
> x86/Hyper-V: Trigger crash enlightenment only once during system
> crash.
> x86/Hyper-V: Report crash register data or ksmg before running crash
> kernel
>
> arch/x86/kernel/cpu/mshyperv.c | 10 ++++++++++
> drivers/hv/channel_mgmt.c | 5 +++++
> drivers/hv/vmbus_drv.c | 35 +++++++++++++++++++++++++----------
> 3 files changed, 40 insertions(+), 10 deletions(-)
>
> --
> 2.14.5
^ permalink raw reply
* [PATCH v7 0/5] x86/kvm/hyper-v: add support for synthetic debugger
From: Jon Doron @ 2020-03-19 5:58 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
Add support for the synthetic debugger interface of hyper-v, the
synthetic debugger has 2 modes.
1. Use a set of MSRs to send/recv information (undocumented so it's not
going to the hyperv-tlfs.h)
2. Use hypercalls
The first mode is based the following MSRs:
1. Control/Status MSRs which either asks for a send/recv .
2. Send/Recv MSRs each holds GPA where the send/recv buffers are.
3. Pending MSR, holds a GPA to a PAGE that simply has a boolean that
indicates if there is data pending to issue a recv VMEXIT.
The first mode implementation is to simply exit to user-space when
either the control MSR or the pending MSR are being set.
Then it's up-to userspace to implement the rest of the logic of sending/recving.
In the second mode instead of using MSRs KNet will simply issue
Hypercalls with the information to send/recv, in this mode the data
being transferred is UDP encapsulated, unlike in the previous mode in
which you get just the data to send.
The new hypercalls will exit to userspace which will be incharge of
re-encapsulating if needed the UDP packets to be sent.
There is an issue though in which KDNet does not respect the hypercall
page and simply issues vmcall/vmmcall instructions depending on the cpu
type expecting them to be handled as it a real hypercall was issued.
It's important to note that part of this feature has been subject to be
removed in future versions of Windows, which is why some of the
defintions will not be present the the TLFS but in the kvm hyperv header
instead.
Jon Doron (5):
x86/kvm/hyper-v: Explicitly align hcall param for kvm_hyperv_exit
x86/hyper-v: Add synthetic debugger definitions
x86/kvm/hyper-v: Add support for synthetic debugger capability
x86/kvm/hyper-v: enable hypercalls without hypercall page with syndbg
x86/kvm/hyper-v: Add support for synthetic debugger via hypercalls
Documentation/virt/kvm/api.rst | 18 +++
arch/x86/include/asm/hyperv-tlfs.h | 6 +
arch/x86/include/asm/kvm_host.h | 14 +++
arch/x86/kvm/hyperv.c | 177 ++++++++++++++++++++++++++++-
arch/x86/kvm/hyperv.h | 33 ++++++
arch/x86/kvm/trace.h | 51 +++++++++
arch/x86/kvm/x86.c | 13 +++
include/uapi/linux/kvm.h | 13 +++
8 files changed, 323 insertions(+), 2 deletions(-)
--
2.24.1
^ permalink raw reply
* [PATCH v7 1/5] x86/kvm/hyper-v: Explicitly align hcall param for kvm_hyperv_exit
From: Jon Doron @ 2020-03-19 5:58 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319055842.673513-1-arilou@gmail.com>
The problem the patch is trying to address is the fact that 'struct
kvm_hyperv_exit' has different layout on when compiling in 32 and 64 bit
modes.
In 64-bit mode the default alignment boundary is 64 bits thus
forcing extra gaps after 'type' and 'msr' but in 32-bit mode the
boundary is at 32 bits thus no extra gaps.
This is an issue as even when the kernel is 64 bit, the userspace using
the interface can be both 32 and 64 bit but the same 32 bit userspace has
to work with 32 bit kernel.
The issue is fixed by forcing the 64 bit layout, this leads to ABI
change for 32 bit builds and while we are obviously breaking '32 bit
userspace with 32 bit kernel' case, we're fixing the '32 bit userspace
with 64 bit kernel' one.
As the interface has no (known) users and 32 bit KVM is rather baroque
nowadays, this seems like a reasonable decision.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
Documentation/virt/kvm/api.rst | 2 ++
include/uapi/linux/kvm.h | 2 ++
2 files changed, 4 insertions(+)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index ebd383fba939..4872c47bbcff 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -5025,9 +5025,11 @@ EOI was received.
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
__u32 type;
+ __u32 pad1;
union {
struct {
__u32 msr;
+ __u32 pad2;
__u64 control;
__u64 evt_page;
__u64 msg_page;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 4b95f9a31a2f..7ee0ddc4c457 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -189,9 +189,11 @@ struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
__u32 type;
+ __u32 pad1;
union {
struct {
__u32 msr;
+ __u32 pad2;
__u64 control;
__u64 evt_page;
__u64 msg_page;
--
2.24.1
^ permalink raw reply related
* [PATCH v7 2/5] x86/hyper-v: Add synthetic debugger definitions
From: Jon Doron @ 2020-03-19 5:58 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319055842.673513-1-arilou@gmail.com>
Hyper-V synthetic debugger has two modes, one that uses MSRs and
the other that use Hypercalls.
Add all the required definitions to both types of synthetic debugger
interface.
Some of the required new CPUIDs and MSRs are not documented in the TLFS
so they are in hyperv.h instead.
The reason they are not documented is because they are subjected to be
removed in future versions of Windows.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
arch/x86/include/asm/hyperv-tlfs.h | 6 ++++++
arch/x86/kvm/hyperv.h | 27 +++++++++++++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index 92abc1e42bfc..671ce2a39d4b 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -131,6 +131,8 @@
#define HV_FEATURE_FREQUENCY_MSRS_AVAILABLE BIT(8)
/* Crash MSR available */
#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE BIT(10)
+/* Support for debug MSRs available */
+#define HV_FEATURE_DEBUG_MSRS_AVAILABLE BIT(11)
/* stimer Direct Mode is available */
#define HV_STIMER_DIRECT_MODE_AVAILABLE BIT(19)
@@ -376,6 +378,9 @@ struct hv_tsc_emulation_status {
#define HVCALL_SEND_IPI_EX 0x0015
#define HVCALL_POST_MESSAGE 0x005c
#define HVCALL_SIGNAL_EVENT 0x005d
+#define HVCALL_POST_DEBUG_DATA 0x0069
+#define HVCALL_RETRIEVE_DEBUG_DATA 0x006a
+#define HVCALL_RESET_DEBUG_SESSION 0x006b
#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
@@ -419,6 +424,7 @@ enum HV_GENERIC_SET_FORMAT {
#define HV_STATUS_INVALID_HYPERCALL_INPUT 3
#define HV_STATUS_INVALID_ALIGNMENT 4
#define HV_STATUS_INVALID_PARAMETER 5
+#define HV_STATUS_OPERATION_DENIED 8
#define HV_STATUS_INSUFFICIENT_MEMORY 11
#define HV_STATUS_INVALID_PORT_ID 17
#define HV_STATUS_INVALID_CONNECTION_ID 18
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index 757cb578101c..5e4780bf6dd7 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -23,6 +23,33 @@
#include <linux/kvm_host.h>
+/*
+ * The #defines related to the synthetic debugger are required by KDNet, but
+ * they are not documented in the Hyper-V TLFS because the synthetic debugger
+ * functionality has been deprecated and is subject to removal in future versions
+ * of Windows.
+ */
+#define HYPERV_CPUID_SYNDBG_VENDOR_AND_MAX_FUNCTIONS 0x40000080
+#define HYPERV_CPUID_SYNDBG_INTERFACE 0x40000081
+#define HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES 0x40000082
+
+/*
+ * Hyper-V synthetic debugger platform capabilities
+ * These are HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX bits.
+ */
+#define HV_X64_SYNDBG_CAP_ALLOW_KERNEL_DEBUGGING BIT(1)
+
+/* Hyper-V Synthetic debug options MSR */
+#define HV_X64_MSR_SYNDBG_CONTROL 0x400000F1
+#define HV_X64_MSR_SYNDBG_STATUS 0x400000F2
+#define HV_X64_MSR_SYNDBG_SEND_BUFFER 0x400000F3
+#define HV_X64_MSR_SYNDBG_RECV_BUFFER 0x400000F4
+#define HV_X64_MSR_SYNDBG_PENDING_BUFFER 0x400000F5
+#define HV_X64_MSR_SYNDBG_OPTIONS 0x400000FF
+
+/* Hyper-V HV_X64_MSR_SYNDBG_OPTIONS bits */
+#define HV_X64_SYNDBG_OPTION_USE_HCALLS BIT(2)
+
static inline struct kvm_vcpu_hv *vcpu_to_hv_vcpu(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.hyperv;
--
2.24.1
^ permalink raw reply related
* [PATCH v7 4/5] x86/kvm/hyper-v: enable hypercalls without hypercall page with syndbg
From: Jon Doron @ 2020-03-19 5:58 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319055842.673513-1-arilou@gmail.com>
Microsoft's kdvm.dll dbgtransport module does not respect the hypercall
page and simply identifies the CPU being used (AMD/Intel) and according
to it simply makes hypercalls with the relevant instruction
(vmmcall/vmcall respectively).
The relevant function in kdvm is KdHvConnectHypervisor which first checks
if the hypercall page has been enabled via HV_X64_MSR_HYPERCALL_ENABLE,
and in case it was not it simply sets the HV_X64_MSR_GUEST_OS_ID to
0x1000101010001 which means:
build_number = 0x0001
service_version = 0x01
minor_version = 0x01
major_version = 0x01
os_id = 0x00 (Undefined)
vendor_id = 1 (Microsoft)
os_type = 0 (A value of 0 indicates a proprietary, closed source OS)
and starts issuing the hypercall without setting the hypercall page.
To resolve this issue simply enable hypercalls also if the guest_os_id
is not 0 and the syndbg feature is enabled.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
arch/x86/kvm/hyperv.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 4c482ef1eaa9..67f9893150c3 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1645,7 +1645,10 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *current_vcpu, u64 ingpa, u64 outgpa,
bool kvm_hv_hypercall_enabled(struct kvm *kvm)
{
- return READ_ONCE(kvm->arch.hyperv.hv_hypercall) & HV_X64_MSR_HYPERCALL_ENABLE;
+ struct kvm_hv *hv = &kvm->arch.hyperv;
+
+ return READ_ONCE(hv->hv_hypercall) & HV_X64_MSR_HYPERCALL_ENABLE ||
+ (hv->hv_syndbg.active && READ_ONCE(hv->hv_guest_os_id) != 0);
}
static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result)
--
2.24.1
^ permalink raw reply related
* [PATCH v7 5/5] x86/kvm/hyper-v: Add support for synthetic debugger via hypercalls
From: Jon Doron @ 2020-03-19 5:58 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319055842.673513-1-arilou@gmail.com>
There is another mode for the synthetic debugger which uses hypercalls
to send/recv network data instead of the MSR interface.
This interface is much slower and less recommended since you might get
a lot of VMExits while KDVM polling for new packets to recv, rather
than simply checking the pending page to see if there is data avialble
and then request.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
arch/x86/kvm/hyperv.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 67f9893150c3..a62a330ed322 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1827,6 +1827,34 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
}
ret = kvm_hv_send_ipi(vcpu, ingpa, outgpa, true, false);
break;
+ case HVCALL_POST_DEBUG_DATA:
+ case HVCALL_RETRIEVE_DEBUG_DATA:
+ if (unlikely(fast)) {
+ ret = HV_STATUS_INVALID_PARAMETER;
+ break;
+ }
+ fallthrough;
+ case HVCALL_RESET_DEBUG_SESSION: {
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ if (!syndbg->active) {
+ ret = HV_STATUS_INVALID_HYPERCALL_CODE;
+ break;
+ }
+
+ if (!(syndbg->options & HV_X64_SYNDBG_OPTION_USE_HCALLS)) {
+ ret = HV_STATUS_OPERATION_DENIED;
+ break;
+ }
+ vcpu->run->exit_reason = KVM_EXIT_HYPERV;
+ vcpu->run->hyperv.type = KVM_EXIT_HYPERV_HCALL;
+ vcpu->run->hyperv.u.hcall.input = param;
+ vcpu->run->hyperv.u.hcall.params[0] = ingpa;
+ vcpu->run->hyperv.u.hcall.params[1] = outgpa;
+ vcpu->arch.complete_userspace_io =
+ kvm_hv_hypercall_complete_userspace;
+ return 0;
+ }
default:
ret = HV_STATUS_INVALID_HYPERCALL_CODE;
break;
--
2.24.1
^ permalink raw reply related
* [PATCH v7 3/5] x86/kvm/hyper-v: Add support for synthetic debugger capability
From: Jon Doron @ 2020-03-19 5:58 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319055842.673513-1-arilou@gmail.com>
Add support for Hyper-V synthetic debugger (syndbg) interface.
The syndbg interface is using MSRs to emulate a way to send/recv packets
data.
The debug transport dll (kdvm/kdnet) will identify if Hyper-V is enabled
and if it supports the synthetic debugger interface it will attempt to
use it, instead of trying to initialize a network adapter.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
Documentation/virt/kvm/api.rst | 16 ++++
arch/x86/include/asm/kvm_host.h | 14 ++++
arch/x86/kvm/hyperv.c | 144 +++++++++++++++++++++++++++++++-
arch/x86/kvm/hyperv.h | 6 ++
arch/x86/kvm/trace.h | 51 +++++++++++
arch/x86/kvm/x86.c | 13 +++
include/uapi/linux/kvm.h | 11 +++
7 files changed, 254 insertions(+), 1 deletion(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 4872c47bbcff..fe992dcf4f93 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -5024,6 +5024,7 @@ EOI was received.
struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
+ #define KVM_EXIT_HYPERV_SYNDBG 3
__u32 type;
__u32 pad1;
union {
@@ -5039,6 +5040,15 @@ EOI was received.
__u64 result;
__u64 params[2];
} hcall;
+ struct {
+ __u32 msr;
+ __u32 pad2;
+ __u64 control;
+ __u64 status;
+ __u64 send_page;
+ __u64 recv_page;
+ __u64 pending_page;
+ } syndbg;
} u;
};
/* KVM_EXIT_HYPERV */
@@ -5055,6 +5065,12 @@ Hyper-V SynIC state change. Notification is used to remap SynIC
event/message pages and to enable/disable SynIC messages/events processing
in userspace.
+ - KVM_EXIT_HYPERV_SYNDBG -- synchronously notify user-space about
+
+Hyper-V Synthetic debugger state change. Notification is used to either update
+the pending_page location or to send a control command (send the buffer located
+in send_page or recv a buffer to recv_page).
+
::
/* KVM_EXIT_ARM_NISV */
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 98959e8cd448..c09fa7401b13 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -854,6 +854,19 @@ struct kvm_apic_map {
struct kvm_lapic *phys_map[];
};
+/* Hyper-V synthetic debugger (SynDbg)*/
+struct kvm_hv_syndbg {
+ struct {
+ u64 control;
+ u64 status;
+ u64 send_page;
+ u64 recv_page;
+ u64 pending_page;
+ } control;
+ u64 options;
+ bool active;
+};
+
/* Hyper-V emulation context */
struct kvm_hv {
struct mutex hv_lock;
@@ -877,6 +890,7 @@ struct kvm_hv {
atomic_t num_mismatched_vp_indexes;
struct hv_partition_assist_pg *hv_pa_pg;
+ struct kvm_hv_syndbg hv_syndbg;
};
enum kvm_irqchip_mode {
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index a86fda7a1d03..4c482ef1eaa9 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -266,6 +266,111 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
return ret;
}
+void kvm_hv_activate_syndbg(struct kvm_vcpu *vcpu)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ syndbg->active = true;
+}
+
+static int kvm_hv_syndbg_complete_userspace(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_hv *hv = &kvm->arch.hyperv;
+
+ if (vcpu->run->hyperv.u.syndbg.msr == HV_X64_MSR_SYNDBG_CONTROL)
+ hv->hv_syndbg.control.status =
+ vcpu->run->hyperv.u.syndbg.status;
+ return 1;
+}
+
+static void syndbg_exit(struct kvm_vcpu *vcpu, u32 msr)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+ struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
+
+ hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNDBG;
+ hv_vcpu->exit.u.syndbg.msr = msr;
+ hv_vcpu->exit.u.syndbg.control = syndbg->control.control;
+ hv_vcpu->exit.u.syndbg.send_page = syndbg->control.send_page;
+ hv_vcpu->exit.u.syndbg.recv_page = syndbg->control.recv_page;
+ hv_vcpu->exit.u.syndbg.pending_page = syndbg->control.pending_page;
+ vcpu->arch.complete_userspace_io =
+ kvm_hv_syndbg_complete_userspace;
+
+ kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
+}
+
+static void syndbg_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ if (!syndbg->active)
+ return;
+
+ trace_kvm_hv_syndbg_set_msr(vcpu->vcpu_id,
+ vcpu_to_hv_vcpu(vcpu)->vp_index, msr, data);
+ switch (msr) {
+ case HV_X64_MSR_SYNDBG_CONTROL:
+ syndbg->control.control = data;
+ syndbg_exit(vcpu, msr);
+ break;
+ case HV_X64_MSR_SYNDBG_STATUS:
+ syndbg->control.status = data;
+ break;
+ case HV_X64_MSR_SYNDBG_SEND_BUFFER:
+ syndbg->control.send_page = data;
+ break;
+ case HV_X64_MSR_SYNDBG_RECV_BUFFER:
+ syndbg->control.recv_page = data;
+ break;
+ case HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ syndbg->control.pending_page = data;
+ syndbg_exit(vcpu, msr);
+ break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ syndbg->options = data;
+ break;
+ default:
+ break;
+ }
+}
+
+static void syndbg_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ if (!syndbg->active)
+ return;
+
+ switch (msr) {
+ case HV_X64_MSR_SYNDBG_CONTROL:
+ *pdata = syndbg->control.control;
+ break;
+ case HV_X64_MSR_SYNDBG_STATUS:
+ *pdata = syndbg->control.status;
+ break;
+ case HV_X64_MSR_SYNDBG_SEND_BUFFER:
+ *pdata = syndbg->control.send_page;
+ break;
+ case HV_X64_MSR_SYNDBG_RECV_BUFFER:
+ *pdata = syndbg->control.recv_page;
+ break;
+ case HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ *pdata = syndbg->control.pending_page;
+ break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ *pdata = syndbg->options;
+ break;
+ default:
+ break;
+ }
+
+ trace_kvm_hv_syndbg_get_msr(vcpu->vcpu_id,
+ vcpu_to_hv_vcpu(vcpu)->vp_index, msr,
+ *pdata);
+}
+
static int synic_get_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 *pdata,
bool host)
{
@@ -800,6 +905,8 @@ static bool kvm_hv_msr_partition_wide(u32 msr)
case HV_X64_MSR_REENLIGHTENMENT_CONTROL:
case HV_X64_MSR_TSC_EMULATION_CONTROL:
case HV_X64_MSR_TSC_EMULATION_STATUS:
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
r = true;
break;
}
@@ -1061,6 +1168,10 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,
if (!host)
return 1;
break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ syndbg_set_msr(vcpu, msr, data);
+ break;
default:
vcpu_unimpl(vcpu, "Hyper-V unhandled wrmsr: 0x%x data 0x%llx\n",
msr, data);
@@ -1227,6 +1338,10 @@ static int kvm_hv_get_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
case HV_X64_MSR_TSC_EMULATION_STATUS:
data = hv->hv_tsc_emulation_status;
break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ syndbg_get_msr(vcpu, msr, pdata);
+ break;
default:
vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
return 1;
@@ -1797,8 +1912,12 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
{ .function = HYPERV_CPUID_ENLIGHTMENT_INFO },
{ .function = HYPERV_CPUID_IMPLEMENT_LIMITS },
{ .function = HYPERV_CPUID_NESTED_FEATURES },
+ { .function = HYPERV_CPUID_SYNDBG_VENDOR_AND_MAX_FUNCTIONS },
+ { .function = HYPERV_CPUID_SYNDBG_INTERFACE },
+ { .function = HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES },
};
int i, nent = ARRAY_SIZE(cpuid_entries);
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
if (kvm_x86_ops->nested_get_evmcs_version)
evmcs_ver = kvm_x86_ops->nested_get_evmcs_version(vcpu);
@@ -1821,7 +1940,9 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
case HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS:
memcpy(signature, "Linux KVM Hv", 12);
- ent->eax = HYPERV_CPUID_NESTED_FEATURES;
+ ent->eax = syndbg->active ?
+ HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES :
+ HYPERV_CPUID_NESTED_FEATURES;
ent->ebx = signature[0];
ent->ecx = signature[1];
ent->edx = signature[2];
@@ -1856,9 +1977,12 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
ent->ebx |= HV_X64_POST_MESSAGES;
ent->ebx |= HV_X64_SIGNAL_EVENTS;
+ ent->ebx |= HV_X64_DEBUGGING;
ent->edx |= HV_FEATURE_FREQUENCY_MSRS_AVAILABLE;
ent->edx |= HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE;
+ ent->edx |= HV_X64_GUEST_DEBUGGING_AVAILABLE;
+ ent->edx |= HV_FEATURE_DEBUG_MSRS_AVAILABLE;
/*
* Direct Synthetic timers only make sense with in-kernel
@@ -1903,6 +2027,24 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
break;
+ case HYPERV_CPUID_SYNDBG_VENDOR_AND_MAX_FUNCTIONS:
+ memcpy(signature, "Linux KVM Hv", 12);
+
+ ent->eax = 0;
+ ent->ebx = signature[0];
+ ent->ecx = signature[1];
+ ent->edx = signature[2];
+ break;
+
+ case HYPERV_CPUID_SYNDBG_INTERFACE:
+ memcpy(signature, "VS#1\0\0\0\0\0\0\0\0", 12);
+ ent->eax = signature[0];
+ break;
+
+ case HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES:
+ ent->eax |= HV_X64_SYNDBG_CAP_ALLOW_KERNEL_DEBUGGING;
+ break;
+
default:
break;
}
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index 5e4780bf6dd7..45aaaf08bf15 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -73,6 +73,11 @@ static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
return hv_vcpu_to_vcpu(container_of(synic, struct kvm_vcpu_hv, synic));
}
+static inline struct kvm_hv_syndbg *vcpu_to_hv_syndbg(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->kvm->arch.hyperv.hv_syndbg;
+}
+
int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host);
int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host);
@@ -83,6 +88,7 @@ void kvm_hv_irq_routing_update(struct kvm *kvm);
int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
int kvm_hv_activate_synic(struct kvm_vcpu *vcpu, bool dont_zero_synic_pages);
+void kvm_hv_activate_syndbg(struct kvm_vcpu *vcpu);
void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
void kvm_hv_vcpu_postcreate(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index f194dd058470..bf6c3852868d 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -1515,6 +1515,57 @@ TRACE_EVENT(kvm_nested_vmenter_failed,
__print_symbolic(__entry->err, VMX_VMENTER_INSTRUCTION_ERRORS))
);
+/*
+ * Tracepoint for syndbg_set_msr.
+ */
+TRACE_EVENT(kvm_hv_syndbg_set_msr,
+ TP_PROTO(int vcpu_id, u32 vp_index, u32 msr, u64 data),
+ TP_ARGS(vcpu_id, vp_index, msr, data),
+
+ TP_STRUCT__entry(
+ __field(int, vcpu_id)
+ __field(u32, vp_index)
+ __field(u32, msr)
+ __field(u64, data)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->vp_index = vp_index;
+ __entry->msr = msr;
+ __entry->data = data;
+ ),
+
+ TP_printk("vcpu_id %d vp_index %u msr 0x%x data 0x%llx",
+ __entry->vcpu_id, __entry->vp_index, __entry->msr,
+ __entry->data)
+);
+
+/*
+ * Tracepoint for syndbg_get_msr.
+ */
+TRACE_EVENT(kvm_hv_syndbg_get_msr,
+ TP_PROTO(int vcpu_id, u32 vp_index, u32 msr, u64 data),
+ TP_ARGS(vcpu_id, vp_index, msr, data),
+
+ TP_STRUCT__entry(
+ __field(int, vcpu_id)
+ __field(u32, vp_index)
+ __field(u32, msr)
+ __field(u64, data)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->vp_index = vp_index;
+ __entry->msr = msr;
+ __entry->data = data;
+ ),
+
+ TP_printk("vcpu_id %d vp_index %u msr 0x%x data 0x%llx",
+ __entry->vcpu_id, __entry->vp_index, __entry->msr,
+ __entry->data)
+);
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 3156e25b0774..7bfa30d7a525 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1214,6 +1214,10 @@ static const u32 emulated_msrs_all[] = {
HV_X64_MSR_VP_ASSIST_PAGE,
HV_X64_MSR_REENLIGHTENMENT_CONTROL, HV_X64_MSR_TSC_EMULATION_CONTROL,
HV_X64_MSR_TSC_EMULATION_STATUS,
+ HV_X64_MSR_SYNDBG_OPTIONS,
+ HV_X64_MSR_SYNDBG_CONTROL, HV_X64_MSR_SYNDBG_STATUS,
+ HV_X64_MSR_SYNDBG_SEND_BUFFER, HV_X64_MSR_SYNDBG_RECV_BUFFER,
+ HV_X64_MSR_SYNDBG_PENDING_BUFFER,
MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
MSR_KVM_PV_EOI_EN,
@@ -2906,6 +2910,8 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
*/
break;
case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ case HV_X64_MSR_SYNDBG_OPTIONS:
case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
case HV_X64_MSR_CRASH_CTL:
case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:
@@ -3151,6 +3157,8 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
msr_info->data = 0x20000000;
break;
case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ case HV_X64_MSR_SYNDBG_OPTIONS:
case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
case HV_X64_MSR_CRASH_CTL:
case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:
@@ -3323,6 +3331,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_HYPERV_TLBFLUSH:
case KVM_CAP_HYPERV_SEND_IPI:
case KVM_CAP_HYPERV_CPUID:
+ case KVM_CAP_HYPERV_SYNDBG:
case KVM_CAP_PCI_SEGMENT:
case KVM_CAP_DEBUGREGS:
case KVM_CAP_X86_ROBUST_SINGLESTEP:
@@ -4178,6 +4187,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
return -EINVAL;
switch (cap->cap) {
+ case KVM_CAP_HYPERV_SYNDBG:
+ kvm_hv_activate_syndbg(vcpu);
+ return 0;
+
case KVM_CAP_HYPERV_SYNIC2:
if (cap->args[0])
return -EINVAL;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 7ee0ddc4c457..0b573e1155c8 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -188,6 +188,7 @@ struct kvm_s390_cmma_log {
struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
+#define KVM_EXIT_HYPERV_SYNDBG 3
__u32 type;
__u32 pad1;
union {
@@ -203,6 +204,15 @@ struct kvm_hyperv_exit {
__u64 result;
__u64 params[2];
} hcall;
+ struct {
+ __u32 msr;
+ __u32 pad2;
+ __u64 control;
+ __u64 status;
+ __u64 send_page;
+ __u64 recv_page;
+ __u64 pending_page;
+ } syndbg;
} u;
};
@@ -1012,6 +1022,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ARM_NISV_TO_USER 177
#define KVM_CAP_ARM_INJECT_EXT_DABT 178
#define KVM_CAP_S390_VCPU_RESETS 179
+#define KVM_CAP_HYPERV_SYNDBG 180
#ifdef KVM_CAP_IRQ_ROUTING
--
2.24.1
^ permalink raw reply related
* [PATCH v8 1/5] x86/kvm/hyper-v: Explicitly align hcall param for kvm_hyperv_exit
From: Jon Doron @ 2020-03-19 6:38 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319063836.678562-1-arilou@gmail.com>
The problem the patch is trying to address is the fact that 'struct
kvm_hyperv_exit' has different layout on when compiling in 32 and 64 bit
modes.
In 64-bit mode the default alignment boundary is 64 bits thus
forcing extra gaps after 'type' and 'msr' but in 32-bit mode the
boundary is at 32 bits thus no extra gaps.
This is an issue as even when the kernel is 64 bit, the userspace using
the interface can be both 32 and 64 bit but the same 32 bit userspace has
to work with 32 bit kernel.
The issue is fixed by forcing the 64 bit layout, this leads to ABI
change for 32 bit builds and while we are obviously breaking '32 bit
userspace with 32 bit kernel' case, we're fixing the '32 bit userspace
with 64 bit kernel' one.
As the interface has no (known) users and 32 bit KVM is rather baroque
nowadays, this seems like a reasonable decision.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
Documentation/virt/kvm/api.rst | 2 ++
include/uapi/linux/kvm.h | 2 ++
2 files changed, 4 insertions(+)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index ebd383fba939..4872c47bbcff 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -5025,9 +5025,11 @@ EOI was received.
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
__u32 type;
+ __u32 pad1;
union {
struct {
__u32 msr;
+ __u32 pad2;
__u64 control;
__u64 evt_page;
__u64 msg_page;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 4b95f9a31a2f..7ee0ddc4c457 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -189,9 +189,11 @@ struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
__u32 type;
+ __u32 pad1;
union {
struct {
__u32 msr;
+ __u32 pad2;
__u64 control;
__u64 evt_page;
__u64 msg_page;
--
2.24.1
^ permalink raw reply related
* [PATCH v8 0/5] x86/kvm/hyper-v: add support for synthetic debugger
From: Jon Doron @ 2020-03-19 6:38 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
Add support for the synthetic debugger interface of hyper-v, the
synthetic debugger has 2 modes.
1. Use a set of MSRs to send/recv information (undocumented so it's not
going to the hyperv-tlfs.h)
2. Use hypercalls
The first mode is based the following MSRs:
1. Control/Status MSRs which either asks for a send/recv .
2. Send/Recv MSRs each holds GPA where the send/recv buffers are.
3. Pending MSR, holds a GPA to a PAGE that simply has a boolean that
indicates if there is data pending to issue a recv VMEXIT.
The first mode implementation is to simply exit to user-space when
either the control MSR or the pending MSR are being set.
Then it's up-to userspace to implement the rest of the logic of sending/recving.
In the second mode instead of using MSRs KNet will simply issue
Hypercalls with the information to send/recv, in this mode the data
being transferred is UDP encapsulated, unlike in the previous mode in
which you get just the data to send.
The new hypercalls will exit to userspace which will be incharge of
re-encapsulating if needed the UDP packets to be sent.
There is an issue though in which KDNet does not respect the hypercall
page and simply issues vmcall/vmmcall instructions depending on the cpu
type expecting them to be handled as it a real hypercall was issued.
It's important to note that part of this feature has been subject to be
removed in future versions of Windows, which is why some of the
defintions will not be present the the TLFS but in the kvm hyperv header
instead.
Jon Doron (5):
x86/kvm/hyper-v: Explicitly align hcall param for kvm_hyperv_exit
x86/hyper-v: Add synthetic debugger definitions
x86/kvm/hyper-v: Add support for synthetic debugger capability
x86/kvm/hyper-v: enable hypercalls without hypercall page with syndbg
x86/kvm/hyper-v: Add support for synthetic debugger via hypercalls
Documentation/virt/kvm/api.rst | 18 +++
arch/x86/include/asm/hyperv-tlfs.h | 6 +
arch/x86/include/asm/kvm_host.h | 14 +++
arch/x86/kvm/hyperv.c | 179 ++++++++++++++++++++++++++++-
arch/x86/kvm/hyperv.h | 33 ++++++
arch/x86/kvm/trace.h | 51 ++++++++
arch/x86/kvm/x86.c | 13 +++
include/uapi/linux/kvm.h | 13 +++
8 files changed, 325 insertions(+), 2 deletions(-)
--
2.24.1
^ permalink raw reply
* [PATCH v8 2/5] x86/hyper-v: Add synthetic debugger definitions
From: Jon Doron @ 2020-03-19 6:38 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319063836.678562-1-arilou@gmail.com>
Hyper-V synthetic debugger has two modes, one that uses MSRs and
the other that use Hypercalls.
Add all the required definitions to both types of synthetic debugger
interface.
Some of the required new CPUIDs and MSRs are not documented in the TLFS
so they are in hyperv.h instead.
The reason they are not documented is because they are subjected to be
removed in future versions of Windows.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
arch/x86/include/asm/hyperv-tlfs.h | 6 ++++++
arch/x86/kvm/hyperv.h | 27 +++++++++++++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index 92abc1e42bfc..671ce2a39d4b 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -131,6 +131,8 @@
#define HV_FEATURE_FREQUENCY_MSRS_AVAILABLE BIT(8)
/* Crash MSR available */
#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE BIT(10)
+/* Support for debug MSRs available */
+#define HV_FEATURE_DEBUG_MSRS_AVAILABLE BIT(11)
/* stimer Direct Mode is available */
#define HV_STIMER_DIRECT_MODE_AVAILABLE BIT(19)
@@ -376,6 +378,9 @@ struct hv_tsc_emulation_status {
#define HVCALL_SEND_IPI_EX 0x0015
#define HVCALL_POST_MESSAGE 0x005c
#define HVCALL_SIGNAL_EVENT 0x005d
+#define HVCALL_POST_DEBUG_DATA 0x0069
+#define HVCALL_RETRIEVE_DEBUG_DATA 0x006a
+#define HVCALL_RESET_DEBUG_SESSION 0x006b
#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
@@ -419,6 +424,7 @@ enum HV_GENERIC_SET_FORMAT {
#define HV_STATUS_INVALID_HYPERCALL_INPUT 3
#define HV_STATUS_INVALID_ALIGNMENT 4
#define HV_STATUS_INVALID_PARAMETER 5
+#define HV_STATUS_OPERATION_DENIED 8
#define HV_STATUS_INSUFFICIENT_MEMORY 11
#define HV_STATUS_INVALID_PORT_ID 17
#define HV_STATUS_INVALID_CONNECTION_ID 18
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index 757cb578101c..5e4780bf6dd7 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -23,6 +23,33 @@
#include <linux/kvm_host.h>
+/*
+ * The #defines related to the synthetic debugger are required by KDNet, but
+ * they are not documented in the Hyper-V TLFS because the synthetic debugger
+ * functionality has been deprecated and is subject to removal in future versions
+ * of Windows.
+ */
+#define HYPERV_CPUID_SYNDBG_VENDOR_AND_MAX_FUNCTIONS 0x40000080
+#define HYPERV_CPUID_SYNDBG_INTERFACE 0x40000081
+#define HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES 0x40000082
+
+/*
+ * Hyper-V synthetic debugger platform capabilities
+ * These are HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX bits.
+ */
+#define HV_X64_SYNDBG_CAP_ALLOW_KERNEL_DEBUGGING BIT(1)
+
+/* Hyper-V Synthetic debug options MSR */
+#define HV_X64_MSR_SYNDBG_CONTROL 0x400000F1
+#define HV_X64_MSR_SYNDBG_STATUS 0x400000F2
+#define HV_X64_MSR_SYNDBG_SEND_BUFFER 0x400000F3
+#define HV_X64_MSR_SYNDBG_RECV_BUFFER 0x400000F4
+#define HV_X64_MSR_SYNDBG_PENDING_BUFFER 0x400000F5
+#define HV_X64_MSR_SYNDBG_OPTIONS 0x400000FF
+
+/* Hyper-V HV_X64_MSR_SYNDBG_OPTIONS bits */
+#define HV_X64_SYNDBG_OPTION_USE_HCALLS BIT(2)
+
static inline struct kvm_vcpu_hv *vcpu_to_hv_vcpu(struct kvm_vcpu *vcpu)
{
return &vcpu->arch.hyperv;
--
2.24.1
^ permalink raw reply related
* [PATCH v8 3/5] x86/kvm/hyper-v: Add support for synthetic debugger capability
From: Jon Doron @ 2020-03-19 6:38 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319063836.678562-1-arilou@gmail.com>
Add support for Hyper-V synthetic debugger (syndbg) interface.
The syndbg interface is using MSRs to emulate a way to send/recv packets
data.
The debug transport dll (kdvm/kdnet) will identify if Hyper-V is enabled
and if it supports the synthetic debugger interface it will attempt to
use it, instead of trying to initialize a network adapter.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
Documentation/virt/kvm/api.rst | 16 ++++
arch/x86/include/asm/kvm_host.h | 14 +++
arch/x86/kvm/hyperv.c | 146 +++++++++++++++++++++++++++++++-
arch/x86/kvm/hyperv.h | 6 ++
arch/x86/kvm/trace.h | 51 +++++++++++
arch/x86/kvm/x86.c | 13 +++
include/uapi/linux/kvm.h | 11 +++
7 files changed, 256 insertions(+), 1 deletion(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 4872c47bbcff..fe992dcf4f93 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -5024,6 +5024,7 @@ EOI was received.
struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
+ #define KVM_EXIT_HYPERV_SYNDBG 3
__u32 type;
__u32 pad1;
union {
@@ -5039,6 +5040,15 @@ EOI was received.
__u64 result;
__u64 params[2];
} hcall;
+ struct {
+ __u32 msr;
+ __u32 pad2;
+ __u64 control;
+ __u64 status;
+ __u64 send_page;
+ __u64 recv_page;
+ __u64 pending_page;
+ } syndbg;
} u;
};
/* KVM_EXIT_HYPERV */
@@ -5055,6 +5065,12 @@ Hyper-V SynIC state change. Notification is used to remap SynIC
event/message pages and to enable/disable SynIC messages/events processing
in userspace.
+ - KVM_EXIT_HYPERV_SYNDBG -- synchronously notify user-space about
+
+Hyper-V Synthetic debugger state change. Notification is used to either update
+the pending_page location or to send a control command (send the buffer located
+in send_page or recv a buffer to recv_page).
+
::
/* KVM_EXIT_ARM_NISV */
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 98959e8cd448..c09fa7401b13 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -854,6 +854,19 @@ struct kvm_apic_map {
struct kvm_lapic *phys_map[];
};
+/* Hyper-V synthetic debugger (SynDbg)*/
+struct kvm_hv_syndbg {
+ struct {
+ u64 control;
+ u64 status;
+ u64 send_page;
+ u64 recv_page;
+ u64 pending_page;
+ } control;
+ u64 options;
+ bool active;
+};
+
/* Hyper-V emulation context */
struct kvm_hv {
struct mutex hv_lock;
@@ -877,6 +890,7 @@ struct kvm_hv {
atomic_t num_mismatched_vp_indexes;
struct hv_partition_assist_pg *hv_pa_pg;
+ struct kvm_hv_syndbg hv_syndbg;
};
enum kvm_irqchip_mode {
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index a86fda7a1d03..b2b50c67badd 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -266,6 +266,115 @@ static int synic_set_msr(struct kvm_vcpu_hv_synic *synic,
return ret;
}
+void kvm_hv_activate_syndbg(struct kvm_vcpu *vcpu)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ syndbg->active = true;
+}
+
+static int kvm_hv_syndbg_complete_userspace(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ struct kvm_hv *hv = &kvm->arch.hyperv;
+
+ if (vcpu->run->hyperv.u.syndbg.msr == HV_X64_MSR_SYNDBG_CONTROL)
+ hv->hv_syndbg.control.status =
+ vcpu->run->hyperv.u.syndbg.status;
+ return 1;
+}
+
+static void syndbg_exit(struct kvm_vcpu *vcpu, u32 msr)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+ struct kvm_vcpu_hv *hv_vcpu = &vcpu->arch.hyperv;
+
+ hv_vcpu->exit.type = KVM_EXIT_HYPERV_SYNDBG;
+ hv_vcpu->exit.u.syndbg.msr = msr;
+ hv_vcpu->exit.u.syndbg.control = syndbg->control.control;
+ hv_vcpu->exit.u.syndbg.send_page = syndbg->control.send_page;
+ hv_vcpu->exit.u.syndbg.recv_page = syndbg->control.recv_page;
+ hv_vcpu->exit.u.syndbg.pending_page = syndbg->control.pending_page;
+ vcpu->arch.complete_userspace_io =
+ kvm_hv_syndbg_complete_userspace;
+
+ kvm_make_request(KVM_REQ_HV_EXIT, vcpu);
+}
+
+static int syndbg_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ if (!syndbg->active)
+ return 1;
+
+ trace_kvm_hv_syndbg_set_msr(vcpu->vcpu_id,
+ vcpu_to_hv_vcpu(vcpu)->vp_index, msr, data);
+ switch (msr) {
+ case HV_X64_MSR_SYNDBG_CONTROL:
+ syndbg->control.control = data;
+ syndbg_exit(vcpu, msr);
+ break;
+ case HV_X64_MSR_SYNDBG_STATUS:
+ syndbg->control.status = data;
+ break;
+ case HV_X64_MSR_SYNDBG_SEND_BUFFER:
+ syndbg->control.send_page = data;
+ break;
+ case HV_X64_MSR_SYNDBG_RECV_BUFFER:
+ syndbg->control.recv_page = data;
+ break;
+ case HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ syndbg->control.pending_page = data;
+ syndbg_exit(vcpu, msr);
+ break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ syndbg->options = data;
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static int syndbg_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
+{
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ if (!syndbg->active)
+ return 1;
+
+ switch (msr) {
+ case HV_X64_MSR_SYNDBG_CONTROL:
+ *pdata = syndbg->control.control;
+ break;
+ case HV_X64_MSR_SYNDBG_STATUS:
+ *pdata = syndbg->control.status;
+ break;
+ case HV_X64_MSR_SYNDBG_SEND_BUFFER:
+ *pdata = syndbg->control.send_page;
+ break;
+ case HV_X64_MSR_SYNDBG_RECV_BUFFER:
+ *pdata = syndbg->control.recv_page;
+ break;
+ case HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ *pdata = syndbg->control.pending_page;
+ break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ *pdata = syndbg->options;
+ break;
+ default:
+ break;
+ }
+
+ trace_kvm_hv_syndbg_get_msr(vcpu->vcpu_id,
+ vcpu_to_hv_vcpu(vcpu)->vp_index, msr,
+ *pdata);
+
+ return 0;
+}
+
static int synic_get_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 *pdata,
bool host)
{
@@ -800,6 +909,8 @@ static bool kvm_hv_msr_partition_wide(u32 msr)
case HV_X64_MSR_REENLIGHTENMENT_CONTROL:
case HV_X64_MSR_TSC_EMULATION_CONTROL:
case HV_X64_MSR_TSC_EMULATION_STATUS:
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
r = true;
break;
}
@@ -1061,6 +1172,9 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,
if (!host)
return 1;
break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ return syndbg_set_msr(vcpu, msr, data);
default:
vcpu_unimpl(vcpu, "Hyper-V unhandled wrmsr: 0x%x data 0x%llx\n",
msr, data);
@@ -1227,6 +1341,9 @@ static int kvm_hv_get_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
case HV_X64_MSR_TSC_EMULATION_STATUS:
data = hv->hv_tsc_emulation_status;
break;
+ case HV_X64_MSR_SYNDBG_OPTIONS:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ return syndbg_get_msr(vcpu, msr, pdata);
default:
vcpu_unimpl(vcpu, "Hyper-V unhandled rdmsr: 0x%x\n", msr);
return 1;
@@ -1797,8 +1914,12 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
{ .function = HYPERV_CPUID_ENLIGHTMENT_INFO },
{ .function = HYPERV_CPUID_IMPLEMENT_LIMITS },
{ .function = HYPERV_CPUID_NESTED_FEATURES },
+ { .function = HYPERV_CPUID_SYNDBG_VENDOR_AND_MAX_FUNCTIONS },
+ { .function = HYPERV_CPUID_SYNDBG_INTERFACE },
+ { .function = HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES },
};
int i, nent = ARRAY_SIZE(cpuid_entries);
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
if (kvm_x86_ops->nested_get_evmcs_version)
evmcs_ver = kvm_x86_ops->nested_get_evmcs_version(vcpu);
@@ -1821,7 +1942,9 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
case HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS:
memcpy(signature, "Linux KVM Hv", 12);
- ent->eax = HYPERV_CPUID_NESTED_FEATURES;
+ ent->eax = syndbg->active ?
+ HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES :
+ HYPERV_CPUID_NESTED_FEATURES;
ent->ebx = signature[0];
ent->ecx = signature[1];
ent->edx = signature[2];
@@ -1856,9 +1979,12 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
ent->ebx |= HV_X64_POST_MESSAGES;
ent->ebx |= HV_X64_SIGNAL_EVENTS;
+ ent->ebx |= HV_X64_DEBUGGING;
ent->edx |= HV_FEATURE_FREQUENCY_MSRS_AVAILABLE;
ent->edx |= HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE;
+ ent->edx |= HV_X64_GUEST_DEBUGGING_AVAILABLE;
+ ent->edx |= HV_FEATURE_DEBUG_MSRS_AVAILABLE;
/*
* Direct Synthetic timers only make sense with in-kernel
@@ -1903,6 +2029,24 @@ int kvm_vcpu_ioctl_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,
break;
+ case HYPERV_CPUID_SYNDBG_VENDOR_AND_MAX_FUNCTIONS:
+ memcpy(signature, "Linux KVM Hv", 12);
+
+ ent->eax = 0;
+ ent->ebx = signature[0];
+ ent->ecx = signature[1];
+ ent->edx = signature[2];
+ break;
+
+ case HYPERV_CPUID_SYNDBG_INTERFACE:
+ memcpy(signature, "VS#1\0\0\0\0\0\0\0\0", 12);
+ ent->eax = signature[0];
+ break;
+
+ case HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES:
+ ent->eax |= HV_X64_SYNDBG_CAP_ALLOW_KERNEL_DEBUGGING;
+ break;
+
default:
break;
}
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index 5e4780bf6dd7..45aaaf08bf15 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -73,6 +73,11 @@ static inline struct kvm_vcpu *synic_to_vcpu(struct kvm_vcpu_hv_synic *synic)
return hv_vcpu_to_vcpu(container_of(synic, struct kvm_vcpu_hv, synic));
}
+static inline struct kvm_hv_syndbg *vcpu_to_hv_syndbg(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->kvm->arch.hyperv.hv_syndbg;
+}
+
int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host);
int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host);
@@ -83,6 +88,7 @@ void kvm_hv_irq_routing_update(struct kvm *kvm);
int kvm_hv_synic_set_irq(struct kvm *kvm, u32 vcpu_id, u32 sint);
void kvm_hv_synic_send_eoi(struct kvm_vcpu *vcpu, int vector);
int kvm_hv_activate_synic(struct kvm_vcpu *vcpu, bool dont_zero_synic_pages);
+void kvm_hv_activate_syndbg(struct kvm_vcpu *vcpu);
void kvm_hv_vcpu_init(struct kvm_vcpu *vcpu);
void kvm_hv_vcpu_postcreate(struct kvm_vcpu *vcpu);
diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index f194dd058470..bf6c3852868d 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -1515,6 +1515,57 @@ TRACE_EVENT(kvm_nested_vmenter_failed,
__print_symbolic(__entry->err, VMX_VMENTER_INSTRUCTION_ERRORS))
);
+/*
+ * Tracepoint for syndbg_set_msr.
+ */
+TRACE_EVENT(kvm_hv_syndbg_set_msr,
+ TP_PROTO(int vcpu_id, u32 vp_index, u32 msr, u64 data),
+ TP_ARGS(vcpu_id, vp_index, msr, data),
+
+ TP_STRUCT__entry(
+ __field(int, vcpu_id)
+ __field(u32, vp_index)
+ __field(u32, msr)
+ __field(u64, data)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->vp_index = vp_index;
+ __entry->msr = msr;
+ __entry->data = data;
+ ),
+
+ TP_printk("vcpu_id %d vp_index %u msr 0x%x data 0x%llx",
+ __entry->vcpu_id, __entry->vp_index, __entry->msr,
+ __entry->data)
+);
+
+/*
+ * Tracepoint for syndbg_get_msr.
+ */
+TRACE_EVENT(kvm_hv_syndbg_get_msr,
+ TP_PROTO(int vcpu_id, u32 vp_index, u32 msr, u64 data),
+ TP_ARGS(vcpu_id, vp_index, msr, data),
+
+ TP_STRUCT__entry(
+ __field(int, vcpu_id)
+ __field(u32, vp_index)
+ __field(u32, msr)
+ __field(u64, data)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu_id = vcpu_id;
+ __entry->vp_index = vp_index;
+ __entry->msr = msr;
+ __entry->data = data;
+ ),
+
+ TP_printk("vcpu_id %d vp_index %u msr 0x%x data 0x%llx",
+ __entry->vcpu_id, __entry->vp_index, __entry->msr,
+ __entry->data)
+);
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 3156e25b0774..7bfa30d7a525 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1214,6 +1214,10 @@ static const u32 emulated_msrs_all[] = {
HV_X64_MSR_VP_ASSIST_PAGE,
HV_X64_MSR_REENLIGHTENMENT_CONTROL, HV_X64_MSR_TSC_EMULATION_CONTROL,
HV_X64_MSR_TSC_EMULATION_STATUS,
+ HV_X64_MSR_SYNDBG_OPTIONS,
+ HV_X64_MSR_SYNDBG_CONTROL, HV_X64_MSR_SYNDBG_STATUS,
+ HV_X64_MSR_SYNDBG_SEND_BUFFER, HV_X64_MSR_SYNDBG_RECV_BUFFER,
+ HV_X64_MSR_SYNDBG_PENDING_BUFFER,
MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME,
MSR_KVM_PV_EOI_EN,
@@ -2906,6 +2910,8 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
*/
break;
case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ case HV_X64_MSR_SYNDBG_OPTIONS:
case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
case HV_X64_MSR_CRASH_CTL:
case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:
@@ -3151,6 +3157,8 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
msr_info->data = 0x20000000;
break;
case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
+ case HV_X64_MSR_SYNDBG_CONTROL ... HV_X64_MSR_SYNDBG_PENDING_BUFFER:
+ case HV_X64_MSR_SYNDBG_OPTIONS:
case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
case HV_X64_MSR_CRASH_CTL:
case HV_X64_MSR_STIMER0_CONFIG ... HV_X64_MSR_STIMER3_COUNT:
@@ -3323,6 +3331,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_HYPERV_TLBFLUSH:
case KVM_CAP_HYPERV_SEND_IPI:
case KVM_CAP_HYPERV_CPUID:
+ case KVM_CAP_HYPERV_SYNDBG:
case KVM_CAP_PCI_SEGMENT:
case KVM_CAP_DEBUGREGS:
case KVM_CAP_X86_ROBUST_SINGLESTEP:
@@ -4178,6 +4187,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
return -EINVAL;
switch (cap->cap) {
+ case KVM_CAP_HYPERV_SYNDBG:
+ kvm_hv_activate_syndbg(vcpu);
+ return 0;
+
case KVM_CAP_HYPERV_SYNIC2:
if (cap->args[0])
return -EINVAL;
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 7ee0ddc4c457..0b573e1155c8 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -188,6 +188,7 @@ struct kvm_s390_cmma_log {
struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_SYNIC 1
#define KVM_EXIT_HYPERV_HCALL 2
+#define KVM_EXIT_HYPERV_SYNDBG 3
__u32 type;
__u32 pad1;
union {
@@ -203,6 +204,15 @@ struct kvm_hyperv_exit {
__u64 result;
__u64 params[2];
} hcall;
+ struct {
+ __u32 msr;
+ __u32 pad2;
+ __u64 control;
+ __u64 status;
+ __u64 send_page;
+ __u64 recv_page;
+ __u64 pending_page;
+ } syndbg;
} u;
};
@@ -1012,6 +1022,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ARM_NISV_TO_USER 177
#define KVM_CAP_ARM_INJECT_EXT_DABT 178
#define KVM_CAP_S390_VCPU_RESETS 179
+#define KVM_CAP_HYPERV_SYNDBG 180
#ifdef KVM_CAP_IRQ_ROUTING
--
2.24.1
^ permalink raw reply related
* [PATCH v8 4/5] x86/kvm/hyper-v: enable hypercalls without hypercall page with syndbg
From: Jon Doron @ 2020-03-19 6:38 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319063836.678562-1-arilou@gmail.com>
Microsoft's kdvm.dll dbgtransport module does not respect the hypercall
page and simply identifies the CPU being used (AMD/Intel) and according
to it simply makes hypercalls with the relevant instruction
(vmmcall/vmcall respectively).
The relevant function in kdvm is KdHvConnectHypervisor which first checks
if the hypercall page has been enabled via HV_X64_MSR_HYPERCALL_ENABLE,
and in case it was not it simply sets the HV_X64_MSR_GUEST_OS_ID to
0x1000101010001 which means:
build_number = 0x0001
service_version = 0x01
minor_version = 0x01
major_version = 0x01
os_id = 0x00 (Undefined)
vendor_id = 1 (Microsoft)
os_type = 0 (A value of 0 indicates a proprietary, closed source OS)
and starts issuing the hypercall without setting the hypercall page.
To resolve this issue simply enable hypercalls also if the guest_os_id
is not 0 and the syndbg feature is enabled.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
arch/x86/kvm/hyperv.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index b2b50c67badd..64f267f10049 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1647,7 +1647,10 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *current_vcpu, u64 ingpa, u64 outgpa,
bool kvm_hv_hypercall_enabled(struct kvm *kvm)
{
- return READ_ONCE(kvm->arch.hyperv.hv_hypercall) & HV_X64_MSR_HYPERCALL_ENABLE;
+ struct kvm_hv *hv = &kvm->arch.hyperv;
+
+ return READ_ONCE(hv->hv_hypercall) & HV_X64_MSR_HYPERCALL_ENABLE ||
+ (hv->hv_syndbg.active && READ_ONCE(hv->hv_guest_os_id) != 0);
}
static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result)
--
2.24.1
^ permalink raw reply related
* [PATCH v8 5/5] x86/kvm/hyper-v: Add support for synthetic debugger via hypercalls
From: Jon Doron @ 2020-03-19 6:38 UTC (permalink / raw)
To: kvm, linux-hyperv; +Cc: vkuznets, Jon Doron
In-Reply-To: <20200319063836.678562-1-arilou@gmail.com>
There is another mode for the synthetic debugger which uses hypercalls
to send/recv network data instead of the MSR interface.
This interface is much slower and less recommended since you might get
a lot of VMExits while KDVM polling for new packets to recv, rather
than simply checking the pending page to see if there is data avialble
and then request.
Signed-off-by: Jon Doron <arilou@gmail.com>
---
arch/x86/kvm/hyperv.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 64f267f10049..0c3d2224dfad 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1829,6 +1829,34 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
}
ret = kvm_hv_send_ipi(vcpu, ingpa, outgpa, true, false);
break;
+ case HVCALL_POST_DEBUG_DATA:
+ case HVCALL_RETRIEVE_DEBUG_DATA:
+ if (unlikely(fast)) {
+ ret = HV_STATUS_INVALID_PARAMETER;
+ break;
+ }
+ fallthrough;
+ case HVCALL_RESET_DEBUG_SESSION: {
+ struct kvm_hv_syndbg *syndbg = vcpu_to_hv_syndbg(vcpu);
+
+ if (!syndbg->active) {
+ ret = HV_STATUS_INVALID_HYPERCALL_CODE;
+ break;
+ }
+
+ if (!(syndbg->options & HV_X64_SYNDBG_OPTION_USE_HCALLS)) {
+ ret = HV_STATUS_OPERATION_DENIED;
+ break;
+ }
+ vcpu->run->exit_reason = KVM_EXIT_HYPERV;
+ vcpu->run->hyperv.type = KVM_EXIT_HYPERV_HCALL;
+ vcpu->run->hyperv.u.hcall.input = param;
+ vcpu->run->hyperv.u.hcall.params[0] = ingpa;
+ vcpu->run->hyperv.u.hcall.params[1] = outgpa;
+ vcpu->arch.complete_userspace_io =
+ kvm_hv_hypercall_complete_userspace;
+ return 0;
+ }
default:
ret = HV_STATUS_INVALID_HYPERCALL_CODE;
break;
--
2.24.1
^ permalink raw reply related
* Re: [PATCH 0/4] x86/Hyper-V: Unload vmbus channel in hv panic callback
From: Tianyu Lan @ 2020-03-19 8:03 UTC (permalink / raw)
To: Vitaly Kuznetsov
Cc: Tianyu Lan, linux-hyperv, linux-kernel, kys, haiyangz, sthemmin,
liuwe, tglx, mingo, bp, hpa, x86, michael.h.kelley
In-Reply-To: <871rpp3ba8.fsf@vitty.brq.redhat.com>
Hi Vitaly:
Thanks for your review.
On 3/18/2020 11:58 PM, Vitaly Kuznetsov wrote:
> ltykernel@gmail.com writes:
>
>> From: Tianyu Lan <Tianyu.Lan@microsoft.com>
>>
>> Customer reported Hyper-V VM still responded network traffic
>> ack packets after kernel panic with kernel parameter "panic=0”.
>> This becauses vmbus driver interrupt handler still works
>> on the panic cpu after kernel panic. Panic cpu falls into
>> infinite loop of panic() with interrupt enabled at that point.
>> Vmbus driver can still handle network traffic.
>>
>> This confuses remote service that the panic system is still
>> alive when it gets ack packets. Unload vmbus channel in hv panic
>> callback and fix it.
>>
>> vmbus_initiate_unload() maybe double called during panic process
>> (e.g, hyperv_panic_event() and hv_crash_handler()). So check
>> and set connection state in vmbus_initiate_unload() to resolve
>> reenter issue.
>>
>> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
>> ---
>> drivers/hv/channel_mgmt.c | 5 +++++
>> drivers/hv/vmbus_drv.c | 17 +++++++++--------
>> 2 files changed, 14 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
>> index 0370364169c4..893493f2b420 100644
>> --- a/drivers/hv/channel_mgmt.c
>> +++ b/drivers/hv/channel_mgmt.c
>> @@ -839,6 +839,9 @@ void vmbus_initiate_unload(bool crash)
>> {
>> struct vmbus_channel_message_header hdr;
>>
>> + if (vmbus_connection.conn_state == DISCONNECTED)
>> + return;
>> +
>
> To make this less racy, can we do something like
>
> if (xchg(&vmbus_connection.conn_state, DISCONNECTED) == DISCONNECTED)
> return;
>
> ?
Agree. Will update in the next version.
>
>> /* Pre-Win2012R2 hosts don't support reconnect */
>> if (vmbus_proto_version < VERSION_WIN8_1)
>> return;
>> @@ -857,6 +860,8 @@ void vmbus_initiate_unload(bool crash)
>> wait_for_completion(&vmbus_connection.unload_event);
>> else
>> vmbus_wait_for_unload();
>> +
>> + vmbus_connection.conn_state = DISCONNECTED;
>> }
>>
>> static void check_ready_for_resume_event(void)
>> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
>> index 029378c27421..b56b9fb9bd90 100644
>> --- a/drivers/hv/vmbus_drv.c
>> +++ b/drivers/hv/vmbus_drv.c
>> @@ -53,9 +53,12 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned long val,
>> {
>> struct pt_regs *regs;
>>
>> - regs = current_pt_regs();
>> + vmbus_initiate_unload(true);
>>
>> - hyperv_report_panic(regs, val);
>> + if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
>
> With Michael's effors to make code in drivers/hv arch agnostic, I think
> we need a better, arch-neutral way.
>
>> + regs = current_pt_regs();
>> + hyperv_report_panic(regs, val);
>> + }
>> return NOTIFY_DONE;
>> }
>>
>> @@ -1391,10 +1394,12 @@ static int vmbus_bus_init(void)
>> }
>>
>> register_die_notifier(&hyperv_die_block);
>> - atomic_notifier_chain_register(&panic_notifier_list,
>> - &hyperv_panic_block);
>> }
>>
>> + /* Vmbus channel is unloaded in panic callback when panic happens.*/
>> + atomic_notifier_chain_register(&panic_notifier_list,
>> + &hyperv_panic_block);
>> +
>> vmbus_request_offers();
>>
>> return 0;
>> @@ -2204,8 +2209,6 @@ static int vmbus_bus_suspend(struct device *dev)
>>
>> vmbus_initiate_unload(false);
>>
>> - vmbus_connection.conn_state = DISCONNECTED;
>> -
>> /* Reset the event for the next resume. */
>> reinit_completion(&vmbus_connection.ready_for_resume_event);
>>
>> @@ -2289,7 +2292,6 @@ static void hv_kexec_handler(void)
>> {
>> hv_stimer_global_cleanup();
>> vmbus_initiate_unload(false);
>> - vmbus_connection.conn_state = DISCONNECTED;
>> /* Make sure conn_state is set as hv_synic_cleanup checks for it */
>> mb();
>> cpuhp_remove_state(hyperv_cpuhp_online);
>> @@ -2306,7 +2308,6 @@ static void hv_crash_handler(struct pt_regs *regs)
>> * doing the cleanup for current CPU only. This should be sufficient
>> * for kdump.
>> */
>> - vmbus_connection.conn_state = DISCONNECTED;
>> cpu = smp_processor_id();
>> hv_stimer_cleanup(cpu);
>> hv_synic_disable_regs(cpu);
>
^ permalink raw reply
* RE: [PATCH 0/4] x86/Hyper-V: Unload vmbus channel in hv panic callback
From: Vitaly Kuznetsov @ 2020-03-19 8:03 UTC (permalink / raw)
To: Michael Kelley, ltykernel@gmail.com
Cc: Tianyu Lan, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Wei Liu, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, hpa@zytor.com, x86@kernel.org
In-Reply-To: <MW2PR2101MB10529A60AF5185BBD7B02E04D7F40@MW2PR2101MB1052.namprd21.prod.outlook.com>
Michael Kelley <mikelley@microsoft.com> writes:
>> > --- a/drivers/hv/vmbus_drv.c
>> > +++ b/drivers/hv/vmbus_drv.c
>> > @@ -53,9 +53,12 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned
>> long val,
>> > {
>> > struct pt_regs *regs;
>> >
>> > - regs = current_pt_regs();
>> > + vmbus_initiate_unload(true);
>> >
>> > - hyperv_report_panic(regs, val);
>> > + if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE) {
>>
>> With Michael's effors to make code in drivers/hv arch agnostic, I think
>> we need a better, arch-neutral way.
>
> Vitaly -- could you elaborate on what part is not arch-neutral? I don't see
> a problem. ms_hyperv and the misc_features field exist for both the x86
> and ARM64 code branches. It turns out the particular bit for
> GUEST_CRASH_MSR_AVAILABLE is different on the two architectures, but
> the compiler will do the right thing.
>
Ah, apologies, missed the fact that we also call it
'HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE' - I have to admit I was confused
by the 'MSR' part. We can probably rename this to something like
HV_FEATURE_GUEST_CRASH_REGS_AVAILABLE - but not as part of the series.
--
Vitaly
^ permalink raw reply
* Re: [PATCH v8 1/5] x86/kvm/hyper-v: Explicitly align hcall param for kvm_hyperv_exit
From: Vitaly Kuznetsov @ 2020-03-19 8:08 UTC (permalink / raw)
To: Jon Doron; +Cc: kvm, linux-hyperv
In-Reply-To: <20200319063836.678562-2-arilou@gmail.com>
Jon Doron <arilou@gmail.com> writes:
> The problem the patch is trying to address is the fact that 'struct
> kvm_hyperv_exit' has different layout on when compiling in 32 and 64 bit
> modes.
>
> In 64-bit mode the default alignment boundary is 64 bits thus
> forcing extra gaps after 'type' and 'msr' but in 32-bit mode the
> boundary is at 32 bits thus no extra gaps.
>
> This is an issue as even when the kernel is 64 bit, the userspace using
> the interface can be both 32 and 64 bit but the same 32 bit userspace has
> to work with 32 bit kernel.
>
> The issue is fixed by forcing the 64 bit layout, this leads to ABI
> change for 32 bit builds and while we are obviously breaking '32 bit
> userspace with 32 bit kernel' case, we're fixing the '32 bit userspace
> with 64 bit kernel' one.
>
> As the interface has no (known) users and 32 bit KVM is rather baroque
> nowadays, this seems like a reasonable decision.
>
> Signed-off-by: Jon Doron <arilou@gmail.com>
> ---
> Documentation/virt/kvm/api.rst | 2 ++
> include/uapi/linux/kvm.h | 2 ++
> 2 files changed, 4 insertions(+)
>
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index ebd383fba939..4872c47bbcff 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -5025,9 +5025,11 @@ EOI was received.
> #define KVM_EXIT_HYPERV_SYNIC 1
> #define KVM_EXIT_HYPERV_HCALL 2
> __u32 type;
> + __u32 pad1;
> union {
> struct {
> __u32 msr;
> + __u32 pad2;
> __u64 control;
> __u64 evt_page;
> __u64 msg_page;
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 4b95f9a31a2f..7ee0ddc4c457 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -189,9 +189,11 @@ struct kvm_hyperv_exit {
> #define KVM_EXIT_HYPERV_SYNIC 1
> #define KVM_EXIT_HYPERV_HCALL 2
> __u32 type;
> + __u32 pad1;
> union {
> struct {
> __u32 msr;
> + __u32 pad2;
> __u64 control;
> __u64 evt_page;
> __u64 msg_page;
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
--
Vitaly
^ permalink raw reply
* Re: [PATCH 2/4] x86/Hyper-V: Free hv_panic_page when fail to register kmsg dump
From: Tianyu Lan @ 2020-03-19 8:12 UTC (permalink / raw)
To: Wei Liu
Cc: kys, haiyangz, sthemmin, liuwe, tglx, mingo, bp, hpa, x86,
michael.h.kelley, Tianyu Lan, linux-hyperv, linux-kernel,
vkuznets
In-Reply-To: <20200317173600.2hqznyabyj4nckjo@debian>
Hi Wei:
Thanks for your review.
On 3/18/2020 1:36 AM, Wei Liu wrote:
> On Tue, Mar 17, 2020 at 06:25:21AM -0700, ltykernel@gmail.com wrote:
>> From: Tianyu Lan <Tianyu.Lan@microsoft.com>
>>
>> If fail to register kmsg dump on Hyper-V platform, hv_panic_page
>> will not be used anywhere. So free and reset it.
>>
>> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
>> ---
>> drivers/hv/vmbus_drv.c | 6 +++++-
>> 1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
>> index b56b9fb9bd90..b043efea092a 100644
>> --- a/drivers/hv/vmbus_drv.c
>> +++ b/drivers/hv/vmbus_drv.c
>> @@ -1385,9 +1385,13 @@ static int vmbus_bus_init(void)
>> hv_panic_page = (void *)hv_alloc_hyperv_zeroed_page();
>> if (hv_panic_page) {
>> ret = kmsg_dump_register(&hv_kmsg_dumper);
>> - if (ret)
>> + if (ret) {
>> pr_err("Hyper-V: kmsg dump register "
>> "error 0x%x\n", ret);
>> + hv_free_hyperv_page(
>> + (unsigned long)hv_panic_page);
>> + hv_panic_page = NULL;
>> + }
>
> While this modification looks correct to me, there is a call to free
> hv_panic_page in the err_alloc path. That makes the error handling a bit
> confusing here.
>
> I think you can just remove that function call in err_alloc path.
OK. Will update in the next version.
>
> Wei.
>
>> } else
>> pr_err("Hyper-V: panic message page memory "
>> "allocation failed");
>> --
>> 2.14.5
>>
^ permalink raw reply
* Re: [PATCH 0/4] x86/Hyper-V: Unload vmbus channel in hv panic callback
From: Tianyu Lan @ 2020-03-19 8:24 UTC (permalink / raw)
To: Wei Liu
Cc: kys, haiyangz, sthemmin, liuwe, tglx, mingo, bp, hpa, x86,
michael.h.kelley, Tianyu Lan, linux-hyperv, linux-kernel,
vkuznets
In-Reply-To: <20200317173553.jerf6gjtaotqjbac@debian>
On 3/18/2020 1:35 AM, Wei Liu wrote:
> On Tue, Mar 17, 2020 at 06:25:20AM -0700, ltykernel@gmail.com wrote:
>> From: Tianyu Lan <Tianyu.Lan@microsoft.com>
>>
>> Customer reported Hyper-V VM still responded network traffic
>> ack packets after kernel panic with kernel parameter "panic=0”.
>> This becauses vmbus driver interrupt handler still works
>> on the panic cpu after kernel panic. Panic cpu falls into
>> infinite loop of panic() with interrupt enabled at that point.
>> Vmbus driver can still handle network traffic.
>>
>> This confuses remote service that the panic system is still
>> alive when it gets ack packets. Unload vmbus channel in hv panic
>> callback and fix it.
>>
>> vmbus_initiate_unload() maybe double called during panic process
>> (e.g, hyperv_panic_event() and hv_crash_handler()). So check
>> and set connection state in vmbus_initiate_unload() to resolve
>> reenter issue.
>>
>> Signed-off-by: Tianyu Lan <Tianyu.Lan@microsoft.com>
>> ---
>> drivers/hv/channel_mgmt.c | 5 +++++
>> drivers/hv/vmbus_drv.c | 17 +++++++++--------
>> 2 files changed, 14 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
>> index 0370364169c4..893493f2b420 100644
>> --- a/drivers/hv/channel_mgmt.c
>> +++ b/drivers/hv/channel_mgmt.c
>> @@ -839,6 +839,9 @@ void vmbus_initiate_unload(bool crash)
>> {
>> struct vmbus_channel_message_header hdr;
>>
>> + if (vmbus_connection.conn_state == DISCONNECTED)
>> + return;
>> +
>> /* Pre-Win2012R2 hosts don't support reconnect */
>> if (vmbus_proto_version < VERSION_WIN8_1)
>> return;
>> @@ -857,6 +860,8 @@ void vmbus_initiate_unload(bool crash)
>> wait_for_completion(&vmbus_connection.unload_event);
>> else
>> vmbus_wait_for_unload();
>> +
>> + vmbus_connection.conn_state = DISCONNECTED;
>
> This is only set at the end of the function. I don't see how this solve
> the re-entrant issue with the check at the beginning. Do I miss anything
> here?
>
For this issue, vmbus_initiate_unload() maybe called on the panic vcpu
twice and so just split check and set conn_state.
> Maybe this function should check and set the state to
> DISCONNECTING/DISCONNECTED at the beginning of this function?
>
Yes, Vitaly also gave suggestion to use "xchg" to check and set
conn_state. Will update in the next version.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox