* [PATCH v5 7/9] Drivers: hv: vmbus: Clean up hv_sock channels by force upon suspend
From: Dexuan Cui @ 2019-09-05 23:01 UTC (permalink / raw)
To: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
sashal@kernel.org, Haiyang Zhang, KY Srinivasan, Michael Kelley,
tglx@linutronix.de
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org,
Dexuan Cui
In-Reply-To: <1567724446-30990-1-git-send-email-decui@microsoft.com>
Fake RESCIND_CHANNEL messages to clean up hv_sock channels by force for
hibernation. There is no better method to clean up the channels since
some of the channels may still be referenced by the userspace apps when
hibernation is triggered: in this case, with this patch, the "rescind"
fields of the channels are set, and the apps will thoroughly destroy
the channels after hibernation.
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
---
drivers/hv/vmbus_drv.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index ce9974b..45b976e 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -24,6 +24,7 @@
#include <linux/sched/task_stack.h>
#include <asm/mshyperv.h>
+#include <linux/delay.h>
#include <linux/notifier.h>
#include <linux/ptrace.h>
#include <linux/screen_info.h>
@@ -1069,6 +1070,41 @@ void vmbus_on_msg_dpc(unsigned long data)
vmbus_signal_eom(msg, message_type);
}
+/*
+ * Fake RESCIND_CHANNEL messages to clean up hv_sock channels by force for
+ * hibernation, because hv_sock connections can not persist across hibernation.
+ */
+static void vmbus_force_channel_rescinded(struct vmbus_channel *channel)
+{
+ struct onmessage_work_context *ctx;
+ struct vmbus_channel_rescind_offer *rescind;
+
+ WARN_ON(!is_hvsock_channel(channel));
+
+ /*
+ * sizeof(*ctx) is small and the allocation should really not fail,
+ * otherwise the state of the hv_sock connections ends up in limbo.
+ */
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL | __GFP_NOFAIL);
+
+ /*
+ * So far, these are not really used by Linux. Just set them to the
+ * reasonable values conforming to the definitions of the fields.
+ */
+ ctx->msg.header.message_type = 1;
+ ctx->msg.header.payload_size = sizeof(*rescind);
+
+ /* These values are actually used by Linux. */
+ rescind = (struct vmbus_channel_rescind_offer *)ctx->msg.u.payload;
+ rescind->header.msgtype = CHANNELMSG_RESCIND_CHANNELOFFER;
+ rescind->child_relid = channel->offermsg.child_relid;
+
+ INIT_WORK(&ctx->work, vmbus_onmessage_work);
+
+ queue_work_on(vmbus_connection.connect_cpu,
+ vmbus_connection.work_queue,
+ &ctx->work);
+}
/*
* Direct callback for channels using other deferred processing
@@ -2091,6 +2127,25 @@ static int vmbus_acpi_add(struct acpi_device *device)
static int vmbus_bus_suspend(struct device *dev)
{
+ struct vmbus_channel *channel;
+
+ while (atomic_read(&vmbus_connection.offer_in_progress) != 0) {
+ /*
+ * We wait here until the completion of any channel
+ * offers that are currently in progress.
+ */
+ msleep(1);
+ }
+
+ mutex_lock(&vmbus_connection.channel_mutex);
+ list_for_each_entry(channel, &vmbus_connection.chn_list, listentry) {
+ if (!is_hvsock_channel(channel))
+ continue;
+
+ vmbus_force_channel_rescinded(channel);
+ }
+ mutex_unlock(&vmbus_connection.channel_mutex);
+
vmbus_initiate_unload(false);
vmbus_connection.conn_state = DISCONNECTED;
--
1.8.3.1
^ permalink raw reply related
* [PATCH v5 5/9] Drivers: hv: vmbus: Ignore the offers when resuming from hibernation
From: Dexuan Cui @ 2019-09-05 23:01 UTC (permalink / raw)
To: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
sashal@kernel.org, Haiyang Zhang, KY Srinivasan, Michael Kelley,
tglx@linutronix.de
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org,
Dexuan Cui
In-Reply-To: <1567724446-30990-1-git-send-email-decui@microsoft.com>
When the VM resumes, the host re-sends the offers. We should not add the
offers to the global vmbus_connection.chn_list again.
This patch assumes the RELIDs of the channels don't change across
hibernation. Actually this is not always true, especially in the case of
NIC SR-IOV the VF vmbus device's RELID sometimes can change. A later patch
will address this issue by mapping the new offers to the old channels and
fixing up the old channels, if necessary.
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
---
drivers/hv/channel_mgmt.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 57 insertions(+), 1 deletion(-)
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index addcef5..44b92fa 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -848,18 +848,74 @@ void vmbus_initiate_unload(bool crash)
}
/*
+ * find_primary_channel_by_offer - Get the channel object given the new offer.
+ * This is only used in the resume path of hibernation.
+ */
+static struct vmbus_channel *
+find_primary_channel_by_offer(const struct vmbus_channel_offer_channel *offer)
+{
+ struct vmbus_channel *channel = NULL, *iter;
+ const guid_t *inst1, *inst2;
+
+ /* Ignore sub-channel offers. */
+ if (offer->offer.sub_channel_index != 0)
+ return NULL;
+
+ mutex_lock(&vmbus_connection.channel_mutex);
+
+ list_for_each_entry(iter, &vmbus_connection.chn_list, listentry) {
+ inst1 = &iter->offermsg.offer.if_instance;
+ inst2 = &offer->offer.if_instance;
+
+ if (guid_equal(inst1, inst2)) {
+ channel = iter;
+ break;
+ }
+ }
+
+ mutex_unlock(&vmbus_connection.channel_mutex);
+
+ return channel;
+}
+
+/*
* vmbus_onoffer - Handler for channel offers from vmbus in parent partition.
*
*/
static void vmbus_onoffer(struct vmbus_channel_message_header *hdr)
{
struct vmbus_channel_offer_channel *offer;
- struct vmbus_channel *newchannel;
+ struct vmbus_channel *oldchannel, *newchannel;
+ size_t offer_sz;
offer = (struct vmbus_channel_offer_channel *)hdr;
trace_vmbus_onoffer(offer);
+ oldchannel = find_primary_channel_by_offer(offer);
+
+ if (oldchannel != NULL) {
+ atomic_dec(&vmbus_connection.offer_in_progress);
+
+ /*
+ * We're resuming from hibernation: we expect the host to send
+ * exactly the same offers that we had before the hibernation.
+ */
+ offer_sz = sizeof(*offer);
+ if (memcmp(offer, &oldchannel->offermsg, offer_sz) == 0)
+ return;
+
+ pr_debug("Mismatched offer from the host (relid=%d)\n",
+ offer->child_relid);
+
+ print_hex_dump_debug("Old vmbus offer: ", DUMP_PREFIX_OFFSET,
+ 16, 4, &oldchannel->offermsg, offer_sz,
+ false);
+ print_hex_dump_debug("New vmbus offer: ", DUMP_PREFIX_OFFSET,
+ 16, 4, offer, offer_sz, false);
+ return;
+ }
+
/* Allocate the channel object and save this offer. */
newchannel = alloc_channel();
if (!newchannel) {
--
1.8.3.1
^ permalink raw reply related
* [PATCH v5 2/9] Drivers: hv: vmbus: Suspend/resume the synic for hibernation
From: Dexuan Cui @ 2019-09-05 23:01 UTC (permalink / raw)
To: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
sashal@kernel.org, Haiyang Zhang, KY Srinivasan, Michael Kelley,
tglx@linutronix.de
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org,
Dexuan Cui
In-Reply-To: <1567724446-30990-1-git-send-email-decui@microsoft.com>
This is needed when we resume the old kernel from the "current" kernel.
Note: when hv_synic_suspend() and hv_synic_resume() run, all the
non-boot CPUs have been offlined, and interrupts are disabled on CPU0.
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
---
drivers/hv/vmbus_drv.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index ebd35fc..2ef375c 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -30,6 +30,7 @@
#include <linux/kdebug.h>
#include <linux/efi.h>
#include <linux/random.h>
+#include <linux/syscore_ops.h>
#include <clocksource/hyperv_timer.h>
#include "hyperv_vmbus.h"
@@ -2086,6 +2087,47 @@ static void hv_crash_handler(struct pt_regs *regs)
hyperv_cleanup();
};
+static int hv_synic_suspend(void)
+{
+ /*
+ * When we reach here, all the non-boot CPUs have been offlined, and
+ * the stimers on them have been unbound in hv_synic_cleanup() ->
+ * hv_stimer_cleanup() -> clockevents_unbind_device().
+ *
+ * hv_synic_suspend() only runs on CPU0 with interrupts disabled. Here
+ * we do not unbind the stimer on CPU0 because: 1) it's unnecessary
+ * because the interrupts remain disabled between syscore_suspend()
+ * and syscore_resume(): see create_image() and resume_target_kernel();
+ * 2) the stimer on CPU0 is automatically disabled later by
+ * syscore_suspend() -> timekeeping_suspend() -> tick_suspend() -> ...
+ * -> clockevents_shutdown() -> ... -> hv_ce_shutdown(); 3) a warning
+ * would be triggered if we call clockevents_unbind_device(), which
+ * may sleep, in an interrupts-disabled context. So, we intentionally
+ * don't call hv_stimer_cleanup(0) here.
+ */
+
+ hv_synic_disable_regs(0);
+
+ return 0;
+}
+
+static void hv_synic_resume(void)
+{
+ hv_synic_enable_regs(0);
+
+ /*
+ * Note: we don't need to call hv_stimer_init(0), because the timer
+ * on CPU0 is not unbound in hv_synic_suspend(), and the timer is
+ * automatically re-enabled in timekeeping_resume().
+ */
+}
+
+/* The callbacks run only on CPU0, with irqs_disabled. */
+static struct syscore_ops hv_synic_syscore_ops = {
+ .suspend = hv_synic_suspend,
+ .resume = hv_synic_resume,
+};
+
static int __init hv_acpi_init(void)
{
int ret, t;
@@ -2116,6 +2158,8 @@ static int __init hv_acpi_init(void)
hv_setup_kexec_handler(hv_kexec_handler);
hv_setup_crash_handler(hv_crash_handler);
+ register_syscore_ops(&hv_synic_syscore_ops);
+
return 0;
cleanup:
@@ -2128,6 +2172,8 @@ static void __exit vmbus_exit(void)
{
int cpu;
+ unregister_syscore_ops(&hv_synic_syscore_ops);
+
hv_remove_kexec_handler();
hv_remove_crash_handler();
vmbus_connection.conn_state = DISCONNECTED;
--
1.8.3.1
^ permalink raw reply related
* [PATCH v5 1/9] Drivers: hv: vmbus: Break out synic enable and disable operations
From: Dexuan Cui @ 2019-09-05 23:01 UTC (permalink / raw)
To: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
sashal@kernel.org, Haiyang Zhang, KY Srinivasan, Michael Kelley,
tglx@linutronix.de
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org,
Dexuan Cui
In-Reply-To: <1567724446-30990-1-git-send-email-decui@microsoft.com>
Break out synic enable and disable operations into separate
hv_synic_disable_regs() and hv_synic_enable_regs() functions for use by a
later patch to support hibernation.
There is no functional change except the unnecessary check
"if (sctrl.enable != 1) return -EFAULT;" which is removed, because when
we're in hv_synic_cleanup(), we're absolutely sure sctrl.enable must be 1.
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
---
drivers/hv/hv.c | 66 ++++++++++++++++++++++++++---------------------
drivers/hv/hyperv_vmbus.h | 2 ++
2 files changed, 39 insertions(+), 29 deletions(-)
diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c
index 6188fb7..fcc5279 100644
--- a/drivers/hv/hv.c
+++ b/drivers/hv/hv.c
@@ -154,7 +154,7 @@ void hv_synic_free(void)
* retrieve the initialized message and event pages. Otherwise, we create and
* initialize the message and event pages.
*/
-int hv_synic_init(unsigned int cpu)
+void hv_synic_enable_regs(unsigned int cpu)
{
struct hv_per_cpu_context *hv_cpu
= per_cpu_ptr(hv_context.cpu_context, cpu);
@@ -196,6 +196,11 @@ int hv_synic_init(unsigned int cpu)
sctrl.enable = 1;
hv_set_synic_state(sctrl.as_uint64);
+}
+
+int hv_synic_init(unsigned int cpu)
+{
+ hv_synic_enable_regs(cpu);
hv_stimer_init(cpu);
@@ -205,20 +210,45 @@ int hv_synic_init(unsigned int cpu)
/*
* hv_synic_cleanup - Cleanup routine for hv_synic_init().
*/
-int hv_synic_cleanup(unsigned int cpu)
+void hv_synic_disable_regs(unsigned int cpu)
{
union hv_synic_sint shared_sint;
union hv_synic_simp simp;
union hv_synic_siefp siefp;
union hv_synic_scontrol sctrl;
+
+ hv_get_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
+
+ shared_sint.masked = 1;
+
+ /* Need to correctly cleanup in the case of SMP!!! */
+ /* Disable the interrupt */
+ hv_set_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
+
+ hv_get_simp(simp.as_uint64);
+ simp.simp_enabled = 0;
+ simp.base_simp_gpa = 0;
+
+ hv_set_simp(simp.as_uint64);
+
+ hv_get_siefp(siefp.as_uint64);
+ siefp.siefp_enabled = 0;
+ siefp.base_siefp_gpa = 0;
+
+ hv_set_siefp(siefp.as_uint64);
+
+ /* Disable the global synic bit */
+ hv_get_synic_state(sctrl.as_uint64);
+ sctrl.enable = 0;
+ hv_set_synic_state(sctrl.as_uint64);
+}
+
+int hv_synic_cleanup(unsigned int cpu)
+{
struct vmbus_channel *channel, *sc;
bool channel_found = false;
unsigned long flags;
- hv_get_synic_state(sctrl.as_uint64);
- if (sctrl.enable != 1)
- return -EFAULT;
-
/*
* Search for channels which are bound to the CPU we're about to
* cleanup. In case we find one and vmbus is still connected we need to
@@ -249,29 +279,7 @@ int hv_synic_cleanup(unsigned int cpu)
hv_stimer_cleanup(cpu);
- hv_get_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
-
- shared_sint.masked = 1;
-
- /* Need to correctly cleanup in the case of SMP!!! */
- /* Disable the interrupt */
- hv_set_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
-
- hv_get_simp(simp.as_uint64);
- simp.simp_enabled = 0;
- simp.base_simp_gpa = 0;
-
- hv_set_simp(simp.as_uint64);
-
- hv_get_siefp(siefp.as_uint64);
- siefp.siefp_enabled = 0;
- siefp.base_siefp_gpa = 0;
-
- hv_set_siefp(siefp.as_uint64);
-
- /* Disable the global synic bit */
- sctrl.enable = 0;
- hv_set_synic_state(sctrl.as_uint64);
+ hv_synic_disable_regs(cpu);
return 0;
}
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 362e70e..26ea161 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -171,8 +171,10 @@ extern int hv_post_message(union hv_connection_id connection_id,
extern void hv_synic_free(void);
+extern void hv_synic_enable_regs(unsigned int cpu);
extern int hv_synic_init(unsigned int cpu);
+extern void hv_synic_disable_regs(unsigned int cpu);
extern int hv_synic_cleanup(unsigned int cpu);
/* Interface */
--
1.8.3.1
^ permalink raw reply related
* [PATCH v5 3/9] Drivers: hv: vmbus: Add a helper function is_sub_channel()
From: Dexuan Cui @ 2019-09-05 23:01 UTC (permalink / raw)
To: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
sashal@kernel.org, Haiyang Zhang, KY Srinivasan, Michael Kelley,
tglx@linutronix.de
Cc: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org,
Dexuan Cui
In-Reply-To: <1567724446-30990-1-git-send-email-decui@microsoft.com>
The existing method of telling if a channel is sub-channel in
vmbus_process_offer() is cumbersome. This new simple helper function
is preferred in future.
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
---
include/linux/hyperv.h | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 6256cc3..2d39248 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -245,7 +245,10 @@ struct vmbus_channel_offer {
} pipe;
} u;
/*
- * The sub_channel_index is defined in win8.
+ * The sub_channel_index is defined in Win8: a value of zero means a
+ * primary channel and a value of non-zero means a sub-channel.
+ *
+ * Before Win8, the field is reserved, meaning it's always zero.
*/
u16 sub_channel_index;
u16 reserved3;
@@ -934,6 +937,11 @@ static inline bool is_hvsock_channel(const struct vmbus_channel *c)
VMBUS_CHANNEL_TLNPI_PROVIDER_OFFER);
}
+static inline bool is_sub_channel(const struct vmbus_channel *c)
+{
+ return c->offermsg.offer.sub_channel_index != 0;
+}
+
static inline void set_channel_affinity_state(struct vmbus_channel *c,
enum hv_numa_policy policy)
{
--
1.8.3.1
^ permalink raw reply related
* RE: [PATCH net-next, 2/2] hv_netvsc: Sync offloading features to VF NIC
From: Haiyang Zhang @ 2019-09-05 23:07 UTC (permalink / raw)
To: Jakub Kicinski
Cc: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, KY Srinivasan, Stephen Hemminger,
olaf@aepfle.de, vkuznets, davem@davemloft.net,
linux-kernel@vger.kernel.org, Mark Bloch
In-Reply-To: <20190830160451.43a61cf9@cakuba.netronome.com>
> -----Original Message-----
> From: Jakub Kicinski <jakub.kicinski@netronome.com>
> Sent: Friday, August 30, 2019 7:05 PM
> To: Haiyang Zhang <haiyangz@microsoft.com>
> Cc: sashal@kernel.org; linux-hyperv@vger.kernel.org;
> netdev@vger.kernel.org; KY Srinivasan <kys@microsoft.com>; Stephen
> Hemminger <sthemmin@microsoft.com>; olaf@aepfle.de; vkuznets
> <vkuznets@redhat.com>; davem@davemloft.net; linux-
> kernel@vger.kernel.org; Mark Bloch <markb@mellanox.com>
> Subject: Re: [PATCH net-next, 2/2] hv_netvsc: Sync offloading features to VF
> NIC
>
> On Fri, 30 Aug 2019 03:45:38 +0000, Haiyang Zhang wrote:
> > VF NIC may go down then come up during host servicing events. This
> > causes the VF NIC offloading feature settings to roll back to the
> > defaults. This patch can synchronize features from synthetic NIC to
> > the VF NIC during ndo_set_features (ethtool -K), and
> > netvsc_register_vf when VF comes back after host events.
> >
> > Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> > Cc: Mark Bloch <markb@mellanox.com>
>
> If we want to make this change in behaviour we should change net_failover
> at the same time.
After checking the net_failover, I found it's for virtio based SRIOV, and very
different from what we did for Hyper-V based SRIOV.
We let the netvsc driver acts as both the synthetic (PV) driver and the transparent
bonding master for the VF NIC. But net_failover acts as a master device on top
of both virtio PV NIC, and VF NIC. And the net_failover doesn't implemented
operations, like ndo_set_features.
So the code change for our netvsc driver cannot be applied to net_failover driver.
I will re-submit my two patches (fixing the extra tab in the 1st one as you pointed
out). Thanks!
- Haiyang
^ permalink raw reply
* RE: [Patch v3] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Michael Kelley @ 2019-09-05 23:18 UTC (permalink / raw)
To: longli@linuxonhyperv.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, James E.J. Bottomley,
Martin K. Petersen, linux-hyperv@vger.kernel.org,
linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: Long Li
In-Reply-To: <1567724073-30192-1-git-send-email-longli@linuxonhyperv.com>
From: Long Li <longli@microsoft.com> Sent: Thursday, September 5, 2019 3:55 PM
>
> storvsc doesn't use a dedicated hardware queue for a given CPU queue. When
> issuing I/O, it selects returning CPU (hardware queue) dynamically based on
> vmbus channel usage across all channels.
>
> This patch advertises num_present_cpus() as number of hardware queues. This
> will have upper layer setup 1:1 mapping between hardware queue and CPU queue
> and avoid unnecessary locking when issuing I/O.
>
> Changes:
> v2: rely on default upper layer function to map queues. (suggested by Ming Lei
> <tom.leiming@gmail.com>)
> v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v doesn't
> support hot-add CPUs. (suggested by Michael Kelley <mikelley@microsoft.com>)
I've mostly seen the "Changes:" section placed below the "---" so that it doesn't
clutter up the commit log. But maybe there's not a strong requirement one
way or the other as I didn't find anything called out in the "Documentation/process"
directory.
Michael
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
> drivers/scsi/storvsc_drv.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
> index b89269120a2d..cf987712041a 100644
> --- a/drivers/scsi/storvsc_drv.c
> +++ b/drivers/scsi/storvsc_drv.c
> @@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,
> /*
> * Set the number of HW queues we are supporting.
> */
> - if (stor_device->num_sc != 0)
> - host->nr_hw_queues = stor_device->num_sc + 1;
> + host->nr_hw_queues = num_present_cpus();
>
> /*
> * Set the error handler work queue.
> --
> 2.17.1
^ permalink raw reply
* [PATCH net-next,v2, 0/2] Enable sg as tunable, sync offload settings to VF NIC
From: Haiyang Zhang @ 2019-09-05 23:22 UTC (permalink / raw)
To: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
This patch set fixes an issue in SG tuning, and sync
offload settings from synthetic NIC to VF NIC.
Haiyang Zhang (2):
hv_netvsc: hv_netvsc: Allow scatter-gather feature to be tunable
hv_netvsc: Sync offloading features to VF NIC
drivers/net/hyperv/hyperv_net.h | 2 +-
drivers/net/hyperv/netvsc_drv.c | 26 ++++++++++++++++++++++----
drivers/net/hyperv/rndis_filter.c | 1 +
3 files changed, 24 insertions(+), 5 deletions(-)
--
1.8.3.1
^ permalink raw reply
* [PATCH net-next,v2, 1/2] hv_netvsc: Allow scatter-gather feature to be tunable
From: Haiyang Zhang @ 2019-09-05 23:23 UTC (permalink / raw)
To: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org
In-Reply-To: <1567725722-33552-1-git-send-email-haiyangz@microsoft.com>
In a previous patch, the NETIF_F_SG was missing after the code changes.
That caused the SG feature to be "fixed". This patch includes it into
hw_features, so it is tunable again.
Fixes: 23312a3be999 ("netvsc: negotiate checksum and segmentation parameters")
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
---
drivers/net/hyperv/hyperv_net.h | 2 +-
drivers/net/hyperv/netvsc_drv.c | 4 ++--
drivers/net/hyperv/rndis_filter.c | 1 +
3 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
index ecc9af0..670ef68 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -822,7 +822,7 @@ struct nvsp_message {
#define NETVSC_SUPPORTED_HW_FEATURES (NETIF_F_RXCSUM | NETIF_F_IP_CSUM | \
NETIF_F_TSO | NETIF_F_IPV6_CSUM | \
- NETIF_F_TSO6 | NETIF_F_LRO)
+ NETIF_F_TSO6 | NETIF_F_LRO | NETIF_F_SG)
#define VRSS_SEND_TAB_SIZE 16 /* must be power of 2 */
#define VRSS_CHANNEL_MAX 64
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 0a6cd2f..1f1192e 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2313,8 +2313,8 @@ static int netvsc_probe(struct hv_device *dev,
/* hw_features computed in rndis_netdev_set_hwcaps() */
net->features = net->hw_features |
- NETIF_F_HIGHDMA | NETIF_F_SG |
- NETIF_F_HW_VLAN_CTAG_TX | NETIF_F_HW_VLAN_CTAG_RX;
+ NETIF_F_HIGHDMA | NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_CTAG_RX;
net->vlan_features = net->features;
netdev_lockdep_set_classes(net);
diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
index 317dbe9..abaf815 100644
--- a/drivers/net/hyperv/rndis_filter.c
+++ b/drivers/net/hyperv/rndis_filter.c
@@ -1207,6 +1207,7 @@ static int rndis_netdev_set_hwcaps(struct rndis_device *rndis_device,
/* Compute tx offload settings based on hw capabilities */
net->hw_features |= NETIF_F_RXCSUM;
+ net->hw_features |= NETIF_F_SG;
if ((hwcaps.csum.ip4_txcsum & NDIS_TXCSUM_ALL_TCP4) == NDIS_TXCSUM_ALL_TCP4) {
/* Can checksum TCP */
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next,v2, 2/2] hv_netvsc: Sync offloading features to VF NIC
From: Haiyang Zhang @ 2019-09-05 23:23 UTC (permalink / raw)
To: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org
Cc: Haiyang Zhang, KY Srinivasan, Stephen Hemminger, olaf@aepfle.de,
vkuznets, davem@davemloft.net, linux-kernel@vger.kernel.org,
Mark Bloch
In-Reply-To: <1567725722-33552-1-git-send-email-haiyangz@microsoft.com>
VF NIC may go down then come up during host servicing events. This
causes the VF NIC offloading feature settings to roll back to the
defaults. This patch can synchronize features from synthetic NIC to
the VF NIC during ndo_set_features (ethtool -K),
and netvsc_register_vf when VF comes back after host events.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Mark Bloch <markb@mellanox.com>
---
drivers/net/hyperv/netvsc_drv.c | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 1f1192e..39dddcd 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -1785,13 +1785,15 @@ static int netvsc_set_features(struct net_device *ndev,
netdev_features_t change = features ^ ndev->features;
struct net_device_context *ndevctx = netdev_priv(ndev);
struct netvsc_device *nvdev = rtnl_dereference(ndevctx->nvdev);
+ struct net_device *vf_netdev = rtnl_dereference(ndevctx->vf_netdev);
struct ndis_offload_params offloads;
+ int ret = 0;
if (!nvdev || nvdev->destroy)
return -ENODEV;
if (!(change & NETIF_F_LRO))
- return 0;
+ goto syncvf;
memset(&offloads, 0, sizeof(struct ndis_offload_params));
@@ -1803,7 +1805,19 @@ static int netvsc_set_features(struct net_device *ndev,
offloads.rsc_ip_v6 = NDIS_OFFLOAD_PARAMETERS_RSC_DISABLED;
}
- return rndis_filter_set_offload_params(ndev, nvdev, &offloads);
+ ret = rndis_filter_set_offload_params(ndev, nvdev, &offloads);
+
+ if (ret)
+ features ^= NETIF_F_LRO;
+
+syncvf:
+ if (!vf_netdev)
+ return ret;
+
+ vf_netdev->wanted_features = features;
+ netdev_update_features(vf_netdev);
+
+ return ret;
}
static u32 netvsc_get_msglevel(struct net_device *ndev)
@@ -2181,6 +2195,10 @@ static int netvsc_register_vf(struct net_device *vf_netdev)
dev_hold(vf_netdev);
rcu_assign_pointer(net_device_ctx->vf_netdev, vf_netdev);
+
+ vf_netdev->wanted_features = ndev->features;
+ netdev_update_features(vf_netdev);
+
return NOTIFY_OK;
}
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH v4 1/2] drivers: hv: vmbus: Introduce latency testing
From: Branden Bonaby @ 2019-09-06 0:13 UTC (permalink / raw)
To: Michael Kelley
Cc: KY Srinivasan, Haiyang Zhang, Stephen Hemminger,
sashal@kernel.org, linux-hyperv@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <DM5PR21MB0137F04798C30379AD6CE553D7BE0@DM5PR21MB0137.namprd21.prod.outlook.com>
On Mon, Sep 02, 2019 at 06:31:04PM +0000, Michael Kelley wrote:
> From: Branden Bonaby <brandonbonaby94@gmail.com> Sent: Wednesday, August 28, 2019 9:24 PM
> >
> > Introduce user specified latency in the packet reception path
> > By exposing the test parameters as part of the debugfs channel
> > attributes. We will control the testing state via these attributes.
> >
> > Signed-off-by: Branden Bonaby <brandonbonaby94@gmail.com>
> > ---
> > diff --git a/Documentation/ABI/testing/debugfs-hyperv
> > b/Documentation/ABI/testing/debugfs-hyperv
> > new file mode 100644
> > index 000000000000..0903e6427a2d
> > --- /dev/null
> > +++ b/Documentation/ABI/testing/debugfs-hyperv
> > @@ -0,0 +1,23 @@
> > +What: /sys/kernel/debug/hyperv/<UUID>/fuzz_test_state
> > +Date: August 2019
> > +KernelVersion: 5.3
>
> Given the way the timing works for adding code to the Linux kernel,
> the earliest your new code will be included is 5.4. The merge window
> for 5.4 will open in two weeks, so your code would need to be accepted
> by then to be included in 5.4. Otherwise, it won't make it until the next
> merge window, which would be for 5.5. I would suggest changing this
> (and the others below) to 5.4 for now, but you might have to change to
> 5.5 if additional revisions are needed.
>
I see, I'll keep this in mind when submitting the further revisions
thanks
> > diff --git a/drivers/hv/Makefile b/drivers/hv/Makefile
> > index a1eec7177c2d..d754bd86ca47 100644
> > --- a/drivers/hv/Makefile
> > +++ b/drivers/hv/Makefile
> > @@ -2,6 +2,7 @@
> > obj-$(CONFIG_HYPERV) += hv_vmbus.o
> > obj-$(CONFIG_HYPERV_UTILS) += hv_utils.o
> > obj-$(CONFIG_HYPERV_BALLOON) += hv_balloon.o
> > +obj-$(CONFIG_HYPERV_TESTING) += hv_debugfs.o
>
> There's an additional complexity here that I failed to describe to
> you in my previous comments. :-( The above line makes the
> hv_debugfs code part of the main Linux OS image -- i.e., the
> vmlinuz file that gets installed into /boot, such that if hv_debugfs
> is built, it is always loaded into the memory of the running system.
> That's OK, but we should consider the alternative, which is to
> make the hv_debugfs code part of the hv_vmbus module that
> is specified a bit further down in the same Makefile. A module
> is installed into /lib/modules/<kernel version>/kernel, and
> is only loaded into memory on the running system if something
> actually references code in the module. This approach helps avoid
> loading code into memory that isn't actually used.
>
> Whether code is built as a separate module or is just built into
> the main vmlinuz file is also controlled by the .config file setting.
> For example, CONFIG_HYPERV=m says to treat hv_vmbus as a
> separate module, while CONFIG_HYPERV=y says to build the
> hv_vmbus module directly into the vmlinuz file even though the
> Makefile constructs it as a module.
>
> Making hv_debugfs part of the hv_vmbus module is generally better
> than just building it into the main vmlinuz because it's best to include
> only things that must always be present in the main vmlinuz file.
> hv_debugfs doesn't have any reason it needs to always be present.
> By comparison, hv_balloon is included in the main vmlinuz, which
> might be due to it dealing with memory mgmt and needing to be
> in the vmlinuz image, but I'm not sure.
>
> You can make hv_debugfs part of the hv_vmbus module with this line
> placed after the line specifying hv_vmbus_y, instead of the line you
> currently have:
>
> hv_vmbus-$(CONFIG_HYPERV_TESTING) += hv_debugfs.o
>
Ah, I see now. That makes sense I'll go ahead and make the adjustments
thanks
> > +
> > +static int hv_debugfs_delay_set(void *data, u64 val)
> > +{
> > + int ret = 0;
> > +
> > + if (val >= 1 && val <= 1000)
> > + *(u32 *)data = val;
> > + else if (val == 0)
> > + *(u32 *)data = 0;
>
> I think this "else if" clause can be folded into the first
> "if" statement by just checking "val >= 0".
>
good call, I'll fold it into one statement
>
> > +/* Remove dentry associated with released hv device */
> > +void hv_debug_rm_dev_dir(struct hv_device *dev)
> > +{
> > + if (!IS_ERR(hv_debug_root))
> > + debugfs_remove_recursive(dev->debug_dir);
> > +}
>
> This function is the first of five functions that are called by
> code outside of hv_debugfs.c. You probably saw the separate
> email from the kbuild test robot showing a build error on each
> of these five functions. Here's why.
>
> When CONFIG_HYPERV=m is set, and hv_vmbus is built as a
> module, there are references to the five functions from the
> module to your hv_debugfs that is built as core code in
> vmlinuz. By default, Linux does not allow such core code to
> be called by modules. Core code must add a statement like:
>
> EXPORT_SYMBOL_GPL(hv_debug_rm_dev_dir);
>
> to allow the module to call it. This gives the code writer
> a very minimal amount of scope control. However, if you build
> hv_debugfs as part of the module hv_vmbus, and the only
> references to the five functions are within the module hv_vmbus,
> then the EXPORT statements aren't needed because all
> references are internal to the hv_vmbus module. But if
> you envision a function like hv_debug_delay_test() being
> called by other Hyper-V drivers that are outside the
> hv_vmbus module, then you will need the EXPORT statement
> at least for that function.
>
> You probably have your .config file setup with
> CONFIG_HYPERV=y. In that case, the hv_vmbus module is
> built-in to the kernel, so you didn't get the errors reported
> by the kbuild test robot. When testing new code, it's a
> good practice to build with the HYPERV settings set to
> 'm' to make sure that any issues with module references
> get flushed out and fixed.
>
> Michael
Oh I see, I'll test it out as a module so I can fix this. As for
exporting hv_debug_delay_test() It might come in handy later but
I think for now I should focus on just the ones in the hv_vmbus
module for now.
thanks
^ permalink raw reply
* Re: [PATCH v3 1/3] drivers: hv: vmbus: Introduce latency testing
From: Branden Bonaby @ 2019-09-06 0:20 UTC (permalink / raw)
To: Stephen Hemminger
Cc: KY Srinivasan, Haiyang Zhang, Stephen Hemminger, sashal,
linux-hyperv
In-Reply-To: <20190829145715.41df52e0@hermes.lan>
On Thu, Aug 29, 2019 at 02:57:15PM -0700, Stephen Hemminger wrote:
> On Tue, 20 Aug 2019 16:39:02 -0700
> "Branden Bonaby" <brandonbonaby94@gmail.com> wrote:
>
> > diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig
> > index 9a59957922d4..d97437ba0626 100644
> > --- a/drivers/hv/Kconfig
> > +++ b/drivers/hv/Kconfig
> > @@ -29,4 +29,11 @@ config HYPERV_BALLOON
> > help
> > Select this option to enable Hyper-V Balloon driver.
> >
> > +config HYPERV_TESTING
> > + bool "Hyper-V testing"
> > + default n
> > + depends on HYPERV && DEBUG_FS
> > + help
> > + Select this option to enable Hyper-V vmbus testing.
> > +
> > endmenu
>
> Maybe this should go under the Kernel hacking
> section in lib/Kconfig.debug?
>
These lines are the same in the v4 patch I sent a bit after this
so in my v5 version I'll update this.
Thanks
^ permalink raw reply
* RE: [Patch v3] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Long Li @ 2019-09-06 3:34 UTC (permalink / raw)
To: Michael Kelley, longli@linuxonhyperv.com, KY Srinivasan,
Haiyang Zhang, Stephen Hemminger, Sasha Levin,
James E.J. Bottomley, Martin K. Petersen,
linux-hyperv@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <DM5PR21MB013716CEE8942CB769E236F2D7BB0@DM5PR21MB0137.namprd21.prod.outlook.com>
>Subject: RE: [Patch v3] storvsc: setup 1:1 mapping between hardware queue
>and CPU queue
>
>From: Long Li <longli@microsoft.com> Sent: Thursday, September 5, 2019 3:55
>PM
>>
>> storvsc doesn't use a dedicated hardware queue for a given CPU queue.
>> When issuing I/O, it selects returning CPU (hardware queue)
>> dynamically based on vmbus channel usage across all channels.
>>
>> This patch advertises num_present_cpus() as number of hardware queues.
>> This will have upper layer setup 1:1 mapping between hardware queue
>> and CPU queue and avoid unnecessary locking when issuing I/O.
>>
>> Changes:
>> v2: rely on default upper layer function to map queues. (suggested by
>> Ming Lei
>> <tom.leiming@gmail.com>)
>> v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v
>> doesn't support hot-add CPUs. (suggested by Michael Kelley
>> <mikelley@microsoft.com>)
>
>I've mostly seen the "Changes:" section placed below the "---" so that it
>doesn't clutter up the commit log. But maybe there's not a strong
>requirement one way or the other as I didn't find anything called out in the
>"Documentation/process"
>directory.
Should I resubmit the patch (but keep it v3)?
>
>Michael
>
>>
>> Signed-off-by: Long Li <longli@microsoft.com>
>> ---
>> drivers/scsi/storvsc_drv.c | 3 +--
>> 1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
>> index b89269120a2d..cf987712041a 100644
>> --- a/drivers/scsi/storvsc_drv.c
>> +++ b/drivers/scsi/storvsc_drv.c
>> @@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,
>> /*
>> * Set the number of HW queues we are supporting.
>> */
>> - if (stor_device->num_sc != 0)
>> - host->nr_hw_queues = stor_device->num_sc + 1;
>> + host->nr_hw_queues = num_present_cpus();
>>
>> /*
>> * Set the error handler work queue.
>> --
>> 2.17.1
^ permalink raw reply
* RE: [Patch v3] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Michael Kelley @ 2019-09-06 15:45 UTC (permalink / raw)
To: Long Li, longli@linuxonhyperv.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, James E.J. Bottomley,
Martin K. Petersen, linux-hyperv@vger.kernel.org,
linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org
In-Reply-To: <CY4PR21MB074110983075A3BC91D73AE4CEBA0@CY4PR21MB0741.namprd21.prod.outlook.com>
From: Long Li <longli@microsoft.com> Sent: Thursday, September 5, 2019 8:35 PM
> >>
> >> Changes:
> >> v2: rely on default upper layer function to map queues. (suggested by
> >> Ming Lei
> >> <tom.leiming@gmail.com>)
> >> v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v
> >> doesn't support hot-add CPUs. (suggested by Michael Kelley
> >> <mikelley@microsoft.com>)
> >
> >I've mostly seen the "Changes:" section placed below the "---" so that it
> >doesn't clutter up the commit log. But maybe there's not a strong
> >requirement one way or the other as I didn't find anything called out in the
> >"Documentation/process"
> >directory.
>
> Should I resubmit the patch (but keep it v3)?
>
I would say do a quick resubmit as v4 so there's no confusion.
Michael
^ permalink raw reply
* [Patch v4] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: longli @ 2019-09-06 17:24 UTC (permalink / raw)
To: K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Sasha Levin,
James E.J. Bottomley, Martin K. Petersen, linux-hyperv,
linux-scsi, linux-kernel
Cc: Long Li
From: Long Li <longli@microsoft.com>
storvsc doesn't use a dedicated hardware queue for a given CPU queue. When
issuing I/O, it selects returning CPU (hardware queue) dynamically based on
vmbus channel usage across all channels.
This patch advertises num_present_cpus() as number of hardware queues. This
will have upper layer setup 1:1 mapping between hardware queue and CPU queue
and avoid unnecessary locking when issuing I/O.
Signed-off-by: Long Li <longli@microsoft.com>
---
Changes:
v2: rely on default upper layer function to map queues. (suggested by Ming Lei
<tom.leiming@gmail.com>)
v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v doesn't
support hot-add CPUs. (suggested by Michael Kelley <mikelley@microsoft.com>)
v4: move change logs to after Signed-of-by
drivers/scsi/storvsc_drv.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
index b89269120a2d..cf987712041a 100644
--- a/drivers/scsi/storvsc_drv.c
+++ b/drivers/scsi/storvsc_drv.c
@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,
/*
* Set the number of HW queues we are supporting.
*/
- if (stor_device->num_sc != 0)
- host->nr_hw_queues = stor_device->num_sc + 1;
+ host->nr_hw_queues = num_present_cpus();
/*
* Set the error handler work queue.
--
2.17.1
^ permalink raw reply related
* RE: [Patch v4] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Michael Kelley @ 2019-09-06 19:14 UTC (permalink / raw)
To: longli@linuxonhyperv.com, KY Srinivasan, Haiyang Zhang,
Stephen Hemminger, Sasha Levin, James E.J. Bottomley,
Martin K. Petersen, linux-hyperv@vger.kernel.org,
linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: Long Li
In-Reply-To: <1567790660-48142-1-git-send-email-longli@linuxonhyperv.com>
From: Long Li <longli@microsoft.com> Sent: Friday, September 6, 2019 10:24 AM
>
> storvsc doesn't use a dedicated hardware queue for a given CPU queue. When
> issuing I/O, it selects returning CPU (hardware queue) dynamically based on
> vmbus channel usage across all channels.
>
> This patch advertises num_present_cpus() as number of hardware queues. This
> will have upper layer setup 1:1 mapping between hardware queue and CPU queue
> and avoid unnecessary locking when issuing I/O.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>
> Changes:
> v2: rely on default upper layer function to map queues. (suggested by Ming Lei
> <tom.leiming@gmail.com>)
> v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v doesn't
> support hot-add CPUs. (suggested by Michael Kelley <mikelley@microsoft.com>)
> v4: move change logs to after Signed-of-by
>
> drivers/scsi/storvsc_drv.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
> index b89269120a2d..cf987712041a 100644
> --- a/drivers/scsi/storvsc_drv.c
> +++ b/drivers/scsi/storvsc_drv.c
> @@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,
> /*
> * Set the number of HW queues we are supporting.
> */
> - if (stor_device->num_sc != 0)
> - host->nr_hw_queues = stor_device->num_sc + 1;
> + host->nr_hw_queues = num_present_cpus();
>
> /*
> * Set the error handler work queue.
> --
> 2.17.1
Reviewed-by: Michael Kelley <mikelley@microsoft.com>
^ permalink raw reply
* Re: [PATCH v5 0/9] Enhance the hv_vmbus driver to support hibernation
From: Sasha Levin @ 2019-09-06 20:03 UTC (permalink / raw)
To: Dexuan Cui
Cc: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
Haiyang Zhang, KY Srinivasan, Michael Kelley, tglx@linutronix.de,
gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org
In-Reply-To: <1567724446-30990-1-git-send-email-decui@microsoft.com>
On Thu, Sep 05, 2019 at 11:01:14PM +0000, Dexuan Cui wrote:
>This patchset (consisting of 9 patches) was part of the v4 patchset (consisting
>of 12 patches):
> https://lkml.org/lkml/2019/9/2/894
>
>The other 3 patches in v4 are posted in another patchset, which will go
>through the tip.git tree.
>
>All the 9 patches here are now rebased to the hyperv tree's hyperv-next branch:
>https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/log/?h=hyperv-next
>, and all the 9 patches have Michael Kelley's Signed-off-by's.
>
>Please review.
Given that these two series depend on each other, I'd much prefer for
them to go through one tree.
But, I may be wrong, and I'm going to see if a scenario such as this
make sense. I've queued this one to the hyperv-next, but I'll wait for
the x86 folks to send their pull request to Linus first before I do it
for these patches.
Usually cases such as these are the exception, but for Hyper-V it seems
to be the norm, so I'm curious to see how this will unfold.
--
Thanks,
Sasha
^ permalink raw reply
* Re: [Patch v4] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Sasha Levin @ 2019-09-06 20:08 UTC (permalink / raw)
To: longli
Cc: K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
James E.J. Bottomley, Martin K. Petersen, linux-hyperv,
linux-scsi, linux-kernel, Long Li
In-Reply-To: <1567790660-48142-1-git-send-email-longli@linuxonhyperv.com>
On Fri, Sep 06, 2019 at 10:24:20AM -0700, longli@linuxonhyperv.com wrote:
>From: Long Li <longli@microsoft.com>
>
>storvsc doesn't use a dedicated hardware queue for a given CPU queue. When
>issuing I/O, it selects returning CPU (hardware queue) dynamically based on
>vmbus channel usage across all channels.
>
>This patch advertises num_present_cpus() as number of hardware queues. This
>will have upper layer setup 1:1 mapping between hardware queue and CPU queue
>and avoid unnecessary locking when issuing I/O.
>
>Signed-off-by: Long Li <longli@microsoft.com>
>---
>
>Changes:
>v2: rely on default upper layer function to map queues. (suggested by Ming Lei
><tom.leiming@gmail.com>)
>v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v doesn't
>support hot-add CPUs. (suggested by Michael Kelley <mikelley@microsoft.com>)
>v4: move change logs to after Signed-of-by
>
> drivers/scsi/storvsc_drv.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
>diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
>index b89269120a2d..cf987712041a 100644
>--- a/drivers/scsi/storvsc_drv.c
>+++ b/drivers/scsi/storvsc_drv.c
>@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,
> /*
> * Set the number of HW queues we are supporting.
> */
>- if (stor_device->num_sc != 0)
>- host->nr_hw_queues = stor_device->num_sc + 1;
>+ host->nr_hw_queues = num_present_cpus();
Just looking at the change notes for v3: why isn't this
num_active_cpus() then? One can still isolate CPUs on hyper-v, no?
--
Thanks,
Sasha
^ permalink raw reply
* RE: [Patch v4] storvsc: setup 1:1 mapping between hardware queue and CPU queue
From: Long Li @ 2019-09-06 21:37 UTC (permalink / raw)
To: Sasha Levin, longli@linuxonhyperv.com
Cc: KY Srinivasan, Haiyang Zhang, Stephen Hemminger,
James E.J. Bottomley, Martin K. Petersen,
linux-hyperv@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <20190906200820.GE1528@sasha-vm>
>Subject: Re: [Patch v4] storvsc: setup 1:1 mapping between hardware queue
>and CPU queue
>
>On Fri, Sep 06, 2019 at 10:24:20AM -0700, longli@linuxonhyperv.com wrote:
>>From: Long Li <longli@microsoft.com>
>>
>>storvsc doesn't use a dedicated hardware queue for a given CPU queue.
>>When issuing I/O, it selects returning CPU (hardware queue) dynamically
>>based on vmbus channel usage across all channels.
>>
>>This patch advertises num_present_cpus() as number of hardware queues.
>>This will have upper layer setup 1:1 mapping between hardware queue and
>>CPU queue and avoid unnecessary locking when issuing I/O.
>>
>>Signed-off-by: Long Li <longli@microsoft.com>
>>---
>>
>>Changes:
>>v2: rely on default upper layer function to map queues. (suggested by
>>Ming Lei
>><tom.leiming@gmail.com>)
>>v3: use num_present_cpus() instead of num_online_cpus(). Hyper-v
>>doesn't support hot-add CPUs. (suggested by Michael Kelley
>><mikelley@microsoft.com>)
>>v4: move change logs to after Signed-of-by
>>
>> drivers/scsi/storvsc_drv.c | 3 +--
>> 1 file changed, 1 insertion(+), 2 deletions(-)
>>
>>diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
>>index b89269120a2d..cf987712041a 100644
>>--- a/drivers/scsi/storvsc_drv.c
>>+++ b/drivers/scsi/storvsc_drv.c
>>@@ -1836,8 +1836,7 @@ static int storvsc_probe(struct hv_device *device,
>> /*
>> * Set the number of HW queues we are supporting.
>> */
>>- if (stor_device->num_sc != 0)
>>- host->nr_hw_queues = stor_device->num_sc + 1;
>>+ host->nr_hw_queues = num_present_cpus();
>
>Just looking at the change notes for v3: why isn't this
>num_active_cpus() then? One can still isolate CPUs on hyper-v, no?
The isolated CPU can be made online at run time. For example, even maxcpus=x is put on the boot line, individual CPUs can still be made online/offline.
>
>--
>Thanks,
>Sasha
^ permalink raw reply
* RE: [PATCH v5 0/9] Enhance the hv_vmbus driver to support hibernation
From: Dexuan Cui @ 2019-09-06 22:45 UTC (permalink / raw)
To: Sasha Levin
Cc: linux-hyperv@vger.kernel.org, Stephen Hemminger, Sasha Levin,
Haiyang Zhang, KY Srinivasan, Michael Kelley, tglx@linutronix.de,
gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org
In-Reply-To: <20190906200325.GD1528@sasha-vm>
> From: Sasha Levin <sashal@kernel.org>
> Sent: Friday, September 6, 2019 1:03 PM
> On Thu, Sep 05, 2019 at 11:01:14PM +0000, Dexuan Cui wrote:
> >This patchset (consisting of 9 patches) was part of the v4 patchset (consisting
> >of 12 patches):
> >
> >The other 3 patches in v4 are posted in another patchset, which will go
> >through the tip.git tree.
> >
> >All the 9 patches here are now rebased to the hyperv tree's hyperv-next
> branch, and all the 9 patches have Michael Kelley's Signed-off-by's.
> >
> >Please review.
>
> Given that these two series depend on each other, I'd much prefer for
> them to go through one tree.
Hi Sasha,
Yeah, that would be ideal. The problem here is: the other patchset conflicts
with the existing patches in the tip.git tree's timers/core branch, so IMO
the 3 patches have to go through the tip tree:
[PATCH v5 1/3] x86/hyper-v: Suspend/resume the hypercall page for hibernation
[PATCH v5 2/3] x86/hyper-v: Implement hv_is_hibernation_supported()
[PATCH v5 3/3] clocksource/drivers: Suspend/resume Hyper-V clocksource for hibernation
> But, I may be wrong, and I'm going to see if a scenario such as this
> make sense. I've queued this one to the hyperv-next, but I'll wait for
> the x86 folks to send their pull request to Linus first before I do it
> for these patches.
Actually IMHO you don't need to wait, because there is not a build
dependency, so either patchset can go into the Linus's tree first.
The 2 patchsets are just the first step to make hibernation work for Linux VM
running on Hyper-V. Next I'm going to post some high-level VSC patches for
hv_balloon, hv_utils, hv_netvsc, hid_hyperv, hv_storvsc, hyperv_keyboard,
hyperv_fb,etc. All of these should go through the hyperv tree, since they're
pure hyper-v changes, and they depend on this 9-patch patchset. I'll make
a note in every patch so the subsystem maintainers will be aware and ack it.
Among the VSC patches, the hv_balloon patch does depend on the 2nd patch:
[PATCH v5 2/3] x86/hyper-v: Implement hv_is_hibernation_supported().
I think I'll wait for the aforementioned 2 patchsets to be in first, before posting
the hv_balloon patch.
> Usually cases such as these are the exception, but for Hyper-V it seems
> to be the norm, so I'm curious to see how this will unfold.
>
> Thanks,
> Sasha
Thanks for taking care all the patches!
Thanks,
-- Dexuan
^ permalink raw reply
* Re: [PATCH net-next, 2/2] hv_netvsc: Sync offloading features to VF NIC
From: Jakub Kicinski @ 2019-09-07 4:25 UTC (permalink / raw)
To: Haiyang Zhang
Cc: sashal@kernel.org, linux-hyperv@vger.kernel.org,
netdev@vger.kernel.org, KY Srinivasan, Stephen Hemminger,
olaf@aepfle.de, vkuznets, davem@davemloft.net,
linux-kernel@vger.kernel.org, Mark Bloch
In-Reply-To: <DM6PR21MB13373166435FD2FC5543D349CABB0@DM6PR21MB1337.namprd21.prod.outlook.com>
On Thu, 5 Sep 2019 23:07:32 +0000, Haiyang Zhang wrote:
> > On Fri, 30 Aug 2019 03:45:38 +0000, Haiyang Zhang wrote:
> > > VF NIC may go down then come up during host servicing events. This
> > > causes the VF NIC offloading feature settings to roll back to the
> > > defaults. This patch can synchronize features from synthetic NIC to
> > > the VF NIC during ndo_set_features (ethtool -K), and
> > > netvsc_register_vf when VF comes back after host events.
> > >
> > > Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> > > Cc: Mark Bloch <markb@mellanox.com>
> >
> > If we want to make this change in behaviour we should change net_failover
> > at the same time.
>
> After checking the net_failover, I found it's for virtio based SRIOV, and very
> different from what we did for Hyper-V based SRIOV.
>
> We let the netvsc driver acts as both the synthetic (PV) driver and the transparent
> bonding master for the VF NIC. But net_failover acts as a master device on top
> of both virtio PV NIC, and VF NIC. And the net_failover doesn't implemented
> operations, like ndo_set_features.
> So the code change for our netvsc driver cannot be applied to net_failover driver.
>
> I will re-submit my two patches (fixing the extra tab in the 1st one as you pointed
> out). Thanks!
I think it stands to reason that two modules which implement the same
functionality behave the same.
^ permalink raw reply
* Re: [PATCH net-next,v2, 0/2] Enable sg as tunable, sync offload settings to VF NIC
From: David Miller @ 2019-09-07 15:43 UTC (permalink / raw)
To: haiyangz
Cc: sashal, linux-hyperv, netdev, kys, sthemmin, olaf, vkuznets,
linux-kernel
In-Reply-To: <1567725722-33552-1-git-send-email-haiyangz@microsoft.com>
From: Haiyang Zhang <haiyangz@microsoft.com>
Date: Thu, 5 Sep 2019 23:22:58 +0000
> This patch set fixes an issue in SG tuning, and sync
> offload settings from synthetic NIC to VF NIC.
Series applied to net-next.
^ permalink raw reply
* [PATCH 0/3] Remove __online_page_set_limits()
From: Souptick Joarder @ 2019-09-07 21:47 UTC (permalink / raw)
To: kys, haiyangz, sthemmin, sashal, boris.ostrovsky, jgross,
sstabellini, akpm, david, osalvador, mhocko, pasha.tatashin,
dan.j.williams, richard.weiyang, cai
Cc: linux-hyperv, xen-devel, linux-mm, linux-kernel, Souptick Joarder
__online_page_set_limits() is a dummy function and an extra call
to this can be avoided.
As both of the callers are now removed, __online_page_set_limits()
can be removed permanently.
Souptick Joarder (3):
hv_ballon: Avoid calling dummy function __online_page_set_limits()
xen/ballon: Avoid calling dummy function __online_page_set_limits()
mm/memory_hotplug.c: Remove __online_page_set_limits()
drivers/hv/hv_balloon.c | 1 -
drivers/xen/balloon.c | 1 -
include/linux/memory_hotplug.h | 1 -
mm/memory_hotplug.c | 5 -----
4 files changed, 8 deletions(-)
--
1.9.1
^ permalink raw reply
* [PATCH 1/3] hv_ballon: Avoid calling dummy function __online_page_set_limits()
From: Souptick Joarder @ 2019-09-07 21:47 UTC (permalink / raw)
To: kys, haiyangz, sthemmin, sashal, boris.ostrovsky, jgross,
sstabellini, akpm, david, osalvador, mhocko, pasha.tatashin,
dan.j.williams, richard.weiyang, cai
Cc: linux-hyperv, xen-devel, linux-mm, linux-kernel, Souptick Joarder
In-Reply-To: <cover.1567889743.git.jrdr.linux@gmail.com>
__online_page_set_limits() is a dummy function and an extra call
to this function can be avoided.
Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
---
drivers/hv/hv_balloon.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c
index 6fb4ea5..9bab443 100644
--- a/drivers/hv/hv_balloon.c
+++ b/drivers/hv/hv_balloon.c
@@ -680,7 +680,6 @@ static void hv_page_online_one(struct hv_hotadd_state *has, struct page *pg)
__ClearPageOffline(pg);
/* This frame is currently backed; online the page. */
- __online_page_set_limits(pg);
__online_page_increment_counters(pg);
__online_page_free(pg);
--
1.9.1
^ permalink raw reply related
* [PATCH 2/3] xen/ballon: Avoid calling dummy function __online_page_set_limits()
From: Souptick Joarder @ 2019-09-07 21:47 UTC (permalink / raw)
To: kys, haiyangz, sthemmin, sashal, boris.ostrovsky, jgross,
sstabellini, akpm, david, osalvador, mhocko, pasha.tatashin,
dan.j.williams, richard.weiyang, cai
Cc: linux-hyperv, xen-devel, linux-mm, linux-kernel, Souptick Joarder
In-Reply-To: <cover.1567889743.git.jrdr.linux@gmail.com>
__online_page_set_limits() is a dummy function and an extra call
to this function can be avoided.
Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
---
drivers/xen/balloon.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index 4e11de6..05b1f7e 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -376,7 +376,6 @@ static void xen_online_page(struct page *page, unsigned int order)
mutex_lock(&balloon_mutex);
for (i = 0; i < size; i++) {
p = pfn_to_page(start_pfn + i);
- __online_page_set_limits(p);
__SetPageOffline(p);
__balloon_append(p);
}
--
1.9.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox