linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH RESEND] netvsc: transfer lower device max tso size
@ 2025-07-18  6:17 Cindy Lu
  2025-07-21 23:28 ` Jakub Kicinski
  0 siblings, 1 reply; 16+ messages in thread
From: Cindy Lu @ 2025-07-18  6:17 UTC (permalink / raw)
  To: K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Andrew Lunn,
	David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Jason Wang, Stanislav Fomichev, Kuniyuki Iwashima,
	Alexander Lobakin, Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list, lulu

From: Jason Wang <jasowang@redhat.com>

When netvsc is accelerated by the lower device, we can advertise the
lower device max tso size in order to get better performance.

One example is that when 802.3ad encap is enabled by netvsc, it has a
lower max tso size than 64K. This will lead to software segmentation
of forwarding GSO packet (e.g the one from VM/tap).

This patch help to recover the performance.

Signed-off-by: Jason Wang <jasowang@redhat.com>
Tested-by: Cindy Lu <lulu@redhat.com>
---
 drivers/net/hyperv/netvsc_drv.c |  2 +-
 include/linux/netdevice.h       |  4 ++++
 net/core/dev.c                  | 18 ++++++++++++++++++
 3 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index c41a025c66f0..7af4aa4f4abe 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2440,7 +2440,7 @@ static int netvsc_vf_changed(struct net_device *vf_netdev, unsigned long event)
 		 * switched over to the VF
 		 */
 		if (vf_is_up)
-			netif_set_tso_max_size(ndev, vf_netdev->tso_max_size);
+			netif_stacked_transfer_tso_max_size(vf_netdev, ndev);
 		else
 			netif_set_tso_max_size(ndev, netvsc_dev->netvsc_gso_max_size);
 	}
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index adb14db25798..c695a3ffecd8 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -5275,6 +5275,9 @@ void netdev_change_features(struct net_device *dev);
 void netif_stacked_transfer_operstate(const struct net_device *rootdev,
 					struct net_device *dev);
 
+void netif_stacked_transfer_tso_max_size(const struct net_device *rootdev,
+					 struct net_device *dev);
+
 netdev_features_t passthru_features_check(struct sk_buff *skb,
 					  struct net_device *dev,
 					  netdev_features_t features);
@@ -5326,6 +5329,7 @@ static inline bool netif_needs_gso(struct sk_buff *skb,
 }
 
 void netif_set_tso_max_size(struct net_device *dev, unsigned int size);
+
 void netif_set_tso_max_segs(struct net_device *dev, unsigned int segs);
 void netif_inherit_tso_max(struct net_device *to,
 			   const struct net_device *from);
diff --git a/net/core/dev.c b/net/core/dev.c
index be97c440ecd5..3bec4284adff 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -3306,6 +3306,24 @@ void netif_set_tso_max_size(struct net_device *dev, unsigned int size)
 }
 EXPORT_SYMBOL(netif_set_tso_max_size);
 
+/**
+ *	netif_stacked_transfer_tso_max_size - transfer tso max size
+ *	@rootdev: the root or lower level device to transfer tso max size from
+ *	@dev: the device to transfer operstate to
+ *
+ *	Transfer tso max size from root to device. This is normally
+ *	called when a stacking relationship exists between the root
+ *	device and the device(a leaf device).
+ */
+void netif_stacked_transfer_tso_max_size(const struct net_device *rootdev,
+					 struct net_device *dev)
+{
+	dev->tso_max_size = rootdev->tso_max_size;
+	netif_set_gso_max_size(dev, READ_ONCE(rootdev->gso_max_size));
+	netif_set_gso_ipv4_max_size(dev, READ_ONCE(rootdev->gso_ipv4_max_size));
+}
+EXPORT_SYMBOL(netif_stacked_transfer_tso_max_size);
+
 /**
  * netif_set_tso_max_segs() - set the max number of segs supported for TSO
  * @dev:	netdev to update
-- 
2.45.0


^ permalink raw reply related	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-18  6:17 [PATCH RESEND] netvsc: transfer lower device max tso size Cindy Lu
@ 2025-07-21 23:28 ` Jakub Kicinski
  2025-07-22  1:04   ` Jason Wang
  0 siblings, 1 reply; 16+ messages in thread
From: Jakub Kicinski @ 2025-07-21 23:28 UTC (permalink / raw)
  To: Cindy Lu
  Cc: K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui, Andrew Lunn,
	David S. Miller, Eric Dumazet, Paolo Abeni, Simon Horman,
	Michael Kelley, Shradha Gupta, Kees Cook, Jason Wang,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Fri, 18 Jul 2025 14:17:55 +0800 Cindy Lu wrote:
> Subject: [PATCH RESEND] netvsc: transfer lower device max tso size

You say RESEND but I don't see a link to previous posting anywhere.

I'd rather we didn't extend the magic behavior of hyperv/netvsc any
further. We have enough problems with it.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-21 23:28 ` Jakub Kicinski
@ 2025-07-22  1:04   ` Jason Wang
  2025-07-22  1:18     ` Jakub Kicinski
  0 siblings, 1 reply; 16+ messages in thread
From: Jason Wang @ 2025-07-22  1:04 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Cindy Lu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Tue, Jul 22, 2025 at 7:28 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Fri, 18 Jul 2025 14:17:55 +0800 Cindy Lu wrote:
> > Subject: [PATCH RESEND] netvsc: transfer lower device max tso size
>
> You say RESEND but I don't see a link to previous posting anywhere.
>
> I'd rather we didn't extend the magic behavior of hyperv/netvsc any
> further.

Are you referring to the netdev coupling model of the VF acceleration?

> We have enough problems with it.
>

But this fixes a real problem, otherwise nested VM performance will be
broken due to the GSO software segmentation.

Thanks


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-22  1:04   ` Jason Wang
@ 2025-07-22  1:18     ` Jakub Kicinski
  2025-07-22  2:04       ` Cindy Lu
  2025-07-23  6:00       ` Jason Wang
  0 siblings, 2 replies; 16+ messages in thread
From: Jakub Kicinski @ 2025-07-22  1:18 UTC (permalink / raw)
  To: Jason Wang
  Cc: Cindy Lu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Tue, 22 Jul 2025 09:04:20 +0800 Jason Wang wrote:
> On Tue, Jul 22, 2025 at 7:28 AM Jakub Kicinski <kuba@kernel.org> wrote:
> > On Fri, 18 Jul 2025 14:17:55 +0800 Cindy Lu wrote:  
> > > Subject: [PATCH RESEND] netvsc: transfer lower device max tso size  
> >
> > You say RESEND but I don't see a link to previous posting anywhere.

Someone should respond to this part, please.

> > I'd rather we didn't extend the magic behavior of hyperv/netvsc any
> > further.  
> 
> Are you referring to the netdev coupling model of the VF acceleration?

Yes, it tries to apply whole bunch of policy automatically in 
the kernel.

> > We have enough problems with it.
> 
> But this fixes a real problem, otherwise nested VM performance will be
> broken due to the GSO software segmentation.

Perhaps, possibly, a migration plan can be devised, away from the
netvsc model, so we don't have to deal with nuggets of joy like:
https://lore.kernel.org/all/1752870014-28909-1-git-send-email-haiyangz@linux.microsoft.com/

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-22  1:18     ` Jakub Kicinski
@ 2025-07-22  2:04       ` Cindy Lu
  2025-07-22  2:46         ` Cindy Lu
  2025-07-23  6:00       ` Jason Wang
  1 sibling, 1 reply; 16+ messages in thread
From: Cindy Lu @ 2025-07-22  2:04 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Jason Wang, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Tue, Jul 22, 2025 at 9:18 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Tue, 22 Jul 2025 09:04:20 +0800 Jason Wang wrote:
> > On Tue, Jul 22, 2025 at 7:28 AM Jakub Kicinski <kuba@kernel.org> wrote:
> > > On Fri, 18 Jul 2025 14:17:55 +0800 Cindy Lu wrote:
> > > > Subject: [PATCH RESEND] netvsc: transfer lower device max tso size
> > >
> > > You say RESEND but I don't see a link to previous posting anywhere.
>
> Someone should respond to this part, please.
>
Hi Jakub,
sorry for the confusion. I previously sent this mail
(https://lore.kernel.org/all/20250718060615.237986-1-lulu@redhat.com/)
to the wrong mailing list, so I'm resended it here.
I've also submitted a v2 of this patch:
https://lore.kernel.org/all/20250718082909.243488-1-lulu@redhat.com/
Sorry again for the mix-up.
thanks

cindy

> > > I'd rather we didn't extend the magic behavior of hyperv/netvsc any
> > > further.
> >
> > Are you referring to the netdev coupling model of the VF acceleration?
>
> Yes, it tries to apply whole bunch of policy automatically in
> the kernel.
>
> > > We have enough problems with it.
> >
> > But this fixes a real problem, otherwise nested VM performance will be
> > broken due to the GSO software segmentation.
>
> Perhaps, possibly, a migration plan can be devised, away from the
> netvsc model, so we don't have to deal with nuggets of joy like:
> https://lore.kernel.org/all/1752870014-28909-1-git-send-email-haiyangz@linux.microsoft.com/
>


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-22  2:04       ` Cindy Lu
@ 2025-07-22  2:46         ` Cindy Lu
  0 siblings, 0 replies; 16+ messages in thread
From: Cindy Lu @ 2025-07-22  2:46 UTC (permalink / raw)
  To: Jakub Kicinski, Long Li, stephen
  Cc: Jason Wang, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Tue, Jul 22, 2025 at 10:04 AM Cindy Lu <lulu@redhat.com> wrote:
>
> On Tue, Jul 22, 2025 at 9:18 AM Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > On Tue, 22 Jul 2025 09:04:20 +0800 Jason Wang wrote:
> > > On Tue, Jul 22, 2025 at 7:28 AM Jakub Kicinski <kuba@kernel.org> wrote:
> > > > On Fri, 18 Jul 2025 14:17:55 +0800 Cindy Lu wrote:
> > > > > Subject: [PATCH RESEND] netvsc: transfer lower device max tso size
> > > >
> > > > You say RESEND but I don't see a link to previous posting anywhere.
> >
> > Someone should respond to this part, please.
> >
> Hi Jakub,
> sorry for the confusion. I previously sent this mail
> (https://lore.kernel.org/all/20250718060615.237986-1-lulu@redhat.com/)
> to the wrong mailing list, so I'm resended it here.
> I've also submitted a v2 of this patch:
> https://lore.kernel.org/all/20250718082909.243488-1-lulu@redhat.com/
> Sorry again for the mix-up.
> thanks
>
> cindy
>
> > > > I'd rather we didn't extend the magic behavior of hyperv/netvsc any
> > > > further.
> > >
> > > Are you referring to the netdev coupling model of the VF acceleration?
> >
> > Yes, it tries to apply whole bunch of policy automatically in
> > the kernel.
> >
> > > > We have enough problems with it.
> > >
> > > But this fixes a real problem, otherwise nested VM performance will be
> > > broken due to the GSO software segmentation.
> >
> > Perhaps, possibly, a migration plan can be devised, away from the
> > netvsc model, so we don't have to deal with nuggets of joy like:
> > https://lore.kernel.org/all/1752870014-28909-1-git-send-email-haiyangz@linux.microsoft.com/

I'm also including Stephen Hemminger and Long Li in this thread and
would greatly appreciate any suggestions.

Thanks
cindy

> >


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-22  1:18     ` Jakub Kicinski
  2025-07-22  2:04       ` Cindy Lu
@ 2025-07-23  6:00       ` Jason Wang
  2025-07-23 15:05         ` Jakub Kicinski
  1 sibling, 1 reply; 16+ messages in thread
From: Jason Wang @ 2025-07-23  6:00 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Cindy Lu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Tue, Jul 22, 2025 at 9:18 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Tue, 22 Jul 2025 09:04:20 +0800 Jason Wang wrote:
> > On Tue, Jul 22, 2025 at 7:28 AM Jakub Kicinski <kuba@kernel.org> wrote:
> > > On Fri, 18 Jul 2025 14:17:55 +0800 Cindy Lu wrote:
> > > > Subject: [PATCH RESEND] netvsc: transfer lower device max tso size
> > >
> > > You say RESEND but I don't see a link to previous posting anywhere.
>
> Someone should respond to this part, please.
>
> > > I'd rather we didn't extend the magic behavior of hyperv/netvsc any
> > > further.
> >
> > Are you referring to the netdev coupling model of the VF acceleration?
>
> Yes, it tries to apply whole bunch of policy automatically in
> the kernel.
>
> > > We have enough problems with it.
> >
> > But this fixes a real problem, otherwise nested VM performance will be
> > broken due to the GSO software segmentation.
>
> Perhaps, possibly, a migration plan can be devised, away from the
> netvsc model, so we don't have to deal with nuggets of joy like:
> https://lore.kernel.org/all/1752870014-28909-1-git-send-email-haiyangz@linux.microsoft.com/

Btw, if I understand this correctly. This is for future development so
it's not a blocker for this patch?

Thanks

>


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-23  6:00       ` Jason Wang
@ 2025-07-23 15:05         ` Jakub Kicinski
  2025-07-23 20:18           ` Haiyang Zhang
  2025-07-31  1:07           ` Jason Wang
  0 siblings, 2 replies; 16+ messages in thread
From: Jakub Kicinski @ 2025-07-23 15:05 UTC (permalink / raw)
  To: Jason Wang
  Cc: Cindy Lu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Wed, 23 Jul 2025 14:00:47 +0800 Jason Wang wrote:
> > > But this fixes a real problem, otherwise nested VM performance will be
> > > broken due to the GSO software segmentation.  
> >
> > Perhaps, possibly, a migration plan can be devised, away from the
> > netvsc model, so we don't have to deal with nuggets of joy like:
> > https://lore.kernel.org/all/1752870014-28909-1-git-send-email-haiyangz@linux.microsoft.com/  
> 
> Btw, if I understand this correctly. This is for future development so
> it's not a blocker for this patch?

Not a blocker, I'm just giving an example of the netvsc auto-weirdness
being a source of tech debt and bugs. Commit d7501e076d859d is another
recent one off the top of my head. IIUC systemd-networkd is broadly
deployed now. It'd be great if there was some migration plan for moving
this sort of VM auto-bonding to user space (with the use of the common
bonding driver, not each hypervisor rolling its own).

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-23 15:05         ` Jakub Kicinski
@ 2025-07-23 20:18           ` Haiyang Zhang
  2025-07-23 22:16             ` Jakub Kicinski
  2025-07-31  1:07           ` Jason Wang
  1 sibling, 1 reply; 16+ messages in thread
From: Haiyang Zhang @ 2025-07-23 20:18 UTC (permalink / raw)
  To: Jakub Kicinski, Jason Wang, Stephen Hemminger
  Cc: Cindy Lu, KY Srinivasan, Wei Liu, Dexuan Cui, Andrew Lunn,
	David S. Miller, Eric Dumazet, Paolo Abeni, Simon Horman,
	Michael Kelley, Shradha Gupta, Kees Cook, Stanislav Fomichev,
	Kuniyuki Iwashima, Alexander Lobakin, Guillaume Nault, Joe Damato,
	Ahmed Zaki, open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list



> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, July 23, 2025 11:06 AM
> To: Jason Wang <jasowang@redhat.com>
> Cc: Cindy Lu <lulu@redhat.com>; KY Srinivasan <kys@microsoft.com>; Haiyang
> Zhang <haiyangz@microsoft.com>; Wei Liu <wei.liu@kernel.org>; Dexuan Cui
> <decui@microsoft.com>; Andrew Lunn <andrew+netdev@lunn.ch>; David S.
> Miller <davem@davemloft.net>; Eric Dumazet <edumazet@google.com>; Paolo
> Abeni <pabeni@redhat.com>; Simon Horman <horms@kernel.org>; Michael Kelley
> <mhklinux@outlook.com>; Shradha Gupta <shradhagupta@linux.microsoft.com>;
> Kees Cook <kees@kernel.org>; Stanislav Fomichev <sdf@fomichev.me>;
> Kuniyuki Iwashima <kuniyu@google.com>; Alexander Lobakin
> <aleksander.lobakin@intel.com>; Guillaume Nault <gnault@redhat.com>; Joe
> Damato <jdamato@fastly.com>; Ahmed Zaki <ahmed.zaki@intel.com>; open
> list:Hyper-V/Azure CORE AND DRIVERS <linux-hyperv@vger.kernel.org>; open
> list:NETWORKING DRIVERS <netdev@vger.kernel.org>; open list <linux-
> kernel@vger.kernel.org>
> Subject: [EXTERNAL] Re: [PATCH RESEND] netvsc: transfer lower device max
> tso size
>
> On Wed, 23 Jul 2025 14:00:47 +0800 Jason Wang wrote:
> > > > But this fixes a real problem, otherwise nested VM performance will
> be
> > > > broken due to the GSO software segmentation.
> > >
> > > Perhaps, possibly, a migration plan can be devised, away from the
> > > netvsc model, so we don't have to deal with nuggets of joy like:
> > >
> https://lore.ker/
> nel.org%2Fall%2F1752870014-28909-1-git-send-email-
> haiyangz%40linux.microsoft.com%2F&data=05%7C02%7Chaiyangz%40microsoft.com%
> 7C27aa73c4b6e9446c7f6508ddc9fa6053%7C72f988bf86f141af91ab2d7cd011db47%7C1%
> 7C0%7C638888799449774753%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsI
> lYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C
> %7C%7C&sdata=HYX4dDie8QlEepzwPiiI3eYASe7WN5o%2BWTXmMGoyFOM%3D&reserved=0
> >
> > Btw, if I understand this correctly. This is for future development so
> > it's not a blocker for this patch?
>
> Not a blocker, I'm just giving an example of the netvsc auto-weirdness
> being a source of tech debt and bugs. Commit d7501e076d859d is another
> recent one off the top of my head. IIUC systemd-networkd is broadly
> deployed now. It'd be great if there was some migration plan for moving
> this sort of VM auto-bonding to user space (with the use of the common
> bonding driver, not each hypervisor rolling its own).

Actually, we had used the common bonding driver 9 years ago. But it's
replaced by this kernel/netvsc based "transparent" bonding mode. See
the patches listed below.

The user mode bonding scripts were unstable, and difficult to deliver
& update for various distros. So Stephen developed the new "transparent"
bonding mode, which greatly improves the situation.
@Stephen Hemminger <stephen@networkplumber.org>, in case he wants to
add more regarding the history.

Related patches:
author  Haiyang Zhang <haiyangz@microsoft.com>  2016-07-11 17:06:42 -0700
committer       David S. Miller <davem@davemloft.net>   2016-07-12 10:41:53 -0700
commit  178cd55f086629cf0bad9c66c793a7e2bcc3abb6 (patch)
tools: hv: Add a script to help bonding synthetic and VF NICs

author  stephen hemminger <stephen@networkplumber.org>  2017-08-01 19:58:55 -0700
committer       David S. Miller <davem@davemloft.net>   2017-08-02 16:55:33 -0700
commit  12aa7469d101e139b3728e540884bc7d72dca70a (patch)
netvsc: remove bonding setup script
No longer needed, now all managed by transparent VF logic.

Thanks,
- Haiyang


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-23 20:18           ` Haiyang Zhang
@ 2025-07-23 22:16             ` Jakub Kicinski
  2025-07-28  3:01               ` Stephen Hemminger
  0 siblings, 1 reply; 16+ messages in thread
From: Jakub Kicinski @ 2025-07-23 22:16 UTC (permalink / raw)
  To: Haiyang Zhang
  Cc: Jason Wang, Stephen Hemminger, Cindy Lu, KY Srinivasan, Wei Liu,
	Dexuan Cui, Andrew Lunn, David S. Miller, Eric Dumazet,
	Paolo Abeni, Simon Horman, Michael Kelley, Shradha Gupta,
	Kees Cook, Stanislav Fomichev, Kuniyuki Iwashima,
	Alexander Lobakin, Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Wed, 23 Jul 2025 20:18:03 +0000 Haiyang Zhang wrote:
> > > Btw, if I understand this correctly. This is for future development so
> > > it's not a blocker for this patch?  
> >
> > Not a blocker, I'm just giving an example of the netvsc auto-weirdness
> > being a source of tech debt and bugs. Commit d7501e076d859d is another
> > recent one off the top of my head. IIUC systemd-networkd is broadly
> > deployed now. It'd be great if there was some migration plan for moving
> > this sort of VM auto-bonding to user space (with the use of the common
> > bonding driver, not each hypervisor rolling its own).  
> 
> Actually, we had used the common bonding driver 9 years ago. But it's
> replaced by this kernel/netvsc based "transparent" bonding mode. See
> the patches listed below.
> 
> The user mode bonding scripts were unstable, and difficult to deliver
> & update for various distros. So Stephen developed the new "transparent"
> bonding mode, which greatly improves the situation.

I specifically highlighted systemd-networkd as the change in the user
space landscape.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-23 22:16             ` Jakub Kicinski
@ 2025-07-28  3:01               ` Stephen Hemminger
  2025-07-28 15:19                 ` Jakub Kicinski
  0 siblings, 1 reply; 16+ messages in thread
From: Stephen Hemminger @ 2025-07-28  3:01 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Haiyang Zhang, Jason Wang, Cindy Lu, KY Srinivasan, Wei Liu,
	Dexuan Cui, Andrew Lunn, David S. Miller, Eric Dumazet,
	Paolo Abeni, Simon Horman, Michael Kelley, Shradha Gupta,
	Kees Cook, Stanislav Fomichev, Kuniyuki Iwashima,
	Alexander Lobakin, Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Wed, 23 Jul 2025 15:16:22 -0700
Jakub Kicinski <kuba@kernel.org> wrote:

> On Wed, 23 Jul 2025 20:18:03 +0000 Haiyang Zhang wrote:
> > > > Btw, if I understand this correctly. This is for future development so
> > > > it's not a blocker for this patch?    
> > >
> > > Not a blocker, I'm just giving an example of the netvsc auto-weirdness
> > > being a source of tech debt and bugs. Commit d7501e076d859d is another
> > > recent one off the top of my head. IIUC systemd-networkd is broadly
> > > deployed now. It'd be great if there was some migration plan for moving
> > > this sort of VM auto-bonding to user space (with the use of the common
> > > bonding driver, not each hypervisor rolling its own).    
> > 
> > Actually, we had used the common bonding driver 9 years ago. But it's
> > replaced by this kernel/netvsc based "transparent" bonding mode. See
> > the patches listed below.
> > 
> > The user mode bonding scripts were unstable, and difficult to deliver
> > & update for various distros. So Stephen developed the new "transparent"
> > bonding mode, which greatly improves the situation.  
> 
> I specifically highlighted systemd-networkd as the change in the user
> space landscape.

Haiyang tried valiantly but getting every distro to do the right thing
with VF's bonding and hot plug was impossible to support.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-28  3:01               ` Stephen Hemminger
@ 2025-07-28 15:19                 ` Jakub Kicinski
  2025-07-28 17:24                   ` Stephen Hemminger
  2025-07-29  2:42                   ` Jason Wang
  0 siblings, 2 replies; 16+ messages in thread
From: Jakub Kicinski @ 2025-07-28 15:19 UTC (permalink / raw)
  To: Stephen Hemminger
  Cc: Haiyang Zhang, Jason Wang, Cindy Lu, KY Srinivasan, Wei Liu,
	Dexuan Cui, Andrew Lunn, David S. Miller, Eric Dumazet,
	Paolo Abeni, Simon Horman, Michael Kelley, Shradha Gupta,
	Kees Cook, Stanislav Fomichev, Kuniyuki Iwashima,
	Alexander Lobakin, Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Sun, 27 Jul 2025 20:01:26 -0700 Stephen Hemminger wrote:
> On Wed, 23 Jul 2025 15:16:22 -0700
> Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > > Actually, we had used the common bonding driver 9 years ago. But it's
> > > replaced by this kernel/netvsc based "transparent" bonding mode. See
> > > the patches listed below.
> > > 
> > > The user mode bonding scripts were unstable, and difficult to deliver
> > > & update for various distros. So Stephen developed the new "transparent"
> > > bonding mode, which greatly improves the situation.    
> > 
> > I specifically highlighted systemd-networkd as the change in the user
> > space landscape.  
> 
> Haiyang tried valiantly but getting every distro to do the right thing
> with VF's bonding and hot plug was impossible to support.

I understand, but I also don't want it to be an upstream Linux problem.

Again, no other cloud provider seems to have this issue, AFAIU.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-28 15:19                 ` Jakub Kicinski
@ 2025-07-28 17:24                   ` Stephen Hemminger
  2025-07-29  2:42                   ` Jason Wang
  1 sibling, 0 replies; 16+ messages in thread
From: Stephen Hemminger @ 2025-07-28 17:24 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Haiyang Zhang, Jason Wang, Cindy Lu, KY Srinivasan, Wei Liu,
	Dexuan Cui, Andrew Lunn, David S. Miller, Eric Dumazet,
	Paolo Abeni, Simon Horman, Michael Kelley, Shradha Gupta,
	Kees Cook, Stanislav Fomichev, Kuniyuki Iwashima,
	Alexander Lobakin, Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Mon, 28 Jul 2025 08:19:07 -0700
Jakub Kicinski <kuba@kernel.org> wrote:

> On Sun, 27 Jul 2025 20:01:26 -0700 Stephen Hemminger wrote:
> > On Wed, 23 Jul 2025 15:16:22 -0700
> > Jakub Kicinski <kuba@kernel.org> wrote:  
> > >  
> > > > Actually, we had used the common bonding driver 9 years ago. But it's
> > > > replaced by this kernel/netvsc based "transparent" bonding mode. See
> > > > the patches listed below.
> > > > 
> > > > The user mode bonding scripts were unstable, and difficult to deliver
> > > > & update for various distros. So Stephen developed the new "transparent"
> > > > bonding mode, which greatly improves the situation.      
> > > 
> > > I specifically highlighted systemd-networkd as the change in the user
> > > space landscape.    
> > 
> > Haiyang tried valiantly but getting every distro to do the right thing
> > with VF's bonding and hot plug was impossible to support.  
> 
> I understand, but I also don't want it to be an upstream Linux problem.
> 
> Again, no other cloud provider seems to have this issue, AFAIU.

The problem is that other cloud providers don't expose the VF, the hide it in HW firmware.
The userspace world is a mess, with systemd, netplan, cloud init, and the SuSe stuff.
And custom appliances that assume that there is a default eth0 device on boot.
Yes, there were users that expect to see eth0 all the time.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-28 15:19                 ` Jakub Kicinski
  2025-07-28 17:24                   ` Stephen Hemminger
@ 2025-07-29  2:42                   ` Jason Wang
  1 sibling, 0 replies; 16+ messages in thread
From: Jason Wang @ 2025-07-29  2:42 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Stephen Hemminger, Haiyang Zhang, Cindy Lu, KY Srinivasan,
	Wei Liu, Dexuan Cui, Andrew Lunn, David S. Miller, Eric Dumazet,
	Paolo Abeni, Simon Horman, Michael Kelley, Shradha Gupta,
	Kees Cook, Stanislav Fomichev, Kuniyuki Iwashima,
	Alexander Lobakin, Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Mon, Jul 28, 2025 at 11:19 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Sun, 27 Jul 2025 20:01:26 -0700 Stephen Hemminger wrote:
> > On Wed, 23 Jul 2025 15:16:22 -0700
> > Jakub Kicinski <kuba@kernel.org> wrote:
> > >
> > > > Actually, we had used the common bonding driver 9 years ago. But it's
> > > > replaced by this kernel/netvsc based "transparent" bonding mode. See
> > > > the patches listed below.
> > > >
> > > > The user mode bonding scripts were unstable, and difficult to deliver
> > > > & update for various distros. So Stephen developed the new "transparent"
> > > > bonding mode, which greatly improves the situation.
> > >
> > > I specifically highlighted systemd-networkd as the change in the user
> > > space landscape.
> >
> > Haiyang tried valiantly but getting every distro to do the right thing
> > with VF's bonding and hot plug was impossible to support.
>
> I understand, but I also don't want it to be an upstream Linux problem.
>
> Again, no other cloud provider seems to have this issue, AFAIU.
>

There's a failover module which is used by virtio-net now. Maybe
that's a good way for netvsc as well?

Thanks


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-23 15:05         ` Jakub Kicinski
  2025-07-23 20:18           ` Haiyang Zhang
@ 2025-07-31  1:07           ` Jason Wang
  2025-07-31  1:18             ` Jakub Kicinski
  1 sibling, 1 reply; 16+ messages in thread
From: Jason Wang @ 2025-07-31  1:07 UTC (permalink / raw)
  To: Jakub Kicinski
  Cc: Cindy Lu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

Hi Jakub,

On Wed, Jul 23, 2025 at 11:05 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Wed, 23 Jul 2025 14:00:47 +0800 Jason Wang wrote:
> > > > But this fixes a real problem, otherwise nested VM performance will be
> > > > broken due to the GSO software segmentation.
> > >
> > > Perhaps, possibly, a migration plan can be devised, away from the
> > > netvsc model, so we don't have to deal with nuggets of joy like:
> > > https://lore.kernel.org/all/1752870014-28909-1-git-send-email-haiyangz@linux.microsoft.com/
> >
> > Btw, if I understand this correctly. This is for future development so
> > it's not a blocker for this patch?
>
> Not a blocker, I'm just giving an example of the netvsc auto-weirdness
> being a source of tech debt and bugs. Commit d7501e076d859d is another
> recent one off the top of my head. IIUC systemd-networkd is broadly
> deployed now. It'd be great if there was some migration plan for moving
> this sort of VM auto-bonding to user space (with the use of the common
> bonding driver, not each hypervisor rolling its own).
>

Please let me know if you want to merge this patch or not. If not, how
to proceed.

Thanks


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH RESEND] netvsc: transfer lower device max tso size
  2025-07-31  1:07           ` Jason Wang
@ 2025-07-31  1:18             ` Jakub Kicinski
  0 siblings, 0 replies; 16+ messages in thread
From: Jakub Kicinski @ 2025-07-31  1:18 UTC (permalink / raw)
  To: Jason Wang
  Cc: Cindy Lu, K. Y. Srinivasan, Haiyang Zhang, Wei Liu, Dexuan Cui,
	Andrew Lunn, David S. Miller, Eric Dumazet, Paolo Abeni,
	Simon Horman, Michael Kelley, Shradha Gupta, Kees Cook,
	Stanislav Fomichev, Kuniyuki Iwashima, Alexander Lobakin,
	Guillaume Nault, Joe Damato, Ahmed Zaki,
	open list:Hyper-V/Azure CORE AND DRIVERS,
	open list:NETWORKING DRIVERS, open list

On Thu, 31 Jul 2025 09:07:27 +0800 Jason Wang wrote:
> > > Btw, if I understand this correctly. This is for future development so
> > > it's not a blocker for this patch?  
> >
> > Not a blocker, I'm just giving an example of the netvsc auto-weirdness
> > being a source of tech debt and bugs. Commit d7501e076d859d is another
> > recent one off the top of my head. IIUC systemd-networkd is broadly
> > deployed now. It'd be great if there was some migration plan for moving
> > this sort of VM auto-bonding to user space (with the use of the common
> > bonding driver, not each hypervisor rolling its own).
> >  
> 
> Please let me know if you want to merge this patch or not. If not, how
> to proceed.

As is its definitely not getting merged.
Please make it look less burdensome or fix it in user space(!!).

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2025-07-31  1:18 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-18  6:17 [PATCH RESEND] netvsc: transfer lower device max tso size Cindy Lu
2025-07-21 23:28 ` Jakub Kicinski
2025-07-22  1:04   ` Jason Wang
2025-07-22  1:18     ` Jakub Kicinski
2025-07-22  2:04       ` Cindy Lu
2025-07-22  2:46         ` Cindy Lu
2025-07-23  6:00       ` Jason Wang
2025-07-23 15:05         ` Jakub Kicinski
2025-07-23 20:18           ` Haiyang Zhang
2025-07-23 22:16             ` Jakub Kicinski
2025-07-28  3:01               ` Stephen Hemminger
2025-07-28 15:19                 ` Jakub Kicinski
2025-07-28 17:24                   ` Stephen Hemminger
2025-07-29  2:42                   ` Jason Wang
2025-07-31  1:07           ` Jason Wang
2025-07-31  1:18             ` Jakub Kicinski

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).