From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCH net] failover: eliminate callback hell Date: Mon, 11 Jun 2018 17:01:48 +0300 Message-ID: <20180611170056-mutt-send-email-mst@kernel.org> References: <20180605034231.31610-1-sthemmin@microsoft.com> <20180606072512.GA2289@nanopsycho.orion> <20180606151955-mutt-send-email-mst@kernel.org> <20180606142447.3c5072d8@xeon-e3> <20180608161801.64afda65@xeon-e3> <20180608170235.7f345b58@xeon-e3> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Siwei Liu , Jiri Pirko , kys@microsoft.com, haiyangz@microsoft.com, David Miller , "Samudrala, Sridhar" , Netdev , Stephen Hemminger To: Stephen Hemminger Return-path: Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46174 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933076AbeFKOBu (ORCPT ); Mon, 11 Jun 2018 10:01:50 -0400 Content-Disposition: inline In-Reply-To: <20180608170235.7f345b58@xeon-e3> Sender: netdev-owner@vger.kernel.org List-ID: On Fri, Jun 08, 2018 at 05:02:35PM -0700, Stephen Hemminger wrote: > On Fri, 8 Jun 2018 16:44:12 -0700 > Siwei Liu wrote: > > > On Fri, Jun 8, 2018 at 4:18 PM, Stephen Hemminger > > wrote: > > > On Fri, 8 Jun 2018 15:25:59 -0700 > > > Siwei Liu wrote: > > > > > >> On Wed, Jun 6, 2018 at 2:24 PM, Stephen Hemminger > > >> wrote: > > >> > On Wed, 6 Jun 2018 15:30:27 +0300 > > >> > "Michael S. Tsirkin" wrote: > > >> > > > >> >> On Wed, Jun 06, 2018 at 09:25:12AM +0200, Jiri Pirko wrote: > > >> >> > Tue, Jun 05, 2018 at 05:42:31AM CEST, stephen@networkplumber.org wrote: > > >> >> > >The net failover should be a simple library, not a virtual > > >> >> > >object with function callbacks (see callback hell). > > >> >> > > > >> >> > Why just a library? It should do a common things. I think it should be a > > >> >> > virtual object. Looks like your patch again splits the common > > >> >> > functionality into multiple drivers. That is kind of backwards attitude. > > >> >> > I don't get it. We should rather focus on fixing the mess the > > >> >> > introduction of netvsc-bonding caused and switch netvsc to 3-netdev > > >> >> > model. > > >> >> > > >> >> So it seems that at least one benefit for netvsc would be better > > >> >> handling of renames. > > >> >> > > >> >> Question is how can this change to 3-netdev happen? Stephen is > > >> >> concerned about risk of breaking some userspace. > > >> >> > > >> >> Stephen, this seems to be the usecase that IFF_HIDDEN was trying to > > >> >> address, and you said then "why not use existing network namespaces > > >> >> rather than inventing a new abstraction". So how about it then? Do you > > >> >> want to find a way to use namespaces to hide the PV device for netvsc > > >> >> compatibility? > > >> >> > > >> > > > >> > Netvsc can't work with 3 dev model. MS has worked with enough distro's and > > >> > startups that all demand eth0 always be present. And VF may come and go. > > >> > After this history, there is a strong motivation not to change how kernel > > >> > behaves. Switching to 3 device model would be perceived as breaking > > >> > existing userspace. > > >> > > > >> > With virtio you can work it out with the distro's yourself. > > >> > There is no pre-existing semantics to deal with. > > >> > > > >> > For the virtio, I don't see the need for IFF_HIDDEN. > > >> > > >> I have a somewhat different view regarding IFF_HIDDEN. The purpose of > > >> that flag, as well as the 1-netdev model, is to have a means to > > >> inherit the interface name from the VF, and to eliminate playing hacks > > >> around renaming devices, customizing udev rules and et al. Why > > >> inheriting VF's name important? To allow existing config/setup around > > >> VF continues to work across kernel feature upgrade. Most of network > > >> config files in all distros are based on interface names. Few are MAC > > >> address based but making lower slaves hidden would cover the rest. And > > >> most importantly, preserving the same level of user experience as > > >> using raw VF interface once getting all ndo_ops and ethtool_ops > > >> exposed. This is essential to realize transparent live migration that > > >> users dont have to learn and be aware of the undertaken. > > > > > > Inheriting the VF name will fail in the migration scenario. > > > It is perfectly reasonable to migrate a guest to another machine where > > > the VF PCI address is different. And since current udev/systemd model > > > is to base network device name off of PCI address, the device will change > > > name when guest is migrated. > > > > > The scenario of having VF on a different PCI address on post migration > > is essentially equal to plugging in a new NIC. Why it has to pair with > > the original PV? A sepearte PV device should be in place to pair the > > new VF. > > The host only guarantees that the PV device will be on the same network. > It does not make any PCI guarantees. The way Windows works is to find > the device based on "serial number" which is an Hyper-V specific attribute > of PCI devices. > > I considered naming off of serial number but that won't work for the > case where PV device is present first and VF arrives later. The serial > number is attribute of VF, not the PV which is there first. > > Your ideas about having the PCI information of the VF form the name > of the failover device have the same problem. The PV device may > be the only one present on boot. We plan to add the serial number to the PV. > > > > On Azure, the VF maybe removed (by host) at any time and then later > > > reattached. There is no guarantee that VF will show back up at > > > the same synthetic PCI address. It will likely have a different > > > PCI domain value. > > > > This is something QEMU can do and make sure the PCI address is > > consistent after migration. > > > > -Siwei