Re: [PATCH net] failover: eliminate callback hell

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 7 Jun 2018 07:17:51 -0700
Alexander Duyck <alexander.duyck@gmail.com> wrote:

> On Wed, Jun 6, 2018 at 3:25 PM, Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> > On Wed, 6 Jun 2018 14:54:04 -0700
> > "Samudrala, Sridhar" <sridhar.samudrala@intel.com> wrote:
> >  
> >> On 6/6/2018 2:24 PM, Stephen Hemminger wrote:  
> >> > On Wed, 6 Jun 2018 15:30:27 +0300
> >> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >> >  
> >> >> On Wed, Jun 06, 2018 at 09:25:12AM +0200, Jiri Pirko wrote:  
> >> >>> Tue, Jun 05, 2018 at 05:42:31AM CEST, stephen@networkplumber.org wrote:  
> >> >>>> The net failover should be a simple library, not a virtual
> >> >>>> object with function callbacks (see callback hell).  
> >> >>> Why just a library? It should do a common things. I think it should be a
> >> >>> virtual object. Looks like your patch again splits the common
> >> >>> functionality into multiple drivers. That is kind of backwards attitude.
> >> >>> I don't get it. We should rather focus on fixing the mess the
> >> >>> introduction of netvsc-bonding caused and switch netvsc to 3-netdev
> >> >>> model.  
> >> >> So it seems that at least one benefit for netvsc would be better
> >> >> handling of renames.
> >> >>
> >> >> Question is how can this change to 3-netdev happen?  Stephen is
> >> >> concerned about risk of breaking some userspace.
> >> >>
> >> >> Stephen, this seems to be the usecase that IFF_HIDDEN was trying to
> >> >> address, and you said then "why not use existing network namespaces
> >> >> rather than inventing a new abstraction". So how about it then? Do you
> >> >> want to find a way to use namespaces to hide the PV device for netvsc
> >> >> compatibility?
> >> >>  
> >> > Netvsc can't work with 3 dev model. MS has worked with enough distro's and
> >> > startups that all demand eth0 always be present. And VF may come and go.
> >> > After this history, there is a strong motivation not to change how kernel
> >> > behaves. Switching to 3 device model would be perceived as breaking
> >> > existing userspace.  
> >>
> >> I think it should be possible for netvsc to work with 3 dev model if the only
> >> requirement is that eth0 will always be present. With net_failover, you will
> >> see eth0 and eth0nsby OR with older distros eth0 and eth1.  It may be an issue
> >> if somehow there is userspace requirement that there can be only 2 netdevs, not 3
> >> when VF is plugged.
> >>
> >> eth0 will be the net_failover device and eth0nsby/eth1 will be the netvsc device
> >> and the IP address gets configured on eth0. Will this be an issue?  
> >
> > DPDK drivers in 18.05 depend on 2 device model. Yes it is a bit of mess
> > but that is the way it is.  
> 
> Why would DPDK care what we do in the kernel? Isn't it just slapping
> vfio-pci on the netdevs it sees?

Alex, you are correct for Intel devices; but DPDK on Azure is not Intel based.,.
The DPDK support uses:
 * Mellanox MLX5 which uses the Infinband hooks to do DMA directly to
   userspace. This means VF netdev device must exist and be visible.
 * Slow path using kernel netvsc device, TAP and BPF to get exception
   path packets to userspace.
 * A autodiscovery mechanism that to set all this up that relies on
   2 device model and sysfs.

In this version, there is no VFIO-PCI. And also Hyper-V does not have virtual
IOMMU so VFIO will not work there at all.