* Re: [PATCH] compat-wireless: updates for orinoco
From: Stephen Hemminger @ 2010-05-05 0:04 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Hauke Mehrtens, David Miller, linux-wireless, mcgrof, netdev
In-Reply-To: <v2u43e72e891005041626n14deaca5z284f2472e909c923@mail.gmail.com>
On Tue, 4 May 2010 16:26:53 -0700
"Luis R. Rodriguez" <lrodriguez@atheros.com> wrote:
> First of all, thanks a lot! Some comments below.
>
> On Tue, May 4, 2010 at 3:40 PM, Hauke Mehrtens <hauke@hauke-m.de> wrote:
> > * Make all the patches apply again.
> > * rename read_pda to avoid conflicts with definitions in kernel <= 2.6.29
>
> I'm going to apply these two changes, if you get time can you send a
> patch to rename read_pda upstream as well, that way we don't have to
> carry this?
>
> > * add orinoco usb
>
> Thanks for this but I've grown tired of updating these netdev ops and
> I think we can do better. I'll add a netdev_attach_ops() which would
> simply do all the backport stuff for us, this way for backporting
> purposes all we have to do is replace the old lines with a
> netdev_attach_ops() call. In fact if we *really* wanted to we could
> add a dummy netdev_attach_ops() upstream and just backport that on
> older kernels, this would mean 0 line changes to backport a newer
> driver.
>
> Something like this maybe on the generic compat module, it builds for
> me, will commit soon.
>
> /*
> * Expand this as drivers require more ops, for now this
> * only sets the ones we need.
> */
> void netdev_attach_ops(struct net_device *dev,
> const struct net_device_ops *ops)
> {
> #define SET_NETDEVOP(_op) (_op ? _op : NULL)
> dev->open = SET_NETDEVOP(ops->ndo_open);
> dev->stop = SET_NETDEVOP(ops->ndo_stop);
> dev->hard_start_xmit = SET_NETDEVOP(ops->ndo_start_xmit);
> dev->set_multicast_list = SET_NETDEVOP(ops->ndo_set_multicast_list);
> dev->change_mtu = SET_NETDEVOP(ops->ndo_change_mtu);
> dev->set_mac_address = SET_NETDEVOP(ops->ndo_set_mac_address);
> dev->tx_timeout = SET_NETDEVOP(ops->ndo_tx_timeout);
> dev->get_stats = SET_NETDEVOP(ops->ndo_get_stats);
> #undef SET_NETDEVOP
> }
> EXPORT_SYMBOL(netdev_attach_ops);
>
> For newer kernels then this would just be:
>
> static inline void netdev_attach_ops(struct net_device *dev,
> const struct net_device_ops *ops)
> {
> dev->netdev_ops = ops;
> }
>
> Stephen, would the above be acceptable upstream on netdevice.h ? It
> would eliminate all needs from having to #ifdef network drivers when
> backporting. If so I can send a respective patch and spatch all the
> setters I think. An example of the nasty ifdef crap we have to do for
> the current backport of netdevop'able drivers is below.
>
No. supporting backporting is not part of the upstream kernel
mission. Honestly, we try for forward compatibility but intentionally
ignore carrying extra backport baggage.
^ permalink raw reply
* Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
From: Stephen Hemminger @ 2010-05-05 0:05 UTC (permalink / raw)
To: Pankaj Thakkar
Cc: linux-kernel, netdev, virtualization, pv-drivers, sbhatewara
In-Reply-To: <20100504230225.GP8323@vmware.com>
On Tue, 4 May 2010 16:02:25 -0700
Pankaj Thakkar <pthakkar@vmware.com> wrote:
> Device passthrough technology allows a guest to bypass the hypervisor and drive
> the underlying physical device. VMware has been exploring various ways to
> deliver this technology to users in a manner which is easy to adopt. In this
> process we have prepared an architecture along with Intel - NPA (Network Plugin
> Architecture). NPA allows the guest to use the virtualized NIC vmxnet3 to
> passthrough to a number of physical NICs which support it. The document below
> provides an overview of NPA.
>
> We intend to upgrade the upstreamed vmxnet3 driver to implement NPA so that
> Linux users can exploit the benefits provided by passthrough devices in a
> seamless manner while retaining the benefits of virtualization. The document
> below tries to answer most of the questions which we anticipated. Please let us
> know your comments and queries.
>
> Thank you.
>
> Signed-off-by: Pankaj Thakkar <pthakkar@vmware.com>
Code please. Also, it has to work for all architectures not just VMware and
Intel.
^ permalink raw reply
* Re: [PATCH] compat-wireless: updates for orinoco
From: Luis R. Rodriguez @ 2010-05-05 0:18 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Luis Rodriguez, Hauke Mehrtens, David Miller,
linux-wireless@vger.kernel.org, mcgrof@kernel.org,
netdev@vger.kernel.org
In-Reply-To: <20100504170409.46914a88@nehalam>
On Tue, May 04, 2010 at 05:04:09PM -0700, Stephen Hemminger wrote:
> On Tue, 4 May 2010 16:26:53 -0700
> "Luis R. Rodriguez" <lrodriguez@atheros.com> wrote:
>
> > First of all, thanks a lot! Some comments below.
> >
> > On Tue, May 4, 2010 at 3:40 PM, Hauke Mehrtens <hauke@hauke-m.de> wrote:
> > > * Make all the patches apply again.
> > > * rename read_pda to avoid conflicts with definitions in kernel <= 2.6.29
> >
> > I'm going to apply these two changes, if you get time can you send a
> > patch to rename read_pda upstream as well, that way we don't have to
> > carry this?
> >
> > > * add orinoco usb
> >
> > Thanks for this but I've grown tired of updating these netdev ops and
> > I think we can do better. I'll add a netdev_attach_ops() which would
> > simply do all the backport stuff for us, this way for backporting
> > purposes all we have to do is replace the old lines with a
> > netdev_attach_ops() call. In fact if we *really* wanted to we could
> > add a dummy netdev_attach_ops() upstream and just backport that on
> > older kernels, this would mean 0 line changes to backport a newer
> > driver.
> >
> > Something like this maybe on the generic compat module, it builds for
> > me, will commit soon.
> >
> > /*
> > * Expand this as drivers require more ops, for now this
> > * only sets the ones we need.
> > */
> > void netdev_attach_ops(struct net_device *dev,
> > const struct net_device_ops *ops)
> > {
> > #define SET_NETDEVOP(_op) (_op ? _op : NULL)
> > dev->open = SET_NETDEVOP(ops->ndo_open);
> > dev->stop = SET_NETDEVOP(ops->ndo_stop);
> > dev->hard_start_xmit = SET_NETDEVOP(ops->ndo_start_xmit);
> > dev->set_multicast_list = SET_NETDEVOP(ops->ndo_set_multicast_list);
> > dev->change_mtu = SET_NETDEVOP(ops->ndo_change_mtu);
> > dev->set_mac_address = SET_NETDEVOP(ops->ndo_set_mac_address);
> > dev->tx_timeout = SET_NETDEVOP(ops->ndo_tx_timeout);
> > dev->get_stats = SET_NETDEVOP(ops->ndo_get_stats);
> > #undef SET_NETDEVOP
> > }
> > EXPORT_SYMBOL(netdev_attach_ops);
> >
> > For newer kernels then this would just be:
> >
> > static inline void netdev_attach_ops(struct net_device *dev,
> > const struct net_device_ops *ops)
> > {
> > dev->netdev_ops = ops;
> > }
> >
> > Stephen, would the above be acceptable upstream on netdevice.h ? It
> > would eliminate all needs from having to #ifdef network drivers when
> > backporting. If so I can send a respective patch and spatch all the
> > setters I think. An example of the nasty ifdef crap we have to do for
> > the current backport of netdevop'able drivers is below.
> >
>
> No. supporting backporting is not part of the upstream kernel
> mission. Honestly, we try for forward compatibility but intentionally
> ignore carrying extra backport baggage.
Sure, understood, just had to try :), if only I could find a *good*
non-backport reason to have the netdev_attach_ops()...
Luis
^ permalink raw reply
* Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
From: Pankaj Thakkar @ 2010-05-05 0:18 UTC (permalink / raw)
To: Stephen Hemminger
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
virtualization@lists.linux-foundation.org, pv-drivers@vmware.com,
Shreyas Bhatewara
In-Reply-To: <20100504170531.1e7122da@nehalam>
The purpose of this email is to introduce the architecture and the design principles. The overall project involves more than just changes to vmxnet3 driver and hence we though an overview email would be better. Once people agree to the design in general we intend to provide the code changes to the vmxnet3 driver.
The architecture supports more than Intel NICs. We started the project with Intel but plan to support all major IHVs including Broadcom, Qlogic, Emulex and others through a certification program. The architecture works on VMware ESX server only as it requires significant support from the hypervisor. Also, the vmxnet3 driver works on VMware platform only. AFAICT Xen has a different model for supporting SR-IOV devices and allowing live migration and the document briefly talks about it (paragraph 6).
Thanks,
-pankaj
On Tue, May 04, 2010 at 05:05:31PM -0700, Stephen Hemminger wrote:
> Date: Tue, 4 May 2010 17:05:31 -0700
> From: Stephen Hemminger <shemminger@vyatta.com>
> To: Pankaj Thakkar <pthakkar@vmware.com>
> CC: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
> "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
> "virtualization@lists.linux-foundation.org"
> <virtualization@lists.linux-foundation.org>,
> "pv-drivers@vmware.com" <pv-drivers@vmware.com>,
> Shreyas Bhatewara <sbhatewara@vmware.com>
> Subject: Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
>
> On Tue, 4 May 2010 16:02:25 -0700
> Pankaj Thakkar <pthakkar@vmware.com> wrote:
>
> > Device passthrough technology allows a guest to bypass the hypervisor and drive
> > the underlying physical device. VMware has been exploring various ways to
> > deliver this technology to users in a manner which is easy to adopt. In this
> > process we have prepared an architecture along with Intel - NPA (Network Plugin
> > Architecture). NPA allows the guest to use the virtualized NIC vmxnet3 to
> > passthrough to a number of physical NICs which support it. The document below
> > provides an overview of NPA.
> >
> > We intend to upgrade the upstreamed vmxnet3 driver to implement NPA so that
> > Linux users can exploit the benefits provided by passthrough devices in a
> > seamless manner while retaining the benefits of virtualization. The document
> > below tries to answer most of the questions which we anticipated. Please let us
> > know your comments and queries.
> >
> > Thank you.
> >
> > Signed-off-by: Pankaj Thakkar <pthakkar@vmware.com>
>
>
> Code please. Also, it has to work for all architectures not just VMware and
> Intel.
^ permalink raw reply
* Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
From: David Miller @ 2010-05-05 0:32 UTC (permalink / raw)
To: pthakkar; +Cc: pv-drivers, netdev, linux-kernel, virtualization, shemminger
In-Reply-To: <20100505001857.GQ8323@vmware.com>
From: Pankaj Thakkar <pthakkar@vmware.com>
Date: Tue, 4 May 2010 17:18:57 -0700
> The purpose of this email is to introduce the architecture and the
> design principles. The overall project involves more than just
> changes to vmxnet3 driver and hence we though an overview email
> would be better. Once people agree to the design in general we
> intend to provide the code changes to the vmxnet3 driver.
Stephen's point is that code talks and bullshit walks.
Talk about high level designs rarely gets any traction, and often goes
nowhere. Give us an example implementation so there is something
concrete for us to sink our teeth into.
^ permalink raw reply
* [PATCH 0/6] netns support in the kobject layer
From: Eric W. Biederman @ 2010-05-05 0:35 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, Greg KH, linux-kernel, Tejun Heo, Cornelia Huck,
Eric Dumazet, Benjamin LaHaise, Serge Hallyn, netdev,
David Miller
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
With the tagged sysfs support finally merged into Greg's tree,
it is time for the last little bits of work to get the kobject
layer and network namespaces to play together properly.
These patches are roughly evenly divided between network layer work
and sysfs layer work. Last time this conundrum came up I believe
we decided that the easiest way to handle this was for Greg to carry
all of the patches. David, Greg does that still make sense?
This patchset adds:
- kobject layer support for sending events in all network namespaces
- netlink support for filtering broadcast packets based on attributes
of the destination socket.
- Enabling the network namespace support for sysfs and the kobject layer.
include/linux/kobject.h | 1 +
include/linux/netlink.h | 4 ++
lib/kobject_uevent.c | 108 +++++++++++++++++++++++++++++++++++++++++-----
net/Kconfig | 8 +++
net/core/dev.c | 28 ++----------
net/core/net-sysfs.c | 62 ++++++++++++++++++++------
net/core/net-sysfs.h | 1 -
net/netlink/af_netlink.c | 21 ++++++++-
8 files changed, 181 insertions(+), 52 deletions(-)
^ permalink raw reply
* [PATCH 1/6] kobject: Send hotplug events in all network namespaces
From: Eric W. Biederman @ 2010-05-05 0:36 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, linux-kernel, Tejun Heo, Cornelia Huck, Eric Dumazet,
Benjamin LaHaise, Serge Hallyn, netdev, David Miller,
Eric W. Biederman
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
From: Eric W. Biederman <ebiederm@xmission.com>
Open a copy of the uevent kernel socket in each network
namespace so we can send uevents in all network namespaces.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
lib/kobject_uevent.c | 68 ++++++++++++++++++++++++++++++++++++++++++++------
1 files changed, 60 insertions(+), 8 deletions(-)
diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c
index c9d3a3e..3f5f17b 100644
--- a/lib/kobject_uevent.c
+++ b/lib/kobject_uevent.c
@@ -23,13 +23,19 @@
#include <linux/skbuff.h>
#include <linux/netlink.h>
#include <net/sock.h>
+#include <net/net_namespace.h>
u64 uevent_seqnum;
char uevent_helper[UEVENT_HELPER_PATH_LEN] = CONFIG_UEVENT_HELPER_PATH;
static DEFINE_SPINLOCK(sequence_lock);
-#if defined(CONFIG_NET)
-static struct sock *uevent_sock;
+#ifdef CONFIG_NET
+struct uevent_sock {
+ struct list_head list;
+ struct sock *sk;
+};
+static LIST_HEAD(uevent_sock_list);
+static DEFINE_MUTEX(uevent_sock_mutex);
#endif
/* the strings here must match the enum in include/linux/kobject.h */
@@ -99,6 +105,9 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
u64 seq;
int i = 0;
int retval = 0;
+#ifdef CONFIG_NET
+ struct uevent_sock *ue_sk;
+#endif
pr_debug("kobject: '%s' (%p): %s\n",
kobject_name(kobj), kobj, __func__);
@@ -210,7 +219,9 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
#if defined(CONFIG_NET)
/* send netlink message */
- if (uevent_sock) {
+ mutex_lock(&uevent_sock_mutex);
+ list_for_each_entry(ue_sk, &uevent_sock_list, list) {
+ struct sock *uevent_sock = ue_sk->sk;
struct sk_buff *skb;
size_t len;
@@ -240,6 +251,7 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
} else
retval = -ENOMEM;
}
+ mutex_unlock(&uevent_sock_mutex);
#endif
/* call uevent_helper, usually only enabled during early boot */
@@ -319,18 +331,58 @@ int add_uevent_var(struct kobj_uevent_env *env, const char *format, ...)
EXPORT_SYMBOL_GPL(add_uevent_var);
#if defined(CONFIG_NET)
-static int __init kobject_uevent_init(void)
+static int uevent_net_init(struct net *net)
{
- uevent_sock = netlink_kernel_create(&init_net, NETLINK_KOBJECT_UEVENT,
- 1, NULL, NULL, THIS_MODULE);
- if (!uevent_sock) {
+ struct uevent_sock *ue_sk;
+
+ ue_sk = kzalloc(sizeof(*ue_sk), GFP_KERNEL);
+ if (!ue_sk)
+ return -ENOMEM;
+
+ ue_sk->sk = netlink_kernel_create(net, NETLINK_KOBJECT_UEVENT,
+ 1, NULL, NULL, THIS_MODULE);
+ if (!ue_sk->sk) {
printk(KERN_ERR
"kobject_uevent: unable to create netlink socket!\n");
return -ENODEV;
}
- netlink_set_nonroot(NETLINK_KOBJECT_UEVENT, NL_NONROOT_RECV);
+ mutex_lock(&uevent_sock_mutex);
+ list_add_tail(&ue_sk->list, &uevent_sock_list);
+ mutex_unlock(&uevent_sock_mutex);
return 0;
}
+static void uevent_net_exit(struct net *net)
+{
+ struct uevent_sock *ue_sk;
+
+ mutex_lock(&uevent_sock_mutex);
+ list_for_each_entry(ue_sk, &uevent_sock_list, list) {
+ if (sock_net(ue_sk->sk) == net)
+ goto found;
+ }
+ mutex_unlock(&uevent_sock_mutex);
+ return;
+
+found:
+ list_del(&ue_sk->list);
+ mutex_unlock(&uevent_sock_mutex);
+
+ netlink_kernel_release(ue_sk->sk);
+ kfree(ue_sk);
+}
+
+static struct pernet_operations uevent_net_ops = {
+ .init = uevent_net_init,
+ .exit = uevent_net_exit,
+};
+
+static int __init kobject_uevent_init(void)
+{
+ netlink_set_nonroot(NETLINK_KOBJECT_UEVENT, NL_NONROOT_RECV);
+ return register_pernet_subsys(&uevent_net_ops);
+}
+
+
postcore_initcall(kobject_uevent_init);
#endif
--
1.6.5.2.143.g8cc62
^ permalink raw reply related
* [PATCH 2/6] netns: Teach network device kobjects which namespace they are in.
From: Eric W. Biederman @ 2010-05-05 0:36 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, linux-kernel, Tejun Heo, Cornelia Huck, Eric Dumazet,
Benjamin LaHaise, Serge Hallyn, netdev, David Miller,
Eric W. Biederman
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
From: Eric W. Biederman <ebiederm@xmission.com>
The problem. Network devices show up in sysfs and with the network
namespace active multiple devices with the same name can show up in
the same directory, ouch!
To avoid that problem and allow existing applications in network namespaces
to see the same interface that is currently presented in sysfs, this
patch enables the tagging directory support in sysfs.
By using the network namespace pointers as tags to separate out the
the sysfs directory entries we ensure that we don't have conflicts
in the directories and applications only see a limited set of
the network devices.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
include/linux/kobject.h | 1 +
net/Kconfig | 8 ++++++++
net/core/net-sysfs.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 55 insertions(+), 0 deletions(-)
diff --git a/include/linux/kobject.h b/include/linux/kobject.h
index b60d2df..cf343a8 100644
--- a/include/linux/kobject.h
+++ b/include/linux/kobject.h
@@ -142,6 +142,7 @@ extern const struct sysfs_ops kobj_sysfs_ops;
*/
enum kobj_ns_type {
KOBJ_NS_TYPE_NONE = 0,
+ KOBJ_NS_TYPE_NET,
KOBJ_NS_TYPES
};
diff --git a/net/Kconfig b/net/Kconfig
index 041c35e..265e33b 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -45,6 +45,14 @@ config COMPAT_NETLINK_MESSAGES
menu "Networking options"
+config NET_NS
+ bool "Network namespace support"
+ default n
+ depends on EXPERIMENTAL && NAMESPACES
+ help
+ Allow user space to create what appear to be multiple instances
+ of the network stack.
+
source "net/packet/Kconfig"
source "net/unix/Kconfig"
source "net/xfrm/Kconfig"
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 099c753..1b98e36 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -13,7 +13,9 @@
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/if_arp.h>
+#include <linux/nsproxy.h>
#include <net/sock.h>
+#include <net/net_namespace.h>
#include <linux/rtnetlink.h>
#include <linux/wireless.h>
#include <net/wext.h>
@@ -466,6 +468,37 @@ static struct attribute_group wireless_group = {
};
#endif
+static const void *net_current_ns(void)
+{
+ return current->nsproxy->net_ns;
+}
+
+static const void *net_initial_ns(void)
+{
+ return &init_net;
+}
+
+static const void *net_netlink_ns(struct sock *sk)
+{
+ return sock_net(sk);
+}
+
+static struct kobj_ns_type_operations net_ns_type_operations = {
+ .type = KOBJ_NS_TYPE_NET,
+ .current_ns = net_current_ns,
+ .netlink_ns = net_netlink_ns,
+ .initial_ns = net_initial_ns,
+};
+
+static void net_kobj_ns_exit(struct net *net)
+{
+ kobj_ns_exit(KOBJ_NS_TYPE_NET, net);
+}
+
+static struct pernet_operations sysfs_net_ops = {
+ .exit = net_kobj_ns_exit,
+};
+
#endif /* CONFIG_SYSFS */
#ifdef CONFIG_HOTPLUG
@@ -506,6 +539,13 @@ static void netdev_release(struct device *d)
kfree((char *)dev - dev->padded);
}
+static const void *net_namespace(struct device *d)
+{
+ struct net_device *dev;
+ dev = container_of(d, struct net_device, dev);
+ return dev_net(dev);
+}
+
static struct class net_class = {
.name = "net",
.dev_release = netdev_release,
@@ -515,6 +555,8 @@ static struct class net_class = {
#ifdef CONFIG_HOTPLUG
.dev_uevent = netdev_uevent,
#endif
+ .ns_type = &net_ns_type_operations,
+ .namespace = net_namespace,
};
/* Delete sysfs entries but hold kobject reference until after all
@@ -587,5 +629,9 @@ void netdev_initialize_kobject(struct net_device *net)
int netdev_kobject_init(void)
{
+ kobj_ns_type_register(&net_ns_type_operations);
+#ifdef CONFIG_SYSFS
+ register_pernet_subsys(&sysfs_net_ops);
+#endif
return class_register(&net_class);
}
--
1.6.5.2.143.g8cc62
^ permalink raw reply related
* [PATCH 4/6] kobj: Send hotplug events in the proper namespace.
From: Eric W. Biederman @ 2010-05-05 0:36 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, linux-kernel, Tejun Heo, Cornelia Huck, Eric Dumazet,
Benjamin LaHaise, Serge Hallyn, netdev, David Miller,
Eric W. Biederman
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
From: Eric W. Biederman <ebiederm@xmission.com>
Utilize netlink_broacast_filtered to allow sending hotplug events
in the proper namespace.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
lib/kobject_uevent.c | 22 ++++++++++++++++++++--
1 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c
index 3f5f17b..9057ec1 100644
--- a/lib/kobject_uevent.c
+++ b/lib/kobject_uevent.c
@@ -82,6 +82,22 @@ out:
return ret;
}
+static int kobj_bcast_filter(struct sock *dsk, struct sk_buff *skb, void *data)
+{
+ struct kobject *kobj = data;
+ const struct kobj_ns_type_operations *ops;
+
+ ops = kobj_ns_ops(kobj);
+ if (ops) {
+ const void *sock_ns, *ns;
+ ns = kobj->ktype->namespace(kobj);
+ sock_ns = ops->netlink_ns(dsk);
+ return sock_ns != ns;
+ }
+
+ return 0;
+}
+
/**
* kobject_uevent_env - send an uevent with environmental data
*
@@ -243,8 +259,10 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
}
NETLINK_CB(skb).dst_group = 1;
- retval = netlink_broadcast(uevent_sock, skb, 0, 1,
- GFP_KERNEL);
+ retval = netlink_broadcast_filtered(uevent_sock, skb,
+ 0, 1, GFP_KERNEL,
+ kobj_bcast_filter,
+ kobj);
/* ENOBUFS should be handled in userspace */
if (retval == -ENOBUFS)
retval = 0;
--
1.6.5.2.143.g8cc62
^ permalink raw reply related
* [PATCH 3/6] netlink: Implment netlink_broadcast_filtered
From: Eric W. Biederman @ 2010-05-05 0:36 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, linux-kernel, Tejun Heo, Cornelia Huck, Eric Dumazet,
Benjamin LaHaise, Serge Hallyn, netdev, David Miller,
Eric W. Biederman
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
From: Eric W. Biederman <ebiederm@xmission.com>
When netlink sockets are used to convey data that is in a namespace
we need a way to select a subset of the listening sockets to deliver
the packet to. For the network namespace we have been doing this
by only transmitting packets in the correct network namespace.
For data belonging to other namespaces netlink_bradcast_filtered
provides a mechanism that allows us to examine the destination
socket and to decide if we should transmit the specified packet
to it.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
include/linux/netlink.h | 4 ++++
net/netlink/af_netlink.c | 21 +++++++++++++++++++--
2 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/include/linux/netlink.h b/include/linux/netlink.h
index fde27c0..4f7bf4b 100644
--- a/include/linux/netlink.h
+++ b/include/linux/netlink.h
@@ -188,6 +188,10 @@ extern int netlink_has_listeners(struct sock *sk, unsigned int group);
extern int netlink_unicast(struct sock *ssk, struct sk_buff *skb, __u32 pid, int nonblock);
extern int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, __u32 pid,
__u32 group, gfp_t allocation);
+extern int netlink_broadcast_filtered(struct sock *ssk, struct sk_buff *skb,
+ __u32 pid, __u32 group, gfp_t allocation,
+ int (*filter)(struct sock *dsk, struct sk_buff *skb, void *data),
+ void *filter_data);
extern void netlink_set_err(struct sock *ssk, __u32 pid, __u32 group, int code);
extern int netlink_register_notifier(struct notifier_block *nb);
extern int netlink_unregister_notifier(struct notifier_block *nb);
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 320d042..4f16d68 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -975,6 +975,8 @@ struct netlink_broadcast_data {
int delivered;
gfp_t allocation;
struct sk_buff *skb, *skb2;
+ int (*tx_filter)(struct sock *dsk, struct sk_buff *skb, void *data);
+ void *tx_data;
};
static inline int do_one_broadcast(struct sock *sk,
@@ -1017,6 +1019,9 @@ static inline int do_one_broadcast(struct sock *sk,
p->failure = 1;
if (nlk->flags & NETLINK_BROADCAST_SEND_ERROR)
p->delivery_failure = 1;
+ } else if (p->tx_filter && p->tx_filter(sk, p->skb2, p->tx_data)) {
+ kfree_skb(p->skb2);
+ p->skb2 = NULL;
} else if (sk_filter(sk, p->skb2)) {
kfree_skb(p->skb2);
p->skb2 = NULL;
@@ -1035,8 +1040,10 @@ out:
return 0;
}
-int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, u32 pid,
- u32 group, gfp_t allocation)
+int netlink_broadcast_filtered(struct sock *ssk, struct sk_buff *skb, u32 pid,
+ u32 group, gfp_t allocation,
+ int (*filter)(struct sock *dsk, struct sk_buff *skb, void *data),
+ void *filter_data)
{
struct net *net = sock_net(ssk);
struct netlink_broadcast_data info;
@@ -1056,6 +1063,8 @@ int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, u32 pid,
info.allocation = allocation;
info.skb = skb;
info.skb2 = NULL;
+ info.tx_filter = filter;
+ info.tx_data = filter_data;
/* While we sleep in clone, do not allow to change socket list */
@@ -1080,6 +1089,14 @@ int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, u32 pid,
}
return -ESRCH;
}
+EXPORT_SYMBOL(netlink_broadcast_filtered);
+
+int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, u32 pid,
+ u32 group, gfp_t allocation)
+{
+ return netlink_broadcast_filtered(ssk, skb, pid, group, allocation,
+ NULL, NULL);
+}
EXPORT_SYMBOL(netlink_broadcast);
struct netlink_set_err_data {
--
1.6.5.2.143.g8cc62
^ permalink raw reply related
* [PATCH 6/6] net: Expose all network devices in a namespaces in sysfs
From: Eric W. Biederman @ 2010-05-05 0:36 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, linux-kernel, Tejun Heo, Cornelia Huck, Eric Dumazet,
Benjamin LaHaise, Serge Hallyn, netdev, David Miller,
Eric W. Biederman
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
From: Eric W. Biederman <ebiederm@xmission.com>
This reverts commit aaf8cdc34ddba08122f02217d9d684e2f9f5d575.
Drivers like the ipw2100 call device_create_group when they
are initialized and device_remove_group when they are shutdown.
Moving them between namespaces deletes their sysfs groups early.
In particular the following call chain results.
netdev_unregister_kobject -> device_del -> kobject_del -> sysfs_remove_dir
With sysfs_remove_dir recursively deleting all of it's subdirectories,
and nothing adding them back.
Ouch!
Therefore we need to call something that ultimate calls sysfs_mv_dir
as that sysfs function can move sysfs directories between namespaces
without deleting their subdirectories or their contents. Allowing
us to avoid placing extra boiler plate into every driver that does
something interesting with sysfs.
Currently the function that provides that capability is device_rename.
That is the code works without nasty side effects as originally written.
So remove the misguided fix for moving devices between namespaces. The
bug in the kobject layer that inspired it has now been recognized and
fixed.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
net/core/dev.c | 28 +++++-----------------------
net/core/net-sysfs.c | 16 +---------------
net/core/net-sysfs.h | 1 -
3 files changed, 6 insertions(+), 39 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index bcc490c..fa54819 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -983,15 +983,10 @@ int dev_change_name(struct net_device *dev, const char *newname)
return err;
rollback:
- /* For now only devices in the initial network namespace
- * are in sysfs.
- */
- if (net_eq(net, &init_net)) {
- ret = device_rename(&dev->dev, dev->name);
- if (ret) {
- memcpy(dev->name, oldname, IFNAMSIZ);
- return ret;
- }
+ ret = device_rename(&dev->dev, dev->name);
+ if (ret) {
+ memcpy(dev->name, oldname, IFNAMSIZ);
+ return ret;
}
write_lock_bh(&dev_base_lock);
@@ -5106,8 +5101,6 @@ int register_netdevice(struct net_device *dev)
if (dev->features & NETIF_F_SG)
dev->features |= NETIF_F_GSO;
- netdev_initialize_kobject(dev);
-
ret = call_netdevice_notifiers(NETDEV_POST_INIT, dev);
ret = notifier_to_errno(ret);
if (ret)
@@ -5628,15 +5621,6 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const char
if (dev->features & NETIF_F_NETNS_LOCAL)
goto out;
-#ifdef CONFIG_SYSFS
- /* Don't allow real devices to be moved when sysfs
- * is enabled.
- */
- err = -EINVAL;
- if (dev->dev.parent)
- goto out;
-#endif
-
/* Ensure the device has been registrered */
err = -EINVAL;
if (dev->reg_state != NETREG_REGISTERED)
@@ -5687,8 +5671,6 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const char
dev_unicast_flush(dev);
dev_addr_discard(dev);
- netdev_unregister_kobject(dev);
-
/* Actually switch the network namespace */
dev_net_set(dev, net);
@@ -5701,7 +5683,7 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const char
}
/* Fixup kobjects */
- err = netdev_register_kobject(dev);
+ err = device_rename(&dev->dev, dev->name);
WARN_ON(err);
/* Add the device back in the hashes */
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 1b98e36..0727c57 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -507,9 +507,6 @@ static int netdev_uevent(struct device *d, struct kobj_uevent_env *env)
struct net_device *dev = to_net_dev(d);
int retval;
- if (!net_eq(dev_net(dev), &init_net))
- return 0;
-
/* pass interface to uevent. */
retval = add_uevent_var(env, "INTERFACE=%s", dev->name);
if (retval)
@@ -568,9 +565,6 @@ void netdev_unregister_kobject(struct net_device * net)
kobject_get(&dev->kobj);
- if (!net_eq(dev_net(net), &init_net))
- return;
-
device_del(dev);
}
@@ -580,6 +574,7 @@ int netdev_register_kobject(struct net_device *net)
struct device *dev = &(net->dev);
const struct attribute_group **groups = net->sysfs_groups;
+ device_initialize(dev);
dev->class = &net_class;
dev->platform_data = net;
dev->groups = groups;
@@ -602,9 +597,6 @@ int netdev_register_kobject(struct net_device *net)
#endif
#endif /* CONFIG_SYSFS */
- if (!net_eq(dev_net(net), &init_net))
- return 0;
-
return device_add(dev);
}
@@ -621,12 +613,6 @@ void netdev_class_remove_file(struct class_attribute *class_attr)
EXPORT_SYMBOL(netdev_class_create_file);
EXPORT_SYMBOL(netdev_class_remove_file);
-void netdev_initialize_kobject(struct net_device *net)
-{
- struct device *device = &(net->dev);
- device_initialize(device);
-}
-
int netdev_kobject_init(void)
{
kobj_ns_type_register(&net_ns_type_operations);
diff --git a/net/core/net-sysfs.h b/net/core/net-sysfs.h
index 14e7524..805555e 100644
--- a/net/core/net-sysfs.h
+++ b/net/core/net-sysfs.h
@@ -4,5 +4,4 @@
int netdev_kobject_init(void);
int netdev_register_kobject(struct net_device *);
void netdev_unregister_kobject(struct net_device *);
-void netdev_initialize_kobject(struct net_device *);
#endif
--
1.6.5.2.143.g8cc62
^ permalink raw reply related
* [PATCH 5/6] hotplug: netns aware uevent_helper
From: Eric W. Biederman @ 2010-05-05 0:36 UTC (permalink / raw)
To: Greg Kroah-Hartman
Cc: Kay Sievers, linux-kernel, Tejun Heo, Cornelia Huck, Eric Dumazet,
Benjamin LaHaise, Serge Hallyn, netdev, David Miller,
Eric W. Biederman
In-Reply-To: <m1fx3hk6gw.fsf@fess.ebiederm.org>
From: Eric W. Biederman <ebiederm@xmission.com>
It only makes sense for uevent_helper to get events
in the intial namespaces. It's invocation is not
per namespace and it is not clear how we could make
it's invocation namespace aware.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
lib/kobject_uevent.c | 18 +++++++++++++++++-
1 files changed, 17 insertions(+), 1 deletions(-)
diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c
index 9057ec1..1b3dbab 100644
--- a/lib/kobject_uevent.c
+++ b/lib/kobject_uevent.c
@@ -18,6 +18,7 @@
#include <linux/string.h>
#include <linux/kobject.h>
#include <linux/module.h>
+#include <linux/user_namespace.h>
#include <linux/socket.h>
#include <linux/skbuff.h>
@@ -98,6 +99,21 @@ static int kobj_bcast_filter(struct sock *dsk, struct sk_buff *skb, void *data)
return 0;
}
+static int kobj_usermode_filter(struct kobject *kobj)
+{
+ const struct kobj_ns_type_operations *ops;
+
+ ops = kobj_ns_ops(kobj);
+ if (ops) {
+ const void *init_ns, *ns;
+ ns = kobj->ktype->namespace(kobj);
+ init_ns = ops->initial_ns();
+ return ns != init_ns;
+ }
+
+ return 0;
+}
+
/**
* kobject_uevent_env - send an uevent with environmental data
*
@@ -273,7 +289,7 @@ int kobject_uevent_env(struct kobject *kobj, enum kobject_action action,
#endif
/* call uevent_helper, usually only enabled during early boot */
- if (uevent_helper[0]) {
+ if (uevent_helper[0] && !kobj_usermode_filter(kobj)) {
char *argv [3];
argv [0] = uevent_helper;
--
1.6.5.2.143.g8cc62
^ permalink raw reply related
* Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
From: Pankaj Thakkar @ 2010-05-05 0:38 UTC (permalink / raw)
To: David Miller
Cc: shemminger@vyatta.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org, virtualization@lists.linux-foundation.org,
pv-drivers@vmware.com, Shreyas Bhatewara
In-Reply-To: <20100504.173236.104064409.davem@davemloft.net>
Sure. We have been working on NPA for a while and have the code internally up
and running. Let me sync up internally on how and when we can provide the
vmxnet3 driver code so that people can look at it.
On Tue, May 04, 2010 at 05:32:36PM -0700, David Miller wrote:
> Date: Tue, 4 May 2010 17:32:36 -0700
> From: David Miller <davem@davemloft.net>
> To: Pankaj Thakkar <pthakkar@vmware.com>
> CC: "shemminger@vyatta.com" <shemminger@vyatta.com>,
> "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
> "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
> "virtualization@lists.linux-foundation.org"
> <virtualization@lists.linux-foundation.org>,
> "pv-drivers@vmware.com" <pv-drivers@vmware.com>,
> Shreyas Bhatewara <sbhatewara@vmware.com>
> Subject: Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
>
> From: Pankaj Thakkar <pthakkar@vmware.com>
> Date: Tue, 4 May 2010 17:18:57 -0700
>
> > The purpose of this email is to introduce the architecture and the
> > design principles. The overall project involves more than just
> > changes to vmxnet3 driver and hence we though an overview email
> > would be better. Once people agree to the design in general we
> > intend to provide the code changes to the vmxnet3 driver.
>
> Stephen's point is that code talks and bullshit walks.
>
> Talk about high level designs rarely gets any traction, and often goes
> nowhere. Give us an example implementation so there is something
> concrete for us to sink our teeth into.
^ permalink raw reply
* Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
From: Chris Wright @ 2010-05-05 0:58 UTC (permalink / raw)
To: Pankaj Thakkar; +Cc: kvm, pv-drivers, netdev, linux-kernel, virtualization
In-Reply-To: <20100504230225.GP8323@vmware.com>
* Pankaj Thakkar (pthakkar@vmware.com) wrote:
> We intend to upgrade the upstreamed vmxnet3 driver to implement NPA so that
> Linux users can exploit the benefits provided by passthrough devices in a
> seamless manner while retaining the benefits of virtualization. The document
> below tries to answer most of the questions which we anticipated. Please let us
> know your comments and queries.
How does the throughput, latency, and host CPU utilization for normal
data path compare with say NetQueue?
And does this obsolete your UPT implementation?
> Network Plugin Architecture
> ---------------------------
>
> VMware has been working on various device passthrough technologies for the past
> few years. Passthrough technology is interesting as it can result in better
> performance/cpu utilization for certain demanding applications. In our vSphere
> product we support direct assignment of PCI devices like networking adapters to
> a guest virtual machine. This allows the guest to drive the device using the
> device drivers installed inside the guest. This is similar to the way KVM
> allows for passthrough of PCI devices to the guests. The hypervisor is bypassed
> for all I/O and control operations and hence it can not provide any value add
> features such as live migration, suspend/resume, etc.
>
>
> Network Plugin Architecture (NPA) is an approach which VMware has developed in
> joint partnership with Intel which allows us to retain the best of passthrough
> technology and virtualization. NPA allows for passthrough of the fast data
> (I/O) path and lets the hypervisor deal with the slow control path using
> traditional emulation/paravirtualization techniques. Through this splitting of
> data and control path the hypervisor can still provide the above mentioned
> value add features and exploit the performance benefits of passthrough.
How many cards actually support this NPA interface? What does it look
like, i.e. where is the NPA specification? (AFAIK, we never got the UPT
one).
> NPA requires SR-IOV hardware which allows for sharing of one single NIC adapter
> by multiple guests. SR-IOV hardware has many logically separate functions
> called virtual functions (VF) which can be independently assigned to the guest
> OS. They also have one or more physical functions (PF) (managed by a PF driver)
> which are used by the hypervisor to control certain aspects of the VFs and the
> rest of the hardware.
How do you handle hardware which has a more symmetric view of the
SR-IOV world (SR-IOV is only PCI sepcification, not a network driver
specification)? Or hardware which has multiple functions per physical
port (multiqueue, hw filtering, embedded switch, etc.)?
> NPA splits the guest driver into two components called
> the Shell and the Plugin. The shell is responsible for interacting with the
> guest networking stack and funneling the control operations to the hypervisor.
> The plugin is responsible for driving the data path of the virtual function
> exposed to the guest and is specific to the NIC hardware. NPA also requires an
> embedded switch in the NIC to allow for switching traffic among the virtual
> functions. The PF is also used as an uplink to provide connectivity to other
> VMs which are in emulation mode. The figure below shows the major components in
> a block diagram.
>
> +------------------------------+
> | Guest VM |
> | |
> | +----------------+ |
> | | vmxnet3 driver | |
> | | Shell | |
> | | +============+ | |
> | | | Plugin | | |
> +------+-+------------+-+------+
> | .
> +---------+ .
> | vmxnet3 | .
> |___+-----+ .
> | .
> | .
> +----------------------------+
> | |
> | virtual switch |
> +----------------------------+
> | . \
> | . \
> +=============+ . \
> | PF control | . \
> | | . \
> | L2 driver | . \
> +-------------+ . \
> | . \
> | . \
> +------------------------+ +------------+
> | PF VF1 VF2 ... VFn | | |
> | | | regular |
> | SR-IOV NIC | | nic |
> | +--------------+ | | +--------+
> | | embedded | | +---+
> | | switch | |
> | +--------------+ |
> | +---------------+
> +--------+
>
> NPA offers several benefits:
> 1. Performance: Critical performance sensitive paths are not trapped and the
> guest can directly drive the hardware without incurring virtualization
> overheads.
Can you demonstrate with data?
> 2. Hypervisor control: All control operations from the guest such as programming
> MAC address go through the hypervisor layer and hence can be subjected to
> hypervisor policies. The PF driver can be further used to put policy decisions
> like which VLAN the guest should be on.
This can happen without NPA as well. VF simply needs to request
the change via the PF (in fact, hw does that right now). Also, we
already have a host side management interface via PF (see, for example,
RTM_SETLINK IFLA_VF_MAC interface).
What is control plane interface? Just something like a fixed register set?
> 3. Guest Management: No hardware specific drivers need to be installed in the
> guest virtual machine and hence no overheads are incurred for guest management.
> All software for the driver (including the PF driver and the plugin) is
> installed in the hypervisor.
So we have a plugin per hardware VF implementation? And the hypervisor
injects this code into the guest?
> 4. IHV independence: The architecture provides guidelines for splitting the
> functionality between the VFs and PF but does not dictate how the hardware
> should be implemented. It gives the IHV the freedom to do asynchronous updates
> either to the software or the hardware to work around any defects.
Yes, this is important, esp. instead of the requirement for hw to
implement a specific interface (I suspect you know all about this issue
already).
> The fundamental tenet in NPA is to let the hypervisor control the passthrough
> functionality with minimal guest intervention. This gives a lot of flexibility
> to the hypervisor which can then treat passthrough as an offload feature (just
> like TSO, LRO, etc) which is offered to the guest virtual machine when there
> are no conflicting features present. For example, if the hypervisor wants to
> migrate the virtual machine from one host to another, the hypervisor can switch
> the virtual machine out of passthrough mode into paravirtualized/emulated mode
> and it can use existing technique to migrate the virtual machine. Once the
> virtual machine is migrated to the destination host the hypervisor can switch
> the virtual machine back to passthrough mode if a supporting SR-IOV nic is
> present. This may involve reloading of a different plugin corresponding to the
> new SR-IOV hardware.
>
> Internally we have explored various other options before settling on the NPA
> approach. For example there are approaches which create a bonding driver on top
> of a complete passthrough of a NIC device and an emulated/paravirtualized
> device. Though this approach allows for live migration to work it adds a lot of
> complexity and dependency. First the hypervisor has to rely on a guest with
> hot-add support. Second the hypervisor has to depend on the guest networking
> stack to cooperate to perform migration. Third the guest has to carry the
> driver images for all possible hardware to which the guest may migrate to.
> Fourth the hypervisor does not get full control for all the policy decisions.
> Another approach we have considered is to have a uniform interface for the data
> path between the emulated/paravirtualized device and the hardware device which
> allows the hypervisor to seamlessly switch from the emulated interface to the
> hardware interface. Though this approach is very attractive and can work
> without any guest involvement it is not acceptable to the IHVs as it does not
> give them the freedom to fix bugs/erratas and differentiate from each other. We
> believe NPA approach provides the right level of control and flexibility to the
> hypervisors while letting the guest exploit the benefits of passthrough.
> The plugin image is provided by the IHVs along with the PF driver and is
> packaged in the hypervisor. The plugin image is OS agnostic and can be loaded
> either into a Linux VM or a Windows VM. The plugin is written against the Shell
And it will need to be GPL AFAICT from what you've said thus far. It
does sound worrisome, although I suppose hw firmware isn't particularly
different.
> API interface which the shell is responsible for implementing. The API
> interface allows the plugin to do TX and RX only by programming the hardware
> rings (along with things like buffer allocation and basic initialization). The
> virtual machine comes up in paravirtualized/emulated mode when it is booted.
> The hypervisor allocates the VF and other resources and notifies the shell of
> the availability of the VF. The hypervisor injects the plugin into memory
> location specified by the shell. The shell initializes the plugin by calling
> into a known entry point and the plugin initializes the data path. The control
> path is already initialized by the PF driver when the VF is allocated. At this
> point the shell switches to using the loaded plugin to do all further TX and RX
> operations. The guest networking stack does not participate in these operations
> and continues to function normally. All the control operations continue being
> trapped by the hypervisor and are directed to the PF driver as needed. For
> example, if the MAC address changes the hypervisor updates its internal state
> and changes the state of the embedded switch as well through the PF control
> API.
How does the shell switch back to emulated mode for live migration?
> We have reworked our existing Linux vmxnet3 driver to accomodate NPA by
> splitting the driver into two parts: Shell and Plugin. The new split driver is
> backwards compatible and continues to work on old/existing vmxnet3 device
> emulations. The shell implements the API interface and contains code to do the
> bookkeeping for TX/RX buffers along with interrupt management. The shell code
> also handles the loading of the plugin and verifying the license of the loaded
> plugin. The plugin contains the code specific to vmxnet3 ring and descriptor
> management. The plugin uses the same Shell API interface which would be used by
> other IHVs. This vmxnet3 plugin is compiled statically along with the shell as
> this is needed to provide connectivity when there is no underlying SR-IOV
> device present. The IHV plugins are required to be distributed under GPL
> license and we are currently looking at ways to verify this both within the
> hypervisor and within the shell.
Please make this shell API interface and the PF/VF requirments available.
thanks,
-chris
^ permalink raw reply
* Re: [RFC] net: change bridge/macvlan hook to be be generic
From: Scott Feldman @ 2010-05-05 0:58 UTC (permalink / raw)
To: Stephen Hemminger, David Miller, Patrick McHardy; +Cc: netdev
In-Reply-To: <20100504153758.0ed3a87d@nehalam>
On 5/4/10 3:37 PM, "Stephen Hemminger" <shemminger@vyatta.com> wrote:
> The existing macvlan and bridge have special hooks in the packet input
> path. This patch changes it to a generic hook chain, like the packet type
> processing. I have been wanting to look into flow based switching, etc...
Can this be further simplified by saying that a netdev can only be hooked by
one mux (macvlan, bridge, etc) at any given time, so there is never more
than one element in the hook chain. If so, then we just need a single hook,
not a chain.
It seems odd to me that a dev would have both macvlan_port != NULL and
br_port != NULL. Can dev be in a macvlan and a bridge at the same time?
-scott
^ permalink raw reply
* Re: [PATCH] compat-wireless: updates for orinoco
From: Luis R. Rodriguez @ 2010-05-05 1:47 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Luis Rodriguez, Hauke Mehrtens, David Miller,
linux-wireless@vger.kernel.org, mcgrof@kernel.org,
netdev@vger.kernel.org
In-Reply-To: <20100505001830.GO2624@tux>
On Tue, May 4, 2010 at 5:18 PM, Luis R. Rodriguez
<lrodriguez@atheros.com> wrote:
> On Tue, May 04, 2010 at 05:04:09PM -0700, Stephen Hemminger wrote:
>> On Tue, 4 May 2010 16:26:53 -0700
>> "Luis R. Rodriguez" <lrodriguez@atheros.com> wrote:
>>
>> > First of all, thanks a lot! Some comments below.
>> >
>> > On Tue, May 4, 2010 at 3:40 PM, Hauke Mehrtens <hauke@hauke-m.de> wrote:
>> > > * Make all the patches apply again.
>> > > * rename read_pda to avoid conflicts with definitions in kernel <= 2.6.29
>> >
>> > I'm going to apply these two changes, if you get time can you send a
>> > patch to rename read_pda upstream as well, that way we don't have to
>> > carry this?
>> >
>> > > * add orinoco usb
>> >
>> > Thanks for this but I've grown tired of updating these netdev ops and
>> > I think we can do better. I'll add a netdev_attach_ops() which would
>> > simply do all the backport stuff for us, this way for backporting
>> > purposes all we have to do is replace the old lines with a
>> > netdev_attach_ops() call. In fact if we *really* wanted to we could
>> > add a dummy netdev_attach_ops() upstream and just backport that on
>> > older kernels, this would mean 0 line changes to backport a newer
>> > driver.
>> >
>> > Something like this maybe on the generic compat module, it builds for
>> > me, will commit soon.
>> >
>> > /*
>> > * Expand this as drivers require more ops, for now this
>> > * only sets the ones we need.
>> > */
>> > void netdev_attach_ops(struct net_device *dev,
>> > const struct net_device_ops *ops)
>> > {
>> > #define SET_NETDEVOP(_op) (_op ? _op : NULL)
>> > dev->open = SET_NETDEVOP(ops->ndo_open);
>> > dev->stop = SET_NETDEVOP(ops->ndo_stop);
>> > dev->hard_start_xmit = SET_NETDEVOP(ops->ndo_start_xmit);
>> > dev->set_multicast_list = SET_NETDEVOP(ops->ndo_set_multicast_list);
>> > dev->change_mtu = SET_NETDEVOP(ops->ndo_change_mtu);
>> > dev->set_mac_address = SET_NETDEVOP(ops->ndo_set_mac_address);
>> > dev->tx_timeout = SET_NETDEVOP(ops->ndo_tx_timeout);
>> > dev->get_stats = SET_NETDEVOP(ops->ndo_get_stats);
>> > #undef SET_NETDEVOP
>> > }
>> > EXPORT_SYMBOL(netdev_attach_ops);
>> >
>> > For newer kernels then this would just be:
>> >
>> > static inline void netdev_attach_ops(struct net_device *dev,
>> > const struct net_device_ops *ops)
>> > {
>> > dev->netdev_ops = ops;
>> > }
>> >
>> > Stephen, would the above be acceptable upstream on netdevice.h ? It
>> > would eliminate all needs from having to #ifdef network drivers when
>> > backporting. If so I can send a respective patch and spatch all the
>> > setters I think. An example of the nasty ifdef crap we have to do for
>> > the current backport of netdevop'able drivers is below.
>> >
>>
>> No. supporting backporting is not part of the upstream kernel
>> mission. Honestly, we try for forward compatibility but intentionally
>> ignore carrying extra backport baggage.
>
> Sure, understood, just had to try :), if only I could find a *good*
> non-backport reason to have the netdev_attach_ops()...
FWIW, it helped a lot, porting an Ethernet driver for example consists
of a 1 line change to the driver, this goes down to 2.6.21 even. With
a netdev_attach_ops() upstream this would require 0 lines of code
changes. But --- I understand, I'll try to find a real value for it on
existing kernels.
patches/01-netdev.patch | 625 ++++++-----------------------------------------
1 files changed, 75 insertions(+), 550 deletions(-)
Luis
^ permalink raw reply
* Re: RFC: Network Plugin Architecture (NPA) for vmxnet3
From: Stephen Hemminger @ 2010-05-05 2:44 UTC (permalink / raw)
To: Pankaj Thakkar
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
virtualization@lists.linux-foundation.org, pv-drivers@vmware.com,
Shreyas Bhatewara
In-Reply-To: <20100505001857.GQ8323@vmware.com>
On Tue, 4 May 2010 17:18:57 -0700
Pankaj Thakkar <pthakkar@vmware.com> wrote:
> The purpose of this email is to introduce the architecture and the design principles. The overall project involves more than just changes to vmxnet3 driver and hence we though an overview email would be better. Once people agree to the design in general we intend to provide the code changes to the vmxnet3 driver.
As Dave said, we care more about what the implementation looks like than the high level
goals of the design. I think we all agree that better management of virtualized devices
is necessary, the problem is that their are so many of them (vmware, xen, HV, Xen),
and vendors seem to to lean on their own specific implementation of a offloading,
which makes a general solution more difficult. Please, Please solve this cleanly.
The little things like API's and locking semantics and handling of dynamic versus
static control can make a good design in principle fall apart when someone does a bad
job of implementing them.
Lastly, projects that have had multiple people involved for long periods of time
in the dark often end up building a legacy mentality "but we convinced vendor XXX to include it
in their Enterprise version 666" and require lots of "retraining" before the code
becomes acceptable.
--
^ permalink raw reply
* MDaemon Notification -- Attachment Removed
From: Postmaster @ 2010-05-05 2:41 UTC (permalink / raw)
To: netdev
-------------------------------------------------------------------
MDaemon has detected restricted attachments within an email message
-------------------------------------------------------------------
>From : netdev@vger.kernel.org
To : ngochphcm@klv.com.vn
Subject : Mail Delivery (failure ngochphcm@klv.com.vn)
Message-ID:
---------------------
Attachment(s) removed
---------------------
message.scr
^ permalink raw reply
* Re: [Patch 1/3] sysctl: refactor integer handling proc code
From: Cong Wang @ 2010-05-05 3:02 UTC (permalink / raw)
To: Changli Gao
Cc: linux-kernel, Octavian Purdila, Eric Dumazet, penguin-kernel,
netdev, Neil Horman, ebiederm, David Miller, adobriyan
In-Reply-To: <u2s412e6f7f1004301549tb0e88a80n4c621e42c0b31015@mail.gmail.com>
Changli Gao wrote:
> On Fri, Apr 30, 2010 at 4:25 PM, Amerigo Wang <amwang@redhat.com> wrote:
>> + if (*p == '-' && *size > 1) {
>> + *neg = 1;
>
> As neg is bool*, you should use true and false instead of 1 and 0.
>
Yeah, I only corrected those lines that I touched, I should
correct them all.
Will fix.
Thanks.
^ permalink raw reply
* Re: [Patch 2/3] sysctl: add proc_do_large_bitmap
From: Cong Wang @ 2010-05-05 3:14 UTC (permalink / raw)
To: Changli Gao
Cc: linux-kernel, Octavian Purdila, Eric Dumazet, penguin-kernel,
netdev, Neil Horman, ebiederm, adobriyan, David Miller
In-Reply-To: <v2y412e6f7f1004301541yb1ede589t8c446966743ca138@mail.gmail.com>
Changli Gao wrote:
> add the following lines to let "echo 1-10 >>
> /proc/..." work as normal.
Hmm, I haven't tested this, what did you see if we append
lines into it?
Also, do we need appending lines to this /proc file when design it?
Octavian? Eric?
Thanks.
^ permalink raw reply
* [PATCH -next 1/3] bnx2: Add GRO support.
From: Michael Chan @ 2010-05-05 5:21 UTC (permalink / raw)
To: davem; +Cc: netdev
Signed-off-by: Michael Chan <mchan@broadcom.com>
Signed-off-by: Matt Carlson <mcarlson@broadcom.com>
---
drivers/net/bnx2.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/bnx2.c b/drivers/net/bnx2.c
index ab26bbc..6ad3184 100644
--- a/drivers/net/bnx2.c
+++ b/drivers/net/bnx2.c
@@ -3207,10 +3207,10 @@ bnx2_rx_int(struct bnx2 *bp, struct bnx2_napi *bnapi, int budget)
#ifdef BCM_VLAN
if (hw_vlan)
- vlan_hwaccel_receive_skb(skb, bp->vlgrp, vtag);
+ vlan_gro_receive(&bnapi->napi, bp->vlgrp, vtag, skb);
else
#endif
- netif_receive_skb(skb);
+ napi_gro_receive(&bnapi->napi, skb);
rx_pkt++;
--
1.6.4.GIT
^ permalink raw reply related
* [PATCH -next 2/3] bnx2: Add prefetches to rx path.
From: Michael Chan @ 2010-05-05 5:21 UTC (permalink / raw)
To: davem; +Cc: netdev
In-Reply-To: <1273036906-29162-1-git-send-email-mchan@broadcom.com>
Add prefetches of the skb and the next rx descriptor to speed up rx path.
Signed-off-by: Michael Chan <mchan@broadcom.com>
Signed-off-by: Matt Carlson <mcarlson@broadcom.com>
---
drivers/net/bnx2.c | 12 +++++++++---
drivers/net/bnx2.h | 1 +
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/drivers/net/bnx2.c b/drivers/net/bnx2.c
index 6ad3184..cdee29b 100644
--- a/drivers/net/bnx2.c
+++ b/drivers/net/bnx2.c
@@ -2719,6 +2719,7 @@ bnx2_alloc_rx_skb(struct bnx2 *bp, struct bnx2_rx_ring_info *rxr, u16 index)
}
rx_buf->skb = skb;
+ rx_buf->desc = (struct l2_fhdr *) skb->data;
dma_unmap_addr_set(rx_buf, mapping, mapping);
rxbd->rx_bd_haddr_hi = (u64) mapping >> 32;
@@ -2941,6 +2942,7 @@ bnx2_reuse_rx_skb(struct bnx2 *bp, struct bnx2_rx_ring_info *rxr,
rxr->rx_prod_bseq += bp->rx_buf_use_size;
prod_rx_buf->skb = skb;
+ prod_rx_buf->desc = (struct l2_fhdr *) skb->data;
if (cons == prod)
return;
@@ -3086,7 +3088,7 @@ bnx2_rx_int(struct bnx2 *bp, struct bnx2_napi *bnapi, int budget)
while (sw_cons != hw_cons) {
unsigned int len, hdr_len;
u32 status;
- struct sw_bd *rx_buf;
+ struct sw_bd *rx_buf, *next_rx_buf;
struct sk_buff *skb;
dma_addr_t dma_addr;
u16 vtag = 0;
@@ -3097,7 +3099,11 @@ bnx2_rx_int(struct bnx2 *bp, struct bnx2_napi *bnapi, int budget)
rx_buf = &rxr->rx_buf_ring[sw_ring_cons];
skb = rx_buf->skb;
+ prefetch(skb);
+ next_rx_buf =
+ &rxr->rx_buf_ring[RX_RING_IDX(NEXT_RX_BD(sw_cons))];
+ prefetch(next_rx_buf->desc);
rx_buf->skb = NULL;
dma_addr = dma_unmap_addr(rx_buf, mapping);
@@ -3106,7 +3112,7 @@ bnx2_rx_int(struct bnx2 *bp, struct bnx2_napi *bnapi, int budget)
BNX2_RX_OFFSET + BNX2_RX_COPY_THRESH,
PCI_DMA_FROMDEVICE);
- rx_hdr = (struct l2_fhdr *) skb->data;
+ rx_hdr = rx_buf->desc;
len = rx_hdr->l2_fhdr_pkt_len;
status = rx_hdr->l2_fhdr_status;
@@ -5764,7 +5770,7 @@ bnx2_run_loopback(struct bnx2 *bp, int loopback_mode)
rx_buf = &rxr->rx_buf_ring[rx_start_idx];
rx_skb = rx_buf->skb;
- rx_hdr = (struct l2_fhdr *) rx_skb->data;
+ rx_hdr = rx_buf->desc;
skb_reserve(rx_skb, BNX2_RX_OFFSET);
pci_dma_sync_single_for_cpu(bp->pdev,
diff --git a/drivers/net/bnx2.h b/drivers/net/bnx2.h
index ab34a5d..dd35bd0 100644
--- a/drivers/net/bnx2.h
+++ b/drivers/net/bnx2.h
@@ -6551,6 +6551,7 @@ struct l2_fhdr {
struct sw_bd {
struct sk_buff *skb;
+ struct l2_fhdr *desc;
DEFINE_DMA_UNMAP_ADDR(mapping);
};
--
1.6.4.GIT
^ permalink raw reply related
* [PATCH net-next-2.6] net: Increase NET_SKB_PAD to 64 bytes
From: Eric Dumazet @ 2010-05-05 5:24 UTC (permalink / raw)
To: David Miller; +Cc: netdev, jamal, Tom Herbert
eth_type_trans() & get_rps_cpus() currently need two 64bytes cache lines
in packet to compute rxhash.
Increasing NET_SKB_PAD from 32 to 64 reduces the need to one cache line
only, and makes RPS faster.
NET_IP_ALIGN(2) + ethernet_header(14) + IP_header(20/40) + ports(8)
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 746a652..fe5798b 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -1356,9 +1356,12 @@ static inline int skb_network_offset(const struct sk_buff *skb)
*
* Various parts of the networking layer expect at least 32 bytes of
* headroom, you should not reduce this.
+ * With RPS, we raised NET_SKB_PAD to 64 so that get_rps_cpus() fetches span
+ * a 64 bytes aligned block to fit modern (>= 64 bytes) cache line sizes
+ * NET_IP_ALIGN(2) + ethernet_header(14) + IP_header(20/40) + ports(8)
*/
#ifndef NET_SKB_PAD
-#define NET_SKB_PAD 32
+#define NET_SKB_PAD 64
#endif
extern int ___pskb_trim(struct sk_buff *skb, unsigned int len);
^ permalink raw reply related
* [RFC PATCH] net: deliver skbs on inactive slaves to exact matches
From: John Fastabend @ 2010-05-05 5:34 UTC (permalink / raw)
To: john.r.fastabend, netdev; +Cc: christopher.leech, andy, kaber, bonding-devel
Currently, the accelerated receive path for VLAN's will
drop packets if the real device is an inactive slave and
is not one of the special pkts tested for in
skb_bond_should_drop(). This behavior is different then
the non-accelerated path and for pkts over a bonded vlan.
For example,
vlanx -> bond0 -> ethx
will be dropped in the vlan path and not delivered to any
packet handlers. However,
bond0 -> vlanx -> ethx
will be delivered to handlers that match the exact dev,
because the VLAN path checks the real_dev which is not a
slave and netif_recv_skb() doesn't drop frames but only
delivers them to exact matches.
This patch adds a pkt_type PACKET_DROP which is now used
to identify skbs that would previously been dropped and
allows the skb to continue to skb_netif_recv(). Here we
add logic to check for PACKET_DROP and if so only deliver
to handlers that match exactly. IMHO this is more
consistent and gives pkt handlers a way to identify skbs
that come from inactive slaves.
This allows a third case to function which is important for
doing multipath with FCoE traffic while LAN traffic bonded,
bond0 -> ethx
|
vlanx -> --
Here the vlan is not in bond0 but the FCoE handler can still
receive the skb. Previously these skbs were dropped.
I have tested the following 4 configurations in failover modes
and load balancing modes and have not seen any duplicate packets
or unexpected bahavior.
# bond0 -> ethx
# vlanx -> bond0 -> ethx
# bond0 -> vlanx -> ethx
# bond0 -> ethx
|
vlanx -> --
Also this removes the PACKET_FASTROUTE define which was not being
used.
Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
---
include/linux/if_packet.h | 2 +-
net/8021q/vlan_core.c | 4 ++--
net/core/dev.c | 25 ++++++++++++++++++-------
3 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/include/linux/if_packet.h b/include/linux/if_packet.h
index 6ac23ef..9d079fa 100644
--- a/include/linux/if_packet.h
+++ b/include/linux/if_packet.h
@@ -28,7 +28,7 @@ struct sockaddr_ll {
#define PACKET_OUTGOING 4 /* Outgoing of any type */
/* These ones are invisible by user level */
#define PACKET_LOOPBACK 5 /* MC/BRD frame looped back */
-#define PACKET_FASTROUTE 6 /* Fastrouted frame */
+#define PACKET_DROP 6 /* Drop packet */
/* Packet socket options */
diff --git a/net/8021q/vlan_core.c b/net/8021q/vlan_core.c
index c584a0a..4510e08 100644
--- a/net/8021q/vlan_core.c
+++ b/net/8021q/vlan_core.c
@@ -12,7 +12,7 @@ int __vlan_hwaccel_rx(struct sk_buff *skb, struct vlan_group *grp,
return NET_RX_DROP;
if (skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
- goto drop;
+ skb->pkt_type = PACKET_DROP;
skb->skb_iif = skb->dev->ifindex;
__vlan_hwaccel_put_tag(skb, vlan_tci);
@@ -84,7 +84,7 @@ vlan_gro_common(struct napi_struct *napi, struct vlan_group *grp,
struct sk_buff *p;
if (skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
- goto drop;
+ skb->pkt_type = PACKET_DROP;
skb->skb_iif = skb->dev->ifindex;
__vlan_hwaccel_put_tag(skb, vlan_tci);
diff --git a/net/core/dev.c b/net/core/dev.c
index 4d43f1a..0a5b0b9 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2776,7 +2776,7 @@ static int __netif_receive_skb(struct sk_buff *skb)
struct net_device *orig_dev;
struct net_device *master;
struct net_device *null_or_orig;
- struct net_device *null_or_bond;
+ struct net_device *dev_or_bond;
int ret = NET_RX_DROP;
__be16 type;
@@ -2793,13 +2793,24 @@ static int __netif_receive_skb(struct sk_buff *skb)
if (!skb->skb_iif)
skb->skb_iif = skb->dev->ifindex;
+ /*
+ * bonding note: skbs received on inactive slaves should only
+ * be delivered to pkt handlers that are exact matches. Also
+ * the pkt_type field will be marked PACKET_DROP. If packet
+ * handlers are sensitive to duplicate packets these skbs will
+ * need to be dropped at the handler. The vlan accel path may
+ * have already set PACKET_DROP.
+ */
null_or_orig = NULL;
orig_dev = skb->dev;
master = ACCESS_ONCE(orig_dev->master);
- if (master) {
- if (skb_bond_should_drop(skb, master))
+ if (skb->pkt_type == PACKET_DROP)
+ null_or_orig = orig_dev;
+ else if (master) {
+ if (skb_bond_should_drop(skb, master)) {
+ skb->pkt_type = PACKET_DROP;
null_or_orig = orig_dev; /* deliver only exact match */
- else
+ } else
skb->dev = master;
}
@@ -2849,10 +2860,10 @@ ncls:
* device that may have registered for a specific ptype. The
* handler may have to adjust skb->dev and orig_dev.
*/
- null_or_bond = NULL;
+ dev_or_bond = skb->dev;
if ((skb->dev->priv_flags & IFF_802_1Q_VLAN) &&
(vlan_dev_real_dev(skb->dev)->priv_flags & IFF_BONDING)) {
- null_or_bond = vlan_dev_real_dev(skb->dev);
+ dev_or_bond = vlan_dev_real_dev(skb->dev);
}
type = skb->protocol;
@@ -2860,7 +2871,7 @@ ncls:
&ptype_base[ntohs(type) & PTYPE_HASH_MASK], list) {
if (ptype->type == type && (ptype->dev == null_or_orig ||
ptype->dev == skb->dev || ptype->dev == orig_dev ||
- ptype->dev == null_or_bond)) {
+ ptype->dev == dev_or_bond)) {
if (pt_prev)
ret = deliver_skb(skb, pt_prev, orig_dev);
pt_prev = ptype;
^ permalink raw reply related
* Re: Receive issues with bonding and vlans
From: John Fastabend @ 2010-05-05 5:35 UTC (permalink / raw)
To: Jay Vosburgh
Cc: Leech, Christopher, netdev@vger.kernel.org, Andy Gospodarek,
Patrick McHardy, bonding-devel@lists.sourceforge.net
In-Reply-To: <12574.1272928653@death.nxdomain.ibm.com>
Jay Vosburgh wrote:
> John Fastabend <john.r.fastabend@intel.com> wrote:
>
>> Jay Vosburgh wrote:
>>> John Fastabend <john.r.fastabend@intel.com> wrote:
>>>
> [...]
>>>> It should be OK to allow packets to be received on the VLAN if it is not
>>>> explicitly in the bond?
>>> Lemme see if I have this straight, because all of these special
>>> cases are making my brain hurt. This one is for a configuration like this:
>>>
>>> bond0 ----- eth0
>>> /
>>> vlan.xxx -/
>>>
>>> I.e., a VLAN configured directly atop an ethernet device, said
>>> ethernet also being a slave to bonding. Is that correct?
>>>
>> Yes, this is the correct scenario that we are considering.
>>
>>> Extrapolating from the ASCII art in a prior message in this
>>> discussion, would this configuration really be something like this:
>>>
>>> vlan.xxx -\
>>> \
>>> bond0 ----- eth1
>>> bond0 ----- eth0
>>> /
>>> vlan.xxx -/
>>>
>>> I.e., two slaves to bonding, with VLAN xxx configured atop both
>>> of the slaves? Or would the eth0 and eth1 use discrete VLAN ids? The
>>> reason I ask is in regards to duplicate suppression. The whole reason
>>> the "inactive" slave drops (most) incoming packets is to eliminate
>>> duplicates when the switch floods traffic to both slave ports.
>>>
>> These vlan ids could be the same or discrete I think both configurations
>> should be valid.
>>
>>> This is a bit tricky, because it's not really about broadcasts /
>>> multicasts so much, but about traffic that the switch sends to all ports
>>> because the switch's MAC address table isn't up to date with the
>>> destination MAC of the traffic (which is a transient condition, so this
>>> would only happen for perhaps one second or so). That would result in
>>> duplicate unicast packets being received by the bond (back in the day
>>> before bonding had the "drop inactive traffic" logic).
>>>
>>> So if the same VLAN is configured atop the two slaves, I wonder
>>> if that will open a window for the duplicate unicast packet problem.
>> OK, this does appear to open a window for duplicated unicast packets. By
>> only allowing handlers with exact matches at least this issue is less
>> obvious and we are assuming the packet handler can deal with this
>> duplication. This seems to be the current assumption made. The same issue
>> exists today for real device in the following setup,
>>
>> vlan --> bond0 --> eth
>
> I just tested this, and I'm not seeing duplicate packets using
> the test that used to show the problem before the "drop dups" logic went
> in (clear the switch's mac address-table, ping -c 25 -f [peer on VLAN],
> compare "packets transmitted" to "packets received").
>
> That doesn't mean there isn't a gap in the logic somewhere, just
> that the original problem hasn't resurfaced (as far as I can tell).
>
>> Specifically for FCoE we use the san mac address so it wouldn't be an
>> issue here. The expectation being that the switch will only ever use the
>> correct san mac on the port.
>
> The issue arises when the switch does not have the destination
> MAC in its address table, and as such is transitory, and only occurs
> after sufficiently long periods of no traffic (or a manual flush of the
> table). The packets are sent to all ports until the MAC table updates
> (which seems to take place asynchronously), which is usually about 1
> second or so (on the midrange Cisco gear I have).
>
> For example, with the switch's mac address table cleared, when
> starting a "ping -f" I can watch as first every port's light blinks,
> then all but two stop blinking. During the time that every port is
> blinking, the switch is sending all the packets to every port because
> the mac address table hasn't updated the switching logic (however that
> works under the covers).
>
>
>
>>> If the VLAN ids are different, then I'll assume this is some
>>> kind of device mapper magic, doing the load balancing elsewhere.
>> Correct device mapper handles load balancing and failover for both cases,
>> when the vlan ids are different and when they are the same.
>>
>>>> Or if we want to be more paranoid deliver packets only to handlers with
>>>> exact matches for the device. For non vlan devices we deliver skb's to
>>>> packet handlers that match exactly even on inactive slaves so doing this
>>>> on vlan devices as well makes sense and shouldn't cause any unexpected
>>>> problems.
>>> Yah, the whole concept of "inactive" slave is pretty mutated
>>> now; it's kind of become the "active-backup with semi-manual load
>>> balance for clever protocols, oh, and duplicate suppression" mode.
>>>
>>>> Also on a somewhat unrelated note I suspect null_or_orig and null_or_bond
>>>> are not working as expected in __netif_receive_skb(). At least the
>>>> comment 'deliver only exact match' could be inaccurate.
>>> I don't think this is unrelated at all. This code (the packet
>>> to device lookup stuff in __netif_receive_skb) has been modified pretty
>>> extensively lately for various bonding-related special cases, and I
>>> think it is getting hard to follow. Whatever comments are there need to
>>> be accurate, and, honestly, I think this code needs comments to explain
>>> what exactly is supposed to happen for these special cases.
>>>
>> Agreed. This should be cleaned up and some explanations added. The
>> current behavior in active-backup mode is receiving packets on the bonded
>> real device in active mode fails but putting that same real device in an
>> inactive state will cause it to receive packets. This is an
>> inconsistency, which should probably be fixed by initializing null_or_bond
>> to orig_dev. And also renaming it orig_or_bond at that point.
>>
>>>> Here's a patch to illustrate what I'm thinking compile tested only. If
>>>> this sounds reasonable I'll work up an official patch.
>>>>
>>>>
>>>> [PATCH] net: allow vlans on bonded real net_devices
>>>>
>>>> For converged I/O it is reasonable to use dm_multipathing to provice
>>>> failover and load balancing for storage traffic and then use bonding
>>>> for the LAN failover and load balancing.
>>>>
>>>> Currently this works if the multipathed devices are using the real
>>>> net_device and those devices are enslaved to a bonded device what
>>>> does not work is creating a vlan on the real device and trying to
>>>> use it outside the bond for multipathing.
>>>>
>>>> This patch adds logic so that if the skb is destined for a vlan
>>>> that is not in the bond the skb will not be dropped.
>>>>
>>>> Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
>>>> ---
>>>>
>>>> net/8021q/vlan_core.c | 31 +++++++++++++++++++++----------
>>>> net/core/dev.c | 11 ++++++++---
>>>> 2 files changed, 29 insertions(+), 13 deletions(-)
>>>>
>>>> diff --git a/net/8021q/vlan_core.c b/net/8021q/vlan_core.c
>>>> index c584a0a..3bce0c3 100644
>>>> --- a/net/8021q/vlan_core.c
>>>> +++ b/net/8021q/vlan_core.c
>>>> @@ -8,18 +8,24 @@
>>>> int __vlan_hwaccel_rx(struct sk_buff *skb, struct vlan_group *grp,
>>>> u16 vlan_tci, int polling)
>>>> {
>>>> + struct net_device *vlan_dev;
>>>> +
>>>> if (netpoll_rx(skb))
>>>> return NET_RX_DROP;
>>>>
>>>> - if (skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
>>>> + vlan_dev = vlan_group_get_device(grp, vlan_tci & VLAN_VID_MASK);
>>>> +
>>>> + if (!vlan_dev)
>>>> + goto drop;
>>>> +
>>>> + if ((vlan_dev->priv_flags & IFF_BONDING ||
>>>> + vlan_dev_real_dev(vlan_dev)->flags & IFF_MASTER) &&
>>>> + skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
>>> I'm not sure this will do the right thing if the VLAN device
>>> itself is a slave to bonding, e.g., bond0 ---> vlan.xxx ---> eth0. In
>>> that case, eth0's dev->master is NULL, and the vlan_dev (vlan.xxx's dev)
>>> doesn't have IFF_MASTER (but does have IFF_SLAVE and IFF_BONDING, I
>>> believe).
>>>
>> correct, vlan_dev does have IFF_BONDING and IFF_SLAVE here and doesn't
>> have IFF_MASTER.
>>
>>
>>> I think this will result in all incoming traffic being accepted
>>> on such a configuration (leading to duplicates, as described above).
>>>
>>> I suspect, but have not tested, that something like this might
>>> do what you're looking for:
>>>
>>> if ((vlan_dev->priv_flags & IFF_BONDING ||
>>> vlan_dev_real_dev(vlan_dev)->flags & (IFF_MASTER | IFF_SLAVE)) &&
>>> skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
>>>
>>> I.e., if the VLAN device is either a MASTER (configured above
>>> the bond) or a slave (configured below the bond) do the duplicate
>>> suppresion.
>> Here are the three basic cases I see,
>>
>> #1. vlanx --> bond0 --> ethx
>>
>> In this case vlanx does not have IFF_BONDING set and real_dev is ethx with
>> IFF_SLAVE set. ethx has master dev->bond0 so this should work. And shows
>> why we need the IFF_SLAVE bit as you pointed out and I dropped.
>>
>> #2. bond --> vlanx --> ethx
>>
>> This case is broke, skb->dev->master is NULL so we would never drop this
>> pkt. As it exists today I suspect this is broken as well.
>
> In the VLAN pass, yes, but the VLAN input path will call into
> netif_receive_skb, and at that point the skb->dev is the vlan device,
> and it has a dev->master. I haven't tested this lately, but I'm fairly
> sure this works.
>
OK, these both seem to work as expected my test was invalid.
>> #3 bond0 --> ethx
>> vlanx --> -|
>>
>> Here is the case where adding the IFF_SLAVE bit doesn't work as I
>> hoped. We don't want to run skb_bond_should_drop here.
>
> Yes, this is tricky because the VLAN device will copy the
> dev->flags from the device it's placed atop, so the VLAN will inherit
> the ethx's IFF_SLAVE flag. This happens regardless of the setup order
> (enslave ethX, then add VLAN, or vice versa).
>
This doesn't appear to be true, adding a VLAN on ethx then enslave ethx
doesn't set the IFF_SLAVE flag on the VLAN. Unless I am missing something.
> I suspect this case may be testable because the VLAN device has
> IFF_SLAVE, but has no dev->master.
>
>> So I think there needs to be a bit of logic here to determine if we need
>> to check skb_bond_should_drop with the vlan device or with the
>> skb->dev->master. Something like might do:
>>
>> should_drop_dev = vlan_dev->master ? vlan_dev->master : skb->dev->master
>>
>> This should fix case #2 without breaking case #1. And the case I want to
>> allow is still not resolved. I'll think about this some more maybe this
>> logic can be fixed for all cases.
>
> As I said above, I don't think case #2 is really broken now.
>
Seems to be working sorry for the noise.
<<snip>>
>
> Hopefully this will be the last futzing around with this, and
> won't make it too complicated.
>
I currently believe the cleanest way to implement this is to add a
pkt_type flag PACKET_DROP to mark skbs that have been received on the
inactive slave. I sent out a functional RFC I would like to run a few
more tests on it, but otherwise I think it ok.
Thanks,
John.
> -J
>
> ---
> -Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox