* RE: [PATCH 4/4] net/mlx4_en: Use atomic counter to decide when queue is full
From: Eric Dumazet @ 2012-06-25 10:12 UTC (permalink / raw)
To: David Laight; +Cc: David Miller, yevgenyp, netdev
In-Reply-To: <AE90C24D6B3A694183C094C60CF0A2F6026B6F61@saturn3.aculab.com>
On Mon, 2012-06-25 at 10:00 +0100, David Laight wrote:
> > > The Transmit and transmit completion flows execute from different
> contexts,
> > > which are not synchronized. Hence naive reading the of consumer
> index might
> > > give wrong value by the time it is being used, That could lead to a
> state of transmit timeout.
> > > Fix that by using atomic variable to maintain that index.
> > >
> > > Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
> >
> > I'm not convinced. There is only one place that actually changes
> > the counter.
> >
> > So it seems more like you have a missing memory barrier somewhere.
>
> Or just keep the two ring indexes - instead of keeping the
> number of 'active' entries as well.
> Then you don't have a variable which the tx setup and
> tx completion routines both update.
This is what was implied by David.
Using a producer/consumer index and appropriate memory barriers.
start_xmit() and tx completion can be truly lockless and atomicless in
their fast path.
There are many drivers doing that correctly.
tg3 driver is a good example.
^ permalink raw reply
* Re: [net-next RFC V4 PATCH 3/4] virtio: introduce a method to get the irq of a specific virtqueue
From: Michael S. Tsirkin @ 2012-06-25 10:14 UTC (permalink / raw)
To: Jason Wang
Cc: krkumar2, habanero, kvm, qemu-devel, netdev, mashirle,
linux-kernel, virtualization, edumazet, tahm, jwhan, davem
In-Reply-To: <1340617278-8022-1-git-send-email-jasowang@redhat.com>
On Mon, Jun 25, 2012 at 05:41:17PM +0800, Jason Wang wrote:
> Device specific irq optimizations such as irq affinity may be used by virtio
> drivers. So this patch introduce a new method to get the irq of a specific
> virtqueue.
>
> After this patch, virtio device drivers could query the irq and do device
> specific optimizations. First user would be virtio-net.
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> drivers/lguest/lguest_device.c | 8 ++++++++
> drivers/s390/kvm/kvm_virtio.c | 6 ++++++
> drivers/virtio/virtio_mmio.c | 8 ++++++++
> drivers/virtio/virtio_pci.c | 12 ++++++++++++
> include/linux/virtio_config.h | 4 ++++
> 5 files changed, 38 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/lguest/lguest_device.c b/drivers/lguest/lguest_device.c
> index 9e8388e..bcd080f 100644
> --- a/drivers/lguest/lguest_device.c
> +++ b/drivers/lguest/lguest_device.c
> @@ -392,6 +392,13 @@ static const char *lg_bus_name(struct virtio_device *vdev)
> return "";
> }
>
> +static int lg_get_vq_irq(struct virtio_device *vdev, struct virtqueue *vq)
> +{
> + struct lguest_vq_info *lvq = vq->priv;
> +
> + return lvq->config.irq;
> +}
> +
> /* The ops structure which hooks everything together. */
> static struct virtio_config_ops lguest_config_ops = {
> .get_features = lg_get_features,
> @@ -404,6 +411,7 @@ static struct virtio_config_ops lguest_config_ops = {
> .find_vqs = lg_find_vqs,
> .del_vqs = lg_del_vqs,
> .bus_name = lg_bus_name,
> + .get_vq_irq = lg_get_vq_irq,
> };
>
> /*
> diff --git a/drivers/s390/kvm/kvm_virtio.c b/drivers/s390/kvm/kvm_virtio.c
> index d74e9ae..a897de2 100644
> --- a/drivers/s390/kvm/kvm_virtio.c
> +++ b/drivers/s390/kvm/kvm_virtio.c
> @@ -268,6 +268,11 @@ static const char *kvm_bus_name(struct virtio_device *vdev)
> return "";
> }
>
> +static int kvm_get_vq_irq(struct virtio_device *vdev, struct virtqueue *vq)
> +{
> + return 0x2603;
> +}
> +
> /*
> * The config ops structure as defined by virtio config
> */
> @@ -282,6 +287,7 @@ static struct virtio_config_ops kvm_vq_configspace_ops = {
> .find_vqs = kvm_find_vqs,
> .del_vqs = kvm_del_vqs,
> .bus_name = kvm_bus_name,
> + .get_vq_irq = kvm_get_vq_irq,
> };
>
> /*
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index f5432b6..2ba37ed 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -411,6 +411,13 @@ static const char *vm_bus_name(struct virtio_device *vdev)
> return vm_dev->pdev->name;
> }
>
> +static int vm_get_vq_irq(struct virtio_device *vdev, struct virtqueue *vq)
> +{
> + struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
> +
> + return platform_get_irq(vm_dev->pdev, 0);
> +}
> +
> static struct virtio_config_ops virtio_mmio_config_ops = {
> .get = vm_get,
> .set = vm_set,
> @@ -422,6 +429,7 @@ static struct virtio_config_ops virtio_mmio_config_ops = {
> .get_features = vm_get_features,
> .finalize_features = vm_finalize_features,
> .bus_name = vm_bus_name,
> + .get_vq_irq = vm_get_vq_irq,
> };
>
>
> diff --git a/drivers/virtio/virtio_pci.c b/drivers/virtio/virtio_pci.c
> index adb24f2..c062227 100644
> --- a/drivers/virtio/virtio_pci.c
> +++ b/drivers/virtio/virtio_pci.c
> @@ -607,6 +607,17 @@ static const char *vp_bus_name(struct virtio_device *vdev)
> return pci_name(vp_dev->pci_dev);
> }
>
> +static int vp_get_vq_irq(struct virtio_device *vdev, struct virtqueue *vq)
> +{
> + struct virtio_pci_device *vp_dev = to_vp_device(vdev);
> + struct virtio_pci_vq_info *info = vq->priv;
> +
> + if (vp_dev->intx_enabled)
> + return vp_dev->pci_dev->irq;
> + else
> + return vp_dev->msix_entries[info->msix_vector].vector;
> +}
> +
> static struct virtio_config_ops virtio_pci_config_ops = {
> .get = vp_get,
> .set = vp_set,
> @@ -618,6 +629,7 @@ static struct virtio_config_ops virtio_pci_config_ops = {
> .get_features = vp_get_features,
> .finalize_features = vp_finalize_features,
> .bus_name = vp_bus_name,
> + .get_vq_irq = vp_get_vq_irq,
> };
>
> static void virtio_pci_release_dev(struct device *_d)
> diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
> index fc457f4..acd6930 100644
> --- a/include/linux/virtio_config.h
> +++ b/include/linux/virtio_config.h
> @@ -98,6 +98,9 @@
> * vdev: the virtio_device
> * This returns a pointer to the bus name a la pci_name from which
> * the caller can then copy.
> + * @get_vq_irq: get the irq numer of the specific virt queue.
> + * vdev: the virtio_device
> + * vq: the virtqueue
What if the vq does not have an IRQ? E.g. control vqs don't.
What if the IRQ is shared between VQs? Between devices?
The need to cleanup affinity on destroy is also nasty.
How about we expose a set_affinity API instead?
Then:
- non PCI can ignore for now
- with a per vq vector we can force it
- with a shared MSI we make it an OR over all affinities
- with a level interrupt we can ignore it
- on cleanup we can do it in core
> */
> typedef void vq_callback_t(struct virtqueue *);
> struct virtio_config_ops {
> @@ -116,6 +119,7 @@ struct virtio_config_ops {
> u32 (*get_features)(struct virtio_device *vdev);
> void (*finalize_features)(struct virtio_device *vdev);
> const char *(*bus_name)(struct virtio_device *vdev);
> + int (*get_vq_irq)(struct virtio_device *vdev, struct virtqueue *vq);
> };
>
> /* If driver didn't advertise the feature, it will never appear. */
> --
> 1.7.1
^ permalink raw reply
* [PATCH] net-next: ipv6: ndisc: allocate a ndisc socket per inet6_dev
From: Menny_Hamburger @ 2012-06-25 10:26 UTC (permalink / raw)
To: netdev
From: mennyh <Menny_Hamburger@Dell.com>
When an IPV6 network discovery packet does not get sent by the NIC,
either because there is some S/W issue or a H/W problem with the NIC, NDP will stop
working and will not be able to send ndisc packets via other NICs on the machine.
The reason for this that there is only one global socket assigned per network for network discovery
(net->ipv6.ndisc_sk), and when this socket is busy, NDP cannot be serviced by
other NICS.
This patch adds a kernel configuration option IPV6_NDISC_SOCKET_PER_INTERFACE,
which when enabled the kernel will allocate a network discovery socket per inet6_dev on creation,
instead of a single socket per network.
Signed-off-by: mennyh <Menny_Hamburger@Dell.com>
---
include/net/if_inet6.h | 3 +++
include/net/ndisc.h | 3 +++
include/net/netns/ipv6.h | 2 ++
net/ipv6/Kconfig | 8 ++++++++
net/ipv6/addrconf.c | 22 +++++++++++++++++++++
net/ipv6/ndisc.c | 48 +++++++++++++++++++++++++++++++++++++++++++++-
6 files changed, 85 insertions(+), 1 deletion(-)
diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h
index 9356322..7134632 100644
--- a/include/net/if_inet6.h
+++ b/include/net/if_inet6.h
@@ -191,6 +191,9 @@ struct inet6_dev {
struct inet6_dev *next;
struct ipv6_devconf cnf;
struct ipv6_devstat stats;
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ struct sock *ndisc_sk;
+#endif
unsigned long tstamp; /* ipv6InterfaceTable update timestamp */
struct rcu_head rcu;
};
diff --git a/include/net/ndisc.h b/include/net/ndisc.h
index c02b6ad..9039d6c 100644
--- a/include/net/ndisc.h
+++ b/include/net/ndisc.h
@@ -153,6 +153,9 @@ extern void ndisc_send_skb(struct sk_buff *skb,
const struct in6_addr *saddr,
struct icmp6hdr *icmp6h);
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+extern int ndisc_socket_init(struct sock **ndisc_sk, struct net *net);
+#endif
/*
diff --git a/include/net/netns/ipv6.h b/include/net/netns/ipv6.h
index df0a545..5d65d60 100644
--- a/include/net/netns/ipv6.h
+++ b/include/net/netns/ipv6.h
@@ -58,7 +58,9 @@ struct netns_ipv6 {
struct fib_rules_ops *fib6_rules_ops;
#endif
struct sock **icmp_sk;
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
struct sock *ndisc_sk;
+#endif
struct sock *tcp_sk;
struct sock *igmp_sk;
#ifdef CONFIG_IPV6_MROUTE
diff --git a/net/ipv6/Kconfig b/net/ipv6/Kconfig
index 5728695..77a09ff 100644
--- a/net/ipv6/Kconfig
+++ b/net/ipv6/Kconfig
@@ -250,4 +250,12 @@ config IPV6_PIMSM_V2
Support for IPv6 PIM multicast routing protocol PIM-SMv2.
If unsure, say N.
+config IPV6_NDISC_SOCKET_PER_INTERFACE
+ bool "IPv6: define socket for network discovery per interface (EXPERIMENTAL)"
+ depends on IPV6 && EXPERIMENTAL
+ ---help---
+ Normally only one socket per network is allocated to service the IPV6 network discovery protocol;
+ This may cause NDP to stop working when ndisc packet is starved by a NIC due to S/W or H/W problems.
+ If you say Y here, a separate socket will be allocated for each IPV6 enabled interface.
+
+ If unsure, say N.
+
endif # IPV6
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index 8f6411c..8cbcb66 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -78,6 +78,9 @@
#include <net/ip.h>
#include <net/netlink.h>
#include <net/pkt_sched.h>
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+#include <net/inet_common.h>
+#endif
#include <linux/if_tunnel.h>
#include <linux/rtnetlink.h>
@@ -336,6 +339,13 @@ void in6_dev_finish_destroy(struct inet6_dev *idev)
pr_warn("Freeing alive inet6 device %p\n", idev);
return;
}
+
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ if (idev->ndisc_sk) {
+ inet_ctl_sock_destroy(idev->ndisc_sk);
+ idev->ndisc_sk = NULL;
+ }
+#endif
snmp6_free_dev(idev);
kfree_rcu(idev, rcu);
}
@@ -392,6 +402,18 @@ static struct inet6_dev *ipv6_add_dev(struct net_device *dev)
return NULL;
}
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ if (ndisc_socket_init(&ndev->ndisc_sk, dev_net(dev)) < 0) {
+ ADBG((KERN_WARNING
+ "%s(): cannot allocate network discovery socket\n"));
+ ndev->ndisc_sk = NULL;
+ neigh_parms_release(&nd_tbl, ndev->nd_parms);
+ ndev->dead = 1;
+ in6_dev_finish_destroy(ndev);
+ return NULL;
+ }
+#endif
+
/* One reference from device. We must do this before
* we invoke __ipv6_regen_rndid().
*/
diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
index 69a6330..08f991b 100644
--- a/net/ipv6/ndisc.c
+++ b/net/ipv6/ndisc.c
@@ -422,8 +422,12 @@ struct sk_buff *ndisc_build_skb(struct net_device *dev,
const struct in6_addr *target,
int llinfo)
{
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
struct net *net = dev_net(dev);
struct sock *sk = net->ipv6.ndisc_sk;
+#else
+ struct sock *sk;
+#endif
struct sk_buff *skb;
struct icmp6hdr *hdr;
int hlen = LL_RESERVED_SPACE(dev);
@@ -432,6 +436,10 @@ struct sk_buff *ndisc_build_skb(struct net_device *dev,
int err;
u8 *opt;
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ sk = in6_dev_get(dev)->ndisc_sk;
+#endif
+
if (!dev->addr_len)
llinfo = 0;
@@ -488,11 +496,19 @@ void ndisc_send_skb(struct sk_buff *skb,
struct flowi6 fl6;
struct dst_entry *dst;
struct net *net = dev_net(dev);
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
struct sock *sk = net->ipv6.ndisc_sk;
+#else
+ struct sock *sk;
+#endif
struct inet6_dev *idev;
int err;
u8 type;
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ sk = in6_dev_get(dev)->ndisc_sk;
+#endif
+
type = icmp6h->icmp6_type;
icmpv6_flow_init(sk, &fl6, type, saddr, daddr, dev->ifindex);
@@ -550,6 +566,11 @@ static void ndisc_send_na(struct net_device *dev, struct neighbour *neigh,
struct icmp6hdr icmp6h = {
.icmp6_type = NDISC_NEIGHBOUR_ADVERTISEMENT,
};
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ struct sock *sk = dev_net(dev)->ipv6.ndisc_sk;
+#else
+ struct sock *sk = in6_dev_get(dev)->ndisc_sk;
+#endif
/* for anycast or proxy, solicited_addr != src_addr */
ifp = ipv6_get_ifaddr(dev_net(dev), solicited_addr, dev, 1);
@@ -561,7 +582,7 @@ static void ndisc_send_na(struct net_device *dev, struct neighbour *neigh,
in6_ifa_put(ifp);
} else {
if (ipv6_dev_get_saddr(dev_net(dev), dev, daddr,
- inet6_sk(dev_net(dev)->ipv6.ndisc_sk)->srcprefs,
+ inet6_sk(sk)->srcprefs,
&tmpaddr))
return;
src_addr = &tmpaddr;
@@ -1470,7 +1491,11 @@ void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target)
{
struct net_device *dev = skb->dev;
struct net *net = dev_net(dev);
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
struct sock *sk = net->ipv6.ndisc_sk;
+#else
+ struct sock *sk;
+#endif
int len = sizeof(struct icmp6hdr) + 2 * sizeof(struct in6_addr);
struct inet_peer *peer;
struct sk_buff *buff;
@@ -1487,6 +1512,10 @@ void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target)
int err;
u8 ha_buf[MAX_ADDR_LEN], *ha = NULL;
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ sk = in6_dev_get(dev)->ndisc_sk;
+#endif
+
if (ipv6_get_lladdr(dev, &saddr_buf, IFA_F_TENTATIVE)) {
ND_PRINTK(2, warn, "Redirect: no link-local address on %s\n",
dev->name);
@@ -1761,7 +1790,11 @@ int ndisc_ifinfo_sysctl_change(struct ctl_table *ctl, int write, void __user *bu
#endif
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+int ndisc_socket_init(struct sock **ndisc_sk, struct net *net)
+#else
static int __net_init ndisc_net_init(struct net *net)
+#endif
{
struct ipv6_pinfo *np;
struct sock *sk;
@@ -1776,7 +1809,11 @@ static int __net_init ndisc_net_init(struct net *net)
return err;
}
+#ifdef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
+ *ndisc_sk = sk;
+#else
net->ipv6.ndisc_sk = sk;
+#endif
np = inet6_sk(sk);
np->hop_limit = 255;
@@ -1786,6 +1823,7 @@ static int __net_init ndisc_net_init(struct net *net)
return 0;
}
:q!
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
static void __net_exit ndisc_net_exit(struct net *net)
{
inet_ctl_sock_destroy(net->ipv6.ndisc_sk);
@@ -1795,14 +1833,18 @@ static struct pernet_operations ndisc_net_ops = {
.init = ndisc_net_init,
.exit = ndisc_net_exit,
};
+#endif
int __init ndisc_init(void)
{
int err;
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
err = register_pernet_subsys(&ndisc_net_ops);
if (err)
return err;
+#endif
+
/*
* Initialize the neighbour table
*/
@@ -1825,7 +1867,9 @@ out_unregister_sysctl:
neigh_sysctl_unregister(&nd_tbl.parms);
out_unregister_pernet:
#endif
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
unregister_pernet_subsys(&ndisc_net_ops);
+#endif
goto out;
}
@@ -1836,5 +1880,7 @@ void ndisc_cleanup(void)
neigh_sysctl_unregister(&nd_tbl.parms);
#endif
neigh_table_clear(&nd_tbl);
+#ifndef CONFIG_IPV6_NDISC_SOCKET_PER_INTERFACE
unregister_pernet_subsys(&ndisc_net_ops);
+#endif
}
--
1.7.10.1
^ permalink raw reply related
* Re: [PATCH] net-next: ipv6: ndisc: allocate a ndisc socket per inet6_dev
From: Eric Dumazet @ 2012-06-25 10:41 UTC (permalink / raw)
To: Menny_Hamburger; +Cc: netdev
In-Reply-To: <D8C50530D6022F40A817A35C40CC06A70B34E34D44@DUBX7MCDUB01.EMEA.DELL.COM>
On Mon, 2012-06-25 at 11:26 +0100, Menny_Hamburger@Dell.com wrote:
> From: mennyh <Menny_Hamburger@Dell.com>
>
> When an IPV6 network discovery packet does not get sent by the NIC,
> either because there is some S/W issue or a H/W problem with the NIC, NDP will stop
> working and will not be able to send ndisc packets via other NICs on the machine.
> The reason for this that there is only one global socket assigned per network for network discovery
> (net->ipv6.ndisc_sk), and when this socket is busy, NDP cannot be serviced by
> other NICS.
>
> This patch adds a kernel configuration option IPV6_NDISC_SOCKET_PER_INTERFACE,
> which when enabled the kernel will allocate a network discovery socket per inet6_dev on creation,
> instead of a single socket per network.
>
> Signed-off-by: mennyh <Menny_Hamburger@Dell.com>
> ---
You obviously didn't see my patch to address this problem ?
I was waiting your feedback and you post this wrong patch instead ?
This sucks.
Test this instead. Please ?
diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
index 69a6330..f149d85 100644
--- a/net/ipv6/ndisc.c
+++ b/net/ipv6/ndisc.c
@@ -429,7 +429,6 @@ struct sk_buff *ndisc_build_skb(struct net_device *dev,
int hlen = LL_RESERVED_SPACE(dev);
int tlen = dev->needed_tailroom;
int len;
- int err;
u8 *opt;
if (!dev->addr_len)
@@ -439,15 +438,10 @@ struct sk_buff *ndisc_build_skb(struct net_device *dev,
if (llinfo)
len += ndisc_opt_addr_space(dev);
- skb = sock_alloc_send_skb(sk,
- (MAX_HEADER + sizeof(struct ipv6hdr) +
- len + hlen + tlen),
- 1, &err);
- if (!skb) {
- ND_PRINTK(0, err, "ND: %s failed to allocate an skb, err=%d\n",
- __func__, err);
+ skb = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + len + hlen + tlen,
+ GFP_ATOMIC);
+ if (!skb)
return NULL;
- }
skb_reserve(skb, hlen);
ip6_nd_hdr(sk, skb, dev, saddr, daddr, IPPROTO_ICMPV6, len);
@@ -1550,16 +1544,10 @@ void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target)
hlen = LL_RESERVED_SPACE(dev);
tlen = dev->needed_tailroom;
- buff = sock_alloc_send_skb(sk,
- (MAX_HEADER + sizeof(struct ipv6hdr) +
- len + hlen + tlen),
- 1, &err);
- if (buff == NULL) {
- ND_PRINTK(0, err,
- "Redirect: %s failed to allocate an skb, err=%d\n",
- __func__, err);
+ buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + len + hlen + tlen,
+ GFP_ATOMIC);
+ if (!buff)
goto release;
- }
skb_reserve(buff, hlen);
ip6_nd_hdr(sk, buff, dev, &saddr_buf, &ipv6_hdr(skb)->saddr,
^ permalink raw reply related
* [PATCH] net: l2tp_eth: fix l2tp_eth_dev_xmit race
From: Eric Dumazet @ 2012-06-25 10:45 UTC (permalink / raw)
To: David Miller; +Cc: netdev, James Chapman
From: Eric Dumazet <edumazet@google.com>
Its illegal to dereference skb after giving it to l2tp_xmit_skb()
as it might be already freed/reused.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: James Chapman <jchapman@katalix.com>
---
diff --git a/net/l2tp/l2tp_eth.c b/net/l2tp/l2tp_eth.c
index 185f12f..c3738f4 100644
--- a/net/l2tp/l2tp_eth.c
+++ b/net/l2tp/l2tp_eth.c
@@ -88,12 +88,12 @@ static int l2tp_eth_dev_xmit(struct sk_buff *skb, struct net_device *dev)
struct l2tp_eth *priv = netdev_priv(dev);
struct l2tp_session *session = priv->session;
- l2tp_xmit_skb(session, skb, session->hdr_len);
-
dev->stats.tx_bytes += skb->len;
dev->stats.tx_packets++;
- return 0;
+ l2tp_xmit_skb(session, skb, session->hdr_len);
+
+ return NETDEV_TX_OK;
}
static struct net_device_ops l2tp_eth_netdev_ops = {
^ permalink raw reply related
* RE: [PATCH] net-next: ipv6: ndisc: allocate a ndisc socket per inet6_dev
From: Menny_Hamburger @ 2012-06-25 11:08 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev
In-Reply-To: <1340620878.10893.26.camel@edumazet-glaptop>
I'm sorry for not responding on your post.
I really want to understand how this fixes our problem.
This fix will make the skb allocations succeed, but what mechanism releases the stuck socket associated with the bad device?
Thanks,
Menny
-----Original Message-----
From: Eric Dumazet [mailto:eric.dumazet@gmail.com]
Sent: 25 June, 2012 13:41
To: Hamburger, Menny
Cc: netdev@vger.kernel.org
Subject: Re: [PATCH] net-next: ipv6: ndisc: allocate a ndisc socket per inet6_dev
On Mon, 2012-06-25 at 11:26 +0100, Menny_Hamburger@Dell.com wrote:
> From: mennyh <Menny_Hamburger@Dell.com>
>
> When an IPV6 network discovery packet does not get sent by the NIC,
> either because there is some S/W issue or a H/W problem with the NIC, NDP will stop
> working and will not be able to send ndisc packets via other NICs on the machine.
> The reason for this that there is only one global socket assigned per network for network discovery
> (net->ipv6.ndisc_sk), and when this socket is busy, NDP cannot be serviced by
> other NICS.
>
> This patch adds a kernel configuration option IPV6_NDISC_SOCKET_PER_INTERFACE,
> which when enabled the kernel will allocate a network discovery socket per inet6_dev on creation,
> instead of a single socket per network.
>
> Signed-off-by: mennyh <Menny_Hamburger@Dell.com>
> ---
You obviously didn't see my patch to address this problem ?
I was waiting your feedback and you post this wrong patch instead ?
This sucks.
Test this instead. Please ?
^ permalink raw reply
* Re: [PATCH 01/13] netfilter: fix problem with proto register
From: Pablo Neira Ayuso @ 2012-06-25 11:12 UTC (permalink / raw)
To: Gao feng; +Cc: netdev, netfilter-devel
In-Reply-To: <1340289410-17642-1-git-send-email-gaofeng@cn.fujitsu.com>
On Thu, Jun 21, 2012 at 10:36:38PM +0800, Gao feng wrote:
> before commit 2c352f444ccfa966a1aa4fd8e9ee29381c467448
> (netfilter: nf_conntrack: prepare namespace support for
> l4 protocol trackers), we register sysctl before register
> protos, so if sysctl is registered faild, the protos will
> not be registered.
>
> but now, we register protos first, and when register
> sysctl failed, we can use protos too, it's different
> from before.
No, this has to be an all-or-nothing game. If one fails, everything
else that you've registered has to be unregistered.
> so change to register sysctl before register protos.
>
> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com>
> ---
> net/netfilter/nf_conntrack_proto.c | 36 +++++++++++++++++++++++-------------
> 1 files changed, 23 insertions(+), 13 deletions(-)
>
> diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
> index 1ea9194..9bd88aa 100644
> --- a/net/netfilter/nf_conntrack_proto.c
> +++ b/net/netfilter/nf_conntrack_proto.c
> @@ -253,18 +253,23 @@ int nf_conntrack_l3proto_register(struct net *net,
> {
> int ret = 0;
>
> - if (net == &init_net)
> - ret = nf_conntrack_l3proto_register_net(proto);
> + if (proto->init_net) {
> + ret = proto->init_net(net);
> + if (ret < 0)
> + return ret;
> + }
>
> + ret = nf_ct_l3proto_register_sysctl(net, proto);
> if (ret < 0)
> return ret;
This is still wrong.
If nf_ct_l3proto_register_sysctl fails, we'll leak the memory that has
been reserved by proto->init_net.
> - if (proto->init_net) {
> - ret = proto->init_net(net);
> + if (net == &init_net) {
> + ret = nf_conntrack_l3proto_register_net(proto);
> if (ret < 0)
> - return ret;
> + nf_ct_l3proto_unregister_sysctl(net, proto);
> }
> - return nf_ct_l3proto_register_sysctl(net, proto);
> +
> + return ret;
> }
> EXPORT_SYMBOL_GPL(nf_conntrack_l3proto_register);
>
> @@ -454,19 +459,24 @@ int nf_conntrack_l4proto_register(struct net *net,
> struct nf_conntrack_l4proto *l4proto)
> {
> int ret = 0;
> - if (net == &init_net)
> - ret = nf_conntrack_l4proto_register_net(l4proto);
>
> - if (ret < 0)
> - return ret;
> -
> - if (l4proto->init_net)
> + if (l4proto->init_net) {
> ret = l4proto->init_net(net);
> + if (ret < 0)
> + return ret;
> + }
>
> + ret = nf_ct_l4proto_register_sysctl(net, l4proto);
> if (ret < 0)
> return ret;
>
> - return nf_ct_l4proto_register_sysctl(net, l4proto);
> + if (net == &init_net) {
> + ret = nf_conntrack_l4proto_register_net(l4proto);
> + if (ret < 0)
> + nf_ct_l4proto_unregister_sysctl(net, l4proto);
> + }
> +
> + return ret;
> }
> EXPORT_SYMBOL_GPL(nf_conntrack_l4proto_register);
>
> --
> 1.7.7.6
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 04/13] netfilter: regard users as refcount for l4proto's per-net data
From: Pablo Neira Ayuso @ 2012-06-25 11:20 UTC (permalink / raw)
To: Gao feng; +Cc: netdev, netfilter-devel
In-Reply-To: <1340289410-17642-4-git-send-email-gaofeng@cn.fujitsu.com>
On Thu, Jun 21, 2012 at 10:36:41PM +0800, Gao feng wrote:
> Now, nf_proto_net's users is confusing.
> we should regard it as the refcount for l4proto's per-net data,
> because maybe there are two l4protos use the same per-net data.
>
> so increment pn->users when nf_conntrack_l4proto_register
> success, and decrement it for nf_conntrack_l4_unregister case.
>
> because nf_conntrack_l3proto_ipv[4|6] don't use the same per-net
> data,so we don't need to add a refcnt for their per-net data.
>
> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com>
> ---
> net/netfilter/nf_conntrack_proto.c | 76 ++++++++++++++++++++++--------------
> 1 files changed, 46 insertions(+), 30 deletions(-)
>
> diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
> index 9d6b6ab..63612e6 100644
> --- a/net/netfilter/nf_conntrack_proto.c
> +++ b/net/netfilter/nf_conntrack_proto.c
[...]
> @@ -458,23 +446,32 @@ int nf_conntrack_l4proto_register(struct net *net,
> struct nf_conntrack_l4proto *l4proto)
> {
> int ret = 0;
> + struct nf_proto_net *pn = NULL;
>
> if (l4proto->init_net) {
> ret = l4proto->init_net(net, l4proto->l3proto);
> if (ret < 0)
> - return ret;
> + goto out;
> }
>
> - ret = nf_ct_l4proto_register_sysctl(net, l4proto);
> + pn = nf_ct_l4proto_net(net, l4proto);
> + if (pn == NULL)
> + goto out;
Same thing here, we're leaking memory allocated by l4proto->init_net.
> + ret = nf_ct_l4proto_register_sysctl(net, pn, l4proto);
> if (ret < 0)
> - return ret;
> + goto out;
>
> if (net == &init_net) {
> ret = nf_conntrack_l4proto_register_net(l4proto);
> - if (ret < 0)
> - nf_ct_l4proto_unregister_sysctl(net, l4proto);
> + if (ret < 0) {
> + nf_ct_l4proto_unregister_sysctl(net, pn, l4proto);
> + goto out;
Better replace the two lines above by:
goto out_register_net;
and then...
> + }
> }
>
> + pn->users++;
out_register_net:
nf_ct_l4proto_unregister_sysctl(net, pn, l4proto);
> +out:
> return ret;
I think that this change is similar to patch 1/1, I think you should
send it as a separated patch.
> }
> EXPORT_SYMBOL_GPL(nf_conntrack_l4proto_register);
> @@ -499,10 +496,18 @@ nf_conntrack_l4proto_unregister_net(struct nf_conntrack_l4proto *l4proto)
> void nf_conntrack_l4proto_unregister(struct net *net,
> struct nf_conntrack_l4proto *l4proto)
> {
> + struct nf_proto_net *pn = NULL;
> +
> if (net == &init_net)
> nf_conntrack_l4proto_unregister_net(l4proto);
>
> - nf_ct_l4proto_unregister_sysctl(net, l4proto);
> + pn = nf_ct_l4proto_net(net, l4proto);
> + if (pn == NULL)
> + return;
> +
> + pn->users--;
> + nf_ct_l4proto_unregister_sysctl(net, pn, l4proto);
> +
> /* Remove all contrack entries for this protocol */
> rtnl_lock();
> nf_ct_iterate_cleanup(net, kill_l4proto, l4proto);
> @@ -514,11 +519,15 @@ int nf_conntrack_proto_init(struct net *net)
> {
> unsigned int i;
> int err;
> + struct nf_proto_net *pn = nf_ct_l4proto_net(net,
> + &nf_conntrack_l4proto_generic);
> +
> err = nf_conntrack_l4proto_generic.init_net(net,
> nf_conntrack_l4proto_generic.l3proto);
> if (err < 0)
> return err;
> err = nf_ct_l4proto_register_sysctl(net,
> + pn,
> &nf_conntrack_l4proto_generic);
> if (err < 0)
> return err;
> @@ -528,13 +537,20 @@ int nf_conntrack_proto_init(struct net *net)
> rcu_assign_pointer(nf_ct_l3protos[i],
> &nf_conntrack_l3proto_generic);
> }
> +
> + pn->users++;
> return 0;
> }
>
> void nf_conntrack_proto_fini(struct net *net)
> {
> unsigned int i;
> + struct nf_proto_net *pn = nf_ct_l4proto_net(net,
> + &nf_conntrack_l4proto_generic);
> +
> + pn->users--;
> nf_ct_l4proto_unregister_sysctl(net,
> + pn,
> &nf_conntrack_l4proto_generic);
> if (net == &init_net) {
> /* free l3proto protocol tables */
> --
> 1.7.7.6
>
^ permalink raw reply
* Re: [RFC net-next 05/14] Fix intel/ixgbevf
From: Yuval Mintz @ 2012-06-25 11:23 UTC (permalink / raw)
To: Greg Rose; +Cc: eilong, Alexander Duyck, netdev, davem, Jeff Kirsher
In-Reply-To: <20120619110704.000045b2@unknown>
>>>> * It's easy to be greedy for MSI-X vectors, but it
>>>> really @@ -2022,8 +2022,9 @@ static int
>>>> ixgbevf_set_interrupt_capability(struct ixgbevf_adapter *adapter)
>>>> * than CPU's. So let's be conservative and only ask for
>>>> * (roughly) twice the number of vectors as there are
>>>> CPU's. */
>>>> + ncpu = min_t(int, num_online_cpus(),
>>>> DEFAULT_MAX_NUM_RSS_QUEUES); v_budget =
>>>> min(adapter->num_rx_queues + adapter->num_tx_queues,
>>>> - (int)(num_online_cpus() * 2)) +
>>>> NON_Q_VECTORS;
>>>> + ncpu * 2) + NON_Q_VECTORS;
>>>>
>>>> /* A failure in MSI-X entry allocation isn't fatal, but
>>>> it does
>>>> * mean we disable MSI-X capabilities of the adapter. */
>>> This change is pointless on the ixgbevf driver. The VF hardware can
>>> support at most 4 RSS queues. As such num_rx_queues + num_tx_queues
>>> will never exceed 8 so you are essentially adding a necessary
>>> min(x,8).
>>
>> It is pointless with the current value, but if someone will edit the
>> kernel source code and replace the 8 with a 2, it will become
>> meaningful. The compiler will optimize this part, and I think that for
>> completion, it is best to keep this reference so a future default
>> number change will not be missed.
>>
>> Eilon
>
> I don't feel there is any real point to making this change to the
> ixgbevf driver. 82599 virtual functions have 3 MSI-X vectors, one of
> which is for the mailbox and the other two can be shared with tx/rx
> queue pairs or assigned separately to tx or rx queues. So this code is
> pointless no matter what value is set for DEFAULT_MAX_NUM_RSS_QUEUES.
> Perhaps the patches to the other drivers in your RFC will have some
> effect but this one looks like a no-op for the ixgbevf driver so there
> is no reason for it.
>
> - Greg
>
Hi Greg,
Since we're changing the RFC to use a new wrapper function which should
replace num_online_cpus (for these purpose), the next RFC version will still
change this driver (for uniformity, if nothing else).
Of course, if you would still have reservations for this change - send them.
Thanks,
Yuval
^ permalink raw reply
* [RFC net-next (v2) 02/14] mlx4: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Or Gerlitz
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx4/main.c | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index ee6f4fe..8f990a3 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -41,6 +41,7 @@
#include <linux/slab.h>
#include <linux/io-mapping.h>
#include <linux/delay.h>
+#include <linux/netdevice.h>
#include <linux/mlx4/device.h>
#include <linux/mlx4/doorbell.h>
@@ -1539,8 +1540,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev)
struct mlx4_priv *priv = mlx4_priv(dev);
struct msix_entry *entries;
int nreq = min_t(int, dev->caps.num_ports *
- min_t(int, num_online_cpus() + 1, MAX_MSIX_P_PORT)
- + MSIX_LEGACY_SZ, MAX_MSIX);
+ min_t(int, netif_get_num_default_rss_queues() + 1,
+ MAX_MSIX_P_PORT) + MSIX_LEGACY_SZ, MAX_MSIX);
int err;
int i;
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 06/14] cxgb4: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Divy Le Ray
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Divy Le Ray <divy@chelsio.com>
---
drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index e1f96fb..5ed49af 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -3493,8 +3493,8 @@ static void __devinit cfg_queues(struct adapter *adap)
*/
if (n10g)
q10g = (MAX_ETH_QSETS - (adap->params.nports - n10g)) / n10g;
- if (q10g > num_online_cpus())
- q10g = num_online_cpus();
+ if (q10g > netif_get_num_default_rss_queues())
+ q10g = netif_get_num_default_rss_queues();
for_each_port(adap, i) {
struct port_info *pi = adap2pinfo(adap, i);
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 05/14] cxgb3: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Divy Le Ray
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Divy Le Ray <divy@chelsio.com>
---
drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
index abb6ce7..9b08749 100644
--- a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
@@ -3050,7 +3050,7 @@ static struct pci_error_handlers t3_err_handler = {
static void set_nqsets(struct adapter *adap)
{
int i, j = 0;
- int num_cpus = num_online_cpus();
+ int num_cpus = netif_get_num_default_rss_queues();
int hwports = adap->params.nports;
int nqsets = adap->msix_nvectors - 1;
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 04/14] qlge: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev
Cc: eilong, Yuval Mintz, Anirban Chakraborty, Jitendra Kalsaria,
Ron Mercer
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Anirban Chakraborty <anirban.chakraborty@qlogic.com>
Cc: Jitendra Kalsaria <jitendra.kalsaria@qlogic.com>
Cc: Ron Mercer <ron.mercer@qlogic.com>
---
drivers/net/ethernet/qlogic/qlge/qlge_main.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qlge/qlge_main.c b/drivers/net/ethernet/qlogic/qlge/qlge_main.c
index 09d8d33..3c3499d 100644
--- a/drivers/net/ethernet/qlogic/qlge/qlge_main.c
+++ b/drivers/net/ethernet/qlogic/qlge/qlge_main.c
@@ -4649,7 +4649,7 @@ static int __devinit qlge_probe(struct pci_dev *pdev,
int err = 0;
ndev = alloc_etherdev_mq(sizeof(struct ql_adapter),
- min(MAX_CPUS, (int)num_online_cpus()));
+ min(MAX_CPUS, netif_get_num_default_rss_queues()));
if (!ndev)
return -ENOMEM;
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 03/14] vxge: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Jon Mason
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Jon Mason <jdmason@kudzu.us>
---
drivers/net/ethernet/neterion/vxge/vxge-main.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/neterion/vxge/vxge-main.c b/drivers/net/ethernet/neterion/vxge/vxge-main.c
index 2578eb1..2fd1edb 100644
--- a/drivers/net/ethernet/neterion/vxge/vxge-main.c
+++ b/drivers/net/ethernet/neterion/vxge/vxge-main.c
@@ -3687,7 +3687,8 @@ static int __devinit vxge_config_vpaths(
return 0;
if (!driver_config->g_no_cpus)
- driver_config->g_no_cpus = num_online_cpus();
+ driver_config->g_no_cpus =
+ netif_get_num_default_rss_queues();
driver_config->vpath_per_dev = driver_config->g_no_cpus >> 1;
if (!driver_config->vpath_per_dev)
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 00/14] default maximal number of RSS queues in mq drivers
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev
Cc: eilong, Yuval Mintz, Divy Le Ray, Or Gerlitz, Jon Mason,
Anirban Chakraborty, Jitendra Kalsaria, Ron Mercer, Jeff Kirsher,
Jon Mason, Andrew Gallatin, Sathya Perla, Subbu Seetharaman,
Ajit Khaparde, Matt Carlson, Michael Chan, Eric Dumazet,
Ben Hutchings
Different vendors support different number of RSS queues by default. Today,
there exists an ethtool API through which users can change the number of
channels their driver supports; This enables us to pursue the goal of using
a default number of RSS queues in various multi-queue drivers.
This RFC intendeds to achieve the above default, by upper-limiting the number
of interrupts multi-queue drivers request (by default, not via the new API)
with correlation to the number of cpus on the machine.
After examining multi-queue drivers that call alloc_etherdev_mq[s],
it became evident that most drivers allocate their devices using hard-coded
values. Changing those defaults directly will most likely cause a regression.
However, (most) multi-queue driver look at the number of online cpus when
requesting for interrupts. We assume that the number of interrupts the
driver manages to request is propagated across the driver, and the number
of RSS queues it configures is based upon it.
This RFC modifies said logic - if the number of cpus is large enough, use
a smaller default value instead. This serves 2 main purposes:
1. A step forward unity in the number of RSS queues of various drivers.
2. It prevents wasteful requests for interrupts on machines with many cpus.
Notice no testing was made on this RFC (other than on the bnx2x driver)
except for compilation test.
Drivers identified as multi-queue, handled in this RFC:
* mellanox mlx4
* neterion vxge
* qlogic qlge
* intel igb, igbxe igbxevf
* chelsio cxgb3, cxgb4
* myricom myri10ge
* emulex benet
* broadcom tg3, bnx2, bnx2x
Driver identified as multi-queue, no reference to number of online cpus found,
and thus unhandled in this RFC:
* neterion s2io
* marvell mv643xx
* freescale gianfar
* ibm ehea
* ti cpmac
* sun niu
* sfc efx
* chelsio cxgb4vf
Cheers,
Yuval Mintz
Cc: Divy Le Ray <divy@chelsio.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Anirban Chakraborty <anirban.chakraborty@qlogic.com>
Cc: Jitendra Kalsaria <jitendra.kalsaria@qlogic.com>
Cc: Ron Mercer <ron.mercer@qlogic.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Jon Mason <mason@myri.com>
Cc: Andrew Gallatin <gallatin@myri.com>
Cc: Sathya Perla <sathya.perla@emulex.com>
Cc: Subbu Seetharaman <subbu.seetharaman@emulex.com>
Cc: Ajit Khaparde <ajit.khaparde@emulex.com>
Cc: Matt Carlson <mcarlson@broadcom.com>
Cc: Michael Chan <mchan@broadcom.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
^ permalink raw reply
* [RFC net-next (v2) 01/14] net-next: Add netif_get_num_default_rss_queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Most multi-queue networking driver consider the number of online cpus when
configuring RSS queues.
This patch adds a wrapper to the number of cpus, setting an upper limit on the
number of cpus a driver should consider (by default) when allocating resources
for his queues.
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
---
include/linux/netdevice.h | 3 +++
net/core/dev.c | 11 +++++++++++
2 files changed, 14 insertions(+), 0 deletions(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 2c2ecea..ab0251d 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -2119,6 +2119,9 @@ static inline int netif_copy_real_num_queues(struct net_device *to_dev,
#endif
}
+#define DEFAULT_MAX_NUM_RSS_QUEUES (8)
+extern int netif_get_num_default_rss_queues(void);
+
/* Use this variant when it is known for sure that it
* is executing from hardware interrupt context or with hardware interrupts
* disabled.
diff --git a/net/core/dev.c b/net/core/dev.c
index 57c4f9b..caae49c 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -1793,6 +1793,17 @@ int netif_set_real_num_rx_queues(struct net_device *dev, unsigned int rxq)
EXPORT_SYMBOL(netif_set_real_num_rx_queues);
#endif
+/* netif_get_num_default_rss_queues - default number of RSS queues
+ *
+ * This routine should set an upper limit on the number of RSS queues
+ * used by default by multiqueue devices.
+ */
+int netif_get_num_default_rss_queues()
+{
+ return min_t(int, DEFAULT_MAX_NUM_RSS_QUEUES, num_online_cpus());
+}
+EXPORT_SYMBOL(netif_get_num_default_rss_queues);
+
static inline void __netif_reschedule(struct Qdisc *q)
{
struct softnet_data *sd;
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 08/14] tg3: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Matt Carlson
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Matt Carlson <mcarlson@broadcom.com>
---
drivers/net/ethernet/broadcom/tg3.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index e47ff8b..6cbab03 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -9908,7 +9908,7 @@ static bool tg3_enable_msix(struct tg3 *tp)
int i, rc;
struct msix_entry msix_ent[tp->irq_max];
- tp->irq_cnt = num_online_cpus();
+ tp->irq_cnt = netif_get_num_default_rss_queues();
if (tp->irq_cnt > 1) {
/* We want as many rx rings enabled as there are cpus.
* In multiqueue MSI-X mode, the first MSI-X vector
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 11/14] igb: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Jeff Kirsher
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 01ced68..b11ee60 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2465,7 +2465,8 @@ static int __devinit igb_sw_init(struct igb_adapter *adapter)
break;
}
- adapter->rss_queues = min_t(u32, max_rss_queues, num_online_cpus());
+ adapter->rss_queues = min_t(u32, max_rss_queues,
+ netif_get_num_default_rss_queues());
/* Determine if we need to pair queues. */
switch (hw->mac.type) {
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 10/14] bnx2x: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
---
drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
index daa894b..53659f3 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
@@ -822,7 +822,8 @@ static inline int bnx2x_calc_num_queues(struct bnx2x *bp)
{
return num_queues ?
min_t(int, num_queues, BNX2X_MAX_QUEUES(bp)) :
- min_t(int, num_online_cpus(), BNX2X_MAX_QUEUES(bp));
+ min_t(int, netif_get_num_default_rss_queues(),
+ BNX2X_MAX_QUEUES(bp));
}
static inline void bnx2x_clear_sge_mask_next_elems(struct bnx2x_fastpath *fp)
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 09/14] bnx2: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Michael Chan
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Michael Chan <mchan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnx2.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnx2.c b/drivers/net/ethernet/broadcom/bnx2.c
index 9b69a62..3f49285 100644
--- a/drivers/net/ethernet/broadcom/bnx2.c
+++ b/drivers/net/ethernet/broadcom/bnx2.c
@@ -6246,7 +6246,7 @@ bnx2_enable_msix(struct bnx2 *bp, int msix_vecs)
static int
bnx2_setup_int_mode(struct bnx2 *bp, int dis_msi)
{
- int cpus = num_online_cpus();
+ int cpus = netif_get_num_default_rss_queues();
int msix_vecs;
if (!bp->num_req_rx_rings)
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 07/14] myri10ge: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev; +Cc: eilong, Yuval Mintz, Jon Mason
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Jon Mason <mason@myri.com>
---
drivers/net/ethernet/myricom/myri10ge/myri10ge.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
index 90153fc..fa85cf1 100644
--- a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
+++ b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
@@ -3775,7 +3775,7 @@ static void myri10ge_probe_slices(struct myri10ge_priv *mgp)
mgp->num_slices = 1;
msix_cap = pci_find_capability(pdev, PCI_CAP_ID_MSIX);
- ncpus = num_online_cpus();
+ ncpus = netif_get_num_default_rss_queues();
if (myri10ge_max_slices == 1 || msix_cap == 0 ||
(myri10ge_max_slices == -1 && ncpus < 2))
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 14/14] ixgbevf: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev
Cc: eilong, Yuval Mintz, Jeff Kirsher, Alexander Duyck, Greg Rose
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Cc: Greg Rose <gregory.v.rose@intel.com>
---
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index f69ec42..7889644 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -2023,7 +2023,7 @@ static int ixgbevf_set_interrupt_capability(struct ixgbevf_adapter *adapter)
* (roughly) twice the number of vectors as there are CPU's.
*/
v_budget = min(adapter->num_rx_queues + adapter->num_tx_queues,
- (int)(num_online_cpus() * 2)) + NON_Q_VECTORS;
+ netif_get_num_default_rss_queues() * 2) + NON_Q_VECTORS;
/* A failure in MSI-X entry allocation isn't fatal, but it does
* mean we disable MSI-X capabilities of the adapter. */
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 12/14] ixgbe: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev
Cc: eilong, Yuval Mintz, Jeff Kirsher, John Fastabend,
Alexander Duyck
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: John Fastabend <john.r.fastabend@intel.com>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c
index af1a531..b352ea8 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c
@@ -802,7 +802,7 @@ static int ixgbe_set_interrupt_capability(struct ixgbe_adapter *adapter)
* The default is to use pairs of vectors.
*/
v_budget = max(adapter->num_rx_queues, adapter->num_tx_queues);
- v_budget = min_t(int, v_budget, num_online_cpus());
+ v_budget = min_t(int, v_budget, netif_get_num_default_rss_queues());
v_budget += NON_Q_VECTORS;
/*
--
1.7.9.rc2
^ permalink raw reply related
* [RFC net-next (v2) 13/14] be2net: set maximal number of default RSS queues
From: Yuval Mintz @ 2012-06-25 11:45 UTC (permalink / raw)
To: davem, netdev
Cc: eilong, Yuval Mintz, Sathya Perla, Subbu Seetharaman,
Ajit Khaparde
In-Reply-To: <1340624745-8650-1-git-send-email-yuvalmin@broadcom.com>
Signed-off-by: Yuval Mintz <yuvalmin@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Cc: Sathya Perla <sathya.perla@emulex.com>
Cc: Subbu Seetharaman <subbu.seetharaman@emulex.com>
Cc: Ajit Khaparde <ajit.khaparde@emulex.com>
---
drivers/net/ethernet/emulex/benet/be_main.c | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index 5a34503..a8564d0 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -2142,12 +2142,14 @@ static void be_msix_disable(struct be_adapter *adapter)
static uint be_num_rss_want(struct be_adapter *adapter)
{
+ u32 num = 0;
if ((adapter->function_caps & BE_FUNCTION_CAPS_RSS) &&
!sriov_want(adapter) && be_physfn(adapter) &&
- !be_is_mc(adapter))
- return (adapter->be3_native) ? BE3_MAX_RSS_QS : BE2_MAX_RSS_QS;
- else
- return 0;
+ !be_is_mc(adapter)) {
+ num = (adapter->be3_native) ? BE3_MAX_RSS_QS : BE2_MAX_RSS_QS;
+ num = min_t(u32, num, (u32)netif_get_num_default_rss_queues());
+ }
+ return num;
}
static void be_msix_enable(struct be_adapter *adapter)
--
1.7.9.rc2
^ permalink raw reply related
* [net-next RFC V3 0/6] Multiqueue support in tun/tap
From: Jason Wang @ 2012-06-25 11:59 UTC (permalink / raw)
To: mst, akong, habanero, tahm, haixiao, jwhan, ernesto.martin,
mashirle, davem, netdev, linux-kernel, krkumar2
Cc: shemminger, edumazet, Jason Wang
In-Reply-To: <20120625060830.6765.27584.stgit@amd-6168-8-1.englab.nay.redhat.com>
Hello All:
This is an update of multiqueue support in tun/tap from V2. Please consider to
merge.
The main idea for this series is to let tun/tap device to benefit from
multiqueue network cards and multi-cores host by letting it to be able to
transmit and receive packets from mmultiple sockets/queues. This series allows
multiple sockets to be attached and detached to the tun/tap devices. Userspace
could utilize this parallism to achiveve higher throughput.
Some quick overview of the design:
- Moving socket from tun_device to tun_file.
- Allowing multiple sockets to be attached to a tun/tap devices.
- Using RCU to synchronize the data path and system call.
- A simple hash based queue selecting algorithm is used to choose the tx queue.
- Two new ioctls were added for the usespace to attach and detach socket to the device.
- ABI compatibility were maintained, userspace that only use one queue won't
need any changes.
Performance test:
This series were originally designed to serve as the backend of the multiqueue
virito-net in kvm guest. But the design is generic enough to let it to be reused
by any other type of userspace.
Since I would post a series of multiqueue virtio-net as RFC, so I would post the
performance result in that thread. To summarize the performance, the multiqueue
improves the transaction in the TCP_RR test but have some regression in small
packets transmission in TCP_STREAM test.
Martin test the multiqueue tap for their userspace, and he see an improvement in
terms of packets per second.
References:
- V2 of multiqueue tun/tap, http://lwn.net/Articles/459270/
- V1 of multiqueue tun/tap, http://www.mail-archive.com/kvm@vger.kernel.org/msg59479.html
Changes from V2:
- Rebase to the latest net-next
- Fix netdev leak when tun_attach fails
- Fix return value of TUNSETOWNER
- Purge the receive queue in socket destructor
- Enable multiqueue tun (V1 and V2 only allows mq to be eanbled for tap
- Add per-queue u64 statistics
- Fix wrong BUG_ON() check in tun_detach()
- Check numqueues instead of tfile[0] in tun_set_iff() to let tunctl -d works
correctly
- Set numqueues to MAX_TAP_QUEUES during tun_detach_all() to prevent the
attaching.
Changes from V1:
- Simplify the sockets array management by not leaving NULL in the slot.
- Optimization on the tx queue selecting.
- Fix the bug in tun_deatch_all()
Jason Wang (6):
tuntap: move socket to tun_file
tuntap: categorize ioctl
tuntap: introduce multiqueue flags
tuntap: multiqueue support
tuntap: per queue 64 bit stats
tuntap: add ioctls to attach or detach a file form tuntap device
drivers/net/tun.c | 797 ++++++++++++++++++++++++++++++------------------
include/linux/if_tun.h | 5 +
2 files changed, 503 insertions(+), 299 deletions(-)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox