* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20 22:30 UTC (permalink / raw)
To: Andi Kleen; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <87aayg539l.fsf@basil.nowhere.org>
Andi Kleen a écrit :
>
> Do you have numbers on this? It seems like a lot of effort to avoid transfering
> a few cache lines.
Lot of efforts ? hmm...
>
> -Andi (who is a bit sceptical and would rather see generic work for this in slab)
>
Yes, I know, but slab/slub is already quite optimized :)
Coding XPS was more a way to try to help push RPS in, I did not benchmarked it BTW.
My new dev machine is up, with two E5530 cpus and a 82599EB 10-Gigabit dual port card :
Pretty amazing :=)
^ permalink raw reply
* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: David Miller @ 2009-11-20 22:32 UTC (permalink / raw)
To: eric.dumazet; +Cc: therbert, netdev
In-Reply-To: <4B05D8DC.7020907@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Fri, 20 Nov 2009 00:46:36 +0100
> Goal of XPS is to free TX completed skbs by the cpu that submitted
> the transmit.
>
> Because I chose to union skb->iif with skb->sending_cpu, I chose
> to introduce a new xps_consume_skb(skb), and not generalize consume_skb() itself.
>
> This means that selected drivers must use new function to benefit from XPS
>
> Preliminary tests are quite good, especially on NUMA machines.
>
> Only NAPI drivers can use this new infrastructure (xps_consume_skb() cannot
> be called from hardirq context, only from softirq)
>
> I converted tg3 and pktgen for my tests
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
I like this work. But as you know it still needs a little bit
more work :-)
Let's also pick a more decent name for the free function since
tons of drivers are going to call this thing. How about
dev_kfree_tx_skb()? :-)
I see Jarek and you have come to a mutual understanding about the
locking. Since you need to change it anyways to fix the deadlock,
what using a netchannel like scheme to do remote SKB queueing?
PAGE_SIZE queue arrays, lockless access to head and tail pointers, and
if queue is full we local free.
I think that's a reasonable policy and the only detail to work out is
to make sure we never race on the IPI send and thus miss processing
the queue.
What do you think?
^ permalink raw reply
* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: David Miller @ 2009-11-20 22:34 UTC (permalink / raw)
To: eric.dumazet; +Cc: jarkao2, therbert, netdev
In-Reply-To: <4B070BAC.1090406@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Fri, 20 Nov 2009 22:35:40 +0100
> Jarek Poplawski a écrit :
>>
>> There is one more problem: this will break things like act_mirred + ifb,
>> and other cases using skb->iif e.g. for filtering on virtual devices at
>> the xmit path.
>>
>
> Following patch might help us to locate real uses of this obscure field :)
>
> [PATCH net-next-2.6] net: rename skb->iif to skb->skb_iif
>
> To help grep games, rename iif to skb_iif
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied, thanks Eric.
^ permalink raw reply
* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20 22:36 UTC (permalink / raw)
To: David Miller; +Cc: therbert, netdev
In-Reply-To: <20091120.143218.252563862.davem@davemloft.net>
David Miller a écrit :
>
> I like this work. But as you know it still needs a little bit
> more work :-)
>
> Let's also pick a more decent name for the free function since
> tons of drivers are going to call this thing. How about
> dev_kfree_tx_skb()? :-)
>
> I see Jarek and you have come to a mutual understanding about the
> locking. Since you need to change it anyways to fix the deadlock,
> what using a netchannel like scheme to do remote SKB queueing?
>
> PAGE_SIZE queue arrays, lockless access to head and tail pointers, and
> if queue is full we local free.
>
> I think that's a reasonable policy and the only detail to work out is
> to make sure we never race on the IPI send and thus miss processing
> the queue.
>
> What do you think?
>
Thats good ideas David, I'll work on them next week, and do benchmarks as well
before sending a new version.
Thanks
^ permalink raw reply
* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Andi Kleen @ 2009-11-20 22:37 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Andi Kleen, David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <4B07188C.2070808@gmail.com>
On Fri, Nov 20, 2009 at 11:30:36PM +0100, Eric Dumazet wrote:
> Andi Kleen a écrit :
> >
> > Do you have numbers on this? It seems like a lot of effort to avoid transfering
> > a few cache lines.
>
> Lot of efforts ? hmm...
Well lots of code at least. Perhaps I'm old fashioned, but I always like to have
each code justify its complexity.
It seems like a very narrow special case. Or perhaps this is something that
should be a general library function for all allocator users?
> Yes, I know, but slab/slub is already quite optimized :)
Well it still has a lot of problems, other benchmarks suffer too.
-Andi
--
ak@linux.intel.com -- Speaking for myself only.
^ permalink raw reply
* Re: [PATCH 1/2] rps: core implementation
From: David Miller @ 2009-11-20 22:52 UTC (permalink / raw)
To: jarkao2; +Cc: therbert, netdev
In-Reply-To: <20091119080831.GA6874@ff.dom.local>
From: Jarek Poplawski <jarkao2@gmail.com>
Date: Thu, 19 Nov 2009 08:08:31 +0000
> On 16-11-2009 17:43, Tom Herbert wrote:
>> NET_RPS_SOFTIRQ is intended to provide coalescing of IPIs.
>
> It seems calling net_rps_action() at the end of net_rx_action() should
> do (mostly) the same, at least for napi drivers. And I'm not sure it's
> worth to add a new softirq because of non-napis.
I agree. This is how we handle all of these kinds of issues.
And with GRO, any arguable latency problem this may have you're
going to eat most of the time anyways.
^ permalink raw reply
* Question about IP_TOS and DSCP handling
From: Philip Prindeville @ 2009-11-20 22:55 UTC (permalink / raw)
To: netdev
I'm looking at net/ipv4/ip_sockglue.c circa line 512 (this is in 2.6.27.38):
case IP_TOS: /* This sets both TOS and Precedence */
if (sk->sk_type == SOCK_STREAM) {
val &= ~3;
val |= inet->tos & 3;
}
if (inet->tos != val) {
inet->tos = val;
sk->sk_priority = rt_tos2priority(val);
sk_dst_reset(sk);
}
break;
and it doesn't look like the kernel handles DSCP marking. Is that
correct? (Also, not clear why SOCK_STREAM gets handled differently...)
Looking at rt_tos2priority(), which uses IPTOS_TOS(x), it doesn't look
like that would be the correct thing to do.
Also, include/linux/ip.h no longer meshes with
/usr/include/netinet/in.h. Please see:
http://sourceware.org/bugzilla/show_bug.cgi?id=10789
I'm trying to add DSCP marking to Thunderbird, Firefox, Apache, and a
few other applications but I wanted to make sure that my test platforms
would handle it correctly first.
Thanks,
-Philip
^ permalink raw reply
* [PATCH v4 1/1] rps: core implementation
From: Tom Herbert @ 2009-11-20 23:28 UTC (permalink / raw)
To: David Miller, Linux Netdev List
Version 4 of RPS patch. I think this addresses most of the comments
on the previous versions. Thanks for all the insightful comments and
patience!
Changes from previous version:
- Added rxhash to kernel-doc for struct sk_buff
- Removed /** on comments not for kernel-doc
- Change get_token declaration to kernel style
- Added struct dev_rps_maps. Each netdevice now has a pointer to this
structure which contains the array of per NAPI rps maps, the number of
this maps. rcu is used to protect pointer
- In store_rps_cpus a new map set is allocated each call.
- Removed dev_isalive check and other locks since rps struct in
netdevice are protected by rcu
- Removed NET_RPS_SOFTIRQ and call net_rps_action from net_rx_action instead
- Queue to remote backlog queues only in NAPI case. This means
rps_remote_softirq_cpus does not need to be accessed with interrupts
disabled and __smp_call_function_single will not be called with
interrupts disabled
- Limit the number of entries in an rps map to min(256, num_possible_cpus())
- Removed unnecessary irq_local_disable
- Renamed skb_tx_hashrnd to just hashrnd and use that for the rps hash as well
- Make index u16 in index=skb_get_rx_queue() and don't check index<0 now
- Replace spin_lock_irqsave with simpler spin_lock_irq in process_backlog
Signed-off-by: Tom Herbert <therbert@google.com>
---
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 97873e3..9b84a38 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -676,6 +676,29 @@ struct net_device_ops {
};
/*
+ * Structure for Receive Packet Steering. Length of map and array of CPU ID's.
+ */
+struct rps_map {
+ int len;
+ u16 map[0];
+};
+
+/*
+ * Structure that contains the rps maps for various NAPI instances of a device.
+ */
+struct dev_rps_maps {
+ int num_maps;
+ struct rcu_head rcu;
+ struct rps_map maps[0];
+};
+
+/* Bound number of CPUs that can be in an rps map */
+#define MAX_RPS_CPUS (num_possible_cpus() < 256 ? num_possible_cpus() : 256)
+
+/* Maximum size of RPS map (for allocation) */
+#define RPS_MAP_SIZE (sizeof(struct rps_map) + (MAX_RPS_CPUS * sizeof(u16)))
+
+/*
* The DEVICE structure.
* Actually, this whole structure is a big mistake. It mixes I/O
* data with strictly "high-level" data, and it has to know about
@@ -861,6 +884,9 @@ struct net_device {
struct netdev_queue rx_queue;
+ struct dev_rps_maps *dev_rps_maps; /* Per-NAPI maps for
+ receive packet steeing */
+
struct netdev_queue *_tx ____cacheline_aligned_in_smp;
/* Number of TX queues allocated at alloc_netdev_mq() time */
@@ -1276,6 +1302,9 @@ struct softnet_data {
struct Qdisc *output_queue;
struct sk_buff_head input_pkt_queue;
struct list_head poll_list;
+
+ struct call_single_data csd;
+
struct sk_buff *completion_queue;
struct napi_struct backlog;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 63f4742..f188301 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -267,6 +267,7 @@ typedef unsigned char *sk_buff_data_t;
* @mac_header: Link layer header
* @_skb_dst: destination entry
* @sp: the security path, used for xfrm
+ * @rxhash: the packet hash computed on receive
* @cb: Control buffer. Free for use by every layer. Put private vars here
* @len: Length of actual data
* @data_len: Data length
@@ -323,6 +324,8 @@ struct sk_buff {
#ifdef CONFIG_XFRM
struct sec_path *sp;
#endif
+ __u32 rxhash;
+
/*
* This is the control buffer. It is free to use for every
* layer. Please put your private variables there. If you
diff --git a/net/core/dev.c b/net/core/dev.c
index 9977288..f2c199c 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -1834,7 +1834,7 @@ out_kfree_skb:
return rc;
}
-static u32 skb_tx_hashrnd;
+static u32 hashrnd;
u16 skb_tx_hash(const struct net_device *dev, const struct sk_buff *skb)
{
@@ -1852,7 +1852,7 @@ u16 skb_tx_hash(const struct net_device *dev,
const struct sk_buff *skb)
else
hash = skb->protocol;
- hash = jhash_1word(hash, skb_tx_hashrnd);
+ hash = jhash_1word(hash, hashrnd);
return (u16) (((u64) hash * dev->real_num_tx_queues) >> 32);
}
@@ -2073,6 +2073,146 @@ int weight_p __read_mostly = 64; /*
old backlog weight */
DEFINE_PER_CPU(struct netif_rx_stats, netdev_rx_stat) = { 0, };
+/*
+ * get_rps_cpu is called from netif_receive_skb and returns the target
+ * CPU from the RPS map of the receiving NAPI instance for a given skb.
+ */
+static int get_rps_cpu(struct net_device *dev, struct sk_buff *skb)
+{
+ u32 addr1, addr2, ports;
+ struct ipv6hdr *ip6;
+ struct iphdr *ip;
+ u32 hash, ihl;
+ u8 ip_proto;
+ int cpu = -1;
+ struct dev_rps_maps *drmap;
+ struct rps_map *map = NULL;
+ u16 index;
+
+ rcu_read_lock();
+
+ drmap = rcu_dereference(dev->dev_rps_maps);
+ if (!drmap)
+ goto done;
+
+ index = skb_get_rx_queue(skb);
+ if (index >= drmap->num_maps)
+ index = 0;
+
+ map = (struct rps_map *)
+ ((void *)drmap->maps + (RPS_MAP_SIZE * index));
+ if (!map->len)
+ goto done;
+
+ hash = skb->rxhash;
+ if (hash)
+ goto got_hash; /* Skip hash computation on packet header */
+
+ switch (skb->protocol) {
+ case __constant_htons(ETH_P_IP):
+ if (!pskb_may_pull(skb, sizeof(*ip)))
+ goto done;
+
+ ip = (struct iphdr *) skb->data;
+ ip_proto = ip->protocol;
+ addr1 = ip->saddr;
+ addr2 = ip->daddr;
+ ihl = ip->ihl;
+ break;
+ case __constant_htons(ETH_P_IPV6):
+ if (!pskb_may_pull(skb, sizeof(*ip6)))
+ goto done;
+
+ ip6 = (struct ipv6hdr *) skb->data;
+ ip_proto = ip6->nexthdr;
+ addr1 = ip6->saddr.s6_addr32[3];
+ addr2 = ip6->daddr.s6_addr32[3];
+ ihl = (40 >> 2);
+ break;
+ default:
+ goto done;
+ }
+ ports = 0;
+ switch (ip_proto) {
+ case IPPROTO_TCP:
+ case IPPROTO_UDP:
+ case IPPROTO_DCCP:
+ case IPPROTO_ESP:
+ case IPPROTO_AH:
+ case IPPROTO_SCTP:
+ case IPPROTO_UDPLITE:
+ if (pskb_may_pull(skb, (ihl * 4) + 4))
+ ports = *((u32 *) (skb->data + (ihl * 4)));
+ break;
+
+ default:
+ break;
+ }
+
+ hash = jhash_3words(addr1, addr2, ports, hashrnd);
+
+got_hash:
+ cpu = map->map[((u64) hash * map->len) >> 32];
+ if (!cpu_online(cpu))
+ cpu = -1;
+done:
+ rcu_read_unlock();
+ return cpu;
+}
+
+static DEFINE_PER_CPU(cpumask_t, rps_remote_softirq_cpus);
+
+/* Called from hardirq (IPI) context */
+static void trigger_softirq(void *data)
+{
+ struct softnet_data *queue = data;
+ __napi_schedule(&queue->backlog);
+}
+
+/*
+ * enqueue_to_backlog is called to queue an skb to a per CPU backlog
+ * queue (may be a remote CPU queue).
+ */
+static int enqueue_to_backlog(struct sk_buff *skb, int cpu)
+{
+ struct softnet_data *queue;
+ unsigned long flags;
+
+ queue = &per_cpu(softnet_data, cpu);
+
+ local_irq_save(flags);
+ __get_cpu_var(netdev_rx_stat).total++;
+
+ spin_lock(&queue->input_pkt_queue.lock);
+ if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
+ if (queue->input_pkt_queue.qlen) {
+enqueue:
+ __skb_queue_tail(&queue->input_pkt_queue, skb);
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock,
+ flags);
+ return NET_RX_SUCCESS;
+ }
+
+ /* Schedule NAPI for backlog device */
+ if (napi_schedule_prep(&queue->backlog)) {
+ if (cpu != smp_processor_id()) {
+ cpu_set(cpu,
+ get_cpu_var(rps_remote_softirq_cpus));
+ __raise_softirq_irqoff(NET_RX_SOFTIRQ);
+ } else
+ __napi_schedule(&queue->backlog);
+ }
+ goto enqueue;
+ }
+
+ spin_unlock(&queue->input_pkt_queue.lock);
+
+ __get_cpu_var(netdev_rx_stat).dropped++;
+ local_irq_restore(flags);
+
+ kfree_skb(skb);
+ return NET_RX_DROP;
+}
/**
* netif_rx - post buffer to the network code
@@ -2091,9 +2231,6 @@ DEFINE_PER_CPU(struct netif_rx_stats,
netdev_rx_stat) = { 0, };
int netif_rx(struct sk_buff *skb)
{
- struct softnet_data *queue;
- unsigned long flags;
-
/* if netpoll wants it, pretend we never saw it */
if (netpoll_rx(skb))
return NET_RX_DROP;
@@ -2101,31 +2238,8 @@ int netif_rx(struct sk_buff *skb)
if (!skb->tstamp.tv64)
net_timestamp(skb);
- /*
- * The code is rearranged so that the path is the most
- * short when CPU is congested, but is still operating.
- */
- local_irq_save(flags);
- queue = &__get_cpu_var(softnet_data);
-
- __get_cpu_var(netdev_rx_stat).total++;
- if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
- if (queue->input_pkt_queue.qlen) {
-enqueue:
- __skb_queue_tail(&queue->input_pkt_queue, skb);
- local_irq_restore(flags);
- return NET_RX_SUCCESS;
- }
-
- napi_schedule(&queue->backlog);
- goto enqueue;
- }
-
- __get_cpu_var(netdev_rx_stat).dropped++;
- local_irq_restore(flags);
-
- kfree_skb(skb);
- return NET_RX_DROP;
+ /* Always send to local backlog, no RPS for non NAPI */
+ return enqueue_to_backlog(skb, smp_processor_id());
}
EXPORT_SYMBOL(netif_rx);
@@ -2363,10 +2477,10 @@ void netif_nit_deliver(struct sk_buff *skb)
}
/**
- * netif_receive_skb - process receive buffer from network
+ * __netif_receive_skb - process receive buffer from network
* @skb: buffer to process
*
- * netif_receive_skb() is the main receive data processing function.
+ * __netif__napireceive_skb() is the main receive data processing function.
* It always succeeds. The buffer may be dropped during processing
* for congestion control or by the protocol layers.
*
@@ -2377,7 +2491,8 @@ void netif_nit_deliver(struct sk_buff *skb)
* NET_RX_SUCCESS: no congestion
* NET_RX_DROP: packet was dropped
*/
-int netif_receive_skb(struct sk_buff *skb)
+
+int __netif_receive_skb(struct sk_buff *skb)
{
struct packet_type *ptype, *pt_prev;
struct net_device *orig_dev;
@@ -2475,6 +2590,16 @@ out:
}
EXPORT_SYMBOL(netif_receive_skb);
+int netif_receive_skb(struct sk_buff *skb)
+{
+ int cpu = get_rps_cpu(skb->dev, skb);
+
+ if (cpu < 0)
+ return __netif_receive_skb(skb);
+ else
+ return enqueue_to_backlog(skb, cpu);
+}
+
/* Network device is going away, flush any packets still pending */
static void flush_backlog(void *arg)
{
@@ -2518,7 +2643,7 @@ static int napi_gro_complete(struct sk_buff *skb)
}
out:
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
}
void napi_gro_flush(struct napi_struct *napi)
@@ -2650,7 +2775,7 @@ gro_result_t napi_skb_finish(gro_result_t ret,
struct sk_buff *skb)
{
switch (ret) {
case GRO_NORMAL:
- if (netif_receive_skb(skb))
+ if (__netif_receive_skb(skb))
ret = GRO_DROP;
break;
@@ -2724,7 +2849,7 @@ gro_result_t napi_frags_finish(struct
napi_struct *napi, struct sk_buff *skb,
if (ret == GRO_HELD)
skb_gro_pull(skb, -ETH_HLEN);
- else if (netif_receive_skb(skb))
+ else if (__netif_receive_skb(skb))
ret = GRO_DROP;
break;
@@ -2799,16 +2924,16 @@ static int process_backlog(struct napi_struct
*napi, int quota)
do {
struct sk_buff *skb;
- local_irq_disable();
+ spin_lock_irq(&queue->input_pkt_queue.lock);
skb = __skb_dequeue(&queue->input_pkt_queue);
if (!skb) {
__napi_complete(napi);
- local_irq_enable();
+ spin_unlock_irq(&queue->input_pkt_queue.lock);
break;
}
- local_irq_enable();
+ spin_unlock_irq(&queue->input_pkt_queue.lock);
- netif_receive_skb(skb);
+ __netif_receive_skb(skb);
} while (++work < quota && jiffies == start_time);
return work;
@@ -2897,6 +3022,21 @@ void netif_napi_del(struct napi_struct *napi)
}
EXPORT_SYMBOL(netif_napi_del);
+/*
+ * net_rps_action sends any pending IPI's for rps. This is only called from
+ * softirq and interrupts must be enabled.
+ */
+static void net_rps_action(void)
+{
+ int cpu;
+
+ /* Send pending IPI's to kick RPS processing on remote cpus. */
+ for_each_cpu_mask_nr(cpu, __get_cpu_var(rps_remote_softirq_cpus)) {
+ struct softnet_data *queue = &per_cpu(softnet_data, cpu);
+ __smp_call_function_single(cpu, &queue->csd, 0);
+ }
+ cpus_clear(__get_cpu_var(rps_remote_softirq_cpus));
+}
static void net_rx_action(struct softirq_action *h)
{
@@ -2968,6 +3108,8 @@ static void net_rx_action(struct softirq_action *h)
out:
local_irq_enable();
+ net_rps_action();
+
#ifdef CONFIG_NET_DMA
/*
* There may not be any more sk_buffs coming right now, so push
@@ -5341,6 +5483,8 @@ void free_netdev(struct net_device *dev)
/* Flush device addresses */
dev_addr_flush(dev);
+ kfree(dev->dev_rps_maps);
+
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
netif_napi_del(p);
@@ -5793,6 +5937,10 @@ static int __init net_dev_init(void)
queue->completion_queue = NULL;
INIT_LIST_HEAD(&queue->poll_list);
+ queue->csd.func = trigger_softirq;
+ queue->csd.info = queue;
+ queue->csd.flags = 0;
+
queue->backlog.poll = process_backlog;
queue->backlog.weight = weight_p;
queue->backlog.gro_list = NULL;
@@ -5831,7 +5979,7 @@ subsys_initcall(net_dev_init);
static int __init initialize_hashrnd(void)
{
- get_random_bytes(&skb_tx_hashrnd, sizeof(skb_tx_hashrnd));
+ get_random_bytes(&hashrnd, sizeof(hashrnd));
return 0;
}
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 157645c..be78dfb 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -18,6 +18,9 @@
#include <linux/wireless.h>
#include <net/wext.h>
+#include <linux/string.h>
+#include <linux/ctype.h>
+
#include "net-sysfs.h"
#ifdef CONFIG_SYSFS
@@ -253,6 +256,132 @@ static ssize_t store_tx_queue_len(struct device *dev,
return netdev_store(dev, attr, buf, len, change_tx_queue_len);
}
+static char *get_token(const char **cp, size_t *len)
+{
+ const char *bp = *cp;
+ char *start;
+
+ while (isspace(*bp))
+ bp++;
+
+ start = (char *)bp;
+ while (!isspace(*bp) && *bp != '\0')
+ bp++;
+
+ if (start != bp)
+ *len = bp - start;
+ else
+ start = NULL;
+
+ *cp = bp;
+ return start;
+}
+
+static void dev_map_release(struct rcu_head *rcu)
+{
+ struct dev_rps_maps *drmap =
+ container_of(rcu, struct dev_rps_maps, rcu);
+
+ kfree(drmap);
+}
+
+static ssize_t store_rps_cpus(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t len)
+{
+ struct net_device *net = to_net_dev(dev);
+ struct napi_struct *napi;
+ cpumask_t mask;
+ int err, cpu, index, i;
+ int cnt = 0;
+ char *token;
+ const char *cp = buf;
+ size_t tlen;
+ struct dev_rps_maps *drmap, *old_drmap;
+
+ if (!capable(CAP_NET_ADMIN))
+ return -EPERM;
+
+ cnt = 0;
+ list_for_each_entry(napi, &net->napi_list, dev_list)
+ cnt++;
+
+ drmap = kzalloc(sizeof (struct dev_rps_maps) +
+ RPS_MAP_SIZE * cnt, GFP_KERNEL);
+ if (!drmap)
+ return -ENOMEM;
+
+ drmap->num_maps = cnt;
+
+ cp = buf;
+ for (index = 0; index < cnt &&
+ (token = get_token(&cp, &tlen)); index++) {
+ struct rps_map *map = (struct rps_map *)
+ ((void *)drmap->maps + (RPS_MAP_SIZE * index));
+ err = bitmap_parse(token, tlen, cpumask_bits(&mask),
+ nr_cpumask_bits);
+
+ if (err) {
+ kfree(drmap);
+ return err;
+ }
+
+ cpus_and(mask, mask, cpu_online_map);
+ i = 0;
+ for_each_cpu_mask(cpu, mask) {
+ if (i >= MAX_RPS_CPUS)
+ break;
+ map->map[i++] = cpu;
+ }
+ map->len = i;
+ }
+
+ rcu_read_lock_bh();
+ old_drmap = rcu_dereference(net->dev_rps_maps);
+ rcu_assign_pointer(net->dev_rps_maps, drmap);
+ rcu_read_unlock_bh();
+
+ if (old_drmap)
+ call_rcu(&old_drmap->rcu, dev_map_release);
+
+ return len;
+}
+
+static ssize_t show_rps_cpus(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct net_device *net = to_net_dev(dev);
+ size_t len = 0;
+ cpumask_t mask;
+ int i, j;
+ struct dev_rps_maps *drmap;
+
+ rcu_read_lock_bh();
+ drmap = rcu_dereference(net->dev_rps_maps);
+
+ if (drmap) {
+ for (j = 0; j < drmap->num_maps; j++) {
+ struct rps_map *map = (struct rps_map *)
+ ((void *)drmap->maps + (RPS_MAP_SIZE * j));
+ cpus_clear(mask);
+ for (i = 0; i < map->len; i++)
+ cpu_set(map->map[i], mask);
+
+ len += cpumask_scnprintf(buf + len, PAGE_SIZE, &mask);
+ if (PAGE_SIZE - len < 3) {
+ rcu_read_unlock();
+ return -EINVAL;
+ }
+ if (j < drmap->num_maps)
+ len += sprintf(buf + len, " ");
+ }
+ }
+
+ rcu_read_unlock_bh();
+
+ len += sprintf(buf + len, "\n");
+ return len;
+}
+
static ssize_t store_ifalias(struct device *dev, struct device_attribute *attr,
const char *buf, size_t len)
{
@@ -309,6 +438,7 @@ static struct device_attribute net_class_attributes[] = {
__ATTR(flags, S_IRUGO | S_IWUSR, show_flags, store_flags),
__ATTR(tx_queue_len, S_IRUGO | S_IWUSR, show_tx_queue_len,
store_tx_queue_len),
+ __ATTR(rps_cpus, S_IRUGO | S_IWUSR, show_rps_cpus, store_rps_cpus),
{}
};
^ permalink raw reply related
* Re: [PATCH v4 1/1] rps: core implementation
From: David Miller @ 2009-11-20 23:39 UTC (permalink / raw)
To: therbert; +Cc: netdev
In-Reply-To: <65634d660911201528k5a07135el471b65fff9dd7c9d@mail.gmail.com>
From: Tom Herbert <therbert@google.com>
Date: Fri, 20 Nov 2009 15:28:58 -0800
> + /* Schedule NAPI for backlog device */
> + if (napi_schedule_prep(&queue->backlog)) {
> + if (cpu != smp_processor_id()) {
> + cpu_set(cpu,
> + get_cpu_var(rps_remote_softirq_cpus));
> + __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> + } else
> + __napi_schedule(&queue->backlog);
> + }
> + goto enqueue;
> + }
I still think, like Jared, this should occur at the end of the NAPI
->poll() run.
Otherwise show us numbers indicating that it makes a big difference.
:-)
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: Stephen Hemminger @ 2009-11-20 23:40 UTC (permalink / raw)
To: Tom Herbert; +Cc: David Miller, Linux Netdev List
In-Reply-To: <65634d660911201528k5a07135el471b65fff9dd7c9d@mail.gmail.com>
On Fri, 20 Nov 2009 15:28:58 -0800
Tom Herbert <therbert@google.com> wrote:
> +static char *get_token(const char **cp, size_t *len)
> +{
> + const char *bp = *cp;
> + char *start;
> +
> + while (isspace(*bp))
> + bp++;
> +
> + start = (char *)bp;
> + while (!isspace(*bp) && *bp != '\0')
> + bp++;
> +
> + if (start != bp)
> + *len = bp - start;
> + else
> + start = NULL;
> +
> + *cp = bp;
> + return start;
> +}
Sysfs is intentionally one value per file. If you need multiple
values, then it is the wrong interface.
--
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: Stephen Hemminger @ 2009-11-20 23:42 UTC (permalink / raw)
To: Tom Herbert; +Cc: David Miller, Linux Netdev List
In-Reply-To: <65634d660911201528k5a07135el471b65fff9dd7c9d@mail.gmail.com>
On Fri, 20 Nov 2009 15:28:58 -0800
Tom Herbert <therbert@google.com> wrote:
> @@ -861,6 +884,9 @@ struct net_device {
>
> struct netdev_queue rx_queue;
>
> + struct dev_rps_maps *dev_rps_maps; /* Per-NAPI maps for
> + receive packet steeing */
> +
How does this work for devices with:
* multiqueue - one device has multiple NAPI instances
* mulitport - one NAPI shared by multiple devices
--
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: Tom Herbert @ 2009-11-20 23:50 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20091120.153913.242783116.davem@davemloft.net>
On Fri, Nov 20, 2009 at 3:39 PM, David Miller <davem@davemloft.net> wrote:
> From: Tom Herbert <therbert@google.com>
> Date: Fri, 20 Nov 2009 15:28:58 -0800
>
>> + /* Schedule NAPI for backlog device */
>> + if (napi_schedule_prep(&queue->backlog)) {
>> + if (cpu != smp_processor_id()) {
>> + cpu_set(cpu,
>> + get_cpu_var(rps_remote_softirq_cpus));
>> + __raise_softirq_irqoff(NET_RX_SOFTIRQ);
>> + } else
>> + __napi_schedule(&queue->backlog);
>> + }
>> + goto enqueue;
>> + }
>
> I still think, like Jared, this should occur at the end of the NAPI
> ->poll() run.
>
We only set the bit in remote_softirq_cpus in here. The actual IPIs
are sent at the end net_rx_action. I'm not exactly sure what you're
thinking on this...
> Otherwise show us numbers indicating that it makes a big difference.
> :-)
>
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: Tom Herbert @ 2009-11-20 23:53 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David Miller, Linux Netdev List
In-Reply-To: <20091120154046.67252d23@nehalam>
On Fri, Nov 20, 2009 at 3:40 PM, Stephen Hemminger
<shemminger@vyatta.com> wrote:
> On Fri, 20 Nov 2009 15:28:58 -0800
> Tom Herbert <therbert@google.com> wrote:
>
>> +static char *get_token(const char **cp, size_t *len)
>> +{
>> + const char *bp = *cp;
>> + char *start;
>> +
>> + while (isspace(*bp))
>> + bp++;
>> +
>> + start = (char *)bp;
>> + while (!isspace(*bp) && *bp != '\0')
>> + bp++;
>> +
>> + if (start != bp)
>> + *len = bp - start;
>> + else
>> + start = NULL;
>> +
>> + *cp = bp;
>> + return start;
>> +}
>
>
> Sysfs is intentionally one value per file. If you need multiple
> values, then it is the wrong interface.
>
What would be a better interface?
> --
>
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: David Miller @ 2009-11-20 23:56 UTC (permalink / raw)
To: shemminger; +Cc: therbert, netdev
In-Reply-To: <20091120154046.67252d23@nehalam>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Fri, 20 Nov 2009 15:40:46 -0800
> Sysfs is intentionally one value per file. If you need multiple
> values, then it is the wrong interface.
This is probably right.
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: Tom Herbert @ 2009-11-21 0:04 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David Miller, Linux Netdev List
In-Reply-To: <20091120154209.01edcbb2@nehalam>
On Fri, Nov 20, 2009 at 3:42 PM, Stephen Hemminger
<shemminger@vyatta.com> wrote:
> On Fri, 20 Nov 2009 15:28:58 -0800
> Tom Herbert <therbert@google.com> wrote:
>
>> @@ -861,6 +884,9 @@ struct net_device {
>>
>> struct netdev_queue rx_queue;
>>
>> + struct dev_rps_maps *dev_rps_maps; /* Per-NAPI maps for
>> + receive packet steeing */
>> +
>
> How does this work for devices with:
> * multiqueue - one device has multiple NAPI instances
Each NAPI instance has its own map (dev_rps_maps hold the array of these maps)
> * mulitport - one NAPI shared by multiple devices
>
I have not tested that, so I'm not sure. But, I believe since the per
NAPI maps are kept in the netdevice (not in napi structure) this would
mean that each of those devices has its own per NAPI rps map(s).
Tom
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: David Miller @ 2009-11-21 0:05 UTC (permalink / raw)
To: therbert; +Cc: netdev
In-Reply-To: <65634d660911201550v34bc463ao17dcd7b16f62264f@mail.gmail.com>
From: Tom Herbert <therbert@google.com>
Date: Fri, 20 Nov 2009 15:50:03 -0800
> We only set the bit in remote_softirq_cpus in here. The actual IPIs
> are sent at the end net_rx_action. I'm not exactly sure what you're
> thinking on this...
You're scheduling a softirq so you can schedule remote softirqs.
Just do that "for each rps cpu" loop at the end of NAPI ->poll()
processing. In fact that seems to be what you're doing :-)
I guess my confusion is from the:
__raise_softirq_irqoff(NET_RX_SOFTIRQ);
you are doing as you set the cpus in rps_remote_softirq_cpus.
Why do you need to schedule the local RX softirq, when we know we're
in a NAPI poll loop and thus that we're in a softirq, and thus that we
will fire off the IPIs at the end of net_rx_action()?
That's what you're doing, the softirq raising just seems superfluous.
^ permalink raw reply
* Re: [PATCH v4 1/1] rps: core implementation
From: Tom Herbert @ 2009-11-21 0:12 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20091120.160507.110482370.davem@davemloft.net>
> I guess my confusion is from the:
>
> __raise_softirq_irqoff(NET_RX_SOFTIRQ);
>
> you are doing as you set the cpus in rps_remote_softirq_cpus.
>
> Why do you need to schedule the local RX softirq, when we know we're
> in a NAPI poll loop and thus that we're in a softirq, and thus that we
> will fire off the IPIs at the end of net_rx_action()?
>
> That's what you're doing, the softirq raising just seems superfluous.
>
Ah, right. If RPS can't be used non-NAPI case, this line is now
superfluous. It can be removed.
Thanks
^ permalink raw reply
* [RFC PATCH] net: persistent device name bitmaps for faster name allocation
From: Octavian Purdila @ 2009-11-21 0:37 UTC (permalink / raw)
To: netdev; +Cc: Stephen Hemminger
This patch solves the current scalability issues of dev_name_alloc() by
making the device name bitmap persistent. It is based on an idea
suggested by Stephen Hemminger.
$ time insmod /lib/modules/dummy.ko numdummies=8000
Without the patch With the patch
real 0m 43.43s real 0m 0.52s
user 0m 0.00s user 0m 0.00s
sys 0m 43.43s sys 0m 0.52s
For each format string (e.g. eth%d) we maintain a device name
bitmap. The bitmap starts by allocating one page and can grow up to
MAX_DEV_NAME_ORDER.
There is also a limit to the number of bitmaps (MAX_DEV_NAME_BITMAPS).
If this limit is reached while trying to allocate a new bitmap we free
the last used one.
The device name bitmaps are stored in a rbtree on a per namespace
basis.
Open issue: Stephen suggested to register "a VM notifier hook" that
could be used to dump the trees in case of memory pressure.
However, I was only able to find the OOM notifier, and I don't know if
that would be as useful as a real memory pressure event. Any hints?
Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
---
include/net/net_namespace.h | 1 +
net/core/dev.c | 293 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 269 insertions(+), 25 deletions(-)
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 0addd45..2da2a09 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -56,6 +56,7 @@ struct net {
struct list_head dev_base_head;
struct hlist_head *dev_name_head;
struct hlist_head *dev_index_head;
+ struct rb_root dev_name_bitmaps;
/* core fib_rules */
struct list_head rules_ops;
diff --git a/net/core/dev.c b/net/core/dev.c
index 9977288..9e8f909 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -205,6 +205,202 @@ static inline struct hlist_head *dev_index_hash(struct net *net, int ifindex)
return &net->dev_index_head[ifindex & (NETDEV_HASHENTRIES - 1)];
}
+/* For fast name allocation we maintain a bitmap of used devices, one for each
+ * format string (e.g. eth%d) */
+struct dev_name_bitmap {
+ char fmt[IFNAMSIZ];
+ unsigned long *data;
+ int size;
+ struct rb_node rb_node;
+ struct list_head list;
+ struct net *net;
+};
+
+#define MAX_DEV_NAME_BITMAPS 10
+#define MAX_DEV_NAME_ORDER 4
+
+static int dev_name_bitmap_count;
+/* List of all device name bitmaps, sorted by how often they are used. Last
+ * bitmap used is last. */
+static LIST_HEAD(dev_name_bitmap_list);
+
+static void kill_dev_name_bitmap(int no)
+{
+ struct dev_name_bitmap *b, *aux;
+ int i = 0;
+
+ list_for_each_entry_safe(b, aux, &dev_name_bitmap_list, list) {
+ if (i++ >= no)
+ return;
+ list_del(&b->list);
+ rb_erase(&b->rb_node, &b->net->dev_name_bitmaps);
+ free_pages((unsigned long)b->data, get_order(b->size/8));
+ dev_name_bitmap_count--;
+ kfree(b);
+ }
+}
+
+/**
+ * get_dev_name_fmt - returns the format string from a device name
+ * @name: the device name (e.g. eth0)
+ * @fmt: the format string which coresponds to the device name (e.g. eth%d)
+ *
+ * If the name contains multiple digit runs then the format string will be
+ * be based on the last one. For example, the format string for eth0a12
+ * will be eth0a%d.
+ */
+static int get_dev_name_fmt(const char *name, char *fmt)
+{
+ int i = 0, last_run_end = -1, last_run_start = -1;
+ char digit[IFNAMSIZ] = { 0, }, c;
+
+ while ((c = name[i++])) {
+ if (c >= '0' && c <= '9')
+ digit[i] = 1;
+ }
+
+ /* find the last digit */
+ for (i = IFNAMSIZ-1; i >= 0; i--)
+ if (digit[i]) {
+ last_run_end = i;
+ break;
+ }
+
+ /* make sure we have space for %d\0 */
+ if (last_run_end < 0 || last_run_start >= IFNAMSIZ - 3)
+ return -EINVAL;
+
+ /* find the first digit of the last digit run */
+ for (i = last_run_end; i >= 0; i--)
+ if (!digit[i]) {
+ last_run_start = i;
+ break;
+ }
+
+ if (last_run_start < 0)
+ return -EINVAL;
+
+ memcpy(fmt, name, last_run_start);
+ memcpy(&fmt[last_run_start], "%d", sizeof("%d"));
+
+ return 0;
+}
+
+static struct dev_name_bitmap *get_name_bitmap(struct net *net, const char *fmt)
+{
+ struct rb_node *i = net->dev_name_bitmaps.rb_node;
+
+ while (i) {
+ struct dev_name_bitmap *b;
+ int cmp;
+
+ b = rb_entry(i, struct dev_name_bitmap, rb_node);
+ cmp = strcmp(fmt, b->fmt);
+
+ if (cmp < 0)
+ i = i->rb_left;
+ else if (cmp > 0)
+ i = i->rb_right;
+ else
+ return b;
+ }
+
+ return NULL;
+}
+
+static int grow_name_bitmap(struct dev_name_bitmap *b, int size)
+{
+ unsigned long *data;
+ int order = get_order(size/8 + size%8);
+
+ if (order > MAX_DEV_NAME_ORDER)
+ return -EDQUOT;
+
+ size = (1<<order)*PAGE_SIZE*8;
+ if (size < b->size)
+ return 0;
+
+ data = (unsigned long *)__get_free_pages(GFP_ATOMIC, order);
+ if (!data)
+ return -ENOMEM;
+
+ memcpy(data, b->data, b->size/8);
+ memset((char *)data + b->size/8, 0, size/8 - b->size/8);
+
+ if (b->size)
+ free_pages((unsigned long)b->data, get_order(b->size/8));
+ b->size = size;
+ b->data = data;
+
+ return 0;
+}
+
+static void fill_name_bitmap(struct dev_name_bitmap *b, int order)
+{
+ struct net_device *d;
+ char buf[IFNAMSIZ];
+ int i;
+
+ for_each_netdev(b->net, d) {
+ if (!sscanf(d->name, b->fmt, &i))
+ continue;
+
+ /* avoid cases where sscanf is not exact inverse of printf */
+ snprintf(buf, IFNAMSIZ, b->fmt, i);
+ if (!strncmp(buf, d->name, IFNAMSIZ))
+ continue;
+
+ if (i < 0)
+ continue;
+
+ if (i >= b->size && grow_name_bitmap(b, i))
+ continue;
+
+ set_bit(i, b->data);
+ }
+}
+
+static void update_name_bitmap(struct net *net, const char *name, bool set)
+{
+ struct dev_name_bitmap *b = NULL;
+ char fmt[IFNAMSIZ], tmp[IFNAMSIZ];
+ int i;
+
+ if (get_dev_name_fmt(name, fmt))
+ return;
+
+ b = get_name_bitmap(net, fmt);
+ if (!b || sscanf(name, fmt, &i) != 1 ||
+ snprintf(tmp, IFNAMSIZ, fmt, i) == 0 ||
+ strcmp(tmp, name) != 0)
+ return;
+
+ if (i > b->size && grow_name_bitmap(b, i))
+ return;
+
+ if (set)
+ set_bit(i, b->data);
+ else
+ clear_bit(i, b->data);
+
+ list_del(&b->list);
+ list_add_tail(&b->list, &dev_name_bitmap_list);
+}
+
+static inline void list_dev_name(struct net_device *dev)
+{
+ struct net *net = dev_net(dev);
+
+ update_name_bitmap(net, dev->name, 1);
+ hlist_add_head_rcu(&dev->name_hlist, dev_name_hash(net, dev->name));
+}
+
+static inline void unlist_dev_name(struct net_device *dev, const char *name)
+{
+ hlist_del_rcu(&dev->name_hlist);
+ update_name_bitmap(dev_net(dev), name, 0);
+}
+
/* Device list insertion */
static int list_netdevice(struct net_device *dev)
{
@@ -214,7 +410,7 @@ static int list_netdevice(struct net_device *dev)
write_lock_bh(&dev_base_lock);
list_add_tail_rcu(&dev->dev_list, &net->dev_base_head);
- hlist_add_head_rcu(&dev->name_hlist, dev_name_hash(net, dev->name));
+ list_dev_name(dev);
hlist_add_head_rcu(&dev->index_hlist,
dev_index_hash(net, dev->ifindex));
write_unlock_bh(&dev_base_lock);
@@ -231,7 +427,7 @@ static void unlist_netdevice(struct net_device *dev)
/* Unlink dev from the device chain */
write_lock_bh(&dev_base_lock);
list_del_rcu(&dev->dev_list);
- hlist_del_rcu(&dev->name_hlist);
+ unlist_dev_name(dev, dev->name);
hlist_del_rcu(&dev->index_hlist);
write_unlock_bh(&dev_base_lock);
}
@@ -839,6 +1035,55 @@ int dev_valid_name(const char *name)
}
EXPORT_SYMBOL(dev_valid_name);
+static struct dev_name_bitmap *new_name_bitmap(struct net *net, const char *fmt)
+{
+ struct rb_node **p = &net->dev_name_bitmaps.rb_node, *parent = NULL;
+ struct dev_name_bitmap *b;
+
+ if (dev_name_bitmap_count == MAX_DEV_NAME_BITMAPS - 1)
+ kill_dev_name_bitmap(1);
+
+ b = kmalloc(sizeof(*b), GFP_ATOMIC);
+ if (!b)
+ return NULL;
+
+ strcpy(b->fmt, fmt);
+ b->size = 0;
+ b->net = net;
+
+ if (grow_name_bitmap(b, 8*PAGE_SIZE)) {
+ kfree(b);
+ return NULL;
+ }
+
+ fill_name_bitmap(b, 0);
+
+ list_add_tail(&b->list, &dev_name_bitmap_list);
+
+ while (*p) {
+ struct dev_name_bitmap *cmp_b;
+ int cmp;
+
+ parent = *p;
+ cmp_b = rb_entry(parent, struct dev_name_bitmap, rb_node);
+ cmp = strcmp(b->fmt, cmp_b->fmt);
+
+ if (cmp < 0)
+ p = &(*p)->rb_left;
+ else if (cmp > 0)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+ rb_link_node(&b->rb_node, parent, p);
+ rb_insert_color(&b->rb_node, &net->dev_name_bitmaps);
+
+ dev_name_bitmap_count++;
+
+ return b;
+}
+
+
/**
* __dev_alloc_name - allocate a name for a device
* @net: network namespace to allocate the device name in
@@ -856,14 +1101,13 @@ EXPORT_SYMBOL(dev_valid_name);
static int __dev_alloc_name(struct net *net, const char *name, char *buf)
{
- int i = 0;
+ int i = 0, retry;
const char *p;
- const int max_netdevices = 8*PAGE_SIZE;
- unsigned long *inuse;
- struct net_device *d;
p = strnchr(name, IFNAMSIZ-1, '%');
if (p) {
+ struct dev_name_bitmap *b;
+
/*
* Verify the string as this thing may have come from
* the user. There must be either one "%d" and no other "%"
@@ -872,25 +1116,23 @@ static int __dev_alloc_name(struct net *net, const char *name, char *buf)
if (p[1] != 'd' || strchr(p + 2, '%'))
return -EINVAL;
- /* Use one page as a bit array of possible slots */
- inuse = (unsigned long *) get_zeroed_page(GFP_ATOMIC);
- if (!inuse)
- return -ENOMEM;
-
- for_each_netdev(net, d) {
- if (!sscanf(d->name, name, &i))
- continue;
- if (i < 0 || i >= max_netdevices)
- continue;
-
- /* avoid cases where sscanf is not exact inverse of printf */
- snprintf(buf, IFNAMSIZ, name, i);
- if (!strncmp(buf, d->name, IFNAMSIZ))
- set_bit(i, inuse);
+ b = get_name_bitmap(net, name);
+ if (!b) {
+ b = new_name_bitmap(net, name);
+ if (!b)
+ return -ENOMEM;
}
- i = find_first_zero_bit(inuse, max_netdevices);
- free_page((unsigned long) inuse);
+ do {
+ retry = 0;
+ i = find_first_zero_bit(b->data, b->size);
+ if (i == b->size) {
+ int err = grow_name_bitmap(b, i + 1);
+ if (err)
+ return err;
+ retry = 1;
+ }
+ } while (retry);
}
if (buf != name)
@@ -994,13 +1236,13 @@ rollback:
}
write_lock_bh(&dev_base_lock);
- hlist_del(&dev->name_hlist);
+ unlist_dev_name(dev, oldname);
write_unlock_bh(&dev_base_lock);
synchronize_rcu();
write_lock_bh(&dev_base_lock);
- hlist_add_head_rcu(&dev->name_hlist, dev_name_hash(net, dev->name));
+ list_dev_name(dev);
write_unlock_bh(&dev_base_lock);
ret = call_netdevice_notifiers(NETDEV_CHANGENAME, dev);
@@ -5655,6 +5897,7 @@ static struct hlist_head *netdev_create_hash(void)
static int __net_init netdev_init(struct net *net)
{
INIT_LIST_HEAD(&net->dev_base_head);
+ net->dev_name_bitmaps = RB_ROOT;
net->dev_name_head = netdev_create_hash();
if (net->dev_name_head == NULL)
--
1.5.6.5
^ permalink raw reply related
* Re: [PATCH v4 1/1] rps: core implementation
From: Jarek Poplawski @ 2009-11-21 0:40 UTC (permalink / raw)
To: Tom Herbert; +Cc: David Miller, netdev
In-Reply-To: <65634d660911201612y12a9b5bnb6685da040ad832d@mail.gmail.com>
Tom Herbert wrote, On 11/21/2009 01:12 AM:
>> I guess my confusion is from the:
>>
>> __raise_softirq_irqoff(NET_RX_SOFTIRQ);
>>
>> you are doing as you set the cpus in rps_remote_softirq_cpus.
>>
>> Why do you need to schedule the local RX softirq, when we know we're
>> in a NAPI poll loop and thus that we're in a softirq, and thus that we
>> will fire off the IPIs at the end of net_rx_action()?
>>
>> That's what you're doing, the softirq raising just seems superfluous.
>>
>
> Ah, right. If RPS can't be used non-NAPI case, this line is now
> superfluous. It can be removed.
Hmm... mostly right. At least if it's wrt. my previous opinion. I meant
only that non-NAPI case would be handled less optimally without this
dedicated softirq, but __raise_softirq_irqoff(NET_RX_SOFTIRQ) or
napi_schedule(&queue->backlog) could be called from netif_rx().
Jarek P.
^ permalink raw reply
* Re: [RFC PATCH] net: persistent device name bitmaps for faster name allocation
From: Stephen Hemminger @ 2009-11-21 0:57 UTC (permalink / raw)
To: Octavian Purdila; +Cc: netdev
In-Reply-To: <200911210237.25855.opurdila@ixiacom.com>
On Sat, 21 Nov 2009 02:37:25 +0200
Octavian Purdila <opurdila@ixiacom.com> wrote:
> This patch solves the current scalability issues of dev_name_alloc() by
> making the device name bitmap persistent. It is based on an idea
> suggested by Stephen Hemminger.
Still not sure that this much code is worth it. Even with 10000 ISP PPPoE links,
I can't see that bringing up a new PPP device would be the real bottleneck
except in the case of some benchmark that starts them all at once. Even TPC-C
benchmarks have a "warm up period".
--
^ permalink raw reply
* [net-2.6 PATCH 2/3] netxen: fix promisc for NX2031.
From: Amit Kumar Salecha @ 2009-11-21 1:09 UTC (permalink / raw)
To: davem; +Cc: netdev, dhananjay.phadke, Narender Kumar, Amit Kumar Salecha
In-Reply-To: <1258765774-27068-1-git-send-email-amit@netxen.com>
From: Narender Kumar <narender.kumar@qlogic.com>
Kernel crashes, if promisc mode set without disabling rx queue.
Before changing mode in NX2031 chip, wait till rx queue drains.
Signed-off-by: Narender Kumar <narender.kumar@qlogic.com>
Signed-off-by: Amit Kumar Salecha <amit.salecha@qlogic.com>
---
drivers/net/netxen/netxen_nic_hdr.h | 2 +
drivers/net/netxen/netxen_nic_hw.c | 49 +++++++++++++++++++++++++++--------
2 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/drivers/net/netxen/netxen_nic_hdr.h b/drivers/net/netxen/netxen_nic_hdr.h
index 1c46da6..17bb381 100644
--- a/drivers/net/netxen/netxen_nic_hdr.h
+++ b/drivers/net/netxen/netxen_nic_hdr.h
@@ -545,6 +545,8 @@ enum {
#define NETXEN_NIU_TEST_MUX_CTL (NETXEN_CRB_NIU + 0x00094)
#define NETXEN_NIU_XG_PAUSE_CTL (NETXEN_CRB_NIU + 0x00098)
#define NETXEN_NIU_XG_PAUSE_LEVEL (NETXEN_CRB_NIU + 0x000dc)
+#define NETXEN_NIU_FRAME_COUNT_SELECT (NETXEN_CRB_NIU + 0x000ac)
+#define NETXEN_NIU_FRAME_COUNT (NETXEN_CRB_NIU + 0x000b0)
#define NETXEN_NIU_XG_SEL (NETXEN_CRB_NIU + 0x00128)
#define NETXEN_NIU_GB_PAUSE_CTL (NETXEN_CRB_NIU + 0x0030c)
diff --git a/drivers/net/netxen/netxen_nic_hw.c b/drivers/net/netxen/netxen_nic_hw.c
index 3185a98..e2c4a01 100644
--- a/drivers/net/netxen/netxen_nic_hw.c
+++ b/drivers/net/netxen/netxen_nic_hw.c
@@ -383,24 +383,51 @@ int netxen_niu_disable_xg_port(struct netxen_adapter *adapter)
int netxen_p2_nic_set_promisc(struct netxen_adapter *adapter, u32 mode)
{
- __u32 reg;
+ u32 mac_cfg;
+ u32 cnt = 0;
+ __u32 reg = 0x0200;
u32 port = adapter->physical_port;
+ u16 board_type = adapter->ahw.board_type;
if (port > NETXEN_NIU_MAX_XG_PORTS)
return -EINVAL;
- reg = NXRD32(adapter, NETXEN_NIU_XGE_CONFIG_1 + (0x10000 * port));
- if (mode == NETXEN_NIU_PROMISC_MODE)
- reg = (reg | 0x2000UL);
- else
- reg = (reg & ~0x2000UL);
+ mac_cfg = NXRD32(adapter, NETXEN_NIU_XGE_CONFIG_0 + (0x10000 * port));
+ mac_cfg &= ~0x4;
+ NXWR32(adapter, NETXEN_NIU_XGE_CONFIG_0 + (0x10000 * port), mac_cfg);
- if (mode == NETXEN_NIU_ALLMULTI_MODE)
- reg = (reg | 0x1000UL);
- else
- reg = (reg & ~0x1000UL);
+ if ((board_type == NETXEN_BRDTYPE_P2_SB31_10G_IMEZ) ||
+ (board_type == NETXEN_BRDTYPE_P2_SB31_10G_HMEZ))
+ reg = (0x20 << port);
+
+ NXWR32(adapter, NETXEN_NIU_FRAME_COUNT_SELECT, reg);
+
+ mdelay(10);
+
+ while (NXRD32(adapter, NETXEN_NIU_FRAME_COUNT) && ++cnt < 20)
+ mdelay(10);
+
+ if (cnt < 20) {
+
+ reg = NXRD32(adapter,
+ NETXEN_NIU_XGE_CONFIG_1 + (0x10000 * port));
+
+ if (mode == NETXEN_NIU_PROMISC_MODE)
+ reg = (reg | 0x2000UL);
+ else
+ reg = (reg & ~0x2000UL);
+
+ if (mode == NETXEN_NIU_ALLMULTI_MODE)
+ reg = (reg | 0x1000UL);
+ else
+ reg = (reg & ~0x1000UL);
+
+ NXWR32(adapter,
+ NETXEN_NIU_XGE_CONFIG_1 + (0x10000 * port), reg);
+ }
- NXWR32(adapter, NETXEN_NIU_XGE_CONFIG_1 + (0x10000 * port), reg);
+ mac_cfg |= 0x4;
+ NXWR32(adapter, NETXEN_NIU_XGE_CONFIG_0 + (0x10000 * port), mac_cfg);
return 0;
}
--
1.6.0.2
^ permalink raw reply related
* [net-2.6 PATCH 3/3] netxen : fix BOND_MODE_TLB/ALB mode.
From: Amit Kumar Salecha @ 2009-11-21 1:09 UTC (permalink / raw)
To: davem; +Cc: netdev, dhananjay.phadke, Narender Kumar, Amit Kumar Salecha
In-Reply-To: <1258765774-27068-1-git-send-email-amit@netxen.com>
From: Narender Kumar <narender.kumar@qlogic.com>
o In case of bonding mode BOND_MODE_TLB and BOND_MODE_ALB,
mac addr can swap of new_active && old_active slave.
o In set_mac, store mac addr in device private structure for future use.
Signed-off-by: Narender Kumar <narender.kumar@qlogic.com>
Signed-off-by: Amit Kumar Salecha <amit.salecha@qlogic.com>
---
drivers/net/netxen/netxen_nic.h | 2 ++
drivers/net/netxen/netxen_nic_hw.c | 6 +++---
drivers/net/netxen/netxen_nic_main.c | 4 +++-
3 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/net/netxen/netxen_nic.h b/drivers/net/netxen/netxen_nic.h
index 7384f59..e1237b8 100644
--- a/drivers/net/netxen/netxen_nic.h
+++ b/drivers/net/netxen/netxen_nic.h
@@ -1163,6 +1163,8 @@ struct netxen_adapter {
u32 int_vec_bit;
u32 heartbit;
+ u8 mac_addr[ETH_ALEN];
+
struct netxen_adapter_stats stats;
struct netxen_recv_context recv_ctx;
diff --git a/drivers/net/netxen/netxen_nic_hw.c b/drivers/net/netxen/netxen_nic_hw.c
index e2c4a01..52a3798 100644
--- a/drivers/net/netxen/netxen_nic_hw.c
+++ b/drivers/net/netxen/netxen_nic_hw.c
@@ -463,7 +463,7 @@ netxen_nic_enable_mcast_filter(struct netxen_adapter *adapter)
{
u32 val = 0;
u16 port = adapter->physical_port;
- u8 *addr = adapter->netdev->dev_addr;
+ u8 *addr = adapter->mac_addr;
if (adapter->mc_enabled)
return 0;
@@ -492,7 +492,7 @@ netxen_nic_disable_mcast_filter(struct netxen_adapter *adapter)
{
u32 val = 0;
u16 port = adapter->physical_port;
- u8 *addr = adapter->netdev->dev_addr;
+ u8 *addr = adapter->mac_addr;
if (!adapter->mc_enabled)
return 0;
@@ -687,7 +687,7 @@ void netxen_p3_nic_set_multi(struct net_device *netdev)
list_splice_tail_init(&adapter->mac_list, &del_list);
- nx_p3_nic_add_mac(adapter, netdev->dev_addr, &del_list);
+ nx_p3_nic_add_mac(adapter, adapter->mac_addr, &del_list);
nx_p3_nic_add_mac(adapter, bcast_addr, &del_list);
if (netdev->flags & IFF_PROMISC) {
diff --git a/drivers/net/netxen/netxen_nic_main.c b/drivers/net/netxen/netxen_nic_main.c
index 0b4a56a..3bf78db 100644
--- a/drivers/net/netxen/netxen_nic_main.c
+++ b/drivers/net/netxen/netxen_nic_main.c
@@ -437,6 +437,7 @@ netxen_read_mac_addr(struct netxen_adapter *adapter)
netdev->dev_addr[i] = *(p + 5 - i);
memcpy(netdev->perm_addr, netdev->dev_addr, netdev->addr_len);
+ memcpy(adapter->mac_addr, netdev->dev_addr, netdev->addr_len);
/* set station address */
@@ -459,6 +460,7 @@ int netxen_nic_set_mac(struct net_device *netdev, void *p)
netxen_napi_disable(adapter);
}
+ memcpy(adapter->mac_addr, addr->sa_data, netdev->addr_len);
memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
adapter->macaddr_set(adapter, addr->sa_data);
@@ -956,7 +958,7 @@ netxen_nic_up(struct netxen_adapter *adapter, struct net_device *netdev)
return err;
}
if (NX_IS_REVISION_P2(adapter->ahw.revision_id))
- adapter->macaddr_set(adapter, netdev->dev_addr);
+ adapter->macaddr_set(adapter, adapter->mac_addr);
adapter->set_multi(netdev);
adapter->set_mtu(adapter, netdev->mtu);
--
1.6.0.2
^ permalink raw reply related
* [net-2.6 PATCH 1/3] netxen: fix memory initialization
From: Amit Kumar Salecha @ 2009-11-21 1:09 UTC (permalink / raw)
To: davem; +Cc: netdev, dhananjay.phadke
In-Reply-To: <1258765774-27068-1-git-send-email-amit@netxen.com>
Avoid resetting memory during initialization, skip this memory
block during driver probe.
Signed-off-by: Amit Kumar Salecha <amit@netxen.com>
---
drivers/net/netxen/netxen_nic_init.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/drivers/net/netxen/netxen_nic_init.c b/drivers/net/netxen/netxen_nic_init.c
index e40b914..8a09043 100644
--- a/drivers/net/netxen/netxen_nic_init.c
+++ b/drivers/net/netxen/netxen_nic_init.c
@@ -544,6 +544,8 @@ int netxen_pinit_from_rom(struct netxen_adapter *adapter, int verbose)
continue;
if (off == (ROMUSB_GLB + 0x1c)) /* MS clock */
continue;
+ if ((off & 0x0ff00000) == NETXEN_CRB_DDR_NET)
+ continue;
if (off == (NETXEN_CRB_PEG_NET_1 + 0x18))
buf[i].data = 0x1020;
/* skip the function enable register */
--
1.6.0.2
^ permalink raw reply related
* [net-2.6 PATCH 0/3] critical fixes
From: Amit Kumar Salecha @ 2009-11-21 1:09 UTC (permalink / raw)
To: davem; +Cc: netdev, dhananjay.phadke
Hi
I am sending three patches, to fix some critical issues.
1) Skip memory initialization during driver load. Device may not come up, if memory is reset.
2) Fix promisc mode, kernel crashes with this bug. Drain rx queue before changing mode.
3) Fix bonding mode TLB/ALB, correct mac address need to communicate to device.
-Amit Salecha
^ permalink raw reply
* Re: A generic kernel compatibilty code
From: Luis R. Rodriguez @ 2009-11-21 2:12 UTC (permalink / raw)
To: John W. Linville
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA, linux-wireless,
netdev-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <43e72e890911201338w4c403d51q651afc3638d1739c-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Fri, Nov 20, 2009 at 1:38 PM, Luis R. Rodriguez <mcgrof-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Fri, Nov 20, 2009 at 1:18 PM, John W. Linville
> <linville-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org> wrote:
>> On Fri, Nov 20, 2009 at 12:53:51PM -0800, Luis R. Rodriguez wrote:
>>> On Fri, Nov 20, 2009 at 12:51 PM, Luis R. Rodriguez <mcgrof@gmail.com> wrote:
>>> > On Fri, Nov 20, 2009 at 12:45 PM, Luis R. Rodriguez <mcgrof@gmail.com> wrote:
>>> >> Everyone and their mother reinvents the wheel when it comes to
>>> >> backporting kernel modules. It a painful job and it seems to me an
>>> >> alternative is possible. If we can write generic compatibilty code for
>>> >> a new routine introduced on the next kernel how about just merging it
>>> >> to the kernel under some generic compat module. This would be
>>> >> completey ignored by everyone using the stable kernel but can be
>>> >> copied by anyone doing backport work.
>>> >>
>>> >> So I'm thinking something as simple as a generic compat/comat.ko with
>>> >> compat-2.6.32.[ch] files.
>>> >
>>> > FWIW, I meant a compat-2.6.32.[ch] and compat-2.6.31.[ch] and so on.
>>> > All these would link to the compat.ko
>>>
>>> I supose this could juse be a separate tree with some generic
>>> compat.ko module. That might work better.
>>
>> This is what I would suggest for pursuing this idea. Perhaps you
>> could split-off from compat-wireless, then make that tree depend on
>> the new tree (compat-core?)...
>
> OK thanks, will try that for the next kernel.
I've packaged a compat.ko here:
git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/compat.git
This should be up as soon as the kernel.org synchs. The missing piece
would just be to define a common place to put the headers for
distributions who want to use external modules which want to use this
thing.
Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox