* Re: xfrm command fails
From: Naveen BN @ 2009-11-11 6:16 UTC (permalink / raw)
To: netdev; +Cc: David Miller
In-Reply-To: <4AFA5499.2090605@globaledgesoft.com>
adding David in loop
Naveen BN wrote:
> I am using fedora with version => 2.6.23.1-42.fc8.
> Please guide me to solve the error message "RTNETLINK answers:
> Function not implemented"
>
> Regards
> Naveen
> Naveen BN wrote:
>> Hi All
>> I am trying to create a security association using ip xfrm command,
>> but i get an message saying
>> "*RTNETLINK answers: Function not implemented*" and ip xfrm state
>> succeeds in displaying sa written using setkey .
>>
>> ip xfrm state add src 192.168.1.20 dst 192.168.1.5 proto esp spi
>> 0x354cb04b mode tunnel reqid 16385 replay-window 32 auth hmac
>> 0xecf02a5cf6568556e1bdcd961c7ec3f92afd01cc enc cbc
>> 0x5c0cfa9672ce67ba545b593076dfb278 sel src 0.0.0.0/0 dst 0.0.0.0/0
>>
>> ERROR message => "RTNETLINK answers: Function not implemented"
>>
>> Regards
>> Naveen
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: [PATCH 01/10] netdev: add netdev_continue_rcu
From: David Miller @ 2009-11-11 6:47 UTC (permalink / raw)
To: eric.dumazet; +Cc: shemminger, paulmck, netdev
In-Reply-To: <4AF9AEAA.1080608@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 19:19:22 +0100
> Stephen Hemminger a écrit :
>> This adds an RCU macro for continuing search, useful for some
>> network devices like vlan.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 02/10] vlan: eliminate use of dev_base_lock
From: David Miller @ 2009-11-11 6:47 UTC (permalink / raw)
To: dada1; +Cc: shemminger, kaber, netdev
In-Reply-To: <4AF9AEDF.2070307@cosmosbay.com>
From: Eric Dumazet <dada1@cosmosbay.com>
Date: Tue, 10 Nov 2009 19:20:15 +0100
> Stephen Hemminger a écrit :
>> Do not need to use read_lock(&dev_base_lock), use RCU instead.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 03/10] net: use rcu for network scheduler API
From: David Miller @ 2009-11-11 6:47 UTC (permalink / raw)
To: eric.dumazet; +Cc: shemminger, netdev
In-Reply-To: <4AF9AF09.60102@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 19:20:57 +0100
> Stephen Hemminger a écrit :
>> Use RCU to walk list of network devices in qdisc dump.
>> This could be optimized for large number of devices.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 04/10] AOE: use rcu to find network device
From: David Miller @ 2009-11-11 6:48 UTC (permalink / raw)
To: shemminger
Cc: karaluh, ecashin, roel.kluin, harvey.harrison, bzolnier, netdev
In-Reply-To: <20091110155316.2c3d7b6e@nehalam>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 15:53:16 -0800
> On Tue, 10 Nov 2009 15:06:17 -0800
> Stephen Hemminger <shemminger@vyatta.com> wrote:
>
>> On Tue, 10 Nov 2009 15:01:49 -0500
>> Ed Cashin <ecashin@coraid.com> wrote:
>>
>> > On Tue Nov 10 13:07:37 EST 2009, shemminger@vyatta.com wrote:
>> > > This gets rid of another use of read_lock(&dev_base_lock) by using
>> > > RCU. Also, it only increments the reference count of the device actually
>> > > used rather than holding and releasing every device
>> > >
>> > > Compile tested only.
>> >
>> > This function runs once a minute when the aoe driver is loaded,
>> > if you'd like to test it a bit more.
>> >
>> > It looks like there's no dev_put corresponding to the dev_hold
>> > after the changes.
>> >
>>
>> Hmm, looks like AOE actually is not ref counting the network device.
>> So my patch is incorrect.
>>
>> As it stands (before my patch), it is UNSAFE. It can decide to queue
>> packets to a device that is removed out from underneath it causing
>> reference to freed memory.
>>
>> Moving the rcu_read_lock up to aoecmd_cfg() would solve that but the
>> whole driver appears to be unsafe about device refcounting and handling
>> device removal properly.
>>
>> It needs to:
>>
>> 1. Get a device ref count when it remembers a device: (ie addif)
>> 2. Install a notifier that looks for device removal events
>> 3. In notifier, remove interface, including flushing all pending
>> skb's for that device.
>>
>> This obviously is beyond the scope of the RCU stuff.
>
> Here is a patch to get you going, it does compile but it probably
> won't work because the code doesn't handle the case of the last
> device going away from a target. This is yet another pre-existing
> bug, since if a timeout happens: ejectif() is called to remove a device,
> resend() will BUG in ifrotate() if all devices are gone.
I'm holding off on the RCU patch until these known refcount bugs are
fixed
^ permalink raw reply
* Re: [PATCH 06/10] s390: use RCU to walk list of network devices
From: David Miller @ 2009-11-11 6:49 UTC (permalink / raw)
To: eric.dumazet; +Cc: shemminger, schwidefsky, heiko.carstens, netdev, linux390
In-Reply-To: <4AF9B3AA.6070104@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 19:40:42 +0100
> Stephen Hemminger a écrit :
>> This is similar to other cases where for_each_netdev_rcu
>> can be used when gathering information.
>>
>> By inspection, don't have platform or cross-build environment
>> to validate.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>>
>>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 05/10] parisc: use RCU to find network device
From: David Miller @ 2009-11-11 6:48 UTC (permalink / raw)
To: shemminger; +Cc: kyle, deller, a.beregalov, netdev, linux-parisc
In-Reply-To: <20091110175647.480041042@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:51 -0800
> Another place where RCU can be used instead of read_lock(&dev_base_lock)
> This is by inspection, don't have platform or cross-build environment
> to validate.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Applied.
^ permalink raw reply
* Re: [PATCH 07/10] decnet: use RCU to find network devices
From: David Miller @ 2009-11-11 6:49 UTC (permalink / raw)
To: shemminger
Cc: christine.caulfield, hannes, adobriyan, swhiteho, netdev,
linux-decnet-users
In-Reply-To: <20091110175647.615305929@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:53 -0800
> When showing device statistics use RCU rather than read_lock(&dev_base_lock)
> Compile tested only.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Applied.
^ permalink raw reply
* Re: [PATCH 08/10] ipv6: use RCU to walk list of network devices
From: David Miller @ 2009-11-11 6:50 UTC (permalink / raw)
To: shemminger; +Cc: netdev
In-Reply-To: <20091110175647.683253741@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:54 -0800
> No longer need read_lock(&dev_base_lock), use RCU instead.
> This also needs to be optimized for large number of devices.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
As Eric mentioned, superceded by his patch.
^ permalink raw reply
* Re: [PATCH 09/10] IPV4: use rcu to walk list of devices in IGMP
From: David Miller @ 2009-11-11 6:50 UTC (permalink / raw)
To: shemminger; +Cc: netdev
In-Reply-To: <20091110175647.754389168@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:55 -0800
> This also needs to be optimized for large number of devices.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Applied.
^ permalink raw reply
* Re: [PATCH 10/10] CAN: use dev_get_by_index_rcu
From: David Miller @ 2009-11-11 6:50 UTC (permalink / raw)
To: oliver; +Cc: eric.dumazet, shemminger, netdev
In-Reply-To: <4AFA5190.60600@hartkopp.net>
From: Oliver Hartkopp <oliver@hartkopp.net>
Date: Wed, 11 Nov 2009 06:54:24 +0100
> Eric Dumazet wrote:
>> Stephen Hemminger a écrit :
>>> Use new function to avoid doing read_lock().
>>>
>>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
...
>> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
>>
>
> Thanks everyone!
>
> Signed-off-by: Oliver Hartkopp <oliver@hartkopp.net>
Applied.
^ permalink raw reply
* [PATCH 0/2] rps: Receive packet steering
From: Tom Herbert @ 2009-11-11 6:53 UTC (permalink / raw)
To: David Miller, netdev
This is the third version of the the patch that implements software receive
side packet steering (RPS). RPS distributes the load of received packet
processing across multiple CPUs. This version allows per NAPI steering maps
as well as using HW provided hash as an optimization.
These patches are also the basis for per-flow steering which was previously
discussed; we are still working on a general solution for the per flow
steering which prevents OOO packets.
Problem statement: Protocol processing done in the NAPI context for received
packets is serialized per device queue and becomes a bottleneck under high
packet load. This substantially limits pps that can be achieved on a single
queue NIC and provides no scaling with multiple cores.
This solution queues packets early on in the receive path on the backlog queues
of other CPUs. This allows protocol processing (e.g. IP and TCP) to be
performed on packets in parallel. For each device (or NAPI instance for
a multi-queue device) a mask of CPUs is set to indicate the CPUs that can
process packets for the device. A CPU is selected on a per packet basis by
hashing contents of the packet header (the TCP or UDP 4-tuple) and using the
result to index into the CPU mask. The IPI mechanism is used to raise
networking receive softirqs between CPUs. This effectively emulates in
software what a multi-queue NIC can provide, but is generic requiring no device
support.
Many devices now provide a hash over the 4-tuple on a per packet basis
(Toeplitz is popular). This patch allow drivers to set the HW reported hash
in an skb field, and that value in turn is used to index into the RPS maps.
Using the HW generated hash can avoid cache misses on the packet when
steering the packet to a remote CPU.
The CPU masks is set on a per device basis in the sysfs variable
/sys/class/net/<device>/rps_cpus. This is a set of canonical bit maps for
each NAPI nstance of the device. For example:
echo "0b 0b0 0b00 0b000" > /sys/class/net/eth0/rps_cpus
would set maps for four NAPI instances on eth0.
The first patch in this set adds the RPS functionality to the core networking.
The second patch adds support to the bnx2x driver to record the Toeplitz hash
reported by the device for received skbs.
Generally, we have found this technique increases pps capabilities of a single
queue device with good CPU utilization. Optimal settings for the CPU mask
seems to depend on architectures and cache hierarcy. Below are some results
running 700 instances of netperf TCP_RR test with 1 byte req. and resp.
Results show cumulative transaction rate and system CPU utilization.
tg3 on 8 core Intel
Without RPS: 90K tps at 34% CPU
With RPS: 285K tps at 70% CPU
e1000 on 8 core Intel
Without RPS: 90K tps at 34% CPU
With RPS: 292K tps at 66% CPU
foredeth on 16 core AMD
Without RPS: 117K tps at 10% CPU
With RPS: 327K tps at 29% CPU
bnx2x on 16 core AMD
Single queue without RPS: 139K tps at 17% CPU
Single queue with RPS: 352K tps at 30% CPU
Multi queue (1 queues per CPU) 204K tps at 12%
We have been running a variant of this patch on production servers for a while
with good results. In some of our more networking intensive applications we
have seen 30-50% gains in end application performance.
Tom
Signed-off-by: Tom Herbert <therbert@google.com>
^ permalink raw reply
* [PATCH 1/2] rps: core implementation
From: Tom Herbert @ 2009-11-11 6:53 UTC (permalink / raw)
To: David Miller, netdev
Third version of RPS.
Signed-off-by: Tom Herbert <therbert@google.com>
---
include/linux/interrupt.h | 1 +
include/linux/netdevice.h | 18 ++++
include/linux/skbuff.h | 2 +
net/core/dev.c | 227 ++++++++++++++++++++++++++++++++++++++-------
net/core/net-sysfs.c | 135 +++++++++++++++++++++++++++
5 files changed, 348 insertions(+), 35 deletions(-)
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index b78cf81..fa91194 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -345,6 +345,7 @@ enum
TIMER_SOFTIRQ,
NET_TX_SOFTIRQ,
NET_RX_SOFTIRQ,
+ NET_RPS_SOFTIRQ,
BLOCK_SOFTIRQ,
BLOCK_IOPOLL_SOFTIRQ,
TASKLET_SOFTIRQ,
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 8380009..c1b1bbb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -639,6 +639,18 @@ struct net_device_ops {
};
/*
+ * Structure for Receive Packet Steering. Length of map and array of CPU ID's.
+ */
+struct rps_map {
+ int len;
+ u16 map[0];
+};
+
+/* Maximum size of RPS map (for allocation) */
+#define RPS_MAP_SIZE (sizeof(struct rps_map) + \
+ (num_possible_cpus() * sizeof(u16)))
+
+/*
* The DEVICE structure.
* Actually, this whole structure is a big mistake. It mixes I/O
* data with strictly "high-level" data, and it has to know about
@@ -807,6 +819,9 @@ struct net_device
void *ax25_ptr; /* AX.25 specific data */
struct wireless_dev *ieee80211_ptr; /* IEEE 802.11 specific data,
assign before registering */
+ void *rps_maps; /* Array of per-NAPI maps for
+ receive packet steeing */
+ int rps_num_maps; /* Number of RPS maps */
/*
* Cache line mostly used on receive path (including eth_type_trans())
@@ -1217,6 +1232,9 @@ struct softnet_data
struct Qdisc *output_queue;
struct sk_buff_head input_pkt_queue;
struct list_head poll_list;
+
+ struct call_single_data csd;
+
struct sk_buff *completion_queue;
struct napi_struct backlog;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 0c68fbd..95feac7 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -396,6 +396,8 @@ struct sk_buff {
__u16 vlan_tci;
+ __u32 rxhash;
+
sk_buff_data_t transport_header;
sk_buff_data_t network_header;
sk_buff_data_t mac_header;
diff --git a/net/core/dev.c b/net/core/dev.c
index 28b0b9e..735e7e3 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -1976,6 +1976,162 @@ int weight_p __read_mostly = 64; /*
old backlog weight */
DEFINE_PER_CPU(struct netif_rx_stats, netdev_rx_stat) = { 0, };
+static u32 simple_hashrnd;
+
+/**
+ * get_rps_cpu is called from netif_receive_skb and returns the target
+ * CPU from the RPS map of the receiving NAPI instance for a given skb.
+ */
+static int get_rps_cpu(struct net_device *dev, struct sk_buff *skb)
+{
+ u32 addr1, addr2, ports;
+ struct ipv6hdr *ip6;
+ struct iphdr *ip;
+ u32 hash, ihl;
+ u8 ip_proto;
+ int cpu;
+ struct rps_map *map = NULL;
+
+ if (dev->rps_num_maps) {
+ /*
+ * Locate the map corresponding to the NAPI queue that
+ * the packet was received on.
+ */
+ int index = skb_get_rx_queue(skb);
+ if (index < 0 || index >= dev->rps_num_maps)
+ index = 0;
+
+ map = (struct rps_map *)
+ (dev->rps_maps + (RPS_MAP_SIZE * index));
+ if (!map->len)
+ map = NULL;
+ }
+
+ if (!map)
+ return -1;
+
+ hash = skb->rxhash;
+ if (hash)
+ goto got_hash; /* Skip hash computation on packet header */
+
+ switch (skb->protocol) {
+ case __constant_htons(ETH_P_IP):
+ if (!pskb_may_pull(skb, sizeof(*ip)))
+ return -1;
+
+ ip = (struct iphdr *) skb->data;
+ ip_proto = ip->protocol;
+ addr1 = ip->saddr;
+ addr2 = ip->daddr;
+ ihl = ip->ihl;
+ break;
+ case __constant_htons(ETH_P_IPV6):
+ if (!pskb_may_pull(skb, sizeof(*ip6)))
+ return -1;
+
+ ip6 = (struct ipv6hdr *) skb->data;
+ ip_proto = ip6->nexthdr;
+ addr1 = ip6->saddr.s6_addr32[3];
+ addr2 = ip6->daddr.s6_addr32[3];
+ ihl = (40 >> 2);
+ break;
+ default:
+ return -1;
+ }
+ ports = 0;
+ switch (ip_proto) {
+ case IPPROTO_TCP:
+ case IPPROTO_UDP:
+ case IPPROTO_DCCP:
+ case IPPROTO_ESP:
+ case IPPROTO_AH:
+ case IPPROTO_SCTP:
+ case IPPROTO_UDPLITE:
+ if (pskb_may_pull(skb, (ihl * 4) + 4))
+ ports = *((u32 *) (skb->data + (ihl * 4)));
+ break;
+
+ default:
+ break;
+ }
+
+ hash = jhash_3words(addr1, addr2, ports, simple_hashrnd);
+
+got_hash:
+ cpu = map->map[((u64) hash * map->len) >> 32];
+
+ return cpu_online(cpu) ? cpu : -1;
+}
+
+static DEFINE_PER_CPU(cpumask_t, rps_remote_softirq_cpus);
+
+/* Called from hardirq (IPI) context */
+static void trigger_softirq(void *data)
+{
+ struct softnet_data *queue = data;
+ __napi_schedule(&queue->backlog);
+}
+
+/**
+ * net_rps_action is called from NET_RPS_SOFTIRQ to do IPIs to schedule RX
+ * softirq on remote CPUs. Called in a separate softirq to allow for
+ * coalescing.
+ */
+static void net_rps_action(struct softirq_action *h)
+{
+ int cpu;
+
+ local_irq_disable();
+
+ for_each_cpu_mask_nr(cpu, __get_cpu_var(rps_remote_softirq_cpus)) {
+ struct softnet_data *queue = &per_cpu(softnet_data, cpu);
+ __smp_call_function_single(cpu, &queue->csd, 0);
+ }
+ cpus_clear(__get_cpu_var(rps_remote_softirq_cpus));
+
+ local_irq_enable();
+}
+
+/**
+ * enqueue_to_backlog is called to queue an skb to a per CPU backlog
+ * queue (may be a remote CPU queue).
+ */
+static int enqueue_to_backlog(struct sk_buff *skb, int cpu)
+{
+ struct softnet_data *queue;
+ unsigned long flags;
+
+ queue = &per_cpu(softnet_data, cpu);
+ spin_lock_irqsave(&queue->input_pkt_queue.lock, flags);
+
+ __get_cpu_var(netdev_rx_stat).total++;
+ if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
+ if (queue->input_pkt_queue.qlen) {
+enqueue:
+ __skb_queue_tail(&queue->input_pkt_queue, skb);
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock,
+ flags);
+ return NET_RX_SUCCESS;
+ }
+
+ /* Schedule NAPI for backlog device */
+ if (napi_schedule_prep(&queue->backlog)) {
+ if (cpu != smp_processor_id()) {
+ cpu_set(cpu,
+ get_cpu_var(rps_remote_softirq_cpus));
+ __raise_softirq_irqoff(NET_RPS_SOFTIRQ);
+ } else
+ __napi_schedule(&queue->backlog);
+ }
+ goto enqueue;
+ }
+
+ __get_cpu_var(netdev_rx_stat).dropped++;
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock, flags);
+
+ kfree_skb(skb);
+ return NET_RX_DROP;
+}
/**
* netif_rx - post buffer to the network code
@@ -1994,8 +2150,7 @@ DEFINE_PER_CPU(struct netif_rx_stats,
netdev_rx_stat) = { 0, };
int netif_rx(struct sk_buff *skb)
{
- struct softnet_data *queue;
- unsigned long flags;
+ int cpu;
/* if netpoll wants it, pretend we never saw it */
if (netpoll_rx(skb))
@@ -2004,31 +2159,11 @@ int netif_rx(struct sk_buff *skb)
if (!skb->tstamp.tv64)
net_timestamp(skb);
- /*
- * The code is rearranged so that the path is the most
- * short when CPU is congested, but is still operating.
- */
- local_irq_save(flags);
- queue = &__get_cpu_var(softnet_data);
+ cpu = get_rps_cpu(skb->dev, skb);
+ if (cpu < 0)
+ cpu = smp_processor_id();
- __get_cpu_var(netdev_rx_stat).total++;
- if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
- if (queue->input_pkt_queue.qlen) {
-enqueue:
- __skb_queue_tail(&queue->input_pkt_queue, skb);
- local_irq_restore(flags);
- return NET_RX_SUCCESS;
- }
-
- napi_schedule(&queue->backlog);
- goto enqueue;
- }
-
- __get_cpu_var(netdev_rx_stat).dropped++;
- local_irq_restore(flags);
-
- kfree_skb(skb);
- return NET_RX_DROP;
+ return enqueue_to_backlog(skb, cpu);
}
EXPORT_SYMBOL(netif_rx);
@@ -2266,10 +2401,10 @@ void netif_nit_deliver(struct sk_buff *skb)
}
/**
- * netif_receive_skb - process receive buffer from network
+ * __netif_receive_skb - process receive buffer from network
* @skb: buffer to process
*
- * netif_receive_skb() is the main receive data processing function.
+ * __netif__napireceive_skb() is the main receive data processing function.
* It always succeeds. The buffer may be dropped during processing
* for congestion control or by the protocol layers.
*
@@ -2280,7 +2415,8 @@ void netif_nit_deliver(struct sk_buff *skb)
* NET_RX_SUCCESS: no congestion
* NET_RX_DROP: packet was dropped
*/
-int netif_receive_skb(struct sk_buff *skb)
+
+int __netif_receive_skb(struct sk_buff *skb)
{
struct packet_type *ptype, *pt_prev;
struct net_device *orig_dev;
@@ -2378,6 +2514,16 @@ out:
}
EXPORT_SYMBOL(netif_receive_skb);
+int netif_receive_skb(struct sk_buff *skb)
+{
+ int cpu = get_rps_cpu(skb->dev, skb);
+
+ if (cpu < 0)
+ return __netif_receive_skb(skb);
+ else
+ return enqueue_to_backlog(skb, cpu);
+}
+
/* Network device is going away, flush any packets still pending */
static void flush_backlog(void *arg)
{
@@ -2421,7 +2567,7 @@ static int napi_gro_complete(struct sk_buff *skb)
}
out:
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
}
void napi_gro_flush(struct napi_struct *napi)
@@ -2554,7 +2700,7 @@ int napi_skb_finish(int ret, struct sk_buff *skb)
switch (ret) {
case GRO_NORMAL:
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
case GRO_DROP:
err = NET_RX_DROP;
@@ -2625,7 +2771,7 @@ int napi_frags_finish(struct napi_struct *napi,
struct sk_buff *skb, int ret)
skb->protocol = eth_type_trans(skb, napi->dev);
if (ret == GRO_NORMAL)
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
skb_gro_pull(skb, -ETH_HLEN);
break;
@@ -2696,21 +2842,24 @@ static int process_backlog(struct napi_struct
*napi, int quota)
int work = 0;
struct softnet_data *queue = &__get_cpu_var(softnet_data);
unsigned long start_time = jiffies;
+ unsigned long flags;
napi->weight = weight_p;
do {
struct sk_buff *skb;
local_irq_disable();
+ spin_lock_irqsave(&queue->input_pkt_queue.lock, flags);
skb = __skb_dequeue(&queue->input_pkt_queue);
if (!skb) {
__napi_complete(napi);
- local_irq_enable();
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock,
+ flags);
break;
}
- local_irq_enable();
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock, flags);
- netif_receive_skb(skb);
+ __netif_receive_skb(skb);
} while (++work < quota && jiffies == start_time);
return work;
@@ -5205,6 +5354,8 @@ void free_netdev(struct net_device *dev)
/* Flush device addresses */
dev_addr_flush(dev);
+ kfree(dev->rps_maps);
+
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
netif_napi_del(p);
@@ -5644,6 +5795,10 @@ static int __init net_dev_init(void)
queue->completion_queue = NULL;
INIT_LIST_HEAD(&queue->poll_list);
+ queue->csd.func = trigger_softirq;
+ queue->csd.info = queue;
+ queue->csd.flags = 0;
+
queue->backlog.poll = process_backlog;
queue->backlog.weight = weight_p;
queue->backlog.gro_list = NULL;
@@ -5669,7 +5824,9 @@ static int __init net_dev_init(void)
open_softirq(NET_TX_SOFTIRQ, net_tx_action);
open_softirq(NET_RX_SOFTIRQ, net_rx_action);
+ open_softirq(NET_RPS_SOFTIRQ, net_rps_action);
+ get_random_bytes(&simple_hashrnd, 4);
hotcpu_notifier(dev_cpu_callback, 0);
dst_init();
dev_mcast_init();
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 753c420..ca250f6 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -18,6 +18,9 @@
#include <linux/wireless.h>
#include <net/wext.h>
+#include <linux/string.h>
+#include <linux/ctype.h>
+
#include "net-sysfs.h"
#ifdef CONFIG_SYSFS
@@ -249,6 +252,137 @@ static ssize_t store_tx_queue_len(struct device *dev,
return netdev_store(dev, attr, buf, len, change_tx_queue_len);
}
+static char *
+get_token(const char **cp, size_t *len)
+{
+ const char *bp = *cp, *start;
+
+ while (isspace(*bp))
+ bp++;
+
+ start = bp;
+ while (!isspace(*bp) && *bp != '\0')
+ bp++;
+
+ if (start != bp)
+ *len = bp - start;
+ else
+ start = NULL;
+
+ *cp = bp;
+ return start;
+}
+
+static ssize_t store_rps_cpus(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t len)
+{
+ struct net_device *net = to_net_dev(dev);
+ struct napi_struct *napi;
+ cpumask_t mask;
+ int err, cpu, index, i;
+ int cnt = 0;
+ char *token;
+ const char *cp = buf;
+ size_t tlen;
+
+ if (!capable(CAP_NET_ADMIN))
+ return -EPERM;
+
+ /*
+ * Pre-check that tokens parse properly before we commit to making
+ * any changes.
+ */
+ while ((token = get_token(&cp, &tlen)))
+ err = bitmap_parse(token, tlen, cpumask_bits(&mask),
+ nr_cpumask_bits);
+
+ if (err)
+ return err;
+
+ rtnl_lock();
+ if (dev_isalive(net)) {
+ if (!net->rps_maps) {
+ /*
+ * Need to allocate the array of RPS maps, one map
+ * for each NAPI instance on the device.
+ */
+ list_for_each_entry(napi, &net->napi_list, dev_list)
+ cnt++;
+ net->rps_maps = kzalloc(RPS_MAP_SIZE * cnt, GFP_KERNEL);
+ if (!net->rps_maps) {
+ rtnl_unlock();
+ return -ENOMEM;
+ }
+ net->rps_num_maps = cnt;
+ }
+
+ cp = buf;
+ for (index = 0; index < net->rps_num_maps &&
+ (token = get_token(&cp, &tlen)); index++) {
+ struct rps_map *map = (struct rps_map *)
+ (net->rps_maps + (RPS_MAP_SIZE * index));
+ err = bitmap_parse(token, tlen, cpumask_bits(&mask),
+ nr_cpumask_bits);
+ if (!err) {
+ cpus_and(mask, mask, cpu_online_map);
+ i = 0;
+ for_each_cpu_mask(cpu, mask)
+ map->map[i++] = cpu;
+ map->len = i;
+ } else {
+ rtnl_unlock();
+ return err;
+ }
+ }
+
+ /*
+ * Any per NAPI maps not being set are "zeroed" by setting
+ * map length length to zero.
+ */
+ for (; index < net->rps_num_maps; index++) {
+ struct rps_map *map = (struct rps_map *)
+ (net->rps_maps + (RPS_MAP_SIZE * index));
+ map->len = 0;
+ }
+ }
+ rtnl_unlock();
+
+ return len;
+}
+
+static ssize_t show_rps_cpus(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct net_device *net = to_net_dev(dev);
+ size_t len = 0;
+ cpumask_t mask;
+ int i, j;
+
+ read_lock(&dev_base_lock);
+ if (dev_isalive(net)) {
+ for (j = 0; j < net->rps_num_maps; j++) {
+ struct rps_map *map = (struct rps_map *)
+ (net->rps_maps + (RPS_MAP_SIZE * j));
+ cpus_clear(mask);
+ for (i = 0; i < map->len; i++)
+ cpu_set(map->map[i], mask);
+
+ len += cpumask_scnprintf(buf + len, PAGE_SIZE, &mask);
+ if (PAGE_SIZE - len < 3) {
+ read_unlock(&dev_base_lock);
+ return -EINVAL;
+ }
+ if (j < net->rps_num_maps)
+ len += sprintf(buf + len, " ");
+ }
+ }
+
+ read_unlock(&dev_base_lock);
+
+ len += sprintf(buf + len, "\n");
+ return len;
+}
+
static ssize_t store_ifalias(struct device *dev, struct device_attribute *attr,
const char *buf, size_t len)
{
@@ -305,6 +439,7 @@ static struct device_attribute net_class_attributes[] = {
__ATTR(flags, S_IRUGO | S_IWUSR, show_flags, store_flags),
__ATTR(tx_queue_len, S_IRUGO | S_IWUSR, show_tx_queue_len,
store_tx_queue_len),
+ __ATTR(rps_cpus, S_IRUGO | S_IWUSR, show_rps_cpus, store_rps_cpus),
{}
};
--
1.5.4.3
^ permalink raw reply related
* [PATCH 2/2] rps: changes to bnx2x to get device hash
From: Tom Herbert @ 2009-11-11 6:53 UTC (permalink / raw)
To: David Miller, netdev
bnx2x changes to get Toeplitz hash on RX and out into skb.
Signed-off-by: Tom Herbert <therbert@google.com>
---
drivers/net/bnx2x.h | 2 +-
drivers/net/bnx2x_main.c | 47 ++++++++++++++++++++++++++++++++++++++++-----
2 files changed, 42 insertions(+), 7 deletions(-)
diff --git a/drivers/net/bnx2x.h b/drivers/net/bnx2x.h
index c3b32f7..a876d78 100644
--- a/drivers/net/bnx2x.h
+++ b/drivers/net/bnx2x.h
@@ -1295,7 +1295,7 @@ static inline u32 reg_poll(struct bnx2x *bp, u32
reg, u32 expected, int ms,
AEU_INPUTS_ATTN_BITS_MISC_PARITY_ERROR)
-#define MULTI_FLAGS(bp) \
+#define RSS_FLAGS(bp) \
(TSTORM_ETH_FUNCTION_COMMON_CONFIG_RSS_IPV4_CAPABILITY | \
TSTORM_ETH_FUNCTION_COMMON_CONFIG_RSS_IPV4_TCP_CAPABILITY | \
TSTORM_ETH_FUNCTION_COMMON_CONFIG_RSS_IPV6_CAPABILITY | \
diff --git a/drivers/net/bnx2x_main.c b/drivers/net/bnx2x_main.c
index 59b58d8..1729c04 100644
--- a/drivers/net/bnx2x_main.c
+++ b/drivers/net/bnx2x_main.c
@@ -83,6 +83,10 @@ module_param(multi_mode, int, 0);
MODULE_PARM_DESC(multi_mode, " Multi queue mode "
"(0 Disable; 1 Enable (default))");
+static int get_hdrhash = 1;
+module_param(get_hdrhash, int, 0);
+MODULE_PARM_DESC(get_hdrhash, " Get Toeplitz hash from device");
+
static int num_rx_queues;
module_param(num_rx_queues, int, 0);
MODULE_PARM_DESC(num_rx_queues, " Number of Rx queues for multi_mode=1"
@@ -1520,7 +1524,7 @@ static int bnx2x_rx_int(struct bnx2x_fastpath
*fp, int budget)
struct sw_rx_bd *rx_buf = NULL;
struct sk_buff *skb;
union eth_rx_cqe *cqe;
- u8 cqe_fp_flags;
+ u8 cqe_fp_flags, cqe_fp_status_flags, cqe_fp_pars_flags;
u16 len, pad;
comp_ring_cons = RCQ_BD(sw_comp_cons);
@@ -1536,6 +1540,8 @@ static int bnx2x_rx_int(struct bnx2x_fastpath
*fp, int budget)
cqe = &fp->rx_comp_ring[comp_ring_cons];
cqe_fp_flags = cqe->fast_path_cqe.type_error_flags;
+ cqe_fp_status_flags = cqe->fast_path_cqe.status_flags;
+ cqe_fp_pars_flags = cqe->fast_path_cqe.pars_flags.flags;
DP(NETIF_MSG_RX_STATUS, "CQE type %x err %x status %x"
" queue %x vlan %x len %u\n", CQE_TYPE(cqe_fp_flags),
@@ -1661,7 +1667,32 @@ reuse_rx:
goto next_rx;
}
- skb->protocol = eth_type_trans(skb, bp->dev);
+
+ if (get_hdrhash && (cqe_fp_status_flags &
+ ETH_FAST_PATH_RX_CQE_RSS_HASH_FLG)) {
+ u8 hash_type = cqe_fp_status_flags &
+ ETH_FAST_PATH_RX_CQE_RSS_HASH_TYPE;
+
+ skb->rxhash = le32_to_cpu(
+ cqe->fast_path_cqe.rss_hash_result);
+ if (!skb->rxhash)
+ skb->rxhash = 1;
+
+ /* unicast IPv4 packet? */
+ if (((hash_type == IPV4_HASH_TYPE) ||
+ (hash_type == TCP_IPV4_HASH_TYPE)) &&
+ (cqe_fp_pars_flags &
+ PARSING_FLAGS_ETHERNET_ADDRESS_TYPE)) {
+ skb->dev = bp->dev;
+ skb_reset_mac_header(skb);
+ skb_pull(skb, ETH_HLEN);
+ skb->protocol =
+ __constant_htons(ETH_P_IP);
+ } else
+ skb->protocol =
+ eth_type_trans(skb, bp->dev);
+ } else
+ skb->protocol = eth_type_trans(skb, bp->dev);
skb->ip_summed = CHECKSUM_NONE;
if (bp->rx_csum) {
@@ -5388,8 +5419,11 @@ static void bnx2x_init_internal_func(struct bnx2x *bp)
u16 max_agg_size;
if (is_multi(bp)) {
- tstorm_config.config_flags = MULTI_FLAGS(bp);
+ tstorm_config.config_flags = RSS_FLAGS(bp);
tstorm_config.rss_result_mask = MULTI_MASK;
+ } else if (get_hdrhash) {
+ /* Set flags so the Toeplitz hash is provided */
+ tstorm_config.config_flags = RSS_FLAGS(bp);
}
/* Enable TPA if needed */
@@ -6223,9 +6257,10 @@ static int bnx2x_init_common(struct bnx2x *bp)
bnx2x_init_block(bp, PBF_BLOCK, COMMON_STAGE);
REG_WR(bp, SRC_REG_SOFT_RST, 1);
- for (i = SRC_REG_KEYRSS0_0; i <= SRC_REG_KEYRSS1_9; i += 4) {
- REG_WR(bp, i, 0xc0cac01a);
- /* TODO: replace with something meaningful */
+ {
+ int i;
+ for (i = SRC_REG_KEYRSS0_0; i <= SRC_REG_KEYRSS1_9; i += 4)
+ REG_WR(bp, i, random32());
}
bnx2x_init_block(bp, SRCH_BLOCK, COMMON_STAGE);
#ifdef BCM_CNIC
--
1.5.4.3
^ permalink raw reply related
* Re: xfrm command fails
From: Naveen BN @ 2009-11-11 7:10 UTC (permalink / raw)
To: netdev; +Cc: David Miller
In-Reply-To: <4AFA56B1.1010109@globaledgesoft.com>
Hi All,
This proper command to create SA using ip xfrm is
ip xfrm state add src 192.168.1.20 dst 192.168.1.5 proto esp spi
0x354cb04b mode tunnel reqid 16385 replay-window 32 auth sha1
0xecf02a5cf6568556e1bdcd961c7ec3f92afd01cc enc aes
0x5c0cfa9672ce67ba545b593076dfb278 sel src 0.0.0.0/0 dst 0.0.0.0/0
But how can create a session based SA with ports also used in sa
selection for the same host .
Please help me on this . In selection of OUTBOUND SA does ports also
involve.
Regards
Naveen
Naveen BN wrote:
> adding David in loop
> Naveen BN wrote:
>> I am using fedora with version => 2.6.23.1-42.fc8.
>> Please guide me to solve the error message "RTNETLINK answers:
>> Function not implemented"
>>
>> Regards
>> Naveen
>> Naveen BN wrote:
>>> Hi All
>>> I am trying to create a security association using ip xfrm command,
>>> but i get an message saying
>>> "*RTNETLINK answers: Function not implemented*" and ip xfrm state
>>> succeeds in displaying sa written using setkey .
>>>
>>> ip xfrm state add src 192.168.1.20 dst 192.168.1.5 proto esp spi
>>> 0x354cb04b mode tunnel reqid 16385 replay-window 32 auth hmac
>>> 0xecf02a5cf6568556e1bdcd961c7ec3f92afd01cc enc cbc
>>> 0x5c0cfa9672ce67ba545b593076dfb278 sel src 0.0.0.0/0 dst 0.0.0.0/0
>>>
>>> ERROR message => "RTNETLINK answers: Function not implemented"
>>>
>>> Regards
>>> Naveen
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: [Bugme-new] [Bug 14546] New: Off-by-two stack buffer overflow in function rpc_uaddr2sockaddr() of net/sunrpc/addr.c
From: Patroklos Argyroudis @ 2009-11-11 7:51 UTC (permalink / raw)
To: Chuck Lever
Cc: bugzilla-daemon-590EEB7GvNiWaY/ihj7yzEB+6BGkLq7r,
bugme-daemon-590EEB7GvNiWaY/ihj7yzEB+6BGkLq7r,
Linux Network Developers, J. Bruce Fields, Trond Myklebust,
Neil Brown, Andrew Morton, Linux NFS Mailing list
In-Reply-To: <967DC2CE-588D-4207-BF2D-59727454DC2E-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
On Nov 10, 2009, at 6:29 PM, Andrew Morton wrote:
> >
> >Please don't submit patches via bugzilla.
> >
> >Please prepare this patch as per Documentation/SubmittingPatches and
> >email it to all the recipients of this email, thanks.
Ok, I will do so.
On Tue, Nov 10, 2009 at 06:38:05PM -0500, Chuck Lever wrote:
> Why wouldn't you bump the size of the buffer by two as well?
> Otherwise valid universal addresses that are RPCBIND_MAXUADDRLEN
> bytes long will fail here.
>
> > memcpy(buf, uaddr, uaddr_len);
There is no need to increase the size of the buffer since the new check
(if (uaddr_len > sizeof(buf) - 2)) will terminate the function in case
the valid universal address is RPCBIND_MAXUADDRLEN bytes.
Cheers,
Patroklos
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] act_mirred: don't go back.
From: jamal @ 2009-11-11 8:11 UTC (permalink / raw)
To: Changli Gao; +Cc: Stephen Hemminger, David S. Miller, netdev
In-Reply-To: <412e6f7f0911100050k39920adfl1ab70e345269833a@mail.gmail.com>
Hi,
On Tue, 2009-11-10 at 16:50 +0800, Changli Gao wrote:
> Where? After skb2 is allocated, there won't be any failure any more.
Sorry, yes, your change does look like an improvement.
If you can test a little more - please add my sign-off and submit.
>
> It's just OK. When using tc, I also found act_mirred doesn't support
> ingress, then I realized that there isn't any difference between
> ingress and egress, as it depends on its parent. However I do think it
> is confused, when it prints:
> filter parent ffff: protocol ip pref 49152 basic handle 0x1
> action order 1: mirred (Egress Redirect to device ifb0) stolen
> index 5 ref 1 bind 1.
> And the TODO note still is in the source code of act_mirred, it do
> make me wonder for a while!
Well, you know if you have the energy - doing socket redirect would
be very interesting indeed. Ingress is another beast, it requires some
thinking and a lot more testing depending on the devices because of
possible loops.
cheers,
jamal
^ permalink raw reply
* Re: [PATCH 1/2] rps: core implementation
From: Eric Dumazet @ 2009-11-11 8:20 UTC (permalink / raw)
To: Tom Herbert; +Cc: David Miller, netdev
In-Reply-To: <65634d660911102253o2b4f7a19kfed5849e5c88bfe1@mail.gmail.com>
Tom Herbert a écrit :
> Third version of RPS.
>
> Signed-off-by: Tom Herbert <therbert@google.com>
> ---
> include/linux/interrupt.h | 1 +
> include/linux/netdevice.h | 18 ++++
> include/linux/skbuff.h | 2 +
> net/core/dev.c | 227 ++++++++++++++++++++++++++++++++++++++-------
> net/core/net-sysfs.c | 135 +++++++++++++++++++++++++++
> 5 files changed, 348 insertions(+), 35 deletions(-)
>
I must say this is really exciting :)
> diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
> index b78cf81..fa91194 100644
> --- a/include/linux/interrupt.h
> +++ b/include/linux/interrupt.h
> @@ -345,6 +345,7 @@ enum
> TIMER_SOFTIRQ,
> NET_TX_SOFTIRQ,
> NET_RX_SOFTIRQ,
> + NET_RPS_SOFTIRQ,
> BLOCK_SOFTIRQ,
> BLOCK_IOPOLL_SOFTIRQ,
> TASKLET_SOFTIRQ,
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 8380009..c1b1bbb 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -639,6 +639,18 @@ struct net_device_ops {
> };
>
> /*
> + * Structure for Receive Packet Steering. Length of map and array of CPU ID's.
> + */
> +struct rps_map {
> + int len;
> + u16 map[0];
> +};
> +
> +/* Maximum size of RPS map (for allocation) */
> +#define RPS_MAP_SIZE (sizeof(struct rps_map) + \
> + (num_possible_cpus() * sizeof(u16)))
> +
Problem of possible cpus is the number can be very large on some arches,
but yet few cpus online....
In this kind of situation, get_rps_cpu() will return -1 most of the time,
defeating goal of RPS ?
> +/*
> * The DEVICE structure.
> * Actually, this whole structure is a big mistake. It mixes I/O
> * data with strictly "high-level" data, and it has to know about
> @@ -807,6 +819,9 @@ struct net_device
> void *ax25_ptr; /* AX.25 specific data */
> struct wireless_dev *ieee80211_ptr; /* IEEE 802.11 specific data,
> assign before registering */
> + void *rps_maps; /* Array of per-NAPI maps for
> + receive packet steeing */
> + int rps_num_maps; /* Number of RPS maps */
>
> /*
> * Cache line mostly used on receive path (including eth_type_trans())
> @@ -1217,6 +1232,9 @@ struct softnet_data
> struct Qdisc *output_queue;
> struct sk_buff_head input_pkt_queue;
> struct list_head poll_list;
> +
> + struct call_single_data csd;
> +
> struct sk_buff *completion_queue;
>
> struct napi_struct backlog;
> diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
> index 0c68fbd..95feac7 100644
> --- a/include/linux/skbuff.h
> +++ b/include/linux/skbuff.h
> @@ -396,6 +396,8 @@ struct sk_buff {
>
> __u16 vlan_tci;
>
> + __u32 rxhash;
> +
> sk_buff_data_t transport_header;
> sk_buff_data_t network_header;
> sk_buff_data_t mac_header;
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 28b0b9e..735e7e3 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -1976,6 +1976,162 @@ int weight_p __read_mostly = 64; /*
> old backlog weight */
>
> DEFINE_PER_CPU(struct netif_rx_stats, netdev_rx_stat) = { 0, };
>
> +static u32 simple_hashrnd;
> +
> +/**
> + * get_rps_cpu is called from netif_receive_skb and returns the target
> + * CPU from the RPS map of the receiving NAPI instance for a given skb.
> + */
> +static int get_rps_cpu(struct net_device *dev, struct sk_buff *skb)
> +{
> + u32 addr1, addr2, ports;
> + struct ipv6hdr *ip6;
> + struct iphdr *ip;
> + u32 hash, ihl;
> + u8 ip_proto;
> + int cpu;
> + struct rps_map *map = NULL;
> +
> + if (dev->rps_num_maps) {
> + /*
> + * Locate the map corresponding to the NAPI queue that
> + * the packet was received on.
> + */
> + int index = skb_get_rx_queue(skb);
> + if (index < 0 || index >= dev->rps_num_maps)
> + index = 0;
> +
> + map = (struct rps_map *)
> + (dev->rps_maps + (RPS_MAP_SIZE * index));
> + if (!map->len)
> + map = NULL;
> + }
> +
> + if (!map)
> + return -1;
> +
> + hash = skb->rxhash;
> + if (hash)
> + goto got_hash; /* Skip hash computation on packet header */
> +
> + switch (skb->protocol) {
> + case __constant_htons(ETH_P_IP):
> + if (!pskb_may_pull(skb, sizeof(*ip)))
> + return -1;
> +
> + ip = (struct iphdr *) skb->data;
> + ip_proto = ip->protocol;
> + addr1 = ip->saddr;
> + addr2 = ip->daddr;
> + ihl = ip->ihl;
> + break;
> + case __constant_htons(ETH_P_IPV6):
> + if (!pskb_may_pull(skb, sizeof(*ip6)))
> + return -1;
> +
> + ip6 = (struct ipv6hdr *) skb->data;
> + ip_proto = ip6->nexthdr;
> + addr1 = ip6->saddr.s6_addr32[3];
> + addr2 = ip6->daddr.s6_addr32[3];
> + ihl = (40 >> 2);
> + break;
> + default:
> + return -1;
> + }
> + ports = 0;
> + switch (ip_proto) {
> + case IPPROTO_TCP:
> + case IPPROTO_UDP:
> + case IPPROTO_DCCP:
> + case IPPROTO_ESP:
> + case IPPROTO_AH:
> + case IPPROTO_SCTP:
> + case IPPROTO_UDPLITE:
> + if (pskb_may_pull(skb, (ihl * 4) + 4))
> + ports = *((u32 *) (skb->data + (ihl * 4)));
> + break;
> +
> + default:
> + break;
> + }
> +
> + hash = jhash_3words(addr1, addr2, ports, simple_hashrnd);
I wonder if you tried to exchange addr1/addr2 port1/port2 so that conntracking/routing
is also speedup ...
ie make sure hash will be the same regardless of the direction of packet.
union {
u32 port;
u16 ports[2];
} p;
if (addr1 < addr2)
swap(addr1, addr2);
if (p.ports[0] < p.ports[1]);
swap(p.ports[0], p.ports[1]);
hash = jhash_3words(addr1, addr2, ports, simple_hashrnd);
I think I'll try to extend your patches with TX completion recycling too.
Ie record in skb the cpu number of original sender, and queue skb to
remote queue for destruction (sock_wfree() call and expensive scheduler calls...)
(This probably needs driver cooperation, instead of calling consume_skb(),
use a different function)
Thanks
^ permalink raw reply
* Re: [PATCH] act_mirred: don't go back.
From: Changli Gao @ 2009-11-11 8:44 UTC (permalink / raw)
To: hadi; +Cc: Stephen Hemminger, David S. Miller, netdev
In-Reply-To: <1257927097.4685.4.camel@bigi>
On Wed, Nov 11, 2009 at 4:11 PM, jamal <hadi@cyberus.ca> wrote:
> Hi,
>
> On Tue, 2009-11-10 at 16:50 +0800, Changli Gao wrote:
>
>> Where? After skb2 is allocated, there won't be any failure any more.
>
>
> Sorry, yes, your change does look like an improvement.
> If you can test a little more - please add my sign-off and submit.
OK. I'll do some test first, then send it out for review.
>
>>
>> It's just OK. When using tc, I also found act_mirred doesn't support
>> ingress, then I realized that there isn't any difference between
>> ingress and egress, as it depends on its parent. However I do think it
>> is confused, when it prints:
>> filter parent ffff: protocol ip pref 49152 basic handle 0x1
>> action order 1: mirred (Egress Redirect to device ifb0) stolen
>> index 5 ref 1 bind 1.
>> And the TODO note still is in the source code of act_mirred, it do
>> make me wonder for a while!
>
> Well, you know if you have the energy - doing socket redirect would
> be very interesting indeed. Ingress is another beast, it requires some
> thinking and a lot more testing depending on the devices because of
> possible loops.
>
The current kernel and iproute2 support ingress in another way, that
is attaching the filter to the corresponding ingress qdisc. refter to
http://www.linuxfoundation.org/en/Net:IFB . The loops is avoided in
netif_receive_skb() and with TTL mechanism. Do you have more
documentation or examples about socket redirect? Sorry, I don't know
what it is used to do. Is it sth. like ipt_REDIRECT?
--
Regards，
Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
* Re: [PATCH] act_mirred: don't go back.
From: jamal @ 2009-11-11 9:03 UTC (permalink / raw)
To: Changli Gao; +Cc: Stephen Hemminger, David S. Miller, netdev
In-Reply-To: <412e6f7f0911110044i22c28ab6pa3a3691c014e76a8@mail.gmail.com>
On Wed, 2009-11-11 at 16:44 +0800, Changli Gao wrote:
> The current kernel and iproute2 support ingress in another way, that
> is attaching the filter to the corresponding ingress qdisc. refter to
> http://www.linuxfoundation.org/en/Net:IFB .
that looks like something i wrote ;->
> The loops is avoided in
> netif_receive_skb() and with TTL mechanism.
> Do you have more
> documentation or examples about socket redirect? Sorry, I don't know
> what it is used to do. Is it sth. like ipt_REDIRECT?
I open a socket and I install a filter to mirror or redirect to this
socket. It is possible to do it today if you use tun device; thats
why i have never been motivated to do it.
cheers,
jamal
^ permalink raw reply
* Re: [PATCH] act_mirred: don't go back.
From: Changli Gao @ 2009-11-11 9:23 UTC (permalink / raw)
To: hadi; +Cc: Stephen Hemminger, David S. Miller, netdev
In-Reply-To: <1257930232.4685.20.camel@bigi>
On Wed, Nov 11, 2009 at 5:03 PM, jamal <hadi@cyberus.ca> wrote:
> On Wed, 2009-11-11 at 16:44 +0800, Changli Gao wrote:
>
> I open a socket and I install a filter to mirror or redirect to this
> socket. It is possible to do it today if you use tun device; thats
> why i have never been motivated to do it.
>
PF_PACKET can mirror to socket. And with the help of ingress polices,
it can emulate redirection too. Am I right?
--
Regards，
Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
* [PATCH] ifb: add multi-queue support
From: Changli Gao @ 2009-11-11 9:51 UTC (permalink / raw)
To: David S. Miller
Cc: Stephen Hemminger, Patrick McHardy, Eric Dumazet, Tom Herbert,
netdev
ifb: add multi-queue support
Add multi-queue support, and one kernel thread is created for per queue.
It can used to emulate multi-queue NIC in software, and distribute work
among CPUs.
gentux linux # modprobe ifb numtxqs=2
gentux linux # ifconfig ifb0 up
gentux linux # pgrep ifb0
18508
18509
gentux linux # taskset -p 1 18508
pid 18508's current affinity mask: 3
pid 18508's new affinity mask: 1
gentux linux # taskset -p 2 18509
pid 18509's current affinity mask: 3
pid 18509's new affinity mask: 2
gentux linux # tc qdisc add dev br0 ingress
gentux linux # tc filter add dev br0 parent ffff: protocol ip basic
action mirred egress redirect dev ifb0
This patch also introduces a ip link option "numtxqs" for specifying the
number of the TX queues. so you can add a new ifb with the command:
ip link add numtxqs 4 type ifb
Signed-off-by: Changli Gao <xiaosuo@gmail.com>
----
drivers/net/ifb.c | 414 ++++++++++++++++++++++++++++++++----------------
include/linux/if_link.h | 1
net/core/rtnetlink.c | 3
3 files changed, 283 insertions(+), 135 deletions(-)
diff --git a/drivers/net/ifb.c b/drivers/net/ifb.c
index 69c2566..ac04e85 100644
--- a/drivers/net/ifb.c
+++ b/drivers/net/ifb.c
@@ -33,161 +33,157 @@
#include <linux/etherdevice.h>
#include <linux/init.h>
#include <linux/moduleparam.h>
+#include <linux/wait.h>
+#include <linux/sched.h>
+#include <linux/kthread.h>
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <net/ip.h>
#include <net/pkt_sched.h>
#include <net/net_namespace.h>
-#define TX_TIMEOUT (2*HZ)
-
#define TX_Q_LIMIT 32
+
+struct ifb_private_q {
+ struct net_device *dev;
+ struct sk_buff_head rq;
+ struct sk_buff_head tq;
+ wait_queue_head_t wq;
+ struct task_struct *task;
+ unsigned long rx_packets;
+ unsigned long rx_bytes;
+ unsigned long rx_dropped;
+} ____cacheline_aligned_in_smp;
+
struct ifb_private {
- struct tasklet_struct ifb_tasklet;
- int tasklet_pending;
- /* mostly debug stats leave in for now */
- unsigned long st_task_enter; /* tasklet entered */
- unsigned long st_txq_refl_try; /* transmit queue refill attempt */
- unsigned long st_rxq_enter; /* receive queue entered */
- unsigned long st_rx2tx_tran; /* receive to trasmit transfers */
- unsigned long st_rxq_notenter; /*receiveQ not entered, resched */
- unsigned long st_rx_frm_egr; /* received from egress path */
- unsigned long st_rx_frm_ing; /* received from ingress path */
- unsigned long st_rxq_check;
- unsigned long st_rxq_rsch;
- struct sk_buff_head rq;
- struct sk_buff_head tq;
+ struct ifb_private_q *pq;
};
+/* Number of ifb devices to be set up by this module. */
static int numifbs = 2;
+module_param(numifbs, int, 0444);
+MODULE_PARM_DESC(numifbs, "Number of ifb devices");
-static void ri_tasklet(unsigned long dev);
-static netdev_tx_t ifb_xmit(struct sk_buff *skb, struct net_device *dev);
-static int ifb_open(struct net_device *dev);
-static int ifb_close(struct net_device *dev);
+/* Number of TX queues per ifb */
+static int numtxqs = 1;
+module_param(numtxqs, int, 0444);
+MODULE_PARM_DESC(numtxqs, "Number of TX queues per ifb");
-static void ri_tasklet(unsigned long dev)
+static int ifb_thread(void *priv)
{
-
- struct net_device *_dev = (struct net_device *)dev;
- struct ifb_private *dp = netdev_priv(_dev);
- struct net_device_stats *stats = &_dev->stats;
- struct netdev_queue *txq;
+ struct ifb_private_q *pq = priv;
+ struct net_device *dev = pq->dev;
+ int num = pq - ((struct ifb_private *)netdev_priv(dev))->pq;
+ struct netdev_queue *txq = netdev_get_tx_queue(dev, num);
struct sk_buff *skb;
-
- txq = netdev_get_tx_queue(_dev, 0);
- dp->st_task_enter++;
- if ((skb = skb_peek(&dp->tq)) == NULL) {
- dp->st_txq_refl_try++;
- if (__netif_tx_trylock(txq)) {
- dp->st_rxq_enter++;
- while ((skb = skb_dequeue(&dp->rq)) != NULL) {
- skb_queue_tail(&dp->tq, skb);
- dp->st_rx2tx_tran++;
- }
- __netif_tx_unlock(txq);
- } else {
- /* reschedule */
- dp->st_rxq_notenter++;
- goto resched;
+ DEFINE_WAIT(wait);
+
+ while (1) {
+ /* move skb from rq to tq */
+ while (1) {
+ prepare_to_wait(&pq->wq, &wait, TASK_UNINTERRUPTIBLE);
+ __netif_tx_lock_bh(txq);
+ while ((skb = skb_dequeue(&pq->rq)) != NULL)
+ skb_queue_tail(&pq->tq, skb);
+ if (netif_queue_stopped(dev))
+ netif_wake_queue(dev);
+ __netif_tx_unlock_bh(txq);
+ if (kthread_should_stop() || !skb_queue_empty(&pq->tq))
+ break;
+ schedule();
}
- }
-
- while ((skb = skb_dequeue(&dp->tq)) != NULL) {
- u32 from = G_TC_FROM(skb->tc_verd);
-
- skb->tc_verd = 0;
- skb->tc_verd = SET_TC_NCLS(skb->tc_verd);
- stats->tx_packets++;
- stats->tx_bytes +=skb->len;
-
- rcu_read_lock();
- skb->dev = dev_get_by_index_rcu(&init_net, skb->iif);
- if (!skb->dev) {
- rcu_read_unlock();
- dev_kfree_skb(skb);
- stats->tx_dropped++;
+ finish_wait(&pq->wq, &wait);
+ if (kthread_should_stop())
break;
+ if (need_resched())
+ schedule();
+
+ /* transfer packets */
+ while ((skb = skb_dequeue(&pq->tq)) != NULL) {
+ u32 from = G_TC_FROM(skb->tc_verd);
+
+ skb->tc_verd = 0;
+ skb->tc_verd = SET_TC_NCLS(skb->tc_verd);
+ txq->tx_packets++;
+ txq->tx_bytes +=skb->len;
+
+ rcu_read_lock();
+ skb->dev = dev_get_by_index_rcu(&init_net, skb->iif);
+ if (!skb->dev) {
+ rcu_read_unlock();
+ dev_kfree_skb(skb);
+ txq->tx_dropped++;
+ break;
+ }
+ rcu_read_unlock();
+ skb->iif = dev->ifindex;
+
+ if (from & AT_EGRESS) {
+ dev_queue_xmit(skb);
+ } else if (from & AT_INGRESS) {
+ skb_pull(skb, skb->dev->hard_header_len);
+ netif_rx_ni(skb);
+ } else
+ BUG();
}
- rcu_read_unlock();
- skb->iif = _dev->ifindex;
-
- if (from & AT_EGRESS) {
- dp->st_rx_frm_egr++;
- dev_queue_xmit(skb);
- } else if (from & AT_INGRESS) {
- dp->st_rx_frm_ing++;
- skb_pull(skb, skb->dev->hard_header_len);
- netif_rx(skb);
- } else
- BUG();
- }
-
- if (__netif_tx_trylock(txq)) {
- dp->st_rxq_check++;
- if ((skb = skb_peek(&dp->rq)) == NULL) {
- dp->tasklet_pending = 0;
- if (netif_queue_stopped(_dev))
- netif_wake_queue(_dev);
- } else {
- dp->st_rxq_rsch++;
- __netif_tx_unlock(txq);
- goto resched;
- }
- __netif_tx_unlock(txq);
- } else {
-resched:
- dp->tasklet_pending = 1;
- tasklet_schedule(&dp->ifb_tasklet);
}
+ return 0;
}
-static const struct net_device_ops ifb_netdev_ops = {
- .ndo_open = ifb_open,
- .ndo_stop = ifb_close,
- .ndo_start_xmit = ifb_xmit,
- .ndo_validate_addr = eth_validate_addr,
-};
-
-static void ifb_setup(struct net_device *dev)
+struct net_device_stats* ifb_get_stats(struct net_device *dev)
{
- /* Initialize the device structure. */
- dev->destructor = free_netdev;
- dev->netdev_ops = &ifb_netdev_ops;
+ struct net_device_stats *stats = &dev->stats;
+ struct ifb_private *dp = netdev_priv(dev);
+ struct ifb_private_q *pq = dp->pq;
+ struct netdev_queue *txq;
+ int i;
+ unsigned long rx_packets = 0, rx_bytes = 0, rx_dropped = 0;
+ unsigned long tx_packets = 0, tx_bytes = 0, tx_dropped = 0;
+
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ rx_packets += pq[i].rx_packets;
+ rx_bytes += pq[i].rx_bytes;
+ rx_dropped += pq[i].rx_dropped;
+ txq = netdev_get_tx_queue(dev, i);
+ tx_packets += txq->tx_packets;
+ tx_bytes += txq->tx_bytes;
+ tx_dropped += txq->tx_dropped;
+ }
- /* Fill in device structure with ethernet-generic values. */
- ether_setup(dev);
- dev->tx_queue_len = TX_Q_LIMIT;
+ stats->rx_packets = rx_packets;
+ stats->rx_bytes = rx_bytes;
+ stats->rx_dropped = rx_dropped;
+ stats->tx_packets = tx_packets;
+ stats->tx_bytes = tx_bytes;
+ stats->tx_dropped = tx_dropped;
- dev->flags |= IFF_NOARP;
- dev->flags &= ~IFF_MULTICAST;
- dev->priv_flags &= ~IFF_XMIT_DST_RELEASE;
- random_ether_addr(dev->dev_addr);
+ return stats;
}
static netdev_tx_t ifb_xmit(struct sk_buff *skb, struct net_device *dev)
{
- struct ifb_private *dp = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
u32 from = G_TC_FROM(skb->tc_verd);
+ int num = skb_get_queue_mapping(skb);
+ struct ifb_private *dp = netdev_priv(dev);
+ struct ifb_private_q *pq = dp->pq + num;
+ struct netdev_queue *txq = netdev_get_tx_queue(dev, num);
- stats->rx_packets++;
- stats->rx_bytes+=skb->len;
+ pq->rx_packets++;
+ pq->rx_bytes += skb->len;
if (!(from & (AT_INGRESS|AT_EGRESS)) || !skb->iif) {
dev_kfree_skb(skb);
- stats->rx_dropped++;
+ pq->rx_dropped++;
return NETDEV_TX_OK;
}
- if (skb_queue_len(&dp->rq) >= dev->tx_queue_len) {
+ txq->trans_start = jiffies;
+ skb_queue_tail(&pq->rq, skb);
+ if (skb_queue_len(&pq->rq) >= dev->tx_queue_len)
netif_stop_queue(dev);
- }
-
- dev->trans_start = jiffies;
- skb_queue_tail(&dp->rq, skb);
- if (!dp->tasklet_pending) {
- dp->tasklet_pending = 1;
- tasklet_schedule(&dp->ifb_tasklet);
- }
+ if (skb_queue_len(&pq->rq) == 1)
+ wake_up(&pq->wq);
return NETDEV_TX_OK;
}
@@ -195,26 +191,162 @@ static netdev_tx_t ifb_xmit(struct sk_buff *skb, struct net_device *dev)
static int ifb_close(struct net_device *dev)
{
struct ifb_private *dp = netdev_priv(dev);
+ struct ifb_private_q *pq = dp->pq;
+ int i;
- tasklet_kill(&dp->ifb_tasklet);
netif_stop_queue(dev);
- skb_queue_purge(&dp->rq);
- skb_queue_purge(&dp->tq);
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ kthread_stop(pq[i].task);
+ skb_queue_purge(&pq[i].tq);
+ skb_queue_purge(&pq[i].rq);
+ }
+
return 0;
}
static int ifb_open(struct net_device *dev)
{
struct ifb_private *dp = netdev_priv(dev);
-
- tasklet_init(&dp->ifb_tasklet, ri_tasklet, (unsigned long)dev);
- skb_queue_head_init(&dp->rq);
- skb_queue_head_init(&dp->tq);
+ struct ifb_private_q *pq = dp->pq;
+ int i;
+
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ pq[i].task = kthread_run(ifb_thread, &pq[i], "%s/%d", dev->name,
+ i);
+ if (IS_ERR(pq[i].task)) {
+ int err = PTR_ERR(pq[i].task);
+ while (--i >= 0)
+ kthread_stop(pq[i].task);
+ return err;
+ }
+ }
netif_start_queue(dev);
return 0;
}
+static u32 simple_tx_hashrnd;
+
+static u16 ifb_select_queue(struct net_device *dev, struct sk_buff *skb)
+{
+ u32 addr1, addr2;
+ u32 hash, ihl;
+ union {
+ u16 in16[2];
+ u32 in32;
+ } ports;
+ u8 ip_proto;
+
+ if ((hash = skb_rx_queue_recorded(skb))) {
+ while (hash >= dev->real_num_tx_queues)
+ hash -= dev->real_num_tx_queues;
+ return hash;
+ }
+
+ switch (skb->protocol) {
+ case __constant_htons(ETH_P_IP):
+ if (!(ip_hdr(skb)->frag_off & htons(IP_MF | IP_OFFSET)))
+ ip_proto = ip_hdr(skb)->protocol;
+ else
+ ip_proto = 0;
+ addr1 = ip_hdr(skb)->saddr;
+ addr2 = ip_hdr(skb)->daddr;
+ ihl = ip_hdr(skb)->ihl << 2;
+ break;
+ case __constant_htons(ETH_P_IPV6):
+ ip_proto = ipv6_hdr(skb)->nexthdr;
+ addr1 = ipv6_hdr(skb)->saddr.s6_addr32[3];
+ addr2 = ipv6_hdr(skb)->daddr.s6_addr32[3];
+ ihl = 10;
+ break;
+ default:
+ return 0;
+ }
+ if (addr1 > addr2)
+ swap(addr1, addr2);
+
+ switch (ip_proto) {
+ case IPPROTO_TCP:
+ case IPPROTO_UDP:
+ case IPPROTO_DCCP:
+ case IPPROTO_ESP:
+ case IPPROTO_AH:
+ case IPPROTO_SCTP:
+ case IPPROTO_UDPLITE:
+ ports.in32 = *((u32 *) (skb_network_header(skb) + ihl));
+ if (ports.in16[0] > ports.in16[1])
+ swap(ports.in16[0], ports.in16[1]);
+ break;
+
+ default:
+ ports.in32 = 0;
+ break;
+ }
+
+ hash = jhash_3words(addr1, addr2, ports.in32,
+ simple_tx_hashrnd ^ ip_proto);
+
+ return (u16) (((u64) hash * dev->real_num_tx_queues) >> 32);
+}
+
+static int ifb_init(struct net_device *dev)
+{
+ struct ifb_private *dp = netdev_priv(dev);
+ struct ifb_private_q *pq = dp->pq;
+ int i;
+
+ pq = kmalloc(sizeof(*pq) * dev->real_num_tx_queues, GFP_KERNEL);
+ if (pq == NULL)
+ return -ENOMEM;
+ dp->pq = pq;
+
+ for (i = 0; i < dev->real_num_tx_queues; i++) {
+ pq[i].dev = dev;
+ skb_queue_head_init(&pq[i].rq);
+ skb_queue_head_init(&pq[i].tq);
+ init_waitqueue_head(&pq[i].wq);
+ pq[i].rx_packets = 0;
+ pq[i].rx_bytes = 0;
+ pq[i].rx_dropped = 0;
+ }
+
+ return 0;
+}
+
+static void ifb_uninit(struct net_device *dev)
+{
+ struct ifb_private *dp = netdev_priv(dev);
+
+ kfree(dp->pq);
+}
+
+static const struct net_device_ops ifb_netdev_ops = {
+ .ndo_init = ifb_init,
+ .ndo_uninit = ifb_uninit,
+ .ndo_open = ifb_open,
+ .ndo_stop = ifb_close,
+ .ndo_start_xmit = ifb_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_select_queue = ifb_select_queue,
+ .ndo_get_stats = ifb_get_stats,
+};
+
+static void ifb_setup(struct net_device *dev)
+{
+ /* Initialize the device structure. */
+ dev->destructor = free_netdev;
+ dev->netdev_ops = &ifb_netdev_ops;
+
+ /* Fill in device structure with ethernet-generic values. */
+ ether_setup(dev);
+ dev->tx_queue_len = TX_Q_LIMIT;
+
+ dev->flags |= IFF_NOARP;
+ dev->flags &= ~IFF_MULTICAST;
+ dev->priv_flags &= ~IFF_XMIT_DST_RELEASE;
+ random_ether_addr(dev->dev_addr);
+}
+
static int ifb_validate(struct nlattr *tb[], struct nlattr *data[])
{
if (tb[IFLA_ADDRESS]) {
@@ -226,24 +358,36 @@ static int ifb_validate(struct nlattr *tb[], struct nlattr *data[])
return 0;
}
+static int ifb_get_tx_queues(struct net *net, struct nlattr *tb[],
+ unsigned int *num_tx_queues,
+ unsigned int *real_num_tx_queues)
+{
+ if (tb[IFLA_NTXQ]) {
+ *num_tx_queues = nla_get_u16(tb[IFLA_NTXQ]);
+ *real_num_tx_queues = *num_tx_queues;
+ } else {
+ *num_tx_queues = numtxqs;
+ *real_num_tx_queues = numtxqs;
+ }
+
+ return 0;
+}
+
static struct rtnl_link_ops ifb_link_ops __read_mostly = {
.kind = "ifb",
- .priv_size = sizeof(struct ifb_private),
.setup = ifb_setup,
.validate = ifb_validate,
+ .get_tx_queues = ifb_get_tx_queues,
+ .priv_size = sizeof(struct ifb_private),
};
-/* Number of ifb devices to be set up by this module. */
-module_param(numifbs, int, 0);
-MODULE_PARM_DESC(numifbs, "Number of ifb devices");
-
static int __init ifb_init_one(int index)
{
struct net_device *dev_ifb;
int err;
- dev_ifb = alloc_netdev(sizeof(struct ifb_private),
- "ifb%d", ifb_setup);
+ dev_ifb = alloc_netdev_mq(sizeof(struct ifb_private), "ifb%d",
+ ifb_setup, numtxqs);
if (!dev_ifb)
return -ENOMEM;
@@ -268,9 +412,9 @@ static int __init ifb_init_module(void)
{
int i, err;
+ get_random_bytes(&simple_tx_hashrnd, 4);
rtnl_lock();
err = __rtnl_link_register(&ifb_link_ops);
-
for (i = 0; i < numifbs && !err; i++)
err = ifb_init_one(i);
if (err)
diff --git a/include/linux/if_link.h b/include/linux/if_link.h
index 1d3b242..99da411 100644
--- a/include/linux/if_link.h
+++ b/include/linux/if_link.h
@@ -78,6 +78,7 @@ enum {
#define IFLA_LINKINFO IFLA_LINKINFO
IFLA_NET_NS_PID,
IFLA_IFALIAS,
+ IFLA_NTXQ,
__IFLA_MAX
};
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 33148a5..7b94fd7 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -597,6 +597,7 @@ static inline size_t if_nlmsg_size(const struct net_device *dev)
+ nla_total_size(4) /* IFLA_MASTER */
+ nla_total_size(1) /* IFLA_OPERSTATE */
+ nla_total_size(1) /* IFLA_LINKMODE */
+ + nla_total_size(2) /* IFLA_NTXQ */
+ rtnl_link_get_size(dev); /* IFLA_LINKINFO */
}
@@ -627,6 +628,7 @@ static int rtnl_fill_ifinfo(struct sk_buff *skb, struct net_device *dev,
netif_running(dev) ? dev->operstate : IF_OPER_DOWN);
NLA_PUT_U8(skb, IFLA_LINKMODE, dev->link_mode);
NLA_PUT_U32(skb, IFLA_MTU, dev->mtu);
+ NLA_PUT_U16(skb, IFLA_NTXQ, dev->real_num_tx_queues);
if (dev->ifindex != dev->iflink)
NLA_PUT_U32(skb, IFLA_LINK, dev->iflink);
@@ -725,6 +727,7 @@ const struct nla_policy ifla_policy[IFLA_MAX+1] = {
[IFLA_LINKINFO] = { .type = NLA_NESTED },
[IFLA_NET_NS_PID] = { .type = NLA_U32 },
[IFLA_IFALIAS] = { .type = NLA_STRING, .len = IFALIASZ-1 },
+ [IFLA_NTXQ] = { .type = NLA_U16 },
};
EXPORT_SYMBOL(ifla_policy);
^ permalink raw reply related
* Re: [PATCH] ifb: add multi-queue support
From: Changli Gao @ 2009-11-11 9:56 UTC (permalink / raw)
To: David S. Miller, Stephen Hemminger
Cc: Patrick McHardy, Eric Dumazet, Tom Herbert, netdev
In-Reply-To: <4AFA8911.7050204@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]
2009/11/11 Changli Gao <xiaosuo@gmail.com>:
> ifb: add multi-queue support
>
> Add multi-queue support, and one kernel thread is created for per queue.
> It can used to emulate multi-queue NIC in software, and distribute work
> among CPUs.
> gentux linux # modprobe ifb numtxqs=2
> gentux linux # ifconfig ifb0 up
> gentux linux # pgrep ifb0
> 18508
> 18509
> gentux linux # taskset -p 1 18508
> pid 18508's current affinity mask: 3
> pid 18508's new affinity mask: 1
> gentux linux # taskset -p 2 18509
> pid 18509's current affinity mask: 3
> pid 18509's new affinity mask: 2
> gentux linux # tc qdisc add dev br0 ingress
> gentux linux # tc filter add dev br0 parent ffff: protocol ip basic
> action mirred egress redirect dev ifb0
>
> This patch also introduces a ip link option "numtxqs" for specifying the
> number of the TX queues. so you can add a new ifb with the command:
>
> ip link add numtxqs 4 type ifb
>
The corresponding patch for iproute2 is attached too. And you need to
update the linux header files too.
--
Regards，
Changli Gao(xiaosuo@gmail.com)
[-- Attachment #2: iproute2-support-numtxqs.diff --]
[-- Type: application/octet-stream, Size: 1214 bytes --]
--- ip/iplink.c.orig 2009-11-11 16:56:41.000000000 +0800
+++ ip/iplink.c 2009-11-11 17:32:52.000000000 +0800
@@ -48,6 +48,7 @@
fprintf(stderr, " [ address LLADDR ]\n");
fprintf(stderr, " [ broadcast LLADDR ]\n");
fprintf(stderr, " [ mtu MTU ]\n");
+ fprintf(stderr, " [ numtxqs NUMTXQS ]\n");
fprintf(stderr, " type TYPE [ ARGS ]\n");
fprintf(stderr, " ip link delete DEV type TYPE [ ARGS ]\n");
fprintf(stderr, "\n");
@@ -180,6 +181,7 @@
int qlen = -1;
int mtu = -1;
int netns = -1;
+ int numtxqs = -1;
ret = argc;
@@ -221,6 +223,13 @@
if (get_integer(&mtu, *argv, 0))
invarg("Invalid \"mtu\" value\n", *argv);
addattr_l(&req->n, sizeof(*req), IFLA_MTU, &mtu, 4);
+ } else if (strcmp(*argv, "numtxqs") == 0) {
+ NEXT_ARG();
+ if (numtxqs != -1)
+ duparg("numtxqs", *argv);
+ if (get_integer(&numtxqs, *argv, 0))
+ invarg("Invalid \"numtxqs\" value\n", *argv);
+ addattr_l(&req->n, sizeof(*req), IFLA_NTXQ, &numtxqs, 2);
} else if (strcmp(*argv, "netns") == 0) {
NEXT_ARG();
if (netns != -1)
^ permalink raw reply
* Re: [PATCH] ifb: add multi-queue support
From: Eric Dumazet @ 2009-11-11 10:30 UTC (permalink / raw)
To: xiaosuo
Cc: David S. Miller, Stephen Hemminger, Patrick McHardy, Tom Herbert,
netdev
In-Reply-To: <4AFA8911.7050204@gmail.com>
Changli Gao a écrit :
> ifb: add multi-queue support
>
> Add multi-queue support, and one kernel thread is created for per queue.
> It can used to emulate multi-queue NIC in software, and distribute work
> among CPUs.
> gentux linux # modprobe ifb numtxqs=2
> gentux linux # ifconfig ifb0 up
> gentux linux # pgrep ifb0
> 18508
> 18509
> gentux linux # taskset -p 1 18508
> pid 18508's current affinity mask: 3
> pid 18508's new affinity mask: 1
> gentux linux # taskset -p 2 18509
> pid 18509's current affinity mask: 3
> pid 18509's new affinity mask: 2
> gentux linux # tc qdisc add dev br0 ingress
> gentux linux # tc filter add dev br0 parent ffff: protocol ip basic
> action mirred egress redirect dev ifb0
>
> This patch also introduces a ip link option "numtxqs" for specifying the
> number of the TX queues. so you can add a new ifb with the command:
>
> ip link add numtxqs 4 type ifb
>
> Signed-off-by: Changli Gao <xiaosuo@gmail.com>
> ----
> drivers/net/ifb.c | 414 ++++++++++++++++++++++++++++++++----------------
> include/linux/if_link.h | 1
> net/core/rtnetlink.c | 3
> 3 files changed, 283 insertions(+), 135 deletions(-)
>
> diff --git a/drivers/net/ifb.c b/drivers/net/ifb.c
> index 69c2566..ac04e85 100644
> --- a/drivers/net/ifb.c
> +++ b/drivers/net/ifb.c
> @@ -33,161 +33,157 @@
> #include <linux/etherdevice.h>
> #include <linux/init.h>
> #include <linux/moduleparam.h>
> +#include <linux/wait.h>
> +#include <linux/sched.h>
> +#include <linux/kthread.h>
> +#include <linux/ip.h>
> +#include <linux/ipv6.h>
> +#include <net/ip.h>
> #include <net/pkt_sched.h>
> #include <net/net_namespace.h>
>
> -#define TX_TIMEOUT (2*HZ)
> -
> #define TX_Q_LIMIT 32
> +
> +struct ifb_private_q {
> + struct net_device *dev;
> + struct sk_buff_head rq;
> + struct sk_buff_head tq;
> + wait_queue_head_t wq;
> + struct task_struct *task;
> + unsigned long rx_packets;
> + unsigned long rx_bytes;
> + unsigned long rx_dropped;
> +} ____cacheline_aligned_in_smp;
> +
Could you split this struct in two parts, one used by ifb_xmit(),
one used by the ifb_thread() ?
This would reduce number of cache line ping pongs...
struct ifb_private_q {
struct net_device *dev;
struct task_struct *task;
/* used by ifb_xmit() */
struct sk_buff_head rq;
unsigned long rx_packets;
unsigned long rx_bytes;
unsigned long rx_dropped;
wait_queue_head_t wq;
/* used by ifb_thread() */
struct sk_buff_head tq ____cacheline_aligned_in_smp;
} ____cacheline_aligned_in_smp;
Or even better, tq could be local to ifb_xmit()
ifb_xmit() would purge it itself, and could use __skb_dequeue() and
other lockless primitives.
The whole :
while ((skb = skb_dequeue(&pq->rq)) != NULL)
skb_queue_tail(&pq->tq, skb);
could be optimized a lot IMHO, its almost a skb_queue_splice() thing...
^ permalink raw reply
* Re: [PATCH] ifb: add multi-queue support
From: Changli Gao @ 2009-11-11 10:57 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S. Miller, Stephen Hemminger, Patrick McHardy, Tom Herbert,
netdev
In-Reply-To: <4AFA9235.1030004@gmail.com>
2009/11/11 Eric Dumazet <eric.dumazet@gmail.com>:
>
> Or even better, tq could be local to ifb_xmit()
>
> ifb_xmit() would purge it itself, and could use __skb_dequeue() and
> other lockless primitives.
>
> The whole :
>
> while ((skb = skb_dequeue(&pq->rq)) != NULL)
> skb_queue_tail(&pq->tq, skb);
>
> could be optimized a lot IMHO, its almost a skb_queue_splice() thing...
>
Thanks, it is a good idea. Any others?
--
Regards，
Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox