* Re: [PATCH net-next-2.6] udp: bind() optimisation
From: David Miller @ 2009-11-11 6:32 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev, lgrijincu, opurdila
In-Reply-To: <4AF834A9.2050005@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Mon, 09 Nov 2009 16:26:33 +0100
> UDP bind() can be O(N^2) in some pathological cases.
>
> Thanks to secondary hash tables, we can make it O(N)
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied to net-next-2.6
^ permalink raw reply
* Re: [net-next 1/4] bnx2x: GSO implies CSUM offload
From: David Miller @ 2009-11-11 6:32 UTC (permalink / raw)
To: eilong; +Cc: netdev
In-Reply-To: <1257782962.4101.40.camel@lb-tlvb-eilong>
From: "Eilon Greenstein" <eilong@broadcom.com>
Date: Mon, 9 Nov 2009 18:09:22 +0200
> Making sure that whenever the FW/HW is configured for GSO, it is also configured
> to CSUM offload
>
> Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Applied.
^ permalink raw reply
* Re: [net-next 2/4] bnx2x: Remove misleading error print
From: David Miller @ 2009-11-11 6:32 UTC (permalink / raw)
To: eilong; +Cc: netdev
In-Reply-To: <1257782968.4101.41.camel@lb-tlvb-eilong>
From: "Eilon Greenstein" <eilong@broadcom.com>
Date: Mon, 9 Nov 2009 18:09:28 +0200
> Failing to allocate MSI-X vectors is not an error and should not be printed as
> such
>
> Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Applied.
^ permalink raw reply
* Re: [net-next 3/4] bnx2x: Change coalescing granularity to 4us
From: David Miller @ 2009-11-11 6:32 UTC (permalink / raw)
To: eilong; +Cc: netdev
In-Reply-To: <1257782975.4101.42.camel@lb-tlvb-eilong>
From: "Eilon Greenstein" <eilong@broadcom.com>
Date: Mon, 9 Nov 2009 18:09:35 +0200
> Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Applied.
^ permalink raw reply
* Re: [net-next 4/4] bnx2x: version 1.52.1-4
From: David Miller @ 2009-11-11 6:32 UTC (permalink / raw)
To: eilong; +Cc: netdev
In-Reply-To: <1257782977.4101.43.camel@lb-tlvb-eilong>
From: "Eilon Greenstein" <eilong@broadcom.com>
Date: Mon, 9 Nov 2009 18:09:37 +0200
> Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Applied.
^ permalink raw reply
* Re: [PATCH] ipv6: Allow inet6_dump_addr() to handle more than 64 addresses
From: David Miller @ 2009-11-11 6:33 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev
In-Reply-To: <4AF85401.3020906@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Mon, 09 Nov 2009 18:40:17 +0100
> Apparently, inet6_dump_addr() is not able to handle more than
> 64 ipv6 addresses per device. We must break from inner loops
> in case skb is full, or else cursor is put at the end of list.
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied to net-next-2.6
^ permalink raw reply
* Re: sit: Clean up DF code by copying from IPIP
From: David Miller @ 2009-11-11 6:33 UTC (permalink / raw)
To: herbert; +Cc: netdev
In-Reply-To: <20091109184201.GA9493@gondor.apana.org.au>
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Mon, 9 Nov 2009 13:42:01 -0500
> sit: Clean up DF code by copying from IPIP
>
> This patch rearranges the SIT DF bit handling using the new IPIP
> DF code. The only externally visible effect should be the case where
> PMTU is enabled and the MTU is exactly 1280 bytes. In this case
> the previous code would send packets out with DF off while the new
> code would set the DF bit. This is inline with RFC 4213.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Applied to net-next-2.6
^ permalink raw reply
* Re: [PATCH] IPv6: use ipv6_addr_v4mapped()
From: David Miller @ 2009-11-11 6:34 UTC (permalink / raw)
To: brian.haley; +Cc: netdev
In-Reply-To: <4AF89241.7060203@hp.com>
From: Brian Haley <brian.haley@hp.com>
Date: Mon, 09 Nov 2009 17:05:53 -0500
> Change udp6_portaddr_hash() to use ipv6_addr_v4mapped()
> inline instead of ipv6_addr_type().
>
> Signed-off-by: Brian Haley <brian.haley@hp.com>
Applied to net-next-2.6
^ permalink raw reply
* Re: [PATCH net-next-2.6] ipv6: speedup inet6_dump_ifinfo()
From: David Miller @ 2009-11-11 6:34 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev
In-Reply-To: <4AF89390.4000902@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Mon, 09 Nov 2009 23:11:28 +0100
> When handling large number of netdevice, inet6_dump_ifinfo()
> is very slow because it has O(N^2) complexity.
>
> Instead of scanning one single list, we can use the 256 sub lists
> of the dev_index hash table, and RCU lookups.
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH net-next-2.6] parisc: led: Use for_each_netdev_rcu()
From: David Miller @ 2009-11-11 6:35 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev
In-Reply-To: <4AF8E700.9080307@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 05:07:28 +0100
> Use for_each_netdev_rcu() and dont lock dev_base_lock anymore
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: xfrm command fails
From: Naveen BN @ 2009-11-11 6:16 UTC (permalink / raw)
To: netdev; +Cc: David Miller
In-Reply-To: <4AFA5499.2090605@globaledgesoft.com>
adding David in loop
Naveen BN wrote:
> I am using fedora with version => 2.6.23.1-42.fc8.
> Please guide me to solve the error message "RTNETLINK answers:
> Function not implemented"
>
> Regards
> Naveen
> Naveen BN wrote:
>> Hi All
>> I am trying to create a security association using ip xfrm command,
>> but i get an message saying
>> "*RTNETLINK answers: Function not implemented*" and ip xfrm state
>> succeeds in displaying sa written using setkey .
>>
>> ip xfrm state add src 192.168.1.20 dst 192.168.1.5 proto esp spi
>> 0x354cb04b mode tunnel reqid 16385 replay-window 32 auth hmac
>> 0xecf02a5cf6568556e1bdcd961c7ec3f92afd01cc enc cbc
>> 0x5c0cfa9672ce67ba545b593076dfb278 sel src 0.0.0.0/0 dst 0.0.0.0/0
>>
>> ERROR message => "RTNETLINK answers: Function not implemented"
>>
>> Regards
>> Naveen
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
* Re: [PATCH 01/10] netdev: add netdev_continue_rcu
From: David Miller @ 2009-11-11 6:47 UTC (permalink / raw)
To: eric.dumazet; +Cc: shemminger, paulmck, netdev
In-Reply-To: <4AF9AEAA.1080608@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 19:19:22 +0100
> Stephen Hemminger a écrit :
>> This adds an RCU macro for continuing search, useful for some
>> network devices like vlan.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 02/10] vlan: eliminate use of dev_base_lock
From: David Miller @ 2009-11-11 6:47 UTC (permalink / raw)
To: dada1; +Cc: shemminger, kaber, netdev
In-Reply-To: <4AF9AEDF.2070307@cosmosbay.com>
From: Eric Dumazet <dada1@cosmosbay.com>
Date: Tue, 10 Nov 2009 19:20:15 +0100
> Stephen Hemminger a écrit :
>> Do not need to use read_lock(&dev_base_lock), use RCU instead.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 03/10] net: use rcu for network scheduler API
From: David Miller @ 2009-11-11 6:47 UTC (permalink / raw)
To: eric.dumazet; +Cc: shemminger, netdev
In-Reply-To: <4AF9AF09.60102@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 19:20:57 +0100
> Stephen Hemminger a écrit :
>> Use RCU to walk list of network devices in qdisc dump.
>> This could be optimized for large number of devices.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 04/10] AOE: use rcu to find network device
From: David Miller @ 2009-11-11 6:48 UTC (permalink / raw)
To: shemminger
Cc: karaluh, ecashin, roel.kluin, harvey.harrison, bzolnier, netdev
In-Reply-To: <20091110155316.2c3d7b6e@nehalam>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 15:53:16 -0800
> On Tue, 10 Nov 2009 15:06:17 -0800
> Stephen Hemminger <shemminger@vyatta.com> wrote:
>
>> On Tue, 10 Nov 2009 15:01:49 -0500
>> Ed Cashin <ecashin@coraid.com> wrote:
>>
>> > On Tue Nov 10 13:07:37 EST 2009, shemminger@vyatta.com wrote:
>> > > This gets rid of another use of read_lock(&dev_base_lock) by using
>> > > RCU. Also, it only increments the reference count of the device actually
>> > > used rather than holding and releasing every device
>> > >
>> > > Compile tested only.
>> >
>> > This function runs once a minute when the aoe driver is loaded,
>> > if you'd like to test it a bit more.
>> >
>> > It looks like there's no dev_put corresponding to the dev_hold
>> > after the changes.
>> >
>>
>> Hmm, looks like AOE actually is not ref counting the network device.
>> So my patch is incorrect.
>>
>> As it stands (before my patch), it is UNSAFE. It can decide to queue
>> packets to a device that is removed out from underneath it causing
>> reference to freed memory.
>>
>> Moving the rcu_read_lock up to aoecmd_cfg() would solve that but the
>> whole driver appears to be unsafe about device refcounting and handling
>> device removal properly.
>>
>> It needs to:
>>
>> 1. Get a device ref count when it remembers a device: (ie addif)
>> 2. Install a notifier that looks for device removal events
>> 3. In notifier, remove interface, including flushing all pending
>> skb's for that device.
>>
>> This obviously is beyond the scope of the RCU stuff.
>
> Here is a patch to get you going, it does compile but it probably
> won't work because the code doesn't handle the case of the last
> device going away from a target. This is yet another pre-existing
> bug, since if a timeout happens: ejectif() is called to remove a device,
> resend() will BUG in ifrotate() if all devices are gone.
I'm holding off on the RCU patch until these known refcount bugs are
fixed
^ permalink raw reply
* Re: [PATCH 06/10] s390: use RCU to walk list of network devices
From: David Miller @ 2009-11-11 6:49 UTC (permalink / raw)
To: eric.dumazet; +Cc: shemminger, schwidefsky, heiko.carstens, netdev, linux390
In-Reply-To: <4AF9B3AA.6070104@gmail.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 10 Nov 2009 19:40:42 +0100
> Stephen Hemminger a écrit :
>> This is similar to other cases where for_each_netdev_rcu
>> can be used when gathering information.
>>
>> By inspection, don't have platform or cross-build environment
>> to validate.
>>
>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
>>
>>
>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
Applied.
^ permalink raw reply
* Re: [PATCH 05/10] parisc: use RCU to find network device
From: David Miller @ 2009-11-11 6:48 UTC (permalink / raw)
To: shemminger; +Cc: kyle, deller, a.beregalov, netdev, linux-parisc
In-Reply-To: <20091110175647.480041042@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:51 -0800
> Another place where RCU can be used instead of read_lock(&dev_base_lock)
> This is by inspection, don't have platform or cross-build environment
> to validate.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Applied.
^ permalink raw reply
* Re: [PATCH 07/10] decnet: use RCU to find network devices
From: David Miller @ 2009-11-11 6:49 UTC (permalink / raw)
To: shemminger
Cc: christine.caulfield, hannes, adobriyan, swhiteho, netdev,
linux-decnet-users
In-Reply-To: <20091110175647.615305929@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:53 -0800
> When showing device statistics use RCU rather than read_lock(&dev_base_lock)
> Compile tested only.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Applied.
^ permalink raw reply
* Re: [PATCH 08/10] ipv6: use RCU to walk list of network devices
From: David Miller @ 2009-11-11 6:50 UTC (permalink / raw)
To: shemminger; +Cc: netdev
In-Reply-To: <20091110175647.683253741@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:54 -0800
> No longer need read_lock(&dev_base_lock), use RCU instead.
> This also needs to be optimized for large number of devices.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
As Eric mentioned, superceded by his patch.
^ permalink raw reply
* Re: [PATCH 09/10] IPV4: use rcu to walk list of devices in IGMP
From: David Miller @ 2009-11-11 6:50 UTC (permalink / raw)
To: shemminger; +Cc: netdev
In-Reply-To: <20091110175647.754389168@vyatta.com>
From: Stephen Hemminger <shemminger@vyatta.com>
Date: Tue, 10 Nov 2009 09:54:55 -0800
> This also needs to be optimized for large number of devices.
>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Applied.
^ permalink raw reply
* Re: [PATCH 10/10] CAN: use dev_get_by_index_rcu
From: David Miller @ 2009-11-11 6:50 UTC (permalink / raw)
To: oliver; +Cc: eric.dumazet, shemminger, netdev
In-Reply-To: <4AFA5190.60600@hartkopp.net>
From: Oliver Hartkopp <oliver@hartkopp.net>
Date: Wed, 11 Nov 2009 06:54:24 +0100
> Eric Dumazet wrote:
>> Stephen Hemminger a écrit :
>>> Use new function to avoid doing read_lock().
>>>
>>> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
...
>> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
>>
>
> Thanks everyone!
>
> Signed-off-by: Oliver Hartkopp <oliver@hartkopp.net>
Applied.
^ permalink raw reply
* [PATCH 0/2] rps: Receive packet steering
From: Tom Herbert @ 2009-11-11 6:53 UTC (permalink / raw)
To: David Miller, netdev
This is the third version of the the patch that implements software receive
side packet steering (RPS). RPS distributes the load of received packet
processing across multiple CPUs. This version allows per NAPI steering maps
as well as using HW provided hash as an optimization.
These patches are also the basis for per-flow steering which was previously
discussed; we are still working on a general solution for the per flow
steering which prevents OOO packets.
Problem statement: Protocol processing done in the NAPI context for received
packets is serialized per device queue and becomes a bottleneck under high
packet load. This substantially limits pps that can be achieved on a single
queue NIC and provides no scaling with multiple cores.
This solution queues packets early on in the receive path on the backlog queues
of other CPUs. This allows protocol processing (e.g. IP and TCP) to be
performed on packets in parallel. For each device (or NAPI instance for
a multi-queue device) a mask of CPUs is set to indicate the CPUs that can
process packets for the device. A CPU is selected on a per packet basis by
hashing contents of the packet header (the TCP or UDP 4-tuple) and using the
result to index into the CPU mask. The IPI mechanism is used to raise
networking receive softirqs between CPUs. This effectively emulates in
software what a multi-queue NIC can provide, but is generic requiring no device
support.
Many devices now provide a hash over the 4-tuple on a per packet basis
(Toeplitz is popular). This patch allow drivers to set the HW reported hash
in an skb field, and that value in turn is used to index into the RPS maps.
Using the HW generated hash can avoid cache misses on the packet when
steering the packet to a remote CPU.
The CPU masks is set on a per device basis in the sysfs variable
/sys/class/net/<device>/rps_cpus. This is a set of canonical bit maps for
each NAPI nstance of the device. For example:
echo "0b 0b0 0b00 0b000" > /sys/class/net/eth0/rps_cpus
would set maps for four NAPI instances on eth0.
The first patch in this set adds the RPS functionality to the core networking.
The second patch adds support to the bnx2x driver to record the Toeplitz hash
reported by the device for received skbs.
Generally, we have found this technique increases pps capabilities of a single
queue device with good CPU utilization. Optimal settings for the CPU mask
seems to depend on architectures and cache hierarcy. Below are some results
running 700 instances of netperf TCP_RR test with 1 byte req. and resp.
Results show cumulative transaction rate and system CPU utilization.
tg3 on 8 core Intel
Without RPS: 90K tps at 34% CPU
With RPS: 285K tps at 70% CPU
e1000 on 8 core Intel
Without RPS: 90K tps at 34% CPU
With RPS: 292K tps at 66% CPU
foredeth on 16 core AMD
Without RPS: 117K tps at 10% CPU
With RPS: 327K tps at 29% CPU
bnx2x on 16 core AMD
Single queue without RPS: 139K tps at 17% CPU
Single queue with RPS: 352K tps at 30% CPU
Multi queue (1 queues per CPU) 204K tps at 12%
We have been running a variant of this patch on production servers for a while
with good results. In some of our more networking intensive applications we
have seen 30-50% gains in end application performance.
Tom
Signed-off-by: Tom Herbert <therbert@google.com>
^ permalink raw reply
* [PATCH 1/2] rps: core implementation
From: Tom Herbert @ 2009-11-11 6:53 UTC (permalink / raw)
To: David Miller, netdev
Third version of RPS.
Signed-off-by: Tom Herbert <therbert@google.com>
---
include/linux/interrupt.h | 1 +
include/linux/netdevice.h | 18 ++++
include/linux/skbuff.h | 2 +
net/core/dev.c | 227 ++++++++++++++++++++++++++++++++++++++-------
net/core/net-sysfs.c | 135 +++++++++++++++++++++++++++
5 files changed, 348 insertions(+), 35 deletions(-)
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index b78cf81..fa91194 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -345,6 +345,7 @@ enum
TIMER_SOFTIRQ,
NET_TX_SOFTIRQ,
NET_RX_SOFTIRQ,
+ NET_RPS_SOFTIRQ,
BLOCK_SOFTIRQ,
BLOCK_IOPOLL_SOFTIRQ,
TASKLET_SOFTIRQ,
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 8380009..c1b1bbb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -639,6 +639,18 @@ struct net_device_ops {
};
/*
+ * Structure for Receive Packet Steering. Length of map and array of CPU ID's.
+ */
+struct rps_map {
+ int len;
+ u16 map[0];
+};
+
+/* Maximum size of RPS map (for allocation) */
+#define RPS_MAP_SIZE (sizeof(struct rps_map) + \
+ (num_possible_cpus() * sizeof(u16)))
+
+/*
* The DEVICE structure.
* Actually, this whole structure is a big mistake. It mixes I/O
* data with strictly "high-level" data, and it has to know about
@@ -807,6 +819,9 @@ struct net_device
void *ax25_ptr; /* AX.25 specific data */
struct wireless_dev *ieee80211_ptr; /* IEEE 802.11 specific data,
assign before registering */
+ void *rps_maps; /* Array of per-NAPI maps for
+ receive packet steeing */
+ int rps_num_maps; /* Number of RPS maps */
/*
* Cache line mostly used on receive path (including eth_type_trans())
@@ -1217,6 +1232,9 @@ struct softnet_data
struct Qdisc *output_queue;
struct sk_buff_head input_pkt_queue;
struct list_head poll_list;
+
+ struct call_single_data csd;
+
struct sk_buff *completion_queue;
struct napi_struct backlog;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 0c68fbd..95feac7 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -396,6 +396,8 @@ struct sk_buff {
__u16 vlan_tci;
+ __u32 rxhash;
+
sk_buff_data_t transport_header;
sk_buff_data_t network_header;
sk_buff_data_t mac_header;
diff --git a/net/core/dev.c b/net/core/dev.c
index 28b0b9e..735e7e3 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -1976,6 +1976,162 @@ int weight_p __read_mostly = 64; /*
old backlog weight */
DEFINE_PER_CPU(struct netif_rx_stats, netdev_rx_stat) = { 0, };
+static u32 simple_hashrnd;
+
+/**
+ * get_rps_cpu is called from netif_receive_skb and returns the target
+ * CPU from the RPS map of the receiving NAPI instance for a given skb.
+ */
+static int get_rps_cpu(struct net_device *dev, struct sk_buff *skb)
+{
+ u32 addr1, addr2, ports;
+ struct ipv6hdr *ip6;
+ struct iphdr *ip;
+ u32 hash, ihl;
+ u8 ip_proto;
+ int cpu;
+ struct rps_map *map = NULL;
+
+ if (dev->rps_num_maps) {
+ /*
+ * Locate the map corresponding to the NAPI queue that
+ * the packet was received on.
+ */
+ int index = skb_get_rx_queue(skb);
+ if (index < 0 || index >= dev->rps_num_maps)
+ index = 0;
+
+ map = (struct rps_map *)
+ (dev->rps_maps + (RPS_MAP_SIZE * index));
+ if (!map->len)
+ map = NULL;
+ }
+
+ if (!map)
+ return -1;
+
+ hash = skb->rxhash;
+ if (hash)
+ goto got_hash; /* Skip hash computation on packet header */
+
+ switch (skb->protocol) {
+ case __constant_htons(ETH_P_IP):
+ if (!pskb_may_pull(skb, sizeof(*ip)))
+ return -1;
+
+ ip = (struct iphdr *) skb->data;
+ ip_proto = ip->protocol;
+ addr1 = ip->saddr;
+ addr2 = ip->daddr;
+ ihl = ip->ihl;
+ break;
+ case __constant_htons(ETH_P_IPV6):
+ if (!pskb_may_pull(skb, sizeof(*ip6)))
+ return -1;
+
+ ip6 = (struct ipv6hdr *) skb->data;
+ ip_proto = ip6->nexthdr;
+ addr1 = ip6->saddr.s6_addr32[3];
+ addr2 = ip6->daddr.s6_addr32[3];
+ ihl = (40 >> 2);
+ break;
+ default:
+ return -1;
+ }
+ ports = 0;
+ switch (ip_proto) {
+ case IPPROTO_TCP:
+ case IPPROTO_UDP:
+ case IPPROTO_DCCP:
+ case IPPROTO_ESP:
+ case IPPROTO_AH:
+ case IPPROTO_SCTP:
+ case IPPROTO_UDPLITE:
+ if (pskb_may_pull(skb, (ihl * 4) + 4))
+ ports = *((u32 *) (skb->data + (ihl * 4)));
+ break;
+
+ default:
+ break;
+ }
+
+ hash = jhash_3words(addr1, addr2, ports, simple_hashrnd);
+
+got_hash:
+ cpu = map->map[((u64) hash * map->len) >> 32];
+
+ return cpu_online(cpu) ? cpu : -1;
+}
+
+static DEFINE_PER_CPU(cpumask_t, rps_remote_softirq_cpus);
+
+/* Called from hardirq (IPI) context */
+static void trigger_softirq(void *data)
+{
+ struct softnet_data *queue = data;
+ __napi_schedule(&queue->backlog);
+}
+
+/**
+ * net_rps_action is called from NET_RPS_SOFTIRQ to do IPIs to schedule RX
+ * softirq on remote CPUs. Called in a separate softirq to allow for
+ * coalescing.
+ */
+static void net_rps_action(struct softirq_action *h)
+{
+ int cpu;
+
+ local_irq_disable();
+
+ for_each_cpu_mask_nr(cpu, __get_cpu_var(rps_remote_softirq_cpus)) {
+ struct softnet_data *queue = &per_cpu(softnet_data, cpu);
+ __smp_call_function_single(cpu, &queue->csd, 0);
+ }
+ cpus_clear(__get_cpu_var(rps_remote_softirq_cpus));
+
+ local_irq_enable();
+}
+
+/**
+ * enqueue_to_backlog is called to queue an skb to a per CPU backlog
+ * queue (may be a remote CPU queue).
+ */
+static int enqueue_to_backlog(struct sk_buff *skb, int cpu)
+{
+ struct softnet_data *queue;
+ unsigned long flags;
+
+ queue = &per_cpu(softnet_data, cpu);
+ spin_lock_irqsave(&queue->input_pkt_queue.lock, flags);
+
+ __get_cpu_var(netdev_rx_stat).total++;
+ if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
+ if (queue->input_pkt_queue.qlen) {
+enqueue:
+ __skb_queue_tail(&queue->input_pkt_queue, skb);
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock,
+ flags);
+ return NET_RX_SUCCESS;
+ }
+
+ /* Schedule NAPI for backlog device */
+ if (napi_schedule_prep(&queue->backlog)) {
+ if (cpu != smp_processor_id()) {
+ cpu_set(cpu,
+ get_cpu_var(rps_remote_softirq_cpus));
+ __raise_softirq_irqoff(NET_RPS_SOFTIRQ);
+ } else
+ __napi_schedule(&queue->backlog);
+ }
+ goto enqueue;
+ }
+
+ __get_cpu_var(netdev_rx_stat).dropped++;
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock, flags);
+
+ kfree_skb(skb);
+ return NET_RX_DROP;
+}
/**
* netif_rx - post buffer to the network code
@@ -1994,8 +2150,7 @@ DEFINE_PER_CPU(struct netif_rx_stats,
netdev_rx_stat) = { 0, };
int netif_rx(struct sk_buff *skb)
{
- struct softnet_data *queue;
- unsigned long flags;
+ int cpu;
/* if netpoll wants it, pretend we never saw it */
if (netpoll_rx(skb))
@@ -2004,31 +2159,11 @@ int netif_rx(struct sk_buff *skb)
if (!skb->tstamp.tv64)
net_timestamp(skb);
- /*
- * The code is rearranged so that the path is the most
- * short when CPU is congested, but is still operating.
- */
- local_irq_save(flags);
- queue = &__get_cpu_var(softnet_data);
+ cpu = get_rps_cpu(skb->dev, skb);
+ if (cpu < 0)
+ cpu = smp_processor_id();
- __get_cpu_var(netdev_rx_stat).total++;
- if (queue->input_pkt_queue.qlen <= netdev_max_backlog) {
- if (queue->input_pkt_queue.qlen) {
-enqueue:
- __skb_queue_tail(&queue->input_pkt_queue, skb);
- local_irq_restore(flags);
- return NET_RX_SUCCESS;
- }
-
- napi_schedule(&queue->backlog);
- goto enqueue;
- }
-
- __get_cpu_var(netdev_rx_stat).dropped++;
- local_irq_restore(flags);
-
- kfree_skb(skb);
- return NET_RX_DROP;
+ return enqueue_to_backlog(skb, cpu);
}
EXPORT_SYMBOL(netif_rx);
@@ -2266,10 +2401,10 @@ void netif_nit_deliver(struct sk_buff *skb)
}
/**
- * netif_receive_skb - process receive buffer from network
+ * __netif_receive_skb - process receive buffer from network
* @skb: buffer to process
*
- * netif_receive_skb() is the main receive data processing function.
+ * __netif__napireceive_skb() is the main receive data processing function.
* It always succeeds. The buffer may be dropped during processing
* for congestion control or by the protocol layers.
*
@@ -2280,7 +2415,8 @@ void netif_nit_deliver(struct sk_buff *skb)
* NET_RX_SUCCESS: no congestion
* NET_RX_DROP: packet was dropped
*/
-int netif_receive_skb(struct sk_buff *skb)
+
+int __netif_receive_skb(struct sk_buff *skb)
{
struct packet_type *ptype, *pt_prev;
struct net_device *orig_dev;
@@ -2378,6 +2514,16 @@ out:
}
EXPORT_SYMBOL(netif_receive_skb);
+int netif_receive_skb(struct sk_buff *skb)
+{
+ int cpu = get_rps_cpu(skb->dev, skb);
+
+ if (cpu < 0)
+ return __netif_receive_skb(skb);
+ else
+ return enqueue_to_backlog(skb, cpu);
+}
+
/* Network device is going away, flush any packets still pending */
static void flush_backlog(void *arg)
{
@@ -2421,7 +2567,7 @@ static int napi_gro_complete(struct sk_buff *skb)
}
out:
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
}
void napi_gro_flush(struct napi_struct *napi)
@@ -2554,7 +2700,7 @@ int napi_skb_finish(int ret, struct sk_buff *skb)
switch (ret) {
case GRO_NORMAL:
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
case GRO_DROP:
err = NET_RX_DROP;
@@ -2625,7 +2771,7 @@ int napi_frags_finish(struct napi_struct *napi,
struct sk_buff *skb, int ret)
skb->protocol = eth_type_trans(skb, napi->dev);
if (ret == GRO_NORMAL)
- return netif_receive_skb(skb);
+ return __netif_receive_skb(skb);
skb_gro_pull(skb, -ETH_HLEN);
break;
@@ -2696,21 +2842,24 @@ static int process_backlog(struct napi_struct
*napi, int quota)
int work = 0;
struct softnet_data *queue = &__get_cpu_var(softnet_data);
unsigned long start_time = jiffies;
+ unsigned long flags;
napi->weight = weight_p;
do {
struct sk_buff *skb;
local_irq_disable();
+ spin_lock_irqsave(&queue->input_pkt_queue.lock, flags);
skb = __skb_dequeue(&queue->input_pkt_queue);
if (!skb) {
__napi_complete(napi);
- local_irq_enable();
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock,
+ flags);
break;
}
- local_irq_enable();
+ spin_unlock_irqrestore(&queue->input_pkt_queue.lock, flags);
- netif_receive_skb(skb);
+ __netif_receive_skb(skb);
} while (++work < quota && jiffies == start_time);
return work;
@@ -5205,6 +5354,8 @@ void free_netdev(struct net_device *dev)
/* Flush device addresses */
dev_addr_flush(dev);
+ kfree(dev->rps_maps);
+
list_for_each_entry_safe(p, n, &dev->napi_list, dev_list)
netif_napi_del(p);
@@ -5644,6 +5795,10 @@ static int __init net_dev_init(void)
queue->completion_queue = NULL;
INIT_LIST_HEAD(&queue->poll_list);
+ queue->csd.func = trigger_softirq;
+ queue->csd.info = queue;
+ queue->csd.flags = 0;
+
queue->backlog.poll = process_backlog;
queue->backlog.weight = weight_p;
queue->backlog.gro_list = NULL;
@@ -5669,7 +5824,9 @@ static int __init net_dev_init(void)
open_softirq(NET_TX_SOFTIRQ, net_tx_action);
open_softirq(NET_RX_SOFTIRQ, net_rx_action);
+ open_softirq(NET_RPS_SOFTIRQ, net_rps_action);
+ get_random_bytes(&simple_hashrnd, 4);
hotcpu_notifier(dev_cpu_callback, 0);
dst_init();
dev_mcast_init();
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index 753c420..ca250f6 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -18,6 +18,9 @@
#include <linux/wireless.h>
#include <net/wext.h>
+#include <linux/string.h>
+#include <linux/ctype.h>
+
#include "net-sysfs.h"
#ifdef CONFIG_SYSFS
@@ -249,6 +252,137 @@ static ssize_t store_tx_queue_len(struct device *dev,
return netdev_store(dev, attr, buf, len, change_tx_queue_len);
}
+static char *
+get_token(const char **cp, size_t *len)
+{
+ const char *bp = *cp, *start;
+
+ while (isspace(*bp))
+ bp++;
+
+ start = bp;
+ while (!isspace(*bp) && *bp != '\0')
+ bp++;
+
+ if (start != bp)
+ *len = bp - start;
+ else
+ start = NULL;
+
+ *cp = bp;
+ return start;
+}
+
+static ssize_t store_rps_cpus(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t len)
+{
+ struct net_device *net = to_net_dev(dev);
+ struct napi_struct *napi;
+ cpumask_t mask;
+ int err, cpu, index, i;
+ int cnt = 0;
+ char *token;
+ const char *cp = buf;
+ size_t tlen;
+
+ if (!capable(CAP_NET_ADMIN))
+ return -EPERM;
+
+ /*
+ * Pre-check that tokens parse properly before we commit to making
+ * any changes.
+ */
+ while ((token = get_token(&cp, &tlen)))
+ err = bitmap_parse(token, tlen, cpumask_bits(&mask),
+ nr_cpumask_bits);
+
+ if (err)
+ return err;
+
+ rtnl_lock();
+ if (dev_isalive(net)) {
+ if (!net->rps_maps) {
+ /*
+ * Need to allocate the array of RPS maps, one map
+ * for each NAPI instance on the device.
+ */
+ list_for_each_entry(napi, &net->napi_list, dev_list)
+ cnt++;
+ net->rps_maps = kzalloc(RPS_MAP_SIZE * cnt, GFP_KERNEL);
+ if (!net->rps_maps) {
+ rtnl_unlock();
+ return -ENOMEM;
+ }
+ net->rps_num_maps = cnt;
+ }
+
+ cp = buf;
+ for (index = 0; index < net->rps_num_maps &&
+ (token = get_token(&cp, &tlen)); index++) {
+ struct rps_map *map = (struct rps_map *)
+ (net->rps_maps + (RPS_MAP_SIZE * index));
+ err = bitmap_parse(token, tlen, cpumask_bits(&mask),
+ nr_cpumask_bits);
+ if (!err) {
+ cpus_and(mask, mask, cpu_online_map);
+ i = 0;
+ for_each_cpu_mask(cpu, mask)
+ map->map[i++] = cpu;
+ map->len = i;
+ } else {
+ rtnl_unlock();
+ return err;
+ }
+ }
+
+ /*
+ * Any per NAPI maps not being set are "zeroed" by setting
+ * map length length to zero.
+ */
+ for (; index < net->rps_num_maps; index++) {
+ struct rps_map *map = (struct rps_map *)
+ (net->rps_maps + (RPS_MAP_SIZE * index));
+ map->len = 0;
+ }
+ }
+ rtnl_unlock();
+
+ return len;
+}
+
+static ssize_t show_rps_cpus(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct net_device *net = to_net_dev(dev);
+ size_t len = 0;
+ cpumask_t mask;
+ int i, j;
+
+ read_lock(&dev_base_lock);
+ if (dev_isalive(net)) {
+ for (j = 0; j < net->rps_num_maps; j++) {
+ struct rps_map *map = (struct rps_map *)
+ (net->rps_maps + (RPS_MAP_SIZE * j));
+ cpus_clear(mask);
+ for (i = 0; i < map->len; i++)
+ cpu_set(map->map[i], mask);
+
+ len += cpumask_scnprintf(buf + len, PAGE_SIZE, &mask);
+ if (PAGE_SIZE - len < 3) {
+ read_unlock(&dev_base_lock);
+ return -EINVAL;
+ }
+ if (j < net->rps_num_maps)
+ len += sprintf(buf + len, " ");
+ }
+ }
+
+ read_unlock(&dev_base_lock);
+
+ len += sprintf(buf + len, "\n");
+ return len;
+}
+
static ssize_t store_ifalias(struct device *dev, struct device_attribute *attr,
const char *buf, size_t len)
{
@@ -305,6 +439,7 @@ static struct device_attribute net_class_attributes[] = {
__ATTR(flags, S_IRUGO | S_IWUSR, show_flags, store_flags),
__ATTR(tx_queue_len, S_IRUGO | S_IWUSR, show_tx_queue_len,
store_tx_queue_len),
+ __ATTR(rps_cpus, S_IRUGO | S_IWUSR, show_rps_cpus, store_rps_cpus),
{}
};
--
1.5.4.3
^ permalink raw reply related
* [PATCH 2/2] rps: changes to bnx2x to get device hash
From: Tom Herbert @ 2009-11-11 6:53 UTC (permalink / raw)
To: David Miller, netdev
bnx2x changes to get Toeplitz hash on RX and out into skb.
Signed-off-by: Tom Herbert <therbert@google.com>
---
drivers/net/bnx2x.h | 2 +-
drivers/net/bnx2x_main.c | 47 ++++++++++++++++++++++++++++++++++++++++-----
2 files changed, 42 insertions(+), 7 deletions(-)
diff --git a/drivers/net/bnx2x.h b/drivers/net/bnx2x.h
index c3b32f7..a876d78 100644
--- a/drivers/net/bnx2x.h
+++ b/drivers/net/bnx2x.h
@@ -1295,7 +1295,7 @@ static inline u32 reg_poll(struct bnx2x *bp, u32
reg, u32 expected, int ms,
AEU_INPUTS_ATTN_BITS_MISC_PARITY_ERROR)
-#define MULTI_FLAGS(bp) \
+#define RSS_FLAGS(bp) \
(TSTORM_ETH_FUNCTION_COMMON_CONFIG_RSS_IPV4_CAPABILITY | \
TSTORM_ETH_FUNCTION_COMMON_CONFIG_RSS_IPV4_TCP_CAPABILITY | \
TSTORM_ETH_FUNCTION_COMMON_CONFIG_RSS_IPV6_CAPABILITY | \
diff --git a/drivers/net/bnx2x_main.c b/drivers/net/bnx2x_main.c
index 59b58d8..1729c04 100644
--- a/drivers/net/bnx2x_main.c
+++ b/drivers/net/bnx2x_main.c
@@ -83,6 +83,10 @@ module_param(multi_mode, int, 0);
MODULE_PARM_DESC(multi_mode, " Multi queue mode "
"(0 Disable; 1 Enable (default))");
+static int get_hdrhash = 1;
+module_param(get_hdrhash, int, 0);
+MODULE_PARM_DESC(get_hdrhash, " Get Toeplitz hash from device");
+
static int num_rx_queues;
module_param(num_rx_queues, int, 0);
MODULE_PARM_DESC(num_rx_queues, " Number of Rx queues for multi_mode=1"
@@ -1520,7 +1524,7 @@ static int bnx2x_rx_int(struct bnx2x_fastpath
*fp, int budget)
struct sw_rx_bd *rx_buf = NULL;
struct sk_buff *skb;
union eth_rx_cqe *cqe;
- u8 cqe_fp_flags;
+ u8 cqe_fp_flags, cqe_fp_status_flags, cqe_fp_pars_flags;
u16 len, pad;
comp_ring_cons = RCQ_BD(sw_comp_cons);
@@ -1536,6 +1540,8 @@ static int bnx2x_rx_int(struct bnx2x_fastpath
*fp, int budget)
cqe = &fp->rx_comp_ring[comp_ring_cons];
cqe_fp_flags = cqe->fast_path_cqe.type_error_flags;
+ cqe_fp_status_flags = cqe->fast_path_cqe.status_flags;
+ cqe_fp_pars_flags = cqe->fast_path_cqe.pars_flags.flags;
DP(NETIF_MSG_RX_STATUS, "CQE type %x err %x status %x"
" queue %x vlan %x len %u\n", CQE_TYPE(cqe_fp_flags),
@@ -1661,7 +1667,32 @@ reuse_rx:
goto next_rx;
}
- skb->protocol = eth_type_trans(skb, bp->dev);
+
+ if (get_hdrhash && (cqe_fp_status_flags &
+ ETH_FAST_PATH_RX_CQE_RSS_HASH_FLG)) {
+ u8 hash_type = cqe_fp_status_flags &
+ ETH_FAST_PATH_RX_CQE_RSS_HASH_TYPE;
+
+ skb->rxhash = le32_to_cpu(
+ cqe->fast_path_cqe.rss_hash_result);
+ if (!skb->rxhash)
+ skb->rxhash = 1;
+
+ /* unicast IPv4 packet? */
+ if (((hash_type == IPV4_HASH_TYPE) ||
+ (hash_type == TCP_IPV4_HASH_TYPE)) &&
+ (cqe_fp_pars_flags &
+ PARSING_FLAGS_ETHERNET_ADDRESS_TYPE)) {
+ skb->dev = bp->dev;
+ skb_reset_mac_header(skb);
+ skb_pull(skb, ETH_HLEN);
+ skb->protocol =
+ __constant_htons(ETH_P_IP);
+ } else
+ skb->protocol =
+ eth_type_trans(skb, bp->dev);
+ } else
+ skb->protocol = eth_type_trans(skb, bp->dev);
skb->ip_summed = CHECKSUM_NONE;
if (bp->rx_csum) {
@@ -5388,8 +5419,11 @@ static void bnx2x_init_internal_func(struct bnx2x *bp)
u16 max_agg_size;
if (is_multi(bp)) {
- tstorm_config.config_flags = MULTI_FLAGS(bp);
+ tstorm_config.config_flags = RSS_FLAGS(bp);
tstorm_config.rss_result_mask = MULTI_MASK;
+ } else if (get_hdrhash) {
+ /* Set flags so the Toeplitz hash is provided */
+ tstorm_config.config_flags = RSS_FLAGS(bp);
}
/* Enable TPA if needed */
@@ -6223,9 +6257,10 @@ static int bnx2x_init_common(struct bnx2x *bp)
bnx2x_init_block(bp, PBF_BLOCK, COMMON_STAGE);
REG_WR(bp, SRC_REG_SOFT_RST, 1);
- for (i = SRC_REG_KEYRSS0_0; i <= SRC_REG_KEYRSS1_9; i += 4) {
- REG_WR(bp, i, 0xc0cac01a);
- /* TODO: replace with something meaningful */
+ {
+ int i;
+ for (i = SRC_REG_KEYRSS0_0; i <= SRC_REG_KEYRSS1_9; i += 4)
+ REG_WR(bp, i, random32());
}
bnx2x_init_block(bp, SRCH_BLOCK, COMMON_STAGE);
#ifdef BCM_CNIC
--
1.5.4.3
^ permalink raw reply related
* Re: xfrm command fails
From: Naveen BN @ 2009-11-11 7:10 UTC (permalink / raw)
To: netdev; +Cc: David Miller
In-Reply-To: <4AFA56B1.1010109@globaledgesoft.com>
Hi All,
This proper command to create SA using ip xfrm is
ip xfrm state add src 192.168.1.20 dst 192.168.1.5 proto esp spi
0x354cb04b mode tunnel reqid 16385 replay-window 32 auth sha1
0xecf02a5cf6568556e1bdcd961c7ec3f92afd01cc enc aes
0x5c0cfa9672ce67ba545b593076dfb278 sel src 0.0.0.0/0 dst 0.0.0.0/0
But how can create a session based SA with ports also used in sa
selection for the same host .
Please help me on this . In selection of OUTBOUND SA does ports also
involve.
Regards
Naveen
Naveen BN wrote:
> adding David in loop
> Naveen BN wrote:
>> I am using fedora with version => 2.6.23.1-42.fc8.
>> Please guide me to solve the error message "RTNETLINK answers:
>> Function not implemented"
>>
>> Regards
>> Naveen
>> Naveen BN wrote:
>>> Hi All
>>> I am trying to create a security association using ip xfrm command,
>>> but i get an message saying
>>> "*RTNETLINK answers: Function not implemented*" and ip xfrm state
>>> succeeds in displaying sa written using setkey .
>>>
>>> ip xfrm state add src 192.168.1.20 dst 192.168.1.5 proto esp spi
>>> 0x354cb04b mode tunnel reqid 16385 replay-window 32 auth hmac
>>> 0xecf02a5cf6568556e1bdcd961c7ec3f92afd01cc enc cbc
>>> 0x5c0cfa9672ce67ba545b593076dfb278 sel src 0.0.0.0/0 dst 0.0.0.0/0
>>>
>>> ERROR message => "RTNETLINK answers: Function not implemented"
>>>
>>> Regards
>>> Naveen
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox