* Re: Re: TSO and IPoIB performance degradation
From: Michael S. Tsirkin @ 2006-03-07 21:53 UTC (permalink / raw)
To: Stephen Hemminger
Cc: netdev, Linux Kernel Mailing List, openib-general,
David S. Miller
In-Reply-To: <20060307134907.733d3d27@localhost.localdomain>
Quoting r. Stephen Hemminger <shemminger@osdl.org>:
> Is IB using NAPI or just doing netif_rx()?
No, IPoIB doesn't use NAPI.
--
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies
^ permalink raw reply
* Re: [PATCH] IPoIB: Fix build now that destructor is in neigh_params
From: David S. Miller @ 2006-03-07 22:53 UTC (permalink / raw)
To: rdreier; +Cc: akpm, netdev, openib-general
In-Reply-To: <adawtf6rrh7.fsf_-_@cisco.com>
From: Roland Dreier <rdreier@cisco.com>
Date: Tue, 07 Mar 2006 11:21:08 -0800
> Dave, here's an incremental patch that fixes the IPoIB build (which is
> broken in net-2.6.17 because of my screw-up, which left out the chunk
> below). I'll also send a full patch that can replace the "Move
> destructor from neigh->ops to neigh_params" patch if you'd rather
> replace it in your tree.
>
> Thanks, and sorry about the screw-up.
Why not just put this into your -ipoib patch set in -mm?
The change was for IPOIB's sake anyways...
^ permalink raw reply
* Re: [PATCH] IPoIB: Fix build now that destructor is in neigh_params
From: Roland Dreier @ 2006-03-07 22:56 UTC (permalink / raw)
To: David S. Miller; +Cc: akpm, netdev, openib-general
In-Reply-To: <20060307.145350.77989471.davem@davemloft.net>
David> Why not just put this into your -ipoib patch set in -mm?
David> The change was for IPOIB's sake anyways...
OK, good idea. I'll put this in my for-2.6.17 and for-mm queues.
- R.
^ permalink raw reply
* Re: [PATCH] [NETFILTER] ip_queue: Fix wrong skb->len == nlmsg_len assumption
From: David S. Miller @ 2006-03-07 23:01 UTC (permalink / raw)
To: kaber; +Cc: netdev, netfilter-devel
In-Reply-To: <440D838D.8000302@trash.net>
From: Patrick McHardy <kaber@trash.net>
Date: Tue, 07 Mar 2006 13:58:53 +0100
> Thomas Graf wrote:
> > The size of the skb carrying the netlink message is not
> > equivalent to the length of the actual netlink message
> > due to padding. ip_queue matches the length of the payload
> > against the original packet size to determine if packet
> > mangling is desired, due to the above wrong assumption
> > arbitary packets may not be mangled depening on their
> > original size.
>
> Looks good, thanks Thomas. I think this should also go in 2.4.
Pushed to 2.6.16, 2.6.x stable, and 2.4.x.
Phew!
Thanks Thomas.
^ permalink raw reply
* Re: Re: TSO and IPoIB performance degradation
From: Matt Leininger @ 2006-03-08 0:11 UTC (permalink / raw)
To: Stephen Hemminger
Cc: netdev, Linux Kernel Mailing List, openib-general,
David S. Miller
In-Reply-To: <20060307134907.733d3d27@localhost.localdomain>
On Tue, 2006-03-07 at 13:49 -0800, Stephen Hemminger wrote:
> On Tue, 07 Mar 2006 13:44:51 -0800
> Matt Leininger <mlleinin@hpcn.ca.sandia.gov> wrote:
>
> > On Mon, 2006-03-06 at 19:13 -0800, Shirley Ma wrote:
> > >
> > > > More likely you are getting hit by the fact that TSO prevents the
> > > congestion
> > > window from increasing properly. This was fixed in 2.6.15 (around mid
> > > of Nov 2005).
> > >
> > > Yep, I noticed the same problem. After updating to the new kernel, the
> > > performance are much better, but it's still lower than before.
> >
> > Here is an updated version of OpenIB IPoIB performance for various
> > kernels with and without one of the TSO patches. The netperf
> > performance for the latest kernels has not improved the TSO performance
> > drop.
> >
> > Any comments or suggestions would be appreciated.
> >
> > - Matt
>
> Configuration information? like did you increase the tcp_rmem, tcp_wmem?
> Tcpdump traces of what is being sent and available window?
> Is IB using NAPI or just doing netif_rx()?
I used the standard setting for tcp_rmem and tcp_wmem. Here are a
few other runs that change those variables. I was able to improve
performance by ~30MB/s to 403 MB/s, but this is still a ways from the
474 MB/s before the TSO patches.
Thanks,
- Matt
All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0
dual EM64T 3.2 GHz PCIe IB HCA (memfull)
patch 1 - remove changeset 314324121f9b94b2ca657a494cf2b9cb0e4a28cc
msi_x=1 for all tests
Kernel OpenIB netperf (MB/s)
2.6.16-rc5 in-kernel 403
tcp_wmem 4096 87380 16777216 tcp_rmem 4096 87380 16777216
2.6.16-rc5 in-kernel 395
tcp_wmem 4096 102400 16777216 tcp_rmem 4096 102400 16777216
2.6.16-rc5 in-kernel 392
tcp_wmem 4096 65536 16777216 tcp_rmem 4096 87380 16777216
2.6.16-rc5 in-kernel 394
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 102400 16777216
2.6.16-rc5 in-kernel 377
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 153600 16777216
2.6.16-rc5 in-kernel 377
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 131072 16777216
2.6.16-rc5 in-kernel 353
tcp_wmem 4096 262144 16777216 tcp_rmem 4096 262144 16777216
2.6.16-rc5 in-kernel 305
tcp_wmem 4096 262144 16777216 tcp_rmem 4096 524288 16777216
2.6.16-rc5 in-kernel 303
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 524288 16777216
2.6.16-rc5 in-kernel 290
tcp_wmem 4096 524288 16777216 tcp_rmem 4096 524288 16777216
2.6.16-rc5 in-kernel 367 default tcp values
--------------------
All with standard tcp settings
Kernel OpenIB netperf (MB/s)
2.6.16-rc5 in-kernel 367
2.6.15 in-kernel 382
2.6.14-rc4 patch 12 in-kernel 436
2.6.14-rc4 patch 1 in-kernel 434
2.6.14-rc4 in-kernel 385
2.6.14-rc3 in-kernel 374
2.6.13.2 svn3627 386
2.6.13.2 patch 1 svn3627 446
2.6.13.2 in-kernel 394
2.6.13-rc3 patch 12 in-kernel 442
2.6.13-rc3 patch 1 in-kernel 450
2.6.13-rc3 in-kernel 395
2.6.12.5-lustre in-kernel 399
2.6.12.5 patch 1 in-kernel 464
2.6.12.5 in-kernel 402
2.6.12 in-kernel 406
2.6.12-rc6 patch 1 in-kernel 470
2.6.12-rc6 in-kernel 407
2.6.12-rc5 in-kernel 405
2.6.12-rc5 patch 1 in-kernel 474
2.6.12-rc4 in-kernel 470
2.6.12-rc3 in-kernel 466
2.6.12-rc2 in-kernel 469
2.6.12-rc1 in-kernel 466
2.6.11 in-kernel 464
2.6.11 svn3687 464
2.6.9-11.ELsmp svn3513 425 (Woody's results, 3.6Ghz EM64T)
^ permalink raw reply
* Re: de2104x: interrupts before interrupt handler is registered
From: Francois Romieu @ 2006-03-08 0:15 UTC (permalink / raw)
To: Martin Michlmayr; +Cc: netdev, linux-kernel
In-Reply-To: <20060307051152.GA1244@deprecation.cyrius.com>
Martin Michlmayr <tbm@cyrius.com> :
[...]
> It seems to help. It's hard to say for sure because I don't have a
> foolproof way to reproduce this panic. It _usually_ occurs after
> copying a few hundred MB but there's no clear trigger. I've now copied
> a few GB around using a kernel with your patch and it hasn't crashed.
netdev watchdog events appear in the dmesg of the patched driver.
The driver survived it. So I'd say that the patch does its job.
OTOH, if you ever saw the unpatched driver survive this event, yell now.
--
Ueimor
^ permalink raw reply
* Re: Re: TSO and IPoIB performance degradation
From: David S. Miller @ 2006-03-08 0:18 UTC (permalink / raw)
To: mlleinin; +Cc: netdev, linux-kernel, openib-general, shemminger
In-Reply-To: <1141776697.6119.938.camel@localhost>
From: Matt Leininger <mlleinin@hpcn.ca.sandia.gov>
Date: Tue, 07 Mar 2006 16:11:37 -0800
> I used the standard setting for tcp_rmem and tcp_wmem. Here are a
> few other runs that change those variables. I was able to improve
> performance by ~30MB/s to 403 MB/s, but this is still a ways from the
> 474 MB/s before the TSO patches.
How limited are the IPoIB devices, TX descriptor wise?
One side effect of the TSO changes is that one extra descriptor
will be used for outgoing packets. This is because we have to
put the headers as well as the user data, into page based
buffers now.
Perhaps you can experiment with increasing the transmit descriptor
table size, if that's possible.
^ permalink raw reply
* Re: Re: TSO and IPoIB performance degradation
From: Roland Dreier @ 2006-03-08 1:17 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, linux-kernel, openib-general, shemminger
In-Reply-To: <20060307.161808.60227862.davem@davemloft.net>
David> How limited are the IPoIB devices, TX descriptor wise?
David> One side effect of the TSO changes is that one extra
David> descriptor will be used for outgoing packets. This is
David> because we have to put the headers as well as the user
David> data, into page based buffers now.
We have essentially no limit on TX descriptors. However I think
there's some confusion about TSO: IPoIB does _not_ do TSO -- generic
InfiniBand hardware does not have any TSO capability. In the future
we might be able to implement TSO for certain hardware that does have
support, but even that requires some firmware help from the from the
HCA vendors, etc. So right now the IPoIB driver does not do TSO.
The reason TSO comes up is that reverting the patch described below
helps (or helped at some point at least) IPoIB throughput quite a bit.
Clearly this was a bug fix so we can't revert it in general but I
think what Michael Tsirkin was suggesting at the beginning of this
thread is to do what the last paragraph of the changelog says -- find
some way to re-enable the trick.
diff-tree 3143241... (from e16fa6b...)
Author: David S. Miller <davem@davemloft.net>
Date: Mon May 23 12:03:06 2005 -0700
[TCP]: Fix stretch ACK performance killer when doing ucopy.
When we are doing ucopy, we try to defer the ACK generation to
cleanup_rbuf(). This works most of the time very well, but if the
ucopy prequeue is large, this ACKing behavior kills performance.
With TSO, it is possible to fill the prequeue so large that by the
time the ACK is sent and gets back to the sender, most of the window
has emptied of data and performance suffers significantly.
This behavior does help in some cases, so we should think about
re-enabling this trick in the future, using some kind of limit in
order to avoid the bug case.
- R.
^ permalink raw reply
* Re: Re: TSO and IPoIB performance degradation
From: David S. Miller @ 2006-03-08 1:23 UTC (permalink / raw)
To: rdreier; +Cc: netdev, linux-kernel, openib-general, shemminger
In-Reply-To: <adaacc1raz9.fsf@cisco.com>
From: Roland Dreier <rdreier@cisco.com>
Date: Tue, 07 Mar 2006 17:17:30 -0800
> The reason TSO comes up is that reverting the patch described below
> helps (or helped at some point at least) IPoIB throughput quite a bit.
I wish you had started the thread by mentioning this specific
patch, we wasted an enormous amount of precious developer time
speculating and asking for arbitrary tests to be run in order
to narrow down the problem, yet you knew the specific change
that introduced the performance regression already...
This is a good example of how not to report a bug.
^ permalink raw reply
* Re: Re: TSO and IPoIB performance degradation
From: Roland Dreier @ 2006-03-08 1:34 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, linux-kernel, openib-general, shemminger
In-Reply-To: <20060307.172336.107863253.davem@davemloft.net>
David> I wish you had started the thread by mentioning this
David> specific patch, we wasted an enormous amount of precious
David> developer time speculating and asking for arbitrary tests
David> to be run in order to narrow down the problem, yet you knew
David> the specific change that introduced the performance
David> regression already...
Sorry, you're right. I was a little confused because I had a memory of
Michael's original email (http://lkml.org/lkml/2006/3/6/150) quoting a
changelog entry, but looking back at the message, it was quoting
something completely different and misleading.
I think the most interesting email in the old thread is
http://openib.org/pipermail/openib-general/2005-October/012482.html
which shows that reverting 314324121 (the "stretch ACK performance
killer" fix) gives ~400 Mbit/sec in extra IPoIB performance.
- R.
^ permalink raw reply
* [patch 0/4] net: percpufy frequently used vars on struct proto
From: Ravikiran G Thirumalai @ 2006-03-08 1:58 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, davem, netdev, shai
Following patchset converts struct proto.memory_allocated to use batching
per-cpu counters, struct proto.sockets_allocated to use per-cpu counters and
changes the proto.inuse per-cpu variable to use alloc_percpu instead of the
NR_CPUS x cacheline size padding.
We observed 5% improvement in apache bench requests per second with this
patchset on a multi NIC 8 way IBM x460 box.
(This was posted earlier
http://marc.theaimsgroup.com/?l=linux-kernel&m=113830220408812&w=2 )
Can this go into -mm please?
Thanks,
Kiran
^ permalink raw reply
* [patch 1/4] net: percpufy frequently used vars -- add percpu_counter_mod_bh
From: Ravikiran G Thirumalai @ 2006-03-08 1:59 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308015808.GA9062@localhost.localdomain>
Add percpu_counter_mod_bh for using these counters safely from
both softirq and process context.
Signed-off by: Pravin B. Shelar <pravins@calsoftinc.com>
Signed-off by: Ravikiran G Thirumalai <kiran@scalex86.org>
Signed-off by: Shai Fultheim <shai@scalex86.org>
Index: linux-2.6.16-rc5mm3/include/linux/percpu_counter.h
===================================================================
--- linux-2.6.16-rc5mm3.orig/include/linux/percpu_counter.h 2006-03-07 15:08:00.000000000 -0800
+++ linux-2.6.16-rc5mm3/include/linux/percpu_counter.h 2006-03-07 15:09:21.000000000 -0800
@@ -11,6 +11,7 @@
#include <linux/smp.h>
#include <linux/threads.h>
#include <linux/percpu.h>
+#include <linux/interrupt.h>
#ifdef CONFIG_SMP
@@ -110,4 +111,11 @@ static inline void percpu_counter_dec(st
percpu_counter_mod(fbc, -1);
}
+static inline void percpu_counter_mod_bh(struct percpu_counter *fbc, long amount)
+{
+ local_bh_disable();
+ percpu_counter_mod(fbc, amount);
+ local_bh_enable();
+}
+
#endif /* _LINUX_PERCPU_COUNTER_H */
^ permalink raw reply
* [patch 2/4] net: percpufy frequently used vars -- struct proto.memory_allocated
From: Ravikiran G Thirumalai @ 2006-03-08 2:01 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308015808.GA9062@localhost.localdomain>
Change struct proto->memory_allocated to a batching per-CPU counter
(percpu_counter) from an atomic_t. A batching counter is better than a
plain per-CPU counter as this field is read often.
Signed-off-by: Pravin B. Shelar <pravins@calsoftinc.com>
Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: Shai Fultheim <shai@scalex86.org>
Index: linux-2.6.16-rc5mm3/include/net/sock.h
===================================================================
--- linux-2.6.16-rc5mm3.orig/include/net/sock.h 2006-03-07 15:07:43.000000000 -0800
+++ linux-2.6.16-rc5mm3/include/net/sock.h 2006-03-07 15:11:48.000000000 -0800
@@ -48,6 +48,7 @@
#include <linux/netdevice.h>
#include <linux/skbuff.h> /* struct sk_buff */
#include <linux/security.h>
+#include <linux/percpu_counter.h>
#include <linux/filter.h>
@@ -541,8 +542,9 @@ struct proto {
/* Memory pressure */
void (*enter_memory_pressure)(void);
- atomic_t *memory_allocated; /* Current allocated memory. */
+ struct percpu_counter *memory_allocated; /* Current allocated memory. */
atomic_t *sockets_allocated; /* Current number of sockets. */
+
/*
* Pressure flag: try to collapse.
* Technical note: it is used by multiple contexts non atomically.
Index: linux-2.6.16-rc5mm3/include/net/tcp.h
===================================================================
--- linux-2.6.16-rc5mm3.orig/include/net/tcp.h 2006-03-07 15:08:00.000000000 -0800
+++ linux-2.6.16-rc5mm3/include/net/tcp.h 2006-03-07 15:11:48.000000000 -0800
@@ -225,7 +225,7 @@ extern int sysctl_tcp_abc;
extern int sysctl_tcp_mtu_probing;
extern int sysctl_tcp_base_mss;
-extern atomic_t tcp_memory_allocated;
+extern struct percpu_counter tcp_memory_allocated;
extern atomic_t tcp_sockets_allocated;
extern int tcp_memory_pressure;
Index: linux-2.6.16-rc5mm3/net/core/sock.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/core/sock.c 2006-03-07 15:07:43.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/core/sock.c 2006-03-07 15:11:48.000000000 -0800
@@ -1616,12 +1616,13 @@ static char proto_method_implemented(con
static void proto_seq_printf(struct seq_file *seq, struct proto *proto)
{
- seq_printf(seq, "%-9s %4u %6d %6d %-3s %6u %-3s %-10s "
+ seq_printf(seq, "%-9s %4u %6d %6lu %-3s %6u %-3s %-10s "
"%2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c\n",
proto->name,
proto->obj_size,
proto->sockets_allocated != NULL ? atomic_read(proto->sockets_allocated) : -1,
- proto->memory_allocated != NULL ? atomic_read(proto->memory_allocated) : -1,
+ proto->memory_allocated != NULL ?
+ percpu_counter_read_positive(proto->memory_allocated) : -1,
proto->memory_pressure != NULL ? *proto->memory_pressure ? "yes" : "no" : "NI",
proto->max_header,
proto->slab == NULL ? "no" : "yes",
Index: linux-2.6.16-rc5mm3/net/core/stream.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/core/stream.c 2006-03-07 15:07:43.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/core/stream.c 2006-03-07 15:11:48.000000000 -0800
@@ -196,11 +196,11 @@ EXPORT_SYMBOL(sk_stream_error);
void __sk_stream_mem_reclaim(struct sock *sk)
{
if (sk->sk_forward_alloc >= SK_STREAM_MEM_QUANTUM) {
- atomic_sub(sk->sk_forward_alloc / SK_STREAM_MEM_QUANTUM,
- sk->sk_prot->memory_allocated);
+ percpu_counter_mod_bh(sk->sk_prot->memory_allocated,
+ -(sk->sk_forward_alloc / SK_STREAM_MEM_QUANTUM));
sk->sk_forward_alloc &= SK_STREAM_MEM_QUANTUM - 1;
if (*sk->sk_prot->memory_pressure &&
- (atomic_read(sk->sk_prot->memory_allocated) <
+ (percpu_counter_read(sk->sk_prot->memory_allocated) <
sk->sk_prot->sysctl_mem[0]))
*sk->sk_prot->memory_pressure = 0;
}
@@ -213,23 +213,26 @@ int sk_stream_mem_schedule(struct sock *
int amt = sk_stream_pages(size);
sk->sk_forward_alloc += amt * SK_STREAM_MEM_QUANTUM;
- atomic_add(amt, sk->sk_prot->memory_allocated);
+ percpu_counter_mod_bh(sk->sk_prot->memory_allocated, amt);
/* Under limit. */
- if (atomic_read(sk->sk_prot->memory_allocated) < sk->sk_prot->sysctl_mem[0]) {
+ if (percpu_counter_read(sk->sk_prot->memory_allocated) <
+ sk->sk_prot->sysctl_mem[0]) {
if (*sk->sk_prot->memory_pressure)
*sk->sk_prot->memory_pressure = 0;
return 1;
}
/* Over hard limit. */
- if (atomic_read(sk->sk_prot->memory_allocated) > sk->sk_prot->sysctl_mem[2]) {
+ if (percpu_counter_read(sk->sk_prot->memory_allocated) >
+ sk->sk_prot->sysctl_mem[2]) {
sk->sk_prot->enter_memory_pressure();
goto suppress_allocation;
}
/* Under pressure. */
- if (atomic_read(sk->sk_prot->memory_allocated) > sk->sk_prot->sysctl_mem[1])
+ if (percpu_counter_read(sk->sk_prot->memory_allocated) >
+ sk->sk_prot->sysctl_mem[1])
sk->sk_prot->enter_memory_pressure();
if (kind) {
@@ -259,7 +262,7 @@ suppress_allocation:
/* Alas. Undo changes. */
sk->sk_forward_alloc -= amt * SK_STREAM_MEM_QUANTUM;
- atomic_sub(amt, sk->sk_prot->memory_allocated);
+ percpu_counter_mod_bh(sk->sk_prot->memory_allocated, -amt);
return 0;
}
Index: linux-2.6.16-rc5mm3/net/decnet/af_decnet.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/decnet/af_decnet.c 2006-03-07 15:07:43.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/decnet/af_decnet.c 2006-03-07 15:09:22.000000000 -0800
@@ -154,7 +154,7 @@ static const struct proto_ops dn_proto_o
static DEFINE_RWLOCK(dn_hash_lock);
static struct hlist_head dn_sk_hash[DN_SK_HASH_SIZE];
static struct hlist_head dn_wild_sk;
-static atomic_t decnet_memory_allocated;
+static struct percpu_counter decnet_memory_allocated;
static int __dn_setsockopt(struct socket *sock, int level, int optname, char __user *optval, int optlen, int flags);
static int __dn_getsockopt(struct socket *sock, int level, int optname, char __user *optval, int __user *optlen, int flags);
@@ -2383,7 +2383,8 @@ static int __init decnet_init(void)
rc = proto_register(&dn_proto, 1);
if (rc != 0)
goto out;
-
+
+ percpu_counter_init(&decnet_memory_allocated);
dn_neigh_init();
dn_dev_init();
dn_route_init();
@@ -2424,6 +2425,7 @@ static void __exit decnet_exit(void)
proc_net_remove("decnet");
proto_unregister(&dn_proto);
+ percpu_counter_destroy(&decnet_memory_allocated);
}
module_exit(decnet_exit);
#endif
Index: linux-2.6.16-rc5mm3/net/ipv4/proc.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/proc.c 2006-03-07 15:07:44.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/proc.c 2006-03-07 15:11:48.000000000 -0800
@@ -61,10 +61,10 @@ static int fold_prot_inuse(struct proto
static int sockstat_seq_show(struct seq_file *seq, void *v)
{
socket_seq_show(seq);
- seq_printf(seq, "TCP: inuse %d orphan %d tw %d alloc %d mem %d\n",
+ seq_printf(seq, "TCP: inuse %d orphan %d tw %d alloc %d mem %lu\n",
fold_prot_inuse(&tcp_prot), atomic_read(&tcp_orphan_count),
tcp_death_row.tw_count, atomic_read(&tcp_sockets_allocated),
- atomic_read(&tcp_memory_allocated));
+ percpu_counter_read_positive(&tcp_memory_allocated));
seq_printf(seq, "UDP: inuse %d\n", fold_prot_inuse(&udp_prot));
seq_printf(seq, "RAW: inuse %d\n", fold_prot_inuse(&raw_prot));
seq_printf(seq, "FRAG: inuse %d memory %d\n", ip_frag_nqueues,
Index: linux-2.6.16-rc5mm3/net/ipv4/tcp.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/tcp.c 2006-03-07 15:07:44.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/tcp.c 2006-03-07 15:11:48.000000000 -0800
@@ -283,7 +283,7 @@ EXPORT_SYMBOL(sysctl_tcp_mem);
EXPORT_SYMBOL(sysctl_tcp_rmem);
EXPORT_SYMBOL(sysctl_tcp_wmem);
-atomic_t tcp_memory_allocated; /* Current allocated memory. */
+struct percpu_counter tcp_memory_allocated; /* Current allocated memory. */
atomic_t tcp_sockets_allocated; /* Current number of TCP sockets. */
EXPORT_SYMBOL(tcp_memory_allocated);
@@ -1593,7 +1593,7 @@ adjudge_to_death:
sk_stream_mem_reclaim(sk);
if (atomic_read(sk->sk_prot->orphan_count) > sysctl_tcp_max_orphans ||
(sk->sk_wmem_queued > SOCK_MIN_SNDBUF &&
- atomic_read(&tcp_memory_allocated) > sysctl_tcp_mem[2])) {
+ percpu_counter_read(&tcp_memory_allocated) > sysctl_tcp_mem[2])) {
if (net_ratelimit())
printk(KERN_INFO "TCP: too many of orphaned "
"sockets\n");
@@ -2127,6 +2127,8 @@ void __init tcp_init(void)
"(established %d bind %d)\n",
tcp_hashinfo.ehash_size << 1, tcp_hashinfo.bhash_size);
+ percpu_counter_init(&tcp_memory_allocated);
+
tcp_register_congestion_control(&tcp_reno);
}
Index: linux-2.6.16-rc5mm3/net/ipv4/tcp_input.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/tcp_input.c 2006-03-07 15:08:01.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/tcp_input.c 2006-03-07 15:09:22.000000000 -0800
@@ -333,7 +333,7 @@ static void tcp_clamp_window(struct sock
if (sk->sk_rcvbuf < sysctl_tcp_rmem[2] &&
!(sk->sk_userlocks & SOCK_RCVBUF_LOCK) &&
!tcp_memory_pressure &&
- atomic_read(&tcp_memory_allocated) < sysctl_tcp_mem[0]) {
+ percpu_counter_read(&tcp_memory_allocated) < sysctl_tcp_mem[0]) {
sk->sk_rcvbuf = min(atomic_read(&sk->sk_rmem_alloc),
sysctl_tcp_rmem[2]);
}
@@ -3555,7 +3555,7 @@ static int tcp_should_expand_sndbuf(stru
return 0;
/* If we are under soft global TCP memory pressure, do not expand. */
- if (atomic_read(&tcp_memory_allocated) >= sysctl_tcp_mem[0])
+ if (percpu_counter_read(&tcp_memory_allocated) >= sysctl_tcp_mem[0])
return 0;
/* If we filled the congestion window, do not expand. */
Index: linux-2.6.16-rc5mm3/net/ipv4/tcp_timer.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/tcp_timer.c 2006-03-07 15:08:01.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/tcp_timer.c 2006-03-07 15:09:22.000000000 -0800
@@ -80,7 +80,7 @@ static int tcp_out_of_resources(struct s
if (orphans >= sysctl_tcp_max_orphans ||
(sk->sk_wmem_queued > SOCK_MIN_SNDBUF &&
- atomic_read(&tcp_memory_allocated) > sysctl_tcp_mem[2])) {
+ percpu_counter_read(&tcp_memory_allocated) > sysctl_tcp_mem[2])) {
if (net_ratelimit())
printk(KERN_INFO "Out of socket memory\n");
^ permalink raw reply
* [patch 3/4] net: percpufy frequently used vars -- proto.sockets_allocated
From: Ravikiran G Thirumalai @ 2006-03-08 2:02 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308015808.GA9062@localhost.localdomain>
Change the atomic_t sockets_allocated member of struct proto to a
per-cpu counter.
Signed-off-by: Pravin B. Shelar <pravins@calsoftinc.com>
Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: Shai Fultheim <shai@scalex86.org>
Index: linux-2.6.16-rc5mm3/include/net/sock.h
===================================================================
--- linux-2.6.16-rc5mm3.orig/include/net/sock.h 2006-03-07 15:09:22.000000000 -0800
+++ linux-2.6.16-rc5mm3/include/net/sock.h 2006-03-07 15:09:52.000000000 -0800
@@ -543,7 +543,7 @@ struct proto {
/* Memory pressure */
void (*enter_memory_pressure)(void);
struct percpu_counter *memory_allocated; /* Current allocated memory. */
- atomic_t *sockets_allocated; /* Current number of sockets. */
+ int *sockets_allocated; /* Current number of sockets (percpu counter). */
/*
* Pressure flag: try to collapse.
@@ -579,6 +579,24 @@ struct proto {
} stats[NR_CPUS];
};
+static inline int read_sockets_allocated(struct proto *prot)
+{
+ int total = 0;
+ int cpu;
+ for_each_cpu(cpu)
+ total += *per_cpu_ptr(prot->sockets_allocated, cpu);
+ return total;
+}
+
+static inline void mod_sockets_allocated(int *sockets_allocated, int count)
+{
+ (*per_cpu_ptr(sockets_allocated, get_cpu())) += count;
+ put_cpu();
+}
+
+#define inc_sockets_allocated(c) mod_sockets_allocated(c, 1)
+#define dec_sockets_allocated(c) mod_sockets_allocated(c, -1)
+
extern int proto_register(struct proto *prot, int alloc_slab);
extern void proto_unregister(struct proto *prot);
Index: linux-2.6.16-rc5mm3/include/net/tcp.h
===================================================================
--- linux-2.6.16-rc5mm3.orig/include/net/tcp.h 2006-03-07 15:09:22.000000000 -0800
+++ linux-2.6.16-rc5mm3/include/net/tcp.h 2006-03-07 15:09:52.000000000 -0800
@@ -226,7 +226,7 @@ extern int sysctl_tcp_mtu_probing;
extern int sysctl_tcp_base_mss;
extern struct percpu_counter tcp_memory_allocated;
-extern atomic_t tcp_sockets_allocated;
+extern int *tcp_sockets_allocated;
extern int tcp_memory_pressure;
/*
Index: linux-2.6.16-rc5mm3/net/core/sock.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/core/sock.c 2006-03-07 15:09:22.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/core/sock.c 2006-03-07 15:09:52.000000000 -0800
@@ -771,7 +771,7 @@ struct sock *sk_clone(const struct sock
newsk->sk_sleep = NULL;
if (newsk->sk_prot->sockets_allocated)
- atomic_inc(newsk->sk_prot->sockets_allocated);
+ inc_sockets_allocated(newsk->sk_prot->sockets_allocated);
}
out:
return newsk;
@@ -1620,7 +1620,7 @@ static void proto_seq_printf(struct seq_
"%2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c %2c\n",
proto->name,
proto->obj_size,
- proto->sockets_allocated != NULL ? atomic_read(proto->sockets_allocated) : -1,
+ proto->sockets_allocated != NULL ? read_sockets_allocated(proto) : -1,
proto->memory_allocated != NULL ?
percpu_counter_read_positive(proto->memory_allocated) : -1,
proto->memory_pressure != NULL ? *proto->memory_pressure ? "yes" : "no" : "NI",
Index: linux-2.6.16-rc5mm3/net/core/stream.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/core/stream.c 2006-03-07 15:09:22.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/core/stream.c 2006-03-07 15:09:52.000000000 -0800
@@ -242,7 +242,7 @@ int sk_stream_mem_schedule(struct sock *
return 1;
if (!*sk->sk_prot->memory_pressure ||
- sk->sk_prot->sysctl_mem[2] > atomic_read(sk->sk_prot->sockets_allocated) *
+ sk->sk_prot->sysctl_mem[2] > read_sockets_allocated(sk->sk_prot) *
sk_stream_pages(sk->sk_wmem_queued +
atomic_read(&sk->sk_rmem_alloc) +
sk->sk_forward_alloc))
Index: linux-2.6.16-rc5mm3/net/ipv4/proc.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/proc.c 2006-03-07 15:09:22.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/proc.c 2006-03-07 15:09:52.000000000 -0800
@@ -63,7 +63,7 @@ static int sockstat_seq_show(struct seq_
socket_seq_show(seq);
seq_printf(seq, "TCP: inuse %d orphan %d tw %d alloc %d mem %lu\n",
fold_prot_inuse(&tcp_prot), atomic_read(&tcp_orphan_count),
- tcp_death_row.tw_count, atomic_read(&tcp_sockets_allocated),
+ tcp_death_row.tw_count, read_sockets_allocated(&tcp_prot),
percpu_counter_read_positive(&tcp_memory_allocated));
seq_printf(seq, "UDP: inuse %d\n", fold_prot_inuse(&udp_prot));
seq_printf(seq, "RAW: inuse %d\n", fold_prot_inuse(&raw_prot));
Index: linux-2.6.16-rc5mm3/net/ipv4/tcp.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/tcp.c 2006-03-07 15:09:22.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/tcp.c 2006-03-07 15:09:52.000000000 -0800
@@ -284,7 +284,7 @@ EXPORT_SYMBOL(sysctl_tcp_rmem);
EXPORT_SYMBOL(sysctl_tcp_wmem);
struct percpu_counter tcp_memory_allocated; /* Current allocated memory. */
-atomic_t tcp_sockets_allocated; /* Current number of TCP sockets. */
+int *tcp_sockets_allocated; /* Current number of TCP sockets. */
EXPORT_SYMBOL(tcp_memory_allocated);
EXPORT_SYMBOL(tcp_sockets_allocated);
@@ -2055,6 +2055,12 @@ void __init tcp_init(void)
if (!tcp_hashinfo.bind_bucket_cachep)
panic("tcp_init: Cannot alloc tcp_bind_bucket cache.");
+ tcp_sockets_allocated = alloc_percpu(int);
+ if (!tcp_sockets_allocated)
+ panic("tcp_init: Cannot alloc tcp_sockets_allocated");
+
+ tcp_prot.sockets_allocated = tcp_sockets_allocated;
+
/* Size and allocate the main established and bind bucket
* hash tables.
*
Index: linux-2.6.16-rc5mm3/net/ipv4/tcp_ipv4.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv4/tcp_ipv4.c 2006-03-07 15:08:01.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv4/tcp_ipv4.c 2006-03-07 15:09:52.000000000 -0800
@@ -1273,7 +1273,7 @@ static int tcp_v4_init_sock(struct sock
sk->sk_sndbuf = sysctl_tcp_wmem[1];
sk->sk_rcvbuf = sysctl_tcp_rmem[1];
- atomic_inc(&tcp_sockets_allocated);
+ inc_sockets_allocated(tcp_sockets_allocated);
return 0;
}
@@ -1307,7 +1307,7 @@ int tcp_v4_destroy_sock(struct sock *sk)
sk->sk_sndmsg_page = NULL;
}
- atomic_dec(&tcp_sockets_allocated);
+ dec_sockets_allocated(tcp_sockets_allocated);
return 0;
}
@@ -1815,7 +1815,6 @@ struct proto tcp_prot = {
.unhash = tcp_unhash,
.get_port = tcp_v4_get_port,
.enter_memory_pressure = tcp_enter_memory_pressure,
- .sockets_allocated = &tcp_sockets_allocated,
.orphan_count = &tcp_orphan_count,
.memory_allocated = &tcp_memory_allocated,
.memory_pressure = &tcp_memory_pressure,
Index: linux-2.6.16-rc5mm3/net/ipv6/tcp_ipv6.c
===================================================================
--- linux-2.6.16-rc5mm3.orig/net/ipv6/tcp_ipv6.c 2006-03-07 15:08:01.000000000 -0800
+++ linux-2.6.16-rc5mm3/net/ipv6/tcp_ipv6.c 2006-03-07 15:11:43.000000000 -0800
@@ -1375,7 +1375,7 @@ static int tcp_v6_init_sock(struct sock
sk->sk_sndbuf = sysctl_tcp_wmem[1];
sk->sk_rcvbuf = sysctl_tcp_rmem[1];
- atomic_inc(&tcp_sockets_allocated);
+ inc_sockets_allocated(tcp_sockets_allocated);
return 0;
}
@@ -1573,7 +1573,6 @@ struct proto tcpv6_prot = {
.unhash = tcp_unhash,
.get_port = tcp_v6_get_port,
.enter_memory_pressure = tcp_enter_memory_pressure,
- .sockets_allocated = &tcp_sockets_allocated,
.memory_allocated = &tcp_memory_allocated,
.memory_pressure = &tcp_memory_pressure,
.orphan_count = &tcp_orphan_count,
@@ -1605,6 +1604,7 @@ static struct inet_protosw tcpv6_protosw
void __init tcpv6_init(void)
{
+ tcpv6_prot.sockets_allocated = tcp_sockets_allocated;
/* register inet6 protocol */
if (inet6_add_protocol(&tcpv6_protocol, IPPROTO_TCP) < 0)
printk(KERN_ERR "tcpv6_init: Could not register protocol\n");
^ permalink raw reply
* [patch 4/4] net: percpufy frequently used vars -- proto.inuse
From: Ravikiran G Thirumalai @ 2006-03-08 2:03 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308015808.GA9062@localhost.localdomain>
This patch uses alloc_percpu to allocate per-CPU memory for the proto->inuse
field. The inuse field is currently per-CPU as in NR_CPUS * cacheline size --
a big bloat on arches with large cachelines. Also marks some frequently
used protos read mostly.
Signed-off-by: Pravin B. Shelar <pravins@calsoftinc.com>
Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: Shai Fultheim <shai@scalex86.org>
Index: linux-2.6.16-rc2/include/net/sock.h
===================================================================
--- linux-2.6.16-rc2.orig/include/net/sock.h 2006-02-09 15:01:47.000000000 -0800
+++ linux-2.6.16-rc2/include/net/sock.h 2006-02-09 15:01:52.000000000 -0800
@@ -573,10 +573,7 @@ struct proto {
#ifdef SOCK_REFCNT_DEBUG
atomic_t socks;
#endif
- struct {
- int inuse;
- u8 __pad[SMP_CACHE_BYTES - sizeof(int)];
- } stats[NR_CPUS];
+ int *inuse;
};
static inline int read_sockets_allocated(struct proto *prot)
@@ -628,12 +625,12 @@ static inline void sk_refcnt_debug_relea
/* Called with local bh disabled */
static __inline__ void sock_prot_inc_use(struct proto *prot)
{
- prot->stats[smp_processor_id()].inuse++;
+ (*per_cpu_ptr(prot->inuse, smp_processor_id())) += 1;
}
static __inline__ void sock_prot_dec_use(struct proto *prot)
{
- prot->stats[smp_processor_id()].inuse--;
+ (*per_cpu_ptr(prot->inuse, smp_processor_id())) -= 1;
}
/* With per-bucket locks this operation is not-atomic, so that
Index: linux-2.6.16-rc2/net/core/sock.c
===================================================================
--- linux-2.6.16-rc2.orig/net/core/sock.c 2006-02-09 15:01:47.000000000 -0800
+++ linux-2.6.16-rc2/net/core/sock.c 2006-02-09 15:01:52.000000000 -0800
@@ -1508,12 +1508,21 @@ int proto_register(struct proto *prot, i
}
}
+ prot->inuse = alloc_percpu(int);
+ if (prot->inuse == NULL) {
+ if (alloc_slab)
+ goto out_free_timewait_sock_slab_name_cache;
+ else
+ goto out;
+ }
write_lock(&proto_list_lock);
list_add(&prot->node, &proto_list);
write_unlock(&proto_list_lock);
rc = 0;
out:
return rc;
+out_free_timewait_sock_slab_name_cache:
+ kmem_cache_destroy(prot->twsk_prot->twsk_slab);
out_free_timewait_sock_slab_name:
kfree(timewait_sock_slab_name);
out_free_request_sock_slab:
@@ -1537,6 +1546,11 @@ void proto_unregister(struct proto *prot
list_del(&prot->node);
write_unlock(&proto_list_lock);
+ if (prot->inuse != NULL) {
+ free_percpu(prot->inuse);
+ prot->inuse = NULL;
+ }
+
if (prot->slab != NULL) {
kmem_cache_destroy(prot->slab);
prot->slab = NULL;
Index: linux-2.6.16-rc2/net/ipv4/proc.c
===================================================================
--- linux-2.6.16-rc2.orig/net/ipv4/proc.c 2006-02-09 15:01:47.000000000 -0800
+++ linux-2.6.16-rc2/net/ipv4/proc.c 2006-02-09 15:01:52.000000000 -0800
@@ -50,7 +50,7 @@ static int fold_prot_inuse(struct proto
int cpu;
for_each_cpu(cpu)
- res += proto->stats[cpu].inuse;
+ res += (*per_cpu_ptr(proto->inuse, cpu));
return res;
}
Index: linux-2.6.16-rc2/net/ipv4/raw.c
===================================================================
--- linux-2.6.16-rc2.orig/net/ipv4/raw.c 2006-02-07 23:14:04.000000000 -0800
+++ linux-2.6.16-rc2/net/ipv4/raw.c 2006-02-09 15:01:52.000000000 -0800
@@ -718,7 +718,7 @@ static int raw_ioctl(struct sock *sk, in
}
}
-struct proto raw_prot = {
+struct proto raw_prot __read_mostly = {
.name = "RAW",
.owner = THIS_MODULE,
.close = raw_close,
Index: linux-2.6.16-rc2/net/ipv4/tcp_ipv4.c
===================================================================
--- linux-2.6.16-rc2.orig/net/ipv4/tcp_ipv4.c 2006-02-09 15:01:47.000000000 -0800
+++ linux-2.6.16-rc2/net/ipv4/tcp_ipv4.c 2006-02-09 15:01:52.000000000 -0800
@@ -1794,7 +1794,7 @@ void tcp4_proc_exit(void)
}
#endif /* CONFIG_PROC_FS */
-struct proto tcp_prot = {
+struct proto tcp_prot __read_mostly = {
.name = "TCP",
.owner = THIS_MODULE,
.close = tcp_close,
Index: linux-2.6.16-rc2/net/ipv4/udp.c
===================================================================
--- linux-2.6.16-rc2.orig/net/ipv4/udp.c 2006-02-07 23:14:04.000000000 -0800
+++ linux-2.6.16-rc2/net/ipv4/udp.c 2006-02-09 15:01:52.000000000 -0800
@@ -1340,7 +1340,7 @@ unsigned int udp_poll(struct file *file,
}
-struct proto udp_prot = {
+struct proto udp_prot __read_mostly = {
.name = "UDP",
.owner = THIS_MODULE,
.close = udp_close,
Index: linux-2.6.16-rc2/net/ipv6/proc.c
===================================================================
--- linux-2.6.16-rc2.orig/net/ipv6/proc.c 2006-02-07 23:19:10.000000000 -0800
+++ linux-2.6.16-rc2/net/ipv6/proc.c 2006-02-09 15:01:52.000000000 -0800
@@ -39,7 +39,7 @@ static int fold_prot_inuse(struct proto
int cpu;
for_each_cpu(cpu)
- res += proto->stats[cpu].inuse;
+ res += (*per_cpu_ptr(proto->inuse, cpu));
return res;
}
^ permalink raw reply
* Re: [patch 1/4] net: percpufy frequently used vars -- add percpu_counter_mod_bh
From: Andrew Morton @ 2006-03-08 2:13 UTC (permalink / raw)
To: Ravikiran G Thirumalai; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308015934.GB9062@localhost.localdomain>
Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
>
> +static inline void percpu_counter_mod_bh(struct percpu_counter *fbc, long amount)
> +{
> + local_bh_disable();
> + percpu_counter_mod(fbc, amount);
> + local_bh_enable();
> +}
> +
percpu_counter_mod() does preempt_disable(), which is redundant in this
context. So just do fbc->count += amount; here.
^ permalink raw reply
* Re: [patch 2/4] net: percpufy frequently used vars -- struct proto.memory_allocated
From: Andrew Morton @ 2006-03-08 2:14 UTC (permalink / raw)
To: Ravikiran G Thirumalai; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308020118.GC9062@localhost.localdomain>
Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
>
> - if (atomic_read(sk->sk_prot->memory_allocated) < sk->sk_prot->sysctl_mem[0]) {
> + if (percpu_counter_read(sk->sk_prot->memory_allocated) <
> + sk->sk_prot->sysctl_mem[0]) {
Bear in mind that percpu_counter_read[_positive] can be inaccurate on large
CPU counts.
It might be worth running percpu_counter_sum() to get the exact count if we
think we're about to cause something to fail.
^ permalink raw reply
* Re: [patch 3/4] net: percpufy frequently used vars -- proto.sockets_allocated
From: Andrew Morton @ 2006-03-08 2:16 UTC (permalink / raw)
To: Ravikiran G Thirumalai; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308020227.GD9062@localhost.localdomain>
Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
>
> --- linux-2.6.16-rc5mm3.orig/include/net/sock.h 2006-03-07 15:09:22.000000000 -0800
> +++ linux-2.6.16-rc5mm3/include/net/sock.h 2006-03-07 15:09:52.000000000 -0800
> @@ -543,7 +543,7 @@ struct proto {
> /* Memory pressure */
> void (*enter_memory_pressure)(void);
> struct percpu_counter *memory_allocated; /* Current allocated memory. */
> - atomic_t *sockets_allocated; /* Current number of sockets. */
> + int *sockets_allocated; /* Current number of sockets (percpu counter). */
>
> /*
> * Pressure flag: try to collapse.
> @@ -579,6 +579,24 @@ struct proto {
> } stats[NR_CPUS];
> };
>
> +static inline int read_sockets_allocated(struct proto *prot)
> +{
> + int total = 0;
> + int cpu;
> + for_each_cpu(cpu)
> + total += *per_cpu_ptr(prot->sockets_allocated, cpu);
> + return total;
> +}
This is likely too big to be inlined, plus sock.h doesn't include enough
headers to reliably compile this code.
> +static inline void mod_sockets_allocated(int *sockets_allocated, int count)
> +{
> + (*per_cpu_ptr(sockets_allocated, get_cpu())) += count;
> + put_cpu();
> +}
> +
Ditto.
^ permalink raw reply
* Re: [patch 2/4] net: percpufy frequently used vars -- struct proto.memory_allocated
From: Ravikiran G Thirumalai @ 2006-03-08 3:08 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060307181422.3e279ca1.akpm@osdl.org>
On Tue, Mar 07, 2006 at 06:14:22PM -0800, Andrew Morton wrote:
> Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
> >
> > - if (atomic_read(sk->sk_prot->memory_allocated) < sk->sk_prot->sysctl_mem[0]) {
> > + if (percpu_counter_read(sk->sk_prot->memory_allocated) <
> > + sk->sk_prot->sysctl_mem[0]) {
>
> Bear in mind that percpu_counter_read[_positive] can be inaccurate on large
> CPU counts.
>
> It might be worth running percpu_counter_sum() to get the exact count if we
> think we're about to cause something to fail.
The problem is percpu_counter_sum has to read all the cpus cachelines. If
we have to use percpu_counter_sum everywhere, then might as well use plain
per-cpu counters instead of batching ones no?
sysctl_mem[0] is about 196K and on a 16 cpu box variance is 512 bytes, which
is OK with just percpu_counter_read I hope. Maybe, on very large cpu counts,
we should just change the FBC_BATCH so that variance does not go quadratic.
Something like 32. So that variance is 32 * NR_CPUS in that case, instead
of (NR_CPUS * NR_CPUS * 2) currently. Comments?
^ permalink raw reply
* Re: de2104x: interrupts before interrupt handler is registered
From: Martin Michlmayr @ 2006-03-08 3:22 UTC (permalink / raw)
To: Francois Romieu; +Cc: netdev, linux-kernel
In-Reply-To: <20060308001556.GA9362@electric-eye.fr.zoreil.com>
* Francois Romieu <romieu@fr.zoreil.com> [2006-03-08 01:15]:
> netdev watchdog events appear in the dmesg of the patched driver.
> The driver survived it. So I'd say that the patch does its job.
>
> OTOH, if you ever saw the unpatched driver survive this event, yell
> now.
No, I've never seen the unpatched driver survive.
--
Martin Michlmayr
http://www.cyrius.com/
^ permalink raw reply
* Re: [patch 2/4] net: percpufy frequently used vars -- struct proto.memory_allocated
From: Andrew Morton @ 2006-03-08 3:22 UTC (permalink / raw)
To: Ravikiran G Thirumalai; +Cc: linux-kernel, davem, netdev, shai
In-Reply-To: <20060308030803.GF9062@localhost.localdomain>
Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
>
> On Tue, Mar 07, 2006 at 06:14:22PM -0800, Andrew Morton wrote:
> > Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
> > >
> > > - if (atomic_read(sk->sk_prot->memory_allocated) < sk->sk_prot->sysctl_mem[0]) {
> > > + if (percpu_counter_read(sk->sk_prot->memory_allocated) <
> > > + sk->sk_prot->sysctl_mem[0]) {
> >
> > Bear in mind that percpu_counter_read[_positive] can be inaccurate on large
> > CPU counts.
> >
> > It might be worth running percpu_counter_sum() to get the exact count if we
> > think we're about to cause something to fail.
>
> The problem is percpu_counter_sum has to read all the cpus cachelines. If
> we have to use percpu_counter_sum everywhere, then might as well use plain
> per-cpu counters instead of batching ones no?
I didn't say "use it everywhere" ;)
Just in places like this:
if (percpu_counter_read(something) > something_else)
make_an_application_fail();
in that case it's worth running percpu_counter_sum(). And bear in mind
that once we've done that, the following percpu_counter_read()s become
accurate, so we won't run the expensive percpu_counter_sum() again
for a while. Unless we're really close to or over the limit, in which case
blowing a few cycles is relatively unimportant.
All that should be captured in library code (per_cpu_counter_exceeds(ptr,
threshold), for example) rather than open-coded everywhere.
> sysctl_mem[0] is about 196K and on a 16 cpu box variance is 512 bytes, which
> is OK with just percpu_counter_read I hope.
You mean a 16 CPU box with NR_CPUS=16 as well...
> Maybe, on very large cpu counts,
> we should just change the FBC_BATCH so that variance does not go quadratic.
> Something like 32. So that variance is 32 * NR_CPUS in that case, instead
> of (NR_CPUS * NR_CPUS * 2) currently. Comments?
Sure, we need to make that happen. But it got all mixed up with the
spinlock removal and it does need quite some thought and testing and
documentation to help developers to choose the right settings and
appropriate selection of defaults, etc.
^ permalink raw reply
* [2.6 patch] drivers/net/wireless/ipw2200.c: make ipw_qos_current_mode() static
From: Adrian Bunk @ 2006-03-08 12:18 UTC (permalink / raw)
To: Andrew Morton; +Cc: yi.zhu, jketreno, linux-kernel, linville, netdev
This patch makes the needlessly global function ipw_qos_current_mode()
static.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
---
This patch was already sent on:
- 4 Mar 2006
--- linux-2.6.16-rc5-mm2-full/drivers/net/wireless/ipw2200.c.old 2006-03-03 17:49:37.000000000 +0100
+++ linux-2.6.16-rc5-mm2-full/drivers/net/wireless/ipw2200.c 2006-03-03 17:50:00.000000000 +0100
@@ -6566,7 +6566,7 @@
* get the modulation type of the current network or
* the card current mode
*/
-u8 ipw_qos_current_mode(struct ipw_priv * priv)
+static u8 ipw_qos_current_mode(struct ipw_priv * priv)
{
u8 mode = 0;
^ permalink raw reply
* Re: Re: TSO and IPoIB performance degradation
From: Michael S. Tsirkin @ 2006-03-08 12:53 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, rdreier, linux-kernel, openib-general, shemminger
In-Reply-To: <20060307.172336.107863253.davem@davemloft.net>
Quoting r. David S. Miller <davem@davemloft.net>:
> Subject: Re: Re: TSO and IPoIB performance degradation
>
> From: Roland Dreier <rdreier@cisco.com>
> Date: Tue, 07 Mar 2006 17:17:30 -0800
>
> > The reason TSO comes up is that reverting the patch described below
> > helps (or helped at some point at least) IPoIB throughput quite a bit.
>
> I wish you had started the thread by mentioning this specific patch
Er, since you mention it, the first message in thread did include this link:
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=314324121f9b94b2ca657a494cf2b9cb0e4a28cc
and I even pasted the patch description there, but oh well.
Now that Roland helped us clear it all up, and now that it has been clarified
that reverting this patch gives us back most of the performance, is the answer
to my question the same?
What I was trying to figure out was, how can we re-enable the trick without
hurting TSO? Could a solution be to simply look at the frame size, and call
tcp_send_delayed_ack if the frame size is small?
--
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies
^ permalink raw reply
* [PATCH][RFC] Notifier mechanism to notify RDMA devices of network events
From: Tom Tucker @ 2006-03-08 16:38 UTC (permalink / raw)
To: netdev, linux-kernel, openib-general
This patch implements a mechanism that allows interested clients to
register for notification of certain network events. The intended use
is to allow RDMA(OpenIB) devices to be notified of neighbour updates,
ICMP redirects, path MTU changes, and route changes.
The reason these devices need update events is because they typically
cache this information in hardware and need to be notified when
this information has been updated.
This approach is one of many possibilities and may be preferred because
it uses an existing notification mechanism that has precedent in the
stack.
This code does not yet implement path MTU change because the number of
places in which this value is updated is large and if this mechanism seems
reasonable, it would be probably be best to funnel these updates through a
single function.
diff -u -r -x '.*' --new-file linux-2.6.14.5/include/net/netevent.h linux-2.6.14.5.tom/include/net/netevent.h
--- linux-2.6.14.5/include/net/netevent.h 1969-12-31 18:00:00.000000000 -0600
+++ linux-2.6.14.5.tom/include/net/netevent.h 2006-01-16 13:42:09.000000000 -0600
@@ -0,0 +1,41 @@
+#ifndef _NET_EVENT_H
+#define _NET_EVENT_H
+
+/*
+ * Generic netevent notifiers
+ *
+ * Authors:
+ * Tom Tucker <tom@opengridcomputing.com>
+ *
+ * Changes:
+ */
+
+#ifdef __KERNEL__
+
+#include <net/dst.h>
+
+struct netevent_redirect {
+ struct dst_entry *old;
+ struct dst_entry *new;
+};
+
+struct netevent_route_change {
+ int event;
+ struct fib_info *fib_info;
+};
+
+enum netevent_notif_type {
+ NETEVENT_NEIGH_UPDATE = 1, /* arg is * struct neighbour */
+ NETEVENT_ROUTE_UPDATE, /* arg is * netevent_route_change */
+ NETEVENT_PMTU_UPDATE,
+ NETEVENT_REDIRECT, /* arg is * struct netevent_redirect */
+};
+
+extern int register_netevent_notifier(struct notifier_block *nb);
+extern int unregister_netevent_notifier(struct notifier_block *nb);
+extern int call_netevent_notifiers(unsigned long val, void *v);
+
+#endif
+#endif
+
+
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/core/Makefile linux-2.6.14.5.tom/net/core/Makefile
--- linux-2.6.14.5/net/core/Makefile 2005-12-26 18:26:33.000000000 -0600
+++ linux-2.6.14.5.tom/net/core/Makefile 2006-01-16 13:41:42.000000000 -0600
@@ -7,7 +7,7 @@
obj-$(CONFIG_SYSCTL) += sysctl_net_core.o
-obj-y += dev.o ethtool.o dev_mcast.o dst.o \
+obj-y += dev.o ethtool.o dev_mcast.o dst.o netevent.o \
neighbour.o rtnetlink.o utils.o link_watch.o filter.o
obj-$(CONFIG_XFRM) += flow.o
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/core/neighbour.c linux-2.6.14.5.tom/net/core/neighbour.c
--- linux-2.6.14.5/net/core/neighbour.c 2005-12-26 18:26:33.000000000 -0600
+++ linux-2.6.14.5.tom/net/core/neighbour.c 2006-01-16 13:41:42.000000000 -0600
@@ -30,9 +30,11 @@
#include <net/neighbour.h>
#include <net/dst.h>
#include <net/sock.h>
+#include <net/netevent.h>
#include <linux/rtnetlink.h>
#include <linux/random.h>
#include <linux/string.h>
+#include <linux/notifier.h>
#define NEIGH_DEBUG 1
@@ -756,6 +758,7 @@
NEIGH_PRINTK2("neigh %p is suspected.\n", neigh);
neigh->nud_state = NUD_STALE;
neigh_suspect(neigh);
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
}
} else if (state & NUD_DELAY) {
if (time_before_eq(now,
@@ -763,6 +766,7 @@
NEIGH_PRINTK2("neigh %p is now reachable.\n", neigh);
neigh->nud_state = NUD_REACHABLE;
neigh_connect(neigh);
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
next = neigh->confirmed + neigh->parms->reachable_time;
} else {
NEIGH_PRINTK2("neigh %p is probed.\n", neigh);
@@ -781,6 +785,7 @@
neigh->nud_state = NUD_FAILED;
notify = 1;
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
NEIGH_CACHE_STAT_INC(neigh->tbl, res_failed);
NEIGH_PRINTK2("neigh %p is failed.\n", neigh);
@@ -1051,6 +1056,9 @@
(neigh->flags | NTF_ROUTER) :
(neigh->flags & ~NTF_ROUTER);
}
+
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
+
write_unlock_bh(&neigh->lock);
#ifdef CONFIG_ARPD
if (notify && neigh->parms->app_probes)
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/core/netevent.c linux-2.6.14.5.tom/net/core/netevent.c
--- linux-2.6.14.5/net/core/netevent.c 1969-12-31 18:00:00.000000000 -0600
+++ linux-2.6.14.5.tom/net/core/netevent.c 2006-01-16 13:50:25.000000000 -0600
@@ -0,0 +1,69 @@
+/*
+ * Network event notifiers
+ *
+ * Authors:
+ * Tom Tucker <tom@opengridcomputing.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ * Fixes:
+ */
+
+#include <linux/rtnetlink.h>
+#include <linux/notifier.h>
+
+static struct notifier_block *netevent_notif_chain;
+
+/**
+ * register_netevent_notifier - register a netevent notifier block
+ * @nb: notifier
+ *
+ * Register a notifier to be called when a netevent occurs.
+ * The notifier passed is linked into the kernel structures and must
+ * not be reused until it has been unregistered. A negative errno code
+ * is returned on a failure.
+ */
+int register_netevent_notifier(struct notifier_block *nb)
+{
+ int err;
+
+ rtnl_lock();
+ err = notifier_chain_register(&netevent_notif_chain, nb);
+ rtnl_unlock();
+ return err;
+}
+
+/**
+ * netevent_unregister_notifier - unregister a netevent notifier block
+ * @nb: notifier
+ *
+ * Unregister a notifier previously registered by
+ * register_neigh_notifier(). The notifier is unlinked into the
+ * kernel structures and may then be reused. A negative errno code
+ * is returned on a failure.
+ */
+
+int unregister_netevent_notifier(struct notifier_block *nb)
+{
+ return notifier_chain_unregister(&netevent_notif_chain, nb);
+}
+
+/**
+ * call_netevent_notifiers - call all netevent notifier blocks
+ * @val: value passed unmodified to notifier function
+ * @v: pointer passed unmodified to notifier function
+ *
+ * Call all neighbour notifier blocks. Parameters and return value
+ * are as for notifier_call_chain().
+ */
+
+int call_netevent_notifiers(unsigned long val, void *v)
+{
+ return notifier_call_chain(&netevent_notif_chain, val, v);
+}
+
+EXPORT_SYMBOL(register_netevent_notifier);
+EXPORT_SYMBOL(unregister_netevent_notifier);
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/ipv4/fib_semantics.c linux-2.6.14.5.tom/net/ipv4/fib_semantics.c
--- linux-2.6.14.5/net/ipv4/fib_semantics.c 2005-12-26 18:26:33.000000000 -0600
+++ linux-2.6.14.5.tom/net/ipv4/fib_semantics.c 2006-01-16 13:41:42.000000000 -0600
@@ -43,6 +43,7 @@
#include <net/sock.h>
#include <net/ip_fib.h>
#include <net/ip_mp_alg.h>
+#include <net/netevent.h>
#include "fib_lookup.h"
@@ -276,9 +277,15 @@
struct nlmsghdr *n, struct netlink_skb_parms *req)
{
struct sk_buff *skb;
+ struct netevent_route_change rev;
+
u32 pid = req ? req->pid : n->nlmsg_pid;
int size = NLMSG_SPACE(sizeof(struct rtmsg)+256);
+ rev.event = event;
+ rev.fib_info = fa->fa_info;
+ call_netevent_notifiers(NETEVENT_ROUTE_UPDATE, &rev);
+
skb = alloc_skb(size, GFP_KERNEL);
if (!skb)
return;
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/ipv4/route.c linux-2.6.14.5.tom/net/ipv4/route.c
--- linux-2.6.14.5/net/ipv4/route.c 2005-12-26 18:26:33.000000000 -0600
+++ linux-2.6.14.5.tom/net/ipv4/route.c 2006-01-16 13:41:42.000000000 -0600
@@ -103,6 +103,7 @@
#include <net/icmp.h>
#include <net/xfrm.h>
#include <net/ip_mp_alg.h>
+#include <net/netevent.h>
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#endif
@@ -1118,6 +1119,7 @@
struct rtable *rth, **rthp;
u32 skeys[2] = { saddr, 0 };
int ikeys[2] = { dev->ifindex, 0 };
+ struct netevent_redirect netevent;
tos &= IPTOS_RT_MASK;
@@ -1213,6 +1215,10 @@
rt_drop(rt);
goto do_next;
}
+
+ netevent.old = &rth->u.dst;
+ netevent.new = &rt->u.dst;
+ call_netevent_notifiers(NETEVENT_REDIRECT, &netevent);
rt_del(hash, rth);
if (!rt_intern_hash(hash, rt, &rt))
^ permalink raw reply
* [PATCH] compat. ifconf: fix limits
From: Randy.Dunlap @ 2006-03-08 17:16 UTC (permalink / raw)
To: netdev, linux-fsdevel; +Cc: Alexandra.Kossovsky, ak, akpm, torvalds
From: Randy Dunlap <rdunlap@xenotime.net>
A recent change to compat. dev_ifconf() in fs/compat_ioctl.c
causes ifconf data to be truncated 1 entry too early when copying it
to userspace. The correct amount of data (length) is returned,
but the final entry is empty (zero, not filled in).
The for-loop 'i' check should use <= to allow the final struct
ifreq32 to be copied. I also used the ifconf-corruption program
in kernel bugzilla #4746 to make sure that this change does not
re-introduce the corruption.
Signed-off-by: Randy Dunlap <rdunlap@xenotime.net>
---
fs/compat_ioctl.c | 2 +-
1 files changed, 1 insertion(+), 1 deletion(-)
--- linux-2616-rc5.orig/fs/compat_ioctl.c
+++ linux-2616-rc5/fs/compat_ioctl.c
@@ -446,7 +446,7 @@ static int dev_ifconf(unsigned int fd, u
ifr = ifc.ifc_req;
ifr32 = compat_ptr(ifc32.ifcbuf);
for (i = 0, j = 0;
- i + sizeof (struct ifreq32) < ifc32.ifc_len && j < ifc.ifc_len;
+ i + sizeof (struct ifreq32) <= ifc32.ifc_len && j < ifc.ifc_len;
i += sizeof (struct ifreq32), j += sizeof (struct ifreq)) {
if (copy_in_user(ifr32, ifr, sizeof (struct ifreq32)))
return -EFAULT;
---
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox