[PATCH] tcp_mem initialization

[PATCH] tcp_mem initialization

[John Heffner]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

The current tcp_mem initialization gives values that are really too 
small for systems with ~256-768 MB of memory, and also for systems with 
larger page sizes (ia64).  This patch gives an alternate method of 
initialization that doesn't depend on the cache allocation functions, 
but I think should still provide a nice curve that gives a smaller 
fraction of total memory with small-memory systems, while maintaining 
the same upper bound (pressure at 1/2, max as 3/4) on larger memory systems.

   -John


[-- Attachment #2: tcp_mem_init.patch --]
[-- Type: text/plain, Size: 1644 bytes --]

Change tcp_mem initialization function.  The fraction of total memory is now
a continuous function of memory size, and independent of page size.

Signed-off-by: John Heffner <jheffner@psc.edu>

---
commit a4461a36efb376bf01399cfd6f1ad15dc89a8794
tree 23b2fb9da52b45de8008fc7ea6bb8c10e3a3724b
parent 8b9909ded6922c33c221b105b26917780cfa497d
author John Heffner <jheffner@psc.edu> Wed, 14 Mar 2007 17:15:06 -0400
committer John Heffner <jheffner@psc.edu> Wed, 14 Mar 2007 17:15:06 -0400

 net/ipv4/tcp.c |   13 ++++++++++---
 1 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 74c4d10..3834b10 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2458,11 +2458,18 @@ void __init tcp_init(void)
 		sysctl_max_syn_backlog = 128;
 	}
 
-	/* Allow no more than 3/4 kernel memory (usually less) allocated to TCP */
-	sysctl_tcp_mem[0] = (1536 / sizeof (struct inet_bind_hashbucket)) << order;
-	sysctl_tcp_mem[1] = sysctl_tcp_mem[0] * 4 / 3;
+	/* Set the pressure threshold to be a fraction of global memory that
+	 * is up to 1/2 at 256 MB, decreasing toward zero with the amount of
+	 * memory, with a floor of 128 pages.
+	 */
+	limit = min(nr_all_pages, 1UL<<(28-PAGE_SHIFT)) >> (20-PAGE_SHIFT);
+	limit = (limit * (nr_all_pages >> (20-PAGE_SHIFT))) >> (PAGE_SHIFT-11);
+	limit = max(limit, 128UL);
+	sysctl_tcp_mem[0] = limit / 4 * 3;
+	sysctl_tcp_mem[1] = limit;
 	sysctl_tcp_mem[2] = sysctl_tcp_mem[0] * 2;
 
+	/* Set per-socket limits to no more than 1/128 the pressure threshold */
 	limit = ((unsigned long)sysctl_tcp_mem[1]) << (PAGE_SHIFT - 7);
 	max_share = min(4UL*1024*1024, limit);
 

Re: [PATCH] tcp_mem initialization

[David Miller]

From: John Heffner <jheffner@psc.edu>
Date: Wed, 14 Mar 2007 17:25:22 -0400

> The current tcp_mem initialization gives values that are really too 
> small for systems with ~256-768 MB of memory, and also for systems with 
> larger page sizes (ia64).  This patch gives an alternate method of 
> initialization that doesn't depend on the cache allocation functions, 
> but I think should still provide a nice curve that gives a smaller 
> fraction of total memory with small-memory systems, while maintaining 
> the same upper bound (pressure at 1/2, max as 3/4) on larger memory systems.

Indeed, it's really dumb for any of these calculations to be
dependant upon the page size.

Your patch looks good, and I'll review it further tomorrow and
push upstream unless I find some issues with it.

Thanks John.

Re: [PATCH] tcp_mem initialization

[John Heffner]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

David Miller wrote:
> From: John Heffner <jheffner@psc.edu>
> Date: Wed, 14 Mar 2007 17:25:22 -0400
> 
>> The current tcp_mem initialization gives values that are really too 
>> small for systems with ~256-768 MB of memory, and also for systems with 
>> larger page sizes (ia64).  This patch gives an alternate method of 
>> initialization that doesn't depend on the cache allocation functions, 
>> but I think should still provide a nice curve that gives a smaller 
>> fraction of total memory with small-memory systems, while maintaining 
>> the same upper bound (pressure at 1/2, max as 3/4) on larger memory systems.
> 
> Indeed, it's really dumb for any of these calculations to be
> dependant upon the page size.
> 
> Your patch looks good, and I'll review it further tomorrow and
> push upstream unless I find some issues with it.
> 
> Thanks John.


The way it's coded is somewhat opaque since it has to be done with 
32-bit integer arithmetic.  These plots might help make the motivation 
behind the code a little clearer.

Thanks,
   -John

[-- Attachment #2: tcp_mem_pressure.png --]
[-- Type: image/png, Size: 9994 bytes --]

[-- Attachment #3: tcp_mem_frac.png --]
[-- Type: image/png, Size: 10756 bytes --]

Re: [PATCH] tcp_mem initialization

[David Miller]

From: John Heffner <jheffner@psc.edu>
Date: Thu, 15 Mar 2007 14:39:29 -0400

> The way it's coded is somewhat opaque since it has to be done with 
> 32-bit integer arithmetic.  These plots might help make the motivation 
> behind the code a little clearer.

Thanks John.

I was going to write a little test program to spit out
the values for a given range of memory sizes but your
graphs are much better :-)