sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

[Robin Holt]

On a 16TB system, we noticed that sysctl_tcp_mem[2] and sysctl_udp_mem[2]
were negative.  Code review indicates that the same should occur with
sysctl_sctp_mem[2].

There are a couple ways we could address this.  The one which appears most
reasonable would be to change the struct proto defintion for sysctl_mem
from an int to a long and handle all the associated fallout.

An alternative is to limit the calculation to 1/2 INT_MAX.  The downside
being that the administrator could not tune the system to use more than
INT_MAX memory when much more is available.

Is there a compelling reason to not change the structure's definition
over to longs instead of ints and deal with the fallout from that change?

Thanks,
Robin Holt

Re: sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

[Willy Tarreau]

Hello Robin,

On Fri, Oct 01, 2010 at 02:39:58PM -0500, Robin Holt wrote:
> 
> On a 16TB system, we noticed that sysctl_tcp_mem[2] and sysctl_udp_mem[2]
> were negative.  Code review indicates that the same should occur with
> sysctl_sctp_mem[2].
> 
> There are a couple ways we could address this.  The one which appears most
> reasonable would be to change the struct proto defintion for sysctl_mem
> from an int to a long and handle all the associated fallout.
> 
> An alternative is to limit the calculation to 1/2 INT_MAX.  The downside
> being that the administrator could not tune the system to use more than
> INT_MAX memory when much more is available.
> 
> Is there a compelling reason to not change the structure's definition
> over to longs instead of ints and deal with the fallout from that change?

Could we not see it differently ? => is there any reason someone would
want to assign more than 8 TB of RAM to the network buffers in the near
future ? Even at 100 Gbps, that's still 10 minutes of traffic stuck in
buffers. Probably that the day we need that large buffers, Linux won't
support 32-bit systems anymore and all such limits will have switched
to 64-bit.

So probably that limiting the value to INT_MAX/2 sounds reasonable ?

Regards,
Willy

Re: sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

[Maciej Żenczykowski]

Isn't INT_MAX/2 just 1GB, which is only ~0.9 seconds at 10 Gbps?

Re: sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

[Robin Holt]

On Sun, Oct 03, 2010 at 01:20:32AM -0700, Maciej Żenczykowski wrote:
> Isn't INT_MAX/2 just 1GB, which is only ~0.9 seconds at 10 Gbps?

Units matter.  1GB pages.  We can limit to 2GB pages or 8TB.

Robin

Re: sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

[Willy Tarreau]

On Sun, Oct 03, 2010 at 01:20:32AM -0700, Maciej ??enczykowski wrote:
> Isn't INT_MAX/2 just 1GB, which is only ~0.9 seconds at 10 Gbps?

no, the size is in pages (and is often wrong when people change it
from the default value).

Regards,
Willy