Re: [RFC] bnx2x: Insane RX rings

[Rick Jones <rick.jones2@hp.com>]

David Miller wrote:
> From: Krzysztof Olędzki <ole@ans.pl>
> Date: Thu, 09 Sep 2010 23:21:01 +0200
> 
> 
>>On 2010-09-09 22:45, Eric Dumazet wrote:
>>
>>>Problem is : With 16 RX queues per device , thats 4078*16*2Kbytes per
>>>ethernet port.
>>>
>>>Total :
>>>
>>>skbuff_head_cache 130747 131025 256 15 1 : tunables 120 60 8 :
>>>slabdata 8735 8735 40
>>>size-2048 130866 130888 2048 2 1 : tunables 24 12 8 : slabdata 65444
>>>65444 28
>>>
>>>Thats about 300 Mbytes of memory, just in case some network trafic
>>>will occur.
>>>
>>>Lets do something about that ?
>>
>>Yep, it is ~8MB per queue, not so much alone, but a lot together. For
>>this reason I use something like bnx2.num_queues=2 on servers where I
>>don't need much CPU power for network workload.
> 
> 
> I think simply that the RX queue size should be scaled by the number
> of queues we have.
> 
> If people want enormous RX ring sizes even when there are many queues,
> they can use ethtool to get that.
> 
> Taking up 130MB of memory per-card, just for RX packet buffers, is
> certainly over the top.

It gets even better if one consideres JumboFrames...  that said, I've had 
customer contacts (indirect) where they were quite keep to have a ring size of 
at least 2048 packets - I never could get it confirmed, but I suspect they had 
applications/systems that might "go out to lunch" for long-enough periods of 
time they wanted that degree of FIFO.

Doesn't necessarily change "what should be the defaults" much but there it is.

rick jones