[LARTC] Which CPU for heavy traffic with much filtering/shaping?

[LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Derek Sims]

Hi

I have a router with a large number of iptables rules and some extensive 
traffic shaping (HTB + RED + ... ) + conntrack.

The router is running Centos5 on a P4 Celeron 2.4 with 512Mb ram


30% soft interrupt cpu utilisation
7000 packets/second on each of eth1 and eth0 (forwarded packets)
20Mbit/second on both eth1 and eth0
e1000 ethernet on both eth0 and eth1 (eth1 running at 100Mbit)

I am trying to optimise the firewall rules and have already managed to 
reduce cpu si by about 40% however I need to get this router to handle a 
throughput rate of 100Mbit or more.

I have seen hints that using SMP (or multicore) processors will not help 
for soft interrupt. My question is this:

1. What processors should I be looking for in order to achieve the best 
routing throughput on a linux router?

2. Is it true that multicore processors will not help much in this 
situation?

Best regards,
Derek
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Marek Kierdelewicz]

>Hi

Hi

>I have a router with a large number of iptables rules and some
>extensive traffic shaping (HTB + RED + ... ) + conntrack.

Performance boost tips:

- Use "set" module instead of sequential iptables rules. It can lower
cpu usage.

- Use hashing filters for shaping if you're using many u32 filters.

- configure conntrack to use bigger hashsize for better performance;
i'm passing following parameter to kernel in grub to achieve this:
ip_conntrack.hashsize\x1048575 

- configure routecache to use bigger to use more memory for better
performance; i'm passing following parameter to kernel in grub to
achieve this: rhash_entries$00000

>1. What processors should I be looking for in order to achieve the
>best routing throughput on a linux router?

I've had good experiences with P4 (with and without HT), Athlon64, Xeon
[dempsey], Xeon [woodcrest]. The last one is the best choice because of
the large cache and architecture. I think you can use Core 2 Duo too
if you want to save some money.

>2. Is it true that multicore processors will not help much in this 
>situation?

Not true. In your setup with two nics with same load you can easily use
two cores. You can assign each nic to different core by the means of
smp_affinity setting in /proc/irq/... or by using irqbalance daemon.

>Best regards,
>Derek

pozdrawiam
Marek Kierdelewicz
KoBa ISP
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Derek Sims]

Marek Kierdelewicz wrote:
>> Hi
>>     
>
> Hi
>
>   
>> I have a router with a large number of iptables rules and some
>> extensive traffic shaping (HTB + RED + ... ) + conntrack.
>>     
>
> Performance boost tips:
>
> - Use "set" module instead of sequential iptables rules. It can lower
> cpu usage.
>   

Hmm - I don't know what the "set" module is - can you point me to some 
documentation please?
> - Use hashing filters for shaping if you're using many u32 filters.
>   
Only 3
> - configure conntrack to use bigger hashsize for better performance;
> i'm passing following parameter to kernel in grub to achieve this:
> ip_conntrack.hashsize\x1048575 
>
>   
I have 64k in conntrack_max and hashsize of 16000
Currently running with about 20000 conntrack connections

I will try increasing this
> - configure routecache to use bigger to use more memory for better
> performance; i'm passing following parameter to kernel in grub to
> achieve this: rhash_entries$00000
>
>   
>> 1. What processors should I be looking for in order to achieve the
>> best routing throughput on a linux router?
>>     
>
> I've had good experiences with P4 (with and without HT), Athlon64, Xeon
> [dempsey], Xeon [woodcrest]. The last one is the best choice because of
> the large cache and architecture. I think you can use Core 2 Duo too
> if you want to save some money.
>
>   
Thanks - I will see what I can get
>> 2. Is it true that multicore processors will not help much in this 
>> situation?
>>     
>
> Not true. In your setup with two nics with same load you can easily use
> two cores. You can assign each nic to different core by the means of
> smp_affinity setting in /proc/irq/... or by using irqbalance daemon.
>
>   
That is good news :) - however I guess 4 core with dual ethernet would 
not help very much!
>> Best regards,
>> Derek
>>     
>
> pozdrawiam
> Marek Kierdelewicz
> KoBa ISP
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>   
Best regards,
Derek
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[sawar]

Hi

is there any how-to which can guide me through all available tuning options 
in /proc/ filesystem 

Pozdrawiam
Szymon Turkiewicz
> >Hi
>
> Hi
>
> >I have a router with a large number of iptables rules and some
> >extensive traffic shaping (HTB + RED + ... ) + conntrack.
>
> Performance boost tips:
>
> - Use "set" module instead of sequential iptables rules. It can lower
> cpu usage.
>
> - Use hashing filters for shaping if you're using many u32 filters.
>
> - configure conntrack to use bigger hashsize for better performance;
> i'm passing following parameter to kernel in grub to achieve this:
> ip_conntrack.hashsize\x1048575
>
> - configure routecache to use bigger to use more memory for better
> performance; i'm passing following parameter to kernel in grub to
> achieve this: rhash_entries$00000
>
> >1. What processors should I be looking for in order to achieve the
> >best routing throughput on a linux router?
>
> I've had good experiences with P4 (with and without HT), Athlon64, Xeon
> [dempsey], Xeon [woodcrest]. The last one is the best choice because of
> the large cache and architecture. I think you can use Core 2 Duo too
> if you want to save some money.
>
> >2. Is it true that multicore processors will not help much in this
> >situation?
>
> Not true. In your setup with two nics with same load you can easily use
> two cores. You can assign each nic to different core by the means of
> smp_affinity setting in /proc/irq/... or by using irqbalance daemon.
>
> >Best regards,
> >Derek
>
> pozdrawiam
> Marek Kierdelewicz
> KoBa ISP
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Marek Kierdelewicz]

>Hi

Hi

>is there any how-to which can guide me through all available tuning
>options in /proc/ filesystem 

Proc filesystem is described in file Documentation/filesystems/proc.txt
in the linux kernel sources. You can find there something about
smp_affinity and linux network stack parameters (and about many more
things). No info about netfilter-related setting is supplied in that
document.

As for guide... Lartc howto: lartc. org / howto / lartc . kernel . html
and many more available throu the google search.

pozdrawiam
Marek Kierdelewicz
KoBa ISP
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Mohan Sundaram]

Derek Sims wrote:
> Marek Kierdelewicz wrote:
> 
> Hmm - I don't know what the "set" module is - can you point me to some 
> documentation please?
Search for ipset extensions for iptables or look up extension projects 
in netfilter.org.

ipset gives the facility to create sets of IPs and use the sets in 
iptables rules. Makes the rules more orderly, easy to read, easy to 
manage and is easier on the CPU.

Mohan
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Mohan Sundaram]

Derek Sims wrote:
> Marek Kierdelewicz wrote
>>> 1. What processors should I be looking for in order to achieve the
>>> best routing throughput on a linux router?
>>>     
>> I've had good experiences with P4 (with and without HT), Athlon64, Xeon
>> [dempsey], Xeon [woodcrest]. The last one is the best choice because of
>> the large cache and architecture. I think you can use Core 2 Duo too
>> if you want to save some money.
>>
>>   
> Thanks - I will see what I can get
I used AMD Opteron 2Ghz and it blazes. In my packet switching 
benchmarks, onchip cache gave no benefit. On price-performance basis, 
you'd be better off going for greater CPU speed than for cache.

Mohan
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Which CPU for heavy traffic with much filtering/shaping?

[Marco C. Coelho]

I like multicore / multicpu amd opteron boards from Tyan.  With a dual 
core (later you can double to 4 core) dual cpu motherboard, you can 
route and shape to your hearts content.  To really improve throughput, 
you should use a higher end network controller with pci-express with 
some smarts and multiple ports if you need it.

Hope that helps.

Mohan Sundaram wrote:
> Derek Sims wrote:
>> Marek Kierdelewicz wrote
>>>> 1. What processors should I be looking for in order to achieve the
>>>> best routing throughput on a linux router?
>>>>     
>>> I've had good experiences with P4 (with and without HT), Athlon64, Xeon
>>> [dempsey], Xeon [woodcrest]. The last one is the best choice because of
>>> the large cache and architecture. I think you can use Core 2 Duo too
>>> if you want to save some money.
>>>
>>>   
>> Thanks - I will see what I can get
> I used AMD Opteron 2Ghz and it blazes. In my packet switching 
> benchmarks, onchip cache gave no benefit. On price-performance basis, 
> you'd be better off going for greater CPU speed than for cache.
>
> Mohan
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc