Re: Network performance forwarding tests on RT

[Clark Williams <williams@redhat.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 11 Nov 2008 14:37:46 +0100
"Tobias Knutsson" <tobias.knutsson@gmail.com> wrote:

> Ok well that assumption is probably not right. I am quite sure that both the
> soft and hard irqs are assigned priority 50. But you probably should raise
> the priority of the irqs related to your implementation.
> 
> I am writing my master's thesis on real-time Linux and I found that the
> worst-case latency dropped from about 2000us to 250us just by changing some
> priorities of the relevant tasks. This was on an XScale based system running
> at 520Mhz.
>

Yeah, they default to 50 and then most RT distros adjust them from there. We generally push the hard IRQ threads up to 85 and keep the soft irq threads down around 75, on the assumption that actual IRQs need to be serviced immediately, with the hope that the service routine runs quicky and then defers further work to a lower priority entity (tasklet, softirq, workqueue, etc.). 

If we're tuning a system for a particular workload and we're on a multi-core box, we'll push all interrupts and non-workload related threads over to cpu0, then distribute the application and interrupt threads across the remaining cpus. Sometimes overkill in terms of dedicating a 2GHz core to handling network interrupts, but if you're tuning for maximum performance for your app, who cares?

Stephen, it might be interesting for you to change the hard IRQ thread for your NIC(s) to be something higher than 50, with the the soft irq threads for networking (softirq-net-{tx,rx}) to be just under that and see what that does to your test. Maybe 55 for hard irqs and 54 for soft irqs?

Clark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkkZkwUACgkQHyuj/+TTEp0rmgCgup9bbrmP23TKcD0DfhjR5v75
qzoAn1xG75DuZAvOEVoB1DyOLUk6PWVX
=AU+y
-----END PGP SIGNATURE-----