ksoftirqd and network processing

[Anna Fischer <a.fischer@sirrix.com>]

Hi there,

I'd like to understand more in detail how ksoftirqd works in combination
with Linux kernel networking.

I'm doing IPSec processing on my Linux machine (kernel 4.4). On one
interface I receive encrypted packets which are decrypted and then
routed out plaintext (TCP) via another interface. Now I can
see that ksoftirqd runs at 100% CPU. Furthermore, I see that initially
it runs on core 0, then after a few seconds, it switches over to core 1
and runs at 100%, then switches over to core 2, and so on.

I have two questions:

1) I assume it switches cores because the kernel thread is scheduled on
different cores, depending on where the softirqs happen. Why does the
load never scale across cores? As far as I understand softirqs can run
concurrently on different cores. So why does it not scale?

2) Is it normal that ksoftirqd runs at such a high CPU load when doing
IPSec, if the core is a small core with small frequency (Intel Atoms,
for example)? And if so, is there any way to optimize this for IPSec?

3) When I pin RX queue interrupts of the NIC to certain cores, I can see
that interrupts are really only processed on that core. However, it does
not change anything about the ksoftirqd behavior. Should it not match,
how I route interrupts?

Many thanks,
Anna