localhost network processing on smp machines

localhost network processing on smp machines

[Matthew Faulkner]

I'm probably being stupid and very confused here. Appologies if it is
a stupid question.

For a particular test I assign a client to core 1 and a server to core
0. My first assumption was all the sending side kernel TCP/IP
processing will be done on core 1 and all the receiving side
processing on core 0. However, having looked at the linux TCP/IP code,
more specifically the loopback device code, when the loopbacks
dev_queue_xmit function gets called it simply passes the skb to the
netif_rx function. Therefore my question is thus, when i assign the
sending / receiving side to different cores is the actual TCP/IP
processing done the assigned sending / receiving cores ? or is TCP/IP
processing done on the sending side only until a certain point? or is
it simply a single core that does all the TCP/IP processing. If it's
the first two and you have any idea, could you point me where in the
kernel the cores the processing swap over.

Thanks for your help

Re: localhost network processing on smp machines

[Andi Kleen]

"Matthew Faulkner" <matthew.faulkner@gmail.com> writes:

> I'm probably being stupid and very confused here. Appologies if it is
> a stupid question.
>
> For a particular test I assign a client to core 1 and a server to core
> 0. My first assumption was all the sending side kernel TCP/IP
> processing will be done on core 1 and all the receiving side
> processing on core 0. However, having looked at the linux TCP/IP code,
> more specifically the loopback device code, when the loopbacks
> dev_queue_xmit function gets called it simply passes the skb to the
> netif_rx function. Therefore my question is thus, when i assign the
> sending / receiving side to different cores is the actual TCP/IP
> processing done the assigned sending / receiving cores ? or is TCP/IP
> processing done on the sending side only until a certain point? 
> it simply a single core that does all the TCP/IP processing. If it's
> the first two and you have any idea, could you point me where in the
> kernel the cores the processing swap over.

You need to define what you mean with TCP/IP processing. It has lots
of little sub-actions.  For the loopback case some will always run on
the sender side (IP, socket demultiplex, some will sometimes run on
either depending on workload (using TCP prequeue and depending on the
tcp_latency setting and how quickly the receiver schedules), and some
always on the receiver side (recvmsg user processing)

-Andi