From mboxrd@z Thu Jan 1 00:00:00 1970 From: gmate.amit@gmail.com (Kumar amit mehta) Date: Fri, 29 Mar 2013 06:19:08 -0700 Subject: relationship between cpu_affinity and Packet RX Processing In-Reply-To: References: <20130326183133.GA6146@gmail.com> <20130326190521.GB6868@gmail.com> Message-ID: <20130329131907.GA3199@gmail.com> To: kernelnewbies@lists.kernelnewbies.org List-Id: kernelnewbies.lists.kernelnewbies.org On Tue, Mar 26, 2013 at 09:31:01PM +0200, Rami Rosen wrote: > Hi, > > 1) You can be sure that it does not use more than one rx queue. > > You can find more info about RX queues in > http://media.wix.com/ugd//295986_4ef6dbdf11fd0a7f74f09741b4b5b2ee.pdf > > (I wrote it, a shameless plug...) > > 2) Does the irqbalance service, which is common in some distros, is > running ? can you try to disable it and run traffic again and check > /proc/interrupts afrer some time ? > > 3) regardless of 2 (meaning not stopping irqbalance if it was active) > can you please try "taskset 02 ping www.google.com" and tell us > whether interrupts were incremented on the second CPU ? (please send > /proc/interrupts | grep eth0 before and after sending, let's say, 5 > pings)? > My apologies for responding after a long time. I lost access to the machine on which this VM was running. Now running this VM on my laptop, which seem to have similar hardware configuration. Please find the details, which you had asked for below:(comment in quotes) amit at ubuntu:~$ cat /proc/cpuinfo |grep processor processor : 0 processor : 1 "2 CPU" amit at ubuntu:~$ cat /proc/interrupts |egrep 'CPU|eth0' CPU0 CPU1 19: 7887 31 IO-APIC-fasteoi eth0 amit at ubuntu:~$ ls -lah /sys/class/net/eth0/device/driver lrwxrwxrwx 1 root root 0 Mar 29 01:13 /sys/class/net/eth0/device/driver -> ../../../../bus/pci/drivers/pcnet32 amit at ubuntu:~$ lsmod|grep -i pcnet32 pcnet32 40671 0 "driver" amit at ubuntu:~$ ls /sys/class/net/eth0/queues rx-0 tx-0 "just one rx-queue" amit at ubuntu:~$ sudo cat /proc/irq/19/smp_affinity 01 "tied to CPU0" "disabling irqbalance" amit at ubuntu:~$ status irqbalance irqbalance start/running, process 1016 amit at ubuntu:~$ sudo stop irqbalance irqbalance stop/waiting amit at ubuntu:~$ taskset 02 ping www.google.com "after sometime" amit at ubuntu:~$ cat /proc/interrupts |egrep 'CPU|eth0' CPU0 CPU1 19: 8046 31 IO-APIC-fasteoi eth0 "No increase in interrupt counts on CPU1" "Now lets use both CPUs" root at ubuntu:/home/amit# echo "3" >/proc/irq/19/smp_affinity root at ubuntu:/home/amit# cat /proc/irq/19/smp_affinity 03 "As before, start ping and expecting both CPUs to be used" amit at ubuntu:~$ taskset 02 ping www.google.com root at ubuntu:/home/amit# cat /proc/interrupts |egrep 'CPU|eth0' CPU0 CPU1 19: 9001 71 IO-APIC-fasteoi eth0 ^^^^ and yes, indeed.. So multiple Rx queues seem to have nothing to do with CPU affinity, as in this case I have a driver implementing a single Rx queue and I see that the rx packet processing happens through both of the CPUs. -Amit