IRQ / SoftIRQ CPU time accounting broken by 457533a7d3402d1d91fbc125c8bd1bd16dcd3cd4

[Andrew Gallatin <gallatin@myri.com>]

When running netperf for some 10GbE tests, I noticed
that  IRQ and SOFTIRQ CPU time is no longer reported for an
otherwise idle CPU on recent kernels, at least for x86_64.

If I take a 2-CPU system, and bind the NIC IRQ to CPU0, and
bind the user-space netserver daemon to CPU1, the problem
is obvious when blasting 10Gb/s of traffic at it.  I see
no CPU used for irq or softirq on CPU0, even though it is
handling 13K interrupts/sec:



% mpstat -P 0 1
Linux 2.6.30-rc1 (venice)       04/22/09

11:25:25     CPU   %user   %nice %system %iowait    %irq   %soft   %idle 
    intr/s
11:25:26       0    0.00    0.00    0.00    0.00    0.00    0.00  100.00 
  13248.00
11:25:27       0    0.00    0.00    0.00    0.00    0.00    0.00  100.00 
  13280.00

Common sense tells me that is wrong, and oprofile verifies there is
a lot happening on CPU0.  Further, when I run a cpu-soaker in
usermode bound to CPU0, I start to see irq, softirq, etc,
being correctly identified:

11:28:02     CPU   %user   %nice %system %iowait    %irq   %soft   %idle 
    intr/s
11:28:03       0   45.10    0.00    0.00    0.00    1.96   52.94    0.00 
  13019.61
11:28:04       0   46.46    0.00    0.00    0.00    2.02   51.52    0.00 
  13414.14

The problem is observable, but much less obvious when using a more
common, e1000 1GbE NIC (15% softirq is missing, rather than 50%).

I spent a few hours git-bisecting until I finally got here:

% git-bisect bad
Bisecting: 0 revisions left to test after this
[457533a7d3402d1d91fbc125c8bd1bd16dcd3cd4] fix scaled & unscaled cputime 
accounting

I have neither CONFIG_NO_HZ, CONFIG_VIRT_CPU_ACCOUNTING, or XEN configured.

Drew