Re: Extreme system overhead on large IP27

[Ralf Baechle <ralf@linux-mips.org>]

On Sat, Oct 21, 2006 at 09:59:02PM +0200, Karl-Johan Karlsson wrote:

> I have an Origin 2000 with 16 R12000 and 16 R10000 CPU:s, running a git 
> snapshot kernel from 20060618 based on 2.6.17.10 (the latest available in 
> Gentoo). Light loads run without problems, but as soon as the load average 
> goes above 4-5 system overhead skyrockets and almost no useful work is being 
> done (see top output below). OProfile is no help, since the daemon just 
> throws away everything the kernel gives it (see output from strace of 
> oprofiled below).
> 
> Does anyone know where this overhead is coming from, or how to get some data 
> from OProfile so I can search for it myself? I'll try booting just the R12000 
> part sometime soon to see if that helps with either problem.

Oprofile is a bit of a bitch on mixed processor systems since it assumes
all processors to have identical performance counters.  However SGI in
it's wisdem decieded the R12000 had to be better than the R10000 and
changed it.  It is possible to work around that but lacking any mixed
CPU configuration I've never done that.

With those annotations, the kernel part of oprofile doesn't yet support
R1x000 processors, I'll try to cook up something.  Should be easy enough
since the interface is nearly identical to MIPS32/MIPS64.

  Ralf