Re: Extreme system overhead on large IP27

[Ralf Baechle <ralf@linux-mips.org>]

On Tue, Oct 24, 2006 at 03:53:56PM +0200, Karl-Johan Karlsson wrote:

> I can't get physical access to the system to pull out CPU boards today, so
> I did the best I could do remotely - powered down all modules but one and
> am now running a kernel built with support for only 4 of the 8 remaining
> R12000 CPU:s.

The kernel has a maxcpus=<somenumber> option which is even easier.
You also can disable processors at the boot prompt.

Pulling node boards is strongly disrecommended; the connectors are very
fragile.

> Overhead is not as extreme as with more CPU:s, but still high. Running
> four copies of "md5sum /dev/zero", top shows around 95% useful work and 5%
> system overhead per CPU, while a "make -j4" of the kernel gives me 20-30%
> system and 70-80% user time (down from a maximum of 80% system time with
> all 32 CPU:s).
> 
> This is still on the Gentoo 2.6.17.10 kernel, by the way (which is a
> mips-git snapshot from 2006-06-18 plus extra patches from e.g.
> <URL:http://ftp.du.se/pub/os/gentoo/distfiles/mips-sources-generic_patches-1.25.tar.bz2>).
> I tried a git snapshot from earlier today, but the only thing that kernel
> did was print the NUMA-link topology and then hang.

To use the linux-mips.org git kernel you also need my IP27 patchset
available from /pub/linux/mips/people/ralf/ip27/ on ftp.linux-mips.org.

> Now that I actually look at Gentoo's patchset, I see there's a large patch
> (misc-2.6.17-ioc3-metadriver-r26.patch) touching serial and ethernet
> drivers for the IOC3. Perhaps the snapshot actually did boot, but just
> couldn't talk to me without that patch? The patch doesn't apply to the
> current git, though, so I think I'll leave that to someone who knows what
> they're doing.

That metadriver thing is primarily necessary for the sake of Octanes.

  Ralf