From: "Martin J. Bligh" <Martin.Bligh@us.ibm.com>
To: Sean Cavanaugh <seanc@gearboxsoftware.com>
Cc: linux-kernel@vger.kernel.org
Subject: Re: P4 SMP load balancing
Date: Fri, 12 Oct 2001 10:59:47 -0700 [thread overview]
Message-ID: <2157008025.1002884387@mbligh.des.sequent.com> (raw)
In-Reply-To: <002601c15300$3a9f0510$150a10ac@gearboxsoftware.com>
> ovendev:~# cat /proc/interrupts
> CPU0 CPU1
> 0: 6348212 0 IO-APIC-edge timer
> 1: 2 0 IO-APIC-edge keyboard
> 2: 0 0 XT-PIC cascade
> 8: 1 0 IO-APIC-edge rtc
> 9: 0 0 IO-APIC-edge acpi
> 16: 92620 0 IO-APIC-level eth0
> 18: 5085 0 IO-APIC-level aic7xxx, aic7xxx
> NMI: 0 0
> LOC: 6348388 6348427
> ERR: 0
> MIS: 0
I don't think this should happen. In the event of both procs having equal
priority (linux never changes them, so they always do), we should fall back
to the arbitration priority of the lapic. Whether you have 1 or 2 I/O apics
working shouldn't make a difference.
The arb priority of the local apic should change whenever a message
is sent (see the Intel docs on developer.intel.com), so we effectively
get round robin. For instance, a 4 way looks like this:
CPU0 CPU1 CPU2 CPU3
0: 1608606 1595657 2168078 1575546 IO-APIC-edge timer
1: 0 0 0 2 IO-APIC-edge keyboard
2: 0 0 0 0 XT-PIC cascade
4: 76 52 62 48 IO-APIC-edge serial
23: 7983 8263 8286 8306 IO-APIC-level qlogicisp
39: 0 0 0 0 IO-APIC-level eth1
40: 6247 6216 6894 6325 IO-APIC-level eth0
NMI: 0 0 0 0
LOC: 6947876 6947859 6947873 6947874
ERR: 0
MIS: 0
Which isn't perfectly balanced, but it looks a damned sight better than
yours does ;-) Do you have something in the log that looks like this?
Oct 11 15:35:04 elm3b76 kernel: IO APIC #13......
Oct 11 15:35:04 elm3b76 kernel: .... register #00: 0D000000
Oct 11 15:35:04 elm3b76 kernel: ....... : physical APIC id: 0D
Oct 11 15:35:04 elm3b76 kernel: .... register #01: 00170011
Oct 11 15:35:04 elm3b76 kernel: ....... : max redirection entries: 0017
Oct 11 15:35:04 elm3b76 kernel: ....... : IO APIC version: 0011
Oct 11 15:35:04 elm3b76 kernel: .... register #02: 00000000
Oct 11 15:35:04 elm3b76 kernel: ....... : arbitration: 00
Oct 11 15:35:04 elm3b76 kernel: .... IRQ redirection table:
Oct 11 15:35:04 elm3b76 kernel: NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
Oct 11 15:35:04 elm3b76 kernel: 00 000 00 1 0 0 0 0 0 0 00
Oct 11 15:35:05 elm3b76 kernel: 01 00F 0F 0 0 0 0 0 0 1 39
Oct 11 15:35:05 elm3b76 kernel: 02 00F 0F 0 0 0 0 0 0 1 31
Oct 11 15:35:05 elm3b76 kernel: 03 00F 0F 0 0 0 0 0 0 1 41
Oct 11 15:35:05 elm3b76 kernel: 04 00F 0F 0 0 0 0 0 0 1 49
Oct 11 15:35:05 elm3b76 kernel: 05 00F 0F 0 0 0 0 0 0 1 51
Oct 11 15:35:05 elm3b76 kernel: 06 00F 0F 0 0 0 0 0 0 1 59
Oct 11 15:35:05 elm3b76 kernel: 07 00F 0F 1 1 0 1 0 0 1 61
Oct 11 15:35:05 elm3b76 kernel: 08 00F 0F 1 1 0 0 0 0 1 69
Oct 11 15:35:05 elm3b76 kernel: 09 00F 0F 0 0 0 0 0 0 1 71
Oct 11 15:35:05 elm3b76 kernel: 0a 00F 0F 0 0 0 0 0 0 1 79
Oct 11 15:35:05 elm3b76 kernel: 0b 00F 0F 1 1 0 1 0 0 1 81
Oct 11 15:35:05 elm3b76 kernel: 0c 00F 0F 0 0 0 0 0 0 1 89
Oct 11 15:35:05 elm3b76 kernel: 0d 00F 0F 1 1 0 1 0 0 1 91
Oct 11 15:35:05 elm3b76 kernel: 0e 00F 0F 0 0 0 0 0 0 1 99
Oct 11 15:35:05 elm3b76 kernel: 0f 00F 0F 1 1 0 1 0 0 1 A1
Oct 11 15:35:05 elm3b76 kernel: 10 00F 0F 1 1 0 1 0 0 1 A9
Oct 11 15:35:05 elm3b76 kernel: 11 00F 0F 1 1 0 1 0 0 1 B1
Oct 11 15:35:05 elm3b76 kernel: 12 00F 0F 1 1 0 1 0 0 1 B9
Oct 11 15:35:05 elm3b76 kernel: 13 00F 0F 1 1 0 1 0 0 1 C1
Oct 11 15:35:05 elm3b76 kernel: 14 00F 0F 1 1 0 1 0 0 1 C9
Oct 11 15:35:05 elm3b76 kernel: 15 00F 0F 1 1 0 1 0 0 1 D1
Oct 11 15:35:05 elm3b76 kernel: 16 00F 0F 1 1 0 1 0 0 1 D9
Oct 11 15:35:05 elm3b76 kernel: 17 00F 0F 1 1 0 1 0 0 1 E1
You might have to tweak syslog.conf to log the debug messages. And
possibly increase LOG_BUF_LEN in kernel/printk.c to something sensible
(63356?)
M.
next prev parent reply other threads:[~2001-10-12 18:05 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2001-10-12 9:28 P4 SMP load balancing Sean Cavanaugh
2001-10-12 17:59 ` Martin J. Bligh [this message]
2001-10-14 9:07 ` Sean Cavanaugh
-- strict thread matches above, loose matches on Subject: below --
2001-10-12 18:38 Manfred Spraul
2001-10-12 21:38 ` Martin J. Bligh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2157008025.1002884387@mbligh.des.sequent.com \
--to=martin.bligh@us.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=seanc@gearboxsoftware.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox