From: Andre Przywara <andre.przywara@amd.com>
To: "marcotte@panix.com" <marcotte@panix.com>
Cc: "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>
Subject: Re: NUMA problem with AMD G34 system
Date: Mon, 21 Feb 2011 09:48:48 +0100 [thread overview]
Message-ID: <4D6226F0.1080007@amd.com> (raw)
In-Reply-To: <20110219085932.E08B624258@panix5.panix.com>
Brian Marcotte wrote:
> I'm having a problem with NUMA on an AMD G34 system. Xen seems to think
> that out of the 16 total cores, there are 12 on node0 and 4 on node3.
>
> The box is:
>
> SuperMicro H8DGU (dual socket G34)
> Pair of AMD 6128 CPUs (8 core each)
> 32 GB of RAM.
> BIOS has been upgraded three times - no help.
>
> xen-unstable from Feb 12 (same problem on 3.4.x).
>
> I'm including the output from the "u" command below to show Xen's idea
> of the CPU->node mapping. Complete boot log is here:
>
> http://www.panix.com/~marcotte/g34/boot.txt
>
> Any help would be appreciated.
Have you checked that you populated the DIMM slots properly?
The SRAT CPU table seems sane, but the memory table seems to say that
there is no memory on node 1 and 3, thus Xen will simply put those CPUs
to node 0 as it cannot cope with memory-less nodes.
If I understand the manual right, you first have to populate all the
blue slots, then the black ones.
Hope that helps,
Andre.
>
> Thanks.
>
> - Brian
>
> (XEN) Xen version 4.1.0-rc5-pre (root@nyc.access.net) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) Sat Feb 12 18:53:49 EST 2011
> (XEN) Command line: dom0_mem=600M com1=57600,8n1 console=com1 no-mce loglvl=debug
>
> (XEN) 'u' pressed -> dumping numa info (now-0x2:5A61B881)
> (XEN) idx0 -> NODE0 start->0 size->4325376
> (XEN) phys_to_nid(0000000000001000) -> 0 should be 0
> (XEN) idx3 -> NODE3 start->4325376 size->4194304
> (XEN) phys_to_nid(0000000420001000) -> 3 should be 3
> (XEN) CPU0 -> NODE0
> (XEN) CPU1 -> NODE0
> (XEN) CPU2 -> NODE0
> (XEN) CPU3 -> NODE0
> (XEN) CPU4 -> NODE0
> (XEN) CPU5 -> NODE0
> (XEN) CPU6 -> NODE0
> (XEN) CPU7 -> NODE0
> (XEN) CPU8 -> NODE3
> (XEN) CPU9 -> NODE3
> (XEN) CPU10 -> NODE3
> (XEN) CPU11 -> NODE3
> (XEN) CPU12 -> NODE0
> (XEN) CPU13 -> NODE0
> (XEN) CPU14 -> NODE0
> (XEN) CPU15 -> NODE0
> (XEN) Memory location of each domain:
> (XEN) Domain 0 (total: 153600):
> (XEN) Node 0: 132232
> (XEN) Node 3: 21368
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12
next prev parent reply other threads:[~2011-02-21 8:48 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-19 8:59 NUMA problem with AMD G34 system Brian Marcotte
2011-02-21 8:48 ` Andre Przywara [this message]
2011-02-21 11:29 ` Brian Marcotte
2011-02-21 11:51 ` Andre Przywara
2011-02-21 13:20 ` Brian Marcotte
2011-02-21 13:42 ` Andre Przywara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D6226F0.1080007@amd.com \
--to=andre.przywara@amd.com \
--cc=marcotte@panix.com \
--cc=xen-devel@lists.xensource.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.