strange phenomenon on CPU affinity

strange phenomenon on CPU affinity

[周理科]

[-- Attachment #1.1: Type: text/plain, Size: 855 bytes --]

Hello,
     My testing machine has 2 quad-core CPU (It supports hyperthreading,
but i disable it in BIOS). I uses Xen 4.0.1 as the hypervisor. When I use 8
VMs to conduct a test, CPU affinity of the VMs is very strange. Like this:

vm_name  vcpu_num  cpu_affinity
Domain-0    8      any
VM1            4      1,3,5,7
VM2            4      1,3,5,7
VM3            4      1,3,5,7
VM4            4      1,3,5,7
VM5            4      1,3,5,7
VM6            4      0,2,4,6
VM7            4      0,2,4,6
VM8            4      0,2,4,6

I do not set the CPU affinity in the configuration file, and I cannot find
when the hypervisor set the CPU affinity in the source code. In this
situation, 4 VCPUs of each VM are binding to 4 PCPUs permanently, and 5 VMs
run on a set of PCPUs, and others run on the other set of PCPUs. It is
unfair to these VMs.

-- 
Like Zhou

[-- Attachment #1.2: Type: text/html, Size: 1040 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: strange phenomenon on CPU affinity

[Juergen Gross]

On 15.03.2013 10:08, 周理科 wrote:
> Hello,
>       My testing machine has 2 quad-core CPU (It supports hyperthreading, but i
> disable it in BIOS). I uses Xen 4.0.1 as the hypervisor. When I use 8 VMs to
> conduct a test, CPU affinity of the VMs is very strange. Like this:
>
> vm_name  vcpu_num  cpu_affinity
> Domain-0    8      any
> VM1            4      1,3,5,7
> VM2            4      1,3,5,7
> VM3            4      1,3,5,7
> VM4            4      1,3,5,7
> VM5            4      1,3,5,7
> VM6            4      0,2,4,6
> VM7            4      0,2,4,6
> VM8            4      0,2,4,6
>
> I do not set the CPU affinity in the configuration file, and I cannot find when
> the hypervisor set the CPU affinity in the source code. In this situation, 4
> VCPUs of each VM are binding to 4 PCPUs permanently, and 5 VMs run on a set of
> PCPUs, and others run on the other set of PCPUs. It is unfair to these VMs.

I'd suspect NUMA optimization. xend tries to optimize domain placement by
pinning the vcpus of the domains to cores in the same NUMA domain. Normally
the overall performance is better with this optimization. You can disable it
by specifying

numa=off

as an additional xen boot parameter for the hypervisor (not the dom0 kernel).
You can see whether you have NUMA active with

xm info

It will display line(s) like:

node_to_cpu            : node0:0-3

If you see multiple nodes you have NUMA active.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
PBG PDG ES&S SWE OS6                   Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: strange phenomenon on CPU affinity

[Like Zhou]

[-- Attachment #1.1: Type: text/plain, Size: 2124 bytes --]

2013/3/15 Juergen Gross <juergen.gross@ts.fujitsu.com>

> On 15.03.2013 10:08, 周理科 wrote:
>
>> Hello,
>>       My testing machine has 2 quad-core CPU (It supports hyperthreading,
>> but i
>> disable it in BIOS). I uses Xen 4.0.1 as the hypervisor. When I use 8 VMs
>> to
>> conduct a test, CPU affinity of the VMs is very strange. Like this:
>>
>> vm_name  vcpu_num  cpu_affinity
>> Domain-0    8      any
>> VM1            4      1,3,5,7
>> VM2            4      1,3,5,7
>> VM3            4      1,3,5,7
>> VM4            4      1,3,5,7
>> VM5            4      1,3,5,7
>> VM6            4      0,2,4,6
>> VM7            4      0,2,4,6
>> VM8            4      0,2,4,6
>>
>> I do not set the CPU affinity in the configuration file, and I cannot
>> find when
>> the hypervisor set the CPU affinity in the source code. In this
>> situation, 4
>> VCPUs of each VM are binding to 4 PCPUs permanently, and 5 VMs run on a
>> set of
>> PCPUs, and others run on the other set of PCPUs. It is unfair to these
>> VMs.
>>
>
> I'd suspect NUMA optimization. xend tries to optimize domain placement by
> pinning the vcpus of the domains to cores in the same NUMA domain. Normally
> the overall performance is better with this optimization. You can disable
> it
> by specifying
>
> numa=off
>
> as an additional xen boot parameter for the hypervisor (not the dom0
> kernel).
> You can see whether you have NUMA active with
>
> xm info
>
> It will display line(s) like:
>
> node_to_cpu            : node0:0-3
>
> If you see multiple nodes you have NUMA active.
>
>
> Juergen
>
> --
> Juergen Gross                 Principal Developer Operating Systems
> PBG PDG ES&S SWE OS6                   Telephone: +49 (0) 89 3222 2967
> Fujitsu Technology Solutions              e-mail:
> juergen.gross@ts.fujitsu.com
> Domagkstr. 28                           Internet: ts.fujitsu.com
> D-80807 Muenchen                 Company details:
> ts.fujitsu.com/imprint.html
>

Yes. When I disable NUMA optimization, CPU affinity of each VCPU becomes
"any". Thanks.


-- 
Like Zhou

[-- Attachment #1.2: Type: text/html, Size: 3558 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel