[Patch 0/6] Cpupools

[Patch 0/6] Cpupools

[Juergen Gross]

Round 2 for cpupools patches.

Changes from 1st version:
- create all domains in Pool-0, move to specific pool afterwards
- don't use cpupool_lock for cpu add/remove. Races with hot swapping cpus are
  avoided by locking cpus to Pool-0 if necessary

libxl patches will be sent soon, I'm still working on them.

The physical cpus of a server are grouped in cpu-pools. Each physical CPU is
assigned at most to one cpu-pool. Domains are each restricted to a single
cpu-pool. Scheduling does not cross cpu-pool boundaries, so each cpu-pool
has an own scheduler.

The cpu-pool of the domain Dom0 will be Pool-0, at boot-time we have to create
this cpu-pool implicitly.

The creation of a cpu-pool is similar to the creation of a domain. In a first
step the attributes of a cpu-pool have to be defined, in the second step the
cpu-pool will be activated (started) by assigning the resources (CPUs) and
initializing the scheduler. An activated cpu-pool owns at least one physical
CPU.

A physical CPU may be removed from an activated cpu-pool. Further CPUs may be
assigned to the cpu-pool. The CPU may be assigned to a cpu-pool only if that
CPU is free (not assigned to any other activated cpu-pool). The free CPUs are
internally managed.

The poolname will be added as an additional parameter to the configfile of a
domain. If no poolname is specified the poolname is defaults to Pool-0. At
domain start the domain is added to its cpu-pool, which must be activated.

A started domain can be moved to another activated cpu-pool.

A cpu-pool may be deactivated if there is no active domain assigned to it. A
cpu-pool is deactivated by removing the resources (CPUs).

A cpu-pool may be deleted if it is deactivated.

The administration of the cpu-pools is done by xm subcommands.


Patch 1/6: hypervisor support of cpupools
Patch 2/6: support in libxc
Patch 3/6: changes in libxen
Patch 4/6: python stuff (xm, xend)
Patch 5/6: xm-test add-ons for cpupools
Patch 6/6: documentation changes


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

RE: [Patch 0/6] Cpupools

[Ian Pratt]

> Round 2 for cpupools patches.
> 
> Changes from 1st version:
> - create all domains in Pool-0, move to specific pool afterwards

Do you make sure that the domain is moved to the right pool *before* any memory is assigned to the domain?  It's necessary that the CPU affinity mask gets set appropriately to ensure memory gets allocated from the right node.

Thanks,
Ian

Re: [Patch 0/6] Cpupools

[Juergen Gross]

Ian Pratt wrote:
>> Round 2 for cpupools patches.
>>
>> Changes from 1st version:
>> - create all domains in Pool-0, move to specific pool afterwards
> 
> Do you make sure that the domain is moved to the right pool *before* any memory is assigned to the domain?  It's necessary that the CPU affinity mask gets set appropriately to ensure memory gets allocated from the right node.

Yep. Right after creation the domain is moved to it's target pool, after that
the resources are allocated.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Re: [Patch 0/6] Cpupools

[Keir Fraser]

On 21/04/2010 12:15, "Juergen Gross" <juergen.gross@ts.fujitsu.com> wrote:

> Round 2 for cpupools patches.
> 
> Changes from 1st version:
> - create all domains in Pool-0, move to specific pool afterwards
> - don't use cpupool_lock for cpu add/remove. Races with hot swapping cpus are
>   avoided by locking cpus to Pool-0 if necessary

Applied. Thanks.

 -- Keir

[Patch 0/6] Cpupools

[Juergen Gross]

Hi,

Following patch-series support cpupools in xen:

The physical cpus of a server are grouped in cpu-pools. Each physical CPU is
assigned at most to one cpu-pool. Domains are each restricted to a single
cpu-pool. Scheduling does not cross cpu-pool boundaries, so each cpu-pool
has an own scheduler.

The cpu-pool of the domain Dom0 will be Pool-0, at boot-time we have to create
this cpu-pool implicitly.

The creation of a cpu-pool is similar to the creation of a domain. In a first
step the attributes of a cpu-pool have to be defined, in the second step the
cpu-pool will be activated (started) by assigning the resources (CPUs) and
initializing the scheduler. An activated cpu-pool owns at least one physical
CPU.

A physical CPU may be removed from an activated cpu-pool. Further CPUs may be
assigned to the cpu-pool. The CPU may be assigned to a cpu-pool only if that
CPU is free (not assigned to any other activated cpu-pool). The free CPUs are
internally managed.

The poolname will be added as an additional parameter to the configfile of a
domain. If no poolname is specified the poolname is defaults to Pool-0. At
domain start the domain is added to its cpu-pool, which must be activated.

A started domain can be moved to another activated cpu-pool.

A cpu-pool may be deactivated if there is no active domain assigned to it. A
cpu-pool is deactivated by removing the resources (CPUs).

A cpu-pool may be deleted if it is deactivated.

The administration of the cpu-pools is done by xm subcommands.


Patch 1/6: hypervisor support of cpupools
Patch 2/6: support in libxc
Patch 3/6: changes in libxen
Patch 4/6: python stuff (xm, xend)
Patch 5/6: xm-test add-ons for cpupools
Patch 6/6: documentation changes


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Re: [Patch 0/6] Cpupools

[Ian Jackson]

Juergen Gross writes ("[Xen-devel] [Patch 0/6] Cpupools"):
> Patch 4/6: python stuff (xm, xend)

Would you care to support this in libxl, please ?  xl is going to be
the toolstack for the future we hope ...

Thanks,
Ian.

Re: [Patch 0/6] Cpupools

[Juergen Gross]

Ian Jackson wrote:
> Juergen Gross writes ("[Xen-devel] [Patch 0/6] Cpupools"):
>> Patch 4/6: python stuff (xm, xend)
> 
> Would you care to support this in libxl, please ?  xl is going to be
> the toolstack for the future we hope ...

Okay, next round will include libxl support.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Re: [Patch 0/6] Cpupools

[Konrad Rzeszutek Wilk]

On Tue, Apr 20, 2010 at 11:37:58AM +0200, Juergen Gross wrote:
> Hi,
> 
> Following patch-series support cpupools in xen:

What is the advantage and disadvantage of this? What kind of problem is
this trying to solve?

Re: [Patch 0/6] Cpupools

[Juergen Gross]

Konrad Rzeszutek Wilk wrote:
> On Tue, Apr 20, 2010 at 11:37:58AM +0200, Juergen Gross wrote:
>> Hi,
>>
>> Following patch-series support cpupools in xen:
> 
> What is the advantage and disadvantage of this? What kind of problem is
> this trying to solve?

Please see the original discussion on xen-devel:
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00333.html
This should answer most of your questions.


Juergen

-- 
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@ts.fujitsu.com
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html