* RE: Unable to set dom0 vcpu to 1
@ 2006-06-12 15:43 Carb, Brian A
2006-06-12 15:50 ` Ryan Harper
0 siblings, 1 reply; 10+ messages in thread
From: Carb, Brian A @ 2006-06-12 15:43 UTC (permalink / raw)
To: Ryan Harper
Cc: xen-devel, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Subrahmanian, Raj, Vessey, Bruce A
Thanks Ryan. We've experienced the wait as well (we contributed the
timeout loop in changeset 10043 for xm-test
01_enforce_dom0_cpus_basic_pos). For some reason, though, vcpu-set is
not releasing procs above #7, even when we wait a long time.
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
-----Original Message-----
From: Ryan Harper [mailto:ryanh@us.ibm.com]
Sent: Monday, June 12, 2006 11:12 AM
To: Carb, Brian A
Cc: xen-devel@lists.xensource.com; McAfee, Tommie M; Krysan, Susan;
Puthiyaparambil, Aravindh; Subrahmanian, Raj; Vessey, Bruce A
Subject: Re: [Xen-devel] Unable to set dom0 vcpu to 1
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-08 15:13]:
> > Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64,
> > 16 procs, 16 GB RAM under SLES 10 RC1.
> >
> > At startup, dom0-cpus is set to 0 (the default) in the
> > xend-config.sxp file, and dom0 uses all 16 cpus. 'xm top' shows 16
procs, 'xm list'
> > shows dom0 using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned
> > to dom0.
> >
> > When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0
> > 1', the number of procs, as shown by /proc/cpuinfo, reduces to 9.
> > 'xm list' shows dom0 using 9 procs, and 'xm vcpu-list' shows that
> > procs
> > 1-7 have been released, while procs 0 and 8 through 15 are still
> > assigned. Nothing is shown on the serial port.
> >
> > Can anyone shed some light on why the rest of the procs might not be
> > released? Thanks.
On large smp boxes, it takes some time to unplug all of the processors
as any processes that are running on those cpus have to be migrated to
another cpu.
On a 32-way, it took 15 seconds to go from 32 to 1. As I ran xm list,
you can see the number of active vcpus reduce over time.
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 21 r----- 4494.6
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 19 r----- 4495.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 17 r----- 4496.2
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 16 r----- 4496.8
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 16 r----- 4497.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 15 r----- 4497.6
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 14 r----- 4498.1
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 13 r----- 4498.6
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 12 r----- 4499.2
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 12 r----- 4499.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 10 r----- 4500.3
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 10 r----- 4500.7
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 9 r----- 4501.0
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 8 r----- 4501.5
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 7 r----- 4502.0
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 6 r----- 4502.5
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 4 r----- 4503.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 1 r----- 4504.4
--
Ryan Harper
Software Engineer; Linux Technology Center IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Unable to set dom0 vcpu to 1
2006-06-12 15:43 Unable to set dom0 vcpu to 1 Carb, Brian A
@ 2006-06-12 15:50 ` Ryan Harper
0 siblings, 0 replies; 10+ messages in thread
From: Ryan Harper @ 2006-06-12 15:50 UTC (permalink / raw)
To: Carb, Brian A
Cc: xen-devel, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Subrahmanian, Raj, Vessey, Bruce A
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-12 10:44]:
> Thanks Ryan. We've experienced the wait as well (we contributed the
> timeout loop in changeset 10043 for xm-test
> 01_enforce_dom0_cpus_basic_pos). For some reason, though, vcpu-set is
> not releasing procs above #7, even when we wait a long time.
There was a recent patch dealing with hotplug cpu and rcu sent to the
list by Harry Butterworth. I think it has been committed yet but you
will probably want to try again with that patch.
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Unable to set dom0 vcpu to 1
@ 2006-06-15 15:11 Carb, Brian A
2006-06-15 16:16 ` Ryan Harper
0 siblings, 1 reply; 10+ messages in thread
From: Carb, Brian A @ 2006-06-15 15:11 UTC (permalink / raw)
To: Ryan Harper
Cc: xen-devel, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Subrahmanian, Raj, Vessey, Bruce A
Thanks Ryan,
We're still seeing this problem as of changeset 10333 (which I believe
includes that patch).
It seems as if the "vcpus" value as shown by "xm list --long 0" command
is getting set to 8, even when we boot dom0 with 16procs. In fact, vcpus
is still set to 8 even if we boot hyperthreaded to a 32x. Since the
release of procs with vcpu-set seems to key off this value, we can never
release more than 7.
--------------------------
m0075:~ # xm list --long 0
(domain
(domid 0)
(uuid 00000000-0000-0000-0000-000000000000)
(vcpus 8)
(cpu_weight 1.0)
(memory 2728)
(maxmem 15016)
(features )
(name Domain-0)
(on_poweroff destroy)
(on_reboot restart)
(on_crash restart)
(state r-----)
(shutdown_reason poweroff)
(cpu_time 1254.97341823)
(online_vcpus 16)
)
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
-----Original Message-----
From: Ryan Harper [mailto:ryanh@us.ibm.com]
Sent: Monday, June 12, 2006 11:50 AM
To: Carb, Brian A
Cc: xen-devel@lists.xensource.com; McAfee, Tommie M; Krysan, Susan;
Puthiyaparambil, Aravindh; Subrahmanian, Raj; Vessey, Bruce A
Subject: Re: [Xen-devel] Unable to set dom0 vcpu to 1
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-12 10:44]:
> Thanks Ryan. We've experienced the wait as well (we contributed the
> timeout loop in changeset 10043 for xm-test
> 01_enforce_dom0_cpus_basic_pos). For some reason, though, vcpu-set is
> not releasing procs above #7, even when we wait a long time.
There was a recent patch dealing with hotplug cpu and rcu sent to the
list by Harry Butterworth. I think it has been committed yet but you
will probably want to try again with that patch.
--
Ryan Harper
Software Engineer; Linux Technology Center IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Unable to set dom0 vcpu to 1
2006-06-15 15:11 Carb, Brian A
@ 2006-06-15 16:16 ` Ryan Harper
0 siblings, 0 replies; 10+ messages in thread
From: Ryan Harper @ 2006-06-15 16:16 UTC (permalink / raw)
To: Carb, Brian A
Cc: xen-devel, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Subrahmanian, Raj, Vessey, Bruce A
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-15 10:12]:
> Thanks Ryan,
>
> We're still seeing this problem as of changeset 10333 (which I believe
> includes that patch).
Yeah, that was a long shot.
>
> It seems as if the "vcpus" value as shown by "xm list --long 0" command
> is getting set to 8, even when we boot dom0 with 16procs. In fact, vcpus
> is still set to 8 even if we boot hyperthreaded to a 32x. Since the
> release of procs with vcpu-set seems to key off this value, we can never
> release more than 7.
the vcpus value for dom0 should be the number of vcpus the domain is
configured to use; usually in the conf file. Domain0 is different since
it has no conf file being built by xen instead of Xend. In this case,
vcpus value is derived from max_vcpu_id, which is the highest vcpu
that as been initialized. For dom0, this value should be the
number of cpus xen enabled during boot.
vcpu-set uses the vcpu_avail variable which is initialized from the
vcpus variable, which, as mentioned able is determined by max_vcpu_id.
e.g. max_vcpu_id=7, vcpus = 8, vcpu_avail = (1 << 8)-1, or 0xff
I'd start fresh by clearing out xenstore:
rm /var/lib/xenstored/tdb
And rebooting.
It should never be the case where vcpus for dom0 is < online vcpus as
that means you are running vcpus that haven't been initialized
(max_vcpu_id indicates the max vcpu that has been allocated to run in
the domain). It would be interesting if you can track what steps you
took to get the machine/xenstore into this state and file a bug.
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Unable to set dom0 vcpu to 1
@ 2006-06-08 21:20 Apparao, Padmashree K
2006-06-12 19:06 ` Pablo Montesinos
0 siblings, 1 reply; 10+ messages in thread
From: Apparao, Padmashree K @ 2006-06-08 21:20 UTC (permalink / raw)
To: Subrahmanian, Raj, Carb, Brian A, xen-devel
Cc: McAfee, Tommie M, Krysan, Susan, Puthiyaparambil, Aravindh,
Vessey, Bruce A
[-- Attachment #1.1: Type: text/plain, Size: 5331 bytes --]
I think it should too, but I ended up doing this to achieve what I
needed. ( only 1 vcpu for dom0)
- Padma
-
________________________________
From: Subrahmanian, Raj [mailto:raj.subrahmanian@unisys.com]
Sent: Thursday, June 08, 2006 2:16 PM
To: Apparao, Padmashree K; Carb, Brian A; xen-devel@lists.xensource.com
Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Vessey,
Bruce A
Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1
Padma,
Shouldn't the xm vcpu-set work regardless of the dom0-cpus value?
Raj
________________________________
From: Apparao, Padmashree K [mailto:padmashree.k.apparao@intel.com]
Sent: Thursday, June 08, 2006 5:03 PM
To: Carb, Brian A; xen-devel@lists.xensource.com
Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh;
Subrahmanian, Raj; Vessey, Bruce A
Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1
Set (dom-cpus=1) in etc/xen/xend-config.sxp.
- Padma
-
________________________________
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Carb, Brian
A
Sent: Thursday, June 08, 2006 1:12 PM
To: xen-devel@lists.xensource.com
Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh;
Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A
Subject: [Xen-devel] Unable to set dom0 vcpu to 1
Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16
procs, 16 GB RAM under SLES 10 RC1.
At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp
file, and dom0 uses all 16 cpus. 'xm top' shows 16 procs, 'xm list'
shows dom0 using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned to
dom0.
When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0 1',
the number of procs, as shown by /proc/cpuinfo, reduces to 9. 'xm list'
shows dom0 using 9 procs, and 'xm vcpu-list' shows that procs 1-7 have
been released, while procs 0 and 8 through 15 are still assigned.
Nothing is shown on the serial port.
Can anyone shed some light on why the rest of the procs might not be
released? Thanks.
The following is output from 'xm list' and 'xm vcpu-list':
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 16 r----- 849.3
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 438.1 any cpu
Domain-0 0 1 1 -b- 4.0 any cpu
Domain-0 0 2 2 -b- 5.4 any cpu
Domain-0 0 3 3 -b- 4.5 any cpu
Domain-0 0 4 4 -b- 8.4 any cpu
Domain-0 0 5 5 -b- 6.1 any cpu
Domain-0 0 6 6 -b- 5.0 any cpu
Domain-0 0 7 7 -b- 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 14.6 any cpu
Domain-0 0 10 10 r-- 29.1 any cpu
Domain-0 0 11 11 -b- 15.1 any cpu
Domain-0 0 12 12 -b- 71.6 any cpu
Domain-0 0 13 13 -b- 177.0 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 21.9 any cpu
m0075:~ # xm vcpu-set 0 1
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 9 r----- 853.2
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 439.3 any cpu
Domain-0 0 1 - --p 4.0 any cpu
Domain-0 0 2 - --p 5.6 any cpu
Domain-0 0 3 - --p 4.5 any cpu
Domain-0 0 4 - --p 8.4 any cpu
Domain-0 0 5 - --p 6.1 any cpu
Domain-0 0 6 - --p 5.0 any cpu
Domain-0 0 7 - --p 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 15.0 any cpu
Domain-0 0 10 10 r-- 29.3 any cpu
Domain-0 0 11 11 -b- 15.2 any cpu
Domain-0 0 12 12 -b- 72.7 any cpu
Domain-0 0 13 13 -b- 177.5 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 22.0 any cpu
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
[-- Attachment #1.2: Type: text/html, Size: 25619 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Unable to set dom0 vcpu to 1
2006-06-08 21:20 Apparao, Padmashree K
@ 2006-06-12 19:06 ` Pablo Montesinos
0 siblings, 0 replies; 10+ messages in thread
From: Pablo Montesinos @ 2006-06-12 19:06 UTC (permalink / raw)
Cc: xen-devel, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Subrahmanian, Raj, Vessey, Bruce A,
Carb, Brian A
Actually, in our machine, no matter what I write in
/etc/xen/xend-config.sxp, dom0 alaways uses the 32 cpus.
On 6/8/06, Apparao, Padmashree K <padmashree.k.apparao@intel.com> wrote:
>
>
>
>
> I think it should too, but I ended up doing this to achieve what I needed.
> ( only 1 vcpu for dom0)
>
>
>
> - Padma
>
> -
>
>
>
> ________________________________
>
>
> From: Subrahmanian, Raj [mailto:raj.subrahmanian@unisys.com]
> Sent: Thursday, June 08, 2006 2:16 PM
> To: Apparao, Padmashree K; Carb, Brian A; xen-devel@lists.xensource.com
> Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh; Vessey,
> Bruce A
>
> Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1
>
>
>
>
>
>
>
> Padma,
>
> Shouldn't the xm vcpu-set work regardless of the dom0-cpus value?
>
> Raj
>
>
> ________________________________
>
>
> From: Apparao, Padmashree K [mailto:padmashree.k.apparao@intel.com]
> Sent: Thursday, June 08, 2006 5:03 PM
> To: Carb, Brian A; xen-devel@lists.xensource.com
> Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh;
> Subrahmanian, Raj; Vessey, Bruce A
> Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1
>
> Set (dom-cpus=1) in etc/xen/xend-config.sxp.
>
>
>
> - Padma
>
> -
>
>
>
>
>
> ________________________________
>
>
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of
> Carb, Brian A
> Sent: Thursday, June 08, 2006 1:12 PM
> To: xen-devel@lists.xensource.com
> Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh;
> Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A
> Subject: [Xen-devel] Unable to set dom0 vcpu to 1
>
>
>
> Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16
> procs, 16 GB RAM under SLES 10 RC1.
>
> At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp file,
> and dom0 uses all 16 cpus. 'xm top' shows 16 procs, 'xm list' shows dom0
> using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned to dom0.
>
> When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0 1', the
> number of procs, as shown by /proc/cpuinfo, reduces to 9. 'xm list' shows
> dom0 using 9 procs, and 'xm vcpu-list' shows that procs 1-7 have been
> released, while procs 0 and 8 through 15 are still assigned. Nothing is
> shown on the serial port.
>
> Can anyone shed some light on why the rest of the procs might not be
> released? Thanks.
>
> The following is output from 'xm list' and 'xm vcpu-list':
>
> m0075:~ # xm list
> Name ID Mem(MiB) VCPUs State
> Time(s)
> Domain-0 0 15912 16 r-----
> 849.3
> m0075:~ # xm vcpu-list
> Name ID VCPU CPU State
> Time(s) CPU Affinity
> Domain-0 0 0 0 -b-
> 438.1 any cpu
> Domain-0 0 1 1 -b-
> 4.0 any cpu
> Domain-0 0 2 2 -b-
> 5.4 any cpu
> Domain-0 0 3 3 -b-
> 4.5 any cpu
> Domain-0 0 4 4 -b-
> 8.4 any cpu
> Domain-0 0 5 5 -b-
> 6.1 any cpu
> Domain-0 0 6 6 -b-
> 5.0 any cpu
> Domain-0 0 7 7 -b-
> 4.5 any cpu
> Domain-0 0 8 8 -b-
> 30.3 any cpu
> Domain-0 0 9 9 -b-
> 14.6 any cpu
> Domain-0 0 10 10 r--
> 29.1 any cpu
> Domain-0 0 11 11 -b-
> 15.1 any cpu
> Domain-0 0 12 12 -b-
> 71.6 any cpu
> Domain-0 0 13 13 -b-
> 177.0 any cpu
> Domain-0 0 14 14 -b-
> 14.5 any cpu
> Domain-0 0 15 15 -b-
> 21.9 any cpu
>
> m0075:~ # xm vcpu-set 0 1
> m0075:~ # xm list
> Name ID Mem(MiB) VCPUs State
> Time(s)
> Domain-0 0 15912 9 r-----
> 853.2
> m0075:~ # xm vcpu-list
> Name ID VCPU CPU State
> Time(s) CPU Affinity
> Domain-0 0 0 0 -b-
> 439.3 any cpu
> Domain-0 0 1 - --p
> 4.0 any cpu
> Domain-0 0 2 - --p
> 5.6 any cpu
> Domain-0 0 3 - --p
> 4.5 any cpu
> Domain-0 0 4 - --p
> 8.4 any cpu
> Domain-0 0 5 - --p
> 6.1 any cpu
> Domain-0 0 6 - --p
> 5.0 any cpu
> Domain-0 0 7 - --p
> 4.5 any cpu
> Domain-0 0 8 8 -b-
> 30.3 any cpu
> Domain-0 0 9 9 -b-
> 15.0 any cpu
> Domain-0 0 10 10 r--
> 29.3 any cpu
> Domain-0 0 11 11 -b-
> 15.2 any cpu
> Domain-0 0 12 12 -b-
> 72.7 any cpu
> Domain-0 0 13 13 -b-
> 177.5 any cpu
> Domain-0 0 14 14 -b-
> 14.5 any cpu
> Domain-0 0 15 15 -b-
> 22.0 any cpu
>
> brian carb
> unisys corporation - malvern, pa
> brian.carb@unisys.com
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Unable to set dom0 vcpu to 1
@ 2006-06-08 21:15 Subrahmanian, Raj
0 siblings, 0 replies; 10+ messages in thread
From: Subrahmanian, Raj @ 2006-06-08 21:15 UTC (permalink / raw)
To: Apparao, Padmashree K, Carb, Brian A, xen-devel
Cc: McAfee, Tommie M, Krysan, Susan, Puthiyaparambil, Aravindh,
Vessey, Bruce A
[-- Attachment #1.1: Type: text/plain, Size: 4822 bytes --]
Padma,
Shouldn't the xm vcpu-set work regardless of the dom0-cpus value?
Raj
________________________________
From: Apparao, Padmashree K [mailto:padmashree.k.apparao@intel.com]
Sent: Thursday, June 08, 2006 5:03 PM
To: Carb, Brian A; xen-devel@lists.xensource.com
Cc: McAfee, Tommie M; Krysan, Susan; Puthiyaparambil, Aravindh;
Subrahmanian, Raj; Vessey, Bruce A
Subject: RE: [Xen-devel] Unable to set dom0 vcpu to 1
Set (dom-cpus=1) in etc/xen/xend-config.sxp.
- Padma
-
________________________________
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Carb, Brian
A
Sent: Thursday, June 08, 2006 1:12 PM
To: xen-devel@lists.xensource.com
Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh;
Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A
Subject: [Xen-devel] Unable to set dom0 vcpu to 1
Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16
procs, 16 GB RAM under SLES 10 RC1.
At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp
file, and dom0 uses all 16 cpus. 'xm top' shows 16 procs, 'xm list'
shows dom0 using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned to
dom0.
When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0 1',
the number of procs, as shown by /proc/cpuinfo, reduces to 9. 'xm list'
shows dom0 using 9 procs, and 'xm vcpu-list' shows that procs 1-7 have
been released, while procs 0 and 8 through 15 are still assigned.
Nothing is shown on the serial port.
Can anyone shed some light on why the rest of the procs might not be
released? Thanks.
The following is output from 'xm list' and 'xm vcpu-list':
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 16 r----- 849.3
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 438.1 any cpu
Domain-0 0 1 1 -b- 4.0 any cpu
Domain-0 0 2 2 -b- 5.4 any cpu
Domain-0 0 3 3 -b- 4.5 any cpu
Domain-0 0 4 4 -b- 8.4 any cpu
Domain-0 0 5 5 -b- 6.1 any cpu
Domain-0 0 6 6 -b- 5.0 any cpu
Domain-0 0 7 7 -b- 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 14.6 any cpu
Domain-0 0 10 10 r-- 29.1 any cpu
Domain-0 0 11 11 -b- 15.1 any cpu
Domain-0 0 12 12 -b- 71.6 any cpu
Domain-0 0 13 13 -b- 177.0 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 21.9 any cpu
m0075:~ # xm vcpu-set 0 1
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 9 r----- 853.2
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 439.3 any cpu
Domain-0 0 1 - --p 4.0 any cpu
Domain-0 0 2 - --p 5.6 any cpu
Domain-0 0 3 - --p 4.5 any cpu
Domain-0 0 4 - --p 8.4 any cpu
Domain-0 0 5 - --p 6.1 any cpu
Domain-0 0 6 - --p 5.0 any cpu
Domain-0 0 7 - --p 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 15.0 any cpu
Domain-0 0 10 10 r-- 29.3 any cpu
Domain-0 0 11 11 -b- 15.2 any cpu
Domain-0 0 12 12 -b- 72.7 any cpu
Domain-0 0 13 13 -b- 177.5 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 22.0 any cpu
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
[-- Attachment #1.2: Type: text/html, Size: 22321 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: Unable to set dom0 vcpu to 1
@ 2006-06-08 21:03 Apparao, Padmashree K
0 siblings, 0 replies; 10+ messages in thread
From: Apparao, Padmashree K @ 2006-06-08 21:03 UTC (permalink / raw)
To: Carb, Brian A, xen-devel
Cc: Subrahmanian, Raj, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Vessey, Bruce A
[-- Attachment #1.1: Type: text/plain, Size: 4382 bytes --]
Set (dom-cpus=1) in etc/xen/xend-config.sxp.
- Padma
-
________________________________
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Carb, Brian
A
Sent: Thursday, June 08, 2006 1:12 PM
To: xen-devel@lists.xensource.com
Cc: McAfee, Tommie M; Krysan,Susan; Puthiyaparambil,Aravindh;
Subrahmanian,Raj; Vessey,Bruce A; Carb, Brian A
Subject: [Xen-devel] Unable to set dom0 vcpu to 1
Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64, 16
procs, 16 GB RAM under SLES 10 RC1.
At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp
file, and dom0 uses all 16 cpus. 'xm top' shows 16 procs, 'xm list'
shows dom0 using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned to
dom0.
When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0 1',
the number of procs, as shown by /proc/cpuinfo, reduces to 9. 'xm list'
shows dom0 using 9 procs, and 'xm vcpu-list' shows that procs 1-7 have
been released, while procs 0 and 8 through 15 are still assigned.
Nothing is shown on the serial port.
Can anyone shed some light on why the rest of the procs might not be
released? Thanks.
The following is output from 'xm list' and 'xm vcpu-list':
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 16 r----- 849.3
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 438.1 any cpu
Domain-0 0 1 1 -b- 4.0 any cpu
Domain-0 0 2 2 -b- 5.4 any cpu
Domain-0 0 3 3 -b- 4.5 any cpu
Domain-0 0 4 4 -b- 8.4 any cpu
Domain-0 0 5 5 -b- 6.1 any cpu
Domain-0 0 6 6 -b- 5.0 any cpu
Domain-0 0 7 7 -b- 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 14.6 any cpu
Domain-0 0 10 10 r-- 29.1 any cpu
Domain-0 0 11 11 -b- 15.1 any cpu
Domain-0 0 12 12 -b- 71.6 any cpu
Domain-0 0 13 13 -b- 177.0 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 21.9 any cpu
m0075:~ # xm vcpu-set 0 1
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 9 r----- 853.2
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 439.3 any cpu
Domain-0 0 1 - --p 4.0 any cpu
Domain-0 0 2 - --p 5.6 any cpu
Domain-0 0 3 - --p 4.5 any cpu
Domain-0 0 4 - --p 8.4 any cpu
Domain-0 0 5 - --p 6.1 any cpu
Domain-0 0 6 - --p 5.0 any cpu
Domain-0 0 7 - --p 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 15.0 any cpu
Domain-0 0 10 10 r-- 29.3 any cpu
Domain-0 0 11 11 -b- 15.2 any cpu
Domain-0 0 12 12 -b- 72.7 any cpu
Domain-0 0 13 13 -b- 177.5 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 22.0 any cpu
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
[-- Attachment #1.2: Type: text/html, Size: 21333 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Unable to set dom0 vcpu to 1
@ 2006-06-08 20:11 Carb, Brian A
2006-06-12 15:11 ` Ryan Harper
0 siblings, 1 reply; 10+ messages in thread
From: Carb, Brian A @ 2006-06-08 20:11 UTC (permalink / raw)
To: xen-devel
Cc: McAfee, Tommie M, Krysan, Susan, Puthiyaparambil, Aravindh,
Subrahmanian, Raj, Vessey, Bruce A, Carb, Brian A
[-- Attachment #1.1: Type: text/plain, Size: 3834 bytes --]
> Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64,
> 16 procs, 16 GB RAM under SLES 10 RC1.
>
> At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp
> file, and dom0 uses all 16 cpus. 'xm top' shows 16 procs, 'xm list'
> shows dom0 using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned to
> dom0.
>
> When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0 1',
> the number of procs, as shown by /proc/cpuinfo, reduces to 9. 'xm
> list' shows dom0 using 9 procs, and 'xm vcpu-list' shows that procs
> 1-7 have been released, while procs 0 and 8 through 15 are still
> assigned. Nothing is shown on the serial port.
>
> Can anyone shed some light on why the rest of the procs might not be
> released? Thanks.
>
The following is output from 'xm list' and 'xm vcpu-list':
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 16 r----- 849.3
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 438.1 any cpu
Domain-0 0 1 1 -b- 4.0 any cpu
Domain-0 0 2 2 -b- 5.4 any cpu
Domain-0 0 3 3 -b- 4.5 any cpu
Domain-0 0 4 4 -b- 8.4 any cpu
Domain-0 0 5 5 -b- 6.1 any cpu
Domain-0 0 6 6 -b- 5.0 any cpu
Domain-0 0 7 7 -b- 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 14.6 any cpu
Domain-0 0 10 10 r-- 29.1 any cpu
Domain-0 0 11 11 -b- 15.1 any cpu
Domain-0 0 12 12 -b- 71.6 any cpu
Domain-0 0 13 13 -b- 177.0 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 21.9 any cpu
m0075:~ # xm vcpu-set 0 1
m0075:~ # xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 15912 9 r----- 853.2
m0075:~ # xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 439.3 any cpu
Domain-0 0 1 - --p 4.0 any cpu
Domain-0 0 2 - --p 5.6 any cpu
Domain-0 0 3 - --p 4.5 any cpu
Domain-0 0 4 - --p 8.4 any cpu
Domain-0 0 5 - --p 6.1 any cpu
Domain-0 0 6 - --p 5.0 any cpu
Domain-0 0 7 - --p 4.5 any cpu
Domain-0 0 8 8 -b- 30.3 any cpu
Domain-0 0 9 9 -b- 15.0 any cpu
Domain-0 0 10 10 r-- 29.3 any cpu
Domain-0 0 11 11 -b- 15.2 any cpu
Domain-0 0 12 12 -b- 72.7 any cpu
Domain-0 0 13 13 -b- 177.5 any cpu
Domain-0 0 14 14 -b- 14.5 any cpu
Domain-0 0 15 15 -b- 22.0 any cpu
brian carb
unisys corporation - malvern, pa
brian.carb@unisys.com
[-- Attachment #1.2: Type: text/html, Size: 13714 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Unable to set dom0 vcpu to 1
2006-06-08 20:11 Carb, Brian A
@ 2006-06-12 15:11 ` Ryan Harper
0 siblings, 0 replies; 10+ messages in thread
From: Ryan Harper @ 2006-06-12 15:11 UTC (permalink / raw)
To: Carb, Brian A
Cc: xen-devel, McAfee, Tommie M, Krysan, Susan,
Puthiyaparambil, Aravindh, Subrahmanian, Raj, Vessey, Bruce A
* Carb, Brian A <Brian.Carb@unisys.com> [2006-06-08 15:13]:
> > Running xen-unstable changeset 10269 on a Unisys ES7000/one, x86_64,
> > 16 procs, 16 GB RAM under SLES 10 RC1.
> >
> > At startup, dom0-cpus is set to 0 (the default) in the xend-config.sxp
> > file, and dom0 uses all 16 cpus. 'xm top' shows 16 procs, 'xm list'
> > shows dom0 using 16 cpus, and 'xm vcpu-list' shows 16 cpus assigned to
> > dom0.
> >
> > When we try to reduce the procs for dom0 to 1 using 'xm vcpu-set 0 1',
> > the number of procs, as shown by /proc/cpuinfo, reduces to 9. 'xm
> > list' shows dom0 using 9 procs, and 'xm vcpu-list' shows that procs
> > 1-7 have been released, while procs 0 and 8 through 15 are still
> > assigned. Nothing is shown on the serial port.
> >
> > Can anyone shed some light on why the rest of the procs might not be
> > released? Thanks.
On large smp boxes, it takes some time to unplug all of the processors as any
processes that are running on those cpus have to be migrated to another cpu.
On a 32-way, it took 15 seconds to go from 32 to 1. As I ran xm list, you can
see the number of active vcpus reduce over time.
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 21 r----- 4494.6
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 19 r----- 4495.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 17 r----- 4496.2
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 16 r----- 4496.8
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 16 r----- 4497.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 15 r----- 4497.6
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 14 r----- 4498.1
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 13 r----- 4498.6
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 12 r----- 4499.2
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 12 r----- 4499.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 10 r----- 4500.3
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 10 r----- 4500.7
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 9 r----- 4501.0
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 8 r----- 4501.5
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 7 r----- 4502.0
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 6 r----- 4502.5
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 4 r----- 4503.4
[root@x460-3 ~]# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 507 1 r----- 4504.4
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2006-06-15 16:16 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-06-12 15:43 Unable to set dom0 vcpu to 1 Carb, Brian A
2006-06-12 15:50 ` Ryan Harper
-- strict thread matches above, loose matches on Subject: below --
2006-06-15 15:11 Carb, Brian A
2006-06-15 16:16 ` Ryan Harper
2006-06-08 21:20 Apparao, Padmashree K
2006-06-12 19:06 ` Pablo Montesinos
2006-06-08 21:15 Subrahmanian, Raj
2006-06-08 21:03 Apparao, Padmashree K
2006-06-08 20:11 Carb, Brian A
2006-06-12 15:11 ` Ryan Harper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.