Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Tue, Feb 24, 2015 at 12:41:01PM +0000, Anthony PERARD wrote:
> Hi,
> 
> A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
> of online cpu of a host. But this API is not implemented for the libvirt
> xen driver.
> 
> The commit:
>   Add handling for offlined CPUs to the nova libvirt driver.
> https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086

FWIW, this should not impact Xen based on my understanding. The code
path in question should only be used when Nova is setup todo NUMA
pinning support, and that is not supported with Xen in OpenStack,
only KVM.  Did it actually cause failures for you, or are you simply
keeping track of all used APIs in Nova as a sanity check ?

> Is there a need to use this under Xen? (Is it possible to have offline
> CPU?).
> What libxl API those provide this information, if it exist?
> 
> I found libxl_get_online_cpus() but that not enough. They want a
> bitmap.


Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Ian Campbell]

On Tue, 2015-02-24 at 12:41 +0000, Anthony PERARD wrote:
> Hi,
> 
> A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
> of online cpu of a host. But this API is not implemented for the libvirt
> xen driver.
> 
> The commit:
>   Add handling for offlined CPUs to the nova libvirt driver.
> https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086
> 
> Is there a need to use this under Xen? (Is it possible to have offline
> CPU?).

Yes, I think so. No idea how you use it though.

> What libxl API those provide this information, if it exist?
> 
> I found libxl_get_online_cpus() but that not enough. They want a
> bitmap.

I think that is all which currently exists, at least at the libxl level,
you may need to add a new interface.

It'd be worth looking into the various host numa interfaces -- perhaps
one of them indirectly exposes what you want?

Ian.

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Anthony PERARD]

On Tue, Feb 24, 2015 at 12:46:44PM +0000, Daniel P. Berrange wrote:
> On Tue, Feb 24, 2015 at 12:41:01PM +0000, Anthony PERARD wrote:
> > Hi,
> > 
> > A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
> > of online cpu of a host. But this API is not implemented for the libvirt
> > xen driver.
> > 
> > The commit:
> >   Add handling for offlined CPUs to the nova libvirt driver.
> > https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086
> 
> FWIW, this should not impact Xen based on my understanding. The code
> path in question should only be used when Nova is setup todo NUMA
> pinning support, and that is not supported with Xen in OpenStack,
> only KVM.  Did it actually cause failures for you, or are you simply
> keeping track of all used APIs in Nova as a sanity check ?

It prevent nova from starting. I do the setup with DevStack.

The error:
libvirtError: this function is not supported by the connection driver: virNodeGetCPUMap

And a part of the traceback:
  File "/opt/stack/nova/nova/openstack/common/service.py", line 491, in run_service
    service.start()
  File "/opt/stack/nova/nova/service.py", line 181, in start
    self.manager.pre_start_hook()
  File "/opt/stack/nova/nova/compute/manager.py", line 1188, in pre_start_hook
    self.update_available_resource(nova.context.get_admin_context())
  File "/opt/stack/nova/nova/compute/manager.py", line 6062, in update_available_resource
    rt.update_available_resource(context)
  File "/opt/stack/nova/nova/compute/resource_tracker.py", line 315, in update_available_resource
    resources = self.driver.get_available_resource(self.nodename)
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4896, in get_available_resource
    numa_topology = self._get_host_numa_topology()
  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4749, in _get_host_numa_topology
    online_cpus = self._host.get_online_cpus()
  File "/opt/stack/nova/nova/virt/libvirt/host.py", line 599, in get_online_cpus
    (cpus, cpu_map, online) = self.get_connection().getCPUMap()

I'll look into why nova is going through NUMA code paths then.

Thanks for your information,

-- 
Anthony PERARD

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Tue, Feb 24, 2015 at 01:15:57PM +0000, Anthony PERARD wrote:
> On Tue, Feb 24, 2015 at 12:46:44PM +0000, Daniel P. Berrange wrote:
> > On Tue, Feb 24, 2015 at 12:41:01PM +0000, Anthony PERARD wrote:
> > > Hi,
> > > 
> > > A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
> > > of online cpu of a host. But this API is not implemented for the libvirt
> > > xen driver.
> > > 
> > > The commit:
> > >   Add handling for offlined CPUs to the nova libvirt driver.
> > > https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086
> > 
> > FWIW, this should not impact Xen based on my understanding. The code
> > path in question should only be used when Nova is setup todo NUMA
> > pinning support, and that is not supported with Xen in OpenStack,
> > only KVM.  Did it actually cause failures for you, or are you simply
> > keeping track of all used APIs in Nova as a sanity check ?
> 
> It prevent nova from starting. I do the setup with DevStack.
> 
> The error:
> libvirtError: this function is not supported by the connection driver: virNodeGetCPUMap
> 
> And a part of the traceback:
>   File "/opt/stack/nova/nova/openstack/common/service.py", line 491, in run_service
>     service.start()
>   File "/opt/stack/nova/nova/service.py", line 181, in start
>     self.manager.pre_start_hook()
>   File "/opt/stack/nova/nova/compute/manager.py", line 1188, in pre_start_hook
>     self.update_available_resource(nova.context.get_admin_context())
>   File "/opt/stack/nova/nova/compute/manager.py", line 6062, in update_available_resource
>     rt.update_available_resource(context)
>   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 315, in update_available_resource
>     resources = self.driver.get_available_resource(self.nodename)
>   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4896, in get_available_resource
>     numa_topology = self._get_host_numa_topology()
>   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4749, in _get_host_numa_topology
>     online_cpus = self._host.get_online_cpus()
>   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 599, in get_online_cpus
>     (cpus, cpu_map, online) = self.get_connection().getCPUMap()
> 
> I'll look into why nova is going through NUMA code paths then.

Oh damn, yes, I understand why now. Please file a bug against Nova for
this, as we must fix it as a high pripority. It was certainly not my
intention to break Xen when I approved this change

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Anthony PERARD]

On Tue, Feb 24, 2015 at 01:22:19PM +0000, Daniel P. Berrange wrote:
> On Tue, Feb 24, 2015 at 01:15:57PM +0000, Anthony PERARD wrote:
> > On Tue, Feb 24, 2015 at 12:46:44PM +0000, Daniel P. Berrange wrote:
> > > On Tue, Feb 24, 2015 at 12:41:01PM +0000, Anthony PERARD wrote:
> > > > Hi,
> > > > 
> > > > A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
> > > > of online cpu of a host. But this API is not implemented for the libvirt
> > > > xen driver.
> > > > 
> > > > The commit:
> > > >   Add handling for offlined CPUs to the nova libvirt driver.
> > > > https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086
> > > 
> > > FWIW, this should not impact Xen based on my understanding. The code
> > > path in question should only be used when Nova is setup todo NUMA
> > > pinning support, and that is not supported with Xen in OpenStack,
> > > only KVM.  Did it actually cause failures for you, or are you simply
> > > keeping track of all used APIs in Nova as a sanity check ?
> > 
> > It prevent nova from starting. I do the setup with DevStack.
> > 
> > The error:
> > libvirtError: this function is not supported by the connection driver: virNodeGetCPUMap
> > 
> > And a part of the traceback:
> >   File "/opt/stack/nova/nova/openstack/common/service.py", line 491, in run_service
> >     service.start()
> >   File "/opt/stack/nova/nova/service.py", line 181, in start
> >     self.manager.pre_start_hook()
> >   File "/opt/stack/nova/nova/compute/manager.py", line 1188, in pre_start_hook
> >     self.update_available_resource(nova.context.get_admin_context())
> >   File "/opt/stack/nova/nova/compute/manager.py", line 6062, in update_available_resource
> >     rt.update_available_resource(context)
> >   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 315, in update_available_resource
> >     resources = self.driver.get_available_resource(self.nodename)
> >   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4896, in get_available_resource
> >     numa_topology = self._get_host_numa_topology()
> >   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4749, in _get_host_numa_topology
> >     online_cpus = self._host.get_online_cpus()
> >   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 599, in get_online_cpus
> >     (cpus, cpu_map, online) = self.get_connection().getCPUMap()
> > 
> > I'll look into why nova is going through NUMA code paths then.
> 
> Oh damn, yes, I understand why now. Please file a bug against Nova for
> this, as we must fix it as a high pripority. It was certainly not my
> intention to break Xen when I approved this change

Here is the bug report: https://bugs.launchpad.net/nova/+bug/1425115

Regards,

-- 
Anthony PERARD

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Tue, Feb 24, 2015 at 03:00:16PM +0000, Anthony PERARD wrote:
> On Tue, Feb 24, 2015 at 01:22:19PM +0000, Daniel P. Berrange wrote:
> > On Tue, Feb 24, 2015 at 01:15:57PM +0000, Anthony PERARD wrote:
> > > On Tue, Feb 24, 2015 at 12:46:44PM +0000, Daniel P. Berrange wrote:
> > > > On Tue, Feb 24, 2015 at 12:41:01PM +0000, Anthony PERARD wrote:
> > > > > Hi,
> > > > > 
> > > > > A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
> > > > > of online cpu of a host. But this API is not implemented for the libvirt
> > > > > xen driver.
> > > > > 
> > > > > The commit:
> > > > >   Add handling for offlined CPUs to the nova libvirt driver.
> > > > > https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086
> > > > 
> > > > FWIW, this should not impact Xen based on my understanding. The code
> > > > path in question should only be used when Nova is setup todo NUMA
> > > > pinning support, and that is not supported with Xen in OpenStack,
> > > > only KVM.  Did it actually cause failures for you, or are you simply
> > > > keeping track of all used APIs in Nova as a sanity check ?
> > > 
> > > It prevent nova from starting. I do the setup with DevStack.
> > > 
> > > The error:
> > > libvirtError: this function is not supported by the connection driver: virNodeGetCPUMap
> > > 
> > > And a part of the traceback:
> > >   File "/opt/stack/nova/nova/openstack/common/service.py", line 491, in run_service
> > >     service.start()
> > >   File "/opt/stack/nova/nova/service.py", line 181, in start
> > >     self.manager.pre_start_hook()
> > >   File "/opt/stack/nova/nova/compute/manager.py", line 1188, in pre_start_hook
> > >     self.update_available_resource(nova.context.get_admin_context())
> > >   File "/opt/stack/nova/nova/compute/manager.py", line 6062, in update_available_resource
> > >     rt.update_available_resource(context)
> > >   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 315, in update_available_resource
> > >     resources = self.driver.get_available_resource(self.nodename)
> > >   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4896, in get_available_resource
> > >     numa_topology = self._get_host_numa_topology()
> > >   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4749, in _get_host_numa_topology
> > >     online_cpus = self._host.get_online_cpus()
> > >   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 599, in get_online_cpus
> > >     (cpus, cpu_map, online) = self.get_connection().getCPUMap()
> > > 
> > > I'll look into why nova is going through NUMA code paths then.
> > 
> > Oh damn, yes, I understand why now. Please file a bug against Nova for
> > this, as we must fix it as a high pripority. It was certainly not my
> > intention to break Xen when I approved this change
> 
> Here is the bug report: https://bugs.launchpad.net/nova/+bug/1425115

If you are able to test a fix easily then try the patch here:

  https://review.openstack.org/#/c/159106/

If you're able to add a comment to the review indicating that you've
confirmed it fixes Xen, that'd be useful, since there's no automated
testing of Xen that reports on reviews.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Manish Jaggi]

On Tuesday 24 February 2015 08:30 PM, Anthony PERARD wrote:
> On Tue, Feb 24, 2015 at 01:22:19PM +0000, Daniel P. Berrange wrote:
>> On Tue, Feb 24, 2015 at 01:15:57PM +0000, Anthony PERARD wrote:
>>> On Tue, Feb 24, 2015 at 12:46:44PM +0000, Daniel P. Berrange wrote:
>>>> On Tue, Feb 24, 2015 at 12:41:01PM +0000, Anthony PERARD wrote:
>>>>> Hi,
>>>>>
>>>>> A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
>>>>> of online cpu of a host. But this API is not implemented for the libvirt
>>>>> xen driver.
>>>>>
>>>>> The commit:
>>>>>    Add handling for offlined CPUs to the nova libvirt driver.
>>>>> https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086
>>>> FWIW, this should not impact Xen based on my understanding. The code
>>>> path in question should only be used when Nova is setup todo NUMA
>>>> pinning support, and that is not supported with Xen in OpenStack,
>>>> only KVM.  Did it actually cause failures for you, or are you simply
>>>> keeping track of all used APIs in Nova as a sanity check ?
>>> It prevent nova from starting. I do the setup with DevStack.
>>>
>>> The error:
>>> libvirtError: this function is not supported by the connection driver: virNodeGetCPUMap
>>>
>>> And a part of the traceback:
>>>    File "/opt/stack/nova/nova/openstack/common/service.py", line 491, in run_service
>>>      service.start()
>>>    File "/opt/stack/nova/nova/service.py", line 181, in start
>>>      self.manager.pre_start_hook()
>>>    File "/opt/stack/nova/nova/compute/manager.py", line 1188, in pre_start_hook
>>>      self.update_available_resource(nova.context.get_admin_context())
>>>    File "/opt/stack/nova/nova/compute/manager.py", line 6062, in update_available_resource
>>>      rt.update_available_resource(context)
>>>    File "/opt/stack/nova/nova/compute/resource_tracker.py", line 315, in update_available_resource
>>>      resources = self.driver.get_available_resource(self.nodename)
>>>    File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4896, in get_available_resource
>>>      numa_topology = self._get_host_numa_topology()
>>>    File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4749, in _get_host_numa_topology
>>>      online_cpus = self._host.get_online_cpus()
>>>    File "/opt/stack/nova/nova/virt/libvirt/host.py", line 599, in get_online_cpus
>>>      (cpus, cpu_map, online) = self.get_connection().getCPUMap()
>>>
>>> I'll look into why nova is going through NUMA code paths then.
>> Oh damn, yes, I understand why now. Please file a bug against Nova for
>> this, as we must fix it as a high pripority. It was certainly not my
>> intention to break Xen when I approved this change
I applied the patch, not getting this python libvirtError, but libvirt 
deamon is throwing error.
2015-03-24 08:46:31.169+0000: 1030: error : virNodeGetCPUMap:1342 : this 
function is not supported by the connection driver: virNodeGetCPUMap

> Here is the bug report: https://bugs.launchpad.net/nova/+bug/1425115
>
> Regards,
>

Re: libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1176 bytes --]

On Tue, 2015-02-24 at 13:10 +0000, Ian Campbell wrote:
> On Tue, 2015-02-24 at 12:41 +0000, Anthony PERARD wrote:

> > What libxl API those provide this information, if it exist?
> > 
> > I found libxl_get_online_cpus() but that not enough. They want a
> > bitmap.
> 
> I think that is all which currently exists, at least at the libxl level,
> you may need to add a new interface.
> 
> It'd be worth looking into the various host numa interfaces -- perhaps
> one of them indirectly exposes what you want?
> 
Given Daniel's latest emails, I'm not sure this is useful but
libxl_get_cpu_topology() should put LIBXL_CPUTOPOLOGY_INVALID_ENTRY in
all the fields of the i-eth element of the array it returns, if the
i-eth pcpu is offline (see the implementation of XEN_SYSCTL_topologyinfo
in xen/common/sysctl.c).

So, scanning that array and constructing the bitmap according to whether
or not we find that marker on the various elements would be the way to
go, I would say.

I've actually never tested this, i.e., I've never tried offlining a pcpu
on the host. I'll give it a go as soon as I find 5 minutes, and let know
if it works.

Regards,
Dario

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Ian Campbell]

On Wed, 2015-02-25 at 10:24 +0100, Dario Faggioli wrote:
> On Tue, 2015-02-24 at 13:10 +0000, Ian Campbell wrote:
> > On Tue, 2015-02-24 at 12:41 +0000, Anthony PERARD wrote:
> 
> > > What libxl API those provide this information, if it exist?
> > > 
> > > I found libxl_get_online_cpus() but that not enough. They want a
> > > bitmap.
> > 
> > I think that is all which currently exists, at least at the libxl level,
> > you may need to add a new interface.
> > 
> > It'd be worth looking into the various host numa interfaces -- perhaps
> > one of them indirectly exposes what you want?
> > 
> Given Daniel's latest emails, I'm not sure this is useful but
> libxl_get_cpu_topology() should put LIBXL_CPUTOPOLOGY_INVALID_ENTRY in
> all the fields of the i-eth element of the array it returns, if the
> i-eth pcpu is offline (see the implementation of XEN_SYSCTL_topologyinfo
> in xen/common/sysctl.c).
> 
> So, scanning that array and constructing the bitmap according to whether
> or not we find that marker on the various elements would be the way to
> go, I would say.

It could work, yes, although if there were other reasons for INVALID
entry it would fall down.

Thinking about it, it might be a better idea long term to expose a some
specific interfaces for managing or interrogating host CPU status rather
than inferring it through other means, it's not like the hypercall would
be very hard to setup and plumb through.

But as you say, Daniel's response might have made this all moot anyway,
or at least deferrable.

Ian.

> I've actually never tested this, i.e., I've never tried offlining a pcpu
> on the host. I'll give it a go as soon as I find 5 minutes, and let know
> if it works.
> 
> Regards,
> Dario

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Wed, Feb 25, 2015 at 10:24:37AM +0100, Dario Faggioli wrote:
> On Tue, 2015-02-24 at 13:10 +0000, Ian Campbell wrote:
> > On Tue, 2015-02-24 at 12:41 +0000, Anthony PERARD wrote:
> 
> > > What libxl API those provide this information, if it exist?
> > > 
> > > I found libxl_get_online_cpus() but that not enough. They want a
> > > bitmap.
> > 
> > I think that is all which currently exists, at least at the libxl level,
> > you may need to add a new interface.
> > 
> > It'd be worth looking into the various host numa interfaces -- perhaps
> > one of them indirectly exposes what you want?
> > 
> Given Daniel's latest emails, I'm not sure this is useful but
> libxl_get_cpu_topology() should put LIBXL_CPUTOPOLOGY_INVALID_ENTRY in
> all the fields of the i-eth element of the array it returns, if the
> i-eth pcpu is offline (see the implementation of XEN_SYSCTL_topologyinfo
> in xen/common/sysctl.c).
> 
> So, scanning that array and constructing the bitmap according to whether
> or not we find that marker on the various elements would be the way to
> go, I would say.
> 
> I've actually never tested this, i.e., I've never tried offlining a pcpu
> on the host. I'll give it a go as soon as I find 5 minutes, and let know
> if it works.

FWIW, this code in openstack was only added for benefit of s390
architecture where apparently it is common to have hosts with
CPUs offlined. Presumably you have to pay IBM for each extra
CPU you turn online :)

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Ian Campbell]

On Wed, 2015-02-25 at 15:03 +0000, Daniel P. Berrange wrote:
> FWIW, this code in openstack was only added for benefit of s390
> architecture where apparently it is common to have hosts with
> CPUs offlined. Presumably you have to pay IBM for each extra
> CPU you turn online :)

Presumably :-)

OOI, why does the code care which CPUs are online rather than just the
total number (IOW why a bitmap)?

Ian.

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Daniel P. Berrange]

On Wed, Feb 25, 2015 at 03:13:36PM +0000, Ian Campbell wrote:
> On Wed, 2015-02-25 at 15:03 +0000, Daniel P. Berrange wrote:
> > FWIW, this code in openstack was only added for benefit of s390
> > architecture where apparently it is common to have hosts with
> > CPUs offlined. Presumably you have to pay IBM for each extra
> > CPU you turn online :)
> 
> Presumably :-)
> 
> OOI, why does the code care which CPUs are online rather than just the
> total number (IOW why a bitmap)?

When doing strict CPU pinning,the openstack scheduler needs to have the
list of all pCPUs available in the host. It then tries to place guests
on pCPUs such that the guest does not span across NUMA nodes. To do this
it needs to know which particular pCPUs in the host are available. So we
need the full bitmap rather than just a total count.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [libvirt] libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Ian Campbell]

On Wed, 2015-02-25 at 15:20 +0000, Daniel P. Berrange wrote:
> On Wed, Feb 25, 2015 at 03:13:36PM +0000, Ian Campbell wrote:
> > On Wed, 2015-02-25 at 15:03 +0000, Daniel P. Berrange wrote:
> > > FWIW, this code in openstack was only added for benefit of s390
> > > architecture where apparently it is common to have hosts with
> > > CPUs offlined. Presumably you have to pay IBM for each extra
> > > CPU you turn online :)
> > 
> > Presumably :-)
> > 
> > OOI, why does the code care which CPUs are online rather than just the
> > total number (IOW why a bitmap)?
> 
> When doing strict CPU pinning,the openstack scheduler needs to have the
> list of all pCPUs available in the host. It then tries to place guests
> on pCPUs such that the guest does not span across NUMA nodes. To do this
> it needs to know which particular pCPUs in the host are available. So we
> need the full bitmap rather than just a total count.

Makes perfect sense, thanks.

Ian.

libvirt/libxl implemetation of get_online_cpu / virNodeGetCPUMap?

[Anthony PERARD]

Hi,

A recent OpenStack nova commit make use of virNodeGetCPUMap to get the list
of online cpu of a host. But this API is not implemented for the libvirt
xen driver.

The commit:
  Add handling for offlined CPUs to the nova libvirt driver.
https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=0696a5cd5f0fdc08951a074961bb8ce0c3310086

Is there a need to use this under Xen? (Is it possible to have offline
CPU?).
What libxl API those provide this information, if it exist?

I found libxl_get_online_cpus() but that not enough. They want a
bitmap.

Thanks,

-- 
Anthony PERARD