RE: [PATCH] xm --version

RE: [PATCH] xm --version

[Ian Pratt]

> Ian, i look at the xen source code, and looks like "noht" 
> option is never used anywhere? also ht_per_core is always set 
> to 1? does xen really support HyperThreading now ? (i doubt that)

Xen supports hyperthreading just fine, and even does initial domain
placement using an algorithm that understands it.

In fact, it looks like we effectively lost the option to ever disable
hyperthreading when the setup code got upgraded. It would be nice if
part of your patch removed any reference to 'noht', including in the
documentation.

Thanks,
Ian

Re: [PATCH] xm --version

[aq]

On 6/7/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> > Ian, i look at the xen source code, and looks like "noht"
> > option is never used anywhere? also ht_per_core is always set
> > to 1? does xen really support HyperThreading now ? (i doubt that)
> 
> Xen supports hyperthreading just fine, and even does initial domain
> placement using an algorithm that understands it.
> 
> In fact, it looks like we effectively lost the option to ever disable
> hyperthreading when the setup code got upgraded. 

can you confirm that ht_per_core is also wrongly set? it is set (=1)
at arch/x86/setup.c or arch/ia64/smpboot.c, but never changed. but
this variable is used somewhere, and got reported in "xm info", for
example.

so all of its usage are also wrong when we use HT? (because in that
case ht_per_core must be 2)

>It would be nice if
> part of your patch removed any reference to 'noht', including in the
> documentation.

fine, i will submit a patch on this problem.

regards,
aq

RE: [PATCH] xm --version

[Ian Pratt]

> can you confirm that ht_per_core is also wrongly set? it is 
> set (=1) at arch/x86/setup.c or arch/ia64/smpboot.c, but 
> never changed. but this variable is used somewhere, and got 
> reported in "xm info", for example.
> 
> so all of its usage are also wrong when we use HT? (because 
> in that case ht_per_core must be 2)

OK, that's bad -- the code that initialised ht_per_core has also gone.

What we need to do is to take the new code from Linux 2.6.12-rc5 that
understands multicore, and correctly initialise ht_per_core and
core_per_socket. We can hardwire socket_per_node=1 for the moment.

Thanks,
Ian

Re: [PATCH] xm --version

[aq]

On 6/9/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> > can you confirm that ht_per_core is also wrongly set? it is
> > set (=1) at arch/x86/setup.c or arch/ia64/smpboot.c, but
> > never changed. but this variable is used somewhere, and got
> > reported in "xm info", for example.
> >
> > so all of its usage are also wrong when we use HT? (because
> > in that case ht_per_core must be 2)
> 
> OK, that's bad -- the code that initialised ht_per_core has also gone.
> 
> What we need to do is to take the new code from Linux 2.6.12-rc5 that
> understands multicore, and correctly initialise ht_per_core and
> core_per_socket. We can hardwire socket_per_node=1 for the moment.
> 

anybody is working on this? if no, i will take it.

regards,
aq

RE: [PATCH] xm --version

[Ian Pratt]

> > Where does 'machine' come from? Shouldn't it be x86_32?
> 
> this is not what the patch does, but in the original code. 
> actually "machine" is from "uname" syscall, run in dom0. 
> libxc just gets the result from "uname", together with 
> dom0_release, dom0_version.
> 
> should we fix it to x86_32 or x86_64?

Ideally the architecture should come from Xen rather than the dom0
kernel, but I guess this isn't a big deal right now since we don't
support running 32 bit paravirt guests on a 64 bit hypervisor, and even
if we did you probably wouldn't want to do it for dom0. 

> > 
> > Also, isn't there a tools version field we could print as well?
> > 
> yes, that is fine, but where to get the xm version? lets put 
> it somewhere into xm code? i am not sure where to put it. and 
> actually what is the current version of xm? we better ask 
> Mike to help this problem?

I guess its more interesting to know the xend or libxc version, but I
guess we can assume them to all come from the same package/rpm. We
probably need to add a version identifier to the tools. 
 
> > > cores                  : 1
> > > hyperthreads_per_core  : 1
> > 
> > I'd like to add a bit more information here, to take of 
> ccNUMA systems 
> > with multicore and hyperthreadsing, e.g. for a system with 
> 2 dual core 
> > hyperthreaded Xeons:
> > 
> > logical_cpus            : 8
> 
> sorry for my ignorance (never play with 2 or more cpus system 
> before, poor me!), how come 2 dual core hyperthreaded Xeons 
> has "8 logical cpus"? you must meant "4 logical cpus"

2 sockets * 2 cores * 2 hyperthreads = 8 logical CPUs.

Xen doesn't currently distinguish between sockets and cores, so for the
moment the interface should just return 'sockets_per_node = 4'. We can
fix Xen up later.

Ian

Re: [PATCH] xm --version

[aq]

On 6/6/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> > > Where does 'machine' come from? Shouldn't it be x86_32?
> >
> > this is not what the patch does, but in the original code.
> > actually "machine" is from "uname" syscall, run in dom0.
> > libxc just gets the result from "uname", together with
> > dom0_release, dom0_version.
> >
> > should we fix it to x86_32 or x86_64?
> 
> Ideally the architecture should come from Xen rather than the dom0
> kernel, but I guess this isn't a big deal right now since we don't
> support running 32 bit paravirt guests on a 64 bit hypervisor, and even
> if we did you probably wouldn't want to do it for dom0.
> 
> > >
> > > Also, isn't there a tools version field we could print as well?
> > >
> > yes, that is fine, but where to get the xm version? lets put
> > it somewhere into xm code? i am not sure where to put it. and
> > actually what is the current version of xm? we better ask
> > Mike to help this problem?
> 
> I guess its more interesting to know the xend or libxc version, but I
> guess we can assume them to all come from the same package/rpm. We
> probably need to add a version identifier to the tools.
> 
> > > > cores                  : 1
> > > > hyperthreads_per_core  : 1
> > >
> > > I'd like to add a bit more information here, to take of
> > ccNUMA systems
> > > with multicore and hyperthreadsing, e.g. for a system with
> > 2 dual core
> > > hyperthreaded Xeons:
> > >
> > > logical_cpus            : 8
> >
> > sorry for my ignorance (never play with 2 or more cpus system
> > before, poor me!), how come 2 dual core hyperthreaded Xeons
> > has "8 logical cpus"? you must meant "4 logical cpus"
> 
> 2 sockets * 2 cores * 2 hyperthreads = 8 logical CPUs.
> 
> Xen doesn't currently distinguish between sockets and cores, so for the
> moment the interface should just return 'sockets_per_node = 4'. We can
> fix Xen up later.
> 

Ian, i look at the xen source code, and looks like "noht" option is
never used anywhere? also ht_per_core is always set to 1? does xen
really support HyperThreading now ? (i doubt that)

regards,
aq

RE: [PATCH] xm --version

[Ian Pratt]

> This patch adds to xm a long-missing function: showing xen 
> version and few extra information (thanks go to Keir and Mark 
> for some suggestions). Here is the ouput of "xm info" after 
> applying the patch (against -unstable ChangeSet@1.1677):

Thanks, but I'd like to make another couple of suggestions:

> ---
> #xm info
> system                 : Linux
> host                   : ubuntu
> xen_release            : 3.0-devel
> xen_compile_by         : root@localdomain
> xen_compiler           : gcc version 3.3.5 (Debian 1:3.3.5-8ubuntu2)
> xen_compile_date       : Mon Jun  6 00:30:22 EST 2005
> dom0_release           : 2.6.11.10-xen0
> dom0_version           : #3 Mon Jun 6 00:32:05 EST 2005
> machine                : i686

Where does 'machine' come from? Shouldn't it be x86_32? 

Also, isn't there a tools version field we could print as well?

> cores                  : 1
> hyperthreads_per_core  : 1

I'd like to add a bit more information here, to take of ccNUMA systems
with multicore and hyperthreadsing, e.g. for a system with 2 dual core
hyperthreaded Xeons:

logical_cpus		: 8
hyperthreads_per_core	: 2
cores_per_socket		: 2
sockets_per_node		: 2
nodes				: 1

This information should reflect the running system, rather than the
capabilities of the hardware. i.e. if someone has booted with 'noht'
then hyperthreads_per_core should be 1 regardless of the hardware. This
will clean up some other code in Xen that uses the flag.

Thanks,
Ian


> cpu_mhz                : 1094
> memory                 : 511
> free_memory            : 122

Re: [PATCH] xm --version

[aq]

On 6/6/05, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> > This patch adds to xm a long-missing function: showing xen
> > version and few extra information (thanks go to Keir and Mark
> > for some suggestions). Here is the ouput of "xm info" after
> > applying the patch (against -unstable ChangeSet@1.1677):
> 
> Thanks, but I'd like to make another couple of suggestions:
> 
> > ---
> > #xm info
> > system                 : Linux
> > host                   : ubuntu
> > xen_release            : 3.0-devel
> > xen_compile_by         : root@localdomain
> > xen_compiler           : gcc version 3.3.5 (Debian 1:3.3.5-8ubuntu2)
> > xen_compile_date       : Mon Jun  6 00:30:22 EST 2005
> > dom0_release           : 2.6.11.10-xen0
> > dom0_version           : #3 Mon Jun 6 00:32:05 EST 2005
> > machine                : i686
> 
> Where does 'machine' come from? Shouldn't it be x86_32?

this is not what the patch does, but in the original code. actually
"machine" is from "uname" syscall, run in dom0. libxc just gets the
result from "uname", together with dom0_release, dom0_version.

should we fix it to x86_32 or x86_64?

> 
> Also, isn't there a tools version field we could print as well?
> 
yes, that is fine, but where to get the xm version? lets put it
somewhere into xm code? i am not sure where to put it. and actually
what is the current version of xm? we better ask Mike to help this
problem?

> > cores                  : 1
> > hyperthreads_per_core  : 1
> 
> I'd like to add a bit more information here, to take of ccNUMA systems
> with multicore and hyperthreadsing, e.g. for a system with 2 dual core
> hyperthreaded Xeons:
> 
> logical_cpus            : 8

sorry for my ignorance (never play with 2 or more cpus system before,
poor me!), how come 2 dual core hyperthreaded Xeons has "8 logical
cpus"? you must meant "4 logical cpus"

> hyperthreads_per_core   : 2
> cores_per_socket                : 2
> sockets_per_node                : 2
> nodes                           : 1
> 
> This information should reflect the running system, rather than the
> capabilities of the hardware. i.e. if someone has booted with 'noht'
> then hyperthreads_per_core should be 1 regardless of the hardware. This
> will clean up some other code in Xen that uses the flag.

i see, but how about applying this patch now, and solve what you
propose above later? i will investigate this problem.

regards,
aq

Re: [PATCH] xm --version

[Keir Fraser]

On 6 Jun 2005, at 08:47, aq wrote:

> sorry for my ignorance (never play with 2 or more cpus system before,
> poor me!), how come 2 dual core hyperthreaded Xeons has "8 logical
> cpus"? you must meant "4 logical cpus"
>
>> hyperthreads_per_core   : 2
>> cores_per_socket                : 2
>> sockets_per_node                : 2
>> nodes                           : 1

The example looks forward to physical packages containing two cores, 
each of which has two threads. So, in a dual socket system, you'd have 
2x2x2=8 logical cpus.

  -- Keir

[PATCH] xm --version

[aq]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]

This patch adds to xm a long-missing function: showing xen version and
few extra information (thanks go to Keir and Mark for some
suggestions). Here is the ouput of "xm info" after applying the patch
(against -unstable ChangeSet@1.1677):

---
#xm info
system                 : Linux
host                   : ubuntu
xen_release            : 3.0-devel
xen_compile_by         : root@localdomain
xen_compiler           : gcc version 3.3.5 (Debian 1:3.3.5-8ubuntu2)
xen_compile_date       : Mon Jun  6 00:30:22 EST 2005
dom0_release           : 2.6.11.10-xen0
dom0_version           : #3 Mon Jun 6 00:32:05 EST 2005
machine                : i686
cores                  : 1
hyperthreads_per_core  : 1
cpu_mhz                : 1094
memory                 : 511
free_memory            : 122
----

You get the point. And here is what the patch does:

- extend xen_version hypercall. the new hypercall keeps the old
behavior if executing it with 0 as its parameter. But if give it the
address of version_op_t structure (see version.h in the patch), it
returns extra stuffs like xen_extraversion, compile date, compiler,
etc...
- libxc now has a new function to get xen version (and stuffs): 
xc_xen_version()
- extend python wrapper for xc_xen_version() and use it in XendNode.py
to get xen version and stuffs.

Any comment?

Signed-off-by: Nguyen Anh Quynh <aquynh@gmail.com>

# diffstat xminfo2.patch 
 tools/libxc/xc.h                  |   11 +++++++++++
 tools/libxc/xc_misc.c             |   18 ++++++++++++++++++
 tools/python/xen/lowlevel/xc/xc.c |   29 +++++++++++++++++++++++++++++
 tools/python/xen/xend/XendNode.py |   12 ++++++++----
 xen/common/kernel.c               |   22 ++++++++++++++++++----
 xen/include/public/version.h      |   22 ++++++++++++++++++++++
 6 files changed, 106 insertions(+), 8 deletions(-)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: xminfo2.patch --]
[-- Type: text/x-patch; name="xminfo2.patch", Size: 7031 bytes --]

diff -Nurp -X dontdiff.xen xeno-today/tools/libxc/xc.h xeno.0606/tools/libxc/xc.h
--- xeno-today/tools/libxc/xc.h	2005-06-06 11:54:29.160666848 -0500
+++ xeno.0606/tools/libxc/xc.h	2005-06-06 12:32:26.210502656 -0500
@@ -25,6 +25,7 @@ typedef int64_t            s64;
 #include <xen/dom0_ops.h>
 #include <xen/event_channel.h>
 #include <xen/sched_ctl.h>
+#include <xen/version.h>
 
 /*
  *  DEFINITIONS FOR CPU BARRIERS
@@ -505,4 +506,14 @@ long xc_get_tot_pages(int xc_handle, u32
 /* Execute a privileged dom0 operation. */
 int xc_dom0_op(int xc_handle, dom0_op_t *op);
 
+/* This struct is for libxc to get xen version and extra stuffs. */ 
+typedef version_op_t xc_version_t;
+
+/**
+ * Get xen version 
+ *
+ * @return a long value which encodes XEN_VERSION and XEN_SUBVERSION on success, -1 on failure.
+ */
+long xc_xen_version(int xc_handle, xc_version_t *version); 
+ 
 #endif /* __XC_H__ */
diff -Nurp -X dontdiff.xen xeno-today/tools/libxc/xc_misc.c xeno.0606/tools/libxc/xc_misc.c
--- xeno-today/tools/libxc/xc_misc.c	2005-06-06 11:54:29.168665632 -0500
+++ xeno.0606/tools/libxc/xc_misc.c	2005-06-06 12:18:11.176487584 -0500
@@ -130,3 +130,21 @@ int xc_msr_write(int xc_handle, int cpu_
     
     return rc;
 }
+
+long xc_xen_version(int xc_handle, xc_version_t *v)
+{
+    long ret;
+    privcmd_hypercall_t hypercall;
+
+    hypercall.op = __HYPERVISOR_xen_version;
+    hypercall.arg[0] = (unsigned long)v;
+
+    if ((ret = do_xen_hypercall(xc_handle, &hypercall)) < 0)
+    {
+        if (errno == EACCES)
+            PERROR("Could not obtain xen version");
+        return -1;
+    }
+
+    return ret;
+}
diff -Nurp -X dontdiff.xen xeno-today/tools/python/xen/lowlevel/xc/xc.c xeno.0606/tools/python/xen/lowlevel/xc/xc.c
--- xeno-today/tools/python/xen/lowlevel/xc/xc.c	2005-06-06 11:54:29.197661224 -0500
+++ xeno.0606/tools/python/xen/lowlevel/xc/xc.c	2005-06-06 12:39:18.826775448 -0500
@@ -827,6 +827,29 @@ static PyObject *pyxc_domain_memory_incr
     return zero;
 }
 
+static PyObject *pyxc_xen_version(PyObject *self,
+                                    PyObject *args,
+                                    PyObject *kwds)
+{
+    xc_version_t v;
+    long ret;
+    XcObject *xc = (XcObject *)self;
+
+    if (!PyArg_ParseTuple(args, ""))
+        return NULL;
+    
+    if ((ret = xc_xen_version(xc->xc_handle, &v)) == -1)
+        return PyErr_SetFromErrno(xc_error);
+
+    return Py_BuildValue("{s:i,s:i,s:s,s:s,s:s,s:s,s:s}", 
+                         "version", ret >> 16, 
+                         "subversion", ret & 0xFFFF,
+                         "extraversion", v.extraversion,
+                         "compile_by", v.compile_by,
+                         "compile_domain", v.compile_domain,
+                         "compiler", v.compiler,
+                         "compile_date", v.compile_date);
+}
 
 static PyMethodDef pyxc_methods[] = {
     { "handle",
@@ -1110,6 +1133,12 @@ static PyMethodDef pyxc_methods[] = {
       " mem_kb [long]: .\n"
       "Returns: [int] 0 on success; -1 on error.\n" },
 
+    { "xen_version", 
+      (PyCFunction)pyxc_xen_version, 
+       METH_KEYWORDS, "\n"
+      "Get Xen version\n"
+      "Returns: [dict] xen version and extra stuffs on success; empty list on error.\n" },
+
     { NULL, NULL, 0, NULL }
 };
 
diff -Nurp -X dontdiff.xen xeno-today/tools/python/xen/xend/XendNode.py xeno.0606/tools/python/xen/xend/XendNode.py
--- xeno-today/tools/python/xen/xend/XendNode.py	2005-05-28 12:19:57.000000000 -0500
+++ xeno.0606/tools/python/xen/xend/XendNode.py	2005-06-06 12:18:11.179487128 -0500
@@ -35,10 +35,15 @@ class XendNode:
 
     def nodeinfo(self):
         (sys, host, rel, ver, mch) = os.uname()
+        xen_ver = self.xc.xen_version()
         return [['system',  sys],
-                ['host',    host],
-                ['release', rel],
-                ['version', ver],
+                ['host', host],
+                ['xen_release', "%i.%i%s" %(xen_ver['version'], xen_ver['subversion'], xen_ver['extraversion'])],
+                ['xen_compile_by', "%s@%s" %(xen_ver['compile_by'], xen_ver['compile_domain'])],
+                ['xen_compiler', xen_ver['compiler']],
+                ['xen_compile_date', xen_ver['compile_date']],
+                ['dom0_release', rel],
+                ['dom0_version', ver],
                 ['machine', mch]]
 
     def physinfo(self):
@@ -50,7 +55,6 @@ class XendNode:
                 ['free_memory', pinfo['free_pages']/256]]
         return info
         
-        
 
 def instance():
     global inst
diff -Nurp -X dontdiff.xen xeno-today/xen/common/kernel.c xeno.0606/xen/common/kernel.c
--- xeno-today/xen/common/kernel.c	2005-06-06 11:54:29.477618664 -0500
+++ xeno.0606/xen/common/kernel.c	2005-06-06 12:18:11.180486976 -0500
@@ -15,6 +15,9 @@
 #include <xen/sched.h>
 #include <asm/current.h>
 
+#include <public/version.h>
+#include <xen/string.h>
+
 void cmdline_parse(char *cmdline)
 {
     char opt[100], *optval, *p = cmdline, *q;
@@ -83,11 +86,22 @@ void cmdline_parse(char *cmdline)
  * Simple hypercalls.
  */
 
-long do_xen_version(int cmd)
+long do_xen_version(version_op_t *op)
 {
-    if ( cmd != 0 )
-        return -ENOSYS;
-    return (XEN_VERSION<<16) | (XEN_SUBVERSION);
+    version_op_t vop;
+    if (op != NULL)  /* return extra stuffs */ 
+    {
+        if (copy_from_user(&vop, op, sizeof(vop)) != 0)
+            return -EFAULT;
+        strncpy(vop.extraversion, XEN_EXTRAVERSION, sizeof(vop.extraversion));
+        strncpy(vop.compile_by, XEN_COMPILE_BY, sizeof(vop.compile_by));
+        strncpy(vop.compile_domain, XEN_COMPILE_DOMAIN, sizeof(vop.compile_domain));
+        strncpy(vop.compiler, XEN_COMPILER, sizeof(vop.compiler));
+        strncpy(vop.compile_date, XEN_COMPILE_DATE, sizeof(vop.compile_date));
+        if (copy_to_user(op, &vop, sizeof(vop)) != 0)
+            return -EFAULT;
+    }
+    return (XEN_VERSION<<16) | (XEN_SUBVERSION); /* always return version & subversion */ 
 }
 
 long do_vm_assist(unsigned int cmd, unsigned int type)
diff -Nurp -X dontdiff.xen xeno-today/xen/include/public/version.h xeno.0606/xen/include/public/version.h
--- xeno-today/xen/include/public/version.h	1969-12-31 19:00:00.000000000 -0500
+++ xeno.0606/xen/include/public/version.h	2005-06-06 12:18:11.181486824 -0500
@@ -0,0 +1,22 @@
+/******************************************************************************
+ * version.h
+ * 
+ * Xen extraversion and stuffs.
+ * 
+ * Copyright (c) 2005, Nguyen Anh Quynh <aquynh@gmail.com>
+ */
+
+#ifndef __XEN_PUBLIC_VERSION_H__
+#define __XEN_PUBLIC_VERSION_H__
+
+/* This struct is for xen hypercall only. Usually you dont need it. */ 
+typedef struct _version_op_st 
+{
+    char extraversion[8];           /*  0 */
+    char compile_by[16];            /*  8 */
+    char compile_domain[32];        /* 24 */
+    char compiler[64];              /* 56 */
+    char compile_date[32];          /* 120 */
+} PACKED version_op_t;  /* 152 bytes */
+
+#endif /* __XEN_PUBLIC_VERSION_H__ */

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel