[patch 0/8] various lscpu patches

[patch 0/8] various lscpu patches

[Heiko Carstens]

Hi Karel,

here are a couple of fixes and improvemts for lscpu on s390.
Hopefully I didn't screw up anything ;)
At least on my Thinkpad everything still works as before and the output
on s390 machines is much improved.

Thanks,
Heiko

[patch 1/8] [PATCH] lscpu: fix s390 bogomips detection coding style

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

Just make the s390 bogomips detection line look like all others.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -434,8 +434,7 @@ read_basicinfo(struct lscpu_desc *desc)
 		else if (lookup(buf, "features", &desc->flags)) ;	/* s390 */
 		else if (lookup(buf, "type", &desc->flags)) ;		/* sparc64 */
 		else if (lookup(buf, "bogomips", &desc->bogomips)) ;
-		/* S390 */
-		else if (lookup(buf, "bogomips per cpu", &desc->bogomips)) ;
+		else if (lookup(buf, "bogomips per cpu", &desc->bogomips)) ; /* s390 */
 		else
 			continue;
 	}

[patch 2/8] [PATCH] lscpu: fix cpu map array sizes

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

Completely virtualized architectures like s390 may have multiple
virtual sockets and/or cores where each of them has a different
number of cores and cpus.
So the general assumption within the allocation scheme that e.g.
each socket contains the same number of cores is not necessarily
true. To make sure the arrays are always large enough we simply
allocate enough memory so that each array could hold cpu masks
for all present cpus.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.c |   13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -660,11 +660,16 @@ read_topology(struct lscpu_desc *desc, i
 		 */
 		if (!desc->nthreads)
 			desc->nthreads = nsockets * ncores * nthreads;
+		/* For each map we make sure that it can have up to ncpus
+		 * entries. This is because we cannot reliably calculate the
+		 * number of cores, sockets and books on all architectures.
+		 * E.g. completely virtualized architectures like s390 may
+		 * have multiple sockets of different sizes.
+		 */
+		desc->coremaps = xcalloc(desc->ncpus, sizeof(cpu_set_t *));
+		desc->socketmaps = xcalloc(desc->ncpus, sizeof(cpu_set_t *));
 		if (book_siblings)
-			desc->bookmaps = xcalloc(nbooks, sizeof(cpu_set_t *));
-
-		desc->socketmaps = xcalloc(nsockets, sizeof(cpu_set_t *));
-		desc->coremaps = xcalloc(ncores * nsockets, sizeof(cpu_set_t *));
+			desc->bookmaps = xcalloc(desc->ncpus, sizeof(cpu_set_t *));
 	}
 
 	add_cpuset_to_array(desc->socketmaps, &desc->nsockets, core_siblings);

[patch 3/8] [PATCH] lscpu: fix fallback nthreads calculation

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

The fallback calculation of nthreads did not consider books.
In order to avoid division/multiply by/with zero make sure each number
used is at least "1".

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.c |   14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -649,17 +649,21 @@ read_topology(struct lscpu_desc *desc, i
 		nthreads = CPU_COUNT_S(setsize, thread_siblings);
 		/* cores within one socket */
 		ncores = CPU_COUNT_S(setsize, core_siblings) / nthreads;
-		/* number of sockets within one book */
-		nsockets = desc->ncpus / nthreads / ncores;
+		/* number of sockets within one book.
+		 * Because of odd / non-present cpu maps and to keep
+		 * calculation easy we make sure that nsockets and
+		 * nbooks is at least 1.
+		 */
+		nsockets = desc->ncpus / nthreads / ncores ?: 1;
 		/* number of books */
-		nbooks = desc->ncpus / nthreads / ncores / nsockets;
+		nbooks = desc->ncpus / nthreads / ncores / nsockets ?: 1;
 
 		/* all threads, see also read_basicinfo()
-		 * -- this is fallback for kernels where is not
+		 * -- fallback for kernels without
 		 *    /sys/devices/system/cpu/online.
 		 */
 		if (!desc->nthreads)
-			desc->nthreads = nsockets * ncores * nthreads;
+			desc->nthreads = nbooks * nsockets * ncores * nthreads;
 		/* For each map we make sure that it can have up to ncpus
 		 * entries. This is because we cannot reliably calculate the
 		 * number of cores, sockets and books on all architectures.

[patch 4/8] [PATCH] lscpu: detect IBM hypervisor

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

Detect if the hypervisor on s390 is from KVM or IBM.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.c |   22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -50,6 +50,7 @@
 #define _PATH_PROC_XENCAP	_PATH_PROC_XEN "/capabilities"
 #define _PATH_PROC_CPUINFO	"/proc/cpuinfo"
 #define _PATH_PROC_PCIDEVS	"/proc/bus/pci/devices"
+#define _PATH_PROC_SYSINFO	"/proc/sysinfo"
 
 /* virtualization types */
 enum {
@@ -69,14 +70,16 @@ enum {
 	HYPER_XEN,
 	HYPER_KVM,
 	HYPER_MSHV,
-	HYPER_VMWARE
+	HYPER_VMWARE,
+	HYPER_IBM
 };
 const char *hv_vendors[] = {
 	[HYPER_NONE]	= NULL,
 	[HYPER_XEN]	= "Xen",
 	[HYPER_KVM]	= "KVM",
 	[HYPER_MSHV]	= "Microsoft",
-	[HYPER_VMWARE]  = "VMware"
+	[HYPER_VMWARE]  = "VMware",
+	[HYPER_IBM]	= "IBM"
 };
 
 /* CPU modes */
@@ -598,6 +601,21 @@ read_hypervisor(struct lscpu_desc *desc)
 		/* Xen full-virt on non-x86_64 */
 		desc->hyper = HYPER_XEN;
 		desc->virtype = VIRT_FULL;
+	} else if (path_exist(_PATH_PROC_SYSINFO)) {
+		FILE *fd = path_fopen("r", 0, _PATH_PROC_SYSINFO);
+		char buf[BUFSIZ];
+
+		desc->hyper = HYPER_IBM;
+		desc->virtype = VIRT_FULL;
+		while (fgets(buf, sizeof(buf), fd) != NULL) {
+			if (!strstr(buf, "Control Program:"))
+				continue;
+			if (!strstr(buf, "KVM"))
+				desc->hyper = HYPER_IBM;
+			else
+				desc->hyper = HYPER_KVM;
+		}
+		fclose(fd);
 	}
 }
 

[patch 5/8] [PATCH] lscpu: use hypervisor generated topology information

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

The readable output prints also informations like cores per socket etc.
On newer kernel versions on s390 this information is available via
/proc/sysinfo. However it does not contain the layout of the guest but
the layout of the real machine.
Nevertheless this is better than random guessing with completely broken
numbers like we have it now on s390. If the information is not available
we fall back to old mechanism with more or less random numbers.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.c |   33 +++++++++++++++++++++++++++++----
 1 file changed, 29 insertions(+), 4 deletions(-)

--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -1007,13 +1007,38 @@ print_readable(struct lscpu_desc *desc,
 	}
 
 	if (desc->nsockets) {
+		int cores_per_socket, sockets_per_book, books;
+
+		cores_per_socket = sockets_per_book = books = 0;
+		/* s390 detects its cpu topology via /proc/sysinfo, if present.
+		 * Using simply the cpu topology masks in sysfs will not give
+		 * usable results since everything is virtualized. E.g.
+		 * virtual core 0 may have only 1 cpu, but virtual core 2 may
+		 * five cpus.
+		 * If the cpu topology is not exported (e.g. 2nd level guest)
+		 * fall back to old calculation scheme.
+		 */
+		if (path_exist(_PATH_PROC_SYSINFO)) {
+			FILE *fd = path_fopen("r", 0, _PATH_PROC_SYSINFO);
+			char buf[BUFSIZ];
+			int t0, t1, t2;
+
+			while (fgets(buf, sizeof(buf), fd) != NULL) {
+				if (sscanf(buf, "CPU Topology SW:%d%d%d%d%d%d",
+					   &t0, &t1, &t2, &books, &sockets_per_book,
+					   &cores_per_socket) == 6)
+					break;
+			}
+		}
 		print_n(_("Thread(s) per core:"), desc->nthreads / desc->ncores);
-		print_n(_("Core(s) per socket:"), desc->ncores / desc->nsockets);
+		print_n(_("Core(s) per socket:"),
+			cores_per_socket ?: desc->ncores / desc->nsockets);
 		if (desc->nbooks) {
-			print_n(_("Socket(s) per book:"), desc->nsockets / desc->nbooks);
-			print_n(_("Book(s):"), desc->nbooks);
+			print_n(_("Socket(s) per book:"),
+				sockets_per_book ?: desc->nsockets / desc->nbooks);
+			print_n(_("Book(s):"), books ?: desc->nbooks);
 		} else {
-			print_n(_("Socket(s):"), desc->nsockets);
+			print_n(_("Socket(s):"), sockets_per_book ?: desc->nsockets);
 		}
 	}
 	if (desc->nnodes)

[patch 6/8] [PATCH] lscpu: show dispatching mode

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

A virtual guest my run in either "horiziontal" or "vertical" mode
which is the mode in which the underlying hypervisor schedules the
guest cpu.
Since this is of interest print this in the readable output.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.c |   20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -97,6 +97,17 @@ struct cpu_cache {
 	cpu_set_t	**sharedmaps;
 };
 
+/* dispatching modes */
+enum {
+	DISP_HORIZONTAL = 0,
+	DISP_VERTICAL	= 1
+};
+
+const char *disp_modes[] = {
+	[DISP_HORIZONTAL]	= N_("horizontal"),
+	[DISP_VERTICAL]		= N_("vertical")
+};
+
 /* global description */
 struct lscpu_desc {
 	char	*arch;
@@ -110,6 +121,7 @@ struct lscpu_desc {
 	char	*stepping;
 	char    *bogomips;
 	char	*flags;
+	int	dispatching;	/* none, horizontal or vertical */
 	int	mode;		/* rm, lm or/and tm */
 
 	int		ncpus;		/* number of CPUs */
@@ -478,6 +490,12 @@ read_basicinfo(struct lscpu_desc *desc)
 		desc->online = path_cpulist(_PATH_SYS_SYSTEM "/cpu/online");
 		desc->nthreads = CPU_COUNT_S(setsize, desc->online);
 	}
+
+	/* get dispatching mode */
+	if (path_exist(_PATH_SYS_SYSTEM "/cpu/dispatching"))
+		desc->dispatching = path_getnum(_PATH_SYS_SYSTEM "/cpu/dispatching");
+	else
+		desc->dispatching = -1;
 }
 
 static int
@@ -1065,6 +1083,8 @@ print_readable(struct lscpu_desc *desc,
 		print_s(_("Hypervisor vendor:"), hv_vendors[desc->hyper]);
 		print_s(_("Virtualization type:"), virt_types[desc->virtype]);
 	}
+	if (desc->dispatching >= 0)
+		print_s(_("Dispatching mode:"), disp_modes[desc->dispatching]);
 	if (desc->ncaches) {
 		char buf[512];
 		int i;

[patch 7/8] [PATCH] lscpu: add cpu polarization to parseable output

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

When running in different dispatching mode the virtual cpus may
have different polarizations.
E.g. in "vertical" mode cpus may have a polarization of "vertical:high"
which means the virtual cpu has dedicated physical cpu assigned.
Print this information in the parsable output.

Note that this breaks the current rule that
a) the parseable output contains only numbers
b) these numbers are equal or increased in each line

Since however this new item must be selected with the "list" argument
this shouldn't be a problem.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.1 |    2 +-
 sys-utils/lscpu.c |   53 +++++++++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 52 insertions(+), 3 deletions(-)

--- a/sys-utils/lscpu.1
+++ b/sys-utils/lscpu.1
@@ -32,7 +32,7 @@ separate CPU cache columns. If no CPU ca
 columns are not printed at all.
 
 The \fIlist\fP argument is comma delimited list of the columns. Currently
-supported are CPU, Core, Node, Socket, Book and Cache columns. If the
+supported are CPU, Core, Node, Socket, Book, Cache and Polarization columns. If the
 \fIlist\fP argument is given then always all requested columns are printed in
 the defined order. The Cache columns are separated by ':'.
 
--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -108,6 +108,23 @@ const char *disp_modes[] = {
 	[DISP_VERTICAL]		= N_("vertical")
 };
 
+/* cpu polarization */
+enum {
+	POLAR_UNKNOWN	= 0,
+	POLAR_VLOW,
+	POLAR_VMEDIUM,
+	POLAR_VHIGH,
+	POLAR_HORIZONTAL
+};
+
+const char *polar_modes[] = {
+	[POLAR_UNKNOWN]		= "U",
+	[POLAR_VLOW]		= "VL",
+	[POLAR_VMEDIUM]		= "VM",
+	[POLAR_VHIGH]		= "VH",
+	[POLAR_HORIZONTAL]	= "H"
+};
+
 /* global description */
 struct lscpu_desc {
 	char	*arch;
@@ -149,6 +166,8 @@ struct lscpu_desc {
 
 	int		ncaches;
 	struct cpu_cache *caches;
+
+	int		*polarization;	/* cpu polarization */
 };
 
 static size_t sysrootlen;
@@ -179,7 +198,8 @@ enum {
 	COL_SOCKET,
 	COL_NODE,
 	COL_BOOK,
-	COL_CACHE
+	COL_CACHE,
+	COL_POLARIZATION
 };
 
 static const char *colnames[] =
@@ -189,7 +209,8 @@ static const char *colnames[] =
 	[COL_SOCKET] = "Socket",
 	[COL_NODE] = "Node",
 	[COL_BOOK] = "Book",
-	[COL_CACHE] = "Cache"
+	[COL_CACHE] = "Cache",
+	[COL_POLARIZATION] = "Polarization"
 };
 
 
@@ -717,6 +738,29 @@ read_topology(struct lscpu_desc *desc, i
 	if (book_siblings)
 		add_cpuset_to_array(desc->bookmaps, &desc->nbooks, book_siblings);
 }
+static void
+read_polarization(struct lscpu_desc *desc, int num)
+{
+	char mode[64];
+
+	if (desc->dispatching < 0)
+		return;
+	if (!path_exist(_PATH_SYS_CPU "/cpu%d/polarization", num))
+		return;
+	if (!desc->polarization)
+		desc->polarization = xcalloc(desc->ncpus, sizeof(int));
+	path_getstr(mode, sizeof(mode), _PATH_SYS_CPU "/cpu%d/polarization", num);
+	if (strncmp(mode, "vertical:low", sizeof(mode)) == 0)
+		desc->polarization[num] = POLAR_VLOW;
+	else if (strncmp(mode, "vertical:medium", sizeof(mode)) == 0)
+		desc->polarization[num] = POLAR_VMEDIUM;
+	else if (strncmp(mode, "vertical:high", sizeof(mode)) == 0)
+		desc->polarization[num] = POLAR_VHIGH;
+	else if (strncmp(mode, "horizontal", sizeof(mode)) == 0)
+		desc->polarization[num] = POLAR_HORIZONTAL;
+	else
+		desc->polarization[num] = POLAR_UNKNOWN;
+}
 
 static int
 cachecmp(const void *a, const void *b)
@@ -868,6 +912,10 @@ print_parsable_cell(struct lscpu_desc *d
 				putchar(',');
 		}
 		break;
+	case COL_POLARIZATION:
+		if (desc->polarization)
+			printf("%s", polar_modes[desc->polarization[i]]);
+		break;
 	}
 }
 
@@ -1181,6 +1229,7 @@ int main(int argc, char *argv[])
 			continue;
 		read_topology(desc, i);
 		read_cache(desc, i);
+		read_polarization(desc, i);
 	}
 
 	qsort(desc->caches, desc->ncaches, sizeof(struct cpu_cache), cachecmp);

[patch 8/8] [PATCH] lscpu: add physical cpu address to parseable output

[Heiko Carstens]

From: Heiko Carstens <heiko.carstens@de.ibm.com>

Print also the physical cpu address for each logical cpu in parsable
output if selected and present via sysfs.

Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
---
 sys-utils/lscpu.1 |    5 +++--
 sys-utils/lscpu.c |   22 ++++++++++++++++++++--
 2 files changed, 23 insertions(+), 4 deletions(-)

--- a/sys-utils/lscpu.1
+++ b/sys-utils/lscpu.1
@@ -32,8 +32,9 @@ separate CPU cache columns. If no CPU ca
 columns are not printed at all.
 
 The \fIlist\fP argument is comma delimited list of the columns. Currently
-supported are CPU, Core, Node, Socket, Book, Cache and Polarization columns. If the
-\fIlist\fP argument is given then always all requested columns are printed in
+supported are CPU, Core, Node, Socket, Book, Cache, Polarization and Address
+columns.
+If the \fIlist\fP argument is given then always all requested columns are printed in
 the defined order. The Cache columns are separated by ':'.
 
 Note that the optional \fIlist\fP argument cannot be separated from the
--- a/sys-utils/lscpu.c
+++ b/sys-utils/lscpu.c
@@ -168,6 +168,7 @@ struct lscpu_desc {
 	struct cpu_cache *caches;
 
 	int		*polarization;	/* cpu polarization */
+	int		*addresses;	/* physical cpu addresses */
 };
 
 static size_t sysrootlen;
@@ -199,7 +200,8 @@ enum {
 	COL_NODE,
 	COL_BOOK,
 	COL_CACHE,
-	COL_POLARIZATION
+	COL_POLARIZATION,
+	COL_ADDRESS
 };
 
 static const char *colnames[] =
@@ -210,7 +212,8 @@ static const char *colnames[] =
 	[COL_NODE] = "Node",
 	[COL_BOOK] = "Book",
 	[COL_CACHE] = "Cache",
-	[COL_POLARIZATION] = "Polarization"
+	[COL_POLARIZATION] = "Polarization",
+	[COL_ADDRESS] = "Address"
 };
 
 
@@ -762,6 +765,16 @@ read_polarization(struct lscpu_desc *des
 		desc->polarization[num] = POLAR_UNKNOWN;
 }
 
+static void
+read_address(struct lscpu_desc *desc, int num)
+{
+	if (!path_exist(_PATH_SYS_CPU "/cpu%d/address", num))
+		return;
+	if (!desc->addresses)
+		desc->addresses = xcalloc(desc->ncpus, sizeof(int));
+	desc->addresses[num] = path_getnum(_PATH_SYS_CPU "/cpu%d/address", num);
+}
+
 static int
 cachecmp(const void *a, const void *b)
 {
@@ -916,6 +929,10 @@ print_parsable_cell(struct lscpu_desc *d
 		if (desc->polarization)
 			printf("%s", polar_modes[desc->polarization[i]]);
 		break;
+	case COL_ADDRESS:
+		if (desc->addresses)
+			printf("%d", desc->addresses[i]);
+		break;
 	}
 }
 
@@ -1230,6 +1247,7 @@ int main(int argc, char *argv[])
 		read_topology(desc, i);
 		read_cache(desc, i);
 		read_polarization(desc, i);
+		read_address(desc, i);
 	}
 
 	qsort(desc->caches, desc->ncaches, sizeof(struct cpu_cache), cachecmp);

Re: [patch 0/8] various lscpu patches

[Heiko Carstens]

On Wed, Aug 10, 2011 at 10:36:45AM +0200, Heiko Carstens wrote:
> Hi Karel,
> 
> here are a couple of fixes and improvemts for lscpu on s390.
> Hopefully I didn't screw up anything ;)
> At least on my Thinkpad everything still works as before and the output
> on s390 machines is much improved.

FWIW, I set up a git branch with the lscpu changes:

 git://git.kernel.org/pub/scm/linux/kernel/git/heiko/util-linux.git lscpu

Thanks,
Heiko

Re: [patch 0/8] various lscpu patches

[Karel Zak]

On Mon, Aug 15, 2011 at 10:45:30AM +0200, Heiko Carstens wrote:
> On Wed, Aug 10, 2011 at 10:36:45AM +0200, Heiko Carstens wrote:
> > Hi Karel,
> > 
> > here are a couple of fixes and improvemts for lscpu on s390.
> > Hopefully I didn't screw up anything ;)
> > At least on my Thinkpad everything still works as before and the output
> > on s390 machines is much improved.
> 
> FWIW, I set up a git branch with the lscpu changes:
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/heiko/util-linux.git lscpu

 Looks good. Thanks. 
 
 I'm going to merge the changes after 2.20 release (~25th Aug).


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [patch 0/8] various lscpu patches

[Karel Zak]

On Mon, Aug 15, 2011 at 10:45:30AM +0200, Heiko Carstens wrote:
> FWIW, I set up a git branch with the lscpu changes:
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/heiko/util-linux.git lscpu

 Merged, thanks.

 Heiko, it would be nice to have a regression tests for various s390
 boxes. See tests/ts/lscpu/, there is mk-input.sh which is able to
 gather all necessary files and create a tarball from /sys and /proc.
 For s390 it will be probably necessary to extend the list of the files
 (e.g add /proc/sysinfo).

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [patch 0/8] various lscpu patches

[Heiko Carstens]

On Tue, Aug 30, 2011 at 12:12:42PM +0200, Karel Zak wrote:
> On Mon, Aug 15, 2011 at 10:45:30AM +0200, Heiko Carstens wrote:
> > FWIW, I set up a git branch with the lscpu changes:
> > 
> >  git://git.kernel.org/pub/scm/linux/kernel/git/heiko/util-linux.git lscpu
> 
>  Merged, thanks.
> 
>  Heiko, it would be nice to have a regression tests for various s390
>  boxes. See tests/ts/lscpu/, there is mk-input.sh which is able to
>  gather all necessary files and create a tarball from /sys and /proc.
>  For s390 it will be probably necessary to extend the list of the files
>  (e.g add /proc/sysinfo).

Ok, I add that on my todo list. I have a couple of more patches for lscpu
and chcpu that I'd like to send to you this week. Afterwards I'll create
the regression test(s).

Thanks,
Heiko