[PATCH RFC/RFT v3 0/9] drivers: cacheinfo support

[PATCH RFC/RFT v3 0/9] drivers: cacheinfo support

[Sudeep Holla]

From: Sudeep Holla <sudeep.holla@arm.com>

Hi,

This series adds a generic cacheinfo support similar to topology. The
implementation is based on x86 cacheinfo support. Currently x86, powerpc,
ia64 and s390 have their own implementations. While adding similar support
to ARM and ARM64, here is the attempt to make it generic quite similar to
topology info support. It also adds the missing ABI documentation for
the cacheinfo sysfs which is already being used.

It moves all the existing different implementations on x86, ia64, powerpc
and s390 to use the generic cacheinfo infrastructure introduced here.
These changes on non-ARM platforms are only compile tested and hence
the request for testing too.

This series also adds support for ARM and ARM64 architectures based on
the generic support.

Changes v2[2]->v3:
	- Added new class "cpu" to group all cpu devices
	- Converted all "raw" kobjects used in cacheinfo to device_attr
	  by creating cache index devices
	- Added back s390 show_cacheinfo for /proc/cpuinfo
	- Added disable_sysfs to cache_info for preventing a cache node
	  to be exposed through sysfs if required(used on s390)

Changes v1[1]->v2[2]:
	- Extended the generic cacheinfo support to accomodate all
	  the existing implementations
	- Moved all the existing implementations to use this new
	  generic infrastructure
	- Added missing ABI documentation as suggested by Greg KH
	- Added support for unimplemented CTR on pre-ARMv6 implementations
	  as suggested by Russell. However the ctr_info_list is not yet
	  populated
	- not yet changed to device_attr as suggested by Greg KH,
	  registering cache as device won't eliminate the need of kobject
	  unless each index of cache is registered as a device which don't
	  seem to be good idea, but now it's unified it can be done easily
	  in one place if needed

[1] https://lkml.org/lkml/2014/1/8/523
[2] https://lkml.org/lkml/2014/2/7/654

Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-ia64@vger.kernel.org
Cc: linux390@de.ibm.com
Cc: linux-s390@vger.kernel.org
Cc: x86@kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-arm-kernel@lists.infradead.org
---
Sudeep Holla (9):
  drivers: base: add new class "cpu" to group cpu devices
  drivers: base: support cpu cache information interface to userspace
    via sysfs
  ia64: move cacheinfo sysfs to generic cacheinfo infrastructure
  s390: move cacheinfo sysfs to generic cacheinfo infrastructure
  x86: move cacheinfo sysfs to generic cacheinfo infrastructure
  powerpc: move cacheinfo sysfs to generic cacheinfo infrastructure
  ARM64: kernel: add support for cpu cache information
  ARM: kernel: add support for cpu cache information
  ARM: kernel: add outer cache support for cacheinfo implementation

 Documentation/ABI/testing/sysfs-devices-system-cpu |  40 +
 arch/arm/include/asm/outercache.h                  |  13 +
 arch/arm/kernel/Makefile                           |   1 +
 arch/arm/kernel/cacheinfo.c                        | 248 ++++++
 arch/arm/mm/Kconfig                                |  13 +
 arch/arm/mm/cache-l2x0.c                           |  14 +
 arch/arm/mm/cache-tauros2.c                        |  35 +
 arch/arm/mm/cache-xsc3l2.c                         |  15 +
 arch/arm64/kernel/Makefile                         |   2 +-
 arch/arm64/kernel/cacheinfo.c                      | 134 ++++
 arch/ia64/kernel/topology.c                        | 399 ++--------
 arch/powerpc/kernel/cacheinfo.c                    | 831 +++------------------
 arch/powerpc/kernel/cacheinfo.h                    |   8 -
 arch/powerpc/kernel/sysfs.c                        |   4 -
 arch/s390/kernel/cache.c                           | 388 +++-------
 arch/x86/kernel/cpu/intel_cacheinfo.c              | 647 ++++------------
 drivers/base/Makefile                              |   2 +-
 drivers/base/cacheinfo.c                           | 485 ++++++++++++
 drivers/base/core.c                                |  35 +-
 drivers/base/cpu.c                                 |   7 +
 include/linux/cacheinfo.h                          |  55 ++
 include/linux/cpu.h                                |   2 +
 22 files changed, 1523 insertions(+), 1855 deletions(-)
 create mode 100644 arch/arm/kernel/cacheinfo.c
 create mode 100644 arch/arm64/kernel/cacheinfo.c
 delete mode 100644 arch/powerpc/kernel/cacheinfo.h
 create mode 100644 drivers/base/cacheinfo.c
 create mode 100644 include/linux/cacheinfo.h

-- 
1.8.3.2

[PATCH RFC/RFT v3 4/9] s390: move cacheinfo sysfs to generic cacheinfo infrastructure

[Sudeep Holla]

From: Sudeep Holla <sudeep.holla@arm.com>

This patch removes the redundant sysfs cacheinfo code by making use of
the newly introduced generic cacheinfo infrastructure.

Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: linux390@de.ibm.com
Cc: linux-s390@vger.kernel.org
---
 arch/s390/kernel/cache.c | 388 ++++++++++++-----------------------------------
 1 file changed, 93 insertions(+), 295 deletions(-)

diff --git a/arch/s390/kernel/cache.c b/arch/s390/kernel/cache.c
index 3a414c0..e064f95 100644
--- a/arch/s390/kernel/cache.c
+++ b/arch/s390/kernel/cache.c
@@ -5,37 +5,11 @@
  *    Author(s): Heiko Carstens <heiko.carstens@de.ibm.com>
  */
 
-#include <linux/notifier.h>
 #include <linux/seq_file.h>
-#include <linux/init.h>
-#include <linux/list.h>
-#include <linux/slab.h>
 #include <linux/cpu.h>
+#include <linux/cacheinfo.h>
 #include <asm/facility.h>
 
-struct cache {
-	unsigned long size;
-	unsigned int line_size;
-	unsigned int associativity;
-	unsigned int nr_sets;
-	unsigned int level   : 3;
-	unsigned int type    : 2;
-	unsigned int private : 1;
-	struct list_head list;
-};
-
-struct cache_dir {
-	struct kobject *kobj;
-	struct cache_index_dir *index;
-};
-
-struct cache_index_dir {
-	struct kobject kobj;
-	int cpu;
-	struct cache *cache;
-	struct cache_index_dir *next;
-};
-
 enum {
 	CACHE_SCOPE_NOTEXISTS,
 	CACHE_SCOPE_PRIVATE,
@@ -44,10 +18,10 @@ enum {
 };
 
 enum {
-	CACHE_TYPE_SEPARATE,
-	CACHE_TYPE_DATA,
-	CACHE_TYPE_INSTRUCTION,
-	CACHE_TYPE_UNIFIED,
+	CTYPE_SEPARATE,
+	CTYPE_DATA,
+	CTYPE_INSTRUCTION,
+	CTYPE_UNIFIED,
 };
 
 enum {
@@ -63,46 +37,65 @@ enum {
 	CACHE_TI_INSTRUCTION,
 };
 
-struct cache_info {
+struct _cacheinfo {
 	unsigned char	    : 4;
 	unsigned char scope : 2;
 	unsigned char type  : 2;
 };
 
 #define CACHE_MAX_LEVEL 8
-
 union cache_topology {
-	struct cache_info ci[CACHE_MAX_LEVEL];
+	struct _cacheinfo ci[CACHE_MAX_LEVEL];
 	unsigned long long raw;
 };
 
 static const char * const cache_type_string[] = {
-	"Data",
+	"",
 	"Instruction",
+	"Data",
+	"",
 	"Unified",
 };
 
-static struct cache_dir *cache_dir_cpu[NR_CPUS];
-static LIST_HEAD(cache_list);
+static const enum cache_type cache_type_map[] = {
+	[CTYPE_SEPARATE] = CACHE_TYPE_SEPARATE,
+	[CTYPE_DATA] = CACHE_TYPE_DATA,
+	[CTYPE_INSTRUCTION] = CACHE_TYPE_INST,
+	[CTYPE_UNIFIED] = CACHE_TYPE_UNIFIED,
+};
 
 void show_cacheinfo(struct seq_file *m)
 {
-	struct cache *cache;
-	int index = 0;
+	int cpu = smp_processor_id(), idx;
+	struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+	struct cache_info *cache = this_cpu_ci->info_list;
 
-	list_for_each_entry(cache, &cache_list, list) {
-		seq_printf(m, "cache%-11d: ", index);
+	for (idx = 0; idx < this_cpu_ci->num_leaves; idx++) {
+		seq_printf(m, "cache%-11d: ", idx);
 		seq_printf(m, "level=%d ", cache->level);
 		seq_printf(m, "type=%s ", cache_type_string[cache->type]);
-		seq_printf(m, "scope=%s ", cache->private ? "Private" : "Shared");
-		seq_printf(m, "size=%luK ", cache->size >> 10);
-		seq_printf(m, "line_size=%u ", cache->line_size);
-		seq_printf(m, "associativity=%d", cache->associativity);
+		seq_printf(m, "scope=%s ",
+			   cache->disable_sysfs ? "Private" : "Shared");
+		seq_printf(m, "size=%dK ", cache->size >> 10);
+		seq_printf(m, "line_size=%u ", cache->coherency_line_size);
+		seq_printf(m, "associativity=%d", cache->ways_of_associativity);
 		seq_puts(m, "\n");
-		index++;
 	}
 }
 
+static inline enum cache_type get_cache_type(struct _cacheinfo *ci, int level)
+{
+	if (level >= CACHE_MAX_LEVEL)
+		return CACHE_TYPE_NOCACHE;
+
+	ci += level;
+
+	if (ci->scope != CACHE_SCOPE_SHARED && ci->scope != CACHE_SCOPE_PRIVATE)
+		return CACHE_TYPE_NOCACHE;
+
+	return cache_type_map[ci->type];
+}
+
 static inline unsigned long ecag(int ai, int li, int ti)
 {
 	unsigned long cmd, val;
@@ -113,274 +106,79 @@ static inline unsigned long ecag(int ai, int li, int ti)
 	return val;
 }
 
-static int __init cache_add(int level, int private, int type)
+static void ci_leaf_init(struct cache_info *this_leaf, int private,
+			 enum cache_type type, unsigned int level)
 {
-	struct cache *cache;
-	int ti;
+	int ti, num_sets;
+	int cpu = smp_processor_id();
 
-	cache = kzalloc(sizeof(*cache), GFP_KERNEL);
-	if (!cache)
-		return -ENOMEM;
-	if (type == CACHE_TYPE_INSTRUCTION)
+	if (type == CACHE_TYPE_INST)
 		ti = CACHE_TI_INSTRUCTION;
 	else
 		ti = CACHE_TI_UNIFIED;
-	cache->size = ecag(EXTRACT_SIZE, level, ti);
-	cache->line_size = ecag(EXTRACT_LINE_SIZE, level, ti);
-	cache->associativity = ecag(EXTRACT_ASSOCIATIVITY, level, ti);
-	cache->nr_sets = cache->size / cache->associativity;
-	cache->nr_sets /= cache->line_size;
-	cache->private = private;
-	cache->level = level + 1;
-	cache->type = type - 1;
-	list_add_tail(&cache->list, &cache_list);
-	return 0;
-}
 
-static void __init cache_build_info(void)
-{
-	struct cache *cache, *next;
-	union cache_topology ct;
-	int level, private, rc;
+	this_leaf->level = level + 1;
+	this_leaf->type = type;
+	this_leaf->coherency_line_size = ecag(EXTRACT_LINE_SIZE, level, ti);
+	this_leaf->ways_of_associativity = ecag(EXTRACT_ASSOCIATIVITY,
+						level, ti);
+	this_leaf->size = ecag(EXTRACT_SIZE, level, ti);
 
-	ct.raw = ecag(EXTRACT_TOPOLOGY, 0, 0);
-	for (level = 0; level < CACHE_MAX_LEVEL; level++) {
-		switch (ct.ci[level].scope) {
-		case CACHE_SCOPE_SHARED:
-			private = 0;
-			break;
-		case CACHE_SCOPE_PRIVATE:
-			private = 1;
-			break;
-		default:
-			return;
-		}
-		if (ct.ci[level].type == CACHE_TYPE_SEPARATE) {
-			rc  = cache_add(level, private, CACHE_TYPE_DATA);
-			rc |= cache_add(level, private, CACHE_TYPE_INSTRUCTION);
-		} else {
-			rc = cache_add(level, private, ct.ci[level].type);
-		}
-		if (rc)
-			goto error;
-	}
-	return;
-error:
-	list_for_each_entry_safe(cache, next, &cache_list, list) {
-		list_del(&cache->list);
-		kfree(cache);
-	}
-}
-
-static struct cache_dir *cache_create_cache_dir(int cpu)
-{
-	struct cache_dir *cache_dir;
-	struct kobject *kobj = NULL;
-	struct device *dev;
-
-	dev = get_cpu_device(cpu);
-	if (!dev)
-		goto out;
-	kobj = kobject_create_and_add("cache", &dev->kobj);
-	if (!kobj)
-		goto out;
-	cache_dir = kzalloc(sizeof(*cache_dir), GFP_KERNEL);
-	if (!cache_dir)
-		goto out;
-	cache_dir->kobj = kobj;
-	cache_dir_cpu[cpu] = cache_dir;
-	return cache_dir;
-out:
-	kobject_put(kobj);
-	return NULL;
-}
-
-static struct cache_index_dir *kobj_to_cache_index_dir(struct kobject *kobj)
-{
-	return container_of(kobj, struct cache_index_dir, kobj);
-}
-
-static void cache_index_release(struct kobject *kobj)
-{
-	struct cache_index_dir *index;
-
-	index = kobj_to_cache_index_dir(kobj);
-	kfree(index);
+	num_sets = this_leaf->size / this_leaf->coherency_line_size;
+	num_sets /= this_leaf->ways_of_associativity;
+	this_leaf->number_of_sets = num_sets;
+	cpumask_set_cpu(cpu, &this_leaf->shared_cpu_map);
+	if (!private)
+		this_leaf->disable_sysfs = true;
 }
 
-static ssize_t cache_index_show(struct kobject *kobj,
-				struct attribute *attr, char *buf)
+int init_cache_level(unsigned int cpu)
 {
-	struct kobj_attribute *kobj_attr;
-
-	kobj_attr = container_of(attr, struct kobj_attribute, attr);
-	return kobj_attr->show(kobj, kobj_attr, buf);
-}
-
-#define DEFINE_CACHE_ATTR(_name, _format, _value)			\
-static ssize_t cache_##_name##_show(struct kobject *kobj,		\
-				    struct kobj_attribute *attr,	\
-				    char *buf)				\
-{									\
-	struct cache_index_dir *index;					\
-									\
-	index = kobj_to_cache_index_dir(kobj);				\
-	return sprintf(buf, _format, _value);				\
-}									\
-static struct kobj_attribute cache_##_name##_attr =			\
-	__ATTR(_name, 0444, cache_##_name##_show, NULL);
-
-DEFINE_CACHE_ATTR(size, "%luK\n", index->cache->size >> 10);
-DEFINE_CACHE_ATTR(coherency_line_size, "%u\n", index->cache->line_size);
-DEFINE_CACHE_ATTR(number_of_sets, "%u\n", index->cache->nr_sets);
-DEFINE_CACHE_ATTR(ways_of_associativity, "%u\n", index->cache->associativity);
-DEFINE_CACHE_ATTR(type, "%s\n", cache_type_string[index->cache->type]);
-DEFINE_CACHE_ATTR(level, "%d\n", index->cache->level);
-
-static ssize_t shared_cpu_map_func(struct kobject *kobj, int type, char *buf)
-{
-	struct cache_index_dir *index;
-	int len;
-
-	index = kobj_to_cache_index_dir(kobj);
-	len = type ?
-		cpulist_scnprintf(buf, PAGE_SIZE - 2, cpumask_of(index->cpu)) :
-		cpumask_scnprintf(buf, PAGE_SIZE - 2, cpumask_of(index->cpu));
-	len += sprintf(&buf[len], "\n");
-	return len;
-}
-
-static ssize_t shared_cpu_map_show(struct kobject *kobj,
-				   struct kobj_attribute *attr, char *buf)
-{
-	return shared_cpu_map_func(kobj, 0, buf);
-}
-static struct kobj_attribute cache_shared_cpu_map_attr =
-	__ATTR(shared_cpu_map, 0444, shared_cpu_map_show, NULL);
-
-static ssize_t shared_cpu_list_show(struct kobject *kobj,
-				    struct kobj_attribute *attr, char *buf)
-{
-	return shared_cpu_map_func(kobj, 1, buf);
-}
-static struct kobj_attribute cache_shared_cpu_list_attr =
-	__ATTR(shared_cpu_list, 0444, shared_cpu_list_show, NULL);
-
-static struct attribute *cache_index_default_attrs[] = {
-	&cache_type_attr.attr,
-	&cache_size_attr.attr,
-	&cache_number_of_sets_attr.attr,
-	&cache_ways_of_associativity_attr.attr,
-	&cache_level_attr.attr,
-	&cache_coherency_line_size_attr.attr,
-	&cache_shared_cpu_map_attr.attr,
-	&cache_shared_cpu_list_attr.attr,
-	NULL,
-};
-
-static const struct sysfs_ops cache_index_ops = {
-	.show = cache_index_show,
-};
-
-static struct kobj_type cache_index_type = {
-	.sysfs_ops = &cache_index_ops,
-	.release = cache_index_release,
-	.default_attrs = cache_index_default_attrs,
-};
-
-static int cache_create_index_dir(struct cache_dir *cache_dir,
-				  struct cache *cache, int index, int cpu)
-{
-	struct cache_index_dir *index_dir;
-	int rc;
-
-	index_dir = kzalloc(sizeof(*index_dir), GFP_KERNEL);
-	if (!index_dir)
-		return -ENOMEM;
-	index_dir->cache = cache;
-	index_dir->cpu = cpu;
-	rc = kobject_init_and_add(&index_dir->kobj, &cache_index_type,
-				  cache_dir->kobj, "index%d", index);
-	if (rc)
-		goto out;
-	index_dir->next = cache_dir->index;
-	cache_dir->index = index_dir;
-	return 0;
-out:
-	kfree(index_dir);
-	return rc;
-}
+	struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+	unsigned int level = 0, leaves = 0;
+	union cache_topology ct;
+	enum cache_type ctype;
 
-static int cache_add_cpu(int cpu)
-{
-	struct cache_dir *cache_dir;
-	struct cache *cache;
-	int rc, index = 0;
+	if (!this_cpu_ci)
+		return -EINVAL;
 
-	if (list_empty(&cache_list))
-		return 0;
-	cache_dir = cache_create_cache_dir(cpu);
-	if (!cache_dir)
-		return -ENOMEM;
-	list_for_each_entry(cache, &cache_list, list) {
-		if (!cache->private)
+	ct.raw = ecag(EXTRACT_TOPOLOGY, 0, 0);
+	do {
+		ctype = get_cache_type(&ct.ci[0], level);
+		if (ctype == CACHE_TYPE_NOCACHE)
 			break;
-		rc = cache_create_index_dir(cache_dir, cache, index, cpu);
-		if (rc)
-			return rc;
-		index++;
-	}
-	return 0;
-}
+		/* Separate instruction and data caches */
+		leaves += (ctype == CACHE_TYPE_SEPARATE) ? 2 : 1;
+	} while (++level < CACHE_MAX_LEVEL);
 
-static void cache_remove_cpu(int cpu)
-{
-	struct cache_index_dir *index, *next;
-	struct cache_dir *cache_dir;
+	this_cpu_ci->num_levels = level;
+	this_cpu_ci->num_leaves = leaves;
 
-	cache_dir = cache_dir_cpu[cpu];
-	if (!cache_dir)
-		return;
-	index = cache_dir->index;
-	while (index) {
-		next = index->next;
-		kobject_put(&index->kobj);
-		index = next;
-	}
-	kobject_put(cache_dir->kobj);
-	kfree(cache_dir);
-	cache_dir_cpu[cpu] = NULL;
+	return 0;
 }
 
-static int cache_hotplug(struct notifier_block *nfb, unsigned long action,
-			 void *hcpu)
+int populate_cache_leaves(unsigned int cpu)
 {
-	int cpu = (long)hcpu;
-	int rc = 0;
+	unsigned int level, idx, pvt;
+	union cache_topology ct;
+	enum cache_type ctype;
+	struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+	struct cache_info *this_leaf = this_cpu_ci->info_list;
 
-	switch (action & ~CPU_TASKS_FROZEN) {
-	case CPU_ONLINE:
-		rc = cache_add_cpu(cpu);
-		if (rc)
-			cache_remove_cpu(cpu);
-		break;
-	case CPU_DEAD:
-		cache_remove_cpu(cpu);
-		break;
+	ct.raw = ecag(EXTRACT_TOPOLOGY, 0, 0);
+	for (idx = 0, level = 0; level < this_cpu_ci->num_levels &&
+			idx < this_cpu_ci->num_leaves; idx++, level++) {
+		if (!this_leaf)
+			return -EINVAL;
+
+		pvt = (ct.ci[level].scope == CACHE_SCOPE_PRIVATE) ? 1 : 0;
+		ctype = get_cache_type(&ct.ci[0], level);
+		if (ctype == CACHE_TYPE_SEPARATE) {
+			ci_leaf_init(this_leaf++, pvt, CACHE_TYPE_DATA, level);
+			ci_leaf_init(this_leaf++, pvt, CACHE_TYPE_INST, level);
+		} else {
+			ci_leaf_init(this_leaf++, pvt, ctype, level);
+		}
 	}
-	return rc ? NOTIFY_BAD : NOTIFY_OK;
-}
-
-static int __init cache_init(void)
-{
-	int cpu;
-
-	if (!test_facility(34))
-		return 0;
-	cache_build_info();
-	for_each_online_cpu(cpu)
-		cache_add_cpu(cpu);
-	hotcpu_notifier(cache_hotplug, 0);
 	return 0;
 }
-device_initcall(cache_init);
-- 
1.8.3.2

Re: [PATCH RFC/RFT v3 4/9] s390: move cacheinfo sysfs to generic cacheinfo infrastructure

[Heiko Carstens]

On Wed, Feb 19, 2014 at 04:06:11PM +0000, Sudeep Holla wrote:
> From: Sudeep Holla <sudeep.holla@arm.com>
> 
> This patch removes the redundant sysfs cacheinfo code by making use of
> the newly introduced generic cacheinfo infrastructure.
> 
> Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
> Cc: linux390@de.ibm.com
> Cc: linux-s390@vger.kernel.org
> ---
>  arch/s390/kernel/cache.c | 388 ++++++++++++-----------------------------------
>  1 file changed, 93 insertions(+), 295 deletions(-)

(FWIW, if you send an update of your series, please cc me on all patches, so I
 don't have to search LKML for the rest of the patch set)

I gave your patches series a test and I get this nice message on boot:

io scheduler cfq registered
DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
------------[ cut here ]------------
WARNING: at kernel/locking/lockdep.c:2742
Modules linked in:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.14.0-rc3-00177-g468cee08c2ea #7
task: 0000000000adcc60 ti: 0000000000ac4000 task.ti: 0000000000ac4000
Krnl PSW : 0404c00180000000 000000000019af3e (lockdep_trace_alloc+0x116/0x120)
           R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 EA:3
Krnl GPRS: 0000000000000020 0000000000adcc60 000000000000002f 0000000000000000
           000000000019af3a 0000000000000000 0000000000000001 0000000000000000
           000000003ff26140 000000000055fb92 00000000000001b0 00000000000080d0
           0400000000000000 00000000000080d0 000000000019af3a 000000000001bc30
Krnl Code: 000000000019af2e: c020003b1e34       larl    %r2,8feb96
           000000000019af34: c0e5002e0304       brasl   %r14,75b53c
          #000000000019af3a: a7f40001           brc     15,19af3c
          >000000000019af3e: e32003100004       lg      %r2,784
           000000000019af44: a7f4ff9f           brc     15,19ae82
           000000000019af48: e31003100004       lg      %r1,784
           000000000019af4e: e32013f00150       sty     %r2,5104(%r1)
           000000000019af54: 07fe               bcr     15,%r14
Call Trace:
([<000000000019af3a>] lockdep_trace_alloc+0x112/0x120)
 [<000000000027a7d2>] __kmalloc+0x5a/0x204
 [<000000000055fb92>] detect_cache_attributes+0x66/0xd4
 [<000000000055fc2e>] _detect_cache_attributes+0x2e/0x3c
 [<00000000001ce7a2>] generic_smp_call_function_single_interrupt+0x9a/0xe8
 [<000000000010b462>] do_ext_interrupt+0x12a/0x24c
 [<00000000001afab0>] handle_irq_event_percpu+0x70/0x240
 [<00000000001b3218>] handle_percpu_irq+0x6c/0x98
 [<00000000001af16e>] generic_handle_irq+0x46/0x68
 [<000000000010b6c2>] do_IRQ+0x5e/0x84
 [<0000000000768b06>] ext_skip+0x44/0x4a
 [<000000000076827c>] vtime_stop_cpu+0x50/0xa4
([<0000000000768262>] vtime_stop_cpu+0x36/0xa4)
 [<0000000000103848>] arch_cpu_idle+0x6c/0xbc
 [<00000000001af032>] cpu_startup_entry+0x146/0x218
 [<0000000000b64932>] start_kernel+0x3fa/0x408
 [<0000000000100020>] _stext+0x20/0x80
INFO: lockdep is turned off.
Last Breaking-Event-Address:
 [<000000000019af3a>] lockdep_trace_alloc+0x112/0x120
---[ end trace 9c0011ccc7b324d6 ]---
brd: module loaded

In addition /proc/cpuinfo is now incorrect.
With your patches it looks like this:

[root@p2345007 ~]# cat /proc/cpuinfo 
vendor_id       : IBM/S390
# processors    : 4
bogomips per cpu: 18115.00
features        : esan3 zarch stfle msa ldisp eimm dfp etf3eh highgprs 
cache0          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
cache1          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
cache2          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
cache3          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
cache4          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
cache5          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
processor 0: version = FF,  identification = 2F6D15,  machine = 2827
processor 1: version = FF,  identification = 2F6D15,  machine = 2827
processor 2: version = FF,  identification = 2F6D15,  machine = 2827
processor 3: version = FF,  identification = 2F6D15,  machine = 2827

However it is supposed to look like this:

[root@p2345007 ~]# cat /proc/cpuinfo 
vendor_id       : IBM/S390
# processors    : 4
bogomips per cpu: 18115.00
features        : esan3 zarch stfle msa ldisp eimm dfp etf3eh highgprs 
cache0          : level=1 type=Data scope=Private size=96K line_size=256 associativity=6
cache1          : level=1 type=Instruction scope=Private size=64K line_size=256 associativity=4
cache2          : level=2 type=Data scope=Private size=1024K line_size=256 associativity=8
cache3          : level=2 type=Instruction scope=Private size=1024K line_size=256 associativity=8
cache4          : level=3 type=Unified scope=Shared size=49152K line_size=256 associativity=12
cache5          : level=4 type=Unified scope=Shared size=393216K line_size=256 associativity=24
processor 0: version = FF,  identification = 2F6D15,  machine = 2827
processor 1: version = FF,  identification = 2F6D15,  machine = 2827
processor 2: version = FF,  identification = 2F6D15,  machine = 2827
processor 3: version = FF,  identification = 2F6D15,  machine = 2827

The sysfs files and it contents seem to be ok.

Re: [PATCH RFC/RFT v3 4/9] s390: move cacheinfo sysfs to generic cacheinfo infrastructure

[Sudeep Holla]

Hi Heiko,

Thanks for testing this series, much appreciated.

On 20/02/14 08:38, Heiko Carstens wrote:
> On Wed, Feb 19, 2014 at 04:06:11PM +0000, Sudeep Holla wrote:
>> From: Sudeep Holla <sudeep.holla@arm.com>
>>
>> This patch removes the redundant sysfs cacheinfo code by making use of
>> the newly introduced generic cacheinfo infrastructure.
>>
>> Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
>> Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
>> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
>> Cc: linux390@de.ibm.com
>> Cc: linux-s390@vger.kernel.org
>> ---
>>  arch/s390/kernel/cache.c | 388 ++++++++++++-----------------------------------
>>  1 file changed, 93 insertions(+), 295 deletions(-)
> 
> (FWIW, if you send an update of your series, please cc me on all patches, so I
>  don't have to search LKML for the rest of the patch set)
> 
Sorry, will make sure you will be cc-ed in the future updates.

> I gave your patches series a test and I get this nice message on boot:
> 
> io scheduler cfq registered
> DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
> ------------[ cut here ]------------
> WARNING: at kernel/locking/lockdep.c:2742
> Modules linked in:
> CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.14.0-rc3-00177-g468cee08c2ea #7
> task: 0000000000adcc60 ti: 0000000000ac4000 task.ti: 0000000000ac4000
> Krnl PSW : 0404c00180000000 000000000019af3e (lockdep_trace_alloc+0x116/0x120)
>            R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 EA:3
> Krnl GPRS: 0000000000000020 0000000000adcc60 000000000000002f 0000000000000000
>            000000000019af3a 0000000000000000 0000000000000001 0000000000000000
>            000000003ff26140 000000000055fb92 00000000000001b0 00000000000080d0
>            0400000000000000 00000000000080d0 000000000019af3a 000000000001bc30
> Krnl Code: 000000000019af2e: c020003b1e34       larl    %r2,8feb96
>            000000000019af34: c0e5002e0304       brasl   %r14,75b53c
>           #000000000019af3a: a7f40001           brc     15,19af3c
>           >000000000019af3e: e32003100004       lg      %r2,784
>            000000000019af44: a7f4ff9f           brc     15,19ae82
>            000000000019af48: e31003100004       lg      %r1,784
>            000000000019af4e: e32013f00150       sty     %r2,5104(%r1)
>            000000000019af54: 07fe               bcr     15,%r14
> Call Trace:
> ([<000000000019af3a>] lockdep_trace_alloc+0x112/0x120)
>  [<000000000027a7d2>] __kmalloc+0x5a/0x204
>  [<000000000055fb92>] detect_cache_attributes+0x66/0xd4
>  [<000000000055fc2e>] _detect_cache_attributes+0x2e/0x3c
>  [<00000000001ce7a2>] generic_smp_call_function_single_interrupt+0x9a/0xe8
>  [<000000000010b462>] do_ext_interrupt+0x12a/0x24c
>  [<00000000001afab0>] handle_irq_event_percpu+0x70/0x240
>  [<00000000001b3218>] handle_percpu_irq+0x6c/0x98
>  [<00000000001af16e>] generic_handle_irq+0x46/0x68
>  [<000000000010b6c2>] do_IRQ+0x5e/0x84
>  [<0000000000768b06>] ext_skip+0x44/0x4a
>  [<000000000076827c>] vtime_stop_cpu+0x50/0xa4
> ([<0000000000768262>] vtime_stop_cpu+0x36/0xa4)
>  [<0000000000103848>] arch_cpu_idle+0x6c/0xbc
>  [<00000000001af032>] cpu_startup_entry+0x146/0x218
>  [<0000000000b64932>] start_kernel+0x3fa/0x408
>  [<0000000000100020>] _stext+0x20/0x80
> INFO: lockdep is turned off.
> Last Breaking-Event-Address:
>  [<000000000019af3a>] lockdep_trace_alloc+0x112/0x120
> ---[ end trace 9c0011ccc7b324d6 ]---
> brd: module loaded
> 

Ah, my bad, I had this fixed initially when I implemented only on ARM and
introduced back while reorganising it to support multiple architectures, it's
now fixed.

> In addition /proc/cpuinfo is now incorrect.
> With your patches it looks like this:
> 
> [root@p2345007 ~]# cat /proc/cpuinfo 
> vendor_id       : IBM/S390
> # processors    : 4
> bogomips per cpu: 18115.00
> features        : esan3 zarch stfle msa ldisp eimm dfp etf3eh highgprs 
> cache0          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
> cache1          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
> cache2          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
> cache3          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
> cache4          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
> cache5          : level=1 type=Data scope=Shared size=96K line_size=256 associativity=6
> processor 0: version = FF,  identification = 2F6D15,  machine = 2827
> processor 1: version = FF,  identification = 2F6D15,  machine = 2827
> processor 2: version = FF,  identification = 2F6D15,  machine = 2827
> processor 3: version = FF,  identification = 2F6D15,  machine = 2827
> 
> However it is supposed to look like this:
> 
> [root@p2345007 ~]# cat /proc/cpuinfo 
> vendor_id       : IBM/S390
> # processors    : 4
> bogomips per cpu: 18115.00
> features        : esan3 zarch stfle msa ldisp eimm dfp etf3eh highgprs 
> cache0          : level=1 type=Data scope=Private size=96K line_size=256 associativity=6
> cache1          : level=1 type=Instruction scope=Private size=64K line_size=256 associativity=4
> cache2          : level=2 type=Data scope=Private size=1024K line_size=256 associativity=8
> cache3          : level=2 type=Instruction scope=Private size=1024K line_size=256 associativity=8
> cache4          : level=3 type=Unified scope=Shared size=49152K line_size=256 associativity=12
> cache5          : level=4 type=Unified scope=Shared size=393216K line_size=256 associativity=24
> processor 0: version = FF,  identification = 2F6D15,  machine = 2827
> processor 1: version = FF,  identification = 2F6D15,  machine = 2827
> processor 2: version = FF,  identification = 2F6D15,  machine = 2827
> processor 3: version = FF,  identification = 2F6D15,  machine = 2827
> 
> The sysfs files and it contents seem to be ok.
> 

Thanks, this info was helpful and looks like it's stupid mistake I did. I
deleted a line unknowingly while trying to minimise the diff for show_cacheinfo.
The below fix-up must work IIUC the issue. I will squash this in my next update
if it works.

Regards,
Sudeep

--->8

From 77690bc34b2b4b492377998019371f6e2f8f90b7 Mon Sep 17 00:00:00 2001
From: Sudeep Holla <sudeep.holla@arm.com>
Date: Thu, 20 Feb 2014 13:14:09 +0000
Subject: [PATCH] fixup! s390: move cacheinfo sysfs to generic cacheinfo
 infrastructure

---
 arch/s390/kernel/cache.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/s390/kernel/cache.c b/arch/s390/kernel/cache.c
index e064f95..aeedb7f 100644
--- a/arch/s390/kernel/cache.c
+++ b/arch/s390/kernel/cache.c
@@ -68,14 +68,15 @@ void show_cacheinfo(struct seq_file *m)
 {
 	int cpu = smp_processor_id(), idx;
 	struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
-	struct cache_info *cache = this_cpu_ci->info_list;
+	struct cache_info *cache;

 	for (idx = 0; idx < this_cpu_ci->num_leaves; idx++) {
+		cache = this_cpu_ci->info_list + idx;
 		seq_printf(m, "cache%-11d: ", idx);
 		seq_printf(m, "level=%d ", cache->level);
 		seq_printf(m, "type=%s ", cache_type_string[cache->type]);
 		seq_printf(m, "scope=%s ",
-			   cache->disable_sysfs ? "Private" : "Shared");
+			   cache->disable_sysfs ? "Shared" : "Private");
 		seq_printf(m, "size=%dK ", cache->size >> 10);
 		seq_printf(m, "line_size=%u ", cache->coherency_line_size);
 		seq_printf(m, "associativity=%d", cache->ways_of_associativity);

Re: [PATCH RFC/RFT v3 4/9] s390: move cacheinfo sysfs to generic cacheinfo infrastructure

[Heiko Carstens]

On Thu, Feb 20, 2014 at 01:33:56PM +0000, Sudeep Holla wrote:
> Thanks, this info was helpful and looks like it's stupid mistake I did. I
> deleted a line unknowingly while trying to minimise the diff for show_cacheinfo.
> The below fix-up must work IIUC the issue. I will squash this in my next update
> if it works.
> 
> Regards,
> Sudeep
> 
> --->8
> 
> From 77690bc34b2b4b492377998019371f6e2f8f90b7 Mon Sep 17 00:00:00 2001
> From: Sudeep Holla <sudeep.holla@arm.com>
> Date: Thu, 20 Feb 2014 13:14:09 +0000
> Subject: [PATCH] fixup! s390: move cacheinfo sysfs to generic cacheinfo
>  infrastructure
> 
> ---
>  arch/s390/kernel/cache.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/s390/kernel/cache.c b/arch/s390/kernel/cache.c
> index e064f95..aeedb7f 100644
> --- a/arch/s390/kernel/cache.c
> +++ b/arch/s390/kernel/cache.c
> @@ -68,14 +68,15 @@ void show_cacheinfo(struct seq_file *m)
>  {
>  	int cpu = smp_processor_id(), idx;
>  	struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
> -	struct cache_info *cache = this_cpu_ci->info_list;
> +	struct cache_info *cache;
> 
>  	for (idx = 0; idx < this_cpu_ci->num_leaves; idx++) {
> +		cache = this_cpu_ci->info_list + idx;
>  		seq_printf(m, "cache%-11d: ", idx);
>  		seq_printf(m, "level=%d ", cache->level);
>  		seq_printf(m, "type=%s ", cache_type_string[cache->type]);
>  		seq_printf(m, "scope=%s ",
> -			   cache->disable_sysfs ? "Private" : "Shared");
> +			   cache->disable_sysfs ? "Shared" : "Private");
>  		seq_printf(m, "size=%dK ", cache->size >> 10);
>  		seq_printf(m, "line_size=%u ", cache->coherency_line_size);
>  		seq_printf(m, "associativity=%d", cache->ways_of_associativity);

With this patch applied the output looks ok again.

Thanks,
Heiko

Re: [PATCH RFC/RFT v3 4/9] s390: move cacheinfo sysfs to generic cacheinfo infrastructure

[Sudeep Holla]

On 20/02/14 14:07, Heiko Carstens wrote:
> On Thu, Feb 20, 2014 at 01:33:56PM +0000, Sudeep Holla wrote:
>> Thanks, this info was helpful and looks like it's stupid mistake I did. I
>> deleted a line unknowingly while trying to minimise the diff for show_cacheinfo.
>> The below fix-up must work IIUC the issue. I will squash this in my next update
>> if it works.
>>
>> Regards,
>> Sudeep
>>
>> --->8
>>
>> From 77690bc34b2b4b492377998019371f6e2f8f90b7 Mon Sep 17 00:00:00 2001
>> From: Sudeep Holla <sudeep.holla@arm.com>
>> Date: Thu, 20 Feb 2014 13:14:09 +0000
>> Subject: [PATCH] fixup! s390: move cacheinfo sysfs to generic cacheinfo
>>  infrastructure
>>
>> ---
>>  arch/s390/kernel/cache.c | 5 +++--
>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/s390/kernel/cache.c b/arch/s390/kernel/cache.c
>> index e064f95..aeedb7f 100644
>> --- a/arch/s390/kernel/cache.c
>> +++ b/arch/s390/kernel/cache.c
>> @@ -68,14 +68,15 @@ void show_cacheinfo(struct seq_file *m)
>>  {
>>  	int cpu = smp_processor_id(), idx;
>>  	struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
>> -	struct cache_info *cache = this_cpu_ci->info_list;
>> +	struct cache_info *cache;
>>
>>  	for (idx = 0; idx < this_cpu_ci->num_leaves; idx++) {
>> +		cache = this_cpu_ci->info_list + idx;
>>  		seq_printf(m, "cache%-11d: ", idx);
>>  		seq_printf(m, "level=%d ", cache->level);
>>  		seq_printf(m, "type=%s ", cache_type_string[cache->type]);
>>  		seq_printf(m, "scope=%s ",
>> -			   cache->disable_sysfs ? "Private" : "Shared");
>> +			   cache->disable_sysfs ? "Shared" : "Private");
>>  		seq_printf(m, "size=%dK ", cache->size >> 10);
>>  		seq_printf(m, "line_size=%u ", cache->coherency_line_size);
>>  		seq_printf(m, "associativity=%d", cache->ways_of_associativity);
> 
> With this patch applied the output looks ok again.
> 

Thanks a lot for retesting and confirming results with this fix-up, will squash
it in my next update.

Regards,
Sudeep