[patch]:MC/MT enabling/identification for IA-64

[patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

Please find attached the patch that enables identification of multi-core
and multi-thread on IA-64 processors.  This patch adds 4 new  fields in
/proc/cpuinfo to clearly identify each logical execution unit.  Two new
data structures cpu_core_map and cpu_sibling_map are also implemented.
This information will be used in scheduler (and possibly by other kernel
components as well).  A new SAL call (SAL_PHYSICAL_ID_INFO) and a PAL
call (PAL_LOGICAL_TO_PHYSICAL) are also added for identification
purposes.

TBD items: Hot-plug of logical processors
Scheduler enhancements:  We will send out this patch in next few days.

Comments and feedback welcome.

Thanks, rohit

Signed-off-by: Gordon Jin <gordon.jin@intel.com>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Rohit Seth <rohit.seth@intel.com>



[-- Attachment #2: mcmt-ia64-021505.patch --]
[-- Type: application/octet-stream, Size: 14429 bytes --]

Add the Multi-core and Multi-threading detection for IPF.
  - Add new core and threading related fields in /proc/cpuinfo.
  - setup the cpu_core_map and cpu_sibling_map appropriately.

Signed-off-by: Gordon Jin <gordon.jin@intel.com>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Rohit Seth <rohit.seth@intel.com>
--

diff -Nru linux-2.6.11-rc3/arch/ia64/kernel/setup.c linux-mcmt/arch/ia64/kernel/setup.c
--- linux-2.6.11-rc3/arch/ia64/kernel/setup.c	2005-02-02 17:55:14.000000000 -0800
+++ linux-mcmt/arch/ia64/kernel/setup.c	2005-02-15 11:58:37.000000000 -0800
@@ -1,13 +1,17 @@
 /*
  * Architecture-specific setup.
  *
+ * Copyright (C) 2004 Gordon Jin <gordon.jin@intel.com>
+ * Copyright (C) 2004 Suresh Siddha <suresh.b.siddha@intel.com>
  * Copyright (C) 1998-2001, 2003-2004 Hewlett-Packard Co
  *	David Mosberger-Tang <davidm@hpl.hp.com>
  *	Stephane Eranian <eranian@hpl.hp.com>
- * Copyright (C) 2000, Rohit Seth <rohit.seth@intel.com>
+ * Copyright (C) 2000,2004 Rohit Seth <rohit.seth@intel.com>
  * Copyright (C) 1999 VA Linux Systems
  * Copyright (C) 1999 Walt Drummond <drummond@valinux.com>
  *
+ * 12/26/04 G.Jin, S.Siddha, R.Seth
+ *			Add multi-threading and multi-core detection
  * 11/12/01 D.Mosberger Convert get_cpuinfo() to seq_file based show_cpuinfo().
  * 04/04/00 D.Mosberger renamed cpu_initialized to cpu_online_map
  * 03/31/00 R.Seth	cpu_initialized and current->processor fixes
@@ -299,6 +303,34 @@
 #endif
 }
 
+#ifdef CONFIG_SMP
+static void
+check_for_logical_procs (void)
+{
+	pal_logical_to_physical_t info;
+	s64 status;
+
+	status = ia64_pal_logical_to_phys(0, &info);
+	if (status == -1) {
+		printk(KERN_INFO "No logical to physical processor mapping "
+		       "available\n");
+		return;
+	}
+	if (status) {
+		printk(KERN_ERR "ia64_pal_logical_to_phys failed with %ld\n",
+		       status);
+		return;
+	}
+	/*
+	 * Total number of siblings that BSP has.
+	 * Though not all of them may have booted successfully.
+	 * The correct number of siblings booted is in info.overview_num_log.
+	 */
+	smp_num_siblings = info.overview_tpc;
+	smp_num_cpucores = info.overview_cpp;
+}
+#endif
+
 void __init
 setup_arch (char **cmdline_p)
 {
@@ -359,6 +391,16 @@
 
 #ifdef CONFIG_SMP
 	cpu_physical_id(0) = hard_smp_processor_id();
+
+	check_for_logical_procs();
+	if (smp_num_cpucores > 1)
+		printk(KERN_INFO
+		       "cpu package is Multi-Core capable: number of cores=%d\n",
+		       smp_num_cpucores);
+	if (smp_num_siblings > 1)
+		printk(KERN_INFO
+		       "cpu package is Multi-Threading capable: number of siblings=%d\n",
+		       smp_num_siblings);
 #endif
 
 	cpu_init();	/* initialize the bootstrap CPU */
@@ -462,12 +504,22 @@
 		   "cpu regs   : %u\n"
 		   "cpu MHz    : %lu.%06lu\n"
 		   "itc MHz    : %lu.%06lu\n"
-		   "BogoMIPS   : %lu.%02lu\n\n",
+		   "BogoMIPS   : %lu.%02lu\n",
 		   cpunum, c->vendor, family, c->model, c->revision, c->archrev,
 		   features, c->ppn, c->number,
 		   c->proc_freq / 1000000, c->proc_freq % 1000000,
 		   c->itc_freq / 1000000, c->itc_freq % 1000000,
 		   lpj*HZ/500000, (lpj*HZ/5000) % 100);
+#ifdef CONFIG_SMP
+	seq_printf(m,
+		   "physical id\t: %u\n"
+		   "siblings\t: %u\n"
+		   "cpu core id\t: %u\n"
+		   "cpu cores\t: %u\n",
+		   c->socket_id, c->tpc * c->cpp, c->cid, c->cpp);
+#endif
+	seq_printf(m,"\n");
+
 	return 0;
 }
 
@@ -503,6 +555,7 @@
 void
 identify_cpu (struct cpuinfo_ia64 *c)
 {
+	extern void identify_siblings (struct cpuinfo_ia64 *);
 	union {
 		unsigned long bits[5];
 		struct {
@@ -536,6 +589,15 @@
 	memcpy(c->vendor, cpuid.field.vendor, 16);
 #ifdef CONFIG_SMP
 	c->cpu = smp_processor_id();
+
+	/* below default values will be overwritten
+	 * by identify_siblings() for Multi-Threading/Multi-Core capable
+	 * cpu's
+	 */
+	c->tpc = c->cpp = c->num_log = 1;
+	c->socket_id = c->cpu;
+
+	identify_siblings(c);
 #endif
 	c->ppn = cpuid.field.ppn;
 	c->number = cpuid.field.number;
diff -Nru linux-2.6.11-rc3/arch/ia64/kernel/smpboot.c linux-mcmt/arch/ia64/kernel/smpboot.c
--- linux-2.6.11-rc3/arch/ia64/kernel/smpboot.c	2005-02-02 17:55:22.000000000 -0800
+++ linux-mcmt/arch/ia64/kernel/smpboot.c	2005-02-15 11:55:49.000000000 -0800
@@ -1,9 +1,17 @@
 /*
  * SMP boot-related support
  *
+ * Copyright (C) 2005 Suresh Siddha <suresh.b.siddha@intel.com>
+ * Copyright (C) 2004 Gordon Jin <gordon.jin@intel.com>
+ * Copyright (C) 2004 Rohit Seth <rohit.seth@intel.com>
  * Copyright (C) 1998-2003 Hewlett-Packard Co
  *	David Mosberger-Tang <davidm@hpl.hp.com>
  *
+ * 01/30/05 Suresh Siddha <suresh.b.siddha@intel.com>
+ *						Setup cpu_sibling_map and cpu_core_map
+ * 12/26/04 Jin Gordon <gordon.jin@intel.com>
+ * 12/26/04 Rohit Seth <rohit.seth@intel.com>
+ *						Add multi-threading and multi-core detection
  * 01/05/16 Rohit Seth <rohit.seth@intel.com>	Moved SMP booting functions from smp.c to here.
  * 01/04/27 David Mosberger <davidm@hpl.hp.com>	Added ITC synching code.
  * 02/07/31 David Mosberger <davidm@hpl.hp.com>	Switch over to hotplug-CPU boot-sequence.
@@ -90,6 +98,11 @@
 cpumask_t cpu_possible_map;
 EXPORT_SYMBOL(cpu_possible_map);
 
+cpumask_t cpu_core_map[NR_CPUS] __cacheline_aligned;
+cpumask_t cpu_sibling_map[NR_CPUS] __cacheline_aligned;
+int smp_num_siblings = 1;
+int smp_num_cpucores = 1;
+
 /* which logical CPU number maps to which CPU (physical APIC ID) */
 volatile int ia64_cpu_to_sapicid[NR_CPUS];
 EXPORT_SYMBOL(ia64_cpu_to_sapicid);
@@ -650,6 +663,55 @@
 
 	printk(KERN_INFO "Total of %d processors activated (%lu.%02lu BogoMIPS).\n",
 	       (int)num_online_cpus(), bogosum/(500000/HZ), (bogosum/(5000/HZ))%100);
+	for (cpu = 0; cpu < NR_CPUS; cpu++) {
+		int siblings = 0, cores = 0;
+		int i;
+
+		if (!cpu_online(cpu))
+			continue;
+
+		if (cpu_data(cpu)->tpc > 1) {
+			for (i = 0; i < NR_CPUS; i++) {
+				if (!cpu_isset(i, cpu_online_map))
+					continue;
+				if ((cpu_data(cpu)->socket_id ==
+				     cpu_data(i)->socket_id) &&
+				    (cpu_data(cpu)->cid == cpu_data(i)->cid)) {
+					siblings++;
+					cpu_set(i, cpu_sibling_map[cpu]);
+				}
+			}
+		} else {
+			siblings++;
+			cpu_set(cpu, cpu_sibling_map[cpu]);
+		}
+
+		if (siblings != cpu_data(cpu)->tpc)
+			printk(KERN_WARNING "WARNING: %d thread siblings found"
+			       " for CPU%d, should be %d\n", siblings, cpu,
+			       cpu_data(cpu)->tpc);
+
+		if (cpu_data(cpu)->cpp > 1) {
+			for (i = 0; i < NR_CPUS; i++) {
+				if (!cpu_isset(i, cpu_online_map))
+					continue;
+				if ((cpu_data(cpu)->socket_id ==
+				     cpu_data(i)->socket_id)) {
+					cores++;
+					cpu_set(i, cpu_core_map[cpu]);
+				}
+			}
+		} else {
+			cores = siblings;
+			cpu_core_map[cpu] = cpu_sibling_map[cpu];
+		}
+
+		if (cores/siblings != cpu_data(cpu)->cpp)
+			printk(KERN_WARNING "WARNING: %d cores found for CPU%d"
+			       ",should be %d\n", cores, cpu,
+			       cpu_data(cpu)->cpp);
+	}
+
 }
 
 int __devinit
@@ -705,3 +767,98 @@
 		       ia64_sal_strerror(sal_ret));
 }
 
+
+/*
+ * mt[] is a temporary store for all info returned by PAL_LOGICAL_TO_PHYSICAL,
+ * to be copied into cpuinfo_ia64 when the specific cpu comes.
+ */
+
+static struct {
+	__u32   socket_id;
+	__u16   cid;
+	__u16   tid;
+	__u16   proc_la;
+} mt[NR_CPUS] __devinit;
+
+/* how many entries have been used in mt[] */
+static unsigned int mt_count __devinit;
+
+static int
+check_for_new_socket (__u16 la, struct cpuinfo_ia64 *c)
+{
+	int i;
+	__u32 sid = c->socket_id;
+
+	for (i = 0; i < mt_count; i++) {
+		if ((mt[i].proc_la == la) && (mt[i].socket_id == sid)) {
+			c->cid = mt[i].cid;
+			c->tid = mt[i].tid;
+			return 1; /* not a new socket */
+		}
+	}
+	return 0;
+}
+
+/*
+ * identify_siblings(cpu) gets called from identify_cpu.
+ * This populates the information related to logical execution units
+ * in per_cpu_data structure.
+ * TBD: fix the cpu hotplug case.
+ */
+void
+identify_siblings (struct cpuinfo_ia64 *c)
+{
+	s64 status;
+	u16 pltid;
+	u64 proc_la;
+	int count, i;
+	pal_logical_to_physical_t info;
+
+	if (smp_num_cpucores == 1 && smp_num_siblings == 1) 
+		return;
+
+	if ((status = ia64_pal_logical_to_phys(0, &info)) 
+	     != PAL_STATUS_SUCCESS) {
+		printk(KERN_ERR "ia64_pal_logical_to_phys failed with %ld\n", 
+		       status);
+		return;
+	}
+	if ((status = ia64_sal_physical_id_info(&pltid)) != PAL_STATUS_SUCCESS) {
+		printk(KERN_ERR "ia64_sal_pltid failed with %ld\n", status);
+		return;
+	}
+	if ((status = ia64_pal_fixed_addr(&proc_la)) != PAL_STATUS_SUCCESS) {
+		printk(KERN_ERR "ia64_pal_fixed_addr failed with %ld\n", status);
+		return;
+	}
+
+	c->socket_id =  (pltid << 8) | info.overview_ppid;
+	c->cpp = info.overview_cpp;
+	c->tpc = info.overview_tpc;
+	count = c->num_log = info.overview_num_log;
+	/*
+	 * This function will search the existing data structure mt to find out
+	 * if we need to make PAL calls to get the cid/tid information.
+	 * If the socket does not exist then fill in the entries and return.
+	 */
+	if (check_for_new_socket(proc_la, c))
+		return;
+	for (i = 0; i < count; i++) {
+		if (i && (status = ia64_pal_logical_to_phys(i, &info)) 
+			  != PAL_STATUS_SUCCESS) {
+                	printk(KERN_ERR "ia64_pal_logical_to_phys failed"
+					" with %ld\n", status);
+                	return;
+		}
+		if (info.log2_la == proc_la) {
+			c->cid = info.log1_cid;
+			c->tid = info.log1_tid;
+		} else {
+			mt[mt_count].socket_id = c->socket_id;
+			mt[mt_count].cid = info.log1_cid;
+			mt[mt_count].tid = info.log1_tid;
+			mt[mt_count].proc_la = info.log2_la;
+			mt_count++;
+		}
+	}
+}
diff -Nru linux-2.6.11-rc3/include/asm-ia64/pal.h linux-mcmt/include/asm-ia64/pal.h
--- linux-2.6.11-rc3/include/asm-ia64/pal.h	2005-02-02 17:56:11.000000000 -0800
+++ linux-mcmt/include/asm-ia64/pal.h	2005-02-15 11:56:09.000000000 -0800
@@ -67,6 +67,9 @@
 #define PAL_REGISTER_INFO	39	/* return AR and CR register information*/
 #define PAL_SHUTDOWN		40	/* enter processor shutdown state */
 #define PAL_PREFETCH_VISIBILITY	41	/* Make Processor Prefetches Visible */
+#define PAL_LOGICAL_TO_PHYSICAL 42	/* get logical to physical  processor
+					 * mapping info
+					 */
 
 #define PAL_COPY_PAL		256	/* relocate PAL procedures and PAL PMI */
 #define PAL_HALT_INFO		257	/* return the low power capabilities of processor */
@@ -1559,6 +1562,73 @@
 	return iprv.status;
 }
 
+/* data structure for getting information on logical to physical mappings */
+typedef union pal_log_overview_u {
+	struct {
+		u64	num_log		:16,	/* Total number of logical
+						 * processors on this die
+						 */
+			tpc		:8,	/* Threads per core */
+			reserved3	:8,	/* Reserved */
+			cpp		:8,	/* Cores per processor */
+			reserved2	:8,	/* Reserved */
+			ppid		:8,	/* Physical processor ID */
+			reserved1	:8;	/* Reserved */
+	} overview_bits;
+	u64 overview_data;
+} pal_log_overview_t;
+
+typedef union pal_proc_n_log_info1_u{
+	struct {
+		u64	tid		:16,	/* Thread id */
+			reserved2	:16,	/* Reserved */
+			cid		:16,	/* Core id */
+			reserved1	:16;	/* Reserved */
+	} ppli1_bits;
+	u64	ppli1_data;
+} pal_proc_n_log_info1_t;
+
+typedef union pal_proc_n_log_info2_u {
+	struct {
+		u64	la		:16,	/* Logical address */
+			reserved	:48;	/* Reserved */
+	} ppli2_bits;
+	u64	ppli2_data;
+} pal_proc_n_log_info2_t;
+
+typedef struct pal_logical_to_physical_s
+{
+	pal_log_overview_t overview;
+	pal_proc_n_log_info1_t ppli1;
+	pal_proc_n_log_info2_t ppli2;
+} pal_logical_to_physical_t;
+
+#define overview_num_log	overview.overview_bits.num_log
+#define overview_tpc		overview.overview_bits.tpc
+#define overview_cpp		overview.overview_bits.cpp
+#define overview_ppid		overview.overview_bits.ppid
+#define log1_tid		ppli1.ppli1_bits.tid
+#define log1_cid		ppli1.ppli1_bits.cid
+#define log2_la			ppli2.ppli2_bits.la
+
+/* Get information on logical to physical processor mappings. */
+static inline s64 
+ia64_pal_logical_to_phys(u64 proc_number, pal_logical_to_physical_t *mapping)
+{
+	struct ia64_pal_retval iprv;
+
+	PAL_CALL(iprv, PAL_LOGICAL_TO_PHYSICAL, proc_number, 0, 0);
+
+	if (iprv.status == PAL_STATUS_SUCCESS)
+	{
+		if (proc_number == 0)
+			mapping->overview.overview_data = iprv.v0;
+		mapping->ppli1.ppli1_data = iprv.v1;
+		mapping->ppli2.ppli2_data = iprv.v2;
+	}
+
+	return iprv.status;
+}
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_IA64_PAL_H */
diff -Nru linux-2.6.11-rc3/include/asm-ia64/processor.h linux-mcmt/include/asm-ia64/processor.h
--- linux-2.6.11-rc3/include/asm-ia64/processor.h	2005-02-02 17:55:36.000000000 -0800
+++ linux-mcmt/include/asm-ia64/processor.h	2005-02-15 11:55:49.000000000 -0800
@@ -162,6 +162,13 @@
 #ifdef CONFIG_SMP
 	__u64 loops_per_jiffy;
 	int cpu;
+	__u32 socket_id;	/* physical processor socket id */
+	__u16 cid;		/* core id */
+	__u16 tid;		/* thread id */
+	__u16 num_log;		/* Total number of logical processors on
+				 * this socket that were successfully booted */
+	__u8  cpp;		/* Cores per processor socket */
+	__u8  tpc;		/* Threads per core */
 #endif
 
 	/* CPUID-derived information: */
diff -Nru linux-2.6.11-rc3/include/asm-ia64/sal.h linux-mcmt/include/asm-ia64/sal.h
--- linux-2.6.11-rc3/include/asm-ia64/sal.h	2005-02-02 17:56:53.000000000 -0800
+++ linux-mcmt/include/asm-ia64/sal.h	2005-02-15 11:55:49.000000000 -0800
@@ -91,6 +91,7 @@
 #define SAL_PCI_CONFIG_READ		0x01000010
 #define SAL_PCI_CONFIG_WRITE		0x01000011
 #define SAL_FREQ_BASE			0x01000012
+#define SAL_PHYSICAL_ID_INFO		0x01000013
 
 #define SAL_UPDATE_PAL			0x01000020
 
@@ -815,6 +816,19 @@
 	return isrv.status;
 }
 
+/* Get physical processor die mapping in the platform. */
+static inline s64
+ia64_sal_physical_id_info(u16 *splid)
+{
+	struct ia64_sal_retval isrv;
+	SAL_CALL(isrv, SAL_PHYSICAL_ID_INFO, 0, 0, 0, 0, 0, 0, 0);
+
+	if (splid)
+		*splid = isrv.v0;
+
+	return isrv.status;
+}
+
 extern unsigned long sal_platform_features;
 
 extern int (*salinfo_platform_oemdata)(const u8 *, u8 **, u64 *);
diff -Nru linux-2.6.11-rc3/include/asm-ia64/smp.h linux-mcmt/include/asm-ia64/smp.h
--- linux-2.6.11-rc3/include/asm-ia64/smp.h	2005-02-02 17:55:52.000000000 -0800
+++ linux-mcmt/include/asm-ia64/smp.h	2005-02-15 11:55:49.000000000 -0800
@@ -39,6 +39,10 @@
 extern char no_int_routing __devinitdata;
 
 extern cpumask_t cpu_online_map;
+extern cpumask_t cpu_core_map[NR_CPUS];
+extern cpumask_t cpu_sibling_map[NR_CPUS];
+extern int smp_num_siblings;
+extern int smp_num_cpucores;
 extern void __iomem *ipi_base_addr;
 extern unsigned char smp_int_redirect;
 

Re: [patch]:MC/MT enabling/identification for IA-64

[David Mosberger]

>>>>> On Wed, 16 Feb 2005 11:19:05 -0800, "Seth, Rohit" <rohit.seth@intel.com> said:

  Rohit> Please find attached the patch that enables identification of
  Rohit> multi-core and multi-thread on IA-64 processors.  This patch
  Rohit> adds 4 new fields in /proc/cpuinfo to clearly identify each
  Rohit> logical execution unit.  Two new data structures cpu_core_map
  Rohit> and cpu_sibling_map are also implemented.  This information
  Rohit> will be used in scheduler (and possibly by other kernel
  Rohit> components as well).  A new SAL call (SAL_PHYSICAL_ID_INFO)
  Rohit> and a PAL call (PAL_LOGICAL_TO_PHYSICAL) are also added for
  Rohit> identification purposes.

  Rohit> TBD items: Hot-plug of logical processors
  Rohit> Scheduler enhancements:  We will send out this patch in next few days.

  Rohit> Comments and feedback welcome.

The main question I have: why is it necessary/beneficial for the
scheduler to distinguish between cores and sockets?

A nit, in setup.c:

  * Architecture-specific setup.
  *
+ * Copyright (C) 2004 Gordon Jin <gordon.jin@intel.com>
+ * Copyright (C) 2004 Suresh Siddha <suresh.b.siddha@intel.com>
  * Copyright (C) 1998-2001, 2003-2004 Hewlett-Packard Co

Normally (at least in the ia64-files), new Copyright headers are
appended (aside from that, you might want to reconsider the usefulness
of adding per-contributor Copyright headers; I started that trend back
in the early days when things were closed source etc.; I obviously
don't know Intel's position, but in HP, a per-company Copyright header
is preferred nowadays).  Same goes for smpboot.c.

+#ifdef CONFIG_SMP
+	seq_printf(m,
+		   "physical id\t: %u\n"
+		   "siblings\t: %u\n"
+		   "cpu core id\t: %u\n"
+		   "cpu cores\t: %u\n",
+		   c->socket_id, c->tpc * c->cpp, c->cid, c->cpp);
+#endif

Please use blanks, not tabs, so it's consistent with the other output
in cpuinfo.  Is it really useful/necessary to print the number of
(thread) siblings and cores for each entry?  We don't do that for the
CPU count either.  I'm thinking it might be cleaner to just print
a triplet like this:

	socket id  : 128
	core id    :   0
        thread id  :   1

This way, it should be more obvious that socket id/core id/thread id
are the unique coordinates of the CPU, no?

Can we put this in a header:

@@ -503,6 +555,7 @@
 void
 identify_cpu (struct cpuinfo_ia64 *c)
 {
+	extern void identify_siblings (struct cpuinfo_ia64 *);
 	union {
 		unsigned long bits[5];
 		struct {

I know there are other places where we do this, but we should
gradually clean them up, not make the situation worse.

Hmmh, the naming here is curious:

+	__u32 socket_id;	/* physical processor socket id */
+	__u16 cid;		/* core id */
+	__u16 tid;		/* thread id */
+	__u16 num_log;		/* Total number of logical processors on
+				 * this socket that were successfully booted */
+	__u8  cpp;		/* Cores per processor socket */
+	__u8  tpc;		/* Threads per core */

To be honest, I'd lean towards using longer names (socket_id, core_id,
thread_id, cores_per_socket, thread_per_core).  There is only a few
places where these are used so readability should be improved without
getting grossly wide lines.

	--david

RE: [patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

David Mosberger <mailto:davidm@napali.hpl.hp.com> wrote on Friday,
February 18, 2005 3:42 PM:

>>>>>> On Wed, 16 Feb 2005 11:19:05 -0800, "Seth, Rohit"
>>>>>> <rohit.seth@intel.com> said: 
>
> 
> The main question I have: why is it necessary/beneficial for the
> scheduler to distinguish between cores and sockets?
> 

Architecturally cores on the same physical socket can share caches and
bus interfaces.  We will have to do analysis to configure the different
sched domains appropiately.

> A nit, in setup.c:
> 
>   * Architecture-specific setup.
>   *
> + * Copyright (C) 2004 Gordon Jin <gordon.jin@intel.com>
> + * Copyright (C) 2004 Suresh Siddha <suresh.b.siddha@intel.com>
>   * Copyright (C) 1998-2001, 2003-2004 Hewlett-Packard Co
> 
> Normally (at least in the ia64-files), new Copyright headers are
> appended (aside from that, you might want to reconsider the usefulness
> of adding per-contributor Copyright headers; I started that trend back
> in the early days when things were closed source etc.; I obviously
> don't know Intel's position, but in HP, a per-company Copyright header
> is preferred nowadays).  Same goes for smpboot.c.
> 

Will fix this.

> +#ifdef CONFIG_SMP
> +	seq_printf(m,
> +		   "physical id\t: %u\n"
> +		   "siblings\t: %u\n"
> +		   "cpu core id\t: %u\n"
> +		   "cpu cores\t: %u\n",
> +		   c->socket_id, c->tpc * c->cpp, c->cid, c->cpp);
> +#endif
> 
> Please use blanks, not tabs, so it's consistent with the other output
> in cpuinfo.  

I agree that the format of these fields should match the format of other
fields in cpuinfo.....though it will be nice if we have the same format
as that of i386 cpuinfo output.

> Is it really useful/necessary to print the number of
> (thread) siblings and cores for each entry?  We don't do that for the
> CPU count either.  I'm thinking it might be cleaner to just print
> a triplet like this:
> 
> 	socket id  : 128
> 	core id    :   0
>         thread id  :   1
> 
> This way, it should be more obvious that socket id/core id/thread id
> are the unique coordinates of the CPU, no?
> 

I was thinking of this information as something that apps can use to
find the information about which logical execution units (leu) are
threads on the same core, which leu are on the same package and so on.
This is similar to i386(HT enabled processors) where siblings gives the
number of threads on the same package.

Complete identification (giving all the ids) can be put in
/proc/pal/cpu[x]/logical_cpu_info or through sysfs. Please let me know
your comments.

> Can we put this in a header:
> 
> @@ -503,6 +555,7 @@
>  void
>  identify_cpu (struct cpuinfo_ia64 *c)
>  {
> +	extern void identify_siblings (struct cpuinfo_ia64 *);
>  	union {
>  		unsigned long bits[5];
>  		struct {
> 
> I know there are other places where we do this, but we should
> gradually clean them up, not make the situation worse.

Will move that declation to header.

> 
> Hmmh, the naming here is curious:
> 
> +	__u32 socket_id;	/* physical processor socket id */
> +	__u16 cid;		/* core id */
> +	__u16 tid;		/* thread id */
> +	__u16 num_log;		/* Total number of logical processors on
> +				 * this socket that were successfully
booted */
> +	__u8  cpp;		/* Cores per processor socket */
> +	__u8  tpc;		/* Threads per core */
> 
> To be honest, I'd lean towards using longer names (socket_id, core_id,
> thread_id, cores_per_socket, thread_per_core).  There is only a few
> places where these are used so readability should be improved without
> getting grossly wide lines.
> 

Typically the field names in various PAL call related data structures
match their definition in SDM....

Thanks,
-rohit

RE: [patch]:MC/MT enabling/identification for IA-64

[David Mosberger]

>>>>> On Tue, 22 Feb 2005 10:24:40 -0800, "Seth, Rohit" <rohit.seth@intel.com> said:

  Rohit> I agree that the format of these fields should match the
  Rohit> format of other fields in cpuinfo.....though it will be nice
  Rohit> if we have the same format as that of i386 cpuinfo output.

I'm not sure there is much point to that:

 (a) The contents of /proc/cpuinfo is by definition architecture-specific

 (b) Applications _should_ allow any whitespace when parsing
     /proc/cpuinfo, so in properly-written applications, it shouldn't
     matter whether whitespace or tabs are used.

Changing the formatting of /proc/cpuinfo only runs the risk of
existing tools, without benefit to properly written applications.

  Rohit> I was thinking of this information as something that apps can
  Rohit> use to find the information about which logical execution
  Rohit> units (leu) are threads on the same core, which leu are on
  Rohit> the same package and so on.  This is similar to i386(HT
  Rohit> enabled processors) where siblings gives the number of
  Rohit> threads on the same package.

I'm not a fan of including redudant info in /proc files.

  Rohit> Typically the field names in various PAL call related data
  Rohit> structures match their definition in SDM....

I don't think we need to constrain ourselves too much to what the PAL
names are.  That code is part of the kernel and it should be readable.

	--david

RE: [patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

David Mosberger <mailto:davidm@napali.hpl.hp.com> wrote on Thursday,
February 24, 2005 5:27 PM:

>>>>>> On Tue, 22 Feb 2005 10:24:40 -0800, "Seth, Rohit"
>>>>>> <rohit.seth@intel.com> said: 
> 
>   Rohit> I agree that the format of these fields should match the
>   Rohit> format of other fields in cpuinfo.....though it will be nice
>   Rohit> if we have the same format as that of i386 cpuinfo output.
> 
> I'm not sure there is much point to that:
> 
>  (a) The contents of /proc/cpuinfo is by definition
> architecture-specific 
> 

No arguments there.

>  (b) Applications _should_ allow any whitespace when parsing
>      /proc/cpuinfo, so in properly-written applications, it shouldn't
>      matter whether whitespace or tabs are used.
> 
> Changing the formatting of /proc/cpuinfo only runs the risk of
> existing tools, without benefit to properly written applications.
> 

I hope there are not that many apps that have this
behavior...particularly if they are (/want to be) portable.

In any case, I will have this changed (tabs replaced with white space)
in next rev of this patch.

>   Rohit> I was thinking of this information as something that apps can
>   Rohit> use to find the information about which logical execution
>   Rohit> units (leu) are threads on the same core, which leu are on
>   Rohit> the same package and so on.  This is similar to i386(HT
>   Rohit> enabled processors) where siblings gives the number of
>   Rohit> threads on the same package.
> 
> I'm not a fan of including redudant info in /proc files.
> 

I think we should have some consistency (wherever possible) in
/proc/cpuinfo fields across architectures.  This will help applications
writers.  Currently siblings and cpu core fields are already added for
i386 and x86_64.  

Thoughts?


>   Rohit> Typically the field names in various PAL call related data
>   Rohit> structures match their definition in SDM....
> 
> I don't think we need to constrain ourselves too much to what the PAL
> names are.  That code is part of the kernel and it should be readable.
> 

I probably should send you a bigger PAL field cleanup patch first ;-)

RE: [patch]:MC/MT enabling/identification for IA-64

[David Mosberger]

>>>>> On Thu, 24 Feb 2005 18:22:44 -0800, "Seth, Rohit" <rohit.seth@intel.com> said:

  >> Changing the formatting of /proc/cpuinfo only runs the risk of
  >> existing tools, without benefit to properly written applications.

  Rohit> I hope there are not that many apps that have this
  Rohit> behavior...particularly if they are (/want to be) portable.

I _hope_ the same, but what's the point to even run the risk of
breaking existing apps when there is no real benefit?

  Rohit> I think we should have some consistency (wherever possible)
  Rohit> in /proc/cpuinfo fields across architectures.  This will help
  Rohit> applications writers.  Currently siblings and cpu core fields
  Rohit> are already added for i386 and x86_64.

Hmmh, I don't see a "cpu core id" field in the x86 code.  Am I missing
something?

Printing the same number (# siblings/cores) for each CPU over and over
just seems stupid, especially when instead you can print a useful
value (core id/thread id).

	--david

RE: [patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

David Mosberger <mailto:davidm@napali.hpl.hp.com> wrote on Thursday,
February 24, 2005 10:12 PM:

>>>>>> On Thu, 24 Feb 2005 18:22:44 -0800, "Seth, Rohit"
>>>>>> <rohit.seth@intel.com> said: 

>   Rohit> I think we should have some consistency (wherever possible)
>   Rohit> in /proc/cpuinfo fields across architectures.  This will help
>   Rohit> applications writers.  Currently siblings and cpu core fields
>   Rohit> are already added for i386 and x86_64.
> 
> Hmmh, I don't see a "cpu core id" field in the x86 code.  Am I missing
> something?
> 

"Cpu Cores" identifying the number of cores in a physical package is
present in x86_64 and not there in i386 tree.  

Will following work:

Physical id: Identifying the unique physical package id for LEU
Core id: Identifying the unique core id on that specific physical
package
Siblings: Identifying the total number LEUs on that physical package.

Siblings is the one that apps have already started using on i386.

-rohit

Re: [patch]:MC/MT enabling/identification for IA-64

[Ashok Raj]

On Fri, Feb 25, 2005 at 12:12:34PM -0800, Seth, Rohit wrote:
> 
>    Will following work:
> 
>    Physical id: Identifying the unique physical package id for LEU

The spec states that physical processor id or (die id) is not required
to be unique.  Are you depending on this being unique?

also PAL_LOGICAL_TO_PHYSICAL states that num_log (or total number of logical
processors in this die) doesnt necessarily need to be cpp * tpc)

just fyi: even if the numbers might be the same in most cases, its possible
that they could be different, because one thread didnt boot due to some 
failure)

>    Core id: Identifying the unique core id on that specific physical
>    package
>    Siblings: Identifying the total number LEUs on that physical package.
> 
>    Siblings is the one that apps have already started using on i386.
> 
>    -rohit
>    -
>    To  unsubscribe from this list: send the line "unsubscribe linux-ia64"
>    in
>    the body of a message to majordomo@vger.kernel.org
>    More majordomo info at  [2]http://vger.kernel.org/majordomo-info.html
> 
> References
> 
>    1. mailto:davidm@napali.hpl.hp.com
>    2. http://vger.kernel.org/majordomo-info.html

-- 
Cheers,
Ashok Raj

RE: [patch]:MC/MT enabling/identification for IA-64

[David Mosberger]

>>>>> On Fri, 25 Feb 2005 12:12:34 -0800, "Seth, Rohit" <rohit.seth@intel.com> said:

  Rohit> "Cpu Cores" identifying the number of cores in a physical
  Rohit> package is present in x86_64 and not there in i386 tree.

  Rohit> Will following work:

  Rohit> Physical id: Identifying the unique physical package id for LEU
  Rohit> Core id: Identifying the unique core id on that specific physical
  Rohit> package
  Rohit> Siblings: Identifying the total number LEUs on that physical package.

Well, I think it would be useful if we can look at /proc/cpuinfo and
get a mapping from Linux CPU number to "topological address" (socket
id, core id, thread id).  With that plus taskset, it would be very
easy to run a test with, for example, two processes pinned to the two
threads on the same core or pinned to separate cores, etc.

  Rohit> Siblings is the one that apps have already started using on i386.

What is it being used for?

	--david

RE: [patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

Ashok Raj <mailto:ashok.raj@intel.com> wrote on Friday, February 25,
2005 12:27 PM:

> On Fri, Feb 25, 2005 at 12:12:34PM -0800, Seth, Rohit wrote:
>> 
>>    Will following work:
>> 
>>    Physical id: Identifying the unique physical package id for LEU
> 
> The spec states that physical processor id or (die id) is not required
> to be unique.  Are you depending on this being unique?
> 

SAL_PHYSICAL_ID_INFO (in conjunction with PAL_LOGICAL_TO_PHYSICAL) is
guranteed to give unique values for each package in a system.

> also PAL_LOGICAL_TO_PHYSICAL states that num_log (or total number of
> logical processors in this die) doesnt necessarily need to be cpp *
> tpc) 
>

Siblings will be equal to num_log.
 
> just fyi: even if the numbers might be the same in most cases, its
> possible 
> that they could be different, because one thread didnt boot due to
> some 
> failure)
> 

RE: [patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

David Mosberger <> wrote on Friday, February 25, 2005 12:47 PM:

>>>>>> On Fri, 25 Feb 2005 12:12:34 -0800, "Seth, Rohit"
>>>>>> <rohit.seth@intel.com> said: 
> 
>   Rohit> "Cpu Cores" identifying the number of cores in a physical
>   Rohit> package is present in x86_64 and not there in i386 tree.
> 
>   Rohit> Will following work:
> 
>   Rohit> Physical id: Identifying the unique physical package id for
>   LEU Rohit> Core id: Identifying the unique core id on that specific
>   physical Rohit> package
>   Rohit> Siblings: Identifying the total number LEUs on that physical
> package. 
> 
> Well, I think it would be useful if we can look at /proc/cpuinfo and
> get a mapping from Linux CPU number to "topological address" (socket
> id, core id, thread id).  With that plus taskset, it would be very
> easy to run a test with, for example, two processes pinned to the two
> threads on the same core or pinned to separate cores, etc.
> 

For pinning a task (or set of tasks) to same core or multiple cores, you
only need package id and core id.  The thread information is really not
required.  Right?

>   Rohit> Siblings is the one that apps have already started using on
> i386. 
> 
> What is it being used for?
>

AFAIK, an easy way for apps (for licensing etc.) to find out if and how
many LEUs are present without doing any other calculation.

-rohit

RE: [patch]:MC/MT enabling/identification for IA-64

[David Mosberger]

>>>>> On Fri, 25 Feb 2005 13:15:05 -0800, "Seth, Rohit" <rohit.seth@intel.com> said:

  Rohit> For pinning a task (or set of tasks) to same core or multiple
  Rohit> cores, you only need package id and core id.  The thread
  Rohit> information is really not required.  Right?

Yeah, that's true.  At least I can't come up with a good reason of
wanting to know which thread exactly you're running on.  Having said
that, from a user-perspective, looking at two entries in /proc/cpuinfo
and seeing that the socket id/core id are the same and only the thread
ids are different probably would give me more of a warm and fuzzy
feeling.

  Rohit> AFAIK, an easy way for apps (for licensing etc.) to find out
  Rohit> if and how many LEUs are present without doing any other
  Rohit> calculation.

Yeah, I guess "MAX(thread_id) + 1" does count as a "calculation".  And
I thought computers where good at that... ;-)

	--david

RE: [patch]:MC/MT enabling/identification for IA-64

[Seth, Rohit]

David Mosberger <> wrote on Friday, February 25, 2005 2:47 PM:

>>>>>> On Fri, 25 Feb 2005 13:15:05 -0800, "Seth, Rohit"
>>>>>> <rohit.seth@intel.com> said: 
> 
>   Rohit> For pinning a task (or set of tasks) to same core or multiple
>   Rohit> cores, you only need package id and core id.  The thread
>   Rohit> information is really not required.  Right?
> 
> Yeah, that's true.  At least I can't come up with a good reason of
> wanting to know which thread exactly you're running on.  Having said
> that, from a user-perspective, looking at two entries in /proc/cpuinfo
> and seeing that the socket id/core id are the same and only the thread
> ids are different probably would give me more of a warm and fuzzy
> feeling.
> 

Well, I was only trying to evaluate what is the most optimal set of
entries for /proc/cpuinfo.  I'm sure there are other entries
present...for precisely the same reason that you listed.

>   Rohit> AFAIK, an easy way for apps (for licensing etc.) to find out
>   Rohit> if and how many LEUs are present without doing any other
>   Rohit> calculation.
> 
> Yeah, I guess "MAX(thread_id) + 1" does count as a "calculation".  And
> I thought computers where good at that... ;-)
> 


Siblings have no direct relationship with thread_id.  Tids(threads ids)
are not guranteed to be contiguous.  

-rohit

RE: [patch]:MC/MT enabling/identification for IA-64

[David Mosberger]

>>>>> On Fri, 25 Feb 2005 15:01:07 -0800, "Seth, Rohit" <rohit.seth@intel.com> said:

  >>  Yeah, I guess "MAX(thread_id) + 1" does count as a
  >> "calculation".  And I thought computers where good at that... ;-)

  Rohit> Siblings have no direct relationship with thread_id.
  Rohit> Tids(threads ids) are not guranteed to be contiguous.

Ah, that's a good point.

Thanks,

	--david