is TIF_NOTIFY_RESUME used?

is TIF_NOTIFY_RESUME used?

[Stephane Eranian]

Hello,

For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2
there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the
first 7 bits of the thread flag. Many architectures, including IA-64, rely
on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK)
tested on kernel exit reside in the low 8-bit or 7-bit because they use
instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit
immediate.

On IA-64, adding that one perfmon flag (as bit 7) would cause some
restructuring in the kernel exit path but also in all the lightweight syscall
handlers.

I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME 
was never set nor used anywhere in any architecture. Is that really the case?

If so, we could get rid of it and free up one low-order TIF bit.

Thanks.

-- 
-Stephane

Re: is TIF_NOTIFY_RESUME used?

[Andrew Morton]

On Tue, 22 May 2007 05:47:13 -0700
Stephane Eranian <eranian@hpl.hp.com> wrote:

> Hello,
> 
> For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2
> there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the
> first 7 bits of the thread flag. Many architectures, including IA-64, rely
> on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK)
> tested on kernel exit reside in the low 8-bit or 7-bit because they use
> instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit
> immediate.
> 
> On IA-64, adding that one perfmon flag (as bit 7) would cause some
> restructuring in the kernel exit path but also in all the lightweight syscall
> handlers.
> 
> I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME 
> was never set nor used anywhere in any architecture. Is that really the case?
> 
> If so, we could get rid of it and free up one low-order TIF bit.
> 

My grepping argees with yours.  The only place where TIF_NOTIFY_RESUME gets
altered is in ./arch/ia64/kernel/perfmon.c.

Re: is TIF_NOTIFY_RESUME used?

[Stephane Eranian]

Andrew,

On Tue, May 22, 2007 at 09:02:10AM -0700, Andrew Morton wrote:
> On Tue, 22 May 2007 05:47:13 -0700
> Stephane Eranian <eranian@hpl.hp.com> wrote:
> 
> > Hello,
> > 
> > For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2
> > there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the
> > first 7 bits of the thread flag. Many architectures, including IA-64, rely
> > on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK)
> > tested on kernel exit reside in the low 8-bit or 7-bit because they use
> > instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit
> > immediate.
> > 
> > On IA-64, adding that one perfmon flag (as bit 7) would cause some
> > restructuring in the kernel exit path but also in all the lightweight syscall
> > handlers.
> > 
> > I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME 
> > was never set nor used anywhere in any architecture. Is that really the case?
> > 
> > If so, we could get rid of it and free up one low-order TIF bit.
> > 
> 
> My grepping argees with yours.  The only place where TIF_NOTIFY_RESUME gets
> altered is in ./arch/ia64/kernel/perfmon.c.

Yes, and that is with the old IA-64 code. In the new one I used a dedicated
TIF flag.

Shall we just get rid of the flag, then?

-- 
-Stephane

Re: is TIF_NOTIFY_RESUME used?

[Andrew Morton]

On Tue, 22 May 2007 09:07:37 -0700
Stephane Eranian <eranian@hpl.hp.com> wrote:

> Andrew,
> 
> On Tue, May 22, 2007 at 09:02:10AM -0700, Andrew Morton wrote:
> > On Tue, 22 May 2007 05:47:13 -0700
> > Stephane Eranian <eranian@hpl.hp.com> wrote:
> > 
> > > Hello,
> > > 
> > > For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2
> > > there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the
> > > first 7 bits of the thread flag. Many architectures, including IA-64, rely
> > > on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK)
> > > tested on kernel exit reside in the low 8-bit or 7-bit because they use
> > > instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit
> > > immediate.
> > > 
> > > On IA-64, adding that one perfmon flag (as bit 7) would cause some
> > > restructuring in the kernel exit path but also in all the lightweight syscall
> > > handlers.
> > > 
> > > I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME 
> > > was never set nor used anywhere in any architecture. Is that really the case?
> > > 
> > > If so, we could get rid of it and free up one low-order TIF bit.
> > > 
> > 
> > My grepping argees with yours.  The only place where TIF_NOTIFY_RESUME gets
> > altered is in ./arch/ia64/kernel/perfmon.c.
> 
> Yes, and that is with the old IA-64 code. In the new one I used a dedicated
> TIF flag.
> 
> Shall we just get rid of the flag, then?
> 

I'd say so, yes.

[PATCH 1/2] Define new percpu interface for shared data -- version 3

[Fenghua Yu]

Changes from previous version:
Fix a compilation error for ARM

per cpu data section contains two types of data. One set which is exclusively
accessed by the local cpu and the other set which is  per cpu, but also shared
by remote cpus. In the current kernel, these two sets are not clearely
separated out. This can potentially cause the same data cacheline shared
between the two sets of data, which will result in unnecessary bouncing of the
cacheline between cpus.

One way to fix the problem is to cacheline align the remotely accessed per cpu
data, both at the beginning and at the end. Because of the padding at both ends,
this will likely cause some memory wastage and also the interface to achieve
this is not clean.

This patch:

Moves the remotely accessed per cpu data (which is currently marked
as ____cacheline_aligned_in_smp) into a different section, where all the data
elements are cacheline aligned. And as such, this differentiates the local
only data and remotely accessed data cleanly.

igned-off-by: Fenghua Yu <fenghua.yu@intel.com>
Acked-by: Suresh Siddha <suresh.b.siddha@intel.com>

---

 arch/alpha/kernel/vmlinux.lds.S   |    5 +----
 arch/arm/kernel/vmlinux.lds.S     |    1 +
 arch/cris/arch-v32/vmlinux.lds.S  |    5 +----
 arch/frv/kernel/vmlinux.lds.S     |    5 +----
 arch/i386/kernel/vmlinux.lds.S    |    7 +------
 arch/ia64/kernel/vmlinux.lds.S    |    1 +
 arch/m32r/kernel/vmlinux.lds.S    |    5 +----
 arch/mips/kernel/vmlinux.lds.S    |    5 +----
 arch/parisc/kernel/vmlinux.lds.S  |    7 +++----
 arch/powerpc/kernel/vmlinux.lds.S |    7 +------
 arch/ppc/kernel/vmlinux.lds.S     |    5 +----
 arch/s390/kernel/vmlinux.lds.S    |    5 +----
 arch/sh/kernel/vmlinux.lds.S      |    5 +----
 arch/sh64/kernel/vmlinux.lds.S    |    5 +----
 arch/sparc/kernel/vmlinux.lds.S   |    5 +----
 arch/sparc64/kernel/vmlinux.lds.S |    5 +----
 arch/x86_64/kernel/vmlinux.lds.S  |    6 ++----
 arch/xtensa/kernel/vmlinux.lds.S  |    5 +----
 include/asm-generic/percpu.h      |    8 ++++++++
 include/asm-generic/vmlinux.lds.h |    8 ++++++++
 include/asm-i386/percpu.h         |    5 +++++
 include/asm-ia64/percpu.h         |   10 ++++++++++
 include/asm-powerpc/percpu.h      |    7 +++++++
 include/asm-s390/percpu.h         |    7 +++++++
 include/asm-sparc64/percpu.h      |    7 +++++++
 include/asm-x86_64/percpu.h       |    7 +++++++
 26 files changed, 80 insertions(+), 68 deletions(-)

diff -Nurp linux-2.6.22-rc1-mm1.0/arch/alpha/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/alpha/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/alpha/kernel/vmlinux.lds.S	2007-05-18 15:37:15.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/alpha/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -69,10 +69,7 @@ SECTIONS
   . = ALIGN(8);
   SECURITY_INIT
 
-  . = ALIGN(8192);
-  __per_cpu_start = .;
-  .data.percpu : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(8192)
 
   . = ALIGN(2*8192);
   __init_end = .;
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/arm/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/arm/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/arm/kernel/vmlinux.lds.S	2007-05-18 15:37:15.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/arm/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -66,6 +66,7 @@ SECTIONS
 		. = ALIGN(4096);
 		__per_cpu_start = .;
 			*(.data.percpu)
+			*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
 #ifndef CONFIG_XIP_KERNEL
 		__init_begin = _stext;
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/cris/arch-v32/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/cris/arch-v32/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/cris/arch-v32/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/cris/arch-v32/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -91,10 +91,7 @@ SECTIONS
 	}
 	SECURITY_INIT
 
-	. =  ALIGN (8192);
-	__per_cpu_start = .;
-	.data.percpu  : { *(.data.percpu) }
-	__per_cpu_end = .;
+	PERCPU(8192)
 
 #ifdef CONFIG_BLK_DEV_INITRD
 	.init.ramfs : {
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/frv/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/frv/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/frv/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/frv/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -57,10 +57,7 @@ SECTIONS
   __alt_instructions_end = .;
  .altinstr_replacement : { *(.altinstr_replacement) }
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
 
 #ifdef CONFIG_BLK_DEV_INITRD
   . = ALIGN(4096);
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/i386/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/i386/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/i386/kernel/vmlinux.lds.S	2007-05-18 15:38:09.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/i386/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -178,12 +178,7 @@ SECTIONS
 	__initramfs_end = .;
   }
 #endif
-  . = ALIGN(4096);
-  .data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) {
-	__per_cpu_start = .;
-	*(.data.percpu)
-	__per_cpu_end = .;
-  }
+  PERCPU(4096)
   . = ALIGN(4096);
   /* freed after init ends here */
 	
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/ia64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/ia64/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/ia64/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/ia64/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -206,6 +206,7 @@ SECTIONS
 	{
 		__per_cpu_start = .;
 		*(.data.percpu)
+		*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
 	}
   . = __phys_per_cpu_start + PERCPU_PAGE_SIZE;	/* ensure percpu data fits
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/m32r/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/m32r/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/m32r/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/m32r/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -110,10 +110,7 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
   . = ALIGN(4096);
   __init_end = .;
   /* freed after init ends here */
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/mips/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/mips/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/mips/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/mips/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -119,10 +119,7 @@ SECTIONS
   .init.ramfs : { *(.init.ramfs) }
   __initramfs_end = .;
 #endif
-  . = ALIGN(_PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(_PAGE_SIZE)
   . = ALIGN(_PAGE_SIZE);
   __init_end = .;
   /* freed after init ends here */
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/parisc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/parisc/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/parisc/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/parisc/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -181,10 +181,9 @@ SECTIONS
   .init.ramfs : { *(.init.ramfs) }
   __initramfs_end = .;
 #endif
-  . = ALIGN(ASM_PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+
+  PERCPU(ASM_PAGE_SIZE)
+
   . = ALIGN(ASM_PAGE_SIZE);
   __init_end = .;
   /* freed after init ends here */
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/powerpc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/powerpc/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/powerpc/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/powerpc/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -139,12 +139,7 @@ SECTIONS
 		__initramfs_end = .;
 	}
 #endif
-	. = ALIGN(PAGE_SIZE);
-	.data.percpu : {
-		__per_cpu_start = .;
-		*(.data.percpu)
-		__per_cpu_end = .;
-	}
+	PERCPU(PAGE_SIZE)
 
 	. = ALIGN(8);
 	.machine.desc : {
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/ppc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/ppc/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/ppc/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/ppc/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -130,10 +130,7 @@ SECTIONS
   __ftr_fixup : { *(__ftr_fixup) }
   __stop___ftr_fixup = .;
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
 
 #ifdef CONFIG_BLK_DEV_INITRD
   . = ALIGN(4096);
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/s390/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/s390/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/s390/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/s390/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -107,10 +107,7 @@ SECTIONS
   . = ALIGN(2);
   __initramfs_end = .;
 #endif
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
   . = ALIGN(4096);
   __init_end = .;
   /* freed after init ends here */
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sh/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sh/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/sh/kernel/vmlinux.lds.S	2007-05-18 15:38:09.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/sh/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -60,10 +60,7 @@ SECTIONS
   . = ALIGN(PAGE_SIZE);
   __nosave_end = .;
 
-  . = ALIGN(PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(PAGE_SIZE)
   .data.cacheline_aligned : { *(.data.cacheline_aligned) }
 
   _edata = .;			/* End of data section */
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sh64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sh64/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/sh64/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/sh64/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -85,10 +85,7 @@ SECTIONS
   . = ALIGN(PAGE_SIZE);
   .data.page_aligned : C_PHYS(.data.page_aligned) { *(.data.page_aligned) }
 
-  . = ALIGN(PAGE_SIZE);
-  __per_cpu_start = .;
-  .data.percpu : C_PHYS(.data.percpu) { *(.data.percpu) }
-  __per_cpu_end = . ;
+  PERCPU(PAGE_SIZE)
   .data.cacheline_aligned : C_PHYS(.data.cacheline_aligned) { *(.data.cacheline_aligned) }
 
   _edata = .;			/* End of data section */
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sparc/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sparc/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/sparc/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/sparc/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -65,10 +65,7 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
   . = ALIGN(4096);
   __init_end = .;
   . = ALIGN(32);
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/sparc64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/sparc64/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/sparc64/kernel/vmlinux.lds.S	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/sparc64/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -89,10 +89,7 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(8192);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(8192)
   . = ALIGN(8192);
   __init_end = .;
   __bss_start = .;
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/x86_64/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/x86_64/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/x86_64/kernel/vmlinux.lds.S	2007-05-18 15:38:09.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/x86_64/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -203,10 +203,8 @@ SECTIONS
   __initramfs_end = .;
 #endif
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
+
   . = ALIGN(4096);
   __init_end = .;
 
diff -Nurp linux-2.6.22-rc1-mm1.0/arch/xtensa/kernel/vmlinux.lds.S linux-2.6.22-rc1-mm1.1/arch/xtensa/kernel/vmlinux.lds.S
--- linux-2.6.22-rc1-mm1.0/arch/xtensa/kernel/vmlinux.lds.S	2007-05-18 15:37:16.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/arch/xtensa/kernel/vmlinux.lds.S	2007-05-18 15:40:08.000000000 -0700
@@ -198,10 +198,7 @@ SECTIONS
   __ftr_fixup : { *(__ftr_fixup) }
   __stop___ftr_fixup = .;
 
-  . = ALIGN(4096);
-  __per_cpu_start = .;
-  .data.percpu  : { *(.data.percpu) }
-  __per_cpu_end = .;
+  PERCPU(4096)
 
 #ifdef CONFIG_BLK_DEV_INITRD
   . = ALIGN(4096);
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-generic/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-generic/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-generic/percpu.h	2007-05-18 15:37:19.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-generic/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -14,6 +14,11 @@ extern unsigned long __per_cpu_offset[NR
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu) (*({				\
 	extern int simple_identifier_##var(void);	\
@@ -34,6 +39,9 @@ do {								\
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
+
 #define per_cpu(var, cpu)			(*((void)(cpu), &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var
 #define __raw_get_cpu_var(var)			per_cpu__##var
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-generic/vmlinux.lds.h linux-2.6.22-rc1-mm1.1/include/asm-generic/vmlinux.lds.h
--- linux-2.6.22-rc1-mm1.0/include/asm-generic/vmlinux.lds.h	2007-05-18 15:38:10.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-generic/vmlinux.lds.h	2007-05-18 15:41:53.000000000 -0700
@@ -251,3 +251,11 @@
   	*(.initcall7.init)						\
   	*(.initcall7s.init)
 
+#define PERCPU(align)							\
+	. = ALIGN(align);						\
+	__per_cpu_start = .;						\
+	.data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) {		\
+		*(.data.percpu)						\
+		*(.data.percpu.shared_aligned)				\
+	}								\
+	__per_cpu_end = .;
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-i386/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-i386/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-i386/percpu.h	2007-05-18 15:37:19.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-i386/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -54,6 +54,11 @@ extern unsigned long __per_cpu_offset[];
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 /* We can use this directly for local CPU (faster). */
 DECLARE_PER_CPU(unsigned long, this_cpu_off);
 
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-ia64/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-ia64/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-ia64/percpu.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-ia64/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -29,6 +29,16 @@
 	__attribute__((__section__(".data.percpu")))		\
 	__SMALL_ADDR_AREA __typeof__(type) per_cpu__##name
 
+#ifdef CONFIG_SMP
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)			\
+	__attribute__((__section__(".data.percpu.shared_aligned")))	\
+	__SMALL_ADDR_AREA __typeof__(type) per_cpu__##name		\
+	____cacheline_aligned_in_smp
+#else
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+	DEFINE_PER_CPU(type, name)
+#endif
+
 /*
  * Pretty much a literal copy of asm-generic/percpu.h, except that percpu_modcopy() is an
  * external routine, to avoid include-hell.
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-powerpc/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-powerpc/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-powerpc/percpu.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-powerpc/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -20,6 +20,11 @@
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu) (*RELOC_HIDE(&per_cpu__##var, __per_cpu_offset(cpu)))
 #define __get_cpu_var(var) (*RELOC_HIDE(&per_cpu__##var, __my_cpu_offset()))
@@ -40,6 +45,8 @@ extern void setup_per_cpu_areas(void);
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define per_cpu(var, cpu)			(*((void)(cpu), &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-s390/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-s390/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-s390/percpu.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-s390/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -41,6 +41,11 @@ extern unsigned long __per_cpu_offset[NR
     __attribute__((__section__(".data.percpu"))) \
     __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 #define __get_cpu_var(var) __reloc_hide(var,S390_lowcore.percpu_offset)
 #define __raw_get_cpu_var(var) __reloc_hide(var,S390_lowcore.percpu_offset)
 #define per_cpu(var,cpu) __reloc_hide(var,__per_cpu_offset[cpu])
@@ -59,6 +64,8 @@ do {								\
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define __get_cpu_var(var) __reloc_hide(var,0)
 #define __raw_get_cpu_var(var) __reloc_hide(var,0)
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-sparc64/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-sparc64/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-sparc64/percpu.h	2007-05-18 15:37:25.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-sparc64/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -17,6 +17,11 @@ extern unsigned long __per_cpu_shift;
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_aligned_in_smp
+
 register unsigned long __local_per_cpu_offset asm("g5");
 
 /* var is in discarded region: offset to particular copy we want */
@@ -36,6 +41,8 @@ do {								\
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define per_cpu(var, cpu)			(*((void)cpu, &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var
diff -Nurp linux-2.6.22-rc1-mm1.0/include/asm-x86_64/percpu.h linux-2.6.22-rc1-mm1.1/include/asm-x86_64/percpu.h
--- linux-2.6.22-rc1-mm1.0/include/asm-x86_64/percpu.h	2007-05-18 15:37:25.000000000 -0700
+++ linux-2.6.22-rc1-mm1.1/include/asm-x86_64/percpu.h	2007-05-18 15:40:08.000000000 -0700
@@ -20,6 +20,11 @@
 #define DEFINE_PER_CPU(type, name) \
     __attribute__((__section__(".data.percpu"))) __typeof__(type) per_cpu__##name
 
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)		\
+    __attribute__((__section__(".data.percpu.shared_aligned"))) \
+    __typeof__(type) per_cpu__##name				\
+    ____cacheline_internodealigned_in_smp
+
 /* var is in discarded region: offset to particular copy we want */
 #define per_cpu(var, cpu) (*({				\
 	extern int simple_identifier_##var(void);	\
@@ -46,6 +51,8 @@ extern void setup_per_cpu_areas(void);
 
 #define DEFINE_PER_CPU(type, name) \
     __typeof__(type) per_cpu__##name
+#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name)	\
+    DEFINE_PER_CPU(type, name)
 
 #define per_cpu(var, cpu)			(*((void)(cpu), &per_cpu__##var))
 #define __get_cpu_var(var)			per_cpu__##var

Re: is TIF_NOTIFY_RESUME used?

[Stephane Eranian]

Andrew,

On Tue, May 22, 2007 at 09:15:17AM -0700, Andrew Morton wrote:
> On Tue, 22 May 2007 09:07:37 -0700
> Stephane Eranian <eranian@hpl.hp.com> wrote:
> 
> > Andrew,
> > 
> > On Tue, May 22, 2007 at 09:02:10AM -0700, Andrew Morton wrote:
> > > On Tue, 22 May 2007 05:47:13 -0700
> > > Stephane Eranian <eranian@hpl.hp.com> wrote:
> > > 
> > > > Hello,
> > > > 
> > > > For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2
> > > > there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the
> > > > first 7 bits of the thread flag. Many architectures, including IA-64, rely
> > > > on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK)
> > > > tested on kernel exit reside in the low 8-bit or 7-bit because they use
> > > > instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit
> > > > immediate.
> > > > 
> > > > On IA-64, adding that one perfmon flag (as bit 7) would cause some
> > > > restructuring in the kernel exit path but also in all the lightweight syscall
> > > > handlers.
> > > > 
> > > > I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME 
> > > > was never set nor used anywhere in any architecture. Is that really the case?
> > > > 
> > > > If so, we could get rid of it and free up one low-order TIF bit.
> > > > 
> > > 
> > > My grepping argees with yours.  The only place where TIF_NOTIFY_RESUME gets
> > > altered is in ./arch/ia64/kernel/perfmon.c.
> > 
> > Yes, and that is with the old IA-64 code. In the new one I used a dedicated
> > TIF flag.
> > 
> > Shall we just get rid of the flag, then?
> > 
> 
> I'd say so, yes.

Do you want a single patch or small patches broken-down by arch (and there are many...)?

-- 
-Stephane

Re: is TIF_NOTIFY_RESUME used?

[Andrew Morton]

On Tue, 22 May 2007 15:51:35 -0700
Stephane Eranian <eranian@hpl.hp.com> wrote:

> Andrew,
> 
> On Tue, May 22, 2007 at 09:15:17AM -0700, Andrew Morton wrote:
> > On Tue, 22 May 2007 09:07:37 -0700
> > Stephane Eranian <eranian@hpl.hp.com> wrote:
> > 
> > > Andrew,
> > > 
> > > On Tue, May 22, 2007 at 09:02:10AM -0700, Andrew Morton wrote:
> > > > On Tue, 22 May 2007 05:47:13 -0700
> > > > Stephane Eranian <eranian@hpl.hp.com> wrote:
> > > > 
> > > > > Hello,
> > > > > 
> > > > > For perfmon, we need a couple of TIF bits. It seems that with 2.6.22-rc2
> > > > > there is now a TIF_RESTORE_SIGMASK which uses the last remaining bit in the
> > > > > first 7 bits of the thread flag. Many architectures, including IA-64, rely
> > > > > on the fact that some of the TIF flags (TIF_ALL_WORKMASK or TIF_ALL_WORK)
> > > > > tested on kernel exit reside in the low 8-bit or 7-bit because they use
> > > > > instructions (such as add r1=imm8,r2 on IA-64) which operate on 8 or 7 bit
> > > > > immediate.
> > > > > 
> > > > > On IA-64, adding that one perfmon flag (as bit 7) would cause some
> > > > > restructuring in the kernel exit path but also in all the lightweight syscall
> > > > > handlers.
> > > > > 
> > > > > I looked at all the low order TIF flags and found that TIF_NOTIFY_RESUME 
> > > > > was never set nor used anywhere in any architecture. Is that really the case?
> > > > > 
> > > > > If so, we could get rid of it and free up one low-order TIF bit.
> > > > > 
> > > > 
> > > > My grepping argees with yours.  The only place where TIF_NOTIFY_RESUME gets
> > > > altered is in ./arch/ia64/kernel/perfmon.c.
> > > 
> > > Yes, and that is with the old IA-64 code. In the new one I used a dedicated
> > > TIF flag.
> > > 
> > > Shall we just get rid of the flag, then?
> > > 
> > 
> > I'd say so, yes.
> 
> Do you want a single patch or small patches broken-down by arch (and there are many...)?

One patch is OK for this, thanks.

[PATCH] remove unused TIF_NOTIFY_RESUME flag

[Stephane Eranian]

Hello,

As per the recent discussion, the folowing patch removes the TIF_NOTIFY_RESUME
flag from all processor architectures. The flag was not used except on IA-64
for the perfmon subsystem. For IA-64, the patch replaces the flag with a new
dedicated TIF flag. This provides an extra low-order bit for the TIF flags.
I tested the kernel compiles on i386,x86_64,mips,ia64,powerpc.

The patch is relative to 2.6.22-rc2 but applies smoothly to 2.6.22-rc2-git3.

Changelog:
	- remove unused TIF_NOTIFY_RESUME flag for all processor architectures.
	   The flag was not used excecpt on IA-64 where the patch replaces it with
	   TIF_PERFMON_WORK.

signed-off-by: stephane eranian <eranian@hpl.hp.com>


diff -urNp linux-2.6.22.orig/arch/arm/kernel/entry-common.S linux-2.6.22.base/arch/arm/kernel/entry-common.S
--- linux-2.6.22.orig/arch/arm/kernel/entry-common.S	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/arch/arm/kernel/entry-common.S	2007-05-22 13:38:39.000000000 -0700
@@ -46,7 +46,7 @@ fast_work_pending:
 work_pending:
 	tst	r1, #_TIF_NEED_RESCHED
 	bne	work_resched
-	tst	r1, #_TIF_NOTIFY_RESUME | _TIF_SIGPENDING
+	tst	r1, #_TIF_SIGPENDING
 	beq	no_work_pending
 	mov	r0, sp				@ 'regs'
 	mov	r2, why				@ 'syscall'
diff -urNp linux-2.6.22.orig/arch/arm26/kernel/entry.S linux-2.6.22.base/arch/arm26/kernel/entry.S
--- linux-2.6.22.orig/arch/arm26/kernel/entry.S	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/arch/arm26/kernel/entry.S	2007-05-22 13:39:00.000000000 -0700
@@ -194,7 +194,7 @@ fast_work_pending:
 work_pending:
 	tst	r1, #_TIF_NEED_RESCHED
 	bne	work_resched
-	tst	r1, #_TIF_NOTIFY_RESUME | _TIF_SIGPENDING
+	tst	r1, #_TIF_SIGPENDING
 	beq	no_work_pending
 	mov	r0, sp				@ 'regs'
 	mov	r2, why				@ 'syscall'
diff -urNp linux-2.6.22.orig/arch/ia64/kernel/perfmon.c linux-2.6.22.base/arch/ia64/kernel/perfmon.c
--- linux-2.6.22.orig/arch/ia64/kernel/perfmon.c	2007-05-21 01:39:04.000000000 -0700
+++ linux-2.6.22.base/arch/ia64/kernel/perfmon.c	2007-05-22 13:41:23.000000000 -0700
@@ -591,13 +591,13 @@ pfm_set_task_notify(struct task_struct *
 	struct thread_info *info;
 
 	info = (struct thread_info *) ((char *) task + IA64_TASK_SIZE);
-	set_bit(TIF_NOTIFY_RESUME, &info->flags);
+	set_bit(TIF_PERFMON_WORK, &info->flags);
 }
 
 static inline void
 pfm_clear_task_notify(void)
 {
-	clear_thread_flag(TIF_NOTIFY_RESUME);
+	clear_thread_flag(TIF_PERFMON_WORK);
 }
 
 static inline void
diff -urNp linux-2.6.22.orig/arch/powerpc/kernel/entry_64.S linux-2.6.22.base/arch/powerpc/kernel/entry_64.S
--- linux-2.6.22.orig/arch/powerpc/kernel/entry_64.S	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/arch/powerpc/kernel/entry_64.S	2007-05-22 13:39:55.000000000 -0700
@@ -546,7 +546,6 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_ISER
 	rfid
 	b	.	/* prevent speculative execution */
 
-/* Note: this must change if we start using the TIF_NOTIFY_RESUME bit */
 do_work:
 #ifdef CONFIG_PREEMPT
 	andi.	r0,r3,MSR_PR	/* Returning to user mode? */
diff -urNp linux-2.6.22.orig/arch/x86_64/kernel/entry.S linux-2.6.22.base/arch/x86_64/kernel/entry.S
--- linux-2.6.22.orig/arch/x86_64/kernel/entry.S	2007-05-21 01:39:05.000000000 -0700
+++ linux-2.6.22.base/arch/x86_64/kernel/entry.S	2007-05-22 13:40:14.000000000 -0700
@@ -282,7 +282,7 @@ sysret_careful:
 sysret_signal:
 	TRACE_IRQS_ON
 	sti
-	testl $(_TIF_SIGPENDING|_TIF_NOTIFY_RESUME|_TIF_SINGLESTEP),%edx
+	testl $(_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
 	jz    1f
 
 	/* Really a signal */
@@ -375,7 +375,7 @@ int_very_careful:
 	jmp int_restore_rest
 	
 int_signal:
-	testl $(_TIF_NOTIFY_RESUME|_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
+	testl $(_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
 	jz 1f
 	movq %rsp,%rdi		# &ptregs -> arg1
 	xorl %esi,%esi		# oldset -> arg2
@@ -599,7 +599,7 @@ retint_careful:
 	jmp retint_check
 	
 retint_signal:
-	testl $(_TIF_SIGPENDING|_TIF_NOTIFY_RESUME|_TIF_SINGLESTEP),%edx
+	testl $(_TIF_SIGPENDING|_TIF_SINGLESTEP),%edx
 	jz    retint_swapgs
 	TRACE_IRQS_ON
 	sti
diff -urNp linux-2.6.22.orig/include/asm-alpha/thread_info.h linux-2.6.22.base/include/asm-alpha/thread_info.h
--- linux-2.6.22.orig/include/asm-alpha/thread_info.h	2007-05-21 01:39:09.000000000 -0700
+++ linux-2.6.22.base/include/asm-alpha/thread_info.h	2007-05-22 13:26:24.000000000 -0700
@@ -67,25 +67,22 @@ register struct thread_info *__current_t
  * TIF_SYSCALL_TRACE is known to be 0 via blbs.
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* poll_idle is polling NEED_RESCHED */
-#define TIF_DIE_IF_KERNEL	5	/* dik recursion lock */
-#define TIF_UAC_NOPRINT		6	/* see sysinfo.h */
-#define TIF_UAC_NOFIX		7
-#define TIF_UAC_SIGBUS		8
-#define TIF_MEMDIE		9
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* poll_idle is polling NEED_RESCHED */
+#define TIF_DIE_IF_KERNEL	4	/* dik recursion lock */
+#define TIF_UAC_NOPRINT		5	/* see sysinfo.h */
+#define TIF_UAC_NOFIX		6
+#define TIF_UAC_SIGBUS		7
+#define TIF_MEMDIE		8
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
 
 /* Work to do on interrupt/exception return.  */
-#define _TIF_WORK_MASK		(_TIF_NOTIFY_RESUME	\
-				 | _TIF_SIGPENDING	\
+#define _TIF_WORK_MASK		(_TIF_SIGPENDING	\
 				 | _TIF_NEED_RESCHED)
 
 /* Work to do on any return to userspace.  */
diff -urNp linux-2.6.22.orig/include/asm-arm/thread_info.h linux-2.6.22.base/include/asm-arm/thread_info.h
--- linux-2.6.22.orig/include/asm-arm/thread_info.h	2007-05-21 01:39:10.000000000 -0700
+++ linux-2.6.22.base/include/asm-arm/thread_info.h	2007-05-22 13:26:48.000000000 -0700
@@ -135,22 +135,19 @@ extern void iwmmxt_task_switch(struct th
 /*
  * thread information flags:
  *  TIF_SYSCALL_TRACE	- syscall trace active
- *  TIF_NOTIFY_RESUME	- resumption notification requested
  *  TIF_SIGPENDING	- signal pending
  *  TIF_NEED_RESCHED	- rescheduling necessary
  *  TIF_USEDFPU		- FPU was used by this task this quantum (SMP)
  *  TIF_POLLING_NRFLAG	- true if poll_idle() is polling TIF_NEED_RESCHED
  */
-#define TIF_NOTIFY_RESUME	0
-#define TIF_SIGPENDING		1
-#define TIF_NEED_RESCHED	2
+#define TIF_SIGPENDING		0
+#define TIF_NEED_RESCHED	1
 #define TIF_SYSCALL_TRACE	8
 #define TIF_POLLING_NRFLAG	16
 #define TIF_USING_IWMMXT	17
 #define TIF_MEMDIE		18
 #define TIF_FREEZE		19
 
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
diff -urNp linux-2.6.22.orig/include/asm-arm26/thread_info.h linux-2.6.22.base/include/asm-arm26/thread_info.h
--- linux-2.6.22.orig/include/asm-arm26/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-arm26/thread_info.h	2007-05-22 13:27:13.000000000 -0700
@@ -110,21 +110,18 @@ extern void free_thread_info(struct thre
 /*
  * thread information flags:
  *  TIF_SYSCALL_TRACE	- syscall trace active
- *  TIF_NOTIFY_RESUME	- resumption notification requested
  *  TIF_SIGPENDING	- signal pending
  *  TIF_NEED_RESCHED	- rescheduling necessary
  *  TIF_USEDFPU		- FPU was used by this task this quantum (SMP)
  *  TIF_POLLING_NRFLAG	- true if poll_idle() is polling TIF_NEED_RESCHED
  */
-#define TIF_NOTIFY_RESUME	0
-#define TIF_SIGPENDING		1
-#define TIF_NEED_RESCHED	2
+#define TIF_SIGPENDING		0
+#define TIF_NEED_RESCHED	1
 #define TIF_SYSCALL_TRACE	8
 #define TIF_USED_FPU		16
 #define TIF_POLLING_NRFLAG	17
 #define TIF_MEMDIE		18
 
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
diff -urNp linux-2.6.22.orig/include/asm-avr32/thread_info.h linux-2.6.22.base/include/asm-avr32/thread_info.h
--- linux-2.6.22.orig/include/asm-avr32/thread_info.h	2007-05-21 01:39:10.000000000 -0700
+++ linux-2.6.22.base/include/asm-avr32/thread_info.h	2007-05-22 13:27:49.000000000 -0700
@@ -74,20 +74,18 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE       0       /* syscall trace active */
-#define TIF_NOTIFY_RESUME       1       /* resumption notification requested */
-#define TIF_SIGPENDING          2       /* signal pending */
-#define TIF_NEED_RESCHED        3       /* rescheduling necessary */
-#define TIF_POLLING_NRFLAG      4       /* true if poll_idle() is polling
+#define TIF_SIGPENDING          1       /* signal pending */
+#define TIF_NEED_RESCHED        2       /* rescheduling necessary */
+#define TIF_POLLING_NRFLAG      3       /* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_BREAKPOINT		5	/* true if we should break after return */
-#define TIF_SINGLE_STEP		6	/* single step after next break */
-#define TIF_MEMDIE		7
-#define TIF_RESTORE_SIGMASK	8	/* restore signal mask in do_signal */
-#define TIF_CPU_GOING_TO_SLEEP	9	/* CPU is entering sleep 0 mode */
+#define TIF_BREAKPOINT		4	/* true if we should break after return */
+#define TIF_SINGLE_STEP		5	/* single step after next break */
+#define TIF_MEMDIE		6
+#define TIF_RESTORE_SIGMASK	7	/* restore signal mask in do_signal */
+#define TIF_CPU_GOING_TO_SLEEP	8	/* CPU is entering sleep 0 mode */
 #define TIF_USERSPACE		31      /* true if FS sets userspace */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1 << TIF_POLLING_NRFLAG)
diff -urNp linux-2.6.22.orig/include/asm-blackfin/thread_info.h linux-2.6.22.base/include/asm-blackfin/thread_info.h
--- linux-2.6.22.orig/include/asm-blackfin/thread_info.h	2007-05-21 01:39:10.000000000 -0700
+++ linux-2.6.22.base/include/asm-blackfin/thread_info.h	2007-05-22 13:28:25.000000000 -0700
@@ -118,18 +118,16 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE              5
-#define TIF_RESTORE_SIGMASK	6	/* restore signal mask in do_signal() */
-#define TIF_FREEZE              7       /* is freezing for suspend */
+#define TIF_MEMDIE              4
+#define TIF_RESTORE_SIGMASK	5	/* restore signal mask in do_signal() */
+#define TIF_FREEZE              6       /* is freezing for suspend */
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
diff -urNp linux-2.6.22.orig/include/asm-cris/thread_info.h linux-2.6.22.base/include/asm-cris/thread_info.h
--- linux-2.6.22.orig/include/asm-cris/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-cris/thread_info.h	2007-05-22 13:31:49.000000000 -0700
@@ -79,14 +79,12 @@ struct thread_info {
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		17
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
diff -urNp linux-2.6.22.orig/include/asm-frv/thread_info.h linux-2.6.22.base/include/asm-frv/thread_info.h
--- linux-2.6.22.orig/include/asm-frv/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-frv/thread_info.h	2007-05-22 13:33:27.000000000 -0700
@@ -108,18 +108,16 @@ register struct thread_info *__current_t
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
-#define TIF_RESTORE_SIGMASK	6	/* restore signal mask in do_signal() */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
+#define TIF_RESTORE_SIGMASK	5	/* restore signal mask in do_signal() */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		17	/* OOM killer killed process */
 #define TIF_FREEZE		18	/* freezing for suspend */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1 << TIF_SINGLESTEP)
diff -urNp linux-2.6.22.orig/include/asm-h8300/thread_info.h linux-2.6.22.base/include/asm-h8300/thread_info.h
--- linux-2.6.22.orig/include/asm-h8300/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-h8300/thread_info.h	2007-05-22 13:28:48.000000000 -0700
@@ -86,16 +86,14 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE		5
+#define TIF_MEMDIE		4
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
diff -urNp linux-2.6.22.orig/include/asm-i386/thread_info.h linux-2.6.22.base/include/asm-i386/thread_info.h
--- linux-2.6.22.orig/include/asm-i386/thread_info.h	2007-05-21 01:39:10.000000000 -0700
+++ linux-2.6.22.base/include/asm-i386/thread_info.h	2007-05-22 13:23:35.000000000 -0700
@@ -124,22 +124,20 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
-#define TIF_SYSCALL_EMU		6	/* syscall emulation active */
-#define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
-#define TIF_SECCOMP		8	/* secure computing */
-#define TIF_RESTORE_SIGMASK	9	/* restore signal mask in do_signal() */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
+#define TIF_SYSCALL_EMU		5	/* syscall emulation active */
+#define TIF_SYSCALL_AUDIT	6	/* syscall auditing active */
+#define TIF_SECCOMP		7	/* secure computing */
+#define TIF_RESTORE_SIGMASK	8	/* restore signal mask in do_signal() */
 #define TIF_MEMDIE		16
 #define TIF_DEBUG		17	/* uses debug registers */
 #define TIF_IO_BITMAP		18	/* uses I/O bitmap */
 #define TIF_FREEZE		19	/* is freezing for suspend */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)
diff -urNp linux-2.6.22.orig/include/asm-ia64/thread_info.h linux-2.6.22.base/include/asm-ia64/thread_info.h
--- linux-2.6.22.orig/include/asm-ia64/thread_info.h	2007-05-21 01:39:10.000000000 -0700
+++ linux-2.6.22.base/include/asm-ia64/thread_info.h	2007-05-23 02:00:11.000000000 -0700
@@ -79,13 +79,13 @@ struct thread_info {
  * - pending work-to-be-done flags are in least-significant 16 bits, other flags
  *   in top 16 bits
  */
-#define TIF_NOTIFY_RESUME	0	/* resumption notification requested */
-#define TIF_SIGPENDING		1	/* signal pending */
-#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
-#define TIF_SYSCALL_TRACE	3	/* syscall trace active */
-#define TIF_SYSCALL_AUDIT	4	/* syscall auditing active */
-#define TIF_SINGLESTEP		5	/* restore singlestep on return to user mode */
-#define TIF_RESTORE_SIGMASK	6	/* restore signal mask in do_signal() */
+#define TIF_SIGPENDING		0	/* signal pending */
+#define TIF_NEED_RESCHED	1	/* rescheduling necessary */
+#define TIF_SYSCALL_TRACE	2	/* syscall trace active */
+#define TIF_SYSCALL_AUDIT	3	/* syscall auditing active */
+#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
+#define TIF_RESTORE_SIGMASK	5	/* restore signal mask in do_signal() */
+#define TIF_PERFMON_WORK	6	/* work for pfm_handle_work() */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		17
 #define TIF_MCA_INIT		18	/* this task is processing MCA or INIT */
@@ -96,8 +96,8 @@ struct thread_info {
 #define _TIF_SYSCALL_AUDIT	(1 << TIF_SYSCALL_AUDIT)
 #define _TIF_SINGLESTEP		(1 << TIF_SINGLESTEP)
 #define _TIF_SYSCALL_TRACEAUDIT	(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SINGLESTEP)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_RESTORE_SIGMASK	(1 << TIF_RESTORE_SIGMASK)
+#define _TIF_PERFMON_WORK	(1 << TIF_PERFMON_WORK)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1 << TIF_POLLING_NRFLAG)
@@ -106,7 +106,9 @@ struct thread_info {
 #define _TIF_FREEZE		(1 << TIF_FREEZE)
 
 /* "work to do on user-return" bits */
-#define TIF_ALLWORK_MASK	(_TIF_NOTIFY_RESUME|_TIF_SIGPENDING|_TIF_NEED_RESCHED|_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_RESTORE_SIGMASK)
+#define TIF_ALLWORK_MASK	(_TIF_SIGPENDING|_TIF_PERFMON_WORK|_TIF_SYSCALL_AUDIT|\
+				 _TIF_NEED_RESCHED| _TIF_SYSCALL_TRACE|\
+				 _TIF_RESTORE_SIGMASK)
 /* like TIF_ALLWORK_BITS but sans TIF_SYSCALL_TRACE or TIF_SYSCALL_AUDIT */
 #define TIF_WORK_MASK		(TIF_ALLWORK_MASK&~(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT))
 
diff -urNp linux-2.6.22.orig/include/asm-m32r/thread_info.h linux-2.6.22.base/include/asm-m32r/thread_info.h
--- linux-2.6.22.orig/include/asm-m32r/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-m32r/thread_info.h	2007-05-22 13:35:09.000000000 -0700
@@ -146,17 +146,15 @@ static inline unsigned int get_thread_fa
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 					/* 31..28 fault code */
 #define TIF_MEMDIE		17
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)
diff -urNp linux-2.6.22.orig/include/asm-m68knommu/thread_info.h linux-2.6.22.base/include/asm-m68knommu/thread_info.h
--- linux-2.6.22.orig/include/asm-m68knommu/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-m68knommu/thread_info.h	2007-05-22 13:32:30.000000000 -0700
@@ -83,16 +83,14 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE		5
+#define TIF_MEMDIE		4
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
diff -urNp linux-2.6.22.orig/include/asm-mips/thread_info.h linux-2.6.22.base/include/asm-mips/thread_info.h
--- linux-2.6.22.orig/include/asm-mips/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-mips/thread_info.h	2007-05-22 13:38:13.000000000 -0700
@@ -109,11 +109,10 @@ register struct thread_info *__current_t
  * - pending work-to-be-done flags are in LSW
  * - other flags in MSW
  */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SYSCALL_AUDIT	4	/* syscall auditing active */
-#define TIF_SECCOMP		5	/* secure computing */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SYSCALL_AUDIT	3	/* syscall auditing active */
+#define TIF_SECCOMP		4	/* secure computing */
 #define TIF_RESTORE_SIGMASK	9	/* restore signal mask in do_signal() */
 #define TIF_USEDFPU		16	/* FPU was used by this task this quantum (SMP) */
 #define TIF_POLLING_NRFLAG	17	/* true if poll_idle() is polling TIF_NEED_RESCHED */
@@ -122,7 +121,6 @@ register struct thread_info *__current_t
 #define TIF_SYSCALL_TRACE	31	/* syscall trace active */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SYSCALL_AUDIT	(1<<TIF_SYSCALL_AUDIT)
diff -urNp linux-2.6.22.orig/include/asm-parisc/thread_info.h linux-2.6.22.base/include/asm-parisc/thread_info.h
--- linux-2.6.22.orig/include/asm-parisc/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-parisc/thread_info.h	2007-05-22 13:35:52.000000000 -0700
@@ -56,23 +56,21 @@ struct thread_info {
  * thread information flags
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling TIF_NEED_RESCHED */
-#define TIF_32BIT               5       /* 32 bit binary */
-#define TIF_MEMDIE		6
-#define TIF_RESTORE_SIGMASK	7	/* restore saved signal mask */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling TIF_NEED_RESCHED */
+#define TIF_32BIT               4       /* 32 bit binary */
+#define TIF_MEMDIE		5
+#define TIF_RESTORE_SIGMASK	6	/* restore saved signal mask */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1 << TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1 << TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1 << TIF_POLLING_NRFLAG)
 #define _TIF_32BIT		(1 << TIF_32BIT)
 #define _TIF_RESTORE_SIGMASK	(1 << TIF_RESTORE_SIGMASK)
 
-#define _TIF_USER_WORK_MASK     (_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | \
+#define _TIF_USER_WORK_MASK     (_TIF_SIGPENDING | \
                                  _TIF_NEED_RESCHED | _TIF_RESTORE_SIGMASK)
 
 #endif /* __KERNEL__ */
diff -urNp linux-2.6.22.orig/include/asm-powerpc/thread_info.h linux-2.6.22.base/include/asm-powerpc/thread_info.h
--- linux-2.6.22.orig/include/asm-powerpc/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-powerpc/thread_info.h	2007-05-22 13:25:36.000000000 -0700
@@ -107,26 +107,24 @@ static inline struct thread_info *curren
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_32BIT		5	/* 32 bit binary */
-#define TIF_RUNLATCH		6	/* Is the runlatch enabled? */
-#define TIF_ABI_PENDING		7	/* 32/64 bit switch needed */
-#define TIF_SYSCALL_AUDIT	8	/* syscall auditing active */
-#define TIF_SINGLESTEP		9	/* singlestepping active */
-#define TIF_MEMDIE		10
-#define TIF_SECCOMP		11	/* secure computing */
-#define TIF_RESTOREALL		12	/* Restore all regs (implies NOERROR) */
+#define TIF_32BIT		4	/* 32 bit binary */
+#define TIF_RUNLATCH		5	/* Is the runlatch enabled? */
+#define TIF_ABI_PENDING		6	/* 32/64 bit switch needed */
+#define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
+#define TIF_SINGLESTEP		8	/* singlestepping active */
+#define TIF_MEMDIE		9
+#define TIF_SECCOMP		10	/* secure computing */
+#define TIF_RESTOREALL		11	/* Restore all regs (implies NOERROR) */
 #define TIF_NOERROR		14	/* Force successful syscall return */
 #define TIF_RESTORE_SIGMASK	15	/* Restore signal mask in do_signal */
 #define TIF_FREEZE		16	/* Freezing for suspend */
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
@@ -142,7 +140,7 @@ static inline struct thread_info *curren
 #define _TIF_FREEZE		(1<<TIF_FREEZE)
 #define _TIF_SYSCALL_T_OR_A	(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SECCOMP)
 
-#define _TIF_USER_WORK_MASK	(_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | \
+#define _TIF_USER_WORK_MASK	( _TIF_SIGPENDING | \
 				 _TIF_NEED_RESCHED | _TIF_RESTORE_SIGMASK)
 #define _TIF_PERSYSCALL_MASK	(_TIF_RESTOREALL|_TIF_NOERROR)
 
diff -urNp linux-2.6.22.orig/include/asm-sh/thread_info.h linux-2.6.22.base/include/asm-sh/thread_info.h
--- linux-2.6.22.orig/include/asm-sh/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-sh/thread_info.h	2007-05-22 13:36:11.000000000 -0700
@@ -107,18 +107,16 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_RESTORE_SIGMASK	4	/* restore signal mask in do_signal() */
-#define TIF_SINGLESTEP		5	/* singlestepping active */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_RESTORE_SIGMASK	3	/* restore signal mask in do_signal() */
+#define TIF_SINGLESTEP		4	/* singlestepping active */
 #define TIF_USEDFPU		16	/* FPU was used by this task this quantum (SMP) */
 #define TIF_POLLING_NRFLAG	17	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_MEMDIE		18
 #define TIF_FREEZE		19
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_RESTORE_SIGMASK	(1<<TIF_RESTORE_SIGMASK)
diff -urNp linux-2.6.22.orig/include/asm-v850/thread_info.h linux-2.6.22.base/include/asm-v850/thread_info.h
--- linux-2.6.22.orig/include/asm-v850/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-v850/thread_info.h	2007-05-22 13:30:32.000000000 -0700
@@ -77,16 +77,14 @@ struct thread_info {
  * thread information flag bit numbers
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_POLLING_NRFLAG	4	/* true if poll_idle() is polling
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_POLLING_NRFLAG	3	/* true if poll_idle() is polling
 					   TIF_NEED_RESCHED */
-#define TIF_MEMDIE		5
+#define TIF_MEMDIE		4
 
 /* as above, but as bit values */
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
diff -urNp linux-2.6.22.orig/include/asm-x86_64/thread_info.h linux-2.6.22.base/include/asm-x86_64/thread_info.h
--- linux-2.6.22.orig/include/asm-x86_64/thread_info.h	2007-05-21 01:39:10.000000000 -0700
+++ linux-2.6.22.base/include/asm-x86_64/thread_info.h	2007-05-22 13:23:58.000000000 -0700
@@ -107,14 +107,13 @@ static inline struct thread_info *stack_
  * Warning: layout of LSW is hardcoded in entry.S
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* reenable singlestep on user return*/
-#define TIF_IRET		5	/* force IRET */
-#define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
-#define TIF_SECCOMP		8	/* secure computing */
-#define TIF_RESTORE_SIGMASK	9	/* restore signal mask in do_signal */
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* reenable singlestep on user return*/
+#define TIF_IRET		4	/* force IRET */
+#define TIF_SYSCALL_AUDIT	5	/* syscall auditing active */
+#define TIF_SECCOMP		6	/* secure computing */
+#define TIF_RESTORE_SIGMASK	7	/* restore signal mask in do_signal */
 /* 16 free */
 #define TIF_IA32		17	/* 32bit process */ 
 #define TIF_FORK		18	/* ret_from_fork */
@@ -125,7 +124,6 @@ static inline struct thread_info *stack_
 #define TIF_FREEZE		23	/* is freezing for suspend */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
diff -urNp linux-2.6.22.orig/include/asm-xtensa/thread_info.h linux-2.6.22.base/include/asm-xtensa/thread_info.h
--- linux-2.6.22.orig/include/asm-xtensa/thread_info.h	2007-04-25 20:08:32.000000000 -0700
+++ linux-2.6.22.base/include/asm-xtensa/thread_info.h	2007-05-22 13:34:20.000000000 -0700
@@ -110,16 +110,14 @@ static inline struct thread_info *curren
  * - other flags in MSW
  */
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
-#define TIF_NOTIFY_RESUME	1	/* resumption notification requested */
-#define TIF_SIGPENDING		2	/* signal pending */
-#define TIF_NEED_RESCHED	3	/* rescheduling necessary */
-#define TIF_SINGLESTEP		4	/* restore singlestep on return to user mode */
-#define TIF_IRET		5	/* return with iret */
-#define TIF_MEMDIE		6
+#define TIF_SIGPENDING		1	/* signal pending */
+#define TIF_NEED_RESCHED	2	/* rescheduling necessary */
+#define TIF_SINGLESTEP		3	/* restore singlestep on return to user mode */
+#define TIF_IRET		4	/* return with iret */
+#define TIF_MEMDIE		5
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
-#define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
 #define _TIF_SIGPENDING		(1<<TIF_SIGPENDING)
 #define _TIF_NEED_RESCHED	(1<<TIF_NEED_RESCHED)
 #define _TIF_SINGLESTEP		(1<<TIF_SINGLESTEP)

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Andrew Morton]

On Tue, 22 May 2007 11:20:03 -0700 Fenghua Yu <fenghua.yu@intel.com> wrote:

> per cpu data section contains two types of data. One set which is exclusively
> accessed by the local cpu and the other set which is  per cpu, but also shared
> by remote cpus. In the current kernel, these two sets are not clearely
> separated out. This can potentially cause the same data cacheline shared
> between the two sets of data, which will result in unnecessary bouncing of the
> cacheline between cpus.
> 
> One way to fix the problem is to cacheline align the remotely accessed per cpu
> data, both at the beginning and at the end. Because of the padding at both ends,
> this will likely cause some memory wastage and also the interface to achieve
> this is not clean.
> 
> This patch:
> 
> Moves the remotely accessed per cpu data (which is currently marked
> as ____cacheline_aligned_in_smp) into a different section, where all the data
> elements are cacheline aligned. And as such, this differentiates the local
> only data and remotely accessed data cleanly.

OK, but could we please have a concise description of the impact
of these changes on kernel memory footprint?  Increase or decrease?
And by approximately how much?

Thanks.

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Yu, Fenghua]

> elements are cacheline aligned. And as such, this differentiates the
local
> only data and remotely accessed data cleanly.

>OK, but could we please have a concise description of the impact
>of these changes on kernel memory footprint?  Increase or decrease?
>And by approximately how much?

Depending on how linker places percpu data, the patches could
increase or decrease percpu section size. Data from 2.6.21-rc7-mm2:

On x86 SMP, the section size is increased from 0x7768 to 0x790c.
1.3% increase.

On X86-64 SMP, the size is decreased from 0x72d0 to 0x6540.
11.8% decrease.

On X86-64 VSMP, the size is increased from 0x72d0 to 0x8340.
14.3% increase.

On IA64 SMP, the size is decreased from 0x8370 to 0x7fc0.
2.8% decrease.

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Ravikiran G Thirumalai]

On Wed, May 23, 2007 at 11:26:53AM -0700, Yu, Fenghua wrote:
> > elements are cacheline aligned. And as such, this differentiates the
> local
> > only data and remotely accessed data cleanly.
> 
> >OK, but could we please have a concise description of the impact
> >of these changes on kernel memory footprint?  Increase or decrease?
> >And by approximately how much?
> 
> Depending on how linker places percpu data, the patches could
> increase or decrease percpu section size. Data from 2.6.21-rc7-mm2:
> 
> On x86 SMP, the section size is increased from 0x7768 to 0x790c.
> 1.3% increase.
> 
> On X86-64 SMP, the size is decreased from 0x72d0 to 0x6540.
> 11.8% decrease.
> 
> On X86-64 VSMP, the size is increased from 0x72d0 to 0x8340.
> 14.3% increase.
> 
> On IA64 SMP, the size is decreased from 0x8370 to 0x7fc0.
> 2.8% decrease.

Has there been any measurable benefit yet due to tail padding?
It would also be interesting to check the wastage/savings on another large
cache architecture like S390 (which has a 256 byte cache line)

Thanks,
Kiran

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Yu, Fenghua]

>Has there been any measurable benefit yet due to tail padding?

We don't have data that tail padding actually helps. It all
depends on what data the linker lays out in the cachelines.

As of now we just want to create the infrastructure (so that
more and more people who need it, can use it).

>It would also be interesting to check the wastage/savings on another
large
>cache architecture like S390 (which has a 256 byte cache line)

S390 folks or others might help get the data. I don't have other
machines
on hand.

Thanks.

-Fenghua

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Ravikiran G Thirumalai]

On Wed, May 23, 2007 at 12:09:56PM -0700, Yu, Fenghua wrote:
> 
> >Has there been any measurable benefit yet due to tail padding?
> 
> We don't have data that tail padding actually helps. It all
> depends on what data the linker lays out in the cachelines.
> 
> As of now we just want to create the infrastructure (so that
> more and more people who need it, can use it).

So what we have now is space wastage on some architectures, space savings on
some, but with no measurable performance benefit due to the infrastructure
itself.  Why not push the infrastructure when we really need it, as against
pushing it now when we are not sure if it benefits?

Thanks,
Kiran 

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Andrew Morton]

On Wed, 23 May 2007 12:20:05 -0700 Ravikiran G Thirumalai <kiran@scalex86.org> wrote:

> On Wed, May 23, 2007 at 12:09:56PM -0700, Yu, Fenghua wrote:
> > 
> > >Has there been any measurable benefit yet due to tail padding?
> > 
> > We don't have data that tail padding actually helps. It all
> > depends on what data the linker lays out in the cachelines.
> > 
> > As of now we just want to create the infrastructure (so that
> > more and more people who need it, can use it).
> 
> So what we have now is space wastage on some architectures, space savings on
> some, but with no measurable performance benefit due to the infrastructure
> itself.  Why not push the infrastructure when we really need it, as against
> pushing it now when we are not sure if it benefits?
> 

It makes sense from a theoretical POV and is pretty much a no-op in terms
of resource consumption.

The problem with the wait-until-it-hurts approach is that by the time
someone hurts from this and we find out about it, they may well be using
some year-old enterprise kernel and it's too late to fix it for them.

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Yu, Fenghua]

>> So what we have now is space wastage on some architectures, space
savings on
>> some, but with no measurable performance benefit due to the
infrastructure
>> itself.  Why not push the infrastructure when we really need it, as
against
>> pushing it now when we are not sure if it benefits?
> 

>It makes sense from a theoretical POV and is pretty much a no-op in
terms
>of resource consumption.

>The problem with the wait-until-it-hurts approach is that by the time
>someone hurts from this and we find out about it, they may well be
using
>some year-old enterprise kernel and it's too late to fix it for them.

Yes, in theory, sharing shared percpu data with local percpu data in one
cache line can cause cache line contention between remote and local
access.
And this undesirable sharing does happen in current git.

On x86, without the patch, runqueue shares cacheline with cpu_domains.
c0633e80 d per_cpu__runqueues
c06347e0 d per_cpu__cpu_domains

Other architectures also have this undesired sharing. IA64 is an
example.

In future, this undesired sharing could happen with other data if we do 
not have a methodology to prevent it.

I just do not have data to show the performance hit for this sharing
now.

Thanks.

-Fenghua

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Martin Schwidefsky]

On Wed, 2007-05-23 at 11:57 -0700, Ravikiran G Thirumalai wrote:
> > >OK, but could we please have a concise description of the impact
> > >of these changes on kernel memory footprint?  Increase or decrease?
> > >And by approximately how much?
> > 
> > Depending on how linker places percpu data, the patches could
> > increase or decrease percpu section size. Data from 2.6.21-rc7-mm2:
> > 
> > On x86 SMP, the section size is increased from 0x7768 to 0x790c.
> > 1.3% increase.
> > 
> > On X86-64 SMP, the size is decreased from 0x72d0 to 0x6540.
> > 11.8% decrease.
> > 
> > On X86-64 VSMP, the size is increased from 0x72d0 to 0x8340.
> > 14.3% increase.
> > 
> > On IA64 SMP, the size is decreased from 0x8370 to 0x7fc0.
> > 2.8% decrease.
> 
> Has there been any measurable benefit yet due to tail padding?
> It would also be interesting to check the wastage/savings on another
> large
> cache architecture like S390 (which has a 256 byte cache line)

Current git with the patches applied and the default configuration for
s390 decreases the section size fof .data.percpu from 0x3e50 to 0x3e00.
0.5% decrease.

-- 
blue skies,
  Martin.

"Reality continues to ruin my life." - Calvin.

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Ravikiran G Thirumalai]

On Thu, May 24, 2007 at 11:03:56AM +0200, Martin Schwidefsky wrote:
> On Wed, 2007-05-23 at 11:57 -0700, Ravikiran G Thirumalai wrote:
> 
> Current git with the patches applied and the default configuration for
> s390 decreases the section size fof .data.percpu from 0x3e50 to 0x3e00.
> 0.5% decrease.
> 

Thanks!!
Fenghua, you could use my 'Acked by' if needed
Acked-by: Ravikiran Thirumalai <kiran@scalex86.org>

Re: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Russell King]

On Wed, May 23, 2007 at 02:13:24PM -0700, Yu, Fenghua wrote:
> Yes, in theory, sharing shared percpu data with local percpu data in one
> cache line can cause cache line contention between remote and local
> access.

What's "shared percpu data" ?  It sounds to me like a contradiction in
terms.  Isn't percpu data supposed to only be accessed by the CPU which
owns it to prevent cache line bouncing?  In which case, what's the point
of sharing that data with other CPUs?  Surely "shared percpu data" is
just the same as normal data?

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

RE: [PATCH 1/2] Define new percpu interface for shared data -- version 3

[Yu, Fenghua]

>What's "shared percpu data" ?  It sounds to me like a contradiction in
>terms.  Isn't percpu data supposed to only be accessed by the CPU which
>owns it to prevent cache line bouncing?  In which case, what's the
point
>of sharing that data with other CPUs?  Surely "shared percpu data" is
>just the same as normal data?

Like said in the patch description:
"per cpu data section contains two types of data. One set which is
exclusively accessed by the local cpu and the other set which is  per
cpu,
but also shared by remote cpus."

The patches separate shared and aligned per cpu data from others to
eliminate cache pouncing between remote and local access.

The patches do not separate shared but non-aligned percpu data, e.g.
cpu_idle_state, flush_state in x86-64. We can do that later if needed.

I think the shared percpu data are "mainly" accessed by local cpu. We
need
to concentrate on their local cpu access i.e. define in percpu. If we
treat them as "normal" data, we may lose percpu benefits e.g. numa
allocation.

Thanks.

-Fenghua