* Re: [PATCH] New per-cpu patch v2.5.1
@ 2001-12-20 12:13 Dipankar Sarma
2001-12-27 6:50 ` Rusty Russell
2002-01-13 5:52 ` Rusty Russell
0 siblings, 2 replies; 6+ messages in thread
From: Dipankar Sarma @ 2001-12-20 12:13 UTC (permalink / raw)
To: rusty; +Cc: linux-kernel
Hi Rusty,
I appreciate the noble gesture of allowing NUMA people to
locate the per-cpu areas in different places in memory ;-)
That said, memcpy of static per-cpu areas doesn't seem right.
Why copy the same area to all the dynamically allocated per-cpu
areas ? Also, shouldn't the size be &__per_cpu_end - &__per_cpu_start ?
And how do we use this with per-cpu data used right
from smp boot, say apic_timer_irqs[] ?
Thanks
Dipankar
--
Dipankar Sarma <dipankar@in.ibm.com> http://lse.sourceforge.net
Linux Technology Center, IBM Software Lab, Bangalore, India.
In article <E16GwUZ-0004xr-00@wagner.rustcorp.com.au> Rusty Russell wrote:
> After some discussion, this may be a more sane (untested) per-cpu area
> patch. It dynamically allocated the sections (and discards the
> original), which would allow (future) NUMA people to make sure their
> CPU area is allocated near them.
> Comments welcome,
> Rusty.
> --
> Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
> /* Called by boot processor to activate the rest. */
> static void __init smp_init(void)
> {
> + unsigned int i;
> + size_t per_cpu_size;
> +
> /* Get other processors into their bootup holding patterns. */
> smp_boot_cpus();
> wait_init_idle = cpu_online_map;
> @@ -324,6 +328,16 @@
> barrier();
> }
> printk("All processors have done init_idle\n");
> +
> + /* Set up per-CPU section pointers. Page align to be safe. */
> + per_cpu_size = ((&__per_cpu_end - &__per_cpu_start) + PAGE_SIZE-1)
> + & ~(PAGE_SIZE-1);
> + per_cpu_off[0] = kmalloc(per_cpu_size * smp_num_cpus, GFP_KERNEL);
> + for (i = 0; i < smp_num_cpus; i++) {
> + per_cpu_off[i] = per_cpu_off[0] + per_cpu_size;
> + memcpy(per_cpu_off[i], &__per_cpu_start,
> + __per_cpu_end - &__per_cpu_start);
> + }
> }
>
> #endif
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] New per-cpu patch v2.5.1
2001-12-20 12:13 [PATCH] New per-cpu patch v2.5.1 Dipankar Sarma
@ 2001-12-27 6:50 ` Rusty Russell
2002-01-13 5:52 ` Rusty Russell
1 sibling, 0 replies; 6+ messages in thread
From: Rusty Russell @ 2001-12-27 6:50 UTC (permalink / raw)
To: dipankar; +Cc: linux-kernel
On Thu, 20 Dec 2001 17:43:35 +0530
Dipankar Sarma <dipankar@in.ibm.com> wrote:
> Hi Rusty,
>
> I appreciate the noble gesture of allowing NUMA people to
> locate the per-cpu areas in different places in memory ;-)
>
> That said, memcpy of static per-cpu areas doesn't seem right.
> Why copy the same area to all the dynamically allocated per-cpu
> areas ?
Initialization. Sure, we can't do it with the current macros...
> Also, shouldn't the size be &__per_cpu_end - &__per_cpu_start ?
Oops... Yes. (I wrote this just before my Xmas break, and didn't have time
to test it).
> And how do we use this with per-cpu data used right
> from smp boot, say apic_timer_irqs[] ?
No. You can't have everthing (without going into the arch-specific code).
Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] New per-cpu patch v2.5.1
2001-12-20 12:13 [PATCH] New per-cpu patch v2.5.1 Dipankar Sarma
2001-12-27 6:50 ` Rusty Russell
@ 2002-01-13 5:52 ` Rusty Russell
1 sibling, 0 replies; 6+ messages in thread
From: Rusty Russell @ 2002-01-13 5:52 UTC (permalink / raw)
To: dipankar; +Cc: linux-kernel, torvalds, paulus, rth
In message <20011220174335.A10791@in.ibm.com> you write:
> Hi Rusty,
>
> I appreciate the noble gesture of allowing NUMA people to
> locate the per-cpu areas in different places in memory ;-)
>
> That said, memcpy of static per-cpu areas doesn't seem right.
> Why copy the same area to all the dynamically allocated per-cpu
> areas ? Also, shouldn't the size be &__per_cpu_end - &__per_cpu_start ?
> And how do we use this with per-cpu data used right
> from smp boot, say apic_timer_irqs[] ?
New patch... this one is better, but runs into the same gcc
assumptions about doing pointer arith outside the object. Now, noone
is putting string literals in the per-cpu areas, but medium-term we
probably need the arch-specific asm hack to stop bogosity.
By Anton's request, an arch can define __HAVE_ARCH_CPU_OFFSET if it
wants to store each CPU's offset in a spare register to save a mem
access.
If no complaints, I'd like this in 2.5 soon, so we can start using it.
Thanks!
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.2-pre10/arch/i386/vmlinux.lds working-2.5.2-pre10-percpu/arch/i386/vmlinux.lds
--- linux-2.5.2-pre10/arch/i386/vmlinux.lds Tue Jul 3 07:40:14 2001
+++ working-2.5.2-pre10-percpu/arch/i386/vmlinux.lds Tue Jan 8 14:23:46 2002
@@ -50,6 +50,10 @@
__initcall_start = .;
.initcall.init : { *(.initcall.init) }
__initcall_end = .;
+ . = ALIGN(32);
+ __per_cpu_start = .;
+ .data.percpu : { *(.data.percpu) }
+ __per_cpu_end = .;
. = ALIGN(4096);
__init_end = .;
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.2-pre10/arch/ppc/vmlinux.lds working-2.5.2-pre10-percpu/arch/ppc/vmlinux.lds
--- linux-2.5.2-pre10/arch/ppc/vmlinux.lds Tue Aug 28 23:58:33 2001
+++ working-2.5.2-pre10-percpu/arch/ppc/vmlinux.lds Tue Jan 8 14:23:46 2002
@@ -96,6 +96,10 @@
__initcall_start = .;
.initcall.init : { *(.initcall.init) }
__initcall_end = .;
+ . = ALIGN(32);
+ __per_cpu_start = .;
+ .data.percpu : { *(.data.percpu) }
+ __per_cpu_end = .;
. = ALIGN(4096);
__init_end = .;
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.2-pre10/include/linux/smp.h working-2.5.2-pre10-percpu/include/linux/smp.h
--- linux-2.5.2-pre10/include/linux/smp.h Tue Jan 8 11:49:34 2002
+++ working-2.5.2-pre10-percpu/include/linux/smp.h Tue Jan 8 14:38:09 2002
@@ -71,7 +71,17 @@
#define MSG_RESCHEDULE 0x0003 /* Reschedule request from master CPU*/
#define MSG_CALL_FUNCTION 0x0004 /* Call function on all other CPUs */
-#else
+#define __per_cpu_data __attribute__((section(".data.percpu")))
+
+#ifndef __HAVE_ARCH_CPU_OFFSET
+#define per_cpu_offset(cpu) (__per_cpu_offset[(cpu)])
+#endif
+
+#define per_cpu(var, cpu) \
+(*(__typeof__(&var)((void *)&var + per_cpu_offset(cpu))))
+
+extern unsigned long __per_cpu_offset[NR_CPUS];
+#else /* !SMP */
/*
* These macros fold the SMP functionality into a single CPU system
@@ -88,6 +98,10 @@
#define cpu_online_map 1
static inline void smp_send_reschedule(int cpu) { }
static inline void smp_send_reschedule_all(void) { }
+#define __per_cpu_data
+#define per_cpu(var, cpu) var
#endif
+
+#define this_cpu(var) per_cpu(var,smp_processor_id())
#endif
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.2-pre10/init/main.c working-2.5.2-pre10-percpu/init/main.c
--- linux-2.5.2-pre10/init/main.c Tue Jan 8 11:49:34 2002
+++ working-2.5.2-pre10-percpu/init/main.c Tue Jan 8 14:38:19 2002
@@ -305,10 +305,36 @@
#else
+unsigned long __per_cpu_offset[NR_CPUS];
+/* Created by linker magic */
+extern char __per_cpu_start, __per_cpu_end;
+
+static void setup_per_cpu_areas(void)
+{
+ unsigned int i;
+ size_t per_cpu_size;
+ char *region;
+
+ /* Set up per-CPU offset pointers. Page align to be safe. */
+ per_cpu_size = ((&__per_cpu_end - &__per_cpu_start) + PAGE_SIZE-1)
+ & ~(PAGE_SIZE-1);
+ region = kmalloc(per_cpu_size * smp_num_cpus, GFP_KERNEL);
+ if (!region)
+ panic("Could not allocate per-cpu regions: %u bytes\n",
+ per_cpu_size * smp_num_cpus);
+ for (i = 0; i < smp_num_cpus; i++) {
+ memcpy(region + per_cpu_size*i, &__per_cpu_start,
+ &__per_cpu_end - &__per_cpu_start);
+ __per_cpu_offset[i]
+ = (region + per_cpu_size*i) - &__per_cpu_start;
+ }
+}
/* Called by boot processor to activate the rest. */
static void __init smp_init(void)
{
+ setup_per_cpu_areas();
+
/* Get other processors into their bootup holding patterns. */
smp_boot_cpus();
wait_init_idle = cpu_online_map;
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH] New per-cpu patch v2.5.1
@ 2001-12-20 6:15 Rusty Russell
2001-12-20 6:24 ` Robert Love
0 siblings, 1 reply; 6+ messages in thread
From: Rusty Russell @ 2001-12-20 6:15 UTC (permalink / raw)
To: linux-kernel
After some discussion, this may be a more sane (untested) per-cpu area
patch. It dynamically allocated the sections (and discards the
original), which would allow (future) NUMA people to make sure their
CPU area is allocated near them.
Comments welcome,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.1/arch/i386/vmlinux.lds working-2.5.1-percpu/arch/i386/vmlinux.lds
--- linux-2.5.1/arch/i386/vmlinux.lds Tue Jul 3 07:40:14 2001
+++ working-2.5.1-percpu/arch/i386/vmlinux.lds Thu Dec 20 14:13:03 2001
@@ -50,6 +50,10 @@
__initcall_start = .;
.initcall.init : { *(.initcall.init) }
__initcall_end = .;
+ . = ALIGN(32);
+ __per_cpu_start = .;
+ .data.percpu : { *(.data.percpu) }
+ __per_cpu_end = .;
. = ALIGN(4096);
__init_end = .;
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.1/arch/ppc/vmlinux.lds working-2.5.1-percpu/arch/ppc/vmlinux.lds
--- linux-2.5.1/arch/ppc/vmlinux.lds Tue Aug 28 23:58:33 2001
+++ working-2.5.1-percpu/arch/ppc/vmlinux.lds Thu Dec 20 14:34:13 2001
@@ -96,6 +96,10 @@
__initcall_start = .;
.initcall.init : { *(.initcall.init) }
__initcall_end = .;
+ . = ALIGN(32);
+ __per_cpu_start = .;
+ .data.percpu : { *(.data.percpu) }
+ __per_cpu_end = .;
. = ALIGN(4096);
__init_end = .;
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.1/include/linux/smp.h working-2.5.1-percpu/include/linux/smp.h
--- linux-2.5.1/include/linux/smp.h Tue Dec 18 19:08:06 2001
+++ working-2.5.1-percpu/include/linux/smp.h Thu Dec 20 14:30:28 2001
@@ -71,7 +71,18 @@
#define MSG_RESCHEDULE 0x0003 /* Reschedule request from master CPU*/
#define MSG_CALL_FUNCTION 0x0004 /* Call function on all other CPUs */
-#else
+#define PER_CPU(decl) decl##__percpu __attribute__((section(".data.percpu")))
+
+/* Created by linker magic */
+extern char __per_cpu_start, __per_cpu_end;
+
+extern void *per_cpu_off[NR_CPUS];
+
+#define per_cpu(var, cpu) \
+(*(__typeof__(&var)((void *)&var##__percpu - &__per_cpu_start) \
+ + per_cpu_off[cpu]))
+
+#else /* !SMP */
/*
* These macros fold the SMP functionality into a single CPU system
@@ -86,6 +97,10 @@
#define cpu_number_map(cpu) 0
#define smp_call_function(func,info,retry,wait) ({ 0; })
#define cpu_online_map 1
-
+#define PER_CPU(decl) decl
+#define per_cpu(var, cpu) var
#endif
+
+#define this_cpu(var) per_cpu(var,smp_processor_id())
+
#endif
diff -urN -I \$.*\$ --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.1/init/main.c working-2.5.1-percpu/init/main.c
--- linux-2.5.1/init/main.c Mon Dec 17 16:09:13 2001
+++ working-2.5.1-percpu/init/main.c Thu Dec 20 17:08:42 2001
@@ -305,10 +305,14 @@
#else
+void *per_cpu_off[NR_CPUS];
/* Called by boot processor to activate the rest. */
static void __init smp_init(void)
{
+ unsigned int i;
+ size_t per_cpu_size;
+
/* Get other processors into their bootup holding patterns. */
smp_boot_cpus();
wait_init_idle = cpu_online_map;
@@ -324,6 +328,16 @@
barrier();
}
printk("All processors have done init_idle\n");
+
+ /* Set up per-CPU section pointers. Page align to be safe. */
+ per_cpu_size = ((&__per_cpu_end - &__per_cpu_start) + PAGE_SIZE-1)
+ & ~(PAGE_SIZE-1);
+ per_cpu_off[0] = kmalloc(per_cpu_size * smp_num_cpus, GFP_KERNEL);
+ for (i = 0; i < smp_num_cpus; i++) {
+ per_cpu_off[i] = per_cpu_off[0] + per_cpu_size;
+ memcpy(per_cpu_off[i], &__per_cpu_start,
+ __per_cpu_end - &__per_cpu_start);
+ }
}
#endif
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] New per-cpu patch v2.5.1
2001-12-20 6:15 Rusty Russell
@ 2001-12-20 6:24 ` Robert Love
2001-12-26 0:39 ` Rusty Russell
0 siblings, 1 reply; 6+ messages in thread
From: Robert Love @ 2001-12-20 6:24 UTC (permalink / raw)
To: Rusty Russell; +Cc: linux-kernel
On Thu, 2001-12-20 at 01:15, Rusty Russell wrote:
> After some discussion, this may be a more sane (untested) per-cpu area
> patch. It dynamically allocated the sections (and discards the
> original), which would allow (future) NUMA people to make sure their
> CPU area is allocated near them.
>
> Comments welcome,
Would the next step be to find the various per-CPU data in the kernel
and convert it to your new form? I.e., is this a general purpose
interface for per-CPU data structures, or am I missing something?
If it is, this is a good idea. Other Unices have this (IRIX comes to
mind). One of the biggest advantages, IMO, is simply the readability --
data structures that are per-CPU have varying methods of creation and
referencing. The implicit locking (i.e., none) can be unclear.
Bring everything together can be a good thing.
Robert Love
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] New per-cpu patch v2.5.1
2001-12-20 6:24 ` Robert Love
@ 2001-12-26 0:39 ` Rusty Russell
0 siblings, 0 replies; 6+ messages in thread
From: Rusty Russell @ 2001-12-26 0:39 UTC (permalink / raw)
To: Robert Love; +Cc: linux-kernel
On 20 Dec 2001 01:24:45 -0500
Robert Love <rml@tech9.net> wrote:
> On Thu, 2001-12-20 at 01:15, Rusty Russell wrote:
> > After some discussion, this may be a more sane (untested) per-cpu area
> > patch. It dynamically allocated the sections (and discards the
> > original), which would allow (future) NUMA people to make sure their
> > CPU area is allocated near them.
> >
> > Comments welcome,
>
> Would the next step be to find the various per-CPU data in the kernel
> and convert it to your new form? I.e., is this a general purpose
> interface for per-CPU data structures, or am I missing something
Yep. One step at a time...
Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2002-01-13 5:52 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-12-20 12:13 [PATCH] New per-cpu patch v2.5.1 Dipankar Sarma
2001-12-27 6:50 ` Rusty Russell
2002-01-13 5:52 ` Rusty Russell
-- strict thread matches above, loose matches on Subject: below --
2001-12-20 6:15 Rusty Russell
2001-12-20 6:24 ` Robert Love
2001-12-26 0:39 ` Rusty Russell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox