* [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines
@ 2008-12-19 16:01 Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
` (7 more replies)
0 siblings, 8 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel
Patches for review only. Will be pushed via one git conduit or another.
Remove 20 more cpumask_t references (down to *only* 518... ;-)
Pulled the following patches from:
http://ozlabs.org/~rusty/kernel/rr-2008-12-18-1.tar.bz2
cpumask:convert-kernel_compat.c.patch
cpumask:convert-kernel_workqueue.c.patch
cpumask:convert-kernel_time.patch
cpumask:convert-kernel_trace.patch
cpumask:convert-kernel.patch
cpumask:convert-mm.patch
cpumask:convert-drivers.patch
cpumask:convert-misc.patch
... and edited slightly/built/tested on x86_64
In addition, the following are OBSOLETE:
cpumask:get-rid-of-_nr-core.patch
(folded into the above patches to avoid inter-patch dependencies)
The affected files are:
(* - interdepency on 'irq_default_affinity')
(*) arch/alpha/kernel/irq.c
drivers/base/cpu.c
drivers/misc/sgi-xp/xpc_main.c
drivers/net/sfc/efx.c
drivers/oprofile/buffer_sync.c
drivers/oprofile/buffer_sync.h
drivers/oprofile/oprof.c
drivers/xen/manage.c
fs/seq_file.c
include/linux/interrupt.h
include/linux/rcuclassic.h
include/linux/seq_file.h
include/linux/stop_machine.h
include/linux/tick.h
kernel/compat.c
kernel/cpu.c
kernel/irq/manage.c
kernel/irq/proc.c
kernel/power/poweroff.c
kernel/profile.c
kernel/rcuclassic.c
kernel/rcupreempt.c
kernel/rcutorture.c
kernel/softlockup.c
kernel/stop_machine.c
kernel/taskstats.c
kernel/time/clocksource.c
kernel/time/tick-broadcast.c
kernel/time/tick-common.c
kernel/trace/ring_buffer.c
kernel/trace/trace.c
kernel/trace/trace_sysprof.c
kernel/workqueue.c
lib/smp_processor_id.c
mm/pdflush.c
mm/slab.c
mm/slub.c
mm/vmscan.c
mm/vmstat.c
net/iucv/iucv.c
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 1/8] cpumask: convert kernel/compat.c
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 16:01 ` [PATCH 2/8] cpumask: convert kernel/workqueue.c Mike Travis
` (6 subsequent siblings)
7 siblings, 0 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel
[-- Attachment #1: cpumask:convert-kernel_compat.c.patch --]
[-- Type: text/plain, Size: 2705 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Straightforward conversion; cpumasks' size is given by cpumask_size() (now
a variable rather than fixed) and on-stack cpu masks use cpumask_var_t.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
---
kernel/compat.c | 49 ++++++++++++++++++++++++++++++-------------------
1 file changed, 30 insertions(+), 19 deletions(-)
--- linux-2.6.28.orig/kernel/compat.c
+++ linux-2.6.28/kernel/compat.c
@@ -454,16 +454,16 @@ asmlinkage long compat_sys_waitid(int wh
}
static int compat_get_user_cpu_mask(compat_ulong_t __user *user_mask_ptr,
- unsigned len, cpumask_t *new_mask)
+ unsigned len, struct cpumask *new_mask)
{
unsigned long *k;
- if (len < sizeof(cpumask_t))
- memset(new_mask, 0, sizeof(cpumask_t));
- else if (len > sizeof(cpumask_t))
- len = sizeof(cpumask_t);
+ if (len < cpumask_size())
+ memset(new_mask, 0, cpumask_size());
+ else if (len > cpumask_size())
+ len = cpumask_size();
- k = cpus_addr(*new_mask);
+ k = cpumask_bits(new_mask);
return compat_get_bitmap(k, user_mask_ptr, len * 8);
}
@@ -471,40 +471,51 @@ asmlinkage long compat_sys_sched_setaffi
unsigned int len,
compat_ulong_t __user *user_mask_ptr)
{
- cpumask_t new_mask;
+ cpumask_var_t new_mask;
int retval;
- retval = compat_get_user_cpu_mask(user_mask_ptr, len, &new_mask);
+ if (!alloc_cpumask_var(&new_mask, GFP_KERNEL))
+ return -ENOMEM;
+
+ retval = compat_get_user_cpu_mask(user_mask_ptr, len, new_mask);
if (retval)
- return retval;
+ goto out;
- return sched_setaffinity(pid, &new_mask);
+ retval = sched_setaffinity(pid, new_mask);
+out:
+ free_cpumask_var(new_mask);
+ return retval;
}
asmlinkage long compat_sys_sched_getaffinity(compat_pid_t pid, unsigned int len,
compat_ulong_t __user *user_mask_ptr)
{
int ret;
- cpumask_t mask;
+ cpumask_var_t mask;
unsigned long *k;
- unsigned int min_length = sizeof(cpumask_t);
+ unsigned int min_length = cpumask_size();
- if (NR_CPUS <= BITS_PER_COMPAT_LONG)
+ if (nr_cpu_ids <= BITS_PER_COMPAT_LONG)
min_length = sizeof(compat_ulong_t);
if (len < min_length)
return -EINVAL;
- ret = sched_getaffinity(pid, &mask);
+ if (!alloc_cpumask_var(&mask, GFP_KERNEL))
+ return -ENOMEM;
+
+ ret = sched_getaffinity(pid, mask);
if (ret < 0)
- return ret;
+ goto out;
- k = cpus_addr(mask);
+ k = cpumask_bits(mask);
ret = compat_put_bitmap(user_mask_ptr, k, min_length * 8);
- if (ret)
- return ret;
+ if (ret == 0)
+ ret = min_length;
- return min_length;
+out:
+ free_cpumask_var(mask);
+ return ret;
}
int get_compat_itimerspec(struct itimerspec *dst,
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 2/8] cpumask: convert kernel/workqueue.c
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
` (5 subsequent siblings)
7 siblings, 0 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Heiko Carstens
[-- Attachment #1: cpumask:convert-kernel_workqueue.c.patch --]
[-- Type: text/plain, Size: 4695 bytes --]
Impact: Reduce memory usage, use new cpumask API.
In this case, we replace cpu_populated_map with a cpumask_var_t rather
than a bitmap because there's an obvious init function for workqueues
which happens after kmalloc is available.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
---
kernel/workqueue.c | 39 +++++++++++++++++++++------------------
1 file changed, 21 insertions(+), 18 deletions(-)
--- linux-2.6-for-ingo.orig/kernel/workqueue.c
+++ linux-2.6-for-ingo/kernel/workqueue.c
@@ -73,15 +73,15 @@ static DEFINE_SPINLOCK(workqueue_lock);
static LIST_HEAD(workqueues);
static int singlethread_cpu __read_mostly;
-static cpumask_t cpu_singlethread_map __read_mostly;
+static const struct cpumask *cpu_singlethread_map __read_mostly;
/*
- * _cpu_down() first removes CPU from cpu_online_map, then CPU_DEAD
+ * _cpu_down() first removes CPU from cpu_online_mask, then CPU_DEAD
* flushes cwq->worklist. This means that flush_workqueue/wait_on_work
* which comes in between can't use for_each_online_cpu(). We could
- * use cpu_possible_map, the cpumask below is more a documentation
+ * use cpu_possible_mask, the cpumask below is more a documentation
* than optimization.
*/
-static cpumask_t cpu_populated_map __read_mostly;
+static cpumask_var_t cpu_populated_map __read_mostly;
/* If it's single threaded, it isn't in the list of workqueues. */
static inline int is_single_threaded(struct workqueue_struct *wq)
@@ -89,10 +89,10 @@ static inline int is_single_threaded(str
return wq->singlethread;
}
-static const cpumask_t *wq_cpu_map(struct workqueue_struct *wq)
+static const struct cpumask *wq_cpu_map(struct workqueue_struct *wq)
{
return is_single_threaded(wq)
- ? &cpu_singlethread_map : &cpu_populated_map;
+ ? cpu_singlethread_map : cpu_populated_map;
}
static
@@ -410,13 +410,13 @@ static int flush_cpu_workqueue(struct cp
*/
void flush_workqueue(struct workqueue_struct *wq)
{
- const cpumask_t *cpu_map = wq_cpu_map(wq);
+ const struct cpumask *cpu_map = wq_cpu_map(wq);
int cpu;
might_sleep();
lock_map_acquire(&wq->lockdep_map);
lock_map_release(&wq->lockdep_map);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
flush_cpu_workqueue(per_cpu_ptr(wq->cpu_wq, cpu));
}
EXPORT_SYMBOL_GPL(flush_workqueue);
@@ -532,7 +532,7 @@ static void wait_on_work(struct work_str
{
struct cpu_workqueue_struct *cwq;
struct workqueue_struct *wq;
- const cpumask_t *cpu_map;
+ const struct cpumask *cpu_map;
int cpu;
might_sleep();
@@ -547,7 +547,7 @@ static void wait_on_work(struct work_str
wq = cwq->wq;
cpu_map = wq_cpu_map(wq);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
wait_on_cpu_work(per_cpu_ptr(wq->cpu_wq, cpu), work);
}
@@ -778,7 +778,7 @@ static int create_workqueue_thread(struc
* if (caller is __create_workqueue)
* nobody should see this wq
* else // caller is CPU_UP_PREPARE
- * cpu is not on cpu_online_map
+ * cpu is not on cpu_online_mask
* so we can abort safely.
*/
if (IS_ERR(p))
@@ -903,7 +903,7 @@ static void cleanup_workqueue_thread(str
*/
void destroy_workqueue(struct workqueue_struct *wq)
{
- const cpumask_t *cpu_map = wq_cpu_map(wq);
+ const struct cpumask *cpu_map = wq_cpu_map(wq);
int cpu;
cpu_maps_update_begin();
@@ -911,7 +911,7 @@ void destroy_workqueue(struct workqueue_
list_del(&wq->list);
spin_unlock(&workqueue_lock);
- for_each_cpu_mask_nr(cpu, *cpu_map)
+ for_each_cpu(cpu, cpu_map)
cleanup_workqueue_thread(per_cpu_ptr(wq->cpu_wq, cpu));
cpu_maps_update_done();
@@ -933,7 +933,7 @@ static int __devinit workqueue_cpu_callb
switch (action) {
case CPU_UP_PREPARE:
- cpu_set(cpu, cpu_populated_map);
+ cpumask_set_cpu(cpu, cpu_populated_map);
}
undo:
list_for_each_entry(wq, &workqueues, list) {
@@ -964,7 +964,7 @@ undo:
switch (action) {
case CPU_UP_CANCELED:
case CPU_POST_DEAD:
- cpu_clear(cpu, cpu_populated_map);
+ cpumask_clear_cpu(cpu, cpu_populated_map);
}
return ret;
@@ -1017,9 +1017,12 @@ EXPORT_SYMBOL_GPL(work_on_cpu);
void __init init_workqueues(void)
{
- cpu_populated_map = cpu_online_map;
- singlethread_cpu = first_cpu(cpu_possible_map);
- cpu_singlethread_map = cpumask_of_cpu(singlethread_cpu);
+ alloc_cpumask_var(&cpu_populated_map, GFP_KERNEL);
+ cpumask_copy(cpu_populated_map, cpu_online_mask);
+
+ singlethread_cpu = cpumask_first(cpu_possible_mask);
+ cpu_singlethread_map = cpumask_of(singlethread_cpu);
+
hotcpu_notifier(workqueue_cpu_callback, 0);
keventd_wq = create_workqueue("events");
BUG_ON(!keventd_wq);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 3/8] cpumask: convert kernel time functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
2008-12-19 16:01 ` [PATCH 2/8] cpumask: convert kernel/workqueue.c Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-20 11:29 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
` (4 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Thomas Gleixner
[-- Attachment #1: cpumask:convert-kernel_time.patch --]
[-- Type: text/plain, Size: 14212 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Convert kernel/time functions to use struct cpumask *.
Note the ugly bitmap declarations in tick-broadcast.c. These should
be cpumask_var_t, but there was no obvious initialization function to
put the alloc_cpumask_var() calls in. This was safe.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/tick.h | 4 -
kernel/time/clocksource.c | 9 +--
kernel/time/tick-broadcast.c | 115 ++++++++++++++++++++++---------------------
kernel/time/tick-common.c | 6 +-
4 files changed, 69 insertions(+), 65 deletions(-)
--- linux-2.6-for-ingo.orig/include/linux/tick.h
+++ linux-2.6-for-ingo/include/linux/tick.h
@@ -84,10 +84,10 @@ static inline void tick_cancel_sched_tim
# ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
extern struct tick_device *tick_get_broadcast_device(void);
-extern cpumask_t *tick_get_broadcast_mask(void);
+extern struct cpumask *tick_get_broadcast_mask(void);
# ifdef CONFIG_TICK_ONESHOT
-extern cpumask_t *tick_get_broadcast_oneshot_mask(void);
+extern struct cpumask *tick_get_broadcast_oneshot_mask(void);
# endif
# endif /* BROADCAST */
--- linux-2.6-for-ingo.orig/kernel/time/clocksource.c
+++ linux-2.6-for-ingo/kernel/time/clocksource.c
@@ -145,10 +145,11 @@ static void clocksource_watchdog(unsigne
* Cycle through CPUs to check if the CPUs stay
* synchronized to each other.
*/
- int next_cpu = next_cpu_nr(raw_smp_processor_id(), cpu_online_map);
+ int next_cpu = cpumask_next(raw_smp_processor_id(),
+ cpu_online_mask);
if (next_cpu >= nr_cpu_ids)
- next_cpu = first_cpu(cpu_online_map);
+ next_cpu = cpumask_first(cpu_online_mask);
watchdog_timer.expires += WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer, next_cpu);
}
@@ -173,7 +174,7 @@ static void clocksource_check_watchdog(s
watchdog_last = watchdog->read();
watchdog_timer.expires = jiffies + WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer,
- first_cpu(cpu_online_map));
+ cpumask_first(cpu_online_mask));
}
} else {
if (cs->flags & CLOCK_SOURCE_IS_CONTINUOUS)
@@ -195,7 +196,7 @@ static void clocksource_check_watchdog(s
watchdog_timer.expires =
jiffies + WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer,
- first_cpu(cpu_online_map));
+ cpumask_first(cpu_online_mask));
}
}
}
--- linux-2.6-for-ingo.orig/kernel/time/tick-broadcast.c
+++ linux-2.6-for-ingo/kernel/time/tick-broadcast.c
@@ -28,7 +28,9 @@
*/
struct tick_device tick_broadcast_device;
-static cpumask_t tick_broadcast_mask;
+/* FIXME: Use cpumask_var_t. */
+static DECLARE_BITMAP(tick_broadcast_mask, CONFIG_NR_CPUS);
+static DECLARE_BITMAP(tmpmask, CONFIG_NR_CPUS);
static DEFINE_SPINLOCK(tick_broadcast_lock);
static int tick_broadcast_force;
@@ -46,9 +48,9 @@ struct tick_device *tick_get_broadcast_d
return &tick_broadcast_device;
}
-cpumask_t *tick_get_broadcast_mask(void)
+struct cpumask *tick_get_broadcast_mask(void)
{
- return &tick_broadcast_mask;
+ return to_cpumask(tick_broadcast_mask);
}
/*
@@ -72,7 +74,7 @@ int tick_check_broadcast_device(struct c
clockevents_exchange_device(NULL, dev);
tick_broadcast_device.evtdev = dev;
- if (!cpus_empty(tick_broadcast_mask))
+ if (!cpumask_empty(tick_get_broadcast_mask()))
tick_broadcast_start_periodic(dev);
return 1;
}
@@ -104,7 +106,7 @@ int tick_device_uses_broadcast(struct cl
*/
if (!tick_device_is_functional(dev)) {
dev->event_handler = tick_handle_periodic;
- cpu_set(cpu, tick_broadcast_mask);
+ cpumask_set_cpu(cpu, tick_get_broadcast_mask());
tick_broadcast_start_periodic(tick_broadcast_device.evtdev);
ret = 1;
} else {
@@ -116,7 +118,7 @@ int tick_device_uses_broadcast(struct cl
if (!(dev->features & CLOCK_EVT_FEAT_C3STOP)) {
int cpu = smp_processor_id();
- cpu_clear(cpu, tick_broadcast_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_mask());
tick_broadcast_clear_oneshot(cpu);
}
}
@@ -125,9 +127,9 @@ int tick_device_uses_broadcast(struct cl
}
/*
- * Broadcast the event to the cpus, which are set in the mask
+ * Broadcast the event to the cpus, which are set in the mask (mangled).
*/
-static void tick_do_broadcast(cpumask_t mask)
+static void tick_do_broadcast(struct cpumask *mask)
{
int cpu = smp_processor_id();
struct tick_device *td;
@@ -135,22 +137,21 @@ static void tick_do_broadcast(cpumask_t
/*
* Check, if the current cpu is in the mask
*/
- if (cpu_isset(cpu, mask)) {
- cpu_clear(cpu, mask);
+ if (cpumask_test_cpu(cpu, mask)) {
+ cpumask_clear_cpu(cpu, mask);
td = &per_cpu(tick_cpu_device, cpu);
td->evtdev->event_handler(td->evtdev);
}
- if (!cpus_empty(mask)) {
+ if (!cpumask_empty(mask)) {
/*
* It might be necessary to actually check whether the devices
* have different broadcast functions. For now, just use the
* one of the first device. This works as long as we have this
* misfeature only on x86 (lapic)
*/
- cpu = first_cpu(mask);
- td = &per_cpu(tick_cpu_device, cpu);
- td->evtdev->broadcast(&mask);
+ td = &per_cpu(tick_cpu_device, cpumask_first(mask));
+ td->evtdev->broadcast(mask);
}
}
@@ -160,12 +161,11 @@ static void tick_do_broadcast(cpumask_t
*/
static void tick_do_periodic_broadcast(void)
{
- cpumask_t mask;
-
spin_lock(&tick_broadcast_lock);
- cpus_and(mask, cpu_online_map, tick_broadcast_mask);
- tick_do_broadcast(mask);
+ cpumask_and(to_cpumask(tmpmask),
+ cpu_online_mask, tick_get_broadcast_mask());
+ tick_do_broadcast(to_cpumask(tmpmask));
spin_unlock(&tick_broadcast_lock);
}
@@ -228,13 +228,13 @@ static void tick_do_broadcast_on_off(voi
if (!tick_device_is_functional(dev))
goto out;
- bc_stopped = cpus_empty(tick_broadcast_mask);
+ bc_stopped = cpumask_empty(tick_get_broadcast_mask());
switch (*reason) {
case CLOCK_EVT_NOTIFY_BROADCAST_ON:
case CLOCK_EVT_NOTIFY_BROADCAST_FORCE:
- if (!cpu_isset(cpu, tick_broadcast_mask)) {
- cpu_set(cpu, tick_broadcast_mask);
+ if (!cpumask_test_cpu(cpu, tick_get_broadcast_mask())) {
+ cpumask_set_cpu(cpu, tick_get_broadcast_mask());
if (tick_broadcast_device.mode ==
TICKDEV_MODE_PERIODIC)
clockevents_shutdown(dev);
@@ -244,8 +244,8 @@ static void tick_do_broadcast_on_off(voi
break;
case CLOCK_EVT_NOTIFY_BROADCAST_OFF:
if (!tick_broadcast_force &&
- cpu_isset(cpu, tick_broadcast_mask)) {
- cpu_clear(cpu, tick_broadcast_mask);
+ cpumask_test_cpu(cpu, tick_get_broadcast_mask())) {
+ cpumask_clear_cpu(cpu, tick_get_broadcast_mask());
if (tick_broadcast_device.mode ==
TICKDEV_MODE_PERIODIC)
tick_setup_periodic(dev, 0);
@@ -253,7 +253,7 @@ static void tick_do_broadcast_on_off(voi
break;
}
- if (cpus_empty(tick_broadcast_mask)) {
+ if (cpumask_empty(tick_get_broadcast_mask())) {
if (!bc_stopped)
clockevents_shutdown(bc);
} else if (bc_stopped) {
@@ -272,7 +272,7 @@ out:
*/
void tick_broadcast_on_off(unsigned long reason, int *oncpu)
{
- if (!cpu_isset(*oncpu, cpu_online_map))
+ if (!cpumask_test_cpu(*oncpu, cpu_online_mask))
printk(KERN_ERR "tick-broadcast: ignoring broadcast for "
"offline CPU #%d\n", *oncpu);
else
@@ -303,10 +303,10 @@ void tick_shutdown_broadcast(unsigned in
spin_lock_irqsave(&tick_broadcast_lock, flags);
bc = tick_broadcast_device.evtdev;
- cpu_clear(cpu, tick_broadcast_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_mask());
if (tick_broadcast_device.mode == TICKDEV_MODE_PERIODIC) {
- if (bc && cpus_empty(tick_broadcast_mask))
+ if (bc && cpumask_empty(tick_get_broadcast_mask()))
clockevents_shutdown(bc);
}
@@ -342,10 +342,10 @@ int tick_resume_broadcast(void)
switch (tick_broadcast_device.mode) {
case TICKDEV_MODE_PERIODIC:
- if(!cpus_empty(tick_broadcast_mask))
+ if (!cpumask_empty(tick_get_broadcast_mask()))
tick_broadcast_start_periodic(bc);
- broadcast = cpu_isset(smp_processor_id(),
- tick_broadcast_mask);
+ broadcast = cpumask_test_cpu(smp_processor_id(),
+ tick_get_broadcast_mask());
break;
case TICKDEV_MODE_ONESHOT:
broadcast = tick_resume_broadcast_oneshot(bc);
@@ -360,14 +360,15 @@ int tick_resume_broadcast(void)
#ifdef CONFIG_TICK_ONESHOT
-static cpumask_t tick_broadcast_oneshot_mask;
+/* FIXME: use cpumask_var_t. */
+static DECLARE_BITMAP(tick_broadcast_oneshot_mask, CONFIG_NR_CPUS);
/*
- * Debugging: see timer_list.c
+ * Exposed for debugging: see timer_list.c
*/
-cpumask_t *tick_get_broadcast_oneshot_mask(void)
+struct cpumask *tick_get_broadcast_oneshot_mask(void)
{
- return &tick_broadcast_oneshot_mask;
+ return to_cpumask(tick_broadcast_oneshot_mask);
}
static int tick_broadcast_set_event(ktime_t expires, int force)
@@ -389,7 +390,7 @@ int tick_resume_broadcast_oneshot(struct
*/
void tick_check_oneshot_broadcast(int cpu)
{
- if (cpu_isset(cpu, tick_broadcast_oneshot_mask)) {
+ if (cpumask_test_cpu(cpu, to_cpumask(tick_broadcast_oneshot_mask))) {
struct tick_device *td = &per_cpu(tick_cpu_device, cpu);
clockevents_set_mode(td->evtdev, CLOCK_EVT_MODE_ONESHOT);
@@ -402,7 +403,6 @@ void tick_check_oneshot_broadcast(int cp
static void tick_handle_oneshot_broadcast(struct clock_event_device *dev)
{
struct tick_device *td;
- cpumask_t mask;
ktime_t now, next_event;
int cpu;
@@ -410,13 +410,13 @@ static void tick_handle_oneshot_broadcas
again:
dev->next_event.tv64 = KTIME_MAX;
next_event.tv64 = KTIME_MAX;
- mask = CPU_MASK_NONE;
+ cpumask_clear(to_cpumask(tmpmask));
now = ktime_get();
/* Find all expired events */
- for_each_cpu_mask_nr(cpu, tick_broadcast_oneshot_mask) {
+ for_each_cpu(cpu, tick_get_broadcast_oneshot_mask()) {
td = &per_cpu(tick_cpu_device, cpu);
if (td->evtdev->next_event.tv64 <= now.tv64)
- cpu_set(cpu, mask);
+ cpumask_set_cpu(cpu, to_cpumask(tmpmask));
else if (td->evtdev->next_event.tv64 < next_event.tv64)
next_event.tv64 = td->evtdev->next_event.tv64;
}
@@ -424,7 +424,7 @@ again:
/*
* Wakeup the cpus which have an expired event.
*/
- tick_do_broadcast(mask);
+ tick_do_broadcast(to_cpumask(tmpmask));
/*
* Two reasons for reprogram:
@@ -476,15 +476,16 @@ void tick_broadcast_oneshot_control(unsi
goto out;
if (reason == CLOCK_EVT_NOTIFY_BROADCAST_ENTER) {
- if (!cpu_isset(cpu, tick_broadcast_oneshot_mask)) {
- cpu_set(cpu, tick_broadcast_oneshot_mask);
+ if (!cpumask_test_cpu(cpu, tick_get_broadcast_oneshot_mask())) {
+ cpumask_set_cpu(cpu, tick_get_broadcast_oneshot_mask());
clockevents_set_mode(dev, CLOCK_EVT_MODE_SHUTDOWN);
if (dev->next_event.tv64 < bc->next_event.tv64)
tick_broadcast_set_event(dev->next_event, 1);
}
} else {
- if (cpu_isset(cpu, tick_broadcast_oneshot_mask)) {
- cpu_clear(cpu, tick_broadcast_oneshot_mask);
+ if (cpumask_test_cpu(cpu, tick_get_broadcast_oneshot_mask())) {
+ cpumask_clear_cpu(cpu,
+ tick_get_broadcast_oneshot_mask());
clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT);
if (dev->next_event.tv64 != KTIME_MAX)
tick_program_event(dev->next_event, 1);
@@ -502,15 +503,16 @@ out:
*/
static void tick_broadcast_clear_oneshot(int cpu)
{
- cpu_clear(cpu, tick_broadcast_oneshot_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_oneshot_mask());
}
-static void tick_broadcast_init_next_event(cpumask_t *mask, ktime_t expires)
+static void tick_broadcast_init_next_event(struct cpumask *mask,
+ ktime_t expires)
{
struct tick_device *td;
int cpu;
- for_each_cpu_mask_nr(cpu, *mask) {
+ for_each_cpu(cpu, mask) {
td = &per_cpu(tick_cpu_device, cpu);
if (td->evtdev)
td->evtdev->next_event = expires;
@@ -526,7 +528,6 @@ void tick_broadcast_setup_oneshot(struct
if (bc->event_handler != tick_handle_oneshot_broadcast) {
int was_periodic = bc->mode == CLOCK_EVT_MODE_PERIODIC;
int cpu = smp_processor_id();
- cpumask_t mask;
bc->event_handler = tick_handle_oneshot_broadcast;
clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
@@ -540,13 +541,15 @@ void tick_broadcast_setup_oneshot(struct
* oneshot_mask bits for those and program the
* broadcast device to fire.
*/
- mask = tick_broadcast_mask;
- cpu_clear(cpu, mask);
- cpus_or(tick_broadcast_oneshot_mask,
- tick_broadcast_oneshot_mask, mask);
-
- if (was_periodic && !cpus_empty(mask)) {
- tick_broadcast_init_next_event(&mask, tick_next_period);
+ cpumask_copy(to_cpumask(tmpmask), tick_get_broadcast_mask());
+ cpumask_clear_cpu(cpu, to_cpumask(tmpmask));
+ cpumask_or(tick_get_broadcast_oneshot_mask(),
+ tick_get_broadcast_oneshot_mask(),
+ to_cpumask(tmpmask));
+
+ if (was_periodic && !cpumask_empty(to_cpumask(tmpmask))) {
+ tick_broadcast_init_next_event(to_cpumask(tmpmask),
+ tick_next_period);
tick_broadcast_set_event(tick_next_period, 1);
} else
bc->next_event.tv64 = KTIME_MAX;
@@ -585,7 +588,7 @@ void tick_shutdown_broadcast_oneshot(uns
* Clear the broadcast mask flag for the dead cpu, but do not
* stop the broadcast device!
*/
- cpu_clear(cpu, tick_broadcast_oneshot_mask);
+ cpumask_clear_cpu(cpu, tick_get_broadcast_oneshot_mask());
spin_unlock_irqrestore(&tick_broadcast_lock, flags);
}
--- linux-2.6-for-ingo.orig/kernel/time/tick-common.c
+++ linux-2.6-for-ingo/kernel/time/tick-common.c
@@ -254,7 +254,7 @@ static int tick_check_new_device(struct
curdev = NULL;
}
clockevents_exchange_device(curdev, newdev);
- tick_setup_device(td, newdev, cpu, &cpumask_of_cpu(cpu));
+ tick_setup_device(td, newdev, cpu, cpumask_of(cpu));
if (newdev->features & CLOCK_EVT_FEAT_ONESHOT)
tick_oneshot_notify();
@@ -299,9 +299,9 @@ static void tick_shutdown(unsigned int *
}
/* Transfer the do_timer job away from this cpu */
if (*cpup == tick_do_timer_cpu) {
- int cpu = first_cpu(cpu_online_map);
+ int cpu = cpumask_first(cpu_online_mask);
- tick_do_timer_cpu = (cpu != NR_CPUS) ? cpu :
+ tick_do_timer_cpu = (cpu < nr_cpu_ids) ? cpu :
TICK_DO_TIMER_NONE;
}
spin_unlock_irqrestore(&tick_device_lock, flags);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (2 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 16:16 ` Steven Rostedt
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
` (3 subsequent siblings)
7 siblings, 1 reply; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Steven Rostedt
[-- Attachment #1: cpumask:convert-kernel_trace.patch --]
[-- Type: text/plain, Size: 11402 bytes --]
Impact: Reduce memory usage, use new cpumask API.
Convert kernel trace functions to use struct cpumask.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
---
kernel/trace/ring_buffer.c | 42 +++++++++++++++++------------
kernel/trace/trace.c | 62 +++++++++++++++++++++++++------------------
kernel/trace/trace_sysprof.c | 13 ++-------
3 files changed, 65 insertions(+), 52 deletions(-)
--- linux-2.6-for-ingo.orig/kernel/trace/ring_buffer.c
+++ linux-2.6-for-ingo/kernel/trace/ring_buffer.c
@@ -189,7 +189,7 @@ void *ring_buffer_event_data(struct ring
}
#define for_each_buffer_cpu(buffer, cpu) \
- for_each_cpu_mask(cpu, buffer->cpumask)
+ for_each_cpu(cpu, buffer->cpumask)
#define TS_SHIFT 27
#define TS_MASK ((1ULL << TS_SHIFT) - 1)
@@ -262,7 +262,7 @@ struct ring_buffer {
unsigned pages;
unsigned flags;
int cpus;
- cpumask_t cpumask;
+ cpumask_var_t cpumask;
atomic_t record_disabled;
struct mutex mutex;
@@ -453,6 +453,9 @@ struct ring_buffer *ring_buffer_alloc(un
if (!buffer)
return NULL;
+ if (!alloc_cpumask_var(&buffer->cpumask, GFP_KERNEL))
+ goto fail_free_buffer;
+
buffer->pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE);
buffer->flags = flags;
@@ -460,14 +463,14 @@ struct ring_buffer *ring_buffer_alloc(un
if (buffer->pages == 1)
buffer->pages++;
- buffer->cpumask = cpu_possible_map;
+ cpumask_copy(buffer->cpumask, cpu_possible_mask);
buffer->cpus = nr_cpu_ids;
bsize = sizeof(void *) * nr_cpu_ids;
buffer->buffers = kzalloc(ALIGN(bsize, cache_line_size()),
GFP_KERNEL);
if (!buffer->buffers)
- goto fail_free_buffer;
+ goto fail_free_cpumask;
for_each_buffer_cpu(buffer, cpu) {
buffer->buffers[cpu] =
@@ -487,6 +490,9 @@ struct ring_buffer *ring_buffer_alloc(un
}
kfree(buffer->buffers);
+ fail_free_cpumask:
+ free_cpumask_var(buffer->cpumask);
+
fail_free_buffer:
kfree(buffer);
return NULL;
@@ -504,6 +510,8 @@ ring_buffer_free(struct ring_buffer *buf
for_each_buffer_cpu(buffer, cpu)
rb_free_cpu_buffer(buffer->buffers[cpu]);
+ free_cpumask_var(buffer->cpumask);
+
kfree(buffer);
}
@@ -1260,7 +1268,7 @@ ring_buffer_lock_reserve(struct ring_buf
cpu = raw_smp_processor_id();
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
goto out;
cpu_buffer = buffer->buffers[cpu];
@@ -1371,7 +1379,7 @@ int ring_buffer_write(struct ring_buffer
cpu = raw_smp_processor_id();
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
goto out;
cpu_buffer = buffer->buffers[cpu];
@@ -1450,7 +1458,7 @@ void ring_buffer_record_disable_cpu(stru
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return;
cpu_buffer = buffer->buffers[cpu];
@@ -1469,7 +1477,7 @@ void ring_buffer_record_enable_cpu(struc
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return;
cpu_buffer = buffer->buffers[cpu];
@@ -1485,7 +1493,7 @@ unsigned long ring_buffer_entries_cpu(st
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return 0;
cpu_buffer = buffer->buffers[cpu];
@@ -1501,7 +1509,7 @@ unsigned long ring_buffer_overrun_cpu(st
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return 0;
cpu_buffer = buffer->buffers[cpu];
@@ -1814,7 +1822,7 @@ rb_buffer_peek(struct ring_buffer *buffe
struct buffer_page *reader;
int nr_loops = 0;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return NULL;
cpu_buffer = buffer->buffers[cpu];
@@ -1987,7 +1995,7 @@ ring_buffer_consume(struct ring_buffer *
struct ring_buffer_event *event;
unsigned long flags;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return NULL;
spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
@@ -2023,7 +2031,7 @@ ring_buffer_read_start(struct ring_buffe
struct ring_buffer_iter *iter;
unsigned long flags;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return NULL;
iter = kmalloc(sizeof(*iter), GFP_KERNEL);
@@ -2129,7 +2137,7 @@ void ring_buffer_reset_cpu(struct ring_b
struct ring_buffer_per_cpu *cpu_buffer = buffer->buffers[cpu];
unsigned long flags;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return;
spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
@@ -2182,7 +2190,7 @@ int ring_buffer_empty_cpu(struct ring_bu
{
struct ring_buffer_per_cpu *cpu_buffer;
- if (!cpu_isset(cpu, buffer->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
return 1;
cpu_buffer = buffer->buffers[cpu];
@@ -2205,8 +2213,8 @@ int ring_buffer_swap_cpu(struct ring_buf
struct ring_buffer_per_cpu *cpu_buffer_a;
struct ring_buffer_per_cpu *cpu_buffer_b;
- if (!cpu_isset(cpu, buffer_a->cpumask) ||
- !cpu_isset(cpu, buffer_b->cpumask))
+ if (!cpumask_test_cpu(cpu, buffer_a->cpumask) ||
+ !cpumask_test_cpu(cpu, buffer_b->cpumask))
return -EINVAL;
/* At least make sure the two buffers are somewhat the same */
--- linux-2.6-for-ingo.orig/kernel/trace/trace.c
+++ linux-2.6-for-ingo/kernel/trace/trace.c
@@ -90,10 +90,10 @@ static inline void ftrace_enable_cpu(voi
preempt_enable();
}
-static cpumask_t __read_mostly tracing_buffer_mask;
+static cpumask_var_t __read_mostly tracing_buffer_mask;
#define for_each_tracing_cpu(cpu) \
- for_each_cpu_mask(cpu, tracing_buffer_mask)
+ for_each_cpu(cpu, tracing_buffer_mask)
/*
* ftrace_dump_on_oops - variable to dump ftrace buffer on oops
@@ -2618,13 +2618,7 @@ static struct file_operations show_trace
/*
* Only trace on a CPU if the bitmask is set:
*/
-static cpumask_t tracing_cpumask = CPU_MASK_ALL;
-
-/*
- * When tracing/tracing_cpu_mask is modified then this holds
- * the new bitmask we are about to install:
- */
-static cpumask_t tracing_cpumask_new;
+static cpumask_var_t tracing_cpumask;
/*
* The tracer itself will not take this lock, but still we want
@@ -2646,7 +2640,7 @@ tracing_cpumask_read(struct file *filp,
mutex_lock(&tracing_cpumask_update_lock);
- len = cpumask_scnprintf(mask_str, count, &tracing_cpumask);
+ len = cpumask_scnprintf(mask_str, count, tracing_cpumask);
if (count - len < 2) {
count = -EINVAL;
goto out_err;
@@ -2665,9 +2659,13 @@ tracing_cpumask_write(struct file *filp,
size_t count, loff_t *ppos)
{
int err, cpu;
+ cpumask_var_t tracing_cpumask_new;
+
+ if (!alloc_cpumask_var(&tracing_cpumask_new, GFP_KERNEL))
+ return -ENOMEM;
mutex_lock(&tracing_cpumask_update_lock);
- err = cpumask_parse_user(ubuf, count, &tracing_cpumask_new);
+ err = cpumask_parse_user(ubuf, count, tracing_cpumask_new);
if (err)
goto err_unlock;
@@ -2678,26 +2676,28 @@ tracing_cpumask_write(struct file *filp,
* Increase/decrease the disabled counter if we are
* about to flip a bit in the cpumask:
*/
- if (cpu_isset(cpu, tracing_cpumask) &&
- !cpu_isset(cpu, tracing_cpumask_new)) {
+ if (cpumask_test_cpu(cpu, tracing_cpumask) &&
+ !cpumask_test_cpu(cpu, tracing_cpumask_new)) {
atomic_inc(&global_trace.data[cpu]->disabled);
}
- if (!cpu_isset(cpu, tracing_cpumask) &&
- cpu_isset(cpu, tracing_cpumask_new)) {
+ if (!cpumask_test_cpu(cpu, tracing_cpumask) &&
+ cpumask_test_cpu(cpu, tracing_cpumask_new)) {
atomic_dec(&global_trace.data[cpu]->disabled);
}
}
__raw_spin_unlock(&ftrace_max_lock);
local_irq_enable();
- tracing_cpumask = tracing_cpumask_new;
+ cpumask_copy(tracing_cpumask, tracing_cpumask_new);
mutex_unlock(&tracing_cpumask_update_lock);
+ free_cpumask_var(tracing_cpumask_new);
return count;
err_unlock:
mutex_unlock(&tracing_cpumask_update_lock);
+ free_cpumask_var(tracing_cpumask);
return err;
}
@@ -3724,7 +3724,6 @@ void ftrace_dump(void)
static DEFINE_SPINLOCK(ftrace_dump_lock);
/* use static because iter can be a bit big for the stack */
static struct trace_iterator iter;
- static cpumask_t mask;
static int dump_ran;
unsigned long flags;
int cnt = 0, cpu;
@@ -3758,8 +3757,6 @@ void ftrace_dump(void)
* and then release the locks again.
*/
- cpus_clear(mask);
-
while (!trace_empty(&iter)) {
if (!cnt)
@@ -3795,19 +3792,28 @@ __init static int tracer_alloc_buffers(v
{
struct trace_array_cpu *data;
int i;
+ int ret = -ENOMEM;
- /* TODO: make the number of buffers hot pluggable with CPUS */
- tracing_buffer_mask = cpu_possible_map;
+ if (!alloc_cpumask_var(&tracing_buffer_mask, GFP_KERNEL))
+ goto out;
+
+ if (!alloc_cpumask_var(&tracing_cpumask, GFP_KERNEL))
+ goto out_free_buffer_mask;
+ cpumask_copy(tracing_buffer_mask, cpu_possible_mask);
+ cpumask_copy(tracing_cpumask, cpu_all_mask);
+
+ /* TODO: make the number of buffers hot pluggable with CPUS */
global_trace.buffer = ring_buffer_alloc(trace_buf_size,
TRACE_BUFFER_FLAGS);
if (!global_trace.buffer) {
printk(KERN_ERR "tracer: failed to allocate ring buffer!\n");
WARN_ON(1);
- return 0;
+ goto out_free_cpumask;
}
global_trace.entries = ring_buffer_size(global_trace.buffer);
+
#ifdef CONFIG_TRACER_MAX_TRACE
max_tr.buffer = ring_buffer_alloc(trace_buf_size,
TRACE_BUFFER_FLAGS);
@@ -3815,7 +3821,7 @@ __init static int tracer_alloc_buffers(v
printk(KERN_ERR "tracer: failed to allocate max ring buffer!\n");
WARN_ON(1);
ring_buffer_free(global_trace.buffer);
- return 0;
+ goto out_free_cpumask;
}
max_tr.entries = ring_buffer_size(max_tr.buffer);
WARN_ON(max_tr.entries != global_trace.entries);
@@ -3842,11 +3848,17 @@ __init static int tracer_alloc_buffers(v
tracing_disabled = 0;
atomic_notifier_chain_register(&panic_notifier_list,
- &trace_panic_notifier);
+ &trace_panic_notifier);
register_die_notifier(&trace_die_notifier);
+ ret = 0;
- return 0;
+out_free_cpumask:
+ free_cpumask_var(tracing_cpumask);
+out_free_buffer_mask:
+ free_cpumask_var(tracing_buffer_mask);
+out:
+ return ret;
}
early_initcall(tracer_alloc_buffers);
fs_initcall(tracer_init_debugfs);
--- linux-2.6-for-ingo.orig/kernel/trace/trace_sysprof.c
+++ linux-2.6-for-ingo/kernel/trace/trace_sysprof.c
@@ -196,9 +196,9 @@ static enum hrtimer_restart stack_trace_
return HRTIMER_RESTART;
}
-static void start_stack_timer(int cpu)
+static void start_stack_timer(void *unused)
{
- struct hrtimer *hrtimer = &per_cpu(stack_trace_hrtimer, cpu);
+ struct hrtimer *hrtimer = &__get_cpu_var(stack_trace_hrtimer);
hrtimer_init(hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
hrtimer->function = stack_trace_timer_fn;
@@ -209,14 +209,7 @@ static void start_stack_timer(int cpu)
static void start_stack_timers(void)
{
- cpumask_t saved_mask = current->cpus_allowed;
- int cpu;
-
- for_each_online_cpu(cpu) {
- set_cpus_allowed_ptr(current, &cpumask_of_cpu(cpu));
- start_stack_timer(cpu);
- }
- set_cpus_allowed_ptr(current, &saved_mask);
+ on_each_cpu(start_stack_timer, NULL, 1);
}
static void stop_stack_timer(int cpu)
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 5/8] cpumask: convert rest of files in kernel/
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (3 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-20 1:33 ` Lai Jiangshan
2008-12-20 13:02 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
` (2 subsequent siblings)
7 siblings, 2 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: Rusty Russell, linux-kernel, Dipankar Sarma, H. Peter Anvin,
Lai Jiangshan, Max Krasnyansky, Richard Henderson,
Thomas Gleixner, Yinghai Lu
[-- Attachment #1: cpumask:convert-kernel.patch --]
[-- Type: text/plain, Size: 27962 bytes --]
Impact: Reduce stack usage, use new cpumask API. ALPHA mod!
A couple of uses of cpumask_any_but() here to avoid temporary cpumasks
and carry the "const" modifier through the seq_cpumask() function all the
way to bitmap_scnprintf().
Note that prof_cpu_mask will be difficult to convert to a cpumask_var_t
since it needs to be ready from the first scheduler tick.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Dipankar Sarma <dipankar@in.ibm.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Max Krasnyansky <maxk@qualcomm.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Yinghai Lu <yinghai@kernel.org>
---
arch/alpha/kernel/irq.c | 3 ++-
fs/seq_file.c | 3 ++-
include/linux/interrupt.h | 2 +-
include/linux/rcuclassic.h | 4 ++--
include/linux/seq_file.h | 5 +++--
include/linux/stop_machine.h | 6 +++---
kernel/cpu.c | 40 +++++++++++++++++++++-------------------
kernel/irq/manage.c | 11 +++++++++--
kernel/irq/proc.c | 32 +++++++++++++++++++++-----------
kernel/power/poweroff.c | 2 +-
kernel/profile.c | 31 ++++++++++++++++++-------------
kernel/rcuclassic.c | 32 +++++++++++++++++---------------
kernel/rcupreempt.c | 19 ++++++++++---------
kernel/rcutorture.c | 27 +++++++++++++++------------
kernel/softlockup.c | 6 ++----
kernel/stop_machine.c | 8 ++++----
kernel/taskstats.c | 39 ++++++++++++++++++++++++---------------
17 files changed, 155 insertions(+), 115 deletions(-)
--- linux-2.6-for-ingo.orig/arch/alpha/kernel/irq.c
+++ linux-2.6-for-ingo/arch/alpha/kernel/irq.c
@@ -50,7 +50,8 @@ int irq_select_affinity(unsigned int irq
if (!irq_desc[irq].chip->set_affinity || irq_user_affinity[irq])
return 1;
- while (!cpu_possible(cpu) || !cpu_isset(cpu, irq_default_affinity))
+ while (!cpu_possible(cpu) ||
+ !cpumask_test_cpu(cpu, irq_default_affinity))
cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
last_cpu = cpu;
--- linux-2.6-for-ingo.orig/fs/seq_file.c
+++ linux-2.6-for-ingo/fs/seq_file.c
@@ -462,7 +462,8 @@ int seq_dentry(struct seq_file *m, struc
return -1;
}
-int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits)
+int seq_bitmap(struct seq_file *m, const unsigned long *bits,
+ unsigned int nr_bits)
{
if (m->count < m->size) {
int len = bitmap_scnprintf(m->buf + m->count,
--- linux-2.6-for-ingo.orig/include/linux/interrupt.h
+++ linux-2.6-for-ingo/include/linux/interrupt.h
@@ -109,7 +109,7 @@ extern void enable_irq(unsigned int irq)
#if defined(CONFIG_SMP) && defined(CONFIG_GENERIC_HARDIRQS)
-extern cpumask_t irq_default_affinity;
+extern cpumask_var_t irq_default_affinity;
extern int irq_set_affinity(unsigned int irq, const struct cpumask *cpumask);
extern int irq_can_set_affinity(unsigned int irq);
--- linux-2.6-for-ingo.orig/include/linux/rcuclassic.h
+++ linux-2.6-for-ingo/include/linux/rcuclassic.h
@@ -59,8 +59,8 @@ struct rcu_ctrlblk {
int signaled;
spinlock_t lock ____cacheline_internodealigned_in_smp;
- cpumask_t cpumask; /* CPUs that need to switch in order */
- /* for current batch to proceed. */
+ DECLARE_BITMAP(cpumask, NR_CPUS); /* CPUs that need to switch for */
+ /* current batch to proceed. */
} ____cacheline_internodealigned_in_smp;
/* Is batch a before batch b ? */
--- linux-2.6-for-ingo.orig/include/linux/seq_file.h
+++ linux-2.6-for-ingo/include/linux/seq_file.h
@@ -50,8 +50,9 @@ int seq_path(struct seq_file *, struct p
int seq_dentry(struct seq_file *, struct dentry *, char *);
int seq_path_root(struct seq_file *m, struct path *path, struct path *root,
char *esc);
-int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits);
-static inline int seq_cpumask(struct seq_file *m, cpumask_t *mask)
+int seq_bitmap(struct seq_file *m, const unsigned long *bits,
+ unsigned int nr_bits);
+static inline int seq_cpumask(struct seq_file *m, const struct cpumask *mask)
{
return seq_bitmap(m, mask->bits, NR_CPUS);
}
--- linux-2.6-for-ingo.orig/include/linux/stop_machine.h
+++ linux-2.6-for-ingo/include/linux/stop_machine.h
@@ -23,7 +23,7 @@
*
* This can be thought of as a very heavy write lock, equivalent to
* grabbing every spinlock in the kernel. */
-int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
+int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
/**
* __stop_machine: freeze the machine on all CPUs and run this function
@@ -34,11 +34,11 @@ int stop_machine(int (*fn)(void *), void
* Description: This is a special version of the above, which assumes cpus
* won't come or go while it's being called. Used by hotplug cpu.
*/
-int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
+int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
#else
static inline int stop_machine(int (*fn)(void *), void *data,
- const cpumask_t *cpus)
+ const struct cpumask *cpus)
{
int ret;
local_irq_disable();
--- linux-2.6-for-ingo.orig/kernel/cpu.c
+++ linux-2.6-for-ingo/kernel/cpu.c
@@ -218,7 +218,7 @@ static int __ref take_cpu_down(void *_pa
static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
{
int err, nr_calls = 0;
- cpumask_t old_allowed, tmp;
+ cpumask_var_t old_allowed;
void *hcpu = (void *)(long)cpu;
unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
struct take_cpu_down_param tcd_param = {
@@ -232,6 +232,9 @@ static int __ref _cpu_down(unsigned int
if (!cpu_online(cpu))
return -EINVAL;
+ if (!alloc_cpumask_var(&old_allowed, GFP_KERNEL))
+ return -ENOMEM;
+
cpu_hotplug_begin();
err = __raw_notifier_call_chain(&cpu_chain, CPU_DOWN_PREPARE | mod,
hcpu, -1, &nr_calls);
@@ -246,13 +249,11 @@ static int __ref _cpu_down(unsigned int
}
/* Ensure that we are not runnable on dying cpu */
- old_allowed = current->cpus_allowed;
- cpus_setall(tmp);
- cpu_clear(cpu, tmp);
- set_cpus_allowed_ptr(current, &tmp);
- tmp = cpumask_of_cpu(cpu);
+ cpumask_copy(old_allowed, ¤t->cpus_allowed);
+ set_cpus_allowed_ptr(current,
+ cpumask_of(cpumask_any_but(cpu_online_mask, cpu)));
- err = __stop_machine(take_cpu_down, &tcd_param, &tmp);
+ err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
if (err) {
/* CPU didn't die: tell everyone. Can't complain. */
if (raw_notifier_call_chain(&cpu_chain, CPU_DOWN_FAILED | mod,
@@ -278,7 +279,7 @@ static int __ref _cpu_down(unsigned int
check_for_tasks(cpu);
out_allowed:
- set_cpus_allowed_ptr(current, &old_allowed);
+ set_cpus_allowed_ptr(current, old_allowed);
out_release:
cpu_hotplug_done();
if (!err) {
@@ -286,6 +287,7 @@ out_release:
hcpu) == NOTIFY_BAD)
BUG();
}
+ free_cpumask_var(old_allowed);
return err;
}
@@ -304,7 +306,7 @@ int __ref cpu_down(unsigned int cpu)
/*
* Make sure the all cpus did the reschedule and are not
- * using stale version of the cpu_active_map.
+ * using stale version of the cpu_active_mask.
* This is not strictly necessary becuase stop_machine()
* that we run down the line already provides the required
* synchronization. But it's really a side effect and we do not
@@ -368,7 +370,7 @@ out_notify:
int __cpuinit cpu_up(unsigned int cpu)
{
int err = 0;
- if (!cpu_isset(cpu, cpu_possible_map)) {
+ if (!cpu_possible(cpu)) {
printk(KERN_ERR "can't online cpu %d because it is not "
"configured as may-hotadd at boot time\n", cpu);
#if defined(CONFIG_IA64) || defined(CONFIG_X86_64)
@@ -393,25 +395,25 @@ out:
}
#ifdef CONFIG_PM_SLEEP_SMP
-static cpumask_t frozen_cpus;
+static DECLARE_BITMAP(frozen_cpus, CONFIG_NR_CPUS);
int disable_nonboot_cpus(void)
{
int cpu, first_cpu, error = 0;
cpu_maps_update_begin();
- first_cpu = first_cpu(cpu_online_map);
+ first_cpu = cpumask_first(cpu_online_mask);
/* We take down all of the non-boot CPUs in one shot to avoid races
* with the userspace trying to use the CPU hotplug at the same time
*/
- cpus_clear(frozen_cpus);
+ cpumask_clear(to_cpumask(frozen_cpus));
printk("Disabling non-boot CPUs ...\n");
for_each_online_cpu(cpu) {
if (cpu == first_cpu)
continue;
error = _cpu_down(cpu, 1);
if (!error) {
- cpu_set(cpu, frozen_cpus);
+ cpumask_set_cpu(cpu, to_cpumask(frozen_cpus));
printk("CPU%d is down\n", cpu);
} else {
printk(KERN_ERR "Error taking CPU%d down: %d\n",
@@ -437,11 +439,11 @@ void __ref enable_nonboot_cpus(void)
/* Allow everyone to use the CPU hotplug again */
cpu_maps_update_begin();
cpu_hotplug_disabled = 0;
- if (cpus_empty(frozen_cpus))
+ if (cpumask_empty(to_cpumask(frozen_cpus)))
goto out;
printk("Enabling non-boot CPUs ...\n");
- for_each_cpu_mask_nr(cpu, frozen_cpus) {
+ for_each_cpu(cpu, to_cpumask(frozen_cpus)) {
error = _cpu_up(cpu, 1);
if (!error) {
printk("CPU%d is up\n", cpu);
@@ -449,7 +451,7 @@ void __ref enable_nonboot_cpus(void)
}
printk(KERN_WARNING "Error taking CPU%d up: %d\n", cpu, error);
}
- cpus_clear(frozen_cpus);
+ cpumask_clear(to_cpumask(frozen_cpus));
out:
cpu_maps_update_done();
}
@@ -468,7 +470,7 @@ void __cpuinit notify_cpu_starting(unsig
unsigned long val = CPU_STARTING;
#ifdef CONFIG_PM_SLEEP_SMP
- if (cpu_isset(cpu, frozen_cpus))
+ if (cpumask_test_cpu(cpu, to_cpumask(frozen_cpus)))
val = CPU_STARTING_FROZEN;
#endif /* CONFIG_PM_SLEEP_SMP */
raw_notifier_call_chain(&cpu_chain, val, (void *)(long)cpu);
@@ -480,7 +482,7 @@ void __cpuinit notify_cpu_starting(unsig
* cpu_bit_bitmap[] is a special, "compressed" data structure that
* represents all NR_CPUS bits binary values of 1<<nr.
*
- * It is used by cpumask_of_cpu() to get a constant address to a CPU
+ * It is used by cpumask_of() to get a constant address to a CPU
* mask value that has a single bit set only.
*/
--- linux-2.6-for-ingo.orig/kernel/irq/manage.c
+++ linux-2.6-for-ingo/kernel/irq/manage.c
@@ -16,8 +16,15 @@
#include "internals.h"
#ifdef CONFIG_SMP
+cpumask_var_t irq_default_affinity;
-cpumask_t irq_default_affinity = CPU_MASK_ALL;
+static int init_irq_default_affinity(void)
+{
+ alloc_cpumask_var(&irq_default_affinity, GFP_KERNEL);
+ cpumask_setall(irq_default_affinity);
+ return 0;
+}
+core_initcall(init_irq_default_affinity);
/**
* synchronize_irq - wait for pending IRQ handlers (on other CPUs)
@@ -127,7 +134,7 @@ int do_irq_select_affinity(unsigned int
desc->status &= ~IRQ_AFFINITY_SET;
}
- cpumask_and(&desc->affinity, cpu_online_mask, &irq_default_affinity);
+ cpumask_and(&desc->affinity, cpu_online_mask, irq_default_affinity);
set_affinity:
desc->chip->set_affinity(irq, &desc->affinity);
--- linux-2.6-for-ingo.orig/kernel/irq/proc.c
+++ linux-2.6-for-ingo/kernel/irq/proc.c
@@ -20,7 +20,7 @@ static struct proc_dir_entry *root_irq_d
static int irq_affinity_proc_show(struct seq_file *m, void *v)
{
struct irq_desc *desc = irq_to_desc((long)m->private);
- cpumask_t *mask = &desc->affinity;
+ const struct cpumask *mask = &desc->affinity;
#ifdef CONFIG_GENERIC_PENDING_IRQ
if (desc->status & IRQ_MOVE_PENDING)
@@ -93,7 +93,7 @@ static const struct file_operations irq_
static int default_affinity_show(struct seq_file *m, void *v)
{
- seq_cpumask(m, &irq_default_affinity);
+ seq_cpumask(m, irq_default_affinity);
seq_putc(m, '\n');
return 0;
}
@@ -101,27 +101,37 @@ static int default_affinity_show(struct
static ssize_t default_affinity_write(struct file *file,
const char __user *buffer, size_t count, loff_t *ppos)
{
- cpumask_t new_value;
+ cpumask_var_t new_value;
int err;
- err = cpumask_parse_user(buffer, count, &new_value);
+ if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
+ return -ENOMEM;
+
+ err = cpumask_parse_user(buffer, count, new_value);
if (err)
- return err;
+ goto out;
- if (!is_affinity_mask_valid(new_value))
- return -EINVAL;
+ if (!is_affinity_mask_valid(new_value)) {
+ err = -EINVAL;
+ goto out;
+ }
/*
* Do not allow disabling IRQs completely - it's a too easy
* way to make the system unusable accidentally :-) At least
* one online CPU still has to be targeted.
*/
- if (!cpus_intersects(new_value, cpu_online_map))
- return -EINVAL;
+ if (!cpumask_intersects(new_value, cpu_online_mask)) {
+ err = -EINVAL;
+ goto out;
+ }
- irq_default_affinity = new_value;
+ cpumask_copy(irq_default_affinity, new_value);
+ err = count;
- return count;
+out:
+ free_cpumask_var(new_value);
+ return err;
}
static int default_affinity_open(struct inode *inode, struct file *file)
--- linux-2.6-for-ingo.orig/kernel/power/poweroff.c
+++ linux-2.6-for-ingo/kernel/power/poweroff.c
@@ -27,7 +27,7 @@ static DECLARE_WORK(poweroff_work, do_po
static void handle_poweroff(int key, struct tty_struct *tty)
{
/* run sysrq poweroff on boot cpu */
- schedule_work_on(first_cpu(cpu_online_map), &poweroff_work);
+ schedule_work_on(cpumask_first(cpu_online_mask), &poweroff_work);
}
static struct sysrq_key_op sysrq_poweroff_op = {
--- linux-2.6-for-ingo.orig/kernel/profile.c
+++ linux-2.6-for-ingo/kernel/profile.c
@@ -45,7 +45,7 @@ static unsigned long prof_len, prof_shif
int prof_on __read_mostly;
EXPORT_SYMBOL_GPL(prof_on);
-static cpumask_t prof_cpu_mask = CPU_MASK_ALL;
+static DECLARE_BITMAP(prof_cpu_mask, CONFIG_NR_CPUS) = CPU_BITS_ALL;
#ifdef CONFIG_SMP
static DEFINE_PER_CPU(struct profile_hit *[2], cpu_profile_hits);
static DEFINE_PER_CPU(int, cpu_profile_flip);
@@ -386,13 +386,13 @@ out_free:
return NOTIFY_BAD;
case CPU_ONLINE:
case CPU_ONLINE_FROZEN:
- cpu_set(cpu, prof_cpu_mask);
+ cpumask_set_cpu(cpu, to_cpumask(prof_cpu_mask));
break;
case CPU_UP_CANCELED:
case CPU_UP_CANCELED_FROZEN:
case CPU_DEAD:
case CPU_DEAD_FROZEN:
- cpu_clear(cpu, prof_cpu_mask);
+ cpumask_clear_cpu(cpu, to_cpumask(prof_cpu_mask));
if (per_cpu(cpu_profile_hits, cpu)[0]) {
page = virt_to_page(per_cpu(cpu_profile_hits, cpu)[0]);
per_cpu(cpu_profile_hits, cpu)[0] = NULL;
@@ -430,7 +430,8 @@ void profile_tick(int type)
if (type == CPU_PROFILING && timer_hook)
timer_hook(regs);
- if (!user_mode(regs) && cpu_isset(smp_processor_id(), prof_cpu_mask))
+ if (!user_mode(regs) &&
+ cpumask_test_cpu(smp_processor_id(), to_cpumask(prof_cpu_mask)))
profile_hit(type, (void *)profile_pc(regs));
}
@@ -442,7 +443,7 @@ void profile_tick(int type)
static int prof_cpu_mask_read_proc(char *page, char **start, off_t off,
int count, int *eof, void *data)
{
- int len = cpumask_scnprintf(page, count, (cpumask_t *)data);
+ int len = cpumask_scnprintf(page, count, data);
if (count - len < 2)
return -EINVAL;
len += sprintf(page + len, "\n");
@@ -452,16 +453,20 @@ static int prof_cpu_mask_read_proc(char
static int prof_cpu_mask_write_proc(struct file *file,
const char __user *buffer, unsigned long count, void *data)
{
- cpumask_t *mask = (cpumask_t *)data;
+ struct cpumask *mask = data;
unsigned long full_count = count, err;
- cpumask_t new_value;
+ cpumask_var_t new_value;
- err = cpumask_parse_user(buffer, count, &new_value);
- if (err)
- return err;
+ if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
+ return -ENOMEM;
- *mask = new_value;
- return full_count;
+ err = cpumask_parse_user(buffer, count, new_value);
+ if (!err) {
+ cpumask_copy(mask, new_value);
+ err = full_count;
+ }
+ free_cpumask_var(new_value);
+ return err;
}
void create_prof_cpu_mask(struct proc_dir_entry *root_irq_dir)
@@ -472,7 +477,7 @@ void create_prof_cpu_mask(struct proc_di
entry = create_proc_entry("prof_cpu_mask", 0600, root_irq_dir);
if (!entry)
return;
- entry->data = (void *)&prof_cpu_mask;
+ entry->data = prof_cpu_mask;
entry->read_proc = prof_cpu_mask_read_proc;
entry->write_proc = prof_cpu_mask_write_proc;
}
--- linux-2.6-for-ingo.orig/kernel/rcuclassic.c
+++ linux-2.6-for-ingo/kernel/rcuclassic.c
@@ -63,14 +63,14 @@ static struct rcu_ctrlblk rcu_ctrlblk =
.completed = -300,
.pending = -300,
.lock = __SPIN_LOCK_UNLOCKED(&rcu_ctrlblk.lock),
- .cpumask = CPU_MASK_NONE,
+ .cpumask = CPU_BITS_NONE,
};
static struct rcu_ctrlblk rcu_bh_ctrlblk = {
.cur = -300,
.completed = -300,
.pending = -300,
.lock = __SPIN_LOCK_UNLOCKED(&rcu_bh_ctrlblk.lock),
- .cpumask = CPU_MASK_NONE,
+ .cpumask = CPU_BITS_NONE,
};
DEFINE_PER_CPU(struct rcu_data, rcu_data) = { 0L };
@@ -85,7 +85,6 @@ static void force_quiescent_state(struct
struct rcu_ctrlblk *rcp)
{
int cpu;
- cpumask_t cpumask;
unsigned long flags;
set_need_resched();
@@ -96,10 +95,10 @@ static void force_quiescent_state(struct
* Don't send IPI to itself. With irqs disabled,
* rdp->cpu is the current cpu.
*
- * cpu_online_map is updated by the _cpu_down()
+ * cpu_online_mask is updated by the _cpu_down()
* using __stop_machine(). Since we're in irqs disabled
* section, __stop_machine() is not exectuting, hence
- * the cpu_online_map is stable.
+ * the cpu_online_mask is stable.
*
* However, a cpu might have been offlined _just_ before
* we disabled irqs while entering here.
@@ -107,13 +106,14 @@ static void force_quiescent_state(struct
* notification, leading to the offlined cpu's bit
* being set in the rcp->cpumask.
*
- * Hence cpumask = (rcp->cpumask & cpu_online_map) to prevent
+ * Hence cpumask = (rcp->cpumask & cpu_online_mask) to prevent
* sending smp_reschedule() to an offlined CPU.
*/
- cpus_and(cpumask, rcp->cpumask, cpu_online_map);
- cpu_clear(rdp->cpu, cpumask);
- for_each_cpu_mask_nr(cpu, cpumask)
- smp_send_reschedule(cpu);
+ for_each_cpu_and(cpu,
+ to_cpumask(rcp->cpumask), cpu_online_mask) {
+ if (cpu != rdp->cpu)
+ smp_send_reschedule(cpu);
+ }
}
spin_unlock_irqrestore(&rcp->lock, flags);
}
@@ -193,7 +193,7 @@ static void print_other_cpu_stall(struct
printk(KERN_ERR "INFO: RCU detected CPU stalls:");
for_each_possible_cpu(cpu) {
- if (cpu_isset(cpu, rcp->cpumask))
+ if (cpumask_test_cpu(cpu, to_cpumask(rcp->cpumask)))
printk(" %d", cpu);
}
printk(" (detected by %d, t=%ld jiffies)\n",
@@ -221,7 +221,8 @@ static void check_cpu_stall(struct rcu_c
long delta;
delta = jiffies - rcp->jiffies_stall;
- if (cpu_isset(smp_processor_id(), rcp->cpumask) && delta >= 0) {
+ if (cpumask_test_cpu(smp_processor_id(), to_cpumask(rcp->cpumask)) &&
+ delta >= 0) {
/* We haven't checked in, so go dump stack. */
print_cpu_stall(rcp);
@@ -393,7 +394,8 @@ static void rcu_start_batch(struct rcu_c
* unnecessarily.
*/
smp_mb();
- cpumask_andnot(&rcp->cpumask, cpu_online_mask, nohz_cpu_mask);
+ cpumask_andnot(to_cpumask(rcp->cpumask),
+ cpu_online_mask, nohz_cpu_mask);
rcp->signaled = 0;
}
@@ -406,8 +408,8 @@ static void rcu_start_batch(struct rcu_c
*/
static void cpu_quiet(int cpu, struct rcu_ctrlblk *rcp)
{
- cpu_clear(cpu, rcp->cpumask);
- if (cpus_empty(rcp->cpumask)) {
+ cpumask_clear_cpu(cpu, to_cpumask(rcp->cpumask));
+ if (cpumask_empty(to_cpumask(rcp->cpumask))) {
/* batch completed ! */
rcp->completed = rcp->cur;
rcu_start_batch(rcp);
--- linux-2.6-for-ingo.orig/kernel/rcupreempt.c
+++ linux-2.6-for-ingo/kernel/rcupreempt.c
@@ -164,7 +164,8 @@ static char *rcu_try_flip_state_names[]
{ "idle", "waitack", "waitzero", "waitmb" };
#endif /* #ifdef CONFIG_RCU_TRACE */
-static cpumask_t rcu_cpu_online_map __read_mostly = CPU_MASK_NONE;
+static DECLARE_BITMAP(rcu_cpu_online_map, CONFIG_NR_CPUS) __read_mostly
+ = CPU_BITS_NONE;
/*
* Enum and per-CPU flag to determine when each CPU has seen
@@ -748,7 +749,7 @@ rcu_try_flip_idle(void)
/* Now ask each CPU for acknowledgement of the flip. */
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
per_cpu(rcu_flip_flag, cpu) = rcu_flipped;
dyntick_save_progress_counter(cpu);
}
@@ -766,7 +767,7 @@ rcu_try_flip_waitack(void)
int cpu;
RCU_TRACE_ME(rcupreempt_trace_try_flip_a1);
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
if (rcu_try_flip_waitack_needed(cpu) &&
per_cpu(rcu_flip_flag, cpu) != rcu_flip_seen) {
RCU_TRACE_ME(rcupreempt_trace_try_flip_ae1);
@@ -798,7 +799,7 @@ rcu_try_flip_waitzero(void)
/* Check to see if the sum of the "last" counters is zero. */
RCU_TRACE_ME(rcupreempt_trace_try_flip_z1);
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
sum += RCU_DATA_CPU(cpu)->rcu_flipctr[lastidx];
if (sum != 0) {
RCU_TRACE_ME(rcupreempt_trace_try_flip_ze1);
@@ -813,7 +814,7 @@ rcu_try_flip_waitzero(void)
smp_mb(); /* ^^^^^^^^^^^^ */
/* Call for a memory barrier from each CPU. */
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
per_cpu(rcu_mb_flag, cpu) = rcu_mb_needed;
dyntick_save_progress_counter(cpu);
}
@@ -833,7 +834,7 @@ rcu_try_flip_waitmb(void)
int cpu;
RCU_TRACE_ME(rcupreempt_trace_try_flip_m1);
- for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
+ for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
if (rcu_try_flip_waitmb_needed(cpu) &&
per_cpu(rcu_mb_flag, cpu) != rcu_mb_done) {
RCU_TRACE_ME(rcupreempt_trace_try_flip_me1);
@@ -1022,7 +1023,7 @@ void rcu_offline_cpu(int cpu)
RCU_DATA_CPU(cpu)->rcu_flipctr[0] = 0;
RCU_DATA_CPU(cpu)->rcu_flipctr[1] = 0;
- cpu_clear(cpu, rcu_cpu_online_map);
+ cpumask_clear_cpu(cpu, to_cpumask(rcu_cpu_online_map));
spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
@@ -1062,7 +1063,7 @@ void __cpuinit rcu_online_cpu(int cpu)
struct rcu_data *rdp;
spin_lock_irqsave(&rcu_ctrlblk.fliplock, flags);
- cpu_set(cpu, rcu_cpu_online_map);
+ cpumask_set_cpu(cpu, to_cpumask(rcu_cpu_online_map));
spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
/*
@@ -1420,7 +1421,7 @@ void __init __rcu_init(void)
* We don't need protection against CPU-Hotplug here
* since
* a) If a CPU comes online while we are iterating over the
- * cpu_online_map below, we would only end up making a
+ * cpu_online_mask below, we would only end up making a
* duplicate call to rcu_online_cpu() which sets the corresponding
* CPU's mask in the rcu_cpu_online_map.
*
--- linux-2.6-for-ingo.orig/kernel/rcutorture.c
+++ linux-2.6-for-ingo/kernel/rcutorture.c
@@ -843,49 +843,52 @@ static int rcu_idle_cpu; /* Force all to
*/
static void rcu_torture_shuffle_tasks(void)
{
- cpumask_t tmp_mask;
+ cpumask_var_t tmp_mask;
int i;
- cpus_setall(tmp_mask);
+ if (!alloc_cpumask_var(&tmp_mask, GFP_KERNEL))
+ BUG();
+
+ cpumask_setall(tmp_mask);
get_online_cpus();
/* No point in shuffling if there is only one online CPU (ex: UP) */
- if (num_online_cpus() == 1) {
- put_online_cpus();
- return;
- }
+ if (num_online_cpus() == 1)
+ goto out;
if (rcu_idle_cpu != -1)
- cpu_clear(rcu_idle_cpu, tmp_mask);
+ cpumask_clear_cpu(rcu_idle_cpu, tmp_mask);
- set_cpus_allowed_ptr(current, &tmp_mask);
+ set_cpus_allowed_ptr(current, tmp_mask);
if (reader_tasks) {
for (i = 0; i < nrealreaders; i++)
if (reader_tasks[i])
set_cpus_allowed_ptr(reader_tasks[i],
- &tmp_mask);
+ tmp_mask);
}
if (fakewriter_tasks) {
for (i = 0; i < nfakewriters; i++)
if (fakewriter_tasks[i])
set_cpus_allowed_ptr(fakewriter_tasks[i],
- &tmp_mask);
+ tmp_mask);
}
if (writer_task)
- set_cpus_allowed_ptr(writer_task, &tmp_mask);
+ set_cpus_allowed_ptr(writer_task, tmp_mask);
if (stats_task)
- set_cpus_allowed_ptr(stats_task, &tmp_mask);
+ set_cpus_allowed_ptr(stats_task, tmp_mask);
if (rcu_idle_cpu == -1)
rcu_idle_cpu = num_online_cpus() - 1;
else
rcu_idle_cpu--;
+out:
put_online_cpus();
+ free_cpumask_var(tmp_mask);
}
/* Shuffle tasks across CPUs, with the intent of allowing each CPU in the
--- linux-2.6-for-ingo.orig/kernel/softlockup.c
+++ linux-2.6-for-ingo/kernel/softlockup.c
@@ -310,10 +310,8 @@ cpu_callback(struct notifier_block *nfb,
case CPU_DOWN_PREPARE:
case CPU_DOWN_PREPARE_FROZEN:
if (hotcpu == check_cpu) {
- cpumask_t temp_cpu_online_map = cpu_online_map;
-
- cpu_clear(hotcpu, temp_cpu_online_map);
- check_cpu = any_online_cpu(temp_cpu_online_map);
+ /* Pick any other online cpu. */
+ check_cpu = cpumask_any_but(cpu_online_mask, hotcpu);
}
break;
--- linux-2.6-for-ingo.orig/kernel/stop_machine.c
+++ linux-2.6-for-ingo/kernel/stop_machine.c
@@ -69,10 +69,10 @@ static void stop_cpu(struct work_struct
int err;
if (!active_cpus) {
- if (cpu == first_cpu(cpu_online_map))
+ if (cpu == cpumask_first(cpu_online_mask))
smdata = &active;
} else {
- if (cpu_isset(cpu, *active_cpus))
+ if (cpumask_test_cpu(cpu, active_cpus))
smdata = &active;
}
/* Simple state machine */
@@ -109,7 +109,7 @@ static int chill(void *unused)
return 0;
}
-int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
+int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
{
struct work_struct *sm_work;
int i, ret;
@@ -142,7 +142,7 @@ int __stop_machine(int (*fn)(void *), vo
return ret;
}
-int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
+int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
{
int ret;
--- linux-2.6-for-ingo.orig/kernel/taskstats.c
+++ linux-2.6-for-ingo/kernel/taskstats.c
@@ -290,18 +290,17 @@ ret:
return;
}
-static int add_del_listener(pid_t pid, cpumask_t *maskp, int isadd)
+static int add_del_listener(pid_t pid, const struct cpumask *mask, int isadd)
{
struct listener_list *listeners;
struct listener *s, *tmp;
unsigned int cpu;
- cpumask_t mask = *maskp;
- if (!cpus_subset(mask, cpu_possible_map))
+ if (!cpumask_subset(mask, cpu_possible_mask))
return -EINVAL;
if (isadd == REGISTER) {
- for_each_cpu_mask_nr(cpu, mask) {
+ for_each_cpu(cpu, mask) {
s = kmalloc_node(sizeof(struct listener), GFP_KERNEL,
cpu_to_node(cpu));
if (!s)
@@ -320,7 +319,7 @@ static int add_del_listener(pid_t pid, c
/* Deregister or cleanup */
cleanup:
- for_each_cpu_mask_nr(cpu, mask) {
+ for_each_cpu(cpu, mask) {
listeners = &per_cpu(listener_array, cpu);
down_write(&listeners->sem);
list_for_each_entry_safe(s, tmp, &listeners->list, list) {
@@ -335,7 +334,7 @@ cleanup:
return 0;
}
-static int parse(struct nlattr *na, cpumask_t *mask)
+static int parse(struct nlattr *na, struct cpumask *mask)
{
char *data;
int len;
@@ -428,23 +427,33 @@ err:
static int taskstats_user_cmd(struct sk_buff *skb, struct genl_info *info)
{
- int rc = 0;
+ int rc;
struct sk_buff *rep_skb;
struct taskstats *stats;
size_t size;
- cpumask_t mask;
+ cpumask_var_t mask;
+
+ if (!alloc_cpumask_var(&mask, GFP_KERNEL))
+ return -ENOMEM;
- rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], &mask);
+ rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], mask);
if (rc < 0)
- return rc;
- if (rc == 0)
- return add_del_listener(info->snd_pid, &mask, REGISTER);
+ goto free_return_rc;
+ if (rc == 0) {
+ rc = add_del_listener(info->snd_pid, mask, REGISTER);
+ goto free_return_rc;
+ }
- rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], &mask);
+ rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], mask);
if (rc < 0)
+ goto free_return_rc;
+ if (rc == 0) {
+ rc = add_del_listener(info->snd_pid, mask, DEREGISTER);
+free_return_rc:
+ free_cpumask_var(mask);
return rc;
- if (rc == 0)
- return add_del_listener(info->snd_pid, &mask, DEREGISTER);
+ }
+ free_cpumask_var(mask);
/*
* Size includes space for nested attributes
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 6/8] cpumask: convert kernel mm functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (4 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-19 18:08 ` Christoph Lameter
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
2008-12-19 16:01 ` [PATCH 8/8] cpumask: convert other misc kernel functions Mike Travis
7 siblings, 1 reply; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: Rusty Russell, linux-kernel, Christoph Lameter, Pekka Enberg,
Matt Mackall
[-- Attachment #1: cpumask:convert-mm.patch --]
[-- Type: text/plain, Size: 5341 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Convert kernel mm functions to use struct cpumask.
We skip include/linux/percpu.h and mm/allocpercpu.c, which are in flux.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Matt Mackall <mpm@selenic.com>
---
mm/pdflush.c | 16 +++++++++++++---
mm/slab.c | 2 +-
mm/slub.c | 20 +++++++++++---------
mm/vmscan.c | 2 +-
mm/vmstat.c | 4 ++--
5 files changed, 28 insertions(+), 16 deletions(-)
diff --git a/mm/pdflush.c b/mm/pdflush.c
--- a/mm/pdflush.c
+++ b/mm/pdflush.c
@@ -172,7 +172,16 @@ static int pdflush(void *dummy)
static int pdflush(void *dummy)
{
struct pdflush_work my_work;
- cpumask_t cpus_allowed;
+ cpumask_var_t cpus_allowed;
+
+ /*
+ * Since the caller doesn't even check kthread_run() worked, let's not
+ * freak out too much if this fails.
+ */
+ if (!alloc_cpumask_var(&cpus_allowed, GFP_KERNEL)) {
+ printk(KERN_WARNING "pdflush failed to allocate cpumask\n");
+ return 0;
+ }
/*
* pdflush can spend a lot of time doing encryption via dm-crypt. We
@@ -187,8 +196,9 @@ static int pdflush(void *dummy)
* This is needed as pdflush's are dynamically created and destroyed.
* The boottime pdflush's are easily placed w/o these 2 lines.
*/
- cpuset_cpus_allowed(current, &cpus_allowed);
- set_cpus_allowed_ptr(current, &cpus_allowed);
+ cpuset_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, cpus_allowed);
+ free_cpumask_var(cpus_allowed);
return __pdflush(&my_work);
}
diff --git a/mm/slab.c b/mm/slab.c
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -2155,7 +2155,7 @@ kmem_cache_create (const char *name, siz
/*
* We use cache_chain_mutex to ensure a consistent view of
- * cpu_online_map as well. Please see cpuup_callback
+ * cpu_online_mask as well. Please see cpuup_callback
*/
get_online_cpus();
mutex_lock(&cache_chain_mutex);
diff --git a/mm/slub.c b/mm/slub.c
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1960,7 +1960,7 @@ static DEFINE_PER_CPU(struct kmem_cache_
kmem_cache_cpu)[NR_KMEM_CACHE_CPU];
static DEFINE_PER_CPU(struct kmem_cache_cpu *, kmem_cache_cpu_free);
-static cpumask_t kmem_cach_cpu_free_init_once = CPU_MASK_NONE;
+static DECLARE_BITMAP(kmem_cach_cpu_free_init_once, CONFIG_NR_CPUS);
static struct kmem_cache_cpu *alloc_kmem_cache_cpu(struct kmem_cache *s,
int cpu, gfp_t flags)
@@ -2035,13 +2035,13 @@ static void init_alloc_cpu_cpu(int cpu)
{
int i;
- if (cpu_isset(cpu, kmem_cach_cpu_free_init_once))
+ if (cpumask_test_cpu(cpu, to_cpumask(kmem_cach_cpu_free_init_once)))
return;
for (i = NR_KMEM_CACHE_CPU - 1; i >= 0; i--)
free_kmem_cache_cpu(&per_cpu(kmem_cache_cpu, cpu)[i], cpu);
- cpu_set(cpu, kmem_cach_cpu_free_init_once);
+ cpumask_set_cpu(cpu, to_cpumask(kmem_cach_cpu_free_init_once));
}
static void __init init_alloc_cpu(void)
@@ -3433,7 +3433,7 @@ struct location {
long max_time;
long min_pid;
long max_pid;
- cpumask_t cpus;
+ DECLARE_BITMAP(cpus, NR_CPUS);
nodemask_t nodes;
};
@@ -3508,7 +3508,8 @@ static int add_location(struct loc_track
if (track->pid > l->max_pid)
l->max_pid = track->pid;
- cpu_set(track->cpu, l->cpus);
+ cpumask_set_cpu(track->cpu,
+ to_cpumask(l->cpus));
}
node_set(page_to_nid(virt_to_page(track)), l->nodes);
return 1;
@@ -3538,8 +3539,8 @@ static int add_location(struct loc_track
l->max_time = age;
l->min_pid = track->pid;
l->max_pid = track->pid;
- cpus_clear(l->cpus);
- cpu_set(track->cpu, l->cpus);
+ cpumask_clear(to_cpumask(l->cpus));
+ cpumask_set_cpu(track->cpu, to_cpumask(l->cpus));
nodes_clear(l->nodes);
node_set(page_to_nid(virt_to_page(track)), l->nodes);
return 1;
@@ -3620,11 +3621,12 @@ static int list_locations(struct kmem_ca
len += sprintf(buf + len, " pid=%ld",
l->min_pid);
- if (num_online_cpus() > 1 && !cpus_empty(l->cpus) &&
+ if (num_online_cpus() > 1 &&
+ !cpumask_empty(to_cpumask(l->cpus)) &&
len < PAGE_SIZE - 60) {
len += sprintf(buf + len, " cpus=");
len += cpulist_scnprintf(buf + len, PAGE_SIZE - len - 50,
- &l->cpus);
+ to_cpumask(l->cpus));
}
if (num_online_nodes() > 1 && !nodes_empty(l->nodes) &&
diff --git a/mm/vmscan.c b/mm/vmscan.c
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1902,7 +1902,7 @@ static int kswapd(void *p)
};
node_to_cpumask_ptr(cpumask, pgdat->node_id);
- if (!cpus_empty(*cpumask))
+ if (!cpumask_empty(cpumask))
set_cpus_allowed_ptr(tsk, cpumask);
current->reclaim_state = &reclaim_state;
diff --git a/mm/vmstat.c b/mm/vmstat.c
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -20,7 +20,7 @@ DEFINE_PER_CPU(struct vm_event_state, vm
DEFINE_PER_CPU(struct vm_event_state, vm_event_states) = {{0}};
EXPORT_PER_CPU_SYMBOL(vm_event_states);
-static void sum_vm_events(unsigned long *ret, cpumask_t *cpumask)
+static void sum_vm_events(unsigned long *ret, const struct cpumask *cpumask)
{
int cpu;
int i;
@@ -43,7 +43,7 @@ void all_vm_events(unsigned long *ret)
void all_vm_events(unsigned long *ret)
{
get_online_cpus();
- sum_vm_events(ret, &cpu_online_map);
+ sum_vm_events(ret, cpu_online_mask);
put_online_cpus();
}
EXPORT_SYMBOL_GPL(all_vm_events);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 7/8] cpumask: convert misc driver functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (5 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
2008-12-20 23:22 ` Ben Hutchings
2008-12-29 15:37 ` Dean Nelson
2008-12-19 16:01 ` [PATCH 8/8] cpumask: convert other misc kernel functions Mike Travis
7 siblings, 2 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar
Cc: Rusty Russell, linux-kernel, Ben Hutchings, Dean Nelson,
Jeremy Fitzhardinge, Robert Richter
[-- Attachment #1: cpumask:convert-drivers.patch --]
[-- Type: text/plain, Size: 9473 bytes --]
Impact: Reduce stack usage, use new cpumask API.
Convert misc driver functions to use struct cpumask.
To Do:
- Convert iucv_buffer_cpumask to cpumask_var_t.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Cc: Dean Nelson <dcn@sgi.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Robert Richter <robert.richter@amd.com>
---
drivers/base/cpu.c | 2 +-
drivers/misc/sgi-xp/xpc_main.c | 2 +-
drivers/net/sfc/efx.c | 29 +++++++++++++++++------------
drivers/oprofile/buffer_sync.c | 22 ++++++++++++++++++----
drivers/oprofile/buffer_sync.h | 4 ++++
drivers/oprofile/oprof.c | 9 ++++++++-
drivers/xen/manage.c | 2 +-
net/iucv/iucv.c | 28 ++++++++++++++++++----------
8 files changed, 68 insertions(+), 30 deletions(-)
--- linux-2.6-for-ingo.orig/drivers/base/cpu.c
+++ linux-2.6-for-ingo/drivers/base/cpu.c
@@ -107,7 +107,7 @@ static SYSDEV_ATTR(crash_notes, 0400, sh
/*
* Print cpu online, possible, present, and system maps
*/
-static ssize_t print_cpus_map(char *buf, cpumask_t *map)
+static ssize_t print_cpus_map(char *buf, const struct cpumask *map)
{
int n = cpulist_scnprintf(buf, PAGE_SIZE-2, map);
--- linux-2.6-for-ingo.orig/drivers/misc/sgi-xp/xpc_main.c
+++ linux-2.6-for-ingo/drivers/misc/sgi-xp/xpc_main.c
@@ -318,7 +318,7 @@ xpc_hb_checker(void *ignore)
/* this thread was marked active by xpc_hb_init() */
- set_cpus_allowed_ptr(current, &cpumask_of_cpu(XPC_HB_CHECK_CPU));
+ set_cpus_allowed_ptr(current, cpumask_of(XPC_HB_CHECK_CPU));
/* set our heartbeating to other partitions into motion */
xpc_hb_check_timeout = jiffies + (xpc_hb_check_interval * HZ);
--- linux-2.6-for-ingo.orig/drivers/net/sfc/efx.c
+++ linux-2.6-for-ingo/drivers/net/sfc/efx.c
@@ -809,19 +809,18 @@ static void efx_fini_io(struct efx_nic *
/* Get number of RX queues wanted. Return number of online CPU
* packages in the expectation that an IRQ balancer will spread
* interrupts across them. */
-static int efx_wanted_rx_queues(void)
+static int efx_wanted_rx_queues(struct cpumask *scratch)
{
- cpumask_t core_mask;
int count;
int cpu;
- cpus_clear(core_mask);
+ cpumask_clear(scratch);
count = 0;
for_each_online_cpu(cpu) {
- if (!cpu_isset(cpu, core_mask)) {
+ if (!cpumask_test_cpu(cpu, scratch)) {
++count;
- cpus_or(core_mask, core_mask,
- topology_core_siblings(cpu));
+ cpumask_or(scratch, scratch,
+ topology_core_cpumask(cpu));
}
}
@@ -831,7 +830,7 @@ static int efx_wanted_rx_queues(void)
/* Probe the number and type of interrupts we are able to obtain, and
* the resulting numbers of channels and RX queues.
*/
-static void efx_probe_interrupts(struct efx_nic *efx)
+static void efx_probe_interrupts(struct efx_nic *efx, struct cpumask *scratch)
{
int max_channels =
min_t(int, efx->type->phys_addr_channels, EFX_MAX_CHANNELS);
@@ -845,7 +844,8 @@ static void efx_probe_interrupts(struct
* (or as specified by the rss_cpus module parameter).
* We will need one channel per interrupt.
*/
- wanted_ints = rss_cpus ? rss_cpus : efx_wanted_rx_queues();
+ wanted_ints = rss_cpus ? rss_cpus :
+ efx_wanted_rx_queues(scratch);
efx->n_rx_queues = min(wanted_ints, max_channels);
for (i = 0; i < efx->n_rx_queues; i++)
@@ -923,24 +923,29 @@ static void efx_set_channels(struct efx_
static int efx_probe_nic(struct efx_nic *efx)
{
int rc;
+ cpumask_var_t scratch;
EFX_LOG(efx, "creating NIC\n");
+ if (!alloc_cpumask_var(&scratch, GFP_KERNEL))
+ return -ENOMEM;
+
/* Carry out hardware-type specific initialisation */
rc = falcon_probe_nic(efx);
if (rc)
- return rc;
+ goto out;
/* Determine the number of channels and RX queues by trying to hook
* in MSI-X interrupts. */
- efx_probe_interrupts(efx);
+ efx_probe_interrupts(efx, scratch);
efx_set_channels(efx);
/* Initialise the interrupt moderation settings */
efx_init_irq_moderation(efx, tx_irq_mod_usec, rx_irq_mod_usec);
-
- return 0;
+out:
+ free_cpumask_var(scratch);
+ return rc;
}
static void efx_remove_nic(struct efx_nic *efx)
--- linux-2.6-for-ingo.orig/drivers/oprofile/buffer_sync.c
+++ linux-2.6-for-ingo/drivers/oprofile/buffer_sync.c
@@ -37,7 +37,7 @@
static LIST_HEAD(dying_tasks);
static LIST_HEAD(dead_tasks);
-static cpumask_t marked_cpus = CPU_MASK_NONE;
+static cpumask_var_t marked_cpus;
static DEFINE_SPINLOCK(task_mortuary);
static void process_task_mortuary(void);
@@ -524,10 +524,10 @@ static void mark_done(int cpu)
{
int i;
- cpu_set(cpu, marked_cpus);
+ cpumask_set_cpu(cpu, marked_cpus);
for_each_online_cpu(i) {
- if (!cpu_isset(i, marked_cpus))
+ if (!cpumask_test_cpu(i, marked_cpus))
return;
}
@@ -536,7 +536,7 @@ static void mark_done(int cpu)
*/
process_task_mortuary();
- cpus_clear(marked_cpus);
+ cpumask_clear(marked_cpus);
}
@@ -632,6 +632,20 @@ void sync_buffer(int cpu)
mutex_unlock(&buffer_mutex);
}
+int __init buffer_sync_init(void)
+{
+ if (!alloc_cpumask_var(&marked_cpus, GFP_KERNEL))
+ return -ENOMEM;
+
+ cpumask_copy(marked_cpus, cpu_none_mask);
+ return 0;
+}
+
+void __exit buffer_sync_cleanup(void)
+{
+ free_cpumask_var(marked_cpus);
+}
+
/* The function can be used to add a buffer worth of data directly to
* the kernel buffer. The buffer is assumed to be a circular buffer.
* Take the entries from index start and end at index end, wrapping
--- linux-2.6-for-ingo.orig/drivers/oprofile/buffer_sync.h
+++ linux-2.6-for-ingo/drivers/oprofile/buffer_sync.h
@@ -19,4 +19,8 @@ void sync_stop(void);
/* sync the given CPU's buffer */
void sync_buffer(int cpu);
+/* initialize/destroy the buffer system. */
+int buffer_sync_init(void);
+void buffer_sync_cleanup(void);
+
#endif /* OPROFILE_BUFFER_SYNC_H */
--- linux-2.6-for-ingo.orig/drivers/oprofile/oprof.c
+++ linux-2.6-for-ingo/drivers/oprofile/oprof.c
@@ -183,6 +183,10 @@ static int __init oprofile_init(void)
{
int err;
+ err = buffer_sync_init();
+ if (err)
+ return err;
+
err = oprofile_arch_init(&oprofile_ops);
if (err < 0 || timer) {
@@ -191,8 +195,10 @@ static int __init oprofile_init(void)
}
err = oprofilefs_register();
- if (err)
+ if (err) {
oprofile_arch_exit();
+ buffer_sync_cleanup();
+ }
return err;
}
@@ -202,6 +208,7 @@ static void __exit oprofile_exit(void)
{
oprofilefs_unregister();
oprofile_arch_exit();
+ buffer_sync_cleanup();
}
--- linux-2.6-for-ingo.orig/drivers/xen/manage.c
+++ linux-2.6-for-ingo/drivers/xen/manage.c
@@ -100,7 +100,7 @@ static void do_suspend(void)
/* XXX use normal device tree? */
xenbus_suspend();
- err = stop_machine(xen_suspend, &cancelled, &cpumask_of_cpu(0));
+ err = stop_machine(xen_suspend, &cancelled, cpumask_of(0));
if (err) {
printk(KERN_ERR "failed to start xen_suspend: %d\n", err);
goto out;
--- linux-2.6-for-ingo.orig/net/iucv/iucv.c
+++ linux-2.6-for-ingo/net/iucv/iucv.c
@@ -489,15 +489,14 @@ static void iucv_setmask_mp(void)
*
* Allow iucv interrupts on a single cpu.
*/
-static void iucv_setmask_up(void)
+static void iucv_setmask_up(struct cpumask *cpumask)
{
- cpumask_t cpumask;
int cpu;
/* Disable all cpu but the first in cpu_irq_cpumask. */
- cpumask = iucv_irq_cpumask;
- cpu_clear(first_cpu(iucv_irq_cpumask), cpumask);
- for_each_cpu_mask_nr(cpu, cpumask)
+ cpumask_copy(cpumask, iucv_irq_cpumask);
+ cpumask_clear_cpu(cpumask_first(iucv_irq_cpumask), cpumask);
+ for_each_cpu(cpu, cpumask)
smp_call_function_single(cpu, iucv_block_cpu, NULL, 1);
}
@@ -555,7 +554,7 @@ static void iucv_disable(void)
static int __cpuinit iucv_cpu_notify(struct notifier_block *self,
unsigned long action, void *hcpu)
{
- cpumask_t cpumask;
+ cpumask_var_t cpumask;
long cpu = (long) hcpu;
switch (action) {
@@ -590,15 +589,20 @@ static int __cpuinit iucv_cpu_notify(str
break;
case CPU_DOWN_PREPARE:
case CPU_DOWN_PREPARE_FROZEN:
- cpumask = iucv_buffer_cpumask;
- cpu_clear(cpu, cpumask);
- if (cpus_empty(cpumask))
+ if (!alloc_cpumask_var(&cpumask, GFP_KERNEL))
+ return NOTIFY_BAD;
+ cpumask_copy(cpumask, &iucv_buffer_cpumask);
+ cpumask_clear_cpu(cpu, cpumask);
+ if (cpumask_empty(cpumask)) {
/* Can't offline last IUCV enabled cpu. */
+ free_cpumask_var(cpumask);
return NOTIFY_BAD;
+ }
smp_call_function_single(cpu, iucv_retrieve_cpu, NULL, 1);
if (cpus_empty(iucv_irq_cpumask))
smp_call_function_single(first_cpu(iucv_buffer_cpumask),
iucv_allow_cpu, NULL, 1);
+ free_cpumask_var(cpumask);
break;
}
return NOTIFY_OK;
@@ -683,9 +687,12 @@ static void iucv_cleanup_queue(void)
int iucv_register(struct iucv_handler *handler, int smp)
{
int rc;
+ cpumask_var_t scratch;
if (!iucv_available)
return -ENOSYS;
+ if (!alloc_cpumask_var(&scratch, GFP_KERNEL))
+ return -ENOMEM;
mutex_lock(&iucv_register_mutex);
if (!smp)
iucv_nonsmp_handler++;
@@ -694,7 +701,7 @@ int iucv_register(struct iucv_handler *h
if (rc)
goto out_mutex;
} else if (!smp && iucv_nonsmp_handler == 1)
- iucv_setmask_up();
+ iucv_setmask_up(scratch);
INIT_LIST_HEAD(&handler->paths);
spin_lock_bh(&iucv_table_lock);
@@ -703,6 +710,7 @@ int iucv_register(struct iucv_handler *h
rc = 0;
out_mutex:
mutex_unlock(&iucv_register_mutex);
+ free_cpumask_var(scratch);
return rc;
}
EXPORT_SYMBOL(iucv_register);
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 8/8] cpumask: convert other misc kernel functions
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
` (6 preceding siblings ...)
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
@ 2008-12-19 16:01 ` Mike Travis
7 siblings, 0 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:01 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Rusty Russell, linux-kernel, Steven Rostedt
[-- Attachment #1: cpumask:convert-misc.patch --]
[-- Type: text/plain, Size: 717 bytes --]
Impact: Use new cpumask API.
Convert other misc kernel functions to use struct cpumask.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Cc: Steven Rostedt <srostedt@redhat.com>
---
lib/smp_processor_id.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- linux-2.6-for-ingo.orig/lib/smp_processor_id.c
+++ linux-2.6-for-ingo/lib/smp_processor_id.c
@@ -22,7 +22,7 @@ notrace unsigned int debug_smp_processor
* Kernel threads bound to a single CPU can safely use
* smp_processor_id():
*/
- if (cpus_equal(current->cpus_allowed, cpumask_of_cpu(this_cpu)))
+ if (cpumask_equal(¤t->cpus_allowed, cpumask_of(this_cpu)))
goto out;
/*
--
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
@ 2008-12-19 16:16 ` Steven Rostedt
2008-12-19 16:54 ` Mike Travis
0 siblings, 1 reply; 21+ messages in thread
From: Steven Rostedt @ 2008-12-19 16:16 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
On Fri, 19 Dec 2008, Mike Travis wrote:
> Impact: Reduce memory usage, use new cpumask API.
>
> Convert kernel trace functions to use struct cpumask.
>
[..]
> #define for_each_buffer_cpu(buffer, cpu) \
> - for_each_cpu_mask(cpu, buffer->cpumask)
> + for_each_cpu(cpu, buffer->cpumask)
>
> #define TS_SHIFT 27
> #define TS_MASK ((1ULL << TS_SHIFT) - 1)
> @@ -262,7 +262,7 @@ struct ring_buffer {
> unsigned pages;
> unsigned flags;
> int cpus;
> - cpumask_t cpumask;
> + cpumask_var_t cpumask;
> atomic_t record_disabled;
>
> struct mutex mutex;
> @@ -453,6 +453,9 @@ struct ring_buffer *ring_buffer_alloc(un
> if (!buffer)
> return NULL;
>
> + if (!alloc_cpumask_var(&buffer->cpumask, GFP_KERNEL))
> + goto fail_free_buffer;
> +
How does this save memory if we just allocate a cpumask var everytime
we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
not NR_CPUS?
Otherwise, I see this as just adding one extra pointer.
-- Steve
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:16 ` Steven Rostedt
@ 2008-12-19 16:54 ` Mike Travis
2008-12-19 17:01 ` Steven Rostedt
2008-12-20 1:41 ` Steven Rostedt
0 siblings, 2 replies; 21+ messages in thread
From: Mike Travis @ 2008-12-19 16:54 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
Steven Rostedt wrote:
> On Fri, 19 Dec 2008, Mike Travis wrote:
>
>> Impact: Reduce memory usage, use new cpumask API.
>>
>> Convert kernel trace functions to use struct cpumask.
>>
>
> [..]
>
>> #define for_each_buffer_cpu(buffer, cpu) \
>> - for_each_cpu_mask(cpu, buffer->cpumask)
>> + for_each_cpu(cpu, buffer->cpumask)
>>
>> #define TS_SHIFT 27
>> #define TS_MASK ((1ULL << TS_SHIFT) - 1)
>> @@ -262,7 +262,7 @@ struct ring_buffer {
>> unsigned pages;
>> unsigned flags;
>> int cpus;
>> - cpumask_t cpumask;
>> + cpumask_var_t cpumask;
>> atomic_t record_disabled;
>>
>> struct mutex mutex;
>> @@ -453,6 +453,9 @@ struct ring_buffer *ring_buffer_alloc(un
>> if (!buffer)
>> return NULL;
>>
>> + if (!alloc_cpumask_var(&buffer->cpumask, GFP_KERNEL))
>> + goto fail_free_buffer;
>> +
>
>
> How does this save memory if we just allocate a cpumask var everytime
> we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
> not NR_CPUS?
>
> Otherwise, I see this as just adding one extra pointer.
>
> -- Steve
Hi Steve,
Yes, eventually, the cpumask_var_t will be allocated based on cpumask_size()
which will become BITS_TO_LONG(nr_cpu_ids) instead of BITS_TO_LONGS(NR_CPUS)
as soon as the kernel becomes "cpumask" clean. (clean being that it ignores
all bits >= nr_cpu_ids.)
Note that on small NR_CPUS count systems, cpumask_var_t is a static array
and no allocation happens, so it only kicks in when CONFIG_CPUMASK_OFFSTACK=y.
Thanks,
Mike
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:54 ` Mike Travis
@ 2008-12-19 17:01 ` Steven Rostedt
2008-12-20 1:41 ` Steven Rostedt
1 sibling, 0 replies; 21+ messages in thread
From: Steven Rostedt @ 2008-12-19 17:01 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
On Fri, 19 Dec 2008, Mike Travis wrote:
> >
> >
> > How does this save memory if we just allocate a cpumask var everytime
> > we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
> > not NR_CPUS?
> >
> > Otherwise, I see this as just adding one extra pointer.
> >
> > -- Steve
>
> Hi Steve,
>
> Yes, eventually, the cpumask_var_t will be allocated based on cpumask_size()
> which will become BITS_TO_LONG(nr_cpu_ids) instead of BITS_TO_LONGS(NR_CPUS)
> as soon as the kernel becomes "cpumask" clean. (clean being that it ignores
> all bits >= nr_cpu_ids.)
>
> Note that on small NR_CPUS count systems, cpumask_var_t is a static array
> and no allocation happens, so it only kicks in when CONFIG_CPUMASK_OFFSTACK=y.
OK, that explains a lot. Thanks for the explanation.
-- Steve
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 6/8] cpumask: convert kernel mm functions
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
@ 2008-12-19 18:08 ` Christoph Lameter
0 siblings, 0 replies; 21+ messages in thread
From: Christoph Lameter @ 2008-12-19 18:08 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Pekka Enberg,
Matt Mackall
Reviewed-by: Christoph Lameter <cl@linux-foundation.org>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5/8] cpumask: convert rest of files in kernel/
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
@ 2008-12-20 1:33 ` Lai Jiangshan
2008-12-20 13:02 ` Rusty Russell
1 sibling, 0 replies; 21+ messages in thread
From: Lai Jiangshan @ 2008-12-20 1:33 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Dipankar Sarma,
H. Peter Anvin, Max Krasnyansky, Richard Henderson,
Thomas Gleixner, Yinghai Lu, Paul E. McKenney
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API. ALPHA mod!
>
> A couple of uses of cpumask_any_but() here to avoid temporary cpumasks
> and carry the "const" modifier through the seq_cpumask() function all the
> way to bitmap_scnprintf().
>
> Note that prof_cpu_mask will be difficult to convert to a cpumask_var_t
> since it needs to be ready from the first scheduler tick.
>
> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Mike Travis <travis@sgi.com>
> Cc: Dipankar Sarma <dipankar@in.ibm.com>
> Cc: H. Peter Anvin <hpa@zytor.com>
> Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
> Cc: Max Krasnyansky <maxk@qualcomm.com>
> Cc: Richard Henderson <rth@twiddle.net>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Yinghai Lu <yinghai@kernel.org>
> ---
> arch/alpha/kernel/irq.c | 3 ++-
> fs/seq_file.c | 3 ++-
> include/linux/interrupt.h | 2 +-
> include/linux/rcuclassic.h | 4 ++--
> include/linux/seq_file.h | 5 +++--
> include/linux/stop_machine.h | 6 +++---
> kernel/cpu.c | 40 +++++++++++++++++++++-------------------
> kernel/irq/manage.c | 11 +++++++++--
> kernel/irq/proc.c | 32 +++++++++++++++++++++-----------
> kernel/power/poweroff.c | 2 +-
> kernel/profile.c | 31 ++++++++++++++++++-------------
> kernel/rcuclassic.c | 32 +++++++++++++++++---------------
> kernel/rcupreempt.c | 19 ++++++++++---------
> kernel/rcutorture.c | 27 +++++++++++++++------------
> kernel/softlockup.c | 6 ++----
> kernel/stop_machine.c | 8 ++++----
> kernel/taskstats.c | 39 ++++++++++++++++++++++++---------------
> 17 files changed, 155 insertions(+), 115 deletions(-)
>
> --- linux-2.6-for-ingo.orig/arch/alpha/kernel/irq.c
> +++ linux-2.6-for-ingo/arch/alpha/kernel/irq.c
> @@ -50,7 +50,8 @@ int irq_select_affinity(unsigned int irq
> if (!irq_desc[irq].chip->set_affinity || irq_user_affinity[irq])
> return 1;
>
> - while (!cpu_possible(cpu) || !cpu_isset(cpu, irq_default_affinity))
> + while (!cpu_possible(cpu) ||
> + !cpumask_test_cpu(cpu, irq_default_affinity))
> cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
> last_cpu = cpu;
>
> --- linux-2.6-for-ingo.orig/fs/seq_file.c
> +++ linux-2.6-for-ingo/fs/seq_file.c
> @@ -462,7 +462,8 @@ int seq_dentry(struct seq_file *m, struc
> return -1;
> }
>
> -int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits)
> +int seq_bitmap(struct seq_file *m, const unsigned long *bits,
> + unsigned int nr_bits)
> {
> if (m->count < m->size) {
> int len = bitmap_scnprintf(m->buf + m->count,
> --- linux-2.6-for-ingo.orig/include/linux/interrupt.h
> +++ linux-2.6-for-ingo/include/linux/interrupt.h
> @@ -109,7 +109,7 @@ extern void enable_irq(unsigned int irq)
>
> #if defined(CONFIG_SMP) && defined(CONFIG_GENERIC_HARDIRQS)
>
> -extern cpumask_t irq_default_affinity;
> +extern cpumask_var_t irq_default_affinity;
>
> extern int irq_set_affinity(unsigned int irq, const struct cpumask *cpumask);
> extern int irq_can_set_affinity(unsigned int irq);
> --- linux-2.6-for-ingo.orig/include/linux/rcuclassic.h
> +++ linux-2.6-for-ingo/include/linux/rcuclassic.h
> @@ -59,8 +59,8 @@ struct rcu_ctrlblk {
> int signaled;
>
> spinlock_t lock ____cacheline_internodealigned_in_smp;
> - cpumask_t cpumask; /* CPUs that need to switch in order */
> - /* for current batch to proceed. */
> + DECLARE_BITMAP(cpumask, NR_CPUS); /* CPUs that need to switch for */
> + /* current batch to proceed. */
> } ____cacheline_internodealigned_in_smp;
>
> /* Is batch a before batch b ? */
> --- linux-2.6-for-ingo.orig/include/linux/seq_file.h
> +++ linux-2.6-for-ingo/include/linux/seq_file.h
> @@ -50,8 +50,9 @@ int seq_path(struct seq_file *, struct p
> int seq_dentry(struct seq_file *, struct dentry *, char *);
> int seq_path_root(struct seq_file *m, struct path *path, struct path *root,
> char *esc);
> -int seq_bitmap(struct seq_file *m, unsigned long *bits, unsigned int nr_bits);
> -static inline int seq_cpumask(struct seq_file *m, cpumask_t *mask)
> +int seq_bitmap(struct seq_file *m, const unsigned long *bits,
> + unsigned int nr_bits);
> +static inline int seq_cpumask(struct seq_file *m, const struct cpumask *mask)
> {
> return seq_bitmap(m, mask->bits, NR_CPUS);
> }
> --- linux-2.6-for-ingo.orig/include/linux/stop_machine.h
> +++ linux-2.6-for-ingo/include/linux/stop_machine.h
> @@ -23,7 +23,7 @@
> *
> * This can be thought of as a very heavy write lock, equivalent to
> * grabbing every spinlock in the kernel. */
> -int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
> +int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
>
> /**
> * __stop_machine: freeze the machine on all CPUs and run this function
> @@ -34,11 +34,11 @@ int stop_machine(int (*fn)(void *), void
> * Description: This is a special version of the above, which assumes cpus
> * won't come or go while it's being called. Used by hotplug cpu.
> */
> -int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus);
> +int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus);
> #else
>
> static inline int stop_machine(int (*fn)(void *), void *data,
> - const cpumask_t *cpus)
> + const struct cpumask *cpus)
> {
> int ret;
> local_irq_disable();
> --- linux-2.6-for-ingo.orig/kernel/cpu.c
> +++ linux-2.6-for-ingo/kernel/cpu.c
> @@ -218,7 +218,7 @@ static int __ref take_cpu_down(void *_pa
> static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
> {
> int err, nr_calls = 0;
> - cpumask_t old_allowed, tmp;
> + cpumask_var_t old_allowed;
> void *hcpu = (void *)(long)cpu;
> unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
> struct take_cpu_down_param tcd_param = {
> @@ -232,6 +232,9 @@ static int __ref _cpu_down(unsigned int
> if (!cpu_online(cpu))
> return -EINVAL;
>
> + if (!alloc_cpumask_var(&old_allowed, GFP_KERNEL))
> + return -ENOMEM;
> +
> cpu_hotplug_begin();
> err = __raw_notifier_call_chain(&cpu_chain, CPU_DOWN_PREPARE | mod,
> hcpu, -1, &nr_calls);
> @@ -246,13 +249,11 @@ static int __ref _cpu_down(unsigned int
> }
>
> /* Ensure that we are not runnable on dying cpu */
> - old_allowed = current->cpus_allowed;
> - cpus_setall(tmp);
> - cpu_clear(cpu, tmp);
> - set_cpus_allowed_ptr(current, &tmp);
> - tmp = cpumask_of_cpu(cpu);
> + cpumask_copy(old_allowed, ¤t->cpus_allowed);
> + set_cpus_allowed_ptr(current,
> + cpumask_of(cpumask_any_but(cpu_online_mask, cpu)));
>
> - err = __stop_machine(take_cpu_down, &tcd_param, &tmp);
> + err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
> if (err) {
> /* CPU didn't die: tell everyone. Can't complain. */
> if (raw_notifier_call_chain(&cpu_chain, CPU_DOWN_FAILED | mod,
> @@ -278,7 +279,7 @@ static int __ref _cpu_down(unsigned int
> check_for_tasks(cpu);
>
> out_allowed:
> - set_cpus_allowed_ptr(current, &old_allowed);
> + set_cpus_allowed_ptr(current, old_allowed);
> out_release:
> cpu_hotplug_done();
> if (!err) {
> @@ -286,6 +287,7 @@ out_release:
> hcpu) == NOTIFY_BAD)
> BUG();
> }
> + free_cpumask_var(old_allowed);
> return err;
> }
>
> @@ -304,7 +306,7 @@ int __ref cpu_down(unsigned int cpu)
>
> /*
> * Make sure the all cpus did the reschedule and are not
> - * using stale version of the cpu_active_map.
> + * using stale version of the cpu_active_mask.
> * This is not strictly necessary becuase stop_machine()
> * that we run down the line already provides the required
> * synchronization. But it's really a side effect and we do not
> @@ -368,7 +370,7 @@ out_notify:
> int __cpuinit cpu_up(unsigned int cpu)
> {
> int err = 0;
> - if (!cpu_isset(cpu, cpu_possible_map)) {
> + if (!cpu_possible(cpu)) {
> printk(KERN_ERR "can't online cpu %d because it is not "
> "configured as may-hotadd at boot time\n", cpu);
> #if defined(CONFIG_IA64) || defined(CONFIG_X86_64)
> @@ -393,25 +395,25 @@ out:
> }
>
> #ifdef CONFIG_PM_SLEEP_SMP
> -static cpumask_t frozen_cpus;
> +static DECLARE_BITMAP(frozen_cpus, CONFIG_NR_CPUS);
>
> int disable_nonboot_cpus(void)
> {
> int cpu, first_cpu, error = 0;
>
> cpu_maps_update_begin();
> - first_cpu = first_cpu(cpu_online_map);
> + first_cpu = cpumask_first(cpu_online_mask);
> /* We take down all of the non-boot CPUs in one shot to avoid races
> * with the userspace trying to use the CPU hotplug at the same time
> */
> - cpus_clear(frozen_cpus);
> + cpumask_clear(to_cpumask(frozen_cpus));
> printk("Disabling non-boot CPUs ...\n");
> for_each_online_cpu(cpu) {
> if (cpu == first_cpu)
> continue;
> error = _cpu_down(cpu, 1);
> if (!error) {
> - cpu_set(cpu, frozen_cpus);
> + cpumask_set_cpu(cpu, to_cpumask(frozen_cpus));
> printk("CPU%d is down\n", cpu);
> } else {
> printk(KERN_ERR "Error taking CPU%d down: %d\n",
> @@ -437,11 +439,11 @@ void __ref enable_nonboot_cpus(void)
> /* Allow everyone to use the CPU hotplug again */
> cpu_maps_update_begin();
> cpu_hotplug_disabled = 0;
> - if (cpus_empty(frozen_cpus))
> + if (cpumask_empty(to_cpumask(frozen_cpus)))
> goto out;
>
> printk("Enabling non-boot CPUs ...\n");
> - for_each_cpu_mask_nr(cpu, frozen_cpus) {
> + for_each_cpu(cpu, to_cpumask(frozen_cpus)) {
> error = _cpu_up(cpu, 1);
> if (!error) {
> printk("CPU%d is up\n", cpu);
> @@ -449,7 +451,7 @@ void __ref enable_nonboot_cpus(void)
> }
> printk(KERN_WARNING "Error taking CPU%d up: %d\n", cpu, error);
> }
> - cpus_clear(frozen_cpus);
> + cpumask_clear(to_cpumask(frozen_cpus));
> out:
> cpu_maps_update_done();
> }
> @@ -468,7 +470,7 @@ void __cpuinit notify_cpu_starting(unsig
> unsigned long val = CPU_STARTING;
>
> #ifdef CONFIG_PM_SLEEP_SMP
> - if (cpu_isset(cpu, frozen_cpus))
> + if (cpumask_test_cpu(cpu, to_cpumask(frozen_cpus)))
> val = CPU_STARTING_FROZEN;
> #endif /* CONFIG_PM_SLEEP_SMP */
> raw_notifier_call_chain(&cpu_chain, val, (void *)(long)cpu);
> @@ -480,7 +482,7 @@ void __cpuinit notify_cpu_starting(unsig
> * cpu_bit_bitmap[] is a special, "compressed" data structure that
> * represents all NR_CPUS bits binary values of 1<<nr.
> *
> - * It is used by cpumask_of_cpu() to get a constant address to a CPU
> + * It is used by cpumask_of() to get a constant address to a CPU
> * mask value that has a single bit set only.
> */
>
> --- linux-2.6-for-ingo.orig/kernel/irq/manage.c
> +++ linux-2.6-for-ingo/kernel/irq/manage.c
> @@ -16,8 +16,15 @@
> #include "internals.h"
>
> #ifdef CONFIG_SMP
> +cpumask_var_t irq_default_affinity;
>
> -cpumask_t irq_default_affinity = CPU_MASK_ALL;
> +static int init_irq_default_affinity(void)
> +{
> + alloc_cpumask_var(&irq_default_affinity, GFP_KERNEL);
> + cpumask_setall(irq_default_affinity);
> + return 0;
> +}
> +core_initcall(init_irq_default_affinity);
>
> /**
> * synchronize_irq - wait for pending IRQ handlers (on other CPUs)
> @@ -127,7 +134,7 @@ int do_irq_select_affinity(unsigned int
> desc->status &= ~IRQ_AFFINITY_SET;
> }
>
> - cpumask_and(&desc->affinity, cpu_online_mask, &irq_default_affinity);
> + cpumask_and(&desc->affinity, cpu_online_mask, irq_default_affinity);
> set_affinity:
> desc->chip->set_affinity(irq, &desc->affinity);
>
> --- linux-2.6-for-ingo.orig/kernel/irq/proc.c
> +++ linux-2.6-for-ingo/kernel/irq/proc.c
> @@ -20,7 +20,7 @@ static struct proc_dir_entry *root_irq_d
> static int irq_affinity_proc_show(struct seq_file *m, void *v)
> {
> struct irq_desc *desc = irq_to_desc((long)m->private);
> - cpumask_t *mask = &desc->affinity;
> + const struct cpumask *mask = &desc->affinity;
>
> #ifdef CONFIG_GENERIC_PENDING_IRQ
> if (desc->status & IRQ_MOVE_PENDING)
> @@ -93,7 +93,7 @@ static const struct file_operations irq_
>
> static int default_affinity_show(struct seq_file *m, void *v)
> {
> - seq_cpumask(m, &irq_default_affinity);
> + seq_cpumask(m, irq_default_affinity);
> seq_putc(m, '\n');
> return 0;
> }
> @@ -101,27 +101,37 @@ static int default_affinity_show(struct
> static ssize_t default_affinity_write(struct file *file,
> const char __user *buffer, size_t count, loff_t *ppos)
> {
> - cpumask_t new_value;
> + cpumask_var_t new_value;
> int err;
>
> - err = cpumask_parse_user(buffer, count, &new_value);
> + if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
> + return -ENOMEM;
> +
> + err = cpumask_parse_user(buffer, count, new_value);
> if (err)
> - return err;
> + goto out;
>
> - if (!is_affinity_mask_valid(new_value))
> - return -EINVAL;
> + if (!is_affinity_mask_valid(new_value)) {
> + err = -EINVAL;
> + goto out;
> + }
>
> /*
> * Do not allow disabling IRQs completely - it's a too easy
> * way to make the system unusable accidentally :-) At least
> * one online CPU still has to be targeted.
> */
> - if (!cpus_intersects(new_value, cpu_online_map))
> - return -EINVAL;
> + if (!cpumask_intersects(new_value, cpu_online_mask)) {
> + err = -EINVAL;
> + goto out;
> + }
>
> - irq_default_affinity = new_value;
> + cpumask_copy(irq_default_affinity, new_value);
> + err = count;
>
> - return count;
> +out:
> + free_cpumask_var(new_value);
> + return err;
> }
>
> static int default_affinity_open(struct inode *inode, struct file *file)
> --- linux-2.6-for-ingo.orig/kernel/power/poweroff.c
> +++ linux-2.6-for-ingo/kernel/power/poweroff.c
> @@ -27,7 +27,7 @@ static DECLARE_WORK(poweroff_work, do_po
> static void handle_poweroff(int key, struct tty_struct *tty)
> {
> /* run sysrq poweroff on boot cpu */
> - schedule_work_on(first_cpu(cpu_online_map), &poweroff_work);
> + schedule_work_on(cpumask_first(cpu_online_mask), &poweroff_work);
> }
>
> static struct sysrq_key_op sysrq_poweroff_op = {
> --- linux-2.6-for-ingo.orig/kernel/profile.c
> +++ linux-2.6-for-ingo/kernel/profile.c
> @@ -45,7 +45,7 @@ static unsigned long prof_len, prof_shif
> int prof_on __read_mostly;
> EXPORT_SYMBOL_GPL(prof_on);
>
> -static cpumask_t prof_cpu_mask = CPU_MASK_ALL;
> +static DECLARE_BITMAP(prof_cpu_mask, CONFIG_NR_CPUS) = CPU_BITS_ALL;
> #ifdef CONFIG_SMP
> static DEFINE_PER_CPU(struct profile_hit *[2], cpu_profile_hits);
> static DEFINE_PER_CPU(int, cpu_profile_flip);
> @@ -386,13 +386,13 @@ out_free:
> return NOTIFY_BAD;
> case CPU_ONLINE:
> case CPU_ONLINE_FROZEN:
> - cpu_set(cpu, prof_cpu_mask);
> + cpumask_set_cpu(cpu, to_cpumask(prof_cpu_mask));
> break;
> case CPU_UP_CANCELED:
> case CPU_UP_CANCELED_FROZEN:
> case CPU_DEAD:
> case CPU_DEAD_FROZEN:
> - cpu_clear(cpu, prof_cpu_mask);
> + cpumask_clear_cpu(cpu, to_cpumask(prof_cpu_mask));
> if (per_cpu(cpu_profile_hits, cpu)[0]) {
> page = virt_to_page(per_cpu(cpu_profile_hits, cpu)[0]);
> per_cpu(cpu_profile_hits, cpu)[0] = NULL;
> @@ -430,7 +430,8 @@ void profile_tick(int type)
>
> if (type == CPU_PROFILING && timer_hook)
> timer_hook(regs);
> - if (!user_mode(regs) && cpu_isset(smp_processor_id(), prof_cpu_mask))
> + if (!user_mode(regs) &&
> + cpumask_test_cpu(smp_processor_id(), to_cpumask(prof_cpu_mask)))
> profile_hit(type, (void *)profile_pc(regs));
> }
>
> @@ -442,7 +443,7 @@ void profile_tick(int type)
> static int prof_cpu_mask_read_proc(char *page, char **start, off_t off,
> int count, int *eof, void *data)
> {
> - int len = cpumask_scnprintf(page, count, (cpumask_t *)data);
> + int len = cpumask_scnprintf(page, count, data);
> if (count - len < 2)
> return -EINVAL;
> len += sprintf(page + len, "\n");
> @@ -452,16 +453,20 @@ static int prof_cpu_mask_read_proc(char
> static int prof_cpu_mask_write_proc(struct file *file,
> const char __user *buffer, unsigned long count, void *data)
> {
> - cpumask_t *mask = (cpumask_t *)data;
> + struct cpumask *mask = data;
> unsigned long full_count = count, err;
> - cpumask_t new_value;
> + cpumask_var_t new_value;
>
> - err = cpumask_parse_user(buffer, count, &new_value);
> - if (err)
> - return err;
> + if (!alloc_cpumask_var(&new_value, GFP_KERNEL))
> + return -ENOMEM;
>
> - *mask = new_value;
> - return full_count;
> + err = cpumask_parse_user(buffer, count, new_value);
> + if (!err) {
> + cpumask_copy(mask, new_value);
> + err = full_count;
> + }
> + free_cpumask_var(new_value);
> + return err;
> }
>
> void create_prof_cpu_mask(struct proc_dir_entry *root_irq_dir)
> @@ -472,7 +477,7 @@ void create_prof_cpu_mask(struct proc_di
> entry = create_proc_entry("prof_cpu_mask", 0600, root_irq_dir);
> if (!entry)
> return;
> - entry->data = (void *)&prof_cpu_mask;
> + entry->data = prof_cpu_mask;
> entry->read_proc = prof_cpu_mask_read_proc;
> entry->write_proc = prof_cpu_mask_write_proc;
> }
> --- linux-2.6-for-ingo.orig/kernel/rcuclassic.c
> +++ linux-2.6-for-ingo/kernel/rcuclassic.c
> @@ -63,14 +63,14 @@ static struct rcu_ctrlblk rcu_ctrlblk =
> .completed = -300,
> .pending = -300,
> .lock = __SPIN_LOCK_UNLOCKED(&rcu_ctrlblk.lock),
> - .cpumask = CPU_MASK_NONE,
> + .cpumask = CPU_BITS_NONE,
> };
> static struct rcu_ctrlblk rcu_bh_ctrlblk = {
> .cur = -300,
> .completed = -300,
> .pending = -300,
> .lock = __SPIN_LOCK_UNLOCKED(&rcu_bh_ctrlblk.lock),
> - .cpumask = CPU_MASK_NONE,
> + .cpumask = CPU_BITS_NONE,
> };
>
> DEFINE_PER_CPU(struct rcu_data, rcu_data) = { 0L };
> @@ -85,7 +85,6 @@ static void force_quiescent_state(struct
> struct rcu_ctrlblk *rcp)
> {
> int cpu;
> - cpumask_t cpumask;
> unsigned long flags;
>
> set_need_resched();
> @@ -96,10 +95,10 @@ static void force_quiescent_state(struct
> * Don't send IPI to itself. With irqs disabled,
> * rdp->cpu is the current cpu.
> *
> - * cpu_online_map is updated by the _cpu_down()
> + * cpu_online_mask is updated by the _cpu_down()
> * using __stop_machine(). Since we're in irqs disabled
> * section, __stop_machine() is not exectuting, hence
> - * the cpu_online_map is stable.
> + * the cpu_online_mask is stable.
> *
> * However, a cpu might have been offlined _just_ before
> * we disabled irqs while entering here.
> @@ -107,13 +106,14 @@ static void force_quiescent_state(struct
> * notification, leading to the offlined cpu's bit
> * being set in the rcp->cpumask.
> *
> - * Hence cpumask = (rcp->cpumask & cpu_online_map) to prevent
> + * Hence cpumask = (rcp->cpumask & cpu_online_mask) to prevent
> * sending smp_reschedule() to an offlined CPU.
> */
> - cpus_and(cpumask, rcp->cpumask, cpu_online_map);
> - cpu_clear(rdp->cpu, cpumask);
> - for_each_cpu_mask_nr(cpu, cpumask)
> - smp_send_reschedule(cpu);
> + for_each_cpu_and(cpu,
> + to_cpumask(rcp->cpumask), cpu_online_mask) {
> + if (cpu != rdp->cpu)
> + smp_send_reschedule(cpu);
> + }
> }
> spin_unlock_irqrestore(&rcp->lock, flags);
> }
> @@ -193,7 +193,7 @@ static void print_other_cpu_stall(struct
>
> printk(KERN_ERR "INFO: RCU detected CPU stalls:");
> for_each_possible_cpu(cpu) {
> - if (cpu_isset(cpu, rcp->cpumask))
> + if (cpumask_test_cpu(cpu, to_cpumask(rcp->cpumask)))
> printk(" %d", cpu);
> }
> printk(" (detected by %d, t=%ld jiffies)\n",
> @@ -221,7 +221,8 @@ static void check_cpu_stall(struct rcu_c
> long delta;
>
> delta = jiffies - rcp->jiffies_stall;
> - if (cpu_isset(smp_processor_id(), rcp->cpumask) && delta >= 0) {
> + if (cpumask_test_cpu(smp_processor_id(), to_cpumask(rcp->cpumask)) &&
> + delta >= 0) {
>
> /* We haven't checked in, so go dump stack. */
> print_cpu_stall(rcp);
> @@ -393,7 +394,8 @@ static void rcu_start_batch(struct rcu_c
> * unnecessarily.
> */
> smp_mb();
> - cpumask_andnot(&rcp->cpumask, cpu_online_mask, nohz_cpu_mask);
> + cpumask_andnot(to_cpumask(rcp->cpumask),
> + cpu_online_mask, nohz_cpu_mask);
>
> rcp->signaled = 0;
> }
> @@ -406,8 +408,8 @@ static void rcu_start_batch(struct rcu_c
> */
> static void cpu_quiet(int cpu, struct rcu_ctrlblk *rcp)
> {
> - cpu_clear(cpu, rcp->cpumask);
> - if (cpus_empty(rcp->cpumask)) {
> + cpumask_clear_cpu(cpu, to_cpumask(rcp->cpumask));
> + if (cpumask_empty(to_cpumask(rcp->cpumask))) {
> /* batch completed ! */
> rcp->completed = rcp->cur;
> rcu_start_batch(rcp);
> --- linux-2.6-for-ingo.orig/kernel/rcupreempt.c
> +++ linux-2.6-for-ingo/kernel/rcupreempt.c
> @@ -164,7 +164,8 @@ static char *rcu_try_flip_state_names[]
> { "idle", "waitack", "waitzero", "waitmb" };
> #endif /* #ifdef CONFIG_RCU_TRACE */
>
> -static cpumask_t rcu_cpu_online_map __read_mostly = CPU_MASK_NONE;
> +static DECLARE_BITMAP(rcu_cpu_online_map, CONFIG_NR_CPUS) __read_mostly
> + = CPU_BITS_NONE;
>
> /*
> * Enum and per-CPU flag to determine when each CPU has seen
> @@ -748,7 +749,7 @@ rcu_try_flip_idle(void)
>
> /* Now ask each CPU for acknowledgement of the flip. */
>
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
> per_cpu(rcu_flip_flag, cpu) = rcu_flipped;
> dyntick_save_progress_counter(cpu);
> }
> @@ -766,7 +767,7 @@ rcu_try_flip_waitack(void)
> int cpu;
>
> RCU_TRACE_ME(rcupreempt_trace_try_flip_a1);
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
> if (rcu_try_flip_waitack_needed(cpu) &&
> per_cpu(rcu_flip_flag, cpu) != rcu_flip_seen) {
> RCU_TRACE_ME(rcupreempt_trace_try_flip_ae1);
> @@ -798,7 +799,7 @@ rcu_try_flip_waitzero(void)
> /* Check to see if the sum of the "last" counters is zero. */
>
> RCU_TRACE_ME(rcupreempt_trace_try_flip_z1);
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
> sum += RCU_DATA_CPU(cpu)->rcu_flipctr[lastidx];
> if (sum != 0) {
> RCU_TRACE_ME(rcupreempt_trace_try_flip_ze1);
> @@ -813,7 +814,7 @@ rcu_try_flip_waitzero(void)
> smp_mb(); /* ^^^^^^^^^^^^ */
>
> /* Call for a memory barrier from each CPU. */
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map) {
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map)) {
> per_cpu(rcu_mb_flag, cpu) = rcu_mb_needed;
> dyntick_save_progress_counter(cpu);
> }
> @@ -833,7 +834,7 @@ rcu_try_flip_waitmb(void)
> int cpu;
>
> RCU_TRACE_ME(rcupreempt_trace_try_flip_m1);
> - for_each_cpu_mask_nr(cpu, rcu_cpu_online_map)
> + for_each_cpu(cpu, to_cpumask(rcu_cpu_online_map))
> if (rcu_try_flip_waitmb_needed(cpu) &&
> per_cpu(rcu_mb_flag, cpu) != rcu_mb_done) {
> RCU_TRACE_ME(rcupreempt_trace_try_flip_me1);
> @@ -1022,7 +1023,7 @@ void rcu_offline_cpu(int cpu)
> RCU_DATA_CPU(cpu)->rcu_flipctr[0] = 0;
> RCU_DATA_CPU(cpu)->rcu_flipctr[1] = 0;
>
> - cpu_clear(cpu, rcu_cpu_online_map);
> + cpumask_clear_cpu(cpu, to_cpumask(rcu_cpu_online_map));
>
> spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
>
> @@ -1062,7 +1063,7 @@ void __cpuinit rcu_online_cpu(int cpu)
> struct rcu_data *rdp;
>
> spin_lock_irqsave(&rcu_ctrlblk.fliplock, flags);
> - cpu_set(cpu, rcu_cpu_online_map);
> + cpumask_set_cpu(cpu, to_cpumask(rcu_cpu_online_map));
> spin_unlock_irqrestore(&rcu_ctrlblk.fliplock, flags);
>
> /*
> @@ -1420,7 +1421,7 @@ void __init __rcu_init(void)
> * We don't need protection against CPU-Hotplug here
> * since
> * a) If a CPU comes online while we are iterating over the
> - * cpu_online_map below, we would only end up making a
> + * cpu_online_mask below, we would only end up making a
> * duplicate call to rcu_online_cpu() which sets the corresponding
> * CPU's mask in the rcu_cpu_online_map.
> *
> --- linux-2.6-for-ingo.orig/kernel/rcutorture.c
> +++ linux-2.6-for-ingo/kernel/rcutorture.c
> @@ -843,49 +843,52 @@ static int rcu_idle_cpu; /* Force all to
> */
> static void rcu_torture_shuffle_tasks(void)
> {
> - cpumask_t tmp_mask;
> + cpumask_var_t tmp_mask;
> int i;
>
> - cpus_setall(tmp_mask);
> + if (!alloc_cpumask_var(&tmp_mask, GFP_KERNEL))
> + BUG();
> +
> + cpumask_setall(tmp_mask);
> get_online_cpus();
>
> /* No point in shuffling if there is only one online CPU (ex: UP) */
> - if (num_online_cpus() == 1) {
> - put_online_cpus();
> - return;
> - }
> + if (num_online_cpus() == 1)
> + goto out;
>
> if (rcu_idle_cpu != -1)
> - cpu_clear(rcu_idle_cpu, tmp_mask);
> + cpumask_clear_cpu(rcu_idle_cpu, tmp_mask);
>
> - set_cpus_allowed_ptr(current, &tmp_mask);
> + set_cpus_allowed_ptr(current, tmp_mask);
>
> if (reader_tasks) {
> for (i = 0; i < nrealreaders; i++)
> if (reader_tasks[i])
> set_cpus_allowed_ptr(reader_tasks[i],
> - &tmp_mask);
> + tmp_mask);
> }
>
> if (fakewriter_tasks) {
> for (i = 0; i < nfakewriters; i++)
> if (fakewriter_tasks[i])
> set_cpus_allowed_ptr(fakewriter_tasks[i],
> - &tmp_mask);
> + tmp_mask);
> }
>
> if (writer_task)
> - set_cpus_allowed_ptr(writer_task, &tmp_mask);
> + set_cpus_allowed_ptr(writer_task, tmp_mask);
>
> if (stats_task)
> - set_cpus_allowed_ptr(stats_task, &tmp_mask);
> + set_cpus_allowed_ptr(stats_task, tmp_mask);
>
> if (rcu_idle_cpu == -1)
> rcu_idle_cpu = num_online_cpus() - 1;
> else
> rcu_idle_cpu--;
>
> +out:
> put_online_cpus();
> + free_cpumask_var(tmp_mask);
> }
>
> /* Shuffle tasks across CPUs, with the intent of allowing each CPU in the
> --- linux-2.6-for-ingo.orig/kernel/softlockup.c
> +++ linux-2.6-for-ingo/kernel/softlockup.c
> @@ -310,10 +310,8 @@ cpu_callback(struct notifier_block *nfb,
> case CPU_DOWN_PREPARE:
> case CPU_DOWN_PREPARE_FROZEN:
> if (hotcpu == check_cpu) {
> - cpumask_t temp_cpu_online_map = cpu_online_map;
> -
> - cpu_clear(hotcpu, temp_cpu_online_map);
> - check_cpu = any_online_cpu(temp_cpu_online_map);
> + /* Pick any other online cpu. */
> + check_cpu = cpumask_any_but(cpu_online_mask, hotcpu);
> }
> break;
>
> --- linux-2.6-for-ingo.orig/kernel/stop_machine.c
> +++ linux-2.6-for-ingo/kernel/stop_machine.c
> @@ -69,10 +69,10 @@ static void stop_cpu(struct work_struct
> int err;
>
> if (!active_cpus) {
> - if (cpu == first_cpu(cpu_online_map))
> + if (cpu == cpumask_first(cpu_online_mask))
> smdata = &active;
> } else {
> - if (cpu_isset(cpu, *active_cpus))
> + if (cpumask_test_cpu(cpu, active_cpus))
> smdata = &active;
> }
> /* Simple state machine */
> @@ -109,7 +109,7 @@ static int chill(void *unused)
> return 0;
> }
>
> -int __stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
> +int __stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
> {
> struct work_struct *sm_work;
> int i, ret;
> @@ -142,7 +142,7 @@ int __stop_machine(int (*fn)(void *), vo
> return ret;
> }
>
> -int stop_machine(int (*fn)(void *), void *data, const cpumask_t *cpus)
> +int stop_machine(int (*fn)(void *), void *data, const struct cpumask *cpus)
> {
> int ret;
>
> --- linux-2.6-for-ingo.orig/kernel/taskstats.c
> +++ linux-2.6-for-ingo/kernel/taskstats.c
> @@ -290,18 +290,17 @@ ret:
> return;
> }
>
> -static int add_del_listener(pid_t pid, cpumask_t *maskp, int isadd)
> +static int add_del_listener(pid_t pid, const struct cpumask *mask, int isadd)
> {
> struct listener_list *listeners;
> struct listener *s, *tmp;
> unsigned int cpu;
> - cpumask_t mask = *maskp;
>
> - if (!cpus_subset(mask, cpu_possible_map))
> + if (!cpumask_subset(mask, cpu_possible_mask))
> return -EINVAL;
>
> if (isadd == REGISTER) {
> - for_each_cpu_mask_nr(cpu, mask) {
> + for_each_cpu(cpu, mask) {
> s = kmalloc_node(sizeof(struct listener), GFP_KERNEL,
> cpu_to_node(cpu));
> if (!s)
> @@ -320,7 +319,7 @@ static int add_del_listener(pid_t pid, c
>
> /* Deregister or cleanup */
> cleanup:
> - for_each_cpu_mask_nr(cpu, mask) {
> + for_each_cpu(cpu, mask) {
> listeners = &per_cpu(listener_array, cpu);
> down_write(&listeners->sem);
> list_for_each_entry_safe(s, tmp, &listeners->list, list) {
> @@ -335,7 +334,7 @@ cleanup:
> return 0;
> }
>
> -static int parse(struct nlattr *na, cpumask_t *mask)
> +static int parse(struct nlattr *na, struct cpumask *mask)
> {
> char *data;
> int len;
> @@ -428,23 +427,33 @@ err:
>
> static int taskstats_user_cmd(struct sk_buff *skb, struct genl_info *info)
> {
> - int rc = 0;
> + int rc;
> struct sk_buff *rep_skb;
> struct taskstats *stats;
> size_t size;
> - cpumask_t mask;
> + cpumask_var_t mask;
> +
> + if (!alloc_cpumask_var(&mask, GFP_KERNEL))
> + return -ENOMEM;
>
> - rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], &mask);
> + rc = parse(info->attrs[TASKSTATS_CMD_ATTR_REGISTER_CPUMASK], mask);
> if (rc < 0)
> - return rc;
> - if (rc == 0)
> - return add_del_listener(info->snd_pid, &mask, REGISTER);
> + goto free_return_rc;
> + if (rc == 0) {
> + rc = add_del_listener(info->snd_pid, mask, REGISTER);
> + goto free_return_rc;
> + }
>
> - rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], &mask);
> + rc = parse(info->attrs[TASKSTATS_CMD_ATTR_DEREGISTER_CPUMASK], mask);
> if (rc < 0)
> + goto free_return_rc;
> + if (rc == 0) {
> + rc = add_del_listener(info->snd_pid, mask, DEREGISTER);
> +free_return_rc:
> + free_cpumask_var(mask);
> return rc;
> - if (rc == 0)
> - return add_del_listener(info->snd_pid, &mask, DEREGISTER);
> + }
> + free_cpumask_var(mask);
>
> /*
> * Size includes space for nested attributes
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-19 16:54 ` Mike Travis
2008-12-19 17:01 ` Steven Rostedt
@ 2008-12-20 1:41 ` Steven Rostedt
2008-12-20 11:22 ` Rusty Russell
1 sibling, 1 reply; 21+ messages in thread
From: Steven Rostedt @ 2008-12-20 1:41 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, Rusty Russell, linux-kernel
On Fri, 19 Dec 2008, Mike Travis wrote:
> Steven Rostedt wrote:
> > On Fri, 19 Dec 2008, Mike Travis wrote:
> >
> >> Impact: Reduce memory usage, use new cpumask API.
> >>
> >> Convert kernel trace functions to use struct cpumask.
> >>
> > How does this save memory if we just allocate a cpumask var everytime
> > we allocate a ring buffer? Is cpumask_var_t a mask of possible CPUS and
> > not NR_CPUS?
> >
> > Otherwise, I see this as just adding one extra pointer.
> >
> > -- Steve
>
> Hi Steve,
>
> Yes, eventually, the cpumask_var_t will be allocated based on cpumask_size()
> which will become BITS_TO_LONG(nr_cpu_ids) instead of BITS_TO_LONGS(NR_CPUS)
> as soon as the kernel becomes "cpumask" clean. (clean being that it ignores
> all bits >= nr_cpu_ids.)
>
> Note that on small NR_CPUS count systems, cpumask_var_t is a static array
> and no allocation happens, so it only kicks in when CONFIG_CPUMASK_OFFSTACK=y.
For future memory savings...
Acked-by: Steven Rostedt <srostedt@redhat.com>
-- Steve
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/8] cpumask: convert kernel trace functions
2008-12-20 1:41 ` Steven Rostedt
@ 2008-12-20 11:22 ` Rusty Russell
0 siblings, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-20 11:22 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Mike Travis, Ingo Molnar, linux-kernel
On Saturday 20 December 2008 12:11:49 Steven Rostedt wrote:
> For future memory savings...
>
> Acked-by: Steven Rostedt <srostedt@redhat.com>
Thanks for this Mike, this seems a significant enhancement over the
one in my tree (AFAICT I didn't do ring_buffer.c, nor use on_each_cpu).
I've updated the description, and removed one gratuitous indent change.
It's now in the linux-next section of my tree:
cpumask: convert kernel trace functions
Impact: Reduce future memory usage, use new cpumask API.
(Eventually, cpumask_var_t will be allocated based on nr_cpu_ids, not NR_CPUS).
Convert kernel trace functions to use struct cpumask API:
1) Use cpumask_copy/cpumask_test_cpu/for_each_cpu.
2) Use cpumask_var_t and alloc_cpumask_var/free_cpumask_var everywhere.
3) Use on_each_cpu instead of playing with current->cpus_allowed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/8] cpumask: convert kernel time functions
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
@ 2008-12-20 11:29 ` Rusty Russell
0 siblings, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-20 11:29 UTC (permalink / raw)
To: Mike Travis; +Cc: Ingo Molnar, linux-kernel, Thomas Gleixner
On Saturday 20 December 2008 02:31:47 Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API.
>
> Convert kernel/time functions to use struct cpumask *.
>
> Note the ugly bitmap declarations in tick-broadcast.c. These should
> be cpumask_var_t, but there was no obvious initialization function to
> put the alloc_cpumask_var() calls in. This was safe.
I still would like see those bitmaps turned into real cpumask_var_t. It
should be easy for someone who knows the code well, or has time to spend
making sure the initialization is early enough.
Cheers,
Rusty.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5/8] cpumask: convert rest of files in kernel/
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
2008-12-20 1:33 ` Lai Jiangshan
@ 2008-12-20 13:02 ` Rusty Russell
1 sibling, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-20 13:02 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, linux-kernel, Dipankar Sarma, H. Peter Anvin,
Lai Jiangshan, Max Krasnyansky, Richard Henderson,
Thomas Gleixner, Yinghai Lu
On Saturday 20 December 2008 02:31:49 Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API. ALPHA mod!
This need to be split, I think. I was just lazy in leaving the conversion
as one big patch.
> Note that prof_cpu_mask will be difficult to convert to a cpumask_var_t
> since it needs to be ready from the first scheduler tick.
Actually, I think this one can be fixed quite easily.
OK, I've done this. Compiles...
Rusty.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] cpumask: convert misc driver functions
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
@ 2008-12-20 23:22 ` Ben Hutchings
2008-12-29 2:28 ` Rusty Russell
2008-12-29 15:37 ` Dean Nelson
1 sibling, 1 reply; 21+ messages in thread
From: Ben Hutchings @ 2008-12-20 23:22 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Dean Nelson,
Jeremy Fitzhardinge, Robert Richter
On Fri, 2008-12-19 at 08:01 -0800, Mike Travis wrote:
[...]
> --- linux-2.6-for-ingo.orig/drivers/net/sfc/efx.c
> +++ linux-2.6-for-ingo/drivers/net/sfc/efx.c
> @@ -809,19 +809,18 @@ static void efx_fini_io(struct efx_nic *
> /* Get number of RX queues wanted. Return number of online CPU
> * packages in the expectation that an IRQ balancer will spread
> * interrupts across them. */
> -static int efx_wanted_rx_queues(void)
> +static int efx_wanted_rx_queues(struct cpumask *scratch)
> {
> - cpumask_t core_mask;
> int count;
> int cpu;
>
> - cpus_clear(core_mask);
> + cpumask_clear(scratch);
> count = 0;
> for_each_online_cpu(cpu) {
> - if (!cpu_isset(cpu, core_mask)) {
> + if (!cpumask_test_cpu(cpu, scratch)) {
> ++count;
> - cpus_or(core_mask, core_mask,
> - topology_core_siblings(cpu));
> + cpumask_or(scratch, scratch,
> + topology_core_cpumask(cpu));
> }
> }
>
> @@ -831,7 +830,7 @@ static int efx_wanted_rx_queues(void)
> /* Probe the number and type of interrupts we are able to obtain, and
> * the resulting numbers of channels and RX queues.
> */
> -static void efx_probe_interrupts(struct efx_nic *efx)
> +static void efx_probe_interrupts(struct efx_nic *efx, struct cpumask *scratch)
> {
> int max_channels =
> min_t(int, efx->type->phys_addr_channels, EFX_MAX_CHANNELS);
> @@ -845,7 +844,8 @@ static void efx_probe_interrupts(struct
> * (or as specified by the rss_cpus module parameter).
> * We will need one channel per interrupt.
> */
> - wanted_ints = rss_cpus ? rss_cpus : efx_wanted_rx_queues();
> + wanted_ints = rss_cpus ? rss_cpus :
> + efx_wanted_rx_queues(scratch);
> efx->n_rx_queues = min(wanted_ints, max_channels);
>
> for (i = 0; i < efx->n_rx_queues; i++)
> @@ -923,24 +923,29 @@ static void efx_set_channels(struct efx_
> static int efx_probe_nic(struct efx_nic *efx)
> {
> int rc;
> + cpumask_var_t scratch;
>
> EFX_LOG(efx, "creating NIC\n");
>
> + if (!alloc_cpumask_var(&scratch, GFP_KERNEL))
> + return -ENOMEM;
> +
> /* Carry out hardware-type specific initialisation */
> rc = falcon_probe_nic(efx);
> if (rc)
> - return rc;
> + goto out;
>
> /* Determine the number of channels and RX queues by trying to hook
> * in MSI-X interrupts. */
> - efx_probe_interrupts(efx);
> + efx_probe_interrupts(efx, scratch);
>
> efx_set_channels(efx);
>
> /* Initialise the interrupt moderation settings */
> efx_init_irq_moderation(efx, tx_irq_mod_usec, rx_irq_mod_usec);
> -
> - return 0;
> +out:
> + free_cpumask_var(scratch);
> + return rc;
> }
>
> static void efx_remove_nic(struct efx_nic *efx)
[...]
I assume you're moving the allocation up to efx_probe_nic() because
efx_wanted_rx_queues() and efx_probe_interrupts() cannot return failure.
It really isn't worth exposing that detail up the call chain though. I
think it's acceptable for efx_wanted_rx_queues() to log an error message
and return 1 in the exceedingly unlikely case that the allocation fails.
Ben.
--
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] cpumask: convert misc driver functions
2008-12-20 23:22 ` Ben Hutchings
@ 2008-12-29 2:28 ` Rusty Russell
0 siblings, 0 replies; 21+ messages in thread
From: Rusty Russell @ 2008-12-29 2:28 UTC (permalink / raw)
To: Ben Hutchings
Cc: Mike Travis, Ingo Molnar, linux-kernel, Dean Nelson,
Jeremy Fitzhardinge, Robert Richter
On Sunday 21 December 2008 09:52:39 Ben Hutchings wrote:
> I assume you're moving the allocation up to efx_probe_nic() because
> efx_wanted_rx_queues() and efx_probe_interrupts() cannot return failure.
> It really isn't worth exposing that detail up the call chain though. I
> think it's acceptable for efx_wanted_rx_queues() to log an error message
> and return 1 in the exceedingly unlikely case that the allocation fails.
OK, fair call. I was trying trying hard not to break anything.
How's this?
cpumask: convert drivers/net/sfc
Remove a cpumask from the stack. Ben Hutchings indicated that printing
a warning and returning 1 was acceptable for the corner case where allocation
fails.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: Ben Hutchings <bhutchings@solarflare.com>
diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -854,20 +854,27 @@ static void efx_fini_io(struct efx_nic *
* interrupts across them. */
static int efx_wanted_rx_queues(void)
{
- cpumask_t core_mask;
+ cpumask_var_t core_mask;
int count;
int cpu;
- cpus_clear(core_mask);
+ if (!alloc_cpumask_var(&core_mask, GFP_KERNEL)) {
+ printk(KERN_WARNING
+ "efx.c: allocation failure, irq balancing hobbled\n");
+ return 1;
+ }
+
+ cpumask_clear(core_mask);
count = 0;
for_each_online_cpu(cpu) {
- if (!cpu_isset(cpu, core_mask)) {
+ if (!cpumask_test_cpu(cpu, core_mask)) {
++count;
- cpumask_or(&core_mask, &core_mask,
+ cpumask_or(core_mask, core_mask,
topology_core_cpumask(cpu));
}
}
+ free_cpumask_var(core_mask);
return count;
}
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 7/8] cpumask: convert misc driver functions
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
2008-12-20 23:22 ` Ben Hutchings
@ 2008-12-29 15:37 ` Dean Nelson
1 sibling, 0 replies; 21+ messages in thread
From: Dean Nelson @ 2008-12-29 15:37 UTC (permalink / raw)
To: Mike Travis
Cc: Ingo Molnar, Rusty Russell, linux-kernel, Ben Hutchings,
Jeremy Fitzhardinge, Robert Richter
On Fri, Dec 19, 2008 at 08:01:51AM -0800, Mike Travis wrote:
> Impact: Reduce stack usage, use new cpumask API.
>
> Convert misc driver functions to use struct cpumask.
>
> To Do:
> - Convert iucv_buffer_cpumask to cpumask_var_t.
>
> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> Signed-off-by: Mike Travis <travis@sgi.com>
[...]
> --- linux-2.6-for-ingo.orig/drivers/misc/sgi-xp/xpc_main.c
> +++ linux-2.6-for-ingo/drivers/misc/sgi-xp/xpc_main.c
> @@ -318,7 +318,7 @@ xpc_hb_checker(void *ignore)
>
> /* this thread was marked active by xpc_hb_init() */
>
> - set_cpus_allowed_ptr(current, &cpumask_of_cpu(XPC_HB_CHECK_CPU));
> + set_cpus_allowed_ptr(current, cpumask_of(XPC_HB_CHECK_CPU));
>
> /* set our heartbeating to other partitions into motion */
> xpc_hb_check_timeout = jiffies + (xpc_hb_check_interval * HZ);
Acked-by: Dean Nelson <dcn@sgi.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2008-12-29 15:37 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-12-19 16:01 [PATCH 0/8] x86 cpumask: more cpumask updates to core kernel routines Mike Travis
2008-12-19 16:01 ` [PATCH 1/8] cpumask: convert kernel/compat.c Mike Travis
2008-12-19 16:01 ` [PATCH 2/8] cpumask: convert kernel/workqueue.c Mike Travis
2008-12-19 16:01 ` [PATCH 3/8] cpumask: convert kernel time functions Mike Travis
2008-12-20 11:29 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 4/8] cpumask: convert kernel trace functions Mike Travis
2008-12-19 16:16 ` Steven Rostedt
2008-12-19 16:54 ` Mike Travis
2008-12-19 17:01 ` Steven Rostedt
2008-12-20 1:41 ` Steven Rostedt
2008-12-20 11:22 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 5/8] cpumask: convert rest of files in kernel/ Mike Travis
2008-12-20 1:33 ` Lai Jiangshan
2008-12-20 13:02 ` Rusty Russell
2008-12-19 16:01 ` [PATCH 6/8] cpumask: convert kernel mm functions Mike Travis
2008-12-19 18:08 ` Christoph Lameter
2008-12-19 16:01 ` [PATCH 7/8] cpumask: convert misc driver functions Mike Travis
2008-12-20 23:22 ` Ben Hutchings
2008-12-29 2:28 ` Rusty Russell
2008-12-29 15:37 ` Dean Nelson
2008-12-19 16:01 ` [PATCH 8/8] cpumask: convert other misc kernel functions Mike Travis
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.