Re: [PATCH 08/35] autonuma: introduce kthread_bind_node()

Andrea Arcangeli
2012-05-29T17:45:38Z

On Tue, May 29, 2012 at 07:04:51PM +0200, Peter Zijlstra wrote:
> doing it without mention and keeping a misleading comment near the
> definition.

Right, I forgot to update the comment, I fixed it now.

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 60a699c..0b84494 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1788,7 +1788,7 @@ extern void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t *
#define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */
#define PF_SPREAD_PAGE 0x01000000 /* Spread page cache over cpuset */
#define PF_SPREAD_SLAB 0x02000000 /* Spread some slab caches over cpuset */
-#define PF_THREAD_BOUND 0x04000000 /* Thread bound to specific cpu */
+#define PF_THREAD_BOUND 0x04000000 /* Thread bound to specific cpus */
#define PF_MCE_EARLY 0x08000000 /* Early kill for mce process policy */
#define PF_MEMPOLICY 0x10000000 /* Non-default NUMA mempolicy */
#define PF_MUTEX_TESTER 0x20000000 /* Thread belongs to the rt mutex tester */

> Just teach each knuma_migrated what node it represents and don't use
> numa_node_id().

It already works like that, I absolutely never use numa_node_id(), I
always use the pgdat passed as parameter to the kernel thread through
the pointer parameter.

But it'd be totally bad not to do the hard bindings to the cpu_s_ of
the node, and not using PF_THREAD_BOUND would just allow userland to
shoot itself in the foot. I mean if PF_THREAD_BOUND wouldn't exist
already I wouldn't add it, but considering somebody bothered to
implement it for the sake to make userland root user "safer", it'd be
really silly not to take advantage of that for knuma_migrated too
(even if it binds to more than 1 CPU).

Additionally I added a bugcheck in the main knuma_migrated loop:

VM_BUG_ON(numa_node_id() != pgdat->node_id);

to be sure it never goes wrong. This above bugcheck is what allowed me
to find a bug in the numa emulation fixed in commit
d71b5a73fe9af42752c4329b087f7911b35f8f79.

> That way you can change the affinity just fine, it'll be sub-optimal,
> copying memory from node x to node y through node z, but it'll still
> work correctly.

I don't think allowing userland to do suboptimal things (even if it
will only decrease performance and still work correctly) makes
sense (considering somebody added PF_THREAD_BOUND already and it's
zero cost to use).

> numa isn't special in the way per-cpu stuff is special.

Agreed that it won't be as bad as getting per-cpu stuff wrong, it only
slowdown -50% in the worst case, but it's a guaranteed regression in
the best case too, so there's no reason to allow root to shoot itself
in the foot.