Re: VolanoMark regression with 2.6.27-rc1

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

Ok, so one last time (I hope!)..

Everybody happy with this?

---
Subject: sched: load-balance bias fixes
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date: Wed Aug 20 15:28:51 CEST 2008

Yanmin spotted a regression with my patch that introduces LB_BIAS:

  commit 93b75217df39e6d75889cc6f8050343286aff4a5
  Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
  Date:   Fri Jun 27 13:41:33 2008 +0200

And I just spotted the brainfart - I should have replaced min/max with avg
instead of removing it completely.

[ray-lk@madrabbit.org: better avg implementation]
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
 include/linux/kernel.h |    6 ++++++
 kernel/sched.c         |   10 ++++++++--
 2 files changed, 14 insertions(+), 2 deletions(-)

Index: linux-2.6/kernel/sched.c
===================================================================
--- linux-2.6.orig/kernel/sched.c
+++ linux-2.6/kernel/sched.c
@@ -2008,9 +2008,12 @@ static unsigned long source_load(int cpu
 	struct rq *rq = cpu_rq(cpu);
 	unsigned long total = weighted_cpuload(cpu);
 
-	if (type == 0 || !sched_feat(LB_BIAS))
+	if (type == 0)
 		return total;
 
+	if (!sched_feat(LB_BIAS))
+		return avg(rq->cpu_load[type-1], total);
+
 	return min(rq->cpu_load[type-1], total);
 }
 
@@ -2023,9 +2026,12 @@ static unsigned long target_load(int cpu
 	struct rq *rq = cpu_rq(cpu);
 	unsigned long total = weighted_cpuload(cpu);
 
-	if (type == 0 || !sched_feat(LB_BIAS))
+	if (type == 0)
 		return total;
 
+	if (!sched_feat(LB_BIAS))
+		return avg(rq->cpu_load[type-1], total);
+
 	return max(rq->cpu_load[type-1], total);
 }
 
Index: linux-2.6/include/linux/kernel.h
===================================================================
--- linux-2.6.orig/include/linux/kernel.h
+++ linux-2.6/include/linux/kernel.h
@@ -367,6 +367,12 @@ static inline char *pack_hex_byte(char *
 	(void) (&_max1 == &_max2);		\
 	_max1 > _max2 ? _max1 : _max2; })
 
+#define avg(x, y) ({				\
+	typeof(x) _avg1 = (x);			\
+	typeof(y) _avg2 = (y);			\
+	(void) (&_avg1 == &_avg2);		\
+	_avg1 + (_avg2 - _avg1)/2; })
+
 /**
  * clamp - return a value clamped to a given range with strict typechecking
  * @val: current value