[RFC] [PATCH] [2.5.35] Run Queue Statistics

[RFC] [PATCH] [2.5.35] Run Queue Statistics

[Lev Makhlis]

This patch adds two counters, runque and runocc, similar to those
in traditional UNIX systems, to measure the run queue occupancy.
Every second, 'runque' is incremented by the run queue size, and
'runocc' is incremented by one if the run queue is not empty.

I am not comfortable about putting the calculation in the same function
as the load average calculation, but I didn't want to call
count_active_tasks() twice. Comments are welcome.

Lev

--------------------------------------------------------------------------
diff -urN linux-2.5.35.orig/fs/proc/proc_misc.c 
linux-2.5.35/fs/proc/proc_misc.c
--- linux-2.5.35.orig/fs/proc/proc_misc.c	Sun Sep 15 22:18:21 2002
+++ linux-2.5.35/fs/proc/proc_misc.c	Mon Sep 16 13:36:14 2002
@@ -386,7 +386,8 @@
 		"allocstall %u\n"
 		"ctxt %lu\n"
 		"btime %lu\n"
-		"processes %lu\n",
+		"processes %lu\n"
+		"runque %u %u\n",
 		kstat.pgalloc,
 		kstat.pgfree,
 		kstat.pgactivate,
@@ -399,7 +400,9 @@
 		kstat.allocstall,
 		nr_context_switches(),
 		xtime.tv_sec - jif / HZ,
-		total_forks);
+		total_forks,
+		kstat.runque,
+		kstat.runocc);
 
 	return proc_calc_metrics(page, start, off, count, eof, len);
 }
diff -urN linux-2.5.35.orig/include/linux/kernel_stat.h 
linux-2.5.35/include/linux/kernel_stat.h
--- linux-2.5.35.orig/include/linux/kernel_stat.h	Sun Sep 15 22:18:27 2002
+++ linux-2.5.35/include/linux/kernel_stat.h	Mon Sep 16 13:35:30 2002
@@ -31,6 +31,7 @@
 	unsigned int pgfault, pgmajfault;
 	unsigned int pgscan, pgsteal;
 	unsigned int pageoutrun, allocstall;
+	unsigned int runque, runocc;
 #if !defined(CONFIG_ARCH_S390)
 	unsigned int irqs[NR_CPUS][NR_IRQS];
 #endif
diff -urN linux-2.5.35.orig/kernel/timer.c linux-2.5.35/kernel/timer.c
--- linux-2.5.35.orig/kernel/timer.c	Sun Sep 15 22:18:24 2002
+++ linux-2.5.35/kernel/timer.c	Mon Sep 16 13:36:31 2002
@@ -592,11 +592,11 @@
 }
 
 /*
- * Nr of active tasks - counted in fixed-point numbers
+ * Nr of active tasks
  */
 static unsigned long count_active_tasks(void)
 {
-	return (nr_running() + nr_uninterruptible()) * FIXED_1;
+	return nr_running() + nr_uninterruptible();
 }
 
 /*
@@ -615,16 +615,29 @@
  */
 static inline void calc_load(unsigned long ticks)
 {
-	unsigned long active_tasks; /* fixed-point */
-	static int count = LOAD_FREQ;
+	unsigned long active_tasks;
+	unsigned long fp_active_tasks; /* fixed-point */
+	static int load_count = LOAD_FREQ;
+	static int runq_count = HZ;
 
-	count -= ticks;
-	if (count < 0) {
-		count += LOAD_FREQ;
+	load_count -= ticks;
+	runq_count -= ticks;
+	if (load_count < 0 || runq_count < 0) {
 		active_tasks = count_active_tasks();
-		CALC_LOAD(avenrun[0], EXP_1, active_tasks);
-		CALC_LOAD(avenrun[1], EXP_5, active_tasks);
-		CALC_LOAD(avenrun[2], EXP_15, active_tasks);
+		if (runq_count < 0) {
+			runq_count += HZ;
+			if (active_tasks) {
+				kstat.runque += active_tasks;
+				kstat.runocc ++;
+			}
+		}
+		if (load_count < 0) {
+			load_count += LOAD_FREQ;
+			fp_active_tasks = active_tasks * FIXED_1;
+			CALC_LOAD(avenrun[0], EXP_1, fp_active_tasks);
+			CALC_LOAD(avenrun[1], EXP_5, fp_active_tasks);
+			CALC_LOAD(avenrun[2], EXP_15, fp_active_tasks);
+		}
 	}
 }
 

Re: [RFC] [PATCH] [2.5.35] Run Queue Statistics

[Anton Blanchard]

Hi,

> This patch adds two counters, runque and runocc, similar to those
> in traditional UNIX systems, to measure the run queue occupancy.
> Every second, 'runque' is incremented by the run queue size, and
> 'runocc' is incremented by one if the run queue is not empty.
> 
> I am not comfortable about putting the calculation in the same function
> as the load average calculation, but I didn't want to call
> count_active_tasks() twice. Comments are welcome.

On a semi related note, vmstat wants to know the number of running,
blocked and swapped processes. strace vmstat one day and you will see it
currently opens /proc/*/stat (ie one open for each process) just to get
these stats.  Yet another place where the monitoring utilities disturb
the system way too much.

Can we get some things in /proc/stat to give us these numbers? Does
"swapped" make any sense on Linux?

Anton

Re: [RFC] [PATCH] [2.5.35] Run Queue Statistics

[Rik van Riel]

On Tue, 17 Sep 2002, Anton Blanchard wrote:

> On a semi related note, vmstat wants to know the number of running,
> blocked and swapped processes. strace vmstat one day and you will see it
> currently opens /proc/*/stat (ie one open for each process) just to get
> these stats.  Yet another place where the monitoring utilities disturb
> the system way too much.
>
> Can we get some things in /proc/stat to give us these numbers? Does
> "swapped" make any sense on Linux?

Runnable can be done currently, blocked on IO is trivial once
Andrew has pushed the iowait stats to Linus.

Swapped doesn't make any sense at the moment, but it should.
A system without load control is just too vulnerable to sudden
load spikes. If Andrew has interest I'll pick up the work I
did in that area ...

I'll also update vmstat to just use /proc/stat instead of
looking at all /proc/*/stat files.

cheers,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Spamtraps of the month:  september@surriel.com trac@trac.org

Re: [despammed] Re: [RFC] [PATCH] [2.5.35] Run Queue Statistics

[Lev Makhlis]

On Monday 16 September 2002 11:08 pm, Rik van Riel wrote:
> On Tue, 17 Sep 2002, Anton Blanchard wrote:
> > On a semi related note, vmstat wants to know the number of running,
> > blocked and swapped processes. strace vmstat one day and you will see it
> > currently opens /proc/*/stat (ie one open for each process) just to get
> > these stats.  Yet another place where the monitoring utilities disturb
> > the system way too much.
> >
> > Can we get some things in /proc/stat to give us these numbers? Does
> > "swapped" make any sense on Linux?
>
> Runnable can be done currently, blocked on IO is trivial once
> Andrew has pushed the iowait stats to Linus.
>
> Swapped doesn't make any sense at the moment, but it should.
> A system without load control is just too vulnerable to sudden
> load spikes. If Andrew has interest I'll pick up the work I
> did in that area ...
>
> I'll also update vmstat to just use /proc/stat instead of
> looking at all /proc/*/stat files.
>
> cheers,
>
> Rik

Amusingly, the number of running processes can be found in
/proc/loadavg, of all places, right now.

Re: [RFC] [PATCH] [2.5.35] Run Queue Statistics

[Andrew Morton]

Rik van Riel wrote:
> 
> On Tue, 17 Sep 2002, Anton Blanchard wrote:
> 
> > On a semi related note, vmstat wants to know the number of running,
> > blocked and swapped processes. strace vmstat one day and you will see it
> > currently opens /proc/*/stat (ie one open for each process) just to get
> > these stats.  Yet another place where the monitoring utilities disturb
> > the system way too much.
> >
> > Can we get some things in /proc/stat to give us these numbers? Does
> > "swapped" make any sense on Linux?

Certainly sounds good.  Opening every /proc/<pid>/stat is gross.

> Runnable can be done currently, blocked on IO is trivial once
> Andrew has pushed the iowait stats to Linus.
> 

That'll be a while off yet. I'd like to make sure that we have
all the externally visible changes stable for a week or so,
/proc/diskstats settled down, userspace updated and tested etc.

Just to minimise the disruption and churn which these changes
will cause.