[PATCH 2.6]: Make packet scheduler clock source configurable

[PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

Hi Dave,

this patch replaces PSCHED_CLOCK_SOURCE by three new config options
and fixes HTB compilation with CONFIG_NET_SCH_CLK_GETTIMEOFDAY.
Patch compiles with all three options. If you like it I will also send
you a 2.4 version.

Regards
Patrick

[-- Attachment #2: psched-clk.diff --]
[-- Type: application/octect-stream, Size: 9056 bytes --]

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[shemminger]

> Hi Dave,
>
> this patch replaces PSCHED_CLOCK_SOURCE by three new config options
> and fixes HTB compilation with CONFIG_NET_SCH_CLK_GETTIMEOFDAY.
> Patch compiles with all three options. If you like it I will also send
> you a 2.4 version.
>
> Regards
> Patrick
>

I have a patch (to the original) which replaces the arch dependant CPU stuff
with the arch neutral sched_clock() which although slightly
slower works on a x86_64 and ia64 as well.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Mon, 12 Jul 2004 20:35:16 -0700 (PDT)
shemminger@osdl.org wrote:

> I have a patch (to the original) which replaces the arch dependant CPU stuff
> with the arch neutral sched_clock() which although slightly
> slower works on a x86_64 and ia64 as well.

I also want to add sparc64 %tick register support here too.
This is an area with a lot of potential cleanup.  We should
made an asm/pkt_sched.h to abstract out the clock source
implementation instead of loading net/pkt_sched.h up with
a pile of ifdefs.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Wed, 14 Jul 2004 05:46:19 +0200
Patrick McHardy <kaber@trash.net> wrote:

> This one actually compiles ;). The assembler of PSCHED_GET_TIME is
> exactly the same on x86. There are 10 architectures that return
> something non-zero for get_cycles(), but I have no idea if it is
> suitable on all of them. Patch applies on top of the dead-code
> removal patch.

This looks great.

As you mention some platforms return zero, for example sparc32,
for get_cycles().

I suggest we just expand the dependency list for NET_SCH_CLK_TSC
to include SPARC64 PPC64 and perhaps some other easy to verify
as having a working get_cycles() implementation.  I believe that
as long as it increments at some rate >= jiffies, the psched
calibration will get things into a working state.

A lot of patches have been posted in this area and I'm losing
track of what to apply first etc.  Can you repost your work
one change at a time?  Thanks.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

David S. Miller wrote:
> This looks great.
> 
> As you mention some platforms return zero, for example sparc32,
> for get_cycles().
> 
> I suggest we just expand the dependency list for NET_SCH_CLK_TSC
> to include SPARC64 PPC64 and perhaps some other easy to verify
> as having a working get_cycles() implementation.  I believe that
> as long as it increments at some rate >= jiffies, the psched
> calibration will get things into a working state.

It needs to increment at slightly above 1Mhz, otherwise delay will
be zero after this division and everything will fall apart:
delay /= rdelay.

> A lot of patches have been posted in this area and I'm losing
> track of what to apply first etc.  Can you repost your work
> one change at a time?  Thanks.

The following two patches remove some dead timer code and
change PSCHED_GET_TIME to use get_cycles. I'm going to send
the configurable clock-source patch tomorrow after checking
the arches.

Regards
Patrick

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Fri, 23 Jul 2004 01:36:22 +0200
Patrick McHardy <kaber@trash.net> wrote:

> > I suggest we just expand the dependency list for NET_SCH_CLK_TSC
> > to include SPARC64 PPC64 and perhaps some other easy to verify
> > as having a working get_cycles() implementation.  I believe that
> > as long as it increments at some rate >= jiffies, the psched
> > calibration will get things into a working state.
> 
> It needs to increment at slightly above 1Mhz, otherwise delay will
> be zero after this division and everything will fall apart:
> delay /= rdelay.

I see.  I know for a fact that sparc64 meets this criterion, and
I'm pretty sure ppc64 does too.

We could bug check this in psched calibration, in fact I think
we should.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

David S. Miller wrote:
>>It needs to increment at slightly above 1Mhz, otherwise delay will
>>be zero after this division and everything will fall apart:
>>delay /= rdelay.
> 
> I see.  I know for a fact that sparc64 meets this criterion, and
> I'm pretty sure ppc64 does too.

I'm pretty sure x86, x86_64, alpha, sparc64, ppc64 and ia64 can
be used. I'm not sure if the frequency of all ppcs is high enough,
so I won't add support for them.

> We could bug check this in psched calibration, in fact I think
> we should.

Done by this patch. I used BUG_ON instead of just printing a warning
because using packet schedulers after this error will cause a division
by zero.

Regards
Patrick

[-- Attachment #2: psched-bug_on_invalid_cycles.diff --]
[-- Type: application/octect-stream, Size: 761 bytes --]

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

Patrick McHardy wrote:
> David S. Miller wrote:
>> We could bug check this in psched calibration, in fact I think
>> we should.
> 
> 
> Done by this patch. I used BUG_ON instead of just printing a warning
> because using packet schedulers after this error will cause a division
> by zero.

Please drop this patch, psched_calibrate_clock already checks if
rdelay > delay a couple of lines above.

> Regards
> Patrick

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Fri, 23 Jul 2004 03:00:52 +0200
Patrick McHardy <kaber@trash.net> wrote:

> Patrick McHardy wrote:
> > David S. Miller wrote:
> >> We could bug check this in psched calibration, in fact I think
> >> we should.
> > 
> > 
> > Done by this patch. I used BUG_ON instead of just printing a warning
> > because using packet schedulers after this error will cause a division
> > by zero.
> 
> Please drop this patch, psched_calibrate_clock already checks if
> rdelay > delay a couple of lines above.

Ok.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 463 bytes --]

Patrick McHardy wrote:
> I'm pretty sure x86, x86_64, alpha, sparc64, ppc64 and ia64 can
> be used. I'm not sure if the frequency of all ppcs is high enough,
> so I won't add support for them.

This is the patch for configurable clock source. I've double-checked
the arches, I think all mentioned above work fine. I've taken some text
from Stephen's patch for the help text .. thanks ;)

Dave, do you want me to provide patches for 2.4 as well ?

Regards
Patrick

[-- Attachment #2: psched-configurable_clock_source.diff --]
[-- Type: text/plain, Size: 9377 bytes --]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/07/23 18:57:36+02:00 kaber@trash.net 
#   [PKT_SCHED]: Make clock source configurable
#   
#   Signed-off-by: Patrick McHardy <kaber@trash.net>
# 
# net/sched/sch_htb.c
#   2004/07/23 18:57:24+02:00 kaber@trash.net +10 -0
#   [PKT_SCHED]: Make clock source configurable
# 
# net/sched/sch_hfsc.c
#   2004/07/23 18:57:24+02:00 kaber@trash.net +5 -5
#   [PKT_SCHED]: Make clock source configurable
# 
# net/sched/sch_api.c
#   2004/07/23 18:57:24+02:00 kaber@trash.net +4 -4
#   [PKT_SCHED]: Make clock source configurable
# 
# net/sched/Kconfig
#   2004/07/23 18:57:24+02:00 kaber@trash.net +55 -0
#   [PKT_SCHED]: Make clock source configurable
# 
# include/net/pkt_sched.h
#   2004/07/23 18:57:24+02:00 kaber@trash.net +11 -22
#   [PKT_SCHED]: Make clock source configurable
# 
diff -Nru a/include/net/pkt_sched.h b/include/net/pkt_sched.h
--- a/include/net/pkt_sched.h	2004-07-23 18:59:48 +02:00
+++ b/include/net/pkt_sched.h	2004-07-23 18:59:48 +02:00
@@ -1,12 +1,6 @@
 #ifndef __NET_PKT_SCHED_H
 #define __NET_PKT_SCHED_H
 
-#define PSCHED_GETTIMEOFDAY	1
-#define PSCHED_JIFFIES 		2
-#define PSCHED_CPU 		3
-
-#define PSCHED_CLOCK_SOURCE	PSCHED_JIFFIES
-
 #include <linux/config.h>
 #include <linux/netdevice.h>
 #include <linux/types.h>
@@ -179,25 +173,19 @@
    The reason is that, when it is not the same thing as
    gettimeofday, it returns invalid timestamp, which is
    not updated, when net_bh is active.
-
-   So, use PSCHED_CLOCK_SOURCE = PSCHED_CPU on alpha and pentiums
-   with rtdsc. And PSCHED_JIFFIES on all other architectures, including [34]86
-   and pentiums without rtdsc.
-   You can use PSCHED_GETTIMEOFDAY on another architectures,
-   which have fast and precise clock source, but it is too expensive.
  */
 
 /* General note about internal clock.
 
    Any clock source returns time intervals, measured in units
-   close to 1usec. With source PSCHED_GETTIMEOFDAY it is precisely
+   close to 1usec. With source CONFIG_NET_SCH_CLK_GETTIMEOFDAY it is precisely
    microseconds, otherwise something close but different chosen to minimize
    arithmetic cost. Ratio usec/internal untis in form nominator/denominator
    may be read from /proc/net/psched.
  */
 
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_GETTIMEOFDAY
+#ifdef CONFIG_NET_SCH_CLK_GETTIMEOFDAY
 
 typedef struct timeval	psched_time_t;
 typedef long		psched_tdiff_t;
@@ -206,12 +194,12 @@
 #define PSCHED_US2JIFFIE(usecs) (((usecs)+(1000000/HZ-1))/(1000000/HZ))
 #define PSCHED_JIFFIE2US(delay) ((delay)*(1000000/HZ))
 
-#else /* PSCHED_CLOCK_SOURCE != PSCHED_GETTIMEOFDAY */
+#else /* !CONFIG_NET_SCH_CLK_GETTIMEOFDAY */
 
 typedef u64	psched_time_t;
 typedef long	psched_tdiff_t;
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_JIFFIES
+#ifdef CONFIG_NET_SCH_CLK_JIFFIES
 
 #if HZ < 96
 #define PSCHED_JSCALE 14
@@ -229,7 +217,8 @@
 #define PSCHED_US2JIFFIE(delay) (((delay)+(1<<PSCHED_JSCALE)-1)>>PSCHED_JSCALE)
 #define PSCHED_JIFFIE2US(delay) ((delay)<<PSCHED_JSCALE)
 
-#elif PSCHED_CLOCK_SOURCE == PSCHED_CPU
+#endif /* CONFIG_NET_SCH_CLK_JIFFIES */
+#ifdef CONFIG_NET_SCH_CLK_CPU
 #include <asm/timex.h>
 
 extern psched_tdiff_t psched_clock_per_hz;
@@ -252,11 +241,11 @@
 #define PSCHED_US2JIFFIE(delay) (((delay)+psched_clock_per_hz-1)/psched_clock_per_hz)
 #define PSCHED_JIFFIE2US(delay) ((delay)*psched_clock_per_hz)
 
-#endif /* PSCHED_CLOCK_SOURCE == PSCHED_JIFFIES */
+#endif /* CONFIG_NET_SCH_CLK_CPU */
 
-#endif /* PSCHED_CLOCK_SOURCE == PSCHED_GETTIMEOFDAY */
+#endif /* !CONFIG_NET_SCH_CLK_GETTIMEOFDAY */
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_GETTIMEOFDAY
+#ifdef CONFIG_NET_SCH_CLK_GETTIMEOFDAY
 #define PSCHED_TDIFF(tv1, tv2) \
 ({ \
 	   int __delta_sec = (tv1).tv_sec - (tv2).tv_sec; \
@@ -320,7 +309,7 @@
 
 #define	PSCHED_AUDIT_TDIFF(t) ({ if ((t) > 2000000) (t) = 2000000; })
 
-#else
+#else /* !CONFIG_NET_SCH_CLK_GETTIMEOFDAY */
 
 #define PSCHED_TDIFF(tv1, tv2) (long)((tv1) - (tv2))
 #define PSCHED_TDIFF_SAFE(tv1, tv2, bound) \
@@ -334,7 +323,7 @@
 #define PSCHED_IS_PASTPERFECT(t)	((t) == 0)
 #define	PSCHED_AUDIT_TDIFF(t)
 
-#endif
+#endif /* !CONFIG_NET_SCH_CLK_GETTIMEOFDAY */
 
 struct tcf_police
 {
diff -Nru a/net/sched/Kconfig b/net/sched/Kconfig
--- a/net/sched/Kconfig	2004-07-23 18:59:48 +02:00
+++ b/net/sched/Kconfig	2004-07-23 18:59:48 +02:00
@@ -1,6 +1,61 @@
 #
 # Traffic control configuration.
 # 
+choice
+	prompt "Packet scheduler clock source"
+	depends on NET_SCHED
+	default NET_SCH_CLK_JIFFIES
+	help
+	  Packet schedulers need a monotonic clock that increments at a static
+	  rate. The kernel provides several suitable interfaces, each with
+	  different properties:
+	  
+	  - high resolution (us or better)
+	  - fast to read (minimal locking, no i/o access)
+	  - synchronized on all processors
+	  - handles cpu clock frequency changes
+
+	  but nothing provides all of the above.
+
+config NET_SCH_CLK_JIFFIES
+	bool "Timer interrupt"
+	help
+	  Say Y here if you want to use the timer interrupt (jiffies) as clock
+	  source. This clock source is fast, synchronized on all processors and
+	  handles cpu clock frequency changes, but its resolution is too low
+	  for accurate shaping except at very low speed.
+
+config NET_SCH_CLK_GETTIMEOFDAY
+	bool "gettimeofday"
+	help
+	  Say Y here if you want to use gettimeofday as clock source. This clock
+	  source has high resolution, is synchronized on all processors and
+	  handles cpu clock frequency changes, but it is slow.
+
+	  Choose this if you need a high resolution clock source but can't use
+	  the CPU's cycle counter.
+
+config NET_SCH_CLK_CPU
+	bool "CPU cycle counter"
+	depends on X86_TSC || X86_64 || ALPHA || SPARC64 || PPC64 || IA64
+	help
+	  Say Y here if you want to use the CPU's cycle counter as clock source.
+	  This is a cheap and high resolution clock source, but on some
+	  architectures it is not synchronized on all processors and doesn't
+	  handle cpu clock frequency changes.
+
+	  The useable cycle counters are:
+
+	  	x86/x86_64	- Timestamp Counter
+		alpha		- Cycle Counter
+		sparc64		- %ticks register
+		ppc64		- Time base
+		ia64		- Interval Time Counter
+
+	  Choose this if your CPU's cycle counter is working properly.
+
+endchoice
+
 config NET_SCH_CBQ
 	tristate "CBQ packet scheduler"
 	depends on NET_SCHED
diff -Nru a/net/sched/sch_api.c b/net/sched/sch_api.c
--- a/net/sched/sch_api.c	2004-07-23 18:59:48 +02:00
+++ b/net/sched/sch_api.c	2004-07-23 18:59:48 +02:00
@@ -1088,7 +1088,7 @@
 };	
 #endif
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_GETTIMEOFDAY
+#ifdef CONFIG_NET_SCH_CLK_GETTIMEOFDAY
 int psched_tod_diff(int delta_sec, int bound)
 {
 	int delta;
@@ -1103,7 +1103,7 @@
 EXPORT_SYMBOL(psched_tod_diff);
 #endif
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_CPU
+#ifdef CONFIG_NET_SCH_CLK_CPU
 psched_tdiff_t psched_clock_per_hz;
 int psched_clock_scale;
 EXPORT_SYMBOL(psched_clock_per_hz);
@@ -1169,10 +1169,10 @@
 {
 	struct rtnetlink_link *link_p;
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_CPU
+#ifdef CONFIG_NET_SCH_CLK_CPU
 	if (psched_calibrate_clock() < 0)
 		return -1;
-#elif PSCHED_CLOCK_SOURCE == PSCHED_JIFFIES
+#elif defined(CONFIG_NET_SCH_CLK_JIFFIES)
 	psched_tick_per_us = HZ<<PSCHED_JSCALE;
 	psched_us_per_tick = 1000000;
 #endif
diff -Nru a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
--- a/net/sched/sch_hfsc.c	2004-07-23 18:59:48 +02:00
+++ b/net/sched/sch_hfsc.c	2004-07-23 18:59:48 +02:00
@@ -193,7 +193,7 @@
 /*
  * macros
  */
-#if PSCHED_CLOCK_SOURCE == PSCHED_GETTIMEOFDAY
+#ifdef CONFIG_NET_SCH_CLK_GETTIMEOFDAY
 #include <linux/time.h>
 #undef PSCHED_GET_TIME
 #define PSCHED_GET_TIME(stamp)						\
@@ -429,10 +429,10 @@
  *	ism: (psched_us/byte) << ISM_SHIFT
  *	dx: psched_us
  *
- * Time source resolution
- *  PSCHED_JIFFIES: for 48<=HZ<=1534 resolution is between 0.63us and 1.27us.
- *  PSCHED_CPU: resolution is between 0.5us and 1us.
- *  PSCHED_GETTIMEOFDAY: resolution is exactly 1us.
+ * Clock source resolution (CONFIG_NET_SCH_CLK_*)
+ *  JIFFIES: for 48<=HZ<=1534 resolution is between 0.63us and 1.27us.
+ *  CPU: resolution is between 0.5us and 1us.
+ *  GETTIMEOFDAY: resolution is exactly 1us.
  *
  * sm and ism are scaled in order to keep effective digits.
  * SM_SHIFT and ISM_SHIFT are selected to keep at least 4 effective
diff -Nru a/net/sched/sch_htb.c b/net/sched/sch_htb.c
--- a/net/sched/sch_htb.c	2004-07-23 18:59:48 +02:00
+++ b/net/sched/sch_htb.c	2004-07-23 18:59:48 +02:00
@@ -856,8 +856,13 @@
 			if (net_ratelimit())
 				printk(KERN_ERR "HTB: bad diff in charge, cl=%X diff=%lX now=%Lu then=%Lu j=%lu\n",
 				       cl->classid, diff,
+#ifdef CONFIG_NET_SCH_CLK_GETTIMEOFDAY
+				       q->now.tv_sec * 1000000ULL + q->now.tv_usec,
+				       cl->t_c.tv_sec * 1000000ULL + cl->t_c.tv_usec,
+#else
 				       (unsigned long long) q->now,
 				       (unsigned long long) cl->t_c,
+#endif
 				       q->jiffies);
 			diff = 1000;
 		}
@@ -927,8 +932,13 @@
 			if (net_ratelimit())
 				printk(KERN_ERR "HTB: bad diff in events, cl=%X diff=%lX now=%Lu then=%Lu j=%lu\n",
 				       cl->classid, diff,
+#ifdef CONFIG_NET_SCH_CLK_GETTIMEOFDAY
+				       q->now.tv_sec * 1000000ULL + q->now.tv_usec,
+				       cl->t_c.tv_sec * 1000000ULL + cl->t_c.tv_usec,
+#else
 				       (unsigned long long) q->now,
 				       (unsigned long long) cl->t_c,
+#endif
 				       q->jiffies);
 			diff = 1000;
 		}

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Stephen Hemminger]

On Fri, 23 Jul 2004 19:14:48 +0200
Patrick McHardy <kaber@trash.net> wrote:

> Patrick McHardy wrote:
> > I'm pretty sure x86, x86_64, alpha, sparc64, ppc64 and ia64 can
> > be used. I'm not sure if the frequency of all ppcs is high enough,
> > so I won't add support for them.
> 
> This is the patch for configurable clock source. I've double-checked
> the arches, I think all mentioned above work fine. I've taken some text
> from Stephen's patch for the help text .. thanks ;)
> 
> Dave, do you want me to provide patches for 2.4 as well ?
> 
> Regards
> Patrick
> 

Looks great, glad to see this cleaned up.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Fri, 23 Jul 2004 19:14:48 +0200
Patrick McHardy <kaber@trash.net> wrote:

> Patrick McHardy wrote:
> > I'm pretty sure x86, x86_64, alpha, sparc64, ppc64 and ia64 can
> > be used. I'm not sure if the frequency of all ppcs is high enough,
> > so I won't add support for them.
> 
> This is the patch for configurable clock source. I've double-checked
> the arches, I think all mentioned above work fine. I've taken some text
> from Stephen's patch for the help text .. thanks ;)

We can't use this stuff on Alpha, it's cycle counter overflows after
just 10 minutes.  It works very strangely, something like only the
lower 32-bits are guarenteed to be continually incrementing.

I'm going to apply your patch and delete the Alpha parts.
Meanwhile, ping Richard Henderson (rth@redhat.com) or one
of the other Alpha experts for me to get confirmation on
this stuff.

Thanks.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

David S. Miller wrote:
> We can't use this stuff on Alpha, it's cycle counter overflows after
> just 10 minutes.  It works very strangely, something like only the
> lower 32-bits are guarenteed to be continually incrementing.

The code (although racy) can handle 32-bit cycle counters. get_cycles()
on alpha returns a 32-bit value, so psched_tick calls PSCHED_GET_TIME
to adjust psched_time_base once a second. It is basically the same as
before. This is the second condition for this too work, besides
get_cycles() incrementing at >1MHz, it needs to return either a 32-bit
value and really use the full 32 bit or return something bigger, which
is assumed not to overflow. Alpha seems to satisfy both conditions.

> I'm going to apply your patch and delete the Alpha parts.
> Meanwhile, ping Richard Henderson (rth@redhat.com) or one
> of the other Alpha experts for me to get confirmation on
> this stuff.

Even better.

Regards
Patrick

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Sat, 24 Jul 2004 01:18:50 +0200
Patrick McHardy <kaber@trash.net> wrote:

> > I'm going to apply your patch and delete the Alpha parts.
> > Meanwhile, ping Richard Henderson (rth@redhat.com) or one
> > of the other Alpha experts for me to get confirmation on
> > this stuff.
> 
> Even better.

Let me know if you get feedback or not.

For now I'm pushing to Linus with Alpha removed from the
list.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

David S. Miller wrote:
> Let me know if you get feedback or not.
> 
> For now I'm pushing to Linus with Alpha removed from the
> list.

According to Richard Henderson it should work fine. The fastest
Alpha cycle counter (1.2 GHz) takes 3.57 seconds to overflow.

Regards
Patrick

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Sun, 25 Jul 2004 23:56:51 +0200
Patrick McHardy <kaber@trash.net> wrote:

> David S. Miller wrote:
> > Let me know if you get feedback or not.
> > 
> > For now I'm pushing to Linus with Alpha removed from the
> > list.
> 
> According to Richard Henderson it should work fine. The fastest
> Alpha cycle counter (1.2 GHz) takes 3.57 seconds to overflow.

That's word enough for me, I'll put ALPHA back into the list.
Thanks.

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[David S. Miller]

On Fri, 23 Jul 2004 19:14:48 +0200
Patrick McHardy <kaber@trash.net> wrote:

> Dave, do you want me to provide patches for 2.4 as well ?

Sure, give it a shot.  If it gets too ugly I may not take
it though, so be warned :-)

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Patrick McHardy]

David S. Miller wrote:
> On Fri, 23 Jul 2004 19:14:48 +0200
> Patrick McHardy <kaber@trash.net> wrote:
> 
> 
>>Dave, do you want me to provide patches for 2.4 as well ?
> 
> 
> Sure, give it a shot.  If it gets too ugly I may not take
> it though, so be warned :-)
> 

I've tried it before, IIRC the 2.4 config system doesn't allow
to express this nicely, so it probably will get ugly. I'm going
to try, but just throw it away if it gets too ugly ..

Regards
Patrick

Re: [PATCH 2.6]: Make packet scheduler clock source configurable

[Stephen Hemminger]

Here is the patch I have been testing against 2.6.8-rc1 to use sched_clock.

diff -Nru a/include/net/pkt_sched.h b/include/net/pkt_sched.h
--- a/include/net/pkt_sched.h	2004-07-12 11:06:52 -07:00
+++ b/include/net/pkt_sched.h	2004-07-12 11:06:52 -07:00
@@ -16,11 +16,6 @@
 #include <linux/module.h>
 #include <linux/rtnetlink.h>
 
-#ifdef CONFIG_X86_TSC
-#include <asm/msr.h>
-#endif
-
-
 struct rtattr;
 struct Qdisc;
 
@@ -179,16 +174,19 @@
    clock evaluated by integration of network data flow
    in the most critical places.
 
-   Note: we do not use fastgettimeofday.
-   The reason is that, when it is not the same thing as
-   gettimeofday, it returns invalid timestamp, which is
-   not updated, when net_bh is active.
-
-   So, use PSCHED_CLOCK_SOURCE = PSCHED_CPU on alpha and pentiums
-   with rtdsc. And PSCHED_JIFFIES on all other architectures, including [34]86
-   and pentiums without rtdsc.
-   You can use PSCHED_GETTIMEOFDAY on another architectures,
-   which have fast and precise clock source, but it is too expensive.
+   The problem is that there are kernel interfaces that provide:
+
+   - high resolution (us or better)
+   - fast to read (minimal locking, no i/o access)
+   - synchronized on all processors and all architectures
+   - handles cpu clock frequency changes
+
+   but no interface provides all of the above. By default
+   the current choice is to use PSCHED_JIFFIES because it is fast
+   to read, synchronized, and stable with frequency changes
+
+   You can use PSCHED_CPU which maps to sched_clock on architectures 
+   that have synchronized clocks. 
  */
 
 /* General note about internal clock.
@@ -237,39 +235,12 @@
 
 #elif PSCHED_CLOCK_SOURCE == PSCHED_CPU
 
-extern psched_tdiff_t psched_clock_per_hz;
-extern int psched_clock_scale;
-
-#define PSCHED_US2JIFFIE(delay) (((delay)+psched_clock_per_hz-1)/psched_clock_per_hz)
-#define PSCHED_JIFFIE2US(delay) ((delay)*psched_clock_per_hz)
-
-#ifdef CONFIG_X86_TSC
-
-#define PSCHED_GET_TIME(stamp) \
-({ u64 __cur; \
-   rdtscll(__cur); \
-   (stamp) = __cur>>psched_clock_scale; \
-})
-
-#elif defined (__alpha__)
-
-#define PSCHED_WATCHER u32
-
-extern PSCHED_WATCHER psched_time_mark;
-
-#define PSCHED_GET_TIME(stamp) \
-({ u32 __res; \
-   __asm__ __volatile__ ("rpcc %0" : "r="(__res)); \
-   if (__res <= psched_time_mark) psched_time_base += 0x100000000UL; \
-   psched_time_mark = __res; \
-   (stamp) = (psched_time_base + __res)>>psched_clock_scale; \
-})
+extern unsigned long long sched_clock(void);
 
-#else
-
-#error PSCHED_CLOCK_SOURCE=PSCHED_CPU is not supported on this arch.
-
-#endif /* ARCH */
+#define PSCHED_GET_TIME(stamp)	\
+	do { (stamp) = sched_clock(); do_div(stamp, 1000); } while(0)
+#define PSCHED_US2JIFFIE(usecs) (((usecs)+(1000000/HZ-1))/(1000000/HZ))
+#define PSCHED_JIFFIE2US(delay)((delay)*(1000000/HZ))
 
 #endif /* PSCHED_CLOCK_SOURCE == PSCHED_JIFFIES */
 
@@ -496,7 +467,7 @@
 /* Calculate maximal size of packet seen by hard_start_xmit
    routine of this device.
  */
-static inline unsigned psched_mtu(struct net_device *dev)
+static inline unsigned psched_mtu(const struct net_device *dev)
 {
 	unsigned mtu = dev->mtu;
 	return dev->hard_header ? mtu + dev->hard_header_len : mtu;
diff -Nru a/net/sched/sch_api.c b/net/sched/sch_api.c
--- a/net/sched/sch_api.c	2004-07-12 11:06:52 -07:00
+++ b/net/sched/sch_api.c	2004-07-12 11:06:52 -07:00
@@ -1101,91 +1101,17 @@
 EXPORT_SYMBOL(psched_tod_diff);
 #endif
 
-psched_time_t psched_time_base;
-
-#if PSCHED_CLOCK_SOURCE == PSCHED_CPU
-psched_tdiff_t psched_clock_per_hz;
-int psched_clock_scale;
-EXPORT_SYMBOL(psched_clock_per_hz);
-EXPORT_SYMBOL(psched_clock_scale);
-#endif
-
-#ifdef PSCHED_WATCHER
-PSCHED_WATCHER psched_time_mark;
-EXPORT_SYMBOL(psched_time_mark);
-EXPORT_SYMBOL(psched_time_base);
-
-static void psched_tick(unsigned long);
-
-static struct timer_list psched_timer = TIMER_INITIALIZER(psched_tick, 0, 0);
-
-static void psched_tick(unsigned long dummy)
-{
-#if PSCHED_CLOCK_SOURCE == PSCHED_CPU
-	psched_time_t dummy_stamp;
-	PSCHED_GET_TIME(dummy_stamp);
-	/* It is OK up to 4GHz cpu */
-	psched_timer.expires = jiffies + 1*HZ;
-#else
-	unsigned long now = jiffies;
-	psched_time_base += ((u64)(now-psched_time_mark))<<PSCHED_JSCALE;
-	psched_time_mark = now;
-	psched_timer.expires = now + 60*60*HZ;
-#endif
-	add_timer(&psched_timer);
-}
-#endif
-
 #if PSCHED_CLOCK_SOURCE == PSCHED_CPU
-int __init psched_calibrate_clock(void)
-{
-	psched_time_t stamp, stamp1;
-	struct timeval tv, tv1;
-	psched_tdiff_t delay;
-	long rdelay;
-	unsigned long stop;
-
-#ifdef PSCHED_WATCHER
-	psched_tick(0);
-#endif
-	stop = jiffies + HZ/10;
-	PSCHED_GET_TIME(stamp);
-	do_gettimeofday(&tv);
-	while (time_before(jiffies, stop)) {
-		barrier();
-		cpu_relax();
-	}
-	PSCHED_GET_TIME(stamp1);
-	do_gettimeofday(&tv1);
-
-	delay = PSCHED_TDIFF(stamp1, stamp);
-	rdelay = tv1.tv_usec - tv.tv_usec;
-	rdelay += (tv1.tv_sec - tv.tv_sec)*1000000;
-	if (rdelay > delay)
-		return -1;
-	delay /= rdelay;
-	psched_tick_per_us = delay;
-	while ((delay>>=1) != 0)
-		psched_clock_scale++;
-	psched_us_per_tick = 1<<psched_clock_scale;
-	psched_clock_per_hz = (psched_tick_per_us*(1000000/HZ))>>psched_clock_scale;
-	return 0;
-}
+EXPORT_SYMBOL(sched_clock);
 #endif
 
 static int __init pktsched_init(void)
 {
 	struct rtnetlink_link *link_p;
 
-#if PSCHED_CLOCK_SOURCE == PSCHED_CPU
-	if (psched_calibrate_clock() < 0)
-		return -1;
-#elif PSCHED_CLOCK_SOURCE == PSCHED_JIFFIES
+#if PSCHED_CLOCK_SOURCE == PSCHED_JIFFIES
 	psched_tick_per_us = HZ<<PSCHED_JSCALE;
 	psched_us_per_tick = 1000000;
-#ifdef PSCHED_WATCHER
-	psched_tick(0);
-#endif
 #endif
 
 	link_p = rtnetlink_links[PF_UNSPEC];