Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Andrew Morton]

On Tue, 29 May 2007 11:01:13 -0700
Venki Pallipadi <venkatesh.pallipadi@intel.com> wrote:

> round_jiffies for net dev watchdog timer.
> 
> Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
> 
> Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> ===================================================================
> --- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> +++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> @@ -224,7 +224,8 @@
>  	if (dev->tx_timeout) {
>  		if (dev->watchdog_timeo <= 0)
>  			dev->watchdog_timeo = 5*HZ;
> -		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> +		if (!mod_timer(&dev->watchdog_timer,
> +			       round_jiffies(jiffies + dev->watchdog_timeo)))
>  			dev_hold(dev);
>  	}
>  }

Please cc netdev on net patches.

Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
then they will get one second, which is grossly different.

And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
significant, too.

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Stephen Hemminger]

On Wed, 30 May 2007 10:59:36 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Tue, 29 May 2007 11:01:13 -0700
> Venki Pallipadi <venkatesh.pallipadi@intel.com> wrote:
> 
> > round_jiffies for net dev watchdog timer.
> > 
> > Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
> > 
> > Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> > ===================================================================
> > --- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> > +++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> > @@ -224,7 +224,8 @@
> >  	if (dev->tx_timeout) {
> >  		if (dev->watchdog_timeo <= 0)
> >  			dev->watchdog_timeo = 5*HZ;
> > -		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> > +		if (!mod_timer(&dev->watchdog_timer,
> > +			       round_jiffies(jiffies + dev->watchdog_timeo)))
> >  			dev_hold(dev);
> >  	}
> >  }
> 
> Please cc netdev on net patches.
> 
> Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> then they will get one second, which is grossly different.
> 
> And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> significant, too.

Alternatively, we could change to a timer that is pushed forward after each
TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
never run in normal case.

-- 
Stephen Hemminger <shemminger@linux-foundation.org>

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Patrick McHardy]

Stephen Hemminger wrote:
>>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
>>>===================================================================
>>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
>>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
>>>@@ -224,7 +224,8 @@
>>> 	if (dev->tx_timeout) {
>>> 		if (dev->watchdog_timeo <= 0)
>>> 			dev->watchdog_timeo = 5*HZ;
>>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
>>>+		if (!mod_timer(&dev->watchdog_timer,
>>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
>>> 			dev_hold(dev);
>>> 	}
>>> }
>>
>>Please cc netdev on net patches.
>>
>>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
>>then they will get one second, which is grossly different.
>>
>>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
>>significant, too.
> 
> 
> Alternatively, we could change to a timer that is pushed forward after each
> TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> never run in normal case.


It seems wasteful to add per-packet overhead for tx timeouts, which
should be an exception. Do drivers really care about the exact
timeout value? Compared to a packet transmission time its incredibly
long anyways ..

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Venki Pallipadi]

On Wed, May 30, 2007 at 08:42:32PM +0200, Patrick McHardy wrote:
> Stephen Hemminger wrote:
> >>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> >>>===================================================================
> >>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> >>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> >>>@@ -224,7 +224,8 @@
> >>> 	if (dev->tx_timeout) {
> >>> 		if (dev->watchdog_timeo <= 0)
> >>> 			dev->watchdog_timeo = 5*HZ;
> >>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> >>>+		if (!mod_timer(&dev->watchdog_timer,
> >>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
> >>> 			dev_hold(dev);
> >>> 	}
> >>> }
> >>
> >>Please cc netdev on net patches.
> >>
> >>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> >>then they will get one second, which is grossly different.
> >>
> >>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> >>significant, too.
> > 
> > 
> > Alternatively, we could change to a timer that is pushed forward after each
> > TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> > never run in normal case.
> 
> 
> It seems wasteful to add per-packet overhead for tx timeouts, which
> should be an exception. Do drivers really care about the exact
> timeout value? Compared to a packet transmission time its incredibly
> long anyways ..

I agree. Doing a mod_timer or hrtimer_forward to push forward may add to the
complexity depending on how often TX happens.

Are the drivers really worried about exact timeouts here? Can we use rounding
for the timers that are more than a second, at least?

Thanks,
Venki

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Patrick McHardy]

Venki Pallipadi wrote:
> On Wed, May 30, 2007 at 08:42:32PM +0200, Patrick McHardy wrote:
> 
>>
>>It seems wasteful to add per-packet overhead for tx timeouts, which
>>should be an exception. Do drivers really care about the exact
>>timeout value? Compared to a packet transmission time its incredibly
>>long anyways ..
> 
> 
> I agree. Doing a mod_timer or hrtimer_forward to push forward may add to the
> complexity depending on how often TX happens.
> 
> Are the drivers really worried about exact timeouts here?


Just guessing, but I don't think they are, after all timers can be late.

> Can we use rounding for the timers that are more than a second, at least?


Also sounds reasonable.

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[David Miller]

From: Patrick McHardy <kaber@trash.net>
Date: Wed, 30 May 2007 20:42:32 +0200

> Stephen Hemminger wrote:
> >>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> >>>===================================================================
> >>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> >>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> >>>@@ -224,7 +224,8 @@
> >>> 	if (dev->tx_timeout) {
> >>> 		if (dev->watchdog_timeo <= 0)
> >>> 			dev->watchdog_timeo = 5*HZ;
> >>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> >>>+		if (!mod_timer(&dev->watchdog_timer,
> >>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
> >>> 			dev_hold(dev);
> >>> 	}
> >>> }
> >>
> >>Please cc netdev on net patches.
> >>
> >>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> >>then they will get one second, which is grossly different.
> >>
> >>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> >>significant, too.
> > 
> > 
> > Alternatively, we could change to a timer that is pushed forward after each
> > TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> > never run in normal case.
> 
> 
> It seems wasteful to add per-packet overhead for tx timeouts, which
> should be an exception. Do drivers really care about the exact
> timeout value? Compared to a packet transmission time its incredibly
> long anyways ..

I agree, this change is absolutely rediculious and is just a blind
cookie-cutter change made without consideration of what the code is
doing and what it's requirements are.

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Stephen Hemminger]

On Wed, 30 May 2007 12:55:51 -0700 (PDT)
David Miller <davem@davemloft.net> wrote:

> From: Patrick McHardy <kaber@trash.net>
> Date: Wed, 30 May 2007 20:42:32 +0200
> 
> > Stephen Hemminger wrote:
> > >>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> > >>>===================================================================
> > >>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> > >>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> > >>>@@ -224,7 +224,8 @@
> > >>> 	if (dev->tx_timeout) {
> > >>> 		if (dev->watchdog_timeo <= 0)
> > >>> 			dev->watchdog_timeo = 5*HZ;
> > >>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> > >>>+		if (!mod_timer(&dev->watchdog_timer,
> > >>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
> > >>> 			dev_hold(dev);
> > >>> 	}
> > >>> }
> > >>
> > >>Please cc netdev on net patches.
> > >>
> > >>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> > >>then they will get one second, which is grossly different.
> > >>
> > >>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> > >>significant, too.
> > > 
> > > 
> > > Alternatively, we could change to a timer that is pushed forward after each
> > > TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> > > never run in normal case.
> > 
> > 
> > It seems wasteful to add per-packet overhead for tx timeouts, which
> > should be an exception. Do drivers really care about the exact
> > timeout value? Compared to a packet transmission time its incredibly
> > long anyways ..
> 
> I agree, this change is absolutely rediculious and is just a blind
> cookie-cutter change made without consideration of what the code is
> doing and what it's requirements are.
> 

what about the obvious compromise:

--- a/net/sched/sch_generic.c	2007-05-30 11:42:18.000000000 -0700
+++ b/net/sched/sch_generic.c	2007-05-30 13:29:34.000000000 -0700
@@ -203,7 +203,11 @@ static void dev_watchdog(unsigned long a
 				       dev->name);
 				dev->tx_timeout(dev);
 			}
-			if (!mod_timer(&dev->watchdog_timer, round_jiffies(jiffies + dev->watchdog_timeo)))
+
+			if (!mod_timer(&dev->watchdog_timer,
+				       dev->watchdog_timeo > 2 * HZ
+				       ? round_jiffies(jiffies + dev->watchdog_timeo)
+				       : jiffies + dev->watchdog_timeo))
 				dev_hold(dev);
 		}
 	}




-- 
Stephen Hemminger <shemminger@linux-foundation.org>

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Venki Pallipadi]

On Wed, May 30, 2007 at 12:55:51PM -0700, David Miller wrote:
> From: Patrick McHardy <kaber@trash.net>
> Date: Wed, 30 May 2007 20:42:32 +0200
> 
> > Stephen Hemminger wrote:
> > >>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> > >>>===================================================================
> > >>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> > >>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> > >>>@@ -224,7 +224,8 @@
> > >>> 	if (dev->tx_timeout) {
> > >>> 		if (dev->watchdog_timeo <= 0)
> > >>> 			dev->watchdog_timeo = 5*HZ;
> > >>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> > >>>+		if (!mod_timer(&dev->watchdog_timer,
> > >>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
> > >>> 			dev_hold(dev);
> > >>> 	}
> > >>> }
> > >>
> > >>Please cc netdev on net patches.
> > >>
> > >>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> > >>then they will get one second, which is grossly different.
> > >>
> > >>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> > >>significant, too.
> > > 
> > > 
> > > Alternatively, we could change to a timer that is pushed forward after each
> > > TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> > > never run in normal case.
> > 
> > 
> > It seems wasteful to add per-packet overhead for tx timeouts, which
> > should be an exception. Do drivers really care about the exact
> > timeout value? Compared to a packet transmission time its incredibly
> > long anyways ..
> 
> I agree, this change is absolutely rediculious and is just a blind
> cookie-cutter change made without consideration of what the code is
> doing and what it's requirements are.

I hope I could atleast highlight the issue here despite the cookie-cutter
patch..
On a totally idle system I have something like 85 wakeups for every 5 seconds
which I am trying to reduce (to reduce the power consumption and increase
battery life. And 1 interrupt out of 85 happens to be netdev watchdog timer.

Thanks,
Venki

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Venki Pallipadi]

On Wed, May 30, 2007 at 01:30:39PM -0700, Stephen Hemminger wrote:
> On Wed, 30 May 2007 12:55:51 -0700 (PDT)
> David Miller <davem@davemloft.net> wrote:
> 
> > From: Patrick McHardy <kaber@trash.net>
> > Date: Wed, 30 May 2007 20:42:32 +0200
> > 
> > > Stephen Hemminger wrote:
> > > >>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> > > >>>===================================================================
> > > >>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> > > >>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> > > >>>@@ -224,7 +224,8 @@
> > > >>> 	if (dev->tx_timeout) {
> > > >>> 		if (dev->watchdog_timeo <= 0)
> > > >>> 			dev->watchdog_timeo = 5*HZ;
> > > >>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> > > >>>+		if (!mod_timer(&dev->watchdog_timer,
> > > >>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
> > > >>> 			dev_hold(dev);
> > > >>> 	}
> > > >>> }
> > > >>
> > > >>Please cc netdev on net patches.
> > > >>
> > > >>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> > > >>then they will get one second, which is grossly different.
> > > >>
> > > >>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> > > >>significant, too.
> > > > 
> > > > 
> > > > Alternatively, we could change to a timer that is pushed forward after each
> > > > TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> > > > never run in normal case.
> > > 
> > > 
> > > It seems wasteful to add per-packet overhead for tx timeouts, which
> > > should be an exception. Do drivers really care about the exact
> > > timeout value? Compared to a packet transmission time its incredibly
> > > long anyways ..
> > 
> > I agree, this change is absolutely rediculious and is just a blind
> > cookie-cutter change made without consideration of what the code is
> > doing and what it's requirements are.
> > 
> 
> what about the obvious compromise:
> 
> --- a/net/sched/sch_generic.c	2007-05-30 11:42:18.000000000 -0700
> +++ b/net/sched/sch_generic.c	2007-05-30 13:29:34.000000000 -0700
> @@ -203,7 +203,11 @@ static void dev_watchdog(unsigned long a
>  				       dev->name);
>  				dev->tx_timeout(dev);
>  			}
> -			if (!mod_timer(&dev->watchdog_timer, round_jiffies(jiffies + dev->watchdog_timeo)))
> +
> +			if (!mod_timer(&dev->watchdog_timer,
> +				       dev->watchdog_timeo > 2 * HZ
> +				       ? round_jiffies(jiffies + dev->watchdog_timeo)
> +				       : jiffies + dev->watchdog_timeo))
>  				dev_hold(dev);
>  		}
>  	}
> 
> 

If this does not work:
Another option is to use 'deferrable timer' here which will be called at
same as before time when CPU is busy and on idle CPU it will be delayed until
CPU comes out of idle due to any other events.

Thanks,
Venki

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Matt Mackall]

On Wed, May 30, 2007 at 12:55:51PM -0700, David Miller wrote:
> From: Patrick McHardy <kaber@trash.net>
> Date: Wed, 30 May 2007 20:42:32 +0200
> 
> > Stephen Hemminger wrote:
> > >>>Index: linux-2.6.22-rc-mm/net/sched/sch_generic.c
> > >>>===================================================================
> > >>>--- linux-2.6.22-rc-mm.orig/net/sched/sch_generic.c	2007-05-24 11:16:03.000000000 -0700
> > >>>+++ linux-2.6.22-rc-mm/net/sched/sch_generic.c	2007-05-25 15:10:02.000000000 -0700
> > >>>@@ -224,7 +224,8 @@
> > >>> 	if (dev->tx_timeout) {
> > >>> 		if (dev->watchdog_timeo <= 0)
> > >>> 			dev->watchdog_timeo = 5*HZ;
> > >>>-		if (!mod_timer(&dev->watchdog_timer, jiffies + dev->watchdog_timeo))
> > >>>+		if (!mod_timer(&dev->watchdog_timer,
> > >>>+			       round_jiffies(jiffies + dev->watchdog_timeo)))
> > >>> 			dev_hold(dev);
> > >>> 	}
> > >>> }
> > >>
> > >>Please cc netdev on net patches.
> > >>
> > >>Again, I worry that if people set the watchdog timeout to, say, 0.1 seconds
> > >>then they will get one second, which is grossly different.
> > >>
> > >>And if they were to set it to 1.5 seconds, they'd get 2.0 which is pretty
> > >>significant, too.
> > > 
> > > 
> > > Alternatively, we could change to a timer that is pushed forward after each
> > > TX, maybe using hrtimer and hrtimer_forward().  That way the timer would
> > > never run in normal case.
> > 
> > 
> > It seems wasteful to add per-packet overhead for tx timeouts, which
> > should be an exception. Do drivers really care about the exact
> > timeout value? Compared to a packet transmission time its incredibly
> > long anyways ..
> 
> I agree, this change is absolutely rediculious and is just a blind
> cookie-cutter change made without consideration of what the code is
> doing and what it's requirements are.

What are you agreeing with, Dave?

Are you agreeing that "it seems wasteful to add per-packet overhead"?
This patch is not doing that.

A quick grep shows that most things are using multi-second timeouts
here. Of the ones that aren't, a number are using .4s, and many more
aren't even in units of HZ. Makes me wonder if the various boards
using 50ms are being overzealous.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[David Miller]

From: Matt Mackall <mpm@selenic.com>
Date: Wed, 30 May 2007 17:10:39 -0500

> Are you agreeing that "it seems wasteful to add per-packet overhead"?
> This patch is not doing that.

Yes, and I know that :-)

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Matt Mackall]

On Wed, May 30, 2007 at 03:29:39PM -0700, David Miller wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 30 May 2007 17:10:39 -0500
> 
> > Are you agreeing that "it seems wasteful to add per-packet overhead"?
> > This patch is not doing that.
> 
> Yes, and I know that :-)

Is there a real reason for watchdog_timeo to be all over the map?

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[David Miller]

From: Matt Mackall <mpm@selenic.com>
Date: Wed, 30 May 2007 17:36:12 -0500

> On Wed, May 30, 2007 at 03:29:39PM -0700, David Miller wrote:
> > From: Matt Mackall <mpm@selenic.com>
> > Date: Wed, 30 May 2007 17:10:39 -0500
> > 
> > > Are you agreeing that "it seems wasteful to add per-packet overhead"?
> > > This patch is not doing that.
> > 
> > Yes, and I know that :-)
> 
> Is there a real reason for watchdog_timeo to be all over the map?

In theory it's a hardware implementation and link speed specific value,
in reality nearly everyone just copies values from other drivers.

Re: [PATCH 3/4] Make net watchdog timers 1 sec jiffy aligned

[Andi Kleen]

Venki Pallipadi <venkatesh.pallipadi@intel.com> writes:
> 
> If this does not work:
> Another option is to use 'deferrable timer' here which will be called at
> same as before time when CPU is busy and on idle CPU it will be delayed until
> CPU comes out of idle due to any other events.

That would sound like a good idea here and at least power wise 
it should be nearly free

-Andi