A one-liner ipvs patch

A one-liner ipvs patch

[Siim Põder]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

Hi!

I have a few very high connection rate services behind LVS and when
keepalived removes them because of problems, the LVS would be rebooted
with tens of thousands of this message in the logs (rebooted by hardware
watchdogs).

I have had it running on production systems for maybe 3 months or so and
this problem hasn't occured any more, which does not prove that the
patch isn't just curing a symptom, but I think it is concievable, that a
lot of printks could hang a system.

Siim


[-- Attachment #2: ip_vs_wrr-ratelimit-printk.patch --]
[-- Type: text/x-patch, Size: 671 bytes --]

ipvs: ratelimit flooding message in wrr scheduler

When all backends are removed from wrr scheduler by a monitoring agent, this
message can flood so fast that a 10s watchdog would reset the system.

Signed-off-by: Siim Po~der <siim@p6drad-teel.net>
--- linux-2.6.24/net/ipv4/ipvs/ip_vs_wrr.c	2008-01-24 22:58:37.000000000 +0000
+++ linux-2.6.24-ipvs_patches/net/ipv4/ipvs/ip_vs_wrr.c	2008-05-06 16:17:17.790662800 +0000
@@ -169,7 +169,7 @@
 				 */
 				if (mark->cw == 0) {
 					mark->cl = &svc->destinations;
-					IP_VS_INFO("ip_vs_wrr_schedule(): "
+					IP_VS_DBG_RL("ip_vs_wrr_schedule(): "
 						   "no available servers\n");
 					dest = NULL;
 					goto out;

Re: A one-liner ipvs patch

[Simon Horman]

On Wed, Sep 03, 2008 at 11:24:03AM +0300, Siim Põder wrote:
> Hi!
> 
> I have a few very high connection rate services behind LVS and when
> keepalived removes them because of problems, the LVS would be rebooted
> with tens of thousands of this message in the logs (rebooted by hardware
> watchdogs).
> 
> I have had it running on production systems for maybe 3 months or so and
> this problem hasn't occured any more, which does not prove that the
> patch isn't just curing a symptom, but I think it is concievable, that a
> lot of printks could hang a system.

Hi Siim,

When trying to apply this I noticed that it has already been
included as "ipvs: Make wrr "no available servers" error message
rate-limited" by Sven Wegener, which was included in 2.6.25-rc1.

Thanks

Re: A one-liner ipvs patch

[Siim Põder]

Siim Põder wrote:
> I have a few very high connection rate services behind LVS and when
> keepalived removes them because of problems, the LVS would be rebooted
> with tens of thousands of this message in the logs (rebooted by hardware
> watchdogs).
> 
> I have had it running on production systems for maybe 3 months or so and
> this problem hasn't occured any more, which does not prove that the
> patch isn't just curing a symptom, but I think it is concievable, that a
> lot of printks could hang a system.

sorry, this has been fixed in 2.6.25 already.

Siim