All of lore.kernel.org
 help / color / mirror / Atom feed
From: Simon Kirby <sim@hostway.ca>
To: lvs-devel@vger.kernel.org
Cc: Changli Gao <xiaosuo@gmail.com>
Subject: activeconns * weight overflowing 32-bit int
Date: Fri, 12 Apr 2013 23:43:55 -0700	[thread overview]
Message-ID: <20130413064355.GB24319@hostway.ca> (raw)

Hello!

We use lblc in some environments to try to maintain some cache locality.

We recently had some problems upgrading beyond 2.6.38 in one environment.
The cluster kept overloading real servers and showed flapping that didn't
occur on 2.6.38 and older. I was never able to figure this out, but I
think now I see the reason.

We need to use fairly high weights, since lblc requires this in order to
do rescheduling in the event of overload. In the event that we have 3000
activeconns to a real server and a weight of 3000, the next connection
will check to see if any other real server has 2*activeconns less than
its weight, and if so, reschedule by wlc.

With b552f7e3a9524abcbcdf86f0a99b2be58e55a9c6, which "git tag --contains"
says appeared in 2.6.39-rc, the open-coded activeconns * 50 + inactconns
was changed to ip_vs_dest_conn_overhead() that matches the implementation
in ip_vs_wlc.c and others. The problem for us is that ip_vs_lblc.c uses
"int" (and wlc uses "unsigned int") for "loh" and "doh" variables that
the ip_vs_dest_conn_overhead() result is stored in, and then these are
multiplied by the weight.

ip_vs_dest_conn_overhead() uses (activeconns << 8) + inactconns (* 256
instead of * 50), so before where 3000 * 3000 * 50 would fit in an int,
3000 * 3000 * 256 does not.

We really don't care about inactconns, so removing the "<< 8" and just
using activeconns would work for us, but I suspect it must be there for a
raeason. "unsigned long" would fix the problem only for 64-bit arches.
Using __u64 would work everywhere, but perhaps be slow on 32-bit arches.
Thoughts?

Simon-

             reply	other threads:[~2013-04-13  6:43 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-04-13  6:43 Simon Kirby [this message]
2013-04-13 15:10 ` activeconns * weight overflowing 32-bit int Julian Anastasov
2013-05-22  6:18   ` Julian Anastasov
2013-05-23 16:58     ` Simon Kirby
2013-05-23 20:44       ` Julian Anastasov
2013-05-24  0:43         ` Simon Kirby
2013-05-24  8:11           ` Julian Anastasov
2013-08-05  6:10             ` Julian Anastasov
2013-08-06  2:41             ` Simon Kirby
2013-08-06  6:45               ` Julian Anastasov
2013-08-08 23:54                 ` [PATCH] ipvs: Use 64-bit comparisons (connections * weight) to avoid overflow Simon Kirby
2013-08-09  9:02                   ` Julian Anastasov
2013-08-10  8:26                     ` [PATCH v2] ipvs: fix overflow on dest weight multiply Simon Kirby
2013-08-10 12:31                       ` Julian Anastasov
2013-08-13  2:23                         ` Simon Horman
2013-08-13  4:45                           ` Simon Horman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130413064355.GB24319@hostway.ca \
    --to=sim@hostway.ca \
    --cc=lvs-devel@vger.kernel.org \
    --cc=xiaosuo@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.