Re: Fwd: LVS on local node

[Simon Horman <horms@verge.net.au>]

On Thu, Jul 22, 2010 at 02:59:43PM +0200, Eric Dumazet wrote:
> Le jeudi 22 juillet 2010 à 21:24 +0900, Simon Horman a écrit :
> > On Thu, Jul 22, 2010 at 08:56:51AM +0200, Eric Dumazet wrote:
> > 
> > [snip]
> > 
> > > lvs seems not very SMP friendly and a bit complex.
> > 
> > I'd be interested to hear some thoughts on
> > how the SMP aspect of that statement could
> > be improved.
> 
> Hi Simon
> 
> I am not familiar with LVS code, so I am probably wrong, but it seems it
> could be changed a bit.
> 
> Some rwlocks might become spinlocks (faster than rwlocks)
> 
> __ip_vs_securetcp_lock for example is always used with
> write_lock()/write_unlock().
> This can be a regular spinlock without even knowing the code.

I'll get that fixed.

> Some lookups could use RCU to avoid cache line misses, and to be able to
> use spinlocks for the write side.

Agreed. I took a look at RCUing things a while back, but got bogged
down and then forgot about it. I'll take anther stab at it.

> It would be good to have a bench setup with the case of 16 legacy
> daemons, and check how many new connections per second can be
> established, in an LVS setup and an iptables based one.
> 
> With 2.6.35 and RPS, a REDIRECT based solution can chose the target port
> without taking any lock (not counting conntrack internal costs of
> course), each cpu accessing local memory only.
> 
> # No need is eth0 is a multiqueue NIC
> echo ffff >/sys/class/net/eth0/queues/rx-0/rps_cpus
> 
> for c in `seq 0 15`
> do
>   iptables -t nat -A PREROUTING -p tcp --dport 80 -m cpu --cpu $c -j
> REDIRECT --to-port $((1000 + $c))
> done

Its hard for lvs to compete with those kind of lightweight solutions and
it probably shouldn't. However, I'd just like to see LVS working as
well as it can within the constraint that, as you pointed out, its rather
complex. Thanks for your suggestions.