Re: [PATCH V2] netfilter: remove extra timer from ecache extension

[Pablo Neira Ayuso <pablo@netfilter.org>]

Hi Florian,

On Tue, Dec 04, 2012 at 10:35:51AM +0100, Florian Westphal wrote:
> This brings the (per-conntrack) ecache extension back to 24 bytes in
> size (was 112 byte on x86_64 with lockdep on).
> 
> Instead we use a per-ns tasklet to re-trigger event delivery.  When we
> enqueue a ct entry into the dying list, the tasklet is scheduled.
> 
> The tasklet will then deliver up to 20 entries.  It will re-sched
> itself unless all the pending events could be delivered.
> 
> While at it, dying list handling is moved into ecache.c, since its only
> revlevant if ct events are enabled.

Just tested this. My testbed consists of two firewalls in HA running
conntrackd with event reliable mode. I've got a client that generates
lots of small TCP flows that goes through the firewalls and reach a
benchmark server.

This is my analysis:

conntrack -C shows:

253244
254798
256754
258807
261548 <--- we hit table full, dropping packets
176849 <--- it seems the tasklet gets a chance to run
            given that we get less interruptions from the NIC
166449 <--- it slightly empty the dying list
131176
55602
28316
28317
28317

#  hits  hits/s  ^h/s  ^bytes   kB/s  errs   rst  tout  mhtime
4796894 15727 16509   2393805   2227     0     0     0   0.005
4813038 15728 16144   2340880   2227     0     0     0   0.005
4828796 15728 15758   2284910   2227     0     0     0   0.005
4845279 15731 16483   2390035   2227     0     0     0   0.005
4860956 15731 15677   2273165   2227     0     0     0   0.005
4876826 15731 15870   2301150   2227     0     0     0   0.005
4883165 15701  6339    919155   2223     0     0     0   0.004
4883165 15651     0         0   2216     0     0     0   0.000  <--- table full
4883165 15601     0         0   2209     0     0     0   0.000
4894657 15588 11492   1666340   2207     0     0     0   3.008
4913408 15598 18751   2718895   2208     0     0     0   0.004
4931896 15607 18488   2680760   2210     0     0     0   0.004

So it seems the tasklet gets starved under heavy load.

This happens on and on, so after some time we hit table full and again
the dying list is empty.

These are old HP proliant DL145G2 from 2005, that's why the maximum
flows/s looks low.

Looking at the number and the behaviour under heavy stress, I think we
have to consider a different approach.