Re: [PATCH 2/4] deferred drop, __parent workaround, reshape_fail

[Harald Welte <laforge@netfilter.org>]

[-- Attachment #1: Type: text/plain, Size: 3452 bytes --]

On Mon, Aug 16, 2004 at 09:29:59AM -0400, jamal wrote:
> > some later data.  It SHOULD not care whether this later data for the
> > same flow has bigger or smaller byte/packet counters. [and
> > it is very unlikely that the total will be lower, since then in the
> > timeframe [snapshot, terminations] more packets have to be dropped than
> > accepted.  Still, if this is documented with ctnetlink I'm perfectly
> > fine which such behaviour.
> 
> I am too. Good stuff.
> I think 99.9% of accounting would be happy with getting data after the
> call is done; the other 0.01% may have to live with extra packets later
> which undo things. 
> Are you working on something along the IPFIX protocol for transport?

Yes, I am working on the next generation of ulogd (ulogd-2).  It will
allow you to stack multiple plugins on top of each other, such as:

ctnetlink -> ipfix

or 

ulog -> packet parser -> flow_aggrgation -> ipfix 

or even exporting per-packet data in ipfix 

ulog -> packet parser -> ipfix

It's far from complete, but I'm almost finished with the core and can
start to write the plugins:
http://svn.gnumonks.org/branches/ulog/ulogd2/

> Let me think about it.
> Clearly the best place to account for things is on the wire once the
> packet has left the box ;-> 

This is arguable.  Why not once the packet was receieved on the incoming
wire?  If you are Ingress router of your network, your ISP bills you for
the amout of traffic it sends to your ingress router.

OTOTH, for egress traffic I agree.

> How open are you to move accounting further down? My thoughts
> are along the lines of incrementing the contrack counters at the qdisc
> level. Since you transport after the structure has been deleted, it
> should work out fine and fair billing will be taken care of.

Sure, it can be done... but you need to grab ip_conntrack_lock in order
to do so.

> Has someone done experimented and figured how expensive it would be to
> do it at the qdisc level? Note, you can probably have a low level
> grained lock just for stats.

It doesn't help.  Currently we still have one global rwlock for all
conntrack's.  There are patches for per-bucket locking.  

But well, as long as the usage count of ip_conntrack doesn't drop to
zero (which we're guaranteed since our skb still references it), and we
don't need to walk the hash table, we don't actually need to grab
ip_conntrack_lock.   A seperate accounting lock would be possible.

Somebody needs to implement this and run profiles...

> INC_STATS is generic (goes in the generic stats code in attached patch)
> and will have an ifdef for contrack billing (which includes unbilling
> depending on reason code). Reason code could be results that are now
> returned.
> As an example NET_XMIT_DROP is definetely unbilling while
> NET_XMIT_SUCCESS implies bill. 

sounds fine to me.  But this basically means that we would hide
conntrack counters behind generic API - the counters themselves are
stored in our own data structure.

> cheers,
> jamal
-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]