How to access conntrack connection expiration in a netfilter module?

How to access conntrack connection expiration in a netfilter module?

[Ed W]

Hi, I'm trying to update the opendpi-netfilter module to work with more 
recent kernels and the ntop ndpi fork of the code (now that opendpi 
seems to have gone away)

The netfilter module registers a conntrack notifier in order to release 
resources once a conntrack is destroyed
https://github.com/ewildgoose/ndpi-netfilter/blob/master/src/main.c
This is not possible though, if you also want to have conntrack events 
through netlink... ( NF_CT_NETLINK grabs the conntrack notifier first)

Now I have read the history on why this is the case, but what are the 
obvious workarounds to this situation?  Can we extend the conntrack 
connection with custom properties? Anything better than simply scanning 
my flows occasionally and comparing with the conntrack list to see if 
they have gone away?

Thanks for any thoughts

Ed W

Re: How to access conntrack connection expiration in a netfilter module?

[Ed W]

Go on... I'm sure someone wants to have a stab at educating this twit on 
ways a netfilter module could do housekeeping and cleanup it's internal 
state when conntrack connections are destroyed (emphasis being on how to 
discover they have been destroyed in some timely manner?)

Thanks

Ed W

On 01/10/2012 22:17, Ed W wrote:
> Hi, I'm trying to update the opendpi-netfilter module to work with 
> more recent kernels and the ntop ndpi fork of the code (now that 
> opendpi seems to have gone away)
>
> The netfilter module registers a conntrack notifier in order to 
> release resources once a conntrack is destroyed
> https://github.com/ewildgoose/ndpi-netfilter/blob/master/src/main.c
> This is not possible though, if you also want to have conntrack events 
> through netlink... ( NF_CT_NETLINK grabs the conntrack notifier first)
>
> Now I have read the history on why this is the case, but what are the 
> obvious workarounds to this situation?  Can we extend the conntrack 
> connection with custom properties? Anything better than simply 
> scanning my flows occasionally and comparing with the conntrack list 
> to see if they have gone away?
>
> Thanks for any thoughts
>
> Ed W
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe 
> netfilter-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to access conntrack connection expiration in a netfilter module?

[Pablo Neira Ayuso]

On Mon, Oct 01, 2012 at 10:17:02PM +0100, Ed W wrote:
> Hi, I'm trying to update the opendpi-netfilter module to work with
> more recent kernels and the ntop ndpi fork of the code (now that
> opendpi seems to have gone away)
> 
> The netfilter module registers a conntrack notifier in order to
> release resources once a conntrack is destroyed
> https://github.com/ewildgoose/ndpi-netfilter/blob/master/src/main.c
> This is not possible though, if you also want to have conntrack
> events through netlink... ( NF_CT_NETLINK grabs the conntrack
> notifier first)
> 
> Now I have read the history on why this is the case, but what are
> the obvious workarounds to this situation?  Can we extend the
> conntrack connection with custom properties? Anything better than
> simply scanning my flows occasionally and comparing with the
> conntrack list to see if they have gone away?

We used to have notifier call chains to deliver in-kernel
nofitications to events. However, since it was overkill for just one
single client (ctnetlink), we removed it and use a single hook
function.

The workaround is to dig into the history, find that code and forward
port it.

But I have to warn you that I won't take that patch into mainstream
since there's only one single client in the official Linux kernel
code, and external clients like that ndpi thing do not justify such
change. Sorry.

Re: How to access conntrack connection expiration in a netfilter module?

[Ed W]

On 02/10/2012 19:17, Pablo Neira Ayuso wrote:

> We used to have notifier call chains to deliver in-kernel
> nofitications to events. However, since it was overkill for just one
> single client (ctnetlink), we removed it and use a single hook
> function.
>
> The workaround is to dig into the history, find that code and forward
> port it.
>
> But I have to warn you that I won't take that patch into mainstream
> since there's only one single client in the official Linux kernel
> code, and external clients like that ndpi thing do not justify such
> change. Sorry.

Understood.  I have temporarily hacked in a crude second notifier 
variable, just about to test it.

However, it seems like a common requirement to want to be able to do 
some housekeeping in netfilter modules - what am I missing, how are 
other modules doing stuff like this?  Is there another technique which 
might be used?  Any other modules which do something similar that I 
could crib from, ie with some internal state augmenting a flow and then 
needing to cleanup sometime after the flow has gone away?

Thanks

Ed W

Re: How to access conntrack connection expiration in a netfilter module?

[Pablo Neira Ayuso]

On Wed, Oct 03, 2012 at 08:35:31AM +0100, Ed W wrote:
> On 02/10/2012 19:17, Pablo Neira Ayuso wrote:
> 
> >We used to have notifier call chains to deliver in-kernel
> >nofitications to events. However, since it was overkill for just one
> >single client (ctnetlink), we removed it and use a single hook
> >function.
> >
> >The workaround is to dig into the history, find that code and forward
> >port it.
> >
> >But I have to warn you that I won't take that patch into mainstream
> >since there's only one single client in the official Linux kernel
> >code, and external clients like that ndpi thing do not justify such
> >change. Sorry.
> 
> Understood.  I have temporarily hacked in a crude second notifier
> variable, just about to test it.
> 
> However, it seems like a common requirement to want to be able to do
> some housekeeping in netfilter modules - what am I missing, how are
> other modules doing stuff like this?  Is there another technique
> which might be used?  Any other modules which do something similar
> that I could crib from, ie with some internal state augmenting a
> flow and then needing to cleanup sometime after the flow has gone
> away?

I guess your modules are not using conntrack extensions:

See net/netfilter/nf_conntrack_extend.c and
net/netfilter/nf_conntrack_acct.c for instance.

Re: How to access conntrack connection expiration in a netfilter module?

[Aidan McGurn]

Pablo Neira Ayuso <pablo <at> netfilter.org> writes:

> 
> On Wed, Oct 03, 2012 at 08:35:31AM +0100, Ed W wrote:
> > On 02/10/2012 19:17, Pablo Neira Ayuso wrote:
> > 
> > >We used to have notifier call chains to deliver in-kernel
> > >nofitications to events. However, since it was overkill for just one
> > >single client (ctnetlink), we removed it and use a single hook
> > >function.
> > >
> > >The workaround is to dig into the history, find that code and forward
> > >port it.
> > >
> > >But I have to warn you that I won't take that patch into mainstream
> > >since there's only one single client in the official Linux kernel
> > >code, and external clients like that ndpi thing do not justify such
> > >change. Sorry.
> > 
> > Understood.  I have temporarily hacked in a crude second notifier
> > variable, just about to test it.
> > 
> > However, it seems like a common requirement to want to be able to do
> > some housekeeping in netfilter modules - what am I missing, how are
> > other modules doing stuff like this?  Is there another technique
> > which might be used?  Any other modules which do something similar
> > that I could crib from, ie with some internal state augmenting a
> > flow and then needing to cleanup sometime after the flow has gone
> > away?
> 
> I guess your modules are not using conntrack extensions:
> 
> See net/netfilter/nf_conntrack_extend.c and
> net/netfilter/nf_conntrack_acct.c for instance.
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
> the body of a message to majordomo <at> vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
Hi Pablo,
i share the same head scratching as Ed here -
Currently i was using a Netfilter kernel module which required notification
of a connection been deleted:
e.g. 
if (events & IPCT_DESTROY) { ....
//do houskeeping, cleanups of my own custom connection data/database
}

Now when upgrading to RHEL6.3./6.4, i face a serious issue of not having any
way to get notified of a connection been removed.
I didn't understand how conntrack extensions help me here. Could you please
explain?

thanks