Counter problem in a new nat target.

Counter problem in a new nat target.

[Emmanuel Guiton]

Hi!

I need to include a counter in my new nat target function (basically to 
know how many connections for a particular target are curently 
registered in the nat table). I had first though about including a 
simple integer in my targinfo, but I just noticed that it is passed to 
the target function as a const. Thus it does not work.
Does someone has a good advice on where I can store my variable? Can I 
use the void *userdata ? (I do not really know to what it points).

Thanks,

          Emmanuel

Re: Counter problem in a new nat target.

[Henrik Nordstrom]

On Mon, 8 Dec 2003, Emmanuel Guiton wrote:

> Does someone has a good advice on where I can store my variable? Can I 
> use the void *userdata ? (I do not really know to what it points).

You should be able to use the userdata I think. It is a pointer to the
original targetinfo sent/seen by userspace iptables command. The kernel
operates on another copy of the table per CPU. But I would not recommend 
doint this. I think it is better if you keep the needed counters 
elsewhere. You can use reference counting from the check/destroy functions 
to determine which counters you need to maintain.

Regards
Henrik

Re: Counter problem in a new nat target.

[Emmanuel Guiton]

Henrik Nordstrom wrote:

>On Mon, 8 Dec 2003, Emmanuel Guiton wrote:
>
>  
>
>>Does someone has a good advice on where I can store my variable? Can I 
>>use the void *userdata ? (I do not really know to what it points).
>>    
>>
>
>You should be able to use the userdata I think. It is a pointer to the
>original targetinfo sent/seen by userspace iptables command. The kernel
>operates on another copy of the table per CPU. But I would not recommend 
>doint this. I think it is better if you keep the needed counters 
>elsewhere. You can use reference counting from the check/destroy functions 
>to determine which counters you need to maintain.
>
>Regards
>Henrik
>  
>

Well, things are still unclear for me... I have to admit that I am not 
familiar with reference counting, so I do not really know about this 
possibility.
My problem is that I do not see which data can be accessed by any of the 
function involved. To be precise, I need to increment a counter each 
time I get one new conntrack. Then I decrement it each time a conntrack 
is destroyed or set as assured (tcp connection). The thing is 
complicated by the fact that I have several counters, one for each 
different IP destination address. the selection of the right counter is 
done in the target function.
Thus, I can easily do the increment and decrement when assured in the 
target function. But what about the decrement when a conntrack is destroyed?
Is the destroy function called when a conntrack is destroyed? 
Previously, I was adviced to use a notifier to catch the "conntrack 
destroyed" event. So is this equivalent (without regarding the data 
which can be accessed)?
The hacking howto also states that the checkentry can be used to 
dynalically allocate resources and they can be freed in the destroy 
function. But here the same issue occurs: I do not see which common data 
I can access from bothe the checkentry and destroy function.

Well, I feel I'm just blind and missing an important basic thing, but I 
can't point it out. I hope I won't seem too stupid when the light comes :)

           Emmanuel

Re: Counter problem in a new nat target.

[Henrik Nordstrom]

On Tue, 9 Dec 2003, Emmanuel Guiton wrote:

> My problem is that I do not see which data can be accessed by any of the 
> function involved. To be precise, I need to increment a counter each 
> time I get one new conntrack. Then I decrement it each time a conntrack 
> is destroyed or set as assured (tcp connection).

Then the problem is a lot more complex.

Increasing is not a big problem, but the decreasing are as it is hard to 
find which counter to decrease from the conntrack and there is no good 
place to store such information for later without extending the 
ip_conntrack structure unless you implement this as a conntrack 
application protocol helper (can be done if no other conntrack helper is 
needed in the relevant traffic).

Do you really need to decrease the counters?

> Is the destroy function called when a conntrack is destroyed? 

The target destroy function is called when the target as such in the 
iptable is destroyed. It is not related to connection tracking.

> The hacking howto also states that the checkentry can be used to 
> dynalically allocate resources and they can be freed in the destroy 
> function. But here the same issue occurs: I do not see which common data 
> I can access from bothe the checkentry and destroy function.

Both target functions operates on the iptable, not the traffic.

When you define your target/match you define what data the target/match
uses, and you are free to use this data as you please. This data MAY
include space for counters etc but due to SMP issues it is better to have
the counters separate as each CPU operates on his own iptable copy.

Regards
Henrik

Re: Counter problem in a new nat target.

[Emmanuel Guiton]

Hi!

>Do you really need to decrease the counters?
>
Well yes. My goal is to have a precise count of the connections that are 
not assured per destination host. I think I will try the helper option.

One more question: is the target function called for each new packets or 
only once per conntrack? I think I read somewhere that it was the first 
option but I cannot remember exactly and I cannot retrieve it.

Thanks a lot for your help,

         Emmanuel

Re: Counter problem in a new nat target.

[Henrik Nordstrom]

On Tue, 9 Dec 2003, Emmanuel Guiton wrote:

> One more question: is the target function called for each new packets or 
> only once per conntrack?

Once each time the rule is executed. How often this is depends on which 
table you are in and your ruleset and ranges from once per connection to 
once per packet (where retransmissions counts as unique packets)

Regards
Henrik

Re: Counter problem in a new nat target.

[Jesse Peng]

[-- Attachment #1: Type: text/plain, Size: 2202 bytes --]

Henrik Nordstrom wrote:

>On Tue, 9 Dec 2003, Emmanuel Guiton wrote:
>
>  
>
>>My problem is that I do not see which data can be accessed by any of the 
>>function involved. To be precise, I need to increment a counter each 
>>time I get one new conntrack. Then I decrement it each time a conntrack 
>>is destroyed or set as assured (tcp connection).
>>    
>>
>
>Then the problem is a lot more complex.
>
>Increasing is not a big problem, but the decreasing are as it is hard to 
>find which counter to decrease from the conntrack and there is no good 
>place to store such information for later without extending the 
>ip_conntrack structure unless you implement this as a conntrack 
>application protocol helper (can be done if no other conntrack helper is 
>needed in the relevant traffic).
>
>Do you really need to decrease the counters?
>  
>
I think there will remain two alternative ways to deal with the 
decrementing counter problem:

1.You maintain a hash struture which list any of your concerning 
conntrack, and while a conntrack facing destroyed then looking  up this 
hash list if it is on the list.
2.I don't think you need any real helper no matter how simple it can 
be.But just to extend your target module while matching ip_ct_new put 
some flags in the help 
information(conntrack->help.Your_patched_private_help_info)indecate that 
this is the conntrack you care. And then while a conntrack facing 
destroyed then looking  up its help info if it is flaged. But if the 
traffic you wanna filter overlaps with other helpers(eg. 
ip_conntrack_ftp),then a somewhat dirty way regarding where you put your 
flags(because help info is only for one helper)is to put them in 
conntrack->nat.help.Your_patched_private_help_info(for this is 
relatively a virgin place seldom touched by famous helpers except as I 
know the pptp helper)

But no matter which above you choose, You can't avoid patching your 
private destroying code like how ip_conntrack_destroyed(destroying code 
for NATed conntrack) been called.Because so far we never see any 
extending conntrack destroying machenism concerning helper's private 
functionality(like helper->destroy).
Hope this can help ;)!

Regards
Jesse

[-- Attachment #2: Type: text/html, Size: 2569 bytes --]

Re: Counter problem in a new nat target.

[Emmanuel Guiton]

>
> 1.You maintain a hash struture which list any of your concerning 
> conntrack, and while a conntrack facing destroyed then looking  up 
> this hash list if it is on the list.
> 2.I don't think you need any real helper no matter how simple it can 
> be.But just to extend your target module while matching ip_ct_new put 
> some flags in the help 
> information(conntrack->help.Your_patched_private_help_info)indecate 
> that this is the conntrack you care. And then while a conntrack facing 
> destroyed then looking  up its help info if it is flaged. But if the 
> traffic you wanna filter overlaps with other helpers(eg. 
> ip_conntrack_ftp),then a somewhat dirty way regarding where you put 
> your flags(because help info is only for one helper)is to put them in 
> conntrack->nat.help.Your_patched_private_help_info(for this is 
> relatively a virgin place seldom touched by famous helpers except as I 
> know the pptp helper)
>
> But no matter which above you choose, You can't avoid patching your 
> private destroying code like how ip_conntrack_destroyed(destroying 
> code for NATed conntrack) been called.Because so far we never see any 
> extending conntrack destroying machenism concerning helper's private 
> functionality(like helper->destroy).
> Hope this can help ;)!
>
> Regards
> Jesse


I think the conntrack->help data can be very useful to identify the 
counter to which the conntack belongs. that's a good idea, thanks.

However, the counters' "storage place" is still an unsolved issue. The 
main function that uses these counters is my nat target function. Only 
the decrease operation cannot be performed in this function. As Jesse 
and Krisztian metioned, I see so far two possibilities to decrease the 
counter:
 - in nat's ip_conntrack_destroyed
 - in a function in a struct notifier_block called upon the receipt of 
an IPCT_DESTROY notifier.
These two possibilities only offer to modify the conntrack structure, 
which does not work in my case.The counters are used to count the number 
of conntrack and the information in one conntrack is not related to the 
information in another conntrack.
In my target function, as Henrik pointed out, I can - easily, for want 
of cleanly - modify my counters in they are in the user data. In fact, I 
had originally placed the counters there, so that they were initialized 
when adding a new rule in the nat table.
Thus, if I could get a pointer to this user data when decreasinging my 
counters, the problem would be solved. I don't think it really fits in 
the ip_conntrack_destroyed function but why not in the notifier 
function? What about modifying the conntrack notifier system so that it 
includes also a pointer to the user data, or then make a brand new 
notifier system (because at first sight I don't see how to include a 
pointer to the user data in the current conntrack notifier system)?

What do you think about that?

             Emmanuel

PS: thanks for your comments, it's very precious help.

Re: Counter problem in a new nat target.

[Henrik Nordstrom]

On Wed, 10 Dec 2003, Emmanuel Guiton wrote:

> Thus, if I could get a pointer to this user data when decreasinging my 
> counters, the problem would be solved.

Won't work, and very dangerous.

You should maintain the storage within your module, and then add pointers 
to this to conntrack and the target userinfo.

> I don't think it really fits in the ip_conntrack_destroyed function but
> why not in the notifier function?

They are the same, and both fits well (except that ip_conntrack_destroyed 
is reserved for use by NAT).

> What about modifying the conntrack notifier system so that it includes
> also a pointer to the user data, or then make a brand new notifier
> system (because at first sight I don't see how to include a pointer to
> the user data in the current conntrack notifier system)?

You don't. You need to store the information you need in the conntrack if 
it cannot be derived from the addressing information or other information 
already present in conntrack.

I would really recommend you to extend conntrack with a custom field for 
the purpose than to abuse the helper slots.

Regards
Henrik

Re: Counter problem in a new nat target.

[Emmanuel Guiton]

>I would really recommend you to extend conntrack with a custom field for 
>the purpose than to abuse the helper slots.
>
>  
>
I was a bit reluctant to do this since I don't think it won't be useful 
for any applications other than mine. I thought it was a waste.

Thanks again for your help,

        Emmanuel

Re: Counter problem in a new nat target.

[Henrik Nordstrom]

On Wed, 10 Dec 2003, Emmanuel Guiton wrote:

> I was a bit reluctant to do this since I don't think it won't be useful 
> for any applications other than mine. I thought it was a waste.

Well.. extending the conntrack structure with the required information is 
a simple job, and can be done compile-time conditionally on if the feature 
is required or not.

Making other designs relating conntrack entries with custom data either 
requires abusing one of the existing fields and risk catastrophic results 
if there ever is a conflict with the originally intended use of these 
fields or by building an index outside of the conntrack table which is 
both complex and error prone.

The amount of information you need to extend the conntrack with is very 
small, just a identity of the counter used allowing your module to find 
which counter was used for this conntrack entry when it is destroyed. In 
the most simple implementation this is a pointer to your counter data, but 
it can be just an integer or a suitable size if you prefer. It is all 
dependent on how you want to identify the module specific data related to 
this specific conntrack entry.

Regards
Henrik

Re: Counter problem in a new nat target.

[Jesse Peng]

Henrik Nordstrom wrote:

>Making other designs relating conntrack entries with custom data either 
>requires abusing one of the existing fields and risk catastrophic results 
>if there ever is a conflict with the originally intended use of these 
>fields or by building an index outside of the conntrack table which is 
>both complex and error prone.
>
>  
>
The conntrack->help.Private_connection_helper_info and 
conntrack->nat.help.Private_nat_helper supposed not that much accused 
been abused.They exist only if you need a helper put his private data, 
so once not overlapsed with other helpers(You know the traffic your 
helper wants is not what any other helper wants) is for sure, than You 
of cource can put your helper data there. I think a helper exist not 
stonely if a registered helper exist or even a conntrack->helper exist, 
that depends on how you judge the "existence", by soul or by body ;)!

Re: Counter problem in a new nat target.

[Henrik Nordstrom]

On Thu, 11 Dec 2003, Jesse Peng wrote:

> The conntrack->help.Private_connection_helper_info and 
> conntrack->nat.help.Private_nat_helper supposed not that much accused 
> been abused.

There is not a problem using these from the helpers, but using these from
a target will give problems if such helper is ever used in the system for 
the same traffic as the target.

By designing the solution using these fields you absolutely rule out the 
possibility to combine the solution with conntrack/nat helpers.

Adding a new field is trivial and does not cause any conflicts. It is open 
source after all.

Regards
Henrik