Re: Quota on SMP AGAIN

[Patrick McHardy <kaber@trash.net>]

Jan Engelhardt wrote:
> On Dec 28 2007 17:13, Patrick McHardy wrote:
>   
>> Gpf's report didn't include an actual description of the problem, so
>> I can only assume its an incorrect value of the quota. What exactly
>> are the effects you're seeing?
>>
>>     
>
> To begin with:
>
> 	ssh root@somebox
>
> when logged in:
>
> 	iptables -I OUTPUT -m quota --quota 1234567
>
> as you do more actions over the interactive ssh session, the quota
> will obviously decrease -- after all, that is what it is supposed
> to do. Enter
>
> 	iptables -nvL OUTPUT
>
> should reduce the quota by some 50-60 bytes per packet (and typing the 
> iptables command costs some bytes), but as you 
> repeatedly call iptables, the quota fluctuates around 1234567.
>
> What one can see is that the packet counter (1st column) increases while 
> the quota has problems.
>
> 17:38 ccgmbh:~ # iptables -nvL | grep quota
>   155 35709            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1234567 bytes
> 17:38 ccgmbh:~ # iptables -nvL | grep quota
>   163 36609            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1234567 bytes
> 17:38 ccgmbh:~ # iptables -nvL | grep quota
>   171 37509            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1197058 bytes
> 17:38 ccgmbh:~ # iptables -nvL | grep quota
>   179 38409            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1234567 bytes
>
> What is interesting to note is that if we do away with all "wrong" 
> results, then it "works":
>
> 17:39 ccgmbh:~ # while :; do iptables -nvL | grep quota; done | grep -v 1234567
> ...
>   830  118K            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1116680 bytes
>   832  118K            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1116399 bytes
>   834  118K            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1116150 bytes
>   834  118K            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1116150 bytes
>   838  119K            all  --  *      *       0.0.0.0/0            0.0.0.0/0           quota: 1115684 bytes
> ...
>
> Want to know what I believe? That r->master = r in the source code is 
> redundant and/or does not solve the problem its comment says.
>
> /* For SMP, we only want to use one set of counters. */
>
> And my bigtime question would be: where is the other counter actually? 
> struct xt_quota_info only has one counter! Does netfilter secretly 
> allocate matchinfos per-cpu?
>   

Not secretly, but yes, the entire ruleset exists once per CPU. That
also seems to be the problem, at the time the master idea was thought
of we always dumped entries from CPU 0, today its from the current
CPU, but the only one that actually has correct counters is CPU 0.

The easy fix would be to revert to that behaviour, but maybe someone
can come up with a better idea that doesn't involve walking over the
entire ruleset and resycing things or adding dump callbacks to
matches.