From mboxrd@z Thu Jan 1 00:00:00 1970 From: Patrick McHardy Subject: Re: Quota on SMP AGAIN Date: Fri, 28 Dec 2007 17:50:32 +0100 Message-ID: <47752958.3010601@trash.net> References: <4773C13D.1040106@simm.ru> <477515BB.1060303@trash.net> <477520C3.8040501@trash.net> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: gpf , netfilter-devel@vger.kernel.org To: Jan Engelhardt Return-path: Received: from stinky.trash.net ([213.144.137.162]:34339 "EHLO stinky.trash.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753451AbXL1Qup (ORCPT ); Fri, 28 Dec 2007 11:50:45 -0500 In-Reply-To: Sender: netfilter-devel-owner@vger.kernel.org List-ID: Jan Engelhardt wrote: > On Dec 28 2007 17:13, Patrick McHardy wrote: > >> Gpf's report didn't include an actual description of the problem, so >> I can only assume its an incorrect value of the quota. What exactly >> are the effects you're seeing? >> >> > > To begin with: > > ssh root@somebox > > when logged in: > > iptables -I OUTPUT -m quota --quota 1234567 > > as you do more actions over the interactive ssh session, the quota > will obviously decrease -- after all, that is what it is supposed > to do. Enter > > iptables -nvL OUTPUT > > should reduce the quota by some 50-60 bytes per packet (and typing the > iptables command costs some bytes), but as you > repeatedly call iptables, the quota fluctuates around 1234567. > > What one can see is that the packet counter (1st column) increases while > the quota has problems. > > 17:38 ccgmbh:~ # iptables -nvL | grep quota > 155 35709 all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1234567 bytes > 17:38 ccgmbh:~ # iptables -nvL | grep quota > 163 36609 all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1234567 bytes > 17:38 ccgmbh:~ # iptables -nvL | grep quota > 171 37509 all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1197058 bytes > 17:38 ccgmbh:~ # iptables -nvL | grep quota > 179 38409 all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1234567 bytes > > What is interesting to note is that if we do away with all "wrong" > results, then it "works": > > 17:39 ccgmbh:~ # while :; do iptables -nvL | grep quota; done | grep -v 1234567 > ... > 830 118K all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1116680 bytes > 832 118K all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1116399 bytes > 834 118K all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1116150 bytes > 834 118K all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1116150 bytes > 838 119K all -- * * 0.0.0.0/0 0.0.0.0/0 quota: 1115684 bytes > ... > > Want to know what I believe? That r->master = r in the source code is > redundant and/or does not solve the problem its comment says. > > /* For SMP, we only want to use one set of counters. */ > > And my bigtime question would be: where is the other counter actually? > struct xt_quota_info only has one counter! Does netfilter secretly > allocate matchinfos per-cpu? > Not secretly, but yes, the entire ruleset exists once per CPU. That also seems to be the problem, at the time the master idea was thought of we always dumped entries from CPU 0, today its from the current CPU, but the only one that actually has correct counters is CPU 0. The easy fix would be to revert to that behaviour, but maybe someone can come up with a better idea that doesn't involve walking over the entire ruleset and resycing things or adding dump callbacks to matches.