From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Enderborg Subject: Re: Iptables 1.3.1 still not very fast. Date: Sat, 02 Apr 2005 07:32:40 +0200 Message-ID: <424E2E78.9010800@ufh.se> References: <89b63d049739b257@mayday.cix.co.uk> <20050402124038.0201.LARK@linux.net.cn> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: netfilter-devel@lists.netfilter.org Return-path: To: Wang Jian In-Reply-To: <20050402124038.0201.LARK@linux.net.cn> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: netfilter-devel-bounces@lists.netfilter.org Errors-To: netfilter-devel-bounces@lists.netfilter.org List-Id: netfilter-devel.vger.kernel.org Wang Jian wrote: >Hi Robert de Bath, > >It seems that your problem can be solved by using 'set' match. I didn't >use 'set' match, so I am not very sure. 'set' match is in pom-ng::base. > >For your situation, to generate so many rules to just match on source >address is not efficient. In natural language, it is > > "Is this source address in a blacklist?" > >and should be implemented this way as > > "Is this source address in a blacklist (hash table)?" > >It has only one rule. And then, you can manipulate this hash table. >(IMHO, the hash table involved is simple and fast) > >On Fri, 1 Apr 2005 22:52:25 +0100 (BST), Robert de Bath wrote: > > > Maybe a dumb question. But how do you hash on a range? >>Okay, I'll admit is it a LOT faster that 1.2.11 it still seems to take a long >>time with big tables. >> >>An example: >>I have a program (I've called iptrange) that will take a file full of ip >>address ranges (192.168.42.10 192.168.42.25) and converts them into a set >>of iptables chains that invokes another chain if the source (or dest) matches >>ie: >> >>iptables -N CHECK >>iptables -A CHECK -s 192.168.42.10/31 -j FOUND >>iptables -A CHECK -s 192.168.42.12/30 -j FOUND >>iptables -A CHECK -s 192.168.42.16/29 -j FOUND >>iptables -A CHECK -s 192.168.42.24/31 -j FOUND >> >>The good bit is that it will create a tree of subchains so that the >>matching is efficient for large numbers of ip addresses. >> >>This works well for a few hundred address ranges; I use it to match >>recent 'evil' ips and ranges from dshield.org. >> >>However, when I tried a 50000 range list from http://blocklist.org I >>ran into problems. After conversion it created 20000 chains with a >>total of 70000 rules. This tables took about an hour to load using >>iptables-1.2.11! >> >>Version 1.3.1 of libiptc is a LOT faster at about 5 minutes to do the >>load. But this still seems very slow when it takes about half a second >>to generate and print out the iptables rules to a script. >> >>I hate to think how slow it would be with a million ranges; assuming it can be >>loaded it looks like it would be about 300000 chains, 1.3M >>rules and about 100 rules checked per packet (to pick some figures out >>of a single test) so it should be useable ... >> >>BTW: iptables-restore is slower than my iptrange! >> >>My problem is that I think that libiptc looks evil and I really don't >>want to dive into messing with that code. So how can I help to make >>libiptc run as fast as I'd like it to? >> >>-- >>Rob. (Robert de Bath ) >> >> >> > > > > > -- foo!