From: Catalin BOIE <util@deuroconsult.ro>
To: lartc@vger.kernel.org
Subject: Re: [LARTC] Filter huge number of hosts
Date: Tue, 11 Nov 2003 08:28:46 +0000 [thread overview]
Message-ID: <marc-lartc-106854052515419@msgid-missing> (raw)
In-Reply-To: <marc-lartc-106853777913308@msgid-missing>
On Tue, 11 Nov 2003, Peter Nelson wrote:
> Ok, I'm pretty new at this and am probably in a bit over my head, but I
> was looking for some pointers. I want to classify and then prioritize
> traffic based on if it is to an Internet2 host, a host on campus or the
> general internet. I got a dump from one of our campus's routers that
> classifies everything as either I2 or local, only problem is that it is
> ~8000 entries. Now a lot of those entries can probably be combined down
> (for example a couple of sequential /24's). My first question is does
> anyone have any script that you give it multiple netmasks and it
> combines it down to the simplest netmasks? Now my second question is
> would using hashtables, maybe even nested ones be practical for this?
> This is how I'm thinking of hashing so far:
>
> Mask hashes children to check
> 0xFF000000: 100 most 50-500
> 0xFFFF0000: 3000 most <20, some 50-200
> 0xFFFFFF00: 8000 all < 10
>
You can link hashes to do what you want:
hash (made from first byte of the ip):
01 -> another_hash_1
10 -> another_hash_2
another_hash_1 entries (hash made from second byte):
01 -> another_hash_on_level_2
10 -> another_hash_on_level_2
and so on.
> Obviously hashing based on 0xFF helps, but it still has to go through up
> to 500 checks for somes hosts. Once I hash based on 0xFFFF the worst
> case gets a lot better, and of course hashing on 0xFFFFFF makes almost
> perfect hashes. Only thing is does a hash table lose it's point once
> you start indexing everything? I was thinking maybe hashing based on
> the 0xFF and then from there making hashes based on 0x00FF (and then
> maybe an other layer of 0x0000FF). I admit I really don't know all that
> much about the hashing algorighm and it's complexity so I don't know how
> to approach this.
>
> Thanks for any information,
> Peter Nelson
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
---
Catalin(ux) BOIE
catab@deuroconsult.ro
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
prev parent reply other threads:[~2003-11-11 8:28 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-11-11 6:48 [LARTC] Filter huge number of hosts Peter Nelson
2003-11-11 6:48 ` Peter Nelson
2003-11-11 6:48 ` Peter Nelson
2003-11-11 8:28 ` Catalin BOIE [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=marc-lartc-106854052515419@msgid-missing \
--to=util@deuroconsult.ro \
--cc=lartc@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox