Can I use iptables instead of hosts to block adservers?

Can I use iptables instead of hosts to block adservers?

[Walter Dnes]

  I downloaded a large hostfile blocking list recently.  Out of
curiousity, I ran it through a bash script, which fed each hostname to
the "host" command.  Approximately 95% of the host queries returned...

Host <hostname> not found: 3(NXDOMAIN)

  Think about it for a minute.  The people who run adservers *KNOW*
about hostfiles.  It's trivial to set up a script to rotate subdomain
names like a.doubleclick.net, b.doubleclick.net, c.doubleclick.net,
abc.doubleclick.net, etc, etc.  Even domain names can be rotated
through, and aliased.  All the names in the downloaded hostsfile were
probably valid at one time or another, but they age out rather quickly.
This strategy...
a) gets around hostfile-based blocking
b) penalizes hostfile-based blocking by slowing users' computers as they
   scan through an oversized list full of dead subdomain names

  Rather than blocking by ephemeral names, howsabout blocking by IP
address?  I don't think IPV4 addresses are plentiful enough for jumping
around.  This is where iptables comes in.  Let's start off with a
script that uses the 5% of valid addresses that I found.  Setup...

* create chain ADBLOCKLOG with rules
  iptables -A ADBLOCKLOG -j LOG --log-prefix "ADBLOCK:" --log-level 6
  iptables -A ADBLOCKLOG -j DROP

* create chain ADBLOCK with rule
  iptables -I -j ACCEPT

* list adserver addresses as follows...
iptables -I ADBLOCK -d <ipaddress1>/32 -j ADBLOCKLOG
iptables -I ADBLOCK -d <ipaddress2>/32 -j ADBLOCKLOG
iptables -I ADBLOCK -d <ipaddress3>/32 -j ADBLOCKLOG
etc, etc

* if adjacent IP addresses show up, we can aggregate them to /31 or /30
  or /29, etc.

  The last rule in the OUTPUT chain is changed to a jump to the ADBLOCK
chain so that all output is filtered.  If a packet "runs the gauntlet"
successfully, it hits the ACCEPT rule.

Questions...
1) Has this been done before, and am I re-inventing the wheel?
2) Is there a major showstopper problem with this idea?
3) Any suggestions for improvements?

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: Can I use iptables instead of hosts to block adservers?

[Jean Weisbuch]

Le 14/10/2017 à 03:02, Walter Dnes a écrit :
> 3) Any suggestions for improvements?
Using ipset to store the list of IPs/prefixes is better suited than 
using an iptables rule for each IP, it would work like this :

ipset -N adblocklist iphash
ipset -A -exist adblocklist <ipaddress1>
ipset -A -exist adblocklist <ipaddress2>
iptables -A OUTPUT -m set --match-set adblocklist dst -j DROP

Re: Can I use iptables instead of hosts to block adservers?

[Neal P. Murphy]

On Fri, 13 Oct 2017 21:02:04 -0400
"Walter Dnes" <waltdnes@waltdnes.org> wrote:

>   I downloaded a large hostfile blocking list recently.  Out of
> curiousity, I ran it through a bash script, which fed each hostname to
> the "host" command.  Approximately 95% of the host queries returned...
> 
> Host <hostname> not found: 3(NXDOMAIN)
> 
>   Think about it for a minute.  The people who run adservers *KNOW*
> about hostfiles.  It's trivial to set up a script to rotate subdomain
> names like a.doubleclick.net, b.doubleclick.net, c.doubleclick.net,
> abc.doubleclick.net, etc, etc.  Even domain names can be rotated
> through, and aliased.  All the names in the downloaded hostsfile were
> probably valid at one time or another, but they age out rather quickly.
> This strategy...
> a) gets around hostfile-based blocking
> b) penalizes hostfile-based blocking by slowing users' computers as they
>    scan through an oversized list full of dead subdomain names
> 
>   Rather than blocking by ephemeral names, howsabout blocking by IP
> address?  I don't think IPV4 addresses are plentiful enough for jumping
> around.  This is where iptables comes in.  Let's start off with a
> script that uses the 5% of valid addresses that I found.  Setup...
> 
> * create chain ADBLOCKLOG with rules
>   iptables -A ADBLOCKLOG -j LOG --log-prefix "ADBLOCK:" --log-level 6
>   iptables -A ADBLOCKLOG -j DROP
> 
> * create chain ADBLOCK with rule
>   iptables -I -j ACCEPT
> 
> * list adserver addresses as follows...
> iptables -I ADBLOCK -d <ipaddress1>/32 -j ADBLOCKLOG
> iptables -I ADBLOCK -d <ipaddress2>/32 -j ADBLOCKLOG
> iptables -I ADBLOCK -d <ipaddress3>/32 -j ADBLOCKLOG
> etc, etc
> 
> * if adjacent IP addresses show up, we can aggregate them to /31 or /30
>   or /29, etc.
> 
>   The last rule in the OUTPUT chain is changed to a jump to the ADBLOCK
> chain so that all output is filtered.  If a packet "runs the gauntlet"
> successfully, it hits the ACCEPT rule.
> 
> Questions...
> 1) Has this been done before, and am I re-inventing the wheel?

Yes, it's been done.

> 2) Is there a major showstopper problem with this idea?

Netfilter starts becoming inefficient after about 16 entries.

> 3) Any suggestions for improvements?

Use ipset; it's efficient to thousands and even tens of thousands of addresses.

I hacked a script for Smoothwall Express that fetches a few free blocklists, consolidates them, and puts their entries into host and net ipsets. Then it sets netfilter to check incoming packets for src or dst addrs among those entries. It works quite nicely.

Re: Can I use iptables instead of hosts to block adservers?

[Imran Geriskovan]

On 10/14/17, Walter Dnes <waltdnes@waltdnes.org> wrote:
>   Rather than blocking by ephemeral names, howsabout blocking by IP
> address?

There may be multiple "virtual" web domains on that same ip, someof
which are adservers you want the block and some of which are valid
ones that you or your users may want to access.
Depends on the ISP setup.

But "in general" you may assume ad companies operate their
own dedicated servers so most of the time your above appoach is ok.
But sometimes not.

May be they are too aware of this and they intentionally have
virtual server over a floaiting set of IPs which also serve valid
web domains to make counter-measures more difficult/complicated.