Mystics of packet forwarding

Mystics of packet forwarding

[Artūras Šlajus]

Hello fellow netfilter users,

I have a strange problem and I think I should blame my ISP for that...

Recently I lost connectivity to some sites (i.e. digg.com, yahoo). The best part 
is that I can regain connectivity by clearing out all the rules from iptables.

So if I have empty chains - I can connect to digg. After I add one rule:

iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE (or SNAT, doesn't 
make a difference)

and bam! in 3-5 seconds I lose connectivity to these sites. Others work just fine.

The delay of impact is one thing that bothers me and makes me wonder about magic 
in ISP side.

Other thingie is if I do:

iptables -t nat -I POSTROUTING -s 192.168.0.0/16 -j DROP

After 3-5 seconds I regain connectivity... It seems that my ISP is somehow 
monitoring nat'ing and dropping access to some sites...

Any ideas why could this be happening and what is so special about digg.com and 
some of yahoo servers that they connectivity to them get affected like this?

Any ideas welcome.

Thanks, Arthur.

Re: Mystics of packet forwarding

[Billy Crook]

On Tue, Jan 6, 2009 at 13:41, Artûras Ðlajus <x11@arturaz.net> wrote:
> Hello fellow netfilter users,
>
> I have a strange problem and I think I should blame my ISP for that...
>
> Recently I lost connectivity to some sites (i.e. digg.com, yahoo). The best
> part is that I can regain connectivity by clearing out all the rules from
> iptables.
>
> So if I have empty chains - I can connect to digg. After I add one rule:
>
> iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE (or SNAT,
> doesn't make a difference)
>
> and bam! in 3-5 seconds I lose connectivity to these sites. Others work just
> fine.
>
> The delay of impact is one thing that bothers me and makes me wonder about
> magic in ISP side.
>
> Other thingie is if I do:
>
> iptables -t nat -I POSTROUTING -s 192.168.0.0/16 -j DROP
>
> After 3-5 seconds I regain connectivity... It seems that my ISP is somehow
> monitoring nat'ing and dropping access to some sites...
>
> Any ideas why could this be happening and what is so special about digg.com
> and some of yahoo servers that they connectivity to them get affected like
> this?
>
> Any ideas welcome.
>
> Thanks, Arthur.

Has your ISP ever indicated hostility verbally, or in writing, toward
NAT?  Is it forbidden by your Terms of Service?

I find it difficult to believe that your ISP would do something so
elaborate to discourage NAT, especially if they don't explicitly
forbid it.  Try disconnecting your lan from this machine, and testing
for this anomaly.  I suspect one of the machines on your lan may be
attacking yahoo or digg, and causing those sites to temporarily block
your [public] IP.

Re: Mystics of packet forwarding

[Artūras Šlajus]

Billy Crook wrote:
> Has your ISP ever indicated hostility verbally, or in writing, toward
> NAT?  Is it forbidden by your Terms of Service?
Well, it says that NAT is allowed in one house. Also a friend of mine that works 
as main admin on that ISP says they didn't do anything like that.

> I find it difficult to believe that your ISP would do something so
> elaborate to discourage NAT, especially if they don't explicitly
> forbid it.  Try disconnecting your lan from this machine, and testing
> for this anomaly.
An idea. But it seems that only actually forwarded packets make it happen...

>  I suspect one of the machines on your lan may be
> attacking yahoo or digg, and causing those sites to temporarily block
> your [public] IP.
An idea too. I'll try to get an temporary ip and monitor logs for a while...

How could this attack look like? Packet bursts? Lot of connections?

Re: Mystics of packet forwarding

[Artūras Šlajus]

Another strange thing - sometimes it does work, sometimes it just hangs... I 
wonder - wtf...

gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0
gw:/var/log/ulog# nc -v -v yui.yahooapis.com 80
DNS fwd/rev mismatch: geoycs-l.yahoo8.akadns.net != l1.ycs.vip.ch1.yahoo.com
geoycs-l.yahoo8.akadns.net [87.248.125.47] 80 (www) open
^C sent 0, rcvd 0

Re: Mystics of packet forwarding

[Amos Jeffries]

Artūras Šlajus wrote:
> Hello fellow netfilter users,
> 
> I have a strange problem and I think I should blame my ISP for that...
> 
> Recently I lost connectivity to some sites (i.e. digg.com, yahoo). The 
> best part is that I can regain connectivity by clearing out all the 
> rules from iptables.
> 
> So if I have empty chains - I can connect to digg. After I add one rule:
> 
> iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE (or SNAT, 
> doesn't make a difference)

I very much doubt it's your ISP. Maybe the

one-sided NAT does not usually work very well. Try adding both the 
symmetrical sides at once:

SNAT on the outbound request packets
   iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j SNAT ...

MASQUERADE on the inbound reply packets
  iptables -t nat -A PREROUTING -d 192.168.0.0/16 -j MASQUERADE


AYJ

Re: Mystics of packet forwarding

[Ivan Petrushev]

SNAT should be sufficient.
Your friend admin said they didn't do anything. Maybe it is the
provider they are getting bandwidth from?
One think I can come with is TTL limiting (largely known here where I
live). Try pinging these "troubling" sites from your home gateway and
see if TTL is 1 or 2 or some bigger value.
And one other thing - you said these sites disappear, but I didin't
understood where from are you testing? From the home gateway or from
the NATed boxes behind it?

Could you add SNAT rule for non-existant box (IP that is not present
on your network, like 192.168.0.200) and see if these sites work.

And one other thing - /16 ? Do you really have such big network? :)

Greetings,
Ivan.

On Wed, Jan 7, 2009 at 8:15 AM, Amos Jeffries <squid3@treenet.co.nz> wrote:
> Artûras Ğlajus wrote:
>>
>> Hello fellow netfilter users,
>>
>> I have a strange problem and I think I should blame my ISP for that...
>>
>> Recently I lost connectivity to some sites (i.e. digg.com, yahoo). The
>> best part is that I can regain connectivity by clearing out all the rules
>> from iptables.
>>
>> So if I have empty chains - I can connect to digg. After I add one rule:
>>
>> iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j MASQUERADE (or SNAT,
>> doesn't make a difference)
>
> I very much doubt it's your ISP. Maybe the
>
> one-sided NAT does not usually work very well. Try adding both the
> symmetrical sides at once:
>
> SNAT on the outbound request packets
>  iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -j SNAT ...
>
> MASQUERADE on the inbound reply packets
>  iptables -t nat -A PREROUTING -d 192.168.0.0/16 -j MASQUERADE
>
>
> AYJ
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Mystics of packet forwarding

[Artūras Šlajus]

Ivan Petrushev wrote:
> One think I can come with is TTL limiting (largely known here where I
> live). Try pinging these "troubling" sites from your home gateway and
> see if TTL is 1 or 2 or some bigger value.
I don't quite understand what are you saying? TTL too small and expires in path? 
TTL too big and gets filtered some how?

> And one other thing - you said these sites disappear, but I didin't
> understood where from are you testing? From the home gateway or from
> the NATed boxes behind it?
 From both sites..

> Could you add SNAT rule for non-existant box (IP that is not present
> on your network, like 192.168.0.200) and see if these sites work.
> 
> And one other thing - /16 ? Do you really have such big network? :)
No, but I have a lot of dumbass users who love to set static ips to ones that 
servers use :)) And I doubt that this is the problem...

Re: Mystics of packet forwarding

[Artūras Šlajus]

Some more debugging info:

netcat to digg.com 80 with no firewall (TRACE target on raw OUTPUT)

Jan  6 22:19:36 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=39286 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687857 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:19:36 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=39286 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687857 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:19:36 gw TRACE: nat:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=39286 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687857 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:19:36 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=39286 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687857 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:19:36 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=39286 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687857 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:19:36 gw TRACE: nat:POSTROUTING:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=39286 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687857 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:19:36 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39287 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK URGP=0
Jan  6 22:19:36 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39287 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK URGP=0
Jan  6 22:19:36 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39287 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK URGP=0
Jan  6 22:19:36 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39287 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK URGP=0
Jan  6 22:19:41 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39288 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK FIN URGP=0
Jan  6 22:19:41 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39288 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK FIN URGP=0
Jan  6 22:19:41 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39288 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK FIN URGP=0
Jan  6 22:19:41 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39288 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687858 ACK=2594171353 WINDOW=5840 ACK FIN URGP=0
Jan  6 22:19:41 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39289 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687859 ACK=2594171354 WINDOW=5840 ACK URGP=0
Jan  6 22:19:41 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39289 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687859 ACK=2594171354 WINDOW=5840 ACK URGP=0
Jan  6 22:19:41 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39289 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687859 ACK=2594171354 WINDOW=5840 ACK URGP=0
Jan  6 22:19:41 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=39289 CE DF 
PROTO=TCP SPT=50725 DPT=80 SEQ=3378687859 ACK=2594171354 WINDOW=5840 ACK URGP=0

with firewall

Jan  6 22:20:28 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24393 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:28 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24393 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:28 gw TRACE: nat:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24393 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:28 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24393 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:28 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24393 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:28 gw TRACE: nat:POSTROUTING:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24393 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:31 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24394 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:31 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24394 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:31 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24394 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:31 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24394 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:37 gw TRACE: raw:OUTPUT:policy:2  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24395 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:37 gw TRACE: mangle:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24395 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:37 gw TRACE: filter:OUTPUT:policy:1  IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24395 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0
Jan  6 22:20:37 gw TRACE: mangle:POSTROUTING:policy IN= OUT=eth1 MAC= 
SRC=87.247.77.88 DST=64.191.203.30 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=24395 DF 
PROTO=TCP SPT=58290 DPT=80 SEQ=4208670647 ACK=0 WINDOW=5840 SYN URGP=0

It seems that it never goes to ack somehow :(

Re: Mystics of packet forwarding

[Artūras Šlajus]

Ok, it seems that really - someone in LAN is attacking the internet.

If I turn on forwarding for few users like me, some other computer-literate 
friends - digg.com still works :))

Now it's the question how do I catch bad guys? What should I look into? Packet 
bursts? Lot's of new connections? Etc?

Re: Mystics of packet forwarding

[Ivan Petrushev]

> Ok, it seems that really - someone in LAN is attacking the internet.
It could be worms or viruses. In my experience every home or student
Windows network is awfully crowded with viruses. Maybe you should work
on filtering your outgoing traffic.

I'm not sure, but search on google what could cause ban on DIGG or
YAHOO. Probably lots of connections or flood could result in banning
you from these sites. But you say that immediately when you remove the
NAT rules the access is restored? I don't believe their firewalls are
quick enough to restore your position two seconds after you stop being
"bad" to them. Maybe something else is the reason.

Anyway, do you have any output filtering rules?

On Wed, Jan 7, 2009 at 11:28 AM, Artûras Ðlajus <x11@arturaz.net> wrote:
> Ok, it seems that really - someone in LAN is attacking the internet.
>
> If I turn on forwarding for few users like me, some other computer-literate
> friends - digg.com still works :))
>
> Now it's the question how do I catch bad guys? What should I look into?
> Packet bursts? Lot's of new connections? Etc?
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Mystics of packet forwarding

[Roman Fiedler]

Ivan Petrushev wrote:
>> Ok, it seems that really - someone in LAN is attacking the internet.
> It could be worms or viruses. In my experience every home or student
> Windows network is awfully crowded with viruses. Maybe you should work
> on filtering your outgoing traffic.
>
> I'm not sure, but search on google what could cause ban on DIGG or
> YAHOO. Probably lots of connections or flood could result in banning
> you from these sites. But you say that immediately when you remove the
> NAT rules the access is restored? I don't believe their firewalls are
> quick enough to restore your position two seconds after you stop being
> "bad" to them. Maybe something else is the reason.

Are there any hardcoded external IPs in your ruleset? These might fail 
if other hosts have some DNS-round-robin.

Re: Mystics of packet forwarding

[Billy Crook]

On Wed, Jan 7, 2009 at 03:28, Artûras Ğlajus <x11@arturaz.net> wrote:
> Ok, it seems that really - someone in LAN is attacking the internet.
>
> If I turn on forwarding for few users like me, some other computer-literate
> friends - digg.com still works :))
>
> Now it's the question how do I catch bad guys? What should I look into?
> Packet bursts? Lot's of new connections? Etc?

Given the nature of yahoo and digg's services, the attacks are
probably happening over TCP.  Log all outdoing tcp connections, then
enable NAT, wait until yahoo block you, turn it back off, and look at
the logs.  I bet it will be fairly obvious which client(s) are
responsible.  If not, sort the logs, and count the number of
connections from each internal client.  Smoothwall has these nice
per-client traffic graphs which would show be which clients on my
network are generating the most traffic.

Re: Mystics of packet forwarding

[Mart Frauenlob]

netfilter-owner@vger.kernel.org wrote:
> Ok, it seems that really - someone in LAN is attacking the internet.
>
> If I turn on forwarding for few users like me, some other 
> computer-literate friends - digg.com still works :))
>
> Now it's the question how do I catch bad guys? What should I look 
> into? Packet bursts? Lot's of new connections? Etc?
>

quick ways could be:
iptraf (you could apply filters for specific traffic)
iptstate (shows conntrack table)
tcpdump (i.e. simple rule: tcpdump -n -i your_ext_iface tcp dst port 80)

any of those tools could give you a quick picture of current connections 
(attempts).

greets

mart

Re: Mystics of packet forwarding

[Ivan Petrushev]

tcpdump could be a bit overwhelming with its mass of output information.
Maybe Wireshark?

If the case is virus/worm it could be doing port scans on the target
sites or something that not involve port 80.
Something just came in my mind. Remove all of the "gray" NAT rules and
start adding them one by one, removing the old ones. In every moment
there should be only 1 NAT rule. AND remove all Internet access except
for the "target" sites. That could be done with something like that:
iptables -t nat -A POSTROUTING -s XXX.XXX.XXX.XXX -d DIGG.COM -j SNAT
--to-source YYY.YYY.YYY.YYY
iptables -t nat -A POSTROUTING -s XXX.XXX.XXX.XXX -d YAHOO.COM -j SNAT
--to-source YYY.YYY.YYY.YYY
The YYY is ofcourse your external IP.
In that way that PC with IP XXX should have access only to digg.com
(it would be nice to resolve the host DIGG.COM and YAHOO.COM and use
every of its IPs if it has many)
What is the purpose of that: if at any point of the experiment you
notice potent use of bandwidth - that LAN host you are currently
adding is the one that is making things wrong. And since you have been
disabled all of regular Internet traffic (torrents, downloads, wgets,
www) there should be nothing generating traffic and it will be easier
to distinguish the virus host.

On Wed, Jan 7, 2009 at 5:07 PM, Mart Frauenlob <mart.frauenlob@chello.at> wrote:
> netfilter-owner@vger.kernel.org wrote:
>>
>> Ok, it seems that really - someone in LAN is attacking the internet.
>>
>> If I turn on forwarding for few users like me, some other
>> computer-literate friends - digg.com still works :))
>>
>> Now it's the question how do I catch bad guys? What should I look into?
>> Packet bursts? Lot's of new connections? Etc?
>>
>
> quick ways could be:
> iptraf (you could apply filters for specific traffic)
> iptstate (shows conntrack table)
> tcpdump (i.e. simple rule: tcpdump -n -i your_ext_iface tcp dst port 80)
>
> any of those tools could give you a quick picture of current connections
> (attempts).
>
> greets
>
> mart
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Mystics of packet forwarding

[Artūras Šlajus]

Ivan Petrushev wrote:
> tcpdump could be a bit overwhelming with its mass of output information.
> Maybe Wireshark?
> 
> If the case is virus/worm it could be doing port scans on the target
> sites or something that not involve port 80.
I'm lost. And desperate.

I added rule to log ALL packets that are forwarded through.

wrote a tiny script to filter them out.

#!/usr/bin/env ruby

targets = ARGV[0..-1]
puts "targets: #{targets.inspect}"

data = {}
File.open('syslogemu.log') do |f|
     f.read.split("\n").each do |line|
         parts = line.split
         dst = parts[10].split('=')[1]
         if targets.size == 0 or targets.include?(dst)
             src = parts[9].split('=')[1]
             data[src] ||= 0
             data[src] += 1
         end
     end
end

data.to_a.sort_by { |e| e[0] }.each do |ip, conns|
     puts "#{ip} => #{conns}"
end

gw:/var/log/ulog# ./newconns.rb 87.248.113.14 206.190.60.37 68.180.206.184 
64.191.203.30
targets: ["87.248.113.14", "206.190.60.37", "68.180.206.184", "64.191.203.30"]
192.168.0.3 => 3

the 0.3 host is me trying to open digg.com

what the hell? No packets there and still it doesn't work? :((( I'm running out 
of ideas.

Re: Mystics of packet forwarding

[/dev/rob0]

On Wed January 7 2009 16:36:20 Artūras Šlajus wrote:
> I'm lost. And desperate.

Where did we see your "iptables-save -c" output? I looked through the 
whole thread just now, can't find it.

My WAG without seeing your rules is that they're complex and insane; 
also, I bet you're missing a state rule for return packets.

SIMPLIFY. Start off with a nice simple ruleset that works, something 
along the lines of Rusty's Really Quick Guide to $FOO (for values of 
FOO of "Packet Filtering" and "NAT".)
-- 
    Offlist mail to this address is discarded unless
    "/dev/rob0" or "not-spam" is in Subject: header