[LARTC] why shape incoming traffic

[LARTC] why shape incoming traffic

[Don Cohen]

 > On Thu, Feb 28, 2002 at 10:08:16AM +0100, Martin Devera wrote:
 > > qdisc (shaper) only for outgoing data. It is generaly
 > > believed to be dumb to throttle data when they already
 > > reached your computer or gateway.
 > 
 > The only reason I've ever been able to see for incoming data shaping is 
 > to reorder packets so that, for example, FINs are sent to their 
 > respective applications before SYNs (just an example) or ACKs before data.

It doesn't seem very important to shape the incoming traffic that will
be forwarded, since the same shaping can be done at output.

However, it does seem useful to be able to shape the incoming traffic 
destined for the local machine.

For example, suppose this machine is running a server that you want
to limit to 10 connections/minute.  It seems reasonable to do this
by limiting the rate at which syns are delivered to that server.
That might be a lot easier than trying to modify the server.

You might argue that doing it in the server would have the advantage
of being able to make more intelligent decisions about which ones to
accept and which to drop, but in fact the opposite could also be the
case.  (I'm working on a project that provides an example.)

Similarly, as a way to limit resource usage, you might want to limit
the rate at which a server or client gets input (and sends output, but
you can already do that).

What I find frustrating is that, as a firewall, I can already do this
stuff for the servers (and clients) running on OTHER hosts, but I
can't do it for those running on the local machine!
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[bert hubert]

On Thu, Feb 28, 2002 at 05:53:15PM -0800, Don Cohen wrote:
>  > On Thu, Feb 28, 2002 at 10:08:16AM +0100, Martin Devera wrote:
>  > > qdisc (shaper) only for outgoing data. It is generaly
>  > > believed to be dumb to throttle data when they already
>  > > reached your computer or gateway.
>  > 
>  > The only reason I've ever been able to see for incoming data shaping is 
>  > to reorder packets so that, for example, FINs are sent to their 
>  > respective applications before SYNs (just an example) or ACKs before data.
> 
> It doesn't seem very important to shape the incoming traffic that will
> be forwarded, since the same shaping can be done at output.

An important reason to be able to do incoming shaping is that you may lose
information while routing, which doesn't help you with shaping on egress.
For example, to shape traffic going via a proxy server, outgoing.

There is no way to recognize on ppp0 that traffic was generated by an
internal host who sent it to eth0, where it was packaged by a proxy, and
sent out over ppp0.

With ingress shaping I would be able to do that.

So in short - if you like CBQ, HTB etc on egress, and you accept that
informationloss occurs between ingress and egress, you want to be able to
shape ingress.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://www.tk                              the dot in .tk
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Don Cohen]

bert hubert writes:
 > > It doesn't seem very important to shape the incoming traffic that will
 > > be forwarded, since the same shaping can be done at output.
 > 
 > An important reason to be able to do incoming shaping is that you may lose
 > information while routing, which doesn't help you with shaping on egress.
Isn't the usual solution to mark the packets at input and use the
marks for shaping?  When is that not adequate?
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[bert hubert]

On Fri, Mar 01, 2002 at 07:47:12AM -0800, Don Cohen wrote:
> bert hubert writes:
>  > > It doesn't seem very important to shape the incoming traffic that will
>  > > be forwarded, since the same shaping can be done at output.
>  > 
>  > An important reason to be able to do incoming shaping is that you may lose
>  > information while routing, which doesn't help you with shaping on egress.
> Isn't the usual solution to mark the packets at input and use the
> marks for shaping?  When is that not adequate?

Try getting your mark to survive squid.

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://www.tk                              the dot in .tk
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Michael T. Babcock]

On Thu, Feb 28, 2002 at 05:53:15PM -0800, Don Cohen wrote:
> It doesn't seem very important to shape the incoming traffic that will
> be forwarded, since the same shaping can be done at output.

Obviously ...

> However, it does seem useful to be able to shape the incoming traffic 
> destined for the local machine.

... the only other option, besides rejection of course ;-)

> For example, suppose this machine is running a server that you want
> to limit to 10 connections/minute.  It seems reasonable to do this
> by limiting the rate at which syns are delivered to that server.
> That might be a lot easier than trying to modify the server.

I use application-level software for that (iplimit in conjunction with
tcpserver allows me to limit per-service connection rates on a per-source
basis).

> You might argue that doing it in the server would have the advantage
> of being able to make more intelligent decisions about which ones to
> accept and which to drop, but in fact the opposite could also be the
> case.  (I'm working on a project that provides an example.)

I agree with the Unix-way of doing things usually, not the emacs way --
don't build it into the program if it works just as well outside the
program; thus iplimit.  Programs that accept their own connections,
like Apache, can't use an external program of course (yet), so it would
make sense to build this in although I proxy my incoming connections
through a Squid service set up as an accelerator with bandwidth pools
turned on.

> What I find frustrating is that, as a firewall, I can already do this
> stuff for the servers (and clients) running on OTHER hosts, but I
> can't do it for those running on the local machine!

I've got around that in some situations by teaching myself to treat the
gateway/firewall box as a non-service box -- it runs nothing but the
firewalls, tunneling, forwarding and shaping.  This allows it to work as
desired (all traffic goes out one of the two interfaces; none goes
to the local host).
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Michael T. Babcock]

On Fri, Mar 01, 2002 at 04:50:10PM +0100, bert hubert wrote:
> > Isn't the usual solution to mark the packets at input and use the
> > marks for shaping?  When is that not adequate?
> 
> Try getting your mark to survive squid.

Of course, one could spend a few months figuring out how to track marks 
within squid so that outgoing requests have the same TCP options set as 
the incoming ones ...

PS, 'months' being estimate based on prior readings of source ...
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Don Cohen]

 > From: "Michael T. Babcock" <mbabcock@fibrespeed.net>

 > > It doesn't seem very important to shape the incoming traffic that will
 > > be forwarded, since the same shaping can be done at output.
 > Obviously ...
Bert didn't think it was so obvious.
(I don't understand yet what squid is doing.)

 > > For example, suppose this machine is running a server that you want
 > > to limit to 10 connections/minute.  It seems reasonable to do this
 > > by limiting the rate at which syns are delivered to that server.
 > > That might be a lot easier than trying to modify the server.

 > I use application-level software for that (iplimit in conjunction with
 > tcpserver allows me to limit per-service connection rates on a per-source
 > basis).
There are benefits to doing this sort of thing at the firewall.
For instance, it works for servers running different OS's.
And once you have the control working for your inside network it
seems unreasonable to have to change the implementation in order
to move a server to the firewall machine.

(Another advantage of doing it at the firewall is that you can then
control the aggregate of traffic to different hosts.  But the ability
to shape traffic to the local host would not be enough to retain that
advantage when moving a server to the firewall.)

 > > You might argue that doing it in the server would have the advantage
 > > of being able to make more intelligent decisions about which ones to
 > > accept and which to drop, but in fact the opposite could also be the
 > > case.  (I'm working on a project that provides an example.)
 > 
 > I agree with the Unix-way of doing things usually, not the emacs way --
 > don't build it into the program if it works just as well outside the
 > program; thus iplimit.  Programs that accept their own connections,
 > like Apache, can't use an external program of course (yet), so it would
 > make sense to build this in although I proxy my incoming connections
 > through a Squid service set up as an accelerator with bandwidth pools
 > turned on.
I'm not sure what the emacs reference was all about.  I'm not
suggesting building the limit into the server, just the opposite.
(My solution also works with apache.) 

 > > What I find frustrating is that, as a firewall, I can already do this
 > > stuff for the servers (and clients) running on OTHER hosts, but I
 > > can't do it for those running on the local machine!
 > 
 > I've got around that in some situations by teaching myself to treat the
 > gateway/firewall box as a non-service box -- it runs nothing but the
 > firewalls, tunneling, forwarding and shaping.  This allows it to work as
 > desired (all traffic goes out one of the two interfaces; none goes
 > to the local host).
That's a solution dictated by the necessity imposed by our inability 
to shape traffic to the local host.  Clearly there are times when 
you'd rather run something on the firewall.  There are also times
when you have to - programs that are supposed to communicate with the
outside world to control the firewall.  It's just as important (maybe 
more) to shape the traffic to those programs as to others.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Michael T. Babcock]

On Fri, Mar 01, 2002 at 02:12:07PM -0800, Don Cohen wrote:
> I'd argue that this request is coming from the local machine and
> should be classified as such.  I gather it's not forwarding the
> original packets. 

You're correct that it is not sending the original packets, but if 300
users from 5 departments with bandwidth sharing limits are browsing the
web with one copy of Squid between them and the 'net, its very nice if
Squid's web usage gets taken into account when it has to fetch sites.

> What I don't understand: is the user sending packets to the site with
> the original web page or to the squib server?  If the original site,
> then how is the squib server getting them?

That depends on your configuration; Squid can be set up as a transparent
proxy so that all requests made to given ports (80, 443, etc.) are forced
through Squid instead so that the user doesn't have the choice.

> The point is that I want to maintain legitimate communication with
> these servers when someone is trying to attack them.

If all you want is to have legitimate use, set up a class for interactive
traffic, reserve it some amount of bandwidth, cap it at another amount (to
keep a user from using SSH to eat all your bandwidth) and add SFQ to it to
make sure no one connection gets all the bits.

My new Internet gateway box is actually set up without any login or port
services except forwarding and firewalls -- there are no listening ports at
all and no way for me to log into it except over serial port or at the
terminal.  The serial port is connected to an external modem; logging in
is therefore "out of band".  Any form of non serial login is assumed to be
an intrusion, but that's off-topic ;-).
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Michael T. Babcock]

On Sat, Mar 02, 2002 at 11:00:20AM -0800, Don Cohen wrote:
>  > That depends on your configuration; Squid can be set up as a transparent
>  > proxy so that all requests made to given ports (80, 443, etc.) are forced
>  > through Squid instead so that the user doesn't have the choice.
> So squid is intercepting packets addressed to somewhere else?
> How is it doing that?

Usually through port redirection using your firewall (or ipchains ;-).

> SFQ is not a good defense - the attacker just sends you random source
> addresses and ports and now his packets have priority over yours
> (which all come from the same address/port).  But you're close.

That only works if traffic is generated on all of those hashed address/port
pairs in which case the attacker's data flow is just as stymied as mine.
-- 
Michael T. Babcock
CTO, FibreSpeed Ltd.     (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] why shape incoming traffic

[Don Cohen]

 > From: "Michael T. Babcock" <mbabcock@fibrespeed.net>

 > On Sat, Mar 02, 2002 at 11:00:20AM -0800, Don Cohen wrote:
 > >  > That depends on your configuration; Squid can be set up as a transparent
 > >  > proxy so that all requests made to given ports (80, 443, etc.) are forced
 > >  > through Squid instead so that the user doesn't have the choice.
 > > So squid is intercepting packets addressed to somewhere else?
 > > How is it doing that?
 > 
 > Usually through port redirection using your firewall (or ipchains ;-).

Ok, so I think it's reasonable to view the outgoing http client
packets as coming from the squid machine, and they can/should be
shaped/accounted as such.

That brings us back to my original argument that it's not necessary
to shape incoming traffic that is to be forwarded, since that traffic
can be shaped on the way out.  This is not strictly true, since the
work at input could save some work between the input and output, but
it's a pretty good approximation.  On the other hand, if you accept my
argument that it's useful to shape traffic to the local machine, then
you agree with me that it would be worth while to add shaping at
input, in which case the case above that I suggest is unnecessary
comes for free.

 > > SFQ is not a good defense - the attacker just sends you random source
 > > addresses and ports and now his packets have priority over yours
 > > (which all come from the same address/port).  But you're close.
 > 
 > That only works if traffic is generated on all of those hashed address/port
 > pairs in which case the attacker's data flow is just as stymied as mine.

Attackers generally don't really want to communicate with you.  They
just want to prevent others from doing so.  In any case, I'm trying to
protect legitimate communication even when under an attack by that
sort of attacker.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/