Discriminate client requests from transparent proxy requests?

Discriminate client requests from transparent proxy requests?

[Jack Bates]

Do you have any advice how to discriminate traffic from clients from 
traffic from our transparent proxy?

Our proxy sends requests to origin servers with the same source address 
as the request from the client, so we can do per-host traffic shaping on 
our router. But consequently I wonder how to discriminate client 
requests from proxy requests, and route the former to the proxy, but not 
route the latter. What options are there?

Re: Discriminate client requests from transparent proxy requests?

[Jan Engelhardt]

On Tuesday 2012-12-18 08:45, Jack Bates wrote:

>Do you have any advice how to discriminate traffic from clients from 
>traffic from our transparent proxy?
>
>Our proxy sends requests to origin servers with the same source address 
>as the request from the client, so we can do per-host traffic shaping 
>on our router. [What options are there?]

I take it you might be using something like squid and tproxy.

>But consequently I wonder how to discriminate client requests from proxy
>requests,

The origin servers can inspect the "Via:" HTTP request header to 
determine the presence of a proxy. There might be other headers your 
proxy server is emitting that you would normally not find in a pure 
client request.

A second possibility, when proxy server and origin server are on the 
same Ethernet subnet, is to look at the L2 address. Of course the L2 
addr can be "tproxified" as well, but usually is not worth doing.

>and route the former to the proxy, but not route the latter.

As you have noticed, if the original client address is used, routing 
topology/rules needs to be laid out such that packets to client 
addresses always pass through the proxy server machine in both 
directions. (This is the same prerequisite as for connection-tracked 
NAT.)

Re: Discriminate client requests from transparent proxy requests?

[Jack Bates]

On 18/12/12 12:27 AM, Jan Engelhardt wrote:
> On Tuesday 2012-12-18 08:45, Jack Bates wrote:
>
>> Do you have any advice how to discriminate traffic from clients from
>> traffic from our transparent proxy?
>>
>> Our proxy sends requests to origin servers with the same source address
>> as the request from the client, so we can do per-host traffic shaping
>> on our router. [What options are there?]
>
> I take it you might be using something like squid and tproxy.

You're right, we're using Apache Traffic Server and tproxy. Traffic 
Server is configured with "CONFIG proxy.config.http.server_ports STRING 
8080:tr-full" to make both connections to clients and connections to 
origin servers transparent.

>> But consequently I wonder how to discriminate client requests from proxy
>> requests,
>
> The origin servers can inspect the "Via:" HTTP request header to
> determine the presence of a proxy. There might be other headers your
> proxy server is emitting that you would normally not find in a pure
> client request.

Interesting suggestion, I hadn't thought of inspecting the application 
layer. I want our router (not the origin server) to discriminate between 
requests from clients and requests from the proxy (to reroute the former 
via the transparent proxy, but not the latter) so I had in mind teaching 
iptables to recognize the difference.

We run Traffic Server as a forward proxy.

> A second possibility, when proxy server and origin server are on the
> same Ethernet subnet, is to look at the L2 address. Of course the L2
> addr can be "tproxified" as well, but usually is not worth doing.

This is a possibility, with "iptables -m mac --mac-source ..." The proxy 
and the router are on the same subnet. Are there any other options?

I tried adding a second IP to the router, as an alias, changing the 
default gateway of the proxy to this other address, and matching traffic 
from the proxy with "iptables -i br-lan:1" but I discovered that 
--in-interface doesn't support aliases (I guess this makes sense, 
traffic doesn't reference the IP of the next hop, so how can you tell 
which alias it arrived on?)

>> and route the former to the proxy, but not route the latter.
>
> As you have noticed, if the original client address is used, routing
> topology/rules needs to be laid out such that packets to client
> addresses always pass through the proxy server machine in both
> directions. (This is the same prerequisite as for connection-tracked
> NAT.)

Discriminating between responses from origin servers and responses from 
the proxy is easier because the proxy is on a different router interface 
than our internet connection, so I use the following to reroute 
responses via the transparent proxy:

iptables -A PREROUTING -t mangle -i eth0.2 -p tcp --sport 80 -j MARK 
--set-mark 1/1
ip rule add fwmark 1/1 table 1
ip route add table 1 via 192.168.1.35

Re: Discriminate client requests from transparent proxy requests?

[Jan Engelhardt]

On Wednesday 2012-12-19 17:41, Jack Bates wrote:
>
>> A second possibility, when proxy server and origin server are on the
>> same Ethernet subnet, is to look at the L2 address. Of course the L2
>> addr can be "tproxified" as well, but usually is not worth doing.
>
> This is a possibility, with "iptables -m mac --mac-source ..." The proxy and
> the router are on the same subnet. Are there any other options?
>
> I tried adding a second IP to the router, as an alias, changing the default
> gateway of the proxy to this other address, and matching traffic from the proxy
> with "iptables -i br-lan:1" but I discovered that --in-interface doesn't
> support aliases (I guess this makes sense, traffic doesn't reference the IP of
> the next hop, so how can you tell which alias it arrived on?)

Obviously, using the iptables -d option.

>>> and route the former to the proxy, but not route the latter.
>>
>> As you have noticed, if the original client address is used, routing
>> topology/rules needs to be laid out such that packets to client
>> addresses always pass through the proxy server machine in both
>> directions. (This is the same prerequisite as for connection-tracked
>> NAT.)
>
> Discriminating between responses from origin servers and responses from the
> proxy is easier because the proxy is on a different router interface than our
> internet connection, so I use the following to reroute responses via the
> transparent proxy:
>
> iptables -A PREROUTING -t mangle -i eth0.2 -p tcp --sport 80 -j MARK --set-mark
> 1/1

You probably know that, by using CONNMARK, you can always mark it..

-i eth0.2 -j CONNMARK --mark 1 all packets coming from the proxy server,
-i internet -j CONNMARK --restore-mark for all packets from $internet

and then routing back to the proxy also works - based solely on fwmark.

> ip rule add fwmark 1/1 table 1
> ip route add table 1 via 192.168.1.35

Re: Discriminate client requests from transparent proxy requests?

[Jack Bates]

On 19/12/12 01:51 PM, Jan Engelhardt wrote:
>
> On Wednesday 2012-12-19 17:41, Jack Bates wrote:
>>
>>> A second possibility, when proxy server and origin server are on the
>>> same Ethernet subnet, is to look at the L2 address. Of course the L2
>>> addr can be "tproxified" as well, but usually is not worth doing.
>>
>> This is a possibility, with "iptables -m mac --mac-source ..." The proxy and
>> the router are on the same subnet. Are there any other options?
>>
>> I tried adding a second IP to the router, as an alias, changing the default
>> gateway of the proxy to this other address, and matching traffic from the proxy
>> with "iptables -i br-lan:1" but I discovered that --in-interface doesn't
>> support aliases (I guess this makes sense, traffic doesn't reference the IP of
>> the next hop, so how can you tell which alias it arrived on?)
>
> Obviously, using the iptables -d option.

--destination is the ultimate destination. Say our "br-lan" router 
interface is 192.168.1.1 and our "br-lan:1" alias is 192.168.1.84. The 
default gateway for clients is 192.168.1.1, so when a client makes a 
request to an origin server, 12.34.56.78, it forwards it to 192.168.1.1 
(--destination is 12.34.56.78)

The router intercepts this request and reroutes it to our transparent 
proxy. The default gateway for the proxy is 192.168.1.84, so when it 
makes a request to the origin server, it forwards it to 192.168.1.84 
(--destination is 12.34.56.78)

I think iptables can't tell whether the request was forwarded to 
192.168.1.1 or 192.168.1.84, so it can't tell whether it arrived on the 
"br-lan" interface or the "br-lan:1" alias?

>>>> and route the former to the proxy, but not route the latter.
>>>
>>> As you have noticed, if the original client address is used, routing
>>> topology/rules needs to be laid out such that packets to client
>>> addresses always pass through the proxy server machine in both
>>> directions. (This is the same prerequisite as for connection-tracked
>>> NAT.)
>>
>> Discriminating between responses from origin servers and responses from the
>> proxy is easier because the proxy is on a different router interface than our
>> internet connection, so I use the following to reroute responses via the
>> transparent proxy:
>>
>> iptables -A PREROUTING -t mangle -i eth0.2 -p tcp --sport 80 -j MARK --set-mark
>> 1/1
>
> You probably know that, by using CONNMARK, you can always mark it..
>
> -i eth0.2 -j CONNMARK --mark 1 all packets coming from the proxy server,
> -i internet -j CONNMARK --restore-mark for all packets from $internet
>
> and then routing back to the proxy also works - based solely on fwmark.

Right, using CONNMARK to reroute responses from origin servers back via 
the proxy would also work. Thanks!

>> ip rule add fwmark 1/1 table 1
>> ip route add table 1 via 192.168.1.35

So some options for discriminating client requests from proxy requests are:

   * Application layer (e.g. Via: header)
   * --mac-source
   * TOS/DSCP field

Are there any other options worth considering? Do you have any advice 
about which to choose?

Re: Discriminate client requests from transparent proxy requests?

[Jan Engelhardt]

On Thursday 2012-12-20 08:42, Jack Bates wrote:

>>> with "iptables -i br-lan:1" but I discovered that --in-interface doesn't
>>> support aliases (I guess this makes sense, traffic doesn't reference the IP
>>> of
>>> the next hop, so how can you tell which alias it arrived on?)
>>
>> Obviously, using the iptables -d option.
>
>--destination is the ultimate destination. The default gateway for the 
>proxy is 192.168.1.84, so when it makes a request to the origin server, 
>it forwards it to 192.168.1.84 (--destination is 12.34.56.78)
>
>I think iptables can't tell whether the request was forwarded to 
>192.168.1.1 or 192.168.1.84, so it can't tell whether it arrived on the 
>"br-lan" interface or the "br-lan:1" alias?

It is not an iptables problem.

The question is, how do you define "arriving on br-lan:1"? That is, 
looking only at one Ethernet packet, how would you tell it is "for 
br-lan:1" rather than for br-lan? Open a hexdump, tell me which byte(s) 
shall represent "br-lan:1".


> So some options for discriminating client requests from proxy requests are:
>
>  * Application layer (e.g. Via: header)
>  * --mac-source

This one usually in the form of -m mac ! --mac-source <proxy's-mac>

>  * TOS/DSCP field

I would not put too much reliance on this one, as hosts take more 
freedom in setting this field to whatever they like.

> Are there any other options worth considering? Do you have any advice about
> which to choose?

With separate interfaces one more option can be constructed.

Re: Discriminate client requests from transparent proxy requests?

[Leonardo Rodrigues]

Em 20/12/12 06:18, Jan Engelhardt escreveu:
>
>>   * TOS/DSCP field
> I would not put too much reliance on this one, as hosts take more
> freedom in setting this field to whatever they like.
>

     as this is, in fact, a useless information for general purposes, 
you can always force all packets with some specific TOS/DSCP field and 
later alter that to meet your criterias. That way, even if hosts set 
those to something that would mess with your internal matching/routing, 
ensuring that it will be changed to your default ones would do it.



-- 


	Atenciosamente / Sincerily,
	Leonardo Rodrigues
	Solutti Tecnologia
	http://www.solutti.com.br

	Minha armadilha de SPAM, NÃO mandem email
	gertrudes@solutti.com.br
	My SPAMTRAP, do not email it

Re: Discriminate client requests from transparent proxy requests?

[Neal Murphy]

On Thursday, December 20, 2012 03:18:43 AM Jan Engelhardt wrote:
> On Thursday 2012-12-20 08:42, Jack Bates wrote:
> >>> with "iptables -i br-lan:1" but I discovered that --in-interface
> >>> doesn't support aliases (I guess this makes sense, traffic doesn't
> >>> reference the IP of
> >>> the next hop, so how can you tell which alias it arrived on?)
> >> 
> >> Obviously, using the iptables -d option.
> >
> >--destination is the ultimate destination. The default gateway for the
> >proxy is 192.168.1.84, so when it makes a request to the origin server,
> >it forwards it to 192.168.1.84 (--destination is 12.34.56.78)
> >
> >I think iptables can't tell whether the request was forwarded to
> >192.168.1.1 or 192.168.1.84, so it can't tell whether it arrived on the
> >"br-lan" interface or the "br-lan:1" alias?
> 
> It is not an iptables problem.
> 
> The question is, how do you define "arriving on br-lan:1"? That is,
> looking only at one Ethernet packet, how would you tell it is "for
> br-lan:1" rather than for br-lan? Open a hexdump, tell me which byte(s)
> shall represent "br-lan:1".

You can... Hmmm... The packet from the proxy to internet (via the gateway) 
doesn't contain 192.168.1.84, does it?

Perhaps the only way to make such a scheme work would be to add a second 
primary address (i.e., in another subnet) to br-lan and to the proxy's NIC. 
Assuming the existing netmask is /24, one could
  ip addr add 192.168.2.1/30 dev br-lan
on the gateway and
  ip addr add 192.168.2.2/30 dev ethN
on the proxy system, then tell the proxy to use 192.168.1.84 on its internal 
side and 192.168.2.2 on its external side. Now netfilter can differentiate the 
traffic using the source address. But now netfilter need do nothing because 
simple routing properly directs the packets.

I'd say you're right. It more closely resembles an addressing/routing issue 
than it does a packet filtering issue.

Re: Discriminate client requests from transparent proxy requests?

[Jan Engelhardt]

On Thursday 2012-12-20 16:54, Neal Murphy wrote:
>> >--destination is the ultimate destination. The default gateway for the
>> >proxy is 192.168.1.84, so when it makes a request to the origin server,
>> >it forwards it to 192.168.1.84 (--destination is 12.34.56.78)
>> >
>> >I think iptables can't tell whether the request was forwarded to
>> >192.168.1.1 or 192.168.1.84, so it can't tell whether it arrived on the
>> >"br-lan" interface or the "br-lan:1" alias?
>> 
>> It is not an iptables problem.
>> 
>> The question is, how do you define "arriving on br-lan:1"? That is,
>> looking only at one Ethernet packet, how would you tell it is "for
>> br-lan:1" rather than for br-lan? Open a hexdump, tell me which byte(s)
>> shall represent "br-lan:1".
>
>You can... Hmmm... The packet from the proxy to internet (via the gateway) 
>doesn't contain 192.168.1.84, does it?
>Perhaps the only way to make such a scheme work would be to add a second 
>primary address (i.e., in another subnet) to br-lan and to the proxy's NIC.
>Assuming the existing netmask is /24, one could `ip addr add 192.168.2.1/30 dev
>br-lan` on the gateway and `ip addr add 192.168.2.2/30 dev ethN`
>on the proxy system, then tell the proxy to use 192.168.1.84 on its internal 
>side and 192.168.2.2 on its external side.

Again: where in the Ethernet/IP packet would that be recorded? Don't
guess; *construct* one. Your favorite IP reference (be that a book,
the RFCs, Wikipedia, ...) can tell you what fields exist and need to
be filled out. Consider yourself a Linux machine sending a packet,
fill in the fields, in the direction from TCP to IP to Ethernet (i.e.
backwards). Once done, parse it (in the forward direction), like a
Linux machine receiving the packet.

The exercise here is that one sees that it is (im)possible.

Re: Discriminate client requests from transparent proxy requests?

[Neal Murphy]

On Thursday, December 20, 2012 02:35:44 PM Jan Engelhardt wrote:
> Again: where in the Ethernet/IP packet would that be recorded? Don't
> guess; *construct* one. Your favorite IP reference (be that a book,
> the RFCs, Wikipedia, ...) can tell you what fields exist and need to
> be filled out. Consider yourself a Linux machine sending a packet,
> fill in the fields, in the direction from TCP to IP to Ethernet (i.e.
> backwards). Once done, parse it (in the forward direction), like a
> Linux machine receiving the packet.
> 
> The exercise here is that one sees that it is (im)possible.

I think I begin to see your point. Bouncy packets don't necessarily work very 
well, especially when there are loops in the path. If the goal is to 
transparently proxy web connections, the least complex solution is to run the 
proxy either on the firewall or on a bridge between the firewall and the LAN.

Re: Discriminate client requests from transparent proxy requests?

[Leonardo Rodrigues]

     How about configuring two ports, one for transparent proxy and 
other for your 'normal' proxy? Doing that, you could create ACLs for 
matching your normal and transparent ports

     Changing the port on your transparent proxy rule will be absolutely 
transparent to your users ...

http_port 12345 transparent
http_port 3128

acl transparent_access myport 12345
acl normal_access myport 3128


     i cant think on any easier way of discriminating normal and 
transparent-intercepted traffic ...


Em 18/12/12 05:45, Jack Bates escreveu:
> Do you have any advice how to discriminate traffic from clients from 
> traffic from our transparent proxy?
>
> Our proxy sends requests to origin servers with the same source 
> address as the request from the client, so we can do per-host traffic 
> shaping on our router. But consequently I wonder how to discriminate 
> client requests from proxy requests, and route the former to the 
> proxy, but not route the latter. What options are there?

-- 


	Atenciosamente / Sincerily,
	Leonardo Rodrigues
	Solutti Tecnologia
	http://www.solutti.com.br

	Minha armadilha de SPAM, NÃO mandem email
	gertrudes@solutti.com.br
	My SPAMTRAP, do not email it

Re: Discriminate client requests from transparent proxy requests?

[Jack Bates]

Thank you, but what I want is for our *router* to be able to tell the 
difference between requests from clients to origin servers (and 
intercept these) and requests from our transparent proxy to origin 
servers (and not intercept these). I'm wondering what options there are 
to do this because our proxy makes "transparent" requests to origin 
servers, with the same source address as the request from the client.

I think what you're describing instead is how the *proxy* can tell the 
difference between requests that were intercepted and requests that were 
explicitly sent to the proxy.

On 18/12/12 05:35 AM, Leonardo Rodrigues wrote:
>
>      How about configuring two ports, one for transparent proxy and
> other for your 'normal' proxy? Doing that, you could create ACLs for
> matching your normal and transparent ports
>
>      Changing the port on your transparent proxy rule will be absolutely
> transparent to your users ...
>
> http_port 12345 transparent
> http_port 3128
>
> acl transparent_access myport 12345
> acl normal_access myport 3128
>
>
>      i cant think on any easier way of discriminating normal and
> transparent-intercepted traffic ...
>
>
> Em 18/12/12 05:45, Jack Bates escreveu:
>> Do you have any advice how to discriminate traffic from clients from
>> traffic from our transparent proxy?
>>
>> Our proxy sends requests to origin servers with the same source
>> address as the request from the client, so we can do per-host traffic
>> shaping on our router. But consequently I wonder how to discriminate
>> client requests from proxy requests, and route the former to the
>> proxy, but not route the latter. What options are there?
>

Re: Discriminate client requests from transparent proxy requests?

[Leonardo Rodrigues]

     how about adjusting TOS values on the packets using those created 
ACLs ?? That would probably make identification easier/possible on 
routing layers, your routers included.

     you can specify a specific TOS value for your 'normal proxy' port 
and another one for your 'transparent proxy'.

     but you're right, i didnt catch your idea and, maybe, my answer was 
for a different scenario than yours. But i think that using the 
transparent port ACL and adjusting TOS on those packets, you could catch 
that on your routers.



from http://www.squid-cache.org/Doc/config/tcp_outgoing_tos/

     Allows you to select a TOS/Diffserv value for packets outgoing
     on the server side, based on an ACL.

     tcp_outgoing_tos ds-field [!]aclname ...

     Example where normal_service_net uses the TOS value 0x00
     and good_service_net uses 0x20

     acl normal_service_net src 10.0.0.0/24
     acl good_service_net src 10.0.1.0/24
     tcp_outgoing_tos 0x00 normal_service_net
     tcp_outgoing_tos 0x20 good_service_net

     TOS/DSCP values really only have local significance - so you should
     know what you're specifying. For more information, see RFC2474,
     RFC2475, and RFC3260.

     The TOS/DSCP byte must be exactly that - a octet value  0 - 255, or
     "default" to use whatever default your host has. Note that in
     practice often only multiples of 4 is usable as the two rightmost bits
     have been redefined for use by ECN (RFC 3168 section 23.1).

     Processing proceeds in the order specified, and stops at first fully
     matching line.


Em 19/12/12 16:33, Jack Bates escreveu:
> Thank you, but what I want is for our *router* to be able to tell the 
> difference between requests from clients to origin servers (and 
> intercept these) and requests from our transparent proxy to origin 
> servers (and not intercept these). I'm wondering what options there 
> are to do this because our proxy makes "transparent" requests to 
> origin servers, with the same source address as the request from the 
> client.
>
> I think what you're describing instead is how the *proxy* can tell the 
> difference between requests that were intercepted and requests that 
> were explicitly sent to the proxy.
>


-- 


	Atenciosamente / Sincerily,
	Leonardo Rodrigues
	Solutti Tecnologia
	http://www.solutti.com.br

	Minha armadilha de SPAM, NÃO mandem email
	gertrudes@solutti.com.br
	My SPAMTRAP, do not email it

Re: Discriminate client requests from transparent proxy requests?

[Jack Bates]

Cool, using the TOS/DSCP field to discriminate client requests from 
proxy requests would work, with "iptables -m tos --tos ..." Thank you!

Are there any other options?

On 19/12/12 11:05 AM, Leonardo Rodrigues wrote:
>
>      how about adjusting TOS values on the packets using those created
> ACLs ?? That would probably make identification easier/possible on
> routing layers, your routers included.
>
>      you can specify a specific TOS value for your 'normal proxy' port
> and another one for your 'transparent proxy'.
>
>      but you're right, i didnt catch your idea and, maybe, my answer was
> for a different scenario than yours. But i think that using the
> transparent port ACL and adjusting TOS on those packets, you could catch
> that on your routers.
>
>
>
> from http://www.squid-cache.org/Doc/config/tcp_outgoing_tos/
>
>      Allows you to select a TOS/Diffserv value for packets outgoing
>      on the server side, based on an ACL.
>
>      tcp_outgoing_tos ds-field [!]aclname ...
>
>      Example where normal_service_net uses the TOS value 0x00
>      and good_service_net uses 0x20
>
>      acl normal_service_net src 10.0.0.0/24
>      acl good_service_net src 10.0.1.0/24
>      tcp_outgoing_tos 0x00 normal_service_net
>      tcp_outgoing_tos 0x20 good_service_net
>
>      TOS/DSCP values really only have local significance - so you should
>      know what you're specifying. For more information, see RFC2474,
>      RFC2475, and RFC3260.
>
>      The TOS/DSCP byte must be exactly that - a octet value  0 - 255, or
>      "default" to use whatever default your host has. Note that in
>      practice often only multiples of 4 is usable as the two rightmost bits
>      have been redefined for use by ECN (RFC 3168 section 23.1).
>
>      Processing proceeds in the order specified, and stops at first fully
>      matching line.
>
>
> Em 19/12/12 16:33, Jack Bates escreveu:
>> Thank you, but what I want is for our *router* to be able to tell the
>> difference between requests from clients to origin servers (and
>> intercept these) and requests from our transparent proxy to origin
>> servers (and not intercept these). I'm wondering what options there
>> are to do this because our proxy makes "transparent" requests to
>> origin servers, with the same source address as the request from the
>> client.
>>
>> I think what you're describing instead is how the *proxy* can tell the
>> difference between requests that were intercepted and requests that
>> were explicitly sent to the proxy.
>>
>
>