Redirect Packets From Interface in Promiscuous Mode

Redirect Packets From Interface in Promiscuous Mode

[Rodre Ghorashi-Zadeh]

Hello List,

I have a problem that I am hoping someone can help me with. I am currently 
conducting some load testing on a test MySQL server that is destined to 
replace our current production MySQL server. What I want to do is send our 
current MySQL traffic, in real time, to the test MySQL server to measure the 
load in comparison with our current production MySQL server.

What I have done so far is setup port mirroring on the switch that is shared 
by the Production MySQL server, the test MySQL server, and my Linux based 
management station, with the Linux based managment station sniffing the 
mirrored port of the Production MySQL server.

I am able to see the MySQL traffic going to the production MySQL server 
using tcpdump. For a next step what I want to be able to do is have iptables 
sniff the port (I set the interface into promiscuous mode using ifconfig), 
grab all packets that are destined for my Production MySQL server from the 
mirrored port, rewrite the source IP address to be my Linux based management 
stations IP address and rewrite the destination address to be my Test MySQL 
Servers IP address. I don't really care about the MySQL results returned to 
tthe Linux based management station, they can go to /dev/null for all I 
care, but I want to see the queries going through the Test MySQL server.

I am pretty familiar with iptables but the two problems that I see/foresee 
are: getting iptables to grap traffic from the interface in promiscuous mode 
that are not destined for the Linux based Management station, and, properly 
setting up the TCP/IP sessions, between the Test MySQL Server and the Linux 
Based Management Station.

If what I am doing is not possible can anyone give me some advice on how to 
achieve what I am trying to do (send queries in realt time from my 
Production MySQL server to my Test MySQL Server)?

~Rodre

Re: Redirect Packets From Interface in Promiscuous Mode

[Henrik Nordstrom]

On Thu, 29 Sep 2005, Rodre Ghorashi-Zadeh wrote:

> I have a problem that I am hoping someone can help me with. I am currently 
> conducting some load testing on a test MySQL server that is destined to 
> replace our current production MySQL server. What I want to do is send our 
> current MySQL traffic, in real time, to the test MySQL server to measure the 
> load in comparison with our current production MySQL server.

Ok. But not at all trivial to do.

> What I have done so far is setup port mirroring on the switch that is shared 
> by the Production MySQL server, the test MySQL server, and my Linux based 
> management station, with the Linux based managment station sniffing the 
> mirrored port of the Production MySQL server.

Ok.

> I am able to see the MySQL traffic going to the production MySQL server using 
> tcpdump. For a next step what I want to be able to do is have iptables sniff 
> the port (I set the interface into promiscuous mode using ifconfig), grab all 
> packets that are destined for my Production MySQL server from the mirrored 
> port, rewrite the source IP address to be my Linux based management stations 
> IP address and rewrite the destination address to be my Test MySQL Servers IP 
> address.

This won't work. MySQL uses TCP and you can't mirror a TCP stream like 
this.

> I don't really care about the MySQL results returned to tthe Linux 
> based management station, they can go to /dev/null for all I care, but I want 
> to see the queries going through the Test MySQL server.

Ok. This simplifies things somewhat.

Can be done in two different manners. Both requires programming.

a) By sniffing and using a TCP stream reassembly tool and when a MySQL 
query has been reassembled from the TCP stream send it to the test server. 
Drawbacks is that TCP stream reassembly is not always reliable (packets 
may have been dropped and a number of other complications).

b) Use a MySQL proxy via iptables REDIRECT and send each query to both 
servers. This requires the intercepting box to be between the old server 
and the network. Main drawback is that the source IP seen by the old 
server will be changed to the address of the proxy.

Regards
Henrik

Re: Redirect Packets From Interface in Promiscuous Mode

[Rodre Ghorashi-Zadeh]

Hi Henrik,

Thanks for your response. Going down the path of your recommendation b) 
below, it raises a few questions for me.

1) If I put the "sniffing" interface in promiscuous mode, can I not get 
iptables to grab traffic that is not destined for the host it is running on? 
For example, if the Production SQL servers IP address is 192.168.1.1, and my 
Linux based management station's IP address is 192.168.1.2, and the Test 
MySQL servers IP address is 192.168.1.3, can I not have a rule that say 
something like:

iptables -t nat -I PREROUTING -i eth0 -p tcp -d 192.168.1.1 --dport 3306 -j 
REDIRECT --to 192.168.1.3

I guess my question is how can I get iptables to grab traffic that is not 
destined for any of the IP addresses on the host system?

2) How am I going to stop the return traffic/SQL results from being returned 
to the original client?

Thanks again for your help.

~Rodre


>From: Henrik Nordstrom <hno@marasystems.com>
>To: Rodre Ghorashi-Zadeh <rodrico7@hotmail.com>
>CC: netfilter@lists.netfilter.org
>Subject: Re: Redirect Packets From Interface in Promiscuous Mode
>Date: Fri, 30 Sep 2005 14:29:55 +0200 (CEST)
>
>On Thu, 29 Sep 2005, Rodre Ghorashi-Zadeh wrote:
>
>>I have a problem that I am hoping someone can help me with. I am currently 
>>conducting some load testing on a test MySQL server that is destined to 
>>replace our current production MySQL server. What I want to do is send our 
>>current MySQL traffic, in real time, to the test MySQL server to measure 
>>the load in comparison with our current production MySQL server.
>
>Ok. But not at all trivial to do.
>
>>What I have done so far is setup port mirroring on the switch that is 
>>shared by the Production MySQL server, the test MySQL server, and my Linux 
>>based management station, with the Linux based managment station sniffing 
>>the mirrored port of the Production MySQL server.
>
>Ok.
>
>>I am able to see the MySQL traffic going to the production MySQL server 
>>using tcpdump. For a next step what I want to be able to do is have 
>>iptables sniff the port (I set the interface into promiscuous mode using 
>>ifconfig), grab all packets that are destined for my Production MySQL 
>>server from the mirrored port, rewrite the source IP address to be my 
>>Linux based management stations IP address and rewrite the destination 
>>address to be my Test MySQL Servers IP address.
>
>This won't work. MySQL uses TCP and you can't mirror a TCP stream like 
>this.
>
>>I don't really care about the MySQL results returned to tthe Linux based 
>>management station, they can go to /dev/null for all I care, but I want to 
>>see the queries going through the Test MySQL server.
>
>Ok. This simplifies things somewhat.
>
>Can be done in two different manners. Both requires programming.
>
>a) By sniffing and using a TCP stream reassembly tool and when a MySQL 
>query has been reassembled from the TCP stream send it to the test server. 
>Drawbacks is that TCP stream reassembly is not always reliable (packets may 
>have been dropped and a number of other complications).
>
>b) Use a MySQL proxy via iptables REDIRECT and send each query to both 
>servers. This requires the intercepting box to be between the old server 
>and the network. Main drawback is that the source IP seen by the old server 
>will be changed to the address of the proxy.
>
>Regards
>Henrik

Re: Redirect Packets From Interface in Promiscuous Mode

[Henrik Nordstrom]

On Fri, 30 Sep 2005, Rodre Ghorashi-Zadeh wrote:

> Hi Henrik,
>
> Thanks for your response. Going down the path of your recommendation b) 
> below, it raises a few questions for me.
>
> 1) If I put the "sniffing" interface in promiscuous mode, can I not get 
> iptables to grab traffic that is not destined for the host it is running on? 
> For example, if the Production SQL servers IP address is 192.168.1.1, and my 
> Linux based management station's IP address is 192.168.1.2, and the Test 
> MySQL servers IP address is 192.168.1.3, can I not have a rule that say 
> something like:
>
> iptables -t nat -I PREROUTING -i eth0 -p tcp -d 192.168.1.1 --dport 3306 -j 
> REDIRECT --to 192.168.1.3

With some additional tricks you can do this, but it won't accomplish what 
you are trying to do. This will either result in a lot of extra network 
traffic, or total failure to communicate.

> I guess my question is how can I get iptables to grab traffic that is not 
> destined for any of the IP addresses on the host system?

Use ebtables or divert to make make the packets directed to the host. 
Iptables will then pick them up.

> 2) How am I going to stop the return traffic/SQL results from being returned 
> to the original client?

You won't even get that far. Either the TCP setup will fail completely, or 
only the production server will decode it properly.

What you describe above is NOT alternative 'b'.

>> b) Use a MySQL proxy via iptables REDIRECT and send each query to both 
>> servers. This requires the intercepting box to be between the old server 
>> and the network. Main drawback is that the source IP seen by the old server 
>> will be changed to the address of the proxy.
>>

To clarify:

To do 'b' the network needs to look like

                                   /--- test server
                                  /
[clients] -> Intercepting box --x
                                  \
                                   \--- production server


On the interception box you use a custom written mysql proxy which not 
only sends the received query to the production server but also echoes it 
to the test server. iptables REDIRECT is used to deliver the requests 
received from the clients to the proxy instead of the production server.



In alternative 'a' the situation is slightly differetn

             (mirror port)
                  /--- Sniffer -> Test server
                  |
[clients]  -> Switch -> Production server


On the sniffer you need to run some sniffing software which does stream 
reassembly of the MySQL traffic and then resends these as MySQL queries to 
the test server. iptables is not involved at all.



In both cases some programming is required. It is not just a simple case 
of making the corret iptables rule.


Regards
Henrik

Re: Redirect Packets From Interface in Promiscuous Mode

[Mart Frauenlob]

Rodre Ghorashi-Zadeh wrote:
> Hello List,
> 
> I have a problem that I am hoping someone can help me with. I am 
> currently conducting some load testing on a test MySQL server that is 
> destined to replace our current production MySQL server. What I want to 
> do is send our current MySQL traffic, in real time, to the test MySQL 
> server to measure the load in comparison with our current production 
> MySQL server.
> 
> What I have done so far is setup port mirroring on the switch that is 
> shared by the Production MySQL server, the test MySQL server, and my 
> Linux based management station, with the Linux based managment station 
> sniffing the mirrored port of the Production MySQL server.
> 
> I am able to see the MySQL traffic going to the production MySQL server 
> using tcpdump. For a next step what I want to be able to do is have 
> iptables sniff the port (I set the interface into promiscuous mode using 
> ifconfig), grab all packets that are destined for my Production MySQL 
> server from the mirrored port, rewrite the source IP address to be my 
> Linux based management stations IP address and rewrite the destination 
> address to be my Test MySQL Servers IP address. I don't really care 
> about the MySQL results returned to tthe Linux based management station, 
> they can go to /dev/null for all I care, but I want to see the queries 
> going through the Test MySQL server.
> 
> I am pretty familiar with iptables but the two problems that I 
> see/foresee are: getting iptables to grap traffic from the interface in 
> promiscuous mode that are not destined for the Linux based Management 
> station, and, properly setting up the TCP/IP sessions, between the Test 
> MySQL Server and the Linux Based Management Station.
> 
> If what I am doing is not possible can anyone give me some advice on how 
> to achieve what I am trying to do (send queries in realt time from my 
> Production MySQL server to my Test MySQL Server)?
> 
> ~Rodre

Hello,

I think there may be a way that would not require programming, and would 
be quite easy to setup. It's theoretical, so not tested, if it would 
really work.

All that is needed is an arp poisoning tool and a few iptables rules.
If you install an arp poisoning program i.e. ettercap at your testing 
mysql server, and use it to establish one way poisoning, traffic will go 
through your testing system and will be transparentely redirected to 
your production server. Now if you us the iptables MIRROR target (i 
think it's so called) on your testing server to redirect the mysql 
queries to localhost, your testing mysql db should receive exactly the 
same queries as the production one. This only works, if iptables  can 
match traffic which does not hat the local IP as destination. But i 
guess it should be able, or asking different, why not?
Now your production server answers all queries as normal, but the 
testing server will also answer. To face this, I think the best would be 
to use either ettercap filters, or to simply drop the packets from the 
testing server using an iptables rule with a source MAC address match.
The only disadvantage I can see is, that the testing server will only 
receive queries as fast as the production server, as clients can only 
reply when the production server answers.

I hope this is understandable and even more I hope it will work.

Regards
Mart

Re: Redirect Packets From Interface in Promiscuous Mode

[Henrik Nordstrom]

On Sun, 2 Oct 2005, Mart Frauenlob wrote:

> All that is needed is an arp poisoning tool and a few iptables rules.
> If you install an arp poisoning program i.e. ettercap at your testing mysql 
> server, and use it to establish one way poisoning, traffic will go through 
> your testing system and will be transparentely redirected to your production 
> server. Now if you us the iptables MIRROR target (i think it's so called) on 
> your testing server to redirect the mysql queries to localhost, your testing 
> mysql db should receive exactly the same queries as the production one.

Nope, this won't work.

MySQL uses TCP for it's communication, and you can not fork a TCP 
connection like this at the packet level as TCP absolutely requires that 
both enpoints talk to each other and agree on a number of things. Thats 
partly why TCP has the two-way SYN handshake (SYN -> SYN+ACK -> ACK) where 
this is initialized. Having three endpoints on the same TCP connection is 
impossible.

What he needs to do is to somehow extract the MySQL queries from the 
connection to the production server and then send these in new TCP queries 
to the test server.

The problem is not iptables, but to get past the test servers TCP/IP stack 
and get the queries delivered to the MySQL server daemon.

Regards
Henrik