[Bridge] Performance loss with bridging+routing.

* [Bridge] Performance loss with bridging+routing.
@ 2007-03-15 17:05 Egi, Norbert
  2007-03-15 17:59 ` Egi, Norbert
  0 siblings, 1 reply; 6+ messages in thread
From: Egi, Norbert @ 2007-03-15 17:05 UTC (permalink / raw)
  To: bridge

Hi,

I have a machine with 4 Ethernet intefaces and I set up the configuration you can see below (only the first "figure"). There are two subnetworks each having a bridge with 2 interfaces and both of the bridges have an IP address for packet forwarding between the two subnetworks.

HOST1 --------------- ETH0                            ETH2 --------------- HOST3
10.1.1.3                           |                                     |                             10.2.2.3
                                     BR0 --- ROUTING --- BR1
                                  10.1.1.1                         10.2.2.1
                                        |                                     |
HOST2 --------------- ETH1                            ETH3 --------------- HOST4
10.1.1.2                                                                                              10.2.2.2

My problem is that if I forward packets (64byte minimum size packets) between two machines on the same network (i.e. only bridging is done) I get ~700.000 pps throughput, while if packets are going from one subnetwork to the other, so packets are also routed between the two subnetwork, the forwarded packet rate is only ~525.000 pps, so ~25% worse.

To find out how bridging and routing perform on their own with only 2 interfaces I set up the following to topologies and forwarded packets from HOST1 to HOST2.

                                        --- BRIDGING ---
                                        |                            |
HOST1 --------------- ETH0                    ETH1 --------------- HOST2
10.1.1.1                Promisc(0.0.0.0)       Promisc(0.0.0.0)            10.1.1.2

                                        --- ROUTING ---
                                        |                           |
HOST1 --------------- ETH0                   ETH1 --------------- HOST2
10.1.1.2                      10.1.1.1                10.2.2.1                      10.2.2.2

With the bridged setup I get ~700.000 pps like before and in the routed setup ~685.000 pps, just a little bit below the bridged setup. This is good, exactly what I would have expect. However when I set up the configuration below, which is very similar to the first one above, only two interfaces are missing, I only get ~525.000 pps again.

                                        --- ROUTING ---
                                        |                           |
                                     BR0                     BR1
                                  10.1.1.1               10.2.2.1
                                        |                           |
HOST1 --------------- ETH0                   ETH1 --------------- HOST2
10.1.1.2                Promisc(0.0.0.0)       Promisc(0.0.0.0)           10.2.2.2

I have expected a little performance degradation compared to plain routing as a packet traverses two bridges besides being routed as well, but I think ~175.000 pps is too much.

I checked the MAC addresses learnt by both bridges and everything seemed alright. My assumption is that the bottleneck is in the bridge when it has to pass the packets up to the IP layer as opposed to when it has to send the packet out on another port, but I wouldn't expect such a big difference as it looks to me that very similar things should go on inside the bridge in both cases.

Has anyone had similar experience or I'm doing something wrong when combining bridging and routing?
If this is a normal behaviour could anyone tell me where and why the packets got dropped? Or why the performance loss occurs?

Any suggestions are very welcome.

Norbert

^ permalink raw reply	[flat|nested] 6+ messages in thread