[Bridge] MTU Question

[Bridge] MTU Question

[Chris Shaw]

I have a bridge that has gigabit interfaces. The machine in question has the
fun job of being a Bridge, Firewall and SMB server. Both of the Gigabit
interfaces are connected to workstations directly via Xover cable (well
MDI-X to be exact). My question is, if I enable jumbo frames on the gigabit
interfaces will that make any difference in overall transfer rate of the
bridge? I was thinking it might because even though the NIC is enslaved, it
IS the device that the workstations communicate to. But also it might not
because when applications talk to the network they're using BR(n) at
1500bytes.. can a bridging guru shed some light on this for me?

-Thank you in advance
Chris

Re: [Bridge] MTU Question

[shemminger]

> I have a bridge that has gigabit interfaces. The machine in question has
> the
> fun job of being a Bridge, Firewall and SMB server. Both of the Gigabit
> interfaces are connected to workstations directly via Xover cable (well
> MDI-X to be exact). My question is, if I enable jumbo frames on the
> gigabit
> interfaces will that make any difference in overall transfer rate of the
> bridge? I was thinking it might because even though the NIC is enslaved,
> it
> IS the device that the workstations communicate to. But also it might not
> because when applications talk to the network they're using BR(n) at
> 1500bytes.. can a bridging guru shed some light on this for me?
>
> -Thank you in advance
> Chris

The bridge does not care about frame size at all. It should be able
to handle large frames, provided:
* all interfaces in the bridge can handle the same big mtu, there is
  no way a bridge can fragment.
* also, the mtu of the bridge itself will always appear as 1500, but
  you could change that with ifconfig.  Thought about having bridge mtu
  appear as min(ports mtu), but don't think that is in the current version.

Re: [Bridge] MTU Question

[Stephen Hemminger]

Try the following (2.4) patch to allow bridge to be MTU aware.

diff -Nru a/net/bridge/br_device.c b/net/bridge/br_device.c
--- a/net/bridge/br_device.c	2004-06-30 10:30:31 -07:00
+++ b/net/bridge/br_device.c	2004-06-30 10:30:31 -07:00
@@ -121,6 +121,22 @@
 	return -1;
 }
 
+static int br_change_mtu(struct net_device *dev, int new_mtu)
+{
+	struct net_bridge *br = dev->priv;
+	int max_mtu;
+
+	read_lock_bh(&br->lock);
+	max_mtu = br_min_mtu(br);
+	read_unlock_bh(&br->lock);
+
+	if (new_mtu < 68 || new_mtu > max_mtu)
+		return -EINVAL;
+
+	dev->mtu = new_mtu;
+	return 0;
+}
+
 void br_dev_setup(struct net_device *dev)
 {
 	memset(dev->dev_addr, 0, ETH_ALEN);
@@ -134,4 +150,5 @@
 	dev->accept_fastpath = br_dev_accept_fastpath;
 	dev->tx_queue_len = 0;
 	dev->set_mac_address = NULL;
+	dev->change_mtu = br_change_mtu;
 }
diff -Nru a/net/bridge/br_if.c b/net/bridge/br_if.c
--- a/net/bridge/br_if.c	2004-06-30 10:30:31 -07:00
+++ b/net/bridge/br_if.c	2004-06-30 10:30:31 -07:00
@@ -220,6 +220,24 @@
 	return 0;
 }
 
+int br_min_mtu(struct net_bridge *br)
+{
+	struct net_bridge_port *p;
+	int mtu;
+
+	p = br->port_list;
+	if (!p)
+		mtu = 1500;
+	else {
+		mtu = p->dev->mtu;
+		while ((p = p->next) != NULL) {
+			if (p->dev->mtu < mtu)
+				mtu = p->dev->mtu;
+		}
+	}
+	return mtu;
+}
+
 int br_add_if(struct net_bridge *br, struct net_device *dev)
 {
 	struct net_bridge_port *p;
@@ -250,6 +268,7 @@
 	br_fdb_insert(br, p, dev->dev_addr, 1);
 	if ((br->dev.flags & IFF_UP) && (dev->flags & IFF_UP))
 		br_stp_enable_port(p);
+	br->dev.mtu = br_min_mtu(dev->priv);
 	write_unlock_bh(&br->lock);
 
 	return 0;
diff -Nru a/net/bridge/br_private.h b/net/bridge/br_private.h
--- a/net/bridge/br_private.h	2004-06-30 10:30:31 -07:00
+++ b/net/bridge/br_private.h	2004-06-30 10:30:31 -07:00
@@ -164,6 +164,7 @@
 			    int num);
 extern void br_get_port_ifindices(struct net_bridge *br,
 			   int *ifindices);
+extern int br_min_mtu(struct net_bridge *br);
 
 /* br_input.c */
 extern void br_handle_frame(struct sk_buff *skb);

RE: [Bridge] MTU Question

[Eble, Dan]

What if someone (or something) attempts to change the MTU on the port after
it has been added to the bridge?  Perhaps SIOCSIFMTU should do something
special for bridge ports, such as return -EBUSY, or maybe ask the bridge
whether or not the new MTU is acceptable?  If a port's MTU changes, the
bridge may have to adjust its own MTU to maintain consistency.  It could get
hairy.

I faced this question recently when trying to implement bridging over Cisco
HDLC.  I finally decided not to try to maintain consistency of MTUs between
two devices, but rather to drop *all* transmitted packets when the MTU of
the lower-layer device was insufficient to handle the largest possible
packets from the upper-layer device.  I decided that would be enough to
prevent someone from wrongly thinking they had configured it properly, since
not even a cheap a 64-byte ping would work.

Regards,
-- 
Dan Eble <dane@aiinet.com>  _____  .
Software Engineer          |  _  |/|
Applied Innovation Inc.    | |_| | |
http://www.aiinet.com/     |__/|_|_|


> -----Original Message-----
> From: bridge-bounces@lists.osdl.org 
> [mailto:bridge-bounces@lists.osdl.org] On Behalf Of Stephen Hemminger
> Sent: Wednesday, June 30, 2004 1:32 PM
> To: Chris Shaw
> Cc: bridge@osdl.org
> Subject: Re: [Bridge] MTU Question
> 
> 
> 
> Try the following (2.4) patch to allow bridge to be MTU aware.
> 
> diff -Nru a/net/bridge/br_device.c b/net/bridge/br_device.c
> --- a/net/bridge/br_device.c	2004-06-30 10:30:31 -07:00
> +++ b/net/bridge/br_device.c	2004-06-30 10:30:31 -07:00
> @@ -121,6 +121,22 @@
>  	return -1;
>  }
>  
> +static int br_change_mtu(struct net_device *dev, int new_mtu)
> +{
> +	struct net_bridge *br = dev->priv;
> +	int max_mtu;
> +
> +	read_lock_bh(&br->lock);
> +	max_mtu = br_min_mtu(br);
> +	read_unlock_bh(&br->lock);
> +
> +	if (new_mtu < 68 || new_mtu > max_mtu)
> +		return -EINVAL;
> +
> +	dev->mtu = new_mtu;
> +	return 0;
> +}
> +
>  void br_dev_setup(struct net_device *dev)
>  {
>  	memset(dev->dev_addr, 0, ETH_ALEN);
> @@ -134,4 +150,5 @@
>  	dev->accept_fastpath = br_dev_accept_fastpath;
>  	dev->tx_queue_len = 0;
>  	dev->set_mac_address = NULL;
> +	dev->change_mtu = br_change_mtu;
>  }
> diff -Nru a/net/bridge/br_if.c b/net/bridge/br_if.c
> --- a/net/bridge/br_if.c	2004-06-30 10:30:31 -07:00
> +++ b/net/bridge/br_if.c	2004-06-30 10:30:31 -07:00
> @@ -220,6 +220,24 @@
>  	return 0;
>  }
>  
> +int br_min_mtu(struct net_bridge *br)
> +{
> +	struct net_bridge_port *p;
> +	int mtu;
> +
> +	p = br->port_list;
> +	if (!p)
> +		mtu = 1500;
> +	else {
> +		mtu = p->dev->mtu;
> +		while ((p = p->next) != NULL) {
> +			if (p->dev->mtu < mtu)
> +				mtu = p->dev->mtu;
> +		}
> +	}
> +	return mtu;
> +}
> +
>  int br_add_if(struct net_bridge *br, struct net_device *dev)
>  {
>  	struct net_bridge_port *p;
> @@ -250,6 +268,7 @@
>  	br_fdb_insert(br, p, dev->dev_addr, 1);
>  	if ((br->dev.flags & IFF_UP) && (dev->flags & IFF_UP))
>  		br_stp_enable_port(p);
> +	br->dev.mtu = br_min_mtu(dev->priv);
>  	write_unlock_bh(&br->lock);
>  
>  	return 0;
> diff -Nru a/net/bridge/br_private.h b/net/bridge/br_private.h
> --- a/net/bridge/br_private.h	2004-06-30 10:30:31 -07:00
> +++ b/net/bridge/br_private.h	2004-06-30 10:30:31 -07:00
> @@ -164,6 +164,7 @@
>  			    int num);
>  extern void br_get_port_ifindices(struct net_bridge *br,
>  			   int *ifindices);
> +extern int br_min_mtu(struct net_bridge *br);
>  
>  /* br_input.c */
>  extern void br_handle_frame(struct sk_buff *skb);
> 

Re: [Bridge] MTU Question

[Stephen Hemminger]

On Wed, 30 Jun 2004 14:03:28 -0400
"Eble, Dan" <DanE@aiinet.com> wrote:

> What if someone (or something) attempts to change the MTU on the port after
> it has been added to the bridge?  Perhaps SIOCSIFMTU should do something
> special for bridge ports, such as return -EBUSY, or maybe ask the bridge
> whether or not the new MTU is acceptable?  If a port's MTU changes, the
> bridge may have to adjust its own MTU to maintain consistency.  It could get
> hairy.

That can be handled by a notifier callback.

> I faced this question recently when trying to implement bridging over Cisco
> HDLC.  I finally decided not to try to maintain consistency of MTUs between
> two devices, but rather to drop *all* transmitted packets when the MTU of
> the lower-layer device was insufficient to handle the largest possible
> packets from the upper-layer device.  I decided that would be enough to
> prevent someone from wrongly thinking they had configured it properly, since
> not even a cheap a 64-byte ping would work.

Probably a good idea as well.

Also, we could check when adding interface to a bridge that the MTU is big
enough for the existing ports in the bridge.

Re: [Bridge] MTU Question

[Chris Saw]

forgot to CC the list on this one...

> * also, the mtu of the bridge itself will always appear as 1500, but
>   you could change that with ifconfig.  Thought about having bridge mtu
>   appear as min(ports mtu), but don't think that is in the current version

I tried changing the MTU > 1500 on the bridge and I get "SIOCSIFMTU: Invalid
argument" (I'm using 2.4.24 W/ latest ebtables) but if I understand you
correctly that dosen't really matter anyway.

> The bridge does not care about frame size at all. It should be able
> to handle large frames, provided:
> * all interfaces in the bridge can handle the same big mtu, there is
>   no way a bridge can fragment.

That's kinda what I was afraid of... If I understand you correctly, if I
have some 100Mbit NICs in the same bridge, then even if I change the MTU of
the 1Gig NICs the MTU of packets coming in/out of the bridge will be that of
the slowest NIC (1500bytes) correct?

The reason I even care is that I am getting horrible speeds from the gigabit
NICs. In the neighborhood of about 150-200Mbps from the gigabit NICs when
doing a netcat between systems using /dev/zero -> /dev/null... That's just
crappy... even from 33Mhz PCI I should be getting like around 40-50MB/sec or
more I would think with RAM -> RAM. They're good NICs too, both EEPRO1000
using NAPI.

I can't really break up the bridges by speed either because you can't
enslave the same NICs to different bridges and I only have 1 cable modem ;)

I may have to just live with it because the bridge is working beautifully
as-is, firewalling cable modem traffic like a champ and serving files
perfectly through SAMBA too. I just don't like to leave things alone :)

Thank you for your quick reply on the last message, hopefully my questions
aren't too dumb... :)

    -Chris



----- Original Message -----
From: <shemminger@osdl.org>
To: "Chris Shaw" <chriss@watertech.com>
Cc: <bridge@lists.osdl.org>
Sent: Tuesday, June 29, 2004 9:22 PM
Subject: Re: [Bridge] MTU Question


> > I have a bridge that has gigabit interfaces. The machine in question has
> > the
> > fun job of being a Bridge, Firewall and SMB server. Both of the Gigabit
> > interfaces are connected to workstations directly via Xover cable (well
> > MDI-X to be exact). My question is, if I enable jumbo frames on the
> > gigabit
> > interfaces will that make any difference in overall transfer rate of the
> > bridge? I was thinking it might because even though the NIC is enslaved,
> > it
> > IS the device that the workstations communicate to. But also it might
not
> > because when applications talk to the network they're using BR(n) at
> > 1500bytes.. can a bridging guru shed some light on this for me?
> >
> > -Thank you in advance
> > Chris
>
> The bridge does not care about frame size at all. It should be able
> to handle large frames, provided:
> * all interfaces in the bridge can handle the same big mtu, there is
>   no way a bridge can fragment.
> * also, the mtu of the bridge itself will always appear as 1500, but
>   you could change that with ifconfig.  Thought about having bridge mtu
>   appear as min(ports mtu), but don't think that is in the current
version.
>
>
> _______________________________________________
> Bridge mailing list
> Bridge@lists.osdl.org
> http://lists.osdl.org/mailman/listinfo/bridge


Chris Shaw
IS Manager
Water Tech Industries
Phone: (888)-254-8412
Fax: (503)-261-9118
E-Mail: chriss@watertech.com