Jumbo frame question...

Jumbo frame question...

[Robin Getz]

Should a gigabit card, configured as 100, be sending jumbo UDP frames?

My understanding, is no - this is a spec violation..


Settings for eth0:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Half
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x000000ff (255)
        Link detected: no

happly sends UDP packets over 1500 bytes in length.

Tested with TFTP, and 2.6.27 (as the tftp server).

If tftp requests block sizes that are 4096 bytes, the sent packet is a single 
4096 byte packet (not multiple MTU sized packets).

?

Thanks
-Robin

Re: Jumbo frame question...

[David Miller]

From: Robin Getz <rgetz@blackfin.uclinux.org>
Date: Fri, 24 Jul 2009 11:41:55 -0400

> Should a gigabit card, configured as 100, be sending jumbo UDP frames?
> 
> My understanding, is no - this is a spec violation..

There is nothing wrong with supporting jumbo frames
when the speed is lower than 1GB.

If you configure the MTU to be jumbo size, it should
be no surprise to you that this is what gets used.

Re: Jumbo frame question...

[Rick Jones]

David Miller wrote:
> From: Robin Getz <rgetz@blackfin.uclinux.org>
> Date: Fri, 24 Jul 2009 11:41:55 -0400
> 
>>Should a gigabit card, configured as 100, be sending jumbo UDP frames?
>>
>>My understanding, is no - this is a spec violation..

In so far as there is no de jure spec for Jumbo Frames, it is rather difficult 
to have a spec violation :).

> There is nothing wrong with supporting jumbo frames
> when the speed is lower than 1GB.
> 
> If you configure the MTU to be jumbo size, it should
> be no surprise to you that this is what gets used.

Not a case of too much rope?  Given that (IIRC) Jumbo Frame was not introduced 
in Ethernet NICs until Gigabit came along (eg Alteon), the chances a (legacy) 
100 Mbit/s network would have JF-capable NICs is epsilon.

rick jones

> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Jumbo frame question...

[Robin Getz]

On Fri 24 Jul 2009 12:39, Rick Jones pondered:
> David Miller wrote:
> > From: Robin Getz <rgetz@blackfin.uclinux.org>
> > Date: Fri, 24 Jul 2009 11:41:55 -0400
> > 
> >>Should a gigabit card, configured as 100, be sending jumbo UDP frames?
> >>
> >>My understanding, is no - this is a spec violation..
> 
> In so far as there is no de jure spec for Jumbo Frames, it is rather
> difficult to have a spec violation :).

The spec I was talking about was the MTU...

rgetz@pinky:~> /sbin/ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:11:11:B0:A5:D4
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::211:11ff:feb0:a5d4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:45978 errors:5 dropped:0 overruns:0 frame:0
          TX packets:44536 errors:0 dropped:0 overruns:0 carrier:0
          collisions:3193 txqueuelen:1000
          RX bytes:11583575 (11.0 Mb)  TX bytes:20025122 (19.0 Mb)
          Interrupt:16


My MTU is 1500, but when tftp requests a block size of over that - the host 
does not fragment it (like I thought it should).

> > There is nothing wrong with supporting jumbo frames
> > when the speed is lower than 1GB.

I would agree - if you had the MTU set up that big.

> > If you configure the MTU to be jumbo size, it should
> > be no surprise to you that this is what gets used.

Which it is not.

> Not a case of too much rope?  Given that (IIRC) Jumbo Frame was not
> introduced in Ethernet NICs until Gigabit came along (eg Alteon), the
> chances a (legacy) 100 Mbit/s network would have JF-capable NICs is epsilon.

Yeah - I think that this is the issue - my old hub (which is what I normally 
use for ethernet testing is only transferring it's MTU (1500 bytes), and 
dropping the rest...

Isn't there a MTU max size discovery that should be done somewhere before the 
host sends jumbo packets?

http://tools.ietf.org/html/rfc1191

And -- in the UDP/TFTP case - isn't the server responsible for determining 
this? (since it need to determine if fragmentation needs to happen or not?)

-Robin

Re: Jumbo frame question...

[Rick Jones]

Robin Getz wrote:
> On Fri 24 Jul 2009 12:39, Rick Jones pondered:
> 
>>David Miller wrote:
>>
>>>From: Robin Getz <rgetz@blackfin.uclinux.org>
>>>Date: Fri, 24 Jul 2009 11:41:55 -0400
>>>
>>>
>>>>Should a gigabit card, configured as 100, be sending jumbo UDP frames?
>>>>
>>>>My understanding, is no - this is a spec violation..
>>
>>In so far as there is no de jure spec for Jumbo Frames, it is rather
>>difficult to have a spec violation :).
> 
> 
> The spec I was talking about was the MTU...
> 
> rgetz@pinky:~> /sbin/ifconfig eth0
> eth0      Link encap:Ethernet  HWaddr 00:11:11:B0:A5:D4
>           inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
>           inet6 addr: fe80::211:11ff:feb0:a5d4/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:45978 errors:5 dropped:0 overruns:0 frame:0
>           TX packets:44536 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:3193 txqueuelen:1000
>           RX bytes:11583575 (11.0 Mb)  TX bytes:20025122 (19.0 Mb)
>           Interrupt:16
> 
> 
> My MTU is 1500, but when tftp requests a block size of over that - the host 
> does not fragment it (like I thought it should).

Well, you should have said that to begin with!-)  Saying "Jumbo Frames" makes 
people think you have enabled JumboFrames - ie increased the MTU to something 
like 9000 bytes.

>>Not a case of too much rope?  Given that (IIRC) Jumbo Frame was not
>>introduced in Ethernet NICs until Gigabit came along (eg Alteon), the
>>chances a (legacy) 100 Mbit/s network would have JF-capable NICs is epsilon.
> 
> 
> Yeah - I think that this is the issue - my old hub (which is what I normally 
> use for ethernet testing is only transferring it's MTU (1500 bytes), and 
> dropping the rest...
> 
> Isn't there a MTU max size discovery that should be done somewhere before the 
> host sends jumbo packets?
> 
> http://tools.ietf.org/html/rfc1191
> 
> And -- in the UDP/TFTP case - isn't the server responsible for determining 
> this? (since it need to determine if fragmentation needs to happen or not?)

PathMTU discovery is based on the receipt of ICMP messages saying in essence 
"this datagram was too big to forwared without fragmenting and the don't 
fragment bit was set, please send nothing larger than <foo> this way"

That only happens when crossing routers, so if this TFTP transfer is over the 
local LAN, there is no router to say so, leaving the choice of fragment size to 
the sending system.

Does this NIC offer UDP Fragmentation Offload but perhaps only in GbE mode, but 
the driver doesn't clear that bit when the speed is 100 Mbit, or perhaps 
something up the stack cached knowledge that changed on it?

rick jones

Re: Jumbo frame question...

[Eric Dumazet]

Robin Getz a écrit :
> On Fri 24 Jul 2009 12:39, Rick Jones pondered:
>> David Miller wrote:
>>> From: Robin Getz <rgetz@blackfin.uclinux.org>
>>> Date: Fri, 24 Jul 2009 11:41:55 -0400
>>>
>>>> Should a gigabit card, configured as 100, be sending jumbo UDP frames?
>>>>
>>>> My understanding, is no - this is a spec violation..
>> In so far as there is no de jure spec for Jumbo Frames, it is rather
>> difficult to have a spec violation :).
> 
> The spec I was talking about was the MTU...
> 
> rgetz@pinky:~> /sbin/ifconfig eth0
> eth0      Link encap:Ethernet  HWaddr 00:11:11:B0:A5:D4
>           inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
>           inet6 addr: fe80::211:11ff:feb0:a5d4/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:45978 errors:5 dropped:0 overruns:0 frame:0
>           TX packets:44536 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:3193 txqueuelen:1000
>           RX bytes:11583575 (11.0 Mb)  TX bytes:20025122 (19.0 Mb)
>           Interrupt:16
> 
> 
> My MTU is 1500, but when tftp requests a block size of over that - the host 
> does not fragment it (like I thought it should).

Which broken driver would do this me asking, and how can you be sure a jumbo frame was ever sent ?

I guess your tcpdump is fooled by gso settings... Did you tried
# ethtool -K eth0 gso off 

Re: Jumbo frame question...

[Robin Getz]

On Fri 24 Jul 2009 15:13, Eric Dumazet pondered:
> I guess your tcpdump is fooled by gso settings... Did you tried
> # ethtool -K eth0 gso off 

No -- that was the problem - my hardware trying to be too smart for my software...

Thanks for the help - sorry for the noise.

-Robin

Re: Jumbo frame question...

[Eric Dumazet]

Robin Getz a écrit :
> On Fri 24 Jul 2009 15:13, Eric Dumazet pondered:
>> I guess your tcpdump is fooled by gso settings... Did you tried
>> # ethtool -K eth0 gso off 
> 
> No -- that was the problem - my hardware trying to be too smart for my software...
> 
> Thanks for the help - sorry for the noise.

No problem, but I wonder why being 'too smart' was a problem in your case.

Re: Jumbo frame question...

[Lennart Sorensen]

On Fri, Jul 24, 2009 at 09:39:43AM -0700, Rick Jones wrote:
> In so far as there is no de jure spec for Jumbo Frames, it is rather 
> difficult to have a spec violation :).
>
> Not a case of too much rope?  Given that (IIRC) Jumbo Frame was not 
> introduced in Ethernet NICs until Gigabit came along (eg Alteon), the 
> chances a (legacy) 100 Mbit/s network would have JF-capable NICs is 
> epsilon.

Nothing would prevent you from using a 100Mbit fiber PHY on a gigabit
capable network port, so you could have a 100Mbit port that supported
jumbo frames.

-- 
Len Sorensen

Re: Jumbo frame question...

[Herbert Xu]

Robin Getz <rgetz@blackfin.uclinux.org> wrote:
> Should a gigabit card, configured as 100, be sending jumbo UDP frames?
> 
> My understanding, is no - this is a spec violation..
> 
> 
> Settings for eth0:
>        Supported ports: [ MII ]
>        Supported link modes:   10baseT/Half 10baseT/Full
>                                100baseT/Half 100baseT/Full
>                                1000baseT/Half 1000baseT/Full
>        Supports auto-negotiation: Yes
>        Advertised link modes:  10baseT/Half 10baseT/Full
>                                100baseT/Half 100baseT/Full
>                                1000baseT/Half 1000baseT/Full
>        Advertised auto-negotiation: Yes
>        Speed: 100Mb/s
>        Duplex: Half
>        Port: Twisted Pair
>        PHYAD: 1
>        Transceiver: internal
>        Auto-negotiation: on
>        Supports Wake-on: g
>        Wake-on: d
>        Current message level: 0x000000ff (255)
>        Link detected: no
> 
> happly sends UDP packets over 1500 bytes in length.
> 
> Tested with TFTP, and 2.6.27 (as the tftp server).

What's the kernel that you're running on the machine doing the
sending, 2.6.27? If it's the latest kernel then it may be related
to the UFO work that went in recently.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt