[RFC] Ethernet drivers to add padding on egress

[RFC] Ethernet drivers to add padding on egress

[Morten Brørup]

Hi networking driver maintainers,

I suggest that the TX functions of Ethernet interface drivers accept packets with less than 60 byte payload, and transmit them on the medium as valid Ethernet frames, i.e. by padding the packets up to the minimum Ethernet packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.

This feature makes it easier for application developers who are using DPDK as the lower layer in an IP stack, where lots of packets have less than 60 bytes Ethernet payload, e.g. TCP SYN and TCP ACK packets.

This feature also makes it easier for application developers who are using DPDK library functions that split, merge or otherwise transform packets into packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation and various tunnel encapsulation/decapsulation functions.

Currently (without this feature), it is required by the application to check if packets originating from the IP stack or having passed through a split/merge/transform function are about to egress on an Ethernet interface, and in that case, if some of the packets are less than 60 bytes (excl. Ethernet FCS), add padding before passing them on to the driver's TX function.

E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers) and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must be padded before it is transmitted on an Ethernet interface.


In my opinion, it should be a requirement that the Ethernet interface drivers ensure correct padding when egressing the packet on the medium. Alternatively, it can be an optional feature, which could be exposed as an TX Capabilities flag of the driver.

What do you think?


I do not suggest any changes regarding ingress - runts (undersize Ethernet packets) received from the medium are invalid, and should be discarded and counted as errors.


My RFC was triggered by this discussion:
https://mails.dpdk.org/archives/dev/2018-November/119135.html

PS: I acknowledge Keith's comment that I am pushing for a feature with wide ranging consequences - modifying all PMDs and possibly costing some performance - based on one assumption.


Med venlig hilsen / kind regards

Morten Brørup
CTO


SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark

Office      +45 70 20 00 93
Direct      +45 89 93 50 22
Mobile     +45 25 40 82 12

mb@smartsharesystems.com
www.smartsharesystems.com

Re: [RFC] Ethernet drivers to add padding on egress

[Shahaf Shuler]

Thursday, November 15, 2018 6:57 PM, Morten Brørup:
> Subject: [RFC] Ethernet drivers to add padding on egress
> 
> Hi networking driver maintainers,
> 
> I suggest that the TX functions of Ethernet interface drivers accept packets
> with less than 60 byte payload, and transmit them on the medium as valid
> Ethernet frames, i.e. by padding the packets up to the minimum Ethernet
> packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.
> 
> This feature makes it easier for application developers who are using DPDK as
> the lower layer in an IP stack, where lots of packets have less than 60 bytes
> Ethernet payload, e.g. TCP SYN and TCP ACK packets.
> 
> This feature also makes it easier for application developers who are using
> DPDK library functions that split, merge or otherwise transform packets into
> packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation
> and various tunnel encapsulation/decapsulation functions.
> 
> Currently (without this feature), it is required by the application to check if
> packets originating from the IP stack or having passed through a
> split/merge/transform function are about to egress on an Ethernet interface,
> and in that case, if some of the packets are less than 60 bytes (excl. Ethernet
> FCS), add padding before passing them on to the driver's TX function.
> 
> E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte
> TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two
> packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers)
> and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must
> be padded before it is transmitted on an Ethernet interface.
> 
> 
> In my opinion, it should be a requirement that the Ethernet interface drivers
> ensure correct padding when egressing the packet on the medium.
> Alternatively, it can be an optional feature, which could be exposed as an TX
> Capabilities flag of the driver.
> 
> What do you think?

I think at the first stage it should be a Tx offload capability - the ability to pad (maybe in HW) the packets and avoid the cost of padding in SW.
PMD vendors who wants to make an easier life for their customers can implement it in SW, however the gain here is only with simplicity of code for application. Performance wise it wouldn't matter. 

When the majority/all PMDs will have this feature we can discuss on making it a standard for each PMD (like the CRC strip we have today). 

> 
> 
> I do not suggest any changes regarding ingress - runts (undersize Ethernet
> packets) received from the medium are invalid, and should be discarded and
> counted as errors.
> 
> 
> My RFC was triggered by this discussion:
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma
> ils.dpdk.org%2Farchives%2Fdev%2F2018-
> November%2F119135.html&data=02%7C01%7Cshahafs%40mellanox.co
> m%7C395ae1ac33674144116508d64b1b59f0%7Ca652971c7d2e4d9ba6a4d1492
> 56f461b%7C0%7C0%7C636778978183517821&sdata=xWcA5bhbhhqhZkg
> CYuwUP4qjLVlCLMjZrJaitQhAZ8c%3D&reserved=0
> 
> PS: I acknowledge Keith's comment that I am pushing for a feature with wide
> ranging consequences - modifying all PMDs and possibly costing some
> performance - based on one assumption.
> 
> 
> Med venlig hilsen / kind regards
> 
> Morten Brørup
> CTO
> 
> 
> SmartShare Systems A/S
> Tonsbakken 16-18
> DK-2740 Skovlunde
> Denmark
> 
> Office      +45 70 20 00 93
> Direct      +45 89 93 50 22
> Mobile     +45 25 40 82 12
> 
> mb@smartsharesystems.com
> https://emea01.safelinks.protection.outlook.com/?url=www.smartsharesyst
> ems.com&data=02%7C01%7Cshahafs%40mellanox.com%7C395ae1ac33
> 674144116508d64b1b59f0%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C
> 0%7C636778978183517821&sdata=gbdMkjhODS74P0Mtm%2FNEyNQ5h
> gFZI0xRgvNdA5M2ydM%3D&reserved=0

Re: [RFC] Ethernet drivers to add padding on egress

[Stephen Hemminger]

On Thu, 15 Nov 2018 17:56:48 +0100
Morten Brørup <mb@smartsharesystems.com> wrote:

> Hi networking driver maintainers,
> 
> I suggest that the TX functions of Ethernet interface drivers accept packets with less than 60 byte payload, and transmit them on the medium as valid Ethernet frames, i.e. by padding the packets up to the minimum Ethernet packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.
> 
> This feature makes it easier for application developers who are using DPDK as the lower layer in an IP stack, where lots of packets have less than 60 bytes Ethernet payload, e.g. TCP SYN and TCP ACK packets.
> 
> This feature also makes it easier for application developers who are using DPDK library functions that split, merge or otherwise transform packets into packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation and various tunnel encapsulation/decapsulation functions.
> 
> Currently (without this feature), it is required by the application to check if packets originating from the IP stack or having passed through a split/merge/transform function are about to egress on an Ethernet interface, and in that case, if some of the packets are less than 60 bytes (excl. Ethernet FCS), add padding before passing them on to the driver's TX function.
> 
> E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers) and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must be padded before it is transmitted on an Ethernet interface.
> 
> 
> In my opinion, it should be a requirement that the Ethernet interface drivers ensure correct padding when egressing the packet on the medium. Alternatively, it can be an optional feature, which could be exposed as an TX Capabilities flag of the driver.
> 
> What do you think?
> 
> 
> I do not suggest any changes regarding ingress - runts (undersize Ethernet packets) received from the medium are invalid, and should be discarded and counted as errors.
> 

Devices drivers should work like Linux and BSD!
They must pad packets to meet any restrictions by the underlying technology.

Please don't add another capability flag or other way for applications to see this.
DPDK already has way to many hardware capability flags.
It should just happen.

Re: [RFC] Ethernet drivers to add padding on egress

[Shahaf Shuler]

Monday, November 19, 2018 6:10 PM, Stephen Hemminger:
> Subject: Re: [dpdk-dev] [RFC] Ethernet drivers to add padding on egress
> 
> On Thu, 15 Nov 2018 17:56:48 +0100
> Morten Brørup <mb@smartsharesystems.com> wrote:
> 
> > Hi networking driver maintainers,
> >
> > I suggest that the TX functions of Ethernet interface drivers accept packets
> with less than 60 byte payload, and transmit them on the medium as valid
> Ethernet frames, i.e. by padding the packets up to the minimum Ethernet
> packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.
> >
> > This feature makes it easier for application developers who are using DPDK
> as the lower layer in an IP stack, where lots of packets have less than 60
> bytes Ethernet payload, e.g. TCP SYN and TCP ACK packets.
> >
> > This feature also makes it easier for application developers who are using
> DPDK library functions that split, merge or otherwise transform packets into
> packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation
> and various tunnel encapsulation/decapsulation functions.
> >
> > Currently (without this feature), it is required by the application to check if
> packets originating from the IP stack or having passed through a
> split/merge/transform function are about to egress on an Ethernet interface,
> and in that case, if some of the packets are less than 60 bytes (excl. Ethernet
> FCS), add padding before passing them on to the driver's TX function.
> >
> > E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte
> TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two
> packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers)
> and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must
> be padded before it is transmitted on an Ethernet interface.
> >
> >
> > In my opinion, it should be a requirement that the Ethernet interface
> drivers ensure correct padding when egressing the packet on the medium.
> Alternatively, it can be an optional feature, which could be exposed as an TX
> Capabilities flag of the driver.
> >
> > What do you think?
> >
> >
> > I do not suggest any changes regarding ingress - runts (undersize Ethernet
> packets) received from the medium are invalid, and should be discarded and
> counted as errors.
> >
> 
> Devices drivers should work like Linux and BSD!
> They must pad packets to meet any restrictions by the underlying
> technology.
> 
> Please don't add another capability flag or other way for applications to see
> this.
> DPDK already has way to many hardware capability flags.
> It should just happen.

I would like to change my previous comment, I fully agree w/ Stephan here. 

> 

Re: [RFC] Ethernet drivers to add padding on egress

[Stephen Hemminger]

On Mon, 19 Nov 2018 08:02:02 +0000
Shahaf Shuler <shahafs@mellanox.com> wrote:

> Thursday, November 15, 2018 6:57 PM, Morten Brørup:
> > Subject: [RFC] Ethernet drivers to add padding on egress
> > 
> > Hi networking driver maintainers,
> > 
> > I suggest that the TX functions of Ethernet interface drivers accept packets
> > with less than 60 byte payload, and transmit them on the medium as valid
> > Ethernet frames, i.e. by padding the packets up to the minimum Ethernet
> > packet size of 64 bytes incl. Ethernet FCS, instead of discarding them.
> > 
> > This feature makes it easier for application developers who are using DPDK as
> > the lower layer in an IP stack, where lots of packets have less than 60 bytes
> > Ethernet payload, e.g. TCP SYN and TCP ACK packets.
> > 
> > This feature also makes it easier for application developers who are using
> > DPDK library functions that split, merge or otherwise transform packets into
> > packets of other sizes, e.g. Generic Segmentation Offload, IP Fragmentation
> > and various tunnel encapsulation/decapsulation functions.
> > 
> > Currently (without this feature), it is required by the application to check if
> > packets originating from the IP stack or having passed through a
> > split/merge/transform function are about to egress on an Ethernet interface,
> > and in that case, if some of the packets are less than 60 bytes (excl. Ethernet
> > FCS), add padding before passing them on to the driver's TX function.
> > 
> > E.g. when using Generic Segmentation Offload, a packet carrying 1461 byte
> > TCP payload (excl. 54 bytes Ethernet+IP+TCP headers) will be split into two
> > packets of respectively 1514 byte (incl. 54 bytes Ethernet+IP+TCP headers)
> > and 55 bytes (incl. 54 bytes Ethernet+IP+TCP headers), and the latter must
> > be padded before it is transmitted on an Ethernet interface.
> > 
> > 
> > In my opinion, it should be a requirement that the Ethernet interface drivers
> > ensure correct padding when egressing the packet on the medium.
> > Alternatively, it can be an optional feature, which could be exposed as an TX
> > Capabilities flag of the driver.
> > 
> > What do you think?  
> 
> I think at the first stage it should be a Tx offload capability - the ability to pad (maybe in HW) the packets and avoid the cost of padding in SW.
> PMD vendors who wants to make an easier life for their customers can implement it in SW, however the gain here is only with simplicity of code for application. Performance wise it wouldn't matter. 
> 
> When the majority/all PMDs will have this feature we can discuss on making it a standard for each PMD (like the CRC strip we have today).

Yet another tx offload flag may look good as a vendor but doesn't add anything useful and hurts useablity.
Every driver should take any size Ethernet packet and pad in hardware (or software) based on what it knows the NIC hardware can do.
For virtual devices where there is no minimum length is required, then nothing needs to be done.

Packets < Ether header are obvious errors and should increment tx_output errors.