From mboxrd@z Thu Jan 1 00:00:00 1970 From: Declan Doherty Subject: =?utf-8?q?=5BPATCH=5D_doc=3A_link_bonding_related_upda?= =?utf-8?q?tes_to_programmers_guide?= Date: Mon, 1 Dec 2014 17:10:12 +0000 Message-ID: <1417453812-14599-1-git-send-email-declan.doherty@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable To: dev-VfR2kkLFssw@public.gmane.org Return-path: List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces-VfR2kkLFssw@public.gmane.org Sender: "dev" Adding details for link status interrupts and link status polling. Adding details for mode 4 / mode 5 Tidying up rst document to conform to 80 character line limit Adding diagrams to explain bonding modes Signed-off-by: Declan Doherty --- doc/guides/prog_guide/img/bond-mode-0.svg | 638 +++++++++++++++= ++ doc/guides/prog_guide/img/bond-mode-1.svg | 724 +++++++++++++++= ++++ doc/guides/prog_guide/img/bond-mode-2.svg | 702 +++++++++++++++= +++ doc/guides/prog_guide/img/bond-mode-3.svg | 702 +++++++++++++++= +++ doc/guides/prog_guide/img/bond-mode-4.svg | 784 +++++++++++++++= ++++++ doc/guides/prog_guide/img/bond-mode-5.svg | 642 +++++++++++++++= ++ doc/guides/prog_guide/img/bond-overview.svg | 121 ++++ .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 366 +++++++--- 8 files changed, 4579 insertions(+), 100 deletions(-) create mode 100644 doc/guides/prog_guide/img/bond-mode-0.svg create mode 100644 doc/guides/prog_guide/img/bond-mode-1.svg create mode 100644 doc/guides/prog_guide/img/bond-mode-2.svg create mode 100644 doc/guides/prog_guide/img/bond-mode-3.svg create mode 100644 doc/guides/prog_guide/img/bond-mode-4.svg create mode 100644 doc/guides/prog_guide/img/bond-mode-5.svg create mode 100644 doc/guides/prog_guide/img/bond-overview.svg diff --git a/doc/guides/prog_guide/img/bond-mode-0.svg b/doc/guides/prog_= guide/img/bond-mode-0.svg new file mode 100644 index 0000000..eff0edb --- /dev/null +++ b/doc/guides/prog_guide/img/bond-mode-0.svg @@ -0,0 +1,638 @@ + + + +image/svg+xmlPage-4Rectangle.7User ApplicationUser Application +Sheet.2Rectangle.38DPDKDPDK +Rectangle.16bonded ethdevbonded ethdev +Rectangle.11ethdev portethdev port +Rectangle.14ethdev portethdev port +Rectangle.15ethdev portethdev port +Simple Double Arrow.14Simple ArrowSimple Arrow.37Simple Arrow.38Simple Arrow.39Square.11411 +Square.11522 +Square.11633 +Square.11744 +Square.11855 +Square.12011 +Square.12122 +Square.12233 +Square.12344 +Square.12455 + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/bond-mode-1.svg b/doc/guides/prog_= guide/img/bond-mode-1.svg new file mode 100644 index 0000000..c177e85 --- /dev/null +++ b/doc/guides/prog_guide/img/bond-mode-1.svg @@ -0,0 +1,724 @@ + + + +image/svg+xml + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.25;filter:url(#filter_2);stroke:#5b9= bd5;stroke-opacity:0.25} + .st3 {fill:#4f87bb;stroke:#40709c;stroke-width:0.75} + .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st5 {fill:url(#grad0-11);stroke:#4f87bb;stroke-width:0.75} + .st6 {fill:#4f87bb;font-family:Calibri;font-size:0.833336em} + .st7 {fill:#759fcc;fill-opacity:0.25;filter:url(#filter_2);stroke:#759= fcc;stroke-opacity:0.25} + .st8 {fill:#668bb3;stroke:#547395;stroke-width:0.75} + .st9 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9= bd5;stroke-opacity:0.22} + .st10 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25} + .st11 {fill:#bdd0e9;fill-opacity:0.22;filter:url(#filter_2);stroke:#bd= d0e9;stroke-opacity:0.22} + .st12 {fill:#bdd0e9;stroke:#c7c8c8;stroke-width:0.25} + .st13 {fill:url(#grad0-40);stroke:#a6b6cd;stroke-width:0.75} + .st14 {fill:#70ad47;fill-opacity:0.25;filter:url(#filter_2);stroke:#70= ad47;stroke-opacity:0.25} + .st15 {fill:#61973d;stroke:#507e31;stroke-width:0.75} + .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;str= oke-linecap:square;stroke-miterlimit:3} + Page-4Rectangle.40User ApplicationUser Application +Sheet.40Rectangle.38DPDKDPDK +Rectangle.16bonded ethdevbonded ethdev +Rectangle.11ethdev portethdev port +Rectangle.14ethdev portethdev port +Rectangle.15ethdev portethdev port +Simple Double Arrow.47Simple Arrow.47Simple Arrow.49Square.10811 +Square.10922 +Square.11033 +Square.11111 +Square.11222 +Square.11333 + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/bond-mode-2.svg b/doc/guides/prog_= guide/img/bond-mode-2.svg new file mode 100644 index 0000000..3dbb598 --- /dev/null +++ b/doc/guides/prog_guide/img/bond-mode-2.svg @@ -0,0 +1,702 @@ + + + +image/svg+xmlPage-4Rectangle.151User ApplicationUser Application +Sheet.56Rectangle.38DPDKDPDK +Rectangle.16bonded ethdevbonded ethdev +Rectangle.11ethdev portethdev port +Rectangle.14ethdev portethdev port +Rectangle.15ethdev portethdev port +Simple Double Arrow.158Simple Arrow.159Simple Arrow.160Simple Arrow.161Simple Arrow.162Square.16311 +Square.16422 +Square.16533 +Square.16644 +Square.16755 +Square.16866 +Sheet.73Square.17222 +Square.17344 +Square.17566 +Sheet.77Square.16911 +Square.17033 +Square.17155 + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/bond-mode-3.svg b/doc/guides/prog_= guide/img/bond-mode-3.svg new file mode 100644 index 0000000..d2dbe3a --- /dev/null +++ b/doc/guides/prog_guide/img/bond-mode-3.svg @@ -0,0 +1,702 @@ + + + +image/svg+xmlPage-4Rectangle.74User ApplicationUser Application +Sheet.82Rectangle.38DPDKDPDK +Rectangle.16bonded ethdevbonded ethdev +Rectangle.11ethdev portethdev port +Rectangle.14ethdev portethdev port +Rectangle.15ethdev portethdev port +Simple Double Arrow.81Simple Arrow.82Simple Arrow.83Simple Arrow.84Simple Arrow.85Sheet.93Square.12511 +Square.12622 +Square.12733 +Sheet.97Square.12511 +Square.12622 +Square.12733 +Sheet.101Square.12511 +Square.12622 +Square.12733 +Sheet.105Square.12511 +Square.12622 +Square.12733 + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/bond-mode-4.svg b/doc/guides/prog_= guide/img/bond-mode-4.svg new file mode 100644 index 0000000..45749bd --- /dev/null +++ b/doc/guides/prog_guide/img/bond-mode-4.svg @@ -0,0 +1,784 @@ + + + +image/svg+xmlPage-4Rectangle.177User ApplicationUser Application +Sheet.110Rectangle.38DPDKDPDK +Rectangle.16bonded ethdevbonded ethdev +Rectangle.11ethdev portethdev port +Rectangle.14ethdev portethdev port +Rectangle.15ethdev portethdev port +Simple Double Arrow.184Simple Arrow.185Simple Arrow.186Simple Arrow.187Simple Arrow.188Square.18911 +Square.19022 +Square.19133 +Square.19244 +Square.19355 +Square.19466 +Square.17222 +Square.17344 +Square.19866 +Square.16911 +Square.17033 +Square.17155 +Square.203OO +Square.204OO +Square.205OO + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/bond-mode-5.svg b/doc/guides/prog_= guide/img/bond-mode-5.svg new file mode 100644 index 0000000..5ee82fc --- /dev/null +++ b/doc/guides/prog_guide/img/bond-mode-5.svg @@ -0,0 +1,642 @@ + + + +image/svg+xmlPage-4Rectangle.209User ApplicationUser Application +Sheet.137Rectangle.38DPDKDPDK +Rectangle.16bonded ethdevbonded ethdev +Rectangle.11ethdev portethdev port +Rectangle.14ethdev portethdev port +Rectangle.15ethdev portethdev port +Simple Double Arrow.216Simple Arrow.217Simple Arrow.218Simple Arrow.219Simple Arrow.220Sheet.148Rectangle50065006 +Rectangle.24250055005 +Rectangle.24300010001 +Rectangle.24400020002 +Rectangle.2461200312003 +Rectangle.24700010001 +Rectangle.24800020002 +Rectangle.24950065006 +Rectangle.25050055005 +Rectangle.2511200312003 + \ No newline at end of file diff --git a/doc/guides/prog_guide/img/bond-overview.svg b/doc/guides/pro= g_guide/img/bond-overview.svg new file mode 100644 index 0000000..489282a --- /dev/null +++ b/doc/guides/prog_guide/img/bond-overview.svg @@ -0,0 +1,121 @@ + + + + + + + + + + + + + + + + + + + + + + + + + Page-1 + + Rectangle.38 + DPDK + + DPDK + + Rectangle.8 + bonded ethdev + + + + + bonded ethdev + + Rectangle + User Application + + + + + User Application= + + Rectangle.5 + ethdev port + + + + + ethdev port + + Rectangle.6 + ethdev port + + + + + ethdev port + + Rectangle.7 + ethdev port + + + + + ethdev port + + Rectangle.9 + ethdev port + + + + + ethdev port + + Rectangle.10 + ethdev port + + + + + ethdev port + + Simple Double Arrow + + + + Simple Double Arrow.30 + + + + Simple Double Arrow.42 + + + + diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/d= oc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index badd891..c4e0f2c 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -35,21 +35,25 @@ In addition to Poll Mode Drivers (PMDs) for physical = and virtual hardware, Intel=C2=AE DPDK also includes a pure-software library that allows physical PMD's to be bonded together to create a single logical P= MD. =20 -|link_bonding| +|bond-overview| =20 -The Link Bonding PMD library(librte_pmd_bond) supports bonding of groups= of physical ports of the same speed (1GbE, 10GbE and 40GbE) and -duplex to provide similar the capabilities to that found in Linux bondin= g driver to allow the aggregation of multiple (slave) NICs -into a single logical interface between a server and a switch. -The new bonded PMD will then process these interfaces based on the mode = of operation specified to provide support for features -such as redundant links, fault tolerance and/or load balancing. +The Link Bonding PMD library(librte_pmd_bond) supports bonding of groups= of +``rte_eth_dev`` ports of the same speed and duplex to provide +similar the capabilities to that found in Linux bonding driver to allow = the +aggregation of multiple (slave) NICs into a single logical interface bet= ween a +server and a switch. The new bonded PMD will then process these interfac= es +based on the mode of operation specified to provide support for features= such +as redundant links, fault tolerance and/or load balancing. =20 -The librte_pmd_bond library exports a C API which provides an API for th= e creation of bonded devices -as well as the configuration and management of the bonded device and its= slave devices. +The librte_pmd_bond library exports a C API which provides an API for th= e +creation of bonded devices as well as the configuration and management o= f the +bonded device and its slave devices. =20 .. note:: =20 - The Link Bonding PMD Library is enabled by default in the build conf= iguration files, - the library can be disabled by setting CONFIG_RTE_LIBRTE_PMD_BOND=3D= n and recompiling the Intel=C2=AE DPDK. + The Link Bonding PMD Library is enabled by default in the build + configuration files, the library can be disabled by setting + ``CONFIG_RTE_LIBRTE_PMD_BOND=3Dn`` and recompiling the Intel=C2=AE D= PDK. =20 Link Bonding Modes Overview --------------------------- @@ -57,143 +61,255 @@ Link Bonding Modes Overview Currently the Link Bonding PMD library supports 4 modes of operation: =20 * **Round-Robin (Mode 0):** - This mode provides load balancing and fault tolerance by transmissio= n of packets - in sequential order from the first available slave device through th= e last. - Packets are bulk dequeued from devices then serviced in round-robin = manner. + +|bond-mode-0| + + This mode provides load balancing and fault tolerance by transmissio= n of + packets in sequential order from the first available slave device th= rough + the last. Packets are bulk dequeued from devices then serviced in a + round-robin manner. This mode does not guarantee in order reception = of + packets and down stream should be able to handle out of order packet= s. =20 * **Active Backup (Mode 1):** - In this mode only one slave in the bond is active at any time, a dif= ferent slave becomes active if, - and only if, the primary active slave fails, - thereby providing fault tolerance to slave failure. - The single logical bonded interface's MAC address is externally visi= ble on only one NIC (port) + +|bond-mode-1| + + In this mode only one slave in the bond is active at any time, a dif= ferent + slave becomes active if, and only if, the primary active slave fails= , + thereby providing fault tolerance to slave failure. The single logic= al + bonded interface's MAC address is externally visible on only one NIC= (port) to avoid confusing the network switch. =20 * **Balance XOR (Mode 2):** - This mode provides load balancing based on transmit packets based on= the selected XOR transmission policy and fault tolerance. - The default policy (layer2) uses a simple XOR calculation on the pac= ket source / destination MAC address to select the slave to transmit on. - Alternate transmission policies supported are layer 2+3, this uses t= he IP source and destination addresses in the calculation of the slave po= rt and - the final supported policy is layer 3+4, this uses IP source and des= tination addresses as well as the UDP source and destination port. + +|bond-mode-2| + + This mode provides transmit load balancing (based on the selected + transmission policy) and fault tolerance. The default policy (layer2= ) uses + a simple calculation based on the packet flow source and destination= MAC + addresses aswell as the number of active slaves available to the bon= ded + device to classify the packet to a specific slave to transmit on. Al= ternate + transmission policies supported are layer 2+3, this takes the IP sou= rce and + destination addresses into the calculation of the transmit slave por= t and + the final supported policy is layer 3+4, this uses IP source and + destination addresses as well as the TCP/UDP source and destination = port. + +.. note:: + The colouring differences of the packets are used to identify differ= ent flow + classification calculated by the selected transmit policy + =20 * **Broadcast (Mode 3):** - This mode provides fault tolerance by transmission of packets on all= slave ports. + +|bond-mode-3| + + This mode provides fault tolerance by transmission of packets on all= slave + ports. + +* **Link Aggregation 802.3AD (Mode 4):** + +|bond-mode-4| + + This mode provides dynamic link aggregation according to the 802.3ad + specification. It negotiates and monitors aggregation groups that sh= are the + same speed and duplex settings using the selected balance transmit p= olicy + for balancing outgoing traffic. + + DPDK implementation of this mode provide some additional requirement= s of + the application. + + 1. It needs to call ``rte_eth_tx_burst`` and ``rte_eth_rx_burst`` wi= th + intervals period of less than 100ms. + + 2. Calls to ``rte_eth_tx_burst`` must have a buffer size of at least= 2xN, + where N is the number of slaves. This is a space required for LAC= P + frames. Additionally LACP packets are included in the statistics,= but + they are not returned to the application. + +* **Transmit Load Balancing (Mode 5):** + +|bond-mode-5| + + This mode provides an adaptive transmit load balancing. It dynamical= ly + changes the transmitting slave, according to the computed load. Stat= istics + are collected in 100ms intervals and scheduled every 10ms. + =20 Implementation Details ---------------------- =20 -The librte_pmd_bond onded device are compatible with the Ethernet device= API exported by the Ethernet PMDs described in the *Intel=C2=AE DPDK API= Reference*. +The librte_pmd_bond bonded device are compatible with the Ethernet devic= e API +exported by the Ethernet PMDs described in the *Intel=C2=AE DPDK API Ref= erence*. =20 -The Link Bonding Library supports the creation of bonded devices at appl= ication startup time during EAL initialization using the ---vdev option as well as programmatically via the C API rte_eth_bond_cre= ate function. +The Link Bonding Library supports the creation of bonded devices at appl= ication +startup time during EAL initialization using the ``--vdev`` option as we= ll as +programmatically via the C API ``rte_eth_bond_create`` function. =20 Bonded devices support the dynamical addition and removal of slave devic= es using -the rte_eth_bond_slave_add / rte_eth_bond_slave_remove APIs. +the ``rte_eth_bond_slave_add`` / ``rte_eth_bond_slave_remove`` APIs. =20 After a slave device is added to a bonded device slave is stopped using -rte_eth_dev_stop and the slave reconfigured using rte_eth_dev_configure = the RX and TX queues are also reconfigured -using rte_eth_tx_queue_setup / rte_eth_rx_queue_setup with the parameter= s use to configure the bonding device. +``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure= `` +the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setu= p`` / +``rte_eth_rx_queue_setup`` with the parameters use to configure the bond= ing +device. + +Link Status Change Interrupts / Polling +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Link bonding devices support the registration of a link status change ca= llback, +using the ``rte_eth_dev_callback_register`` API, this will be called whe= n the +status of the bonding device changes. For example in the case of a bondi= ng +device which has 3 slaves, the link status will change to up when one sl= ave +becomes active or change to down when all slaves become inactive. There = is no +callback notification when a single slave changes state and the previous +conditions are not met. If a user wishes to monitor individual slaves th= en they +must register callbacks with that slave directly. + +The link bonding library also supports devices which do not implement li= nk +status change interrupts, this is achieve by polling the devices link st= atus at +a defined period which is set using the ``rte_eth_bond_link_monitoring_s= et`` +API, the default polling interval is 10ms. When a device is added as a s= lave to +a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` fla= g +whether the device supports interrupts or whether the link status should= be +monitored by polling it. =20 Requirements / Limitations ~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -The current implementation only supports physical devices of the same ty= pe, speed and duplex to be added as slaves. -The bonded device inherits these values from the first active slave adde= d to the bonded device -and then all further slaves added to the bonded device must match these = parameters. +The current implementation only supports devices that support the same s= peed +and duplex to be added as a slaves to the same bonded device. The bonded= device +inherits these attributes from the first active slave added to the bonde= d +device and then all further slaves added to the bonded device must suppo= rt +these parameters. =20 -A bonding device must have a minimum of one slave before the bonding dev= ice itself can be started. +A bonding device must have a minimum of one slave before the bonding dev= ice +itself can be started. =20 -Like all other PMD, all functions exported by a PMD are lock-free functi= ons that are assumed -not to be invoked in parallel on different logical cores to work on the = same target object. +Like all other PMD, all functions exported by a PMD are lock-free functi= ons +that are assumed not to be invoked in parallel on different logical core= s to +work on the same target object. =20 -It should also be noted that the PMD receive function should not be invo= ked directly on a slave devices after they have -been to a bonded device since packets read directly from the slave devic= e will no longer be available to the bonded device to read. +It should also be noted that the PMD receive function should not be invo= ked +directly on a slave devices after they have been to a bonded device sinc= e +packets read directly from the slave device will no longer be available = to the +bonded device to read. =20 Configuration ~~~~~~~~~~~~~ =20 -Link bonding devices are created using the rte_eth_bond_create API +Link bonding devices are created using the ``rte_eth_bond_create`` API which requires a unique device name, the bonding mode, and the socket Id to allocate the bonding device's resources on. -The other configurable parameters for a bonded device are its slave devi= ces, its primary slave, -a user defined MAC address and transmission policy to use if the device = is balance XOR mode. +The other configurable parameters for a bonded device are its slave devi= ces, +its primary slave, a user defined MAC address and transmission policy to= use if +the device is in balance XOR mode. =20 Slave Devices ^^^^^^^^^^^^^ =20 -Bonding devices support up to a maximum of RTE_MAX_ETHPORTS slave device= s of the same speed and duplex. -Ethernet devices can be added as a slave to a maximum of one bonded devi= ce. -Slave devices are reconfigured with the configuration of the bonded devi= ce on being added to a bonded device. +Bonding devices support up to a maximum of ``RTE_MAX_ETHPORTS`` slave de= vices +of the same speed and duplex. Ethernet devices can be added as a slave t= o a +maximum of one bonded device. Slave devices are reconfigured with the +configuration of the bonded device on being added to a bonded device. =20 -The bonded also guarantees to return the MAC address of the slave device= to its original value of removal of a slave from it. +The bonded also guarantees to return the MAC address of the slave device= to its +original value of removal of a slave from it. =20 Primary Slave ^^^^^^^^^^^^^ =20 -The primary slave is used to define the default port to use when a bonde= d device is in active backup mode. -A different port will only be used if, and only if, the current primary = port goes down. -If the user does not specify a primary port it will default to being the= first port added to the bonded device. +The primary slave is used to define the default port to use when a bonde= d +device is in active backup mode. A different port will only be used if, = and +only if, the current primary port goes down. If the user does not specif= y a +primary port it will default to being the first port added to the bonded= device. =20 MAC Address ^^^^^^^^^^^ =20 -The bonded device can be configured with a user specified MAC address, -this address will be inherited by the some/all slave devices depending o= n the operating mode. -If the device is in active backup mode then only the primary device will= have the user specified MAC, -all other slaves will retain their original MAC address. -In mode 0, 2, and 3 all slaves devices are configure with the bonded dev= ices MAC address. +The bonded device can be configured with a user specified MAC address, t= his +address will be inherited by the some/all slave devices depending on the +operating mode. If the device is in active backup mode then only the pri= mary +device will have the user specified MAC, all other slaves will retain th= eir +original MAC address. In mode 0, 2, 3, 4 all slaves devices are configur= e with +the bonded devices MAC address. =20 -If a user defined MAC address is not defined then the bonded device will= default to using the primary slaves MAC address. +If a user defined MAC address is not defined then the bonded device will +default to using the primary slaves MAC address. =20 Balance XOR Transmit Policies ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ =20 -There are 3 supported transmission policies for bonded device running in= Balance XOR mode. Layer 2, Layer 2+3, Layer 3+4. +There are 3 supported transmission policies for bonded device running in +Balance XOR mode. Layer 2, Layer 2+3, Layer 3+4. =20 -* **Layer 2:** Ethernet MAC address based balancing is the default t= ransmission policy for Balance XOR bonding mode. - It uses a simple XOR calculation on the source MAC address and desti= nation MAC address of the packet and - then calculate the modulus of this value to calculate the slave devi= ce to transmit the packet on. +* **Layer 2:** Ethernet MAC address based balancing is the default + transmission policy for Balance XOR bonding mode. It uses a simple X= OR + calculation on the source MAC address and destination MAC address of= the + packet and then calculate the modulus of this value to calculate the= slave + device to transmit the packet on. =20 -* **Layer 2 + 3:** Ethernet MAC address & IP Address based balancing u= ses a combination of source/destination MAC addresses and - the source/destination IP addresses of the data packet to decide whi= ch slave port the packet will be transmitted on. +* **Layer 2 + 3:** Ethernet MAC address & IP Address based balancing u= ses a + combination of source/destination MAC addresses and the source/desti= nation + IP addresses of the data packet to decide which slave port the packe= t will + be transmitted on. =20 -* **Layer 3 + 4:** IP Address & UDP Port based balancing uses a comb= ination of source/destination IP Address and - the source/destination UDP ports of the packet of the data packet to= decide which slave port the packet will be transmitted on. +* **Layer 3 + 4:** IP Address & UDP Port based balancing uses a comb= ination + of source/destination IP Address and the source/destination UDP port= s of + the packet of the data packet to decide which slave port the packet = will be + transmitted on. =20 -All these policies support 802.1Q VLAN Ethernet packets, as well as IPv4= , IPv6 and UDP protocols for load balancing. +All these policies support 802.1Q VLAN Ethernet packets, as well as IPv4= , IPv6 +and UDP protocols for load balancing. =20 Using Link Bonding Devices -------------------------- =20 -The librte_pmd_bond library support two modes of device creation, the li= braries export full C API or -using the EAL command line to statically configure link bonding devices = at application startup. -Using the EAL option it is possible to use link bonding functionality tr= ansparently without specific knowledge of the libraries API, -this can be used, for example, to add bonding functionality, such as act= ive backup, -to an existing application which has no knowledge of the link bonding C = API. +The librte_pmd_bond library support two modes of device creation, the li= braries +export full C API or using the EAL command line to statically configure = link +bonding devices at application startup. Using the EAL option it is possi= ble to +use link bonding functionality transparently without specific knowledge = of the +libraries API, this can be used, for example, to add bonding functionali= ty, +such as active backup, to an existing application which has no knowledge= of +the link bonding C API. =20 Using the Poll Mode Driver from an Application ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -Using the librte_pmd_bond libraries API it is possible to dynamicall cre= ate and manage link bonding device from within any application. -Link bonding device are created using the rte_eth_bond_create API which = requires a unqiue device name, -the link bonding mode to initial the device in and finally the socket Id= which to allocate the devices resources onto. -After successful creation of a bonding device it must be configured usin= g the generic Ethernet device configure API rte_eth_dev_configure -and then the RX and TX queues which will be used must be setup using rte= _eth_tx_queue_setup / rte_eth_rx_queue_setup. - -Slave devices can be dynamically added and removed from a link bonding d= evice using the rte_eth_bond_slave_add / rte_eth_bond_slave_remove -APIs but at least one slave device must be added to the link bonding dev= ice before it can be started using rte_eth_dev_start. - -The link status of a bonded device is dictated by that of its slaves, if= all slave device link status are down or -if all slaves are removed from the link bonding device then the link sta= tus of the bonding device will go down. - -It is also possible to configure / query the configuration of the contro= l parameters of a bonded device using the provided APIs -rte_eth_bond_mode_set/get, rte_eth_bond_primary_set/get, rte_eth_bond_ma= c_set/reset and rte_eth_bond_xmit_policy_set/get. +Using the librte_pmd_bond libraries API it is possible to dynamically cr= eate +and manage link bonding device from within any application. Link bonding +device are created using the ``rte_eth_bond_create`` API which requires = a +unique device name, the link bonding mode to initial the device in and f= inally +the socket Id which to allocate the devices resources onto. After succes= sful +creation of a bonding device it must be configured using the generic Eth= ernet +device configure API ``rte_eth_dev_configure`` and then the RX and TX qu= eues +which will be used must be setup using ``rte_eth_tx_queue_setup`` / +``rte_eth_rx_queue_setup``. + +Slave devices can be dynamically added and removed from a link bonding d= evice +using the ``rte_eth_bond_slave_add`` / ``rte_eth_bond_slave_remove`` +APIs but at least one slave device must be added to the link bonding dev= ice +before it can be started using ``rte_eth_dev_start``. + +The link status of a bonded device is dictated by that of its slaves, if= all +slave device link status are down or if all slaves are removed from the = link +bonding device then the link status of the bonding device will go down. + +It is also possible to configure / query the configuration of the contro= l +parameters of a bonded device using the provided APIs +``rte_eth_bond_mode_set/ get``, ``rte_eth_bond_primary_set/get``, +``rte_eth_bond_mac_set/reset`` and ``rte_eth_bond_xmit_policy_set/get``. =20 Using Link Bonding Devices from the EAL Command Line ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 -Link bonding devices can be created at application startup time using th= e --vdev EAL command line option. -The device name must start with the eth_bond prefix followed by numbers = or letters. The name must be unique for each device. -Each device can have multiple options arranged in a comma separated list= . -Multiple devices definitions can be arranged by calling the --vdev optio= n multiple times. +Link bonding devices can be created at application startup time using th= e +``--vdev`` EAL command line option. The device name must start with the +eth_bond prefix followed by numbers or letters. The name must be unique = for +each device. Each device can have multiple options arranged in a comma +separated list. Multiple devices definitions can be arranged by calling = the +``--vdev`` option multiple times. + Device names and bonding options must be separated by commas as shown be= low: =20 .. code-block:: console @@ -203,7 +319,8 @@ Device names and bonding options must be separated by= commas as shown below: Link Bonding EAL Options ^^^^^^^^^^^^^^^^^^^^^^^^ =20 -There are multiple ways of definitions that can be assessed and combined= as long as the following two rules are respected: +There are multiple ways of definitions that can be assessed and combined= as +long as the following two rules are respected: =20 * A unique device name, in the format of eth_bondX is provided, where X can be any combination of numbers and/or letters, @@ -216,37 +333,79 @@ There are multiple ways of definitions that can be = assessed and combined as long The different options are: =20 * mode: Integer value defining the bonding mode of the device. - Currently supports modes 0,1,2,3 (round-robin, active backup, balanc= e, and broadcast). + Currently supports modes 0,1,2,3,4,5 (round-robin, active backup, ba= lance, + broadcast, link aggregation, transmit load balancing). + +.. code-block:: console =20 mode=3D2 =20 -* slave: Defines the PMD device which will be added as slave to the bo= nded device. - This option can be selected multiple time, for each device to be add= ed as a slave. - Physical devices should be specified using their PCI address, in the= format domain:bus:devid.function +* slave: Defines the PMD device which will be added as slave to the bo= nded + device. This option can be selected multiple time, for each device t= o be + added as a slave. Physical devices should be specified using their P= CI + address, in the format domain:bus:devid.function + +.. code-block:: console =20 slave=3D0000:0a:00.0,slave=3D0000:0a:00.1 =20 * primary: Optional parameter which defines the primary slave port, - is used in active backup mode to select the primary slave for data T= X/RX if it is available. - The primary port also is used to select the MAC address to use when = it is not defined by the user. - This defaults to the first slave added to the device if it is specif= ied. - The primary device must be a slave of the bonded device. + is used in active backup mode to select the primary slave for data T= X/RX if + it is available. The primary port also is used to select the MAC add= ress to + use when it is not defined by the user. This defaults to the first s= lave + added to the device if it is specified. The primary device must be a= slave + of the bonded device. + +.. code-block:: console =20 primary=3D0000:0a:00.0 =20 -* socket_id: Optional parameter used to select which socket on a NUMA = device the bonded devices resources will be allocated on. +* socket_id: Optional parameter used to select which socket on a NUMA = device + the bonded devices resources will be allocated on. + +.. code-block:: console =20 socket_id=3D0 =20 -* mac: Optional parameter to select a MAC address for link bonding dev= ice, this overrides the value of the primary slave device. +* mac: Optional parameter to select a MAC address for link bonding dev= ice, + this overrides the value of the primary slave device. + +.. code-block:: console =20 mac=3D00:1e:67:1d:fd:1d =20 -* xmit_policy: Optional parameter which defines the transmission polic= y when the bonded device is in balance mode. - If not user specified this defaults to l2 (layer 2) forwarding, - the other transmission policies available are l23 (layer 2+3) and l3= 4 (layer 3+4) +* xmit_policy: Optional parameter which defines the transmission polic= y when + the bonded device is in balance mode. If not user specified this de= faults + to l2 (layer 2) forwarding, the other transmission policies availabl= e are + l23 (layer 2+3) and l34 (layer 3+4) + +.. code-block:: console + + xmit_policy=3Dl23 =20 - xmit_policy=3Dl2 +* lsc_poll_period_ms: Optional parameter which defines the polling int= erval + in milli-seconds at which devices which don't support lsc interrupts= are + checked for a change in the devices link status + +.. code-block:: console + + lsc_poll_period_ms=3D100 + +* up_delay: Optional parameter which adds a delay in milli-seconds to = the + propagation of a devices link status changing to up, by default this + parameter is zero. + +.. code-block:: console + + up_delay=3D10 + +* down_delay: Optional parameter which adds a delay in milli-seconds t= o the + propagation of a devices link status changing to down, by default th= is + parameter is zero. + +.. code-block:: console + + down_delay=3D50 =20 Examples of Usage ^^^^^^^^^^^^^^^^^ @@ -275,4 +434,11 @@ Create a bonded device in balance mode with two slav= es specified by their PCI ad =20 $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=3D2, sl= ave=3D0000:00a:00.01,slave=3D0000:004:00.00,xmit_policy=3Dl34' -- --port-= topology=3Dchained =20 -.. |link_bonding| image:: img/link_bonding.png +.. |bond-overview| image:: img/bond-overview.svg + +.. |bond-mode-0| image:: img/bond-mode-0.svg +.. |bond-mode-1| image:: img/bond-mode-1.svg +.. |bond-mode-2| image:: img/bond-mode-2.svg +.. |bond-mode-3| image:: img/bond-mode-3.svg +.. |bond-mode-4| image:: img/bond-mode-4.svg +.. |bond-mode-5| image:: img/bond-mode-5.svg --=20 1.7.12.2