bonding xmit_policy

bonding xmit_policy

[Simon Chen]

Hi folks,

It looks like that there are three entries in xmit_policy (hashing for
deciding egress phy int) for bonded interfaces:

const struct bond_parm_tbl xmit_hashtype_tbl[] = {
{       "layer2",               BOND_XMIT_POLICY_LAYER2},
{       "layer3+4",             BOND_XMIT_POLICY_LAYER34},
{       "layer2+3",             BOND_XMIT_POLICY_LAYER23},
{       NULL,                   -1},
};


We can set the xmit_policy either at module initiation, or later via
/sys. However, this xmit_policy isn't really read anywhere. Are
different policies really implemented?

Thanks.
-Simon

Re: bonding xmit_policy

[Nicolas de Pesloüan]

Le 13/11/2011 03:55, Simon Chen a écrit :
> Hi folks,
>
> It looks like that there are three entries in xmit_policy (hashing for
> deciding egress phy int) for bonded interfaces:
>
> const struct bond_parm_tbl xmit_hashtype_tbl[] = {
> {       "layer2",               BOND_XMIT_POLICY_LAYER2},
> {       "layer3+4",             BOND_XMIT_POLICY_LAYER34},
> {       "layer2+3",             BOND_XMIT_POLICY_LAYER23},
> {       NULL,                   -1},
> };
>
>
> We can set the xmit_policy either at module initiation, or later via
> /sys. However, this xmit_policy isn't really read anywhere. Are
> different policies really implemented?

The table is used in two different locations:

In drivers/net/bonding/bond_main.c:

	xmit_hashtype = bond_parse_parm(xmit_hash_policy, xmit_hashtype_tbl);
	[...]
	params->xmit_policy = xmit_hashtype;

In drivers/net/bonding/bond_sysfs.c:

	new_value = bond_parse_parm(buf, xmit_hashtype_tbl);
	[...]
	bonds->params.xmit_policy = new_value;

Then, in drivers/net/bonding/bond_main.c, the value in bonds->params.xmit_policy is used to setup a 
callback into bond->xmit_hash_policy.

	static void bond_set_xmit_hash_policy(struct bonding *bond)
	{
	        switch (bond->params.xmit_policy) {
	        case BOND_XMIT_POLICY_LAYER23:
	                bond->xmit_hash_policy = bond_xmit_hash_policy_l23;
	                break;
	        case BOND_XMIT_POLICY_LAYER34:
	                bond->xmit_hash_policy = bond_xmit_hash_policy_l34;
	                break;
	        case BOND_XMIT_POLICY_LAYER2:
         	default:
	                bond->xmit_hash_policy = bond_xmit_hash_policy_l2;
	                break;
	        }
	}

Then, in drivers/net/bonding/bond_3ad.c and in drivers/net/bonding/bond_main.c, the callback is used 
to select a slave for xmit, for the two modes where xmit_hash_policy have a meaning:

         slave_agg_no = bond->xmit_hash_policy(skb, slaves_in_agg);

         slave_no = bond->xmit_hash_policy(skb, bond->slave_cnt);


So, yes, those policies are really implemented.

	Nicolas.

Re: bonding xmit_policy

[Simon Chen]

Thanks, my bad.

It is pretty strange that when I use 802.3ad mode, all my packets
(from different TCP flows) egress the same NIC even though I choose
xmit_policy to be layer3+4. That's why I wasn't quite sure whether the
policy is indeed in place.

When I switch to balance-xor mode, the packets are roughly evenly
distributed across two NICs.

-Simon

On Sun, Nov 13, 2011 at 3:26 PM, Nicolas de Pesloüan
<nicolas.2p.debian@gmail.com> wrote:
> Le 13/11/2011 03:55, Simon Chen a écrit :
>>
>> Hi folks,
>>
>> It looks like that there are three entries in xmit_policy (hashing for
>> deciding egress phy int) for bonded interfaces:
>>
>> const struct bond_parm_tbl xmit_hashtype_tbl[] = {
>> {       "layer2",               BOND_XMIT_POLICY_LAYER2},
>> {       "layer3+4",             BOND_XMIT_POLICY_LAYER34},
>> {       "layer2+3",             BOND_XMIT_POLICY_LAYER23},
>> {       NULL,                   -1},
>> };
>>
>>
>> We can set the xmit_policy either at module initiation, or later via
>> /sys. However, this xmit_policy isn't really read anywhere. Are
>> different policies really implemented?
>
> The table is used in two different locations:
>
> In drivers/net/bonding/bond_main.c:
>
>        xmit_hashtype = bond_parse_parm(xmit_hash_policy, xmit_hashtype_tbl);
>        [...]
>        params->xmit_policy = xmit_hashtype;
>
> In drivers/net/bonding/bond_sysfs.c:
>
>        new_value = bond_parse_parm(buf, xmit_hashtype_tbl);
>        [...]
>        bonds->params.xmit_policy = new_value;
>
> Then, in drivers/net/bonding/bond_main.c, the value in
> bonds->params.xmit_policy is used to setup a callback into
> bond->xmit_hash_policy.
>
>        static void bond_set_xmit_hash_policy(struct bonding *bond)
>        {
>                switch (bond->params.xmit_policy) {
>                case BOND_XMIT_POLICY_LAYER23:
>                        bond->xmit_hash_policy = bond_xmit_hash_policy_l23;
>                        break;
>                case BOND_XMIT_POLICY_LAYER34:
>                        bond->xmit_hash_policy = bond_xmit_hash_policy_l34;
>                        break;
>                case BOND_XMIT_POLICY_LAYER2:
>                default:
>                        bond->xmit_hash_policy = bond_xmit_hash_policy_l2;
>                        break;
>                }
>        }
>
> Then, in drivers/net/bonding/bond_3ad.c and in
> drivers/net/bonding/bond_main.c, the callback is used to select a slave for
> xmit, for the two modes where xmit_hash_policy have a meaning:
>
>        slave_agg_no = bond->xmit_hash_policy(skb, slaves_in_agg);
>
>        slave_no = bond->xmit_hash_policy(skb, bond->slave_cnt);
>
>
> So, yes, those policies are really implemented.
>
>        Nicolas.
>

Re: bonding xmit_policy

[Eric Dumazet]

Le mardi 15 novembre 2011 à 01:20 -0500, Simon Chen a écrit :
> Thanks, my bad.
> 
> It is pretty strange that when I use 802.3ad mode, all my packets
> (from different TCP flows) egress the same NIC even though I choose
> xmit_policy to be layer3+4. That's why I wasn't quite sure whether the
> policy is indeed in place.
> 
> When I switch to balance-xor mode, the packets are roughly evenly
> distributed across two NICs.

AFAIK, layer4 information (soure/dst ports) is limited to ipv4 TCP/UDP
trafic.

Moreover, if your bond has 2 ports, only low order bit of source and dst
port is used.

layer4_xor = ntohs((*layer4hdr ^ *(layer4hdr + 1)));

So if your trafic is RTP, it probably use only even ports, (RTCP using
odd ports), and uses a single slave.