[PATCH net-next 0/2] bonding: extend round-robin mode

[PATCH net-next 0/2] bonding: extend round-robin mode

[Nikolay Aleksandrov]

This small patchset adds a new option called packets_per_slave to the
bonding which aims to extend round-robin mode with the following effects:
0 - choose the slave id at random
1 - packet per slave (standard round-robin, default option value)
 >1 - transmit >1 packets per slave, switch the slaves in round-robin
Patch02 adds a description for the new option to the bonding documentation.

Best regards,
 Nikolay Aleksandrov

Nikolay Aleksandrov (2):
  bonding: extend round-robin mode with packets_per_slave
  bonding: document the new packets_per_slave option

 Documentation/networking/bonding.txt |  9 ++++++
 drivers/net/bonding/bond_main.c      | 55 ++++++++++++++++++++++++++++++++----
 drivers/net/bonding/bond_sysfs.c     | 49 ++++++++++++++++++++++++++++++++
 drivers/net/bonding/bonding.h        |  3 +-
 4 files changed, 110 insertions(+), 6 deletions(-)

-- 
1.8.1.4

[PATCH net-next 1/2] bonding: extend round-robin mode with packets_per_slave

[Nikolay Aleksandrov]

This patch aims to extend round-robin mode with a new option called
packets_per_slave which can have the following values and effects:
0 - choose a random slave
1 (default) - standard round-robin, 1 packet per slave
 >1 - round-robin when >1 packets have been transmitted per slave
The allowed values are between 0 and 65535.
This patch also fixes the comment style in bond_xmit_roundrobin().

Signed-off-by: Nikolay Aleksandrov <nikolay@redhat.com>
---
 drivers/net/bonding/bond_main.c  | 55 ++++++++++++++++++++++++++++++++++++----
 drivers/net/bonding/bond_sysfs.c | 49 +++++++++++++++++++++++++++++++++++
 drivers/net/bonding/bonding.h    |  3 ++-
 3 files changed, 101 insertions(+), 6 deletions(-)

diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index a141f40..4dd5ee2 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -79,6 +79,7 @@
 #include <net/pkt_sched.h>
 #include <linux/rculist.h>
 #include <net/flow_keys.h>
+#include <linux/reciprocal_div.h>
 #include "bonding.h"
 #include "bond_3ad.h"
 #include "bond_alb.h"
@@ -111,6 +112,7 @@ static char *fail_over_mac;
 static int all_slaves_active;
 static struct bond_params bonding_defaults;
 static int resend_igmp = BOND_DEFAULT_RESEND_IGMP;
+static int packets_per_slave = 1;
 
 module_param(max_bonds, int, 0);
 MODULE_PARM_DESC(max_bonds, "Max number of bonded devices");
@@ -183,6 +185,10 @@ MODULE_PARM_DESC(all_slaves_active, "Keep all frames received on an interface"
 module_param(resend_igmp, int, 0);
 MODULE_PARM_DESC(resend_igmp, "Number of IGMP membership reports to send on "
 			      "link failure");
+module_param(packets_per_slave, int, 0);
+MODULE_PARM_DESC(packets_per_slave, "Packets to send per slave in balance-rr "
+				    "mode; 0 for a random slave, 1 packet per "
+				    "slave (default), >1 packets per slave.");
 
 /*----------------------------- Global variables ----------------------------*/
 
@@ -3574,14 +3580,44 @@ void bond_xmit_slave_id(struct bonding *bond, struct sk_buff *skb, int slave_id)
 	kfree_skb(skb);
 }
 
+/**
+ * bond_rr_gen_slave_id - generate slave id based on packets_per_slave
+ * @bond: bonding device to use
+ *
+ * Based on the value of the bonding device's packets_per_slave parameter
+ * this function generates a slave id, which is usually used as the next
+ * slave to transmit through.
+ */
+static u32 bond_rr_gen_slave_id(struct bonding *bond)
+{
+	int packets_per_slave = bond->params.packets_per_slave;
+	u32 slave_id;
+
+	switch (packets_per_slave) {
+	case 0:
+		slave_id = prandom_u32();
+		break;
+	case 1:
+		slave_id = bond->rr_tx_counter;
+		break;
+	default:
+		slave_id = reciprocal_divide(bond->rr_tx_counter,
+					     packets_per_slave);
+		break;
+	}
+	bond->rr_tx_counter++;
+
+	return slave_id;
+}
+
 static int bond_xmit_roundrobin(struct sk_buff *skb, struct net_device *bond_dev)
 {
 	struct bonding *bond = netdev_priv(bond_dev);
 	struct iphdr *iph = ip_hdr(skb);
 	struct slave *slave;
+	u32 slave_id;
 
-	/*
-	 * Start with the curr_active_slave that joined the bond as the
+	/* Start with the curr_active_slave that joined the bond as the
 	 * default for sending IGMP traffic.  For failover purposes one
 	 * needs to maintain some consistency for the interface that will
 	 * send the join/membership reports.  The curr_active_slave found
@@ -3594,8 +3630,8 @@ static int bond_xmit_roundrobin(struct sk_buff *skb, struct net_device *bond_dev
 		else
 			bond_xmit_slave_id(bond, skb, 0);
 	} else {
-		bond_xmit_slave_id(bond, skb,
-				   bond->rr_tx_counter++ % bond->slave_cnt);
+		slave_id = bond_rr_gen_slave_id(bond);
+		bond_xmit_slave_id(bond, skb, slave_id % bond->slave_cnt);
 	}
 
 	return NETDEV_TX_OK;
@@ -4099,6 +4135,12 @@ static int bond_check_params(struct bond_params *params)
 		resend_igmp = BOND_DEFAULT_RESEND_IGMP;
 	}
 
+	if (packets_per_slave < 0 || packets_per_slave > USHRT_MAX) {
+		pr_warn("Warning: packets_per_slave (%d) should be between 0 and %u resetting to 1\n",
+			packets_per_slave, USHRT_MAX);
+		packets_per_slave = 1;
+	}
+
 	/* reset values for TLB/ALB */
 	if ((bond_mode == BOND_MODE_TLB) ||
 	    (bond_mode == BOND_MODE_ALB)) {
@@ -4288,7 +4330,10 @@ static int bond_check_params(struct bond_params *params)
 	params->resend_igmp = resend_igmp;
 	params->min_links = min_links;
 	params->lp_interval = BOND_ALB_DEFAULT_LP_INTERVAL;
-
+	if (packets_per_slave > 1)
+		params->packets_per_slave = reciprocal_value(packets_per_slave);
+	else
+		params->packets_per_slave = packets_per_slave;
 	if (primary) {
 		strncpy(params->primary, primary, IFNAMSIZ);
 		params->primary[IFNAMSIZ - 1] = 0;
diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
index 47749c9..75dc4d0 100644
--- a/drivers/net/bonding/bond_sysfs.c
+++ b/drivers/net/bonding/bond_sysfs.c
@@ -40,6 +40,7 @@
 #include <net/net_namespace.h>
 #include <net/netns/generic.h>
 #include <linux/nsproxy.h>
+#include <linux/reciprocal_div.h>
 
 #include "bonding.h"
 
@@ -1640,6 +1641,53 @@ out:
 static DEVICE_ATTR(lp_interval, S_IRUGO | S_IWUSR,
 		   bonding_show_lp_interval, bonding_store_lp_interval);
 
+static ssize_t bonding_show_packets_per_slave(struct device *d,
+					      struct device_attribute *attr,
+					      char *buf)
+{
+	struct bonding *bond = to_bond(d);
+	int packets_per_slave = bond->params.packets_per_slave;
+
+	if (packets_per_slave > 1)
+		packets_per_slave = reciprocal_value(packets_per_slave);
+
+	return sprintf(buf, "%d\n", packets_per_slave);
+}
+
+static ssize_t bonding_store_packets_per_slave(struct device *d,
+					       struct device_attribute *attr,
+					       const char *buf, size_t count)
+{
+	struct bonding *bond = to_bond(d);
+	int new_value, ret = count;
+
+	if (sscanf(buf, "%d", &new_value) != 1) {
+		pr_err("%s: no packets_per_slave value specified.\n",
+		       bond->dev->name);
+		ret = -EINVAL;
+		goto out;
+	}
+	if (new_value < 0 || new_value > USHRT_MAX) {
+		pr_err("%s: packets_per_slave must be between 0 and %u\n",
+		       bond->dev->name, USHRT_MAX);
+		ret = -EINVAL;
+		goto out;
+	}
+	if (bond->params.mode != BOND_MODE_ROUNDROBIN)
+		pr_warn("%s: Warning: packets_per_slave has effect only in balance-rr mode\n",
+			bond->dev->name);
+	if (new_value > 1)
+		bond->params.packets_per_slave = reciprocal_value(new_value);
+	else
+		bond->params.packets_per_slave = new_value;
+out:
+	return ret;
+}
+
+static DEVICE_ATTR(packets_per_slave, S_IRUGO | S_IWUSR,
+		   bonding_show_packets_per_slave,
+		   bonding_store_packets_per_slave);
+
 static struct attribute *per_bond_attrs[] = {
 	&dev_attr_slaves.attr,
 	&dev_attr_mode.attr,
@@ -1671,6 +1719,7 @@ static struct attribute *per_bond_attrs[] = {
 	&dev_attr_resend_igmp.attr,
 	&dev_attr_min_links.attr,
 	&dev_attr_lp_interval.attr,
+	&dev_attr_packets_per_slave.attr,
 	NULL,
 };
 
diff --git a/drivers/net/bonding/bonding.h b/drivers/net/bonding/bonding.h
index 046a605..77a07a1 100644
--- a/drivers/net/bonding/bonding.h
+++ b/drivers/net/bonding/bonding.h
@@ -156,6 +156,7 @@ struct bond_params {
 	int all_slaves_active;
 	int resend_igmp;
 	int lp_interval;
+	int packets_per_slave;
 };
 
 struct bond_parm_tbl {
@@ -222,7 +223,7 @@ struct bonding {
 	char     proc_file_name[IFNAMSIZ];
 #endif /* CONFIG_PROC_FS */
 	struct   list_head bond_list;
-	u16      rr_tx_counter;
+	u32      rr_tx_counter;
 	struct   ad_bond_info ad_info;
 	struct   alb_bond_info alb_info;
 	struct   bond_params params;
-- 
1.8.1.4

Re: [PATCH net-next 1/2] bonding: extend round-robin mode with packets_per_slave

[Veaceslav Falico]

On Tue, Nov 05, 2013 at 01:51:41PM +0100, Nikolay Aleksandrov wrote:
>This patch aims to extend round-robin mode with a new option called
>packets_per_slave which can have the following values and effects:
>0 - choose a random slave
>1 (default) - standard round-robin, 1 packet per slave
> >1 - round-robin when >1 packets have been transmitted per slave
>The allowed values are between 0 and 65535.
>This patch also fixes the comment style in bond_xmit_roundrobin().
>
>Signed-off-by: Nikolay Aleksandrov <nikolay@redhat.com>

Clean and simple, thanks a lot!

Acked-by: Veaceslav Falico <vfalico@redhat.com>

>---
> drivers/net/bonding/bond_main.c  | 55 ++++++++++++++++++++++++++++++++++++----
> drivers/net/bonding/bond_sysfs.c | 49 +++++++++++++++++++++++++++++++++++
> drivers/net/bonding/bonding.h    |  3 ++-
> 3 files changed, 101 insertions(+), 6 deletions(-)
>
>diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
>index a141f40..4dd5ee2 100644
>--- a/drivers/net/bonding/bond_main.c
>+++ b/drivers/net/bonding/bond_main.c
>@@ -79,6 +79,7 @@
> #include <net/pkt_sched.h>
> #include <linux/rculist.h>
> #include <net/flow_keys.h>
>+#include <linux/reciprocal_div.h>
> #include "bonding.h"
> #include "bond_3ad.h"
> #include "bond_alb.h"
>@@ -111,6 +112,7 @@ static char *fail_over_mac;
> static int all_slaves_active;
> static struct bond_params bonding_defaults;
> static int resend_igmp = BOND_DEFAULT_RESEND_IGMP;
>+static int packets_per_slave = 1;
>
> module_param(max_bonds, int, 0);
> MODULE_PARM_DESC(max_bonds, "Max number of bonded devices");
>@@ -183,6 +185,10 @@ MODULE_PARM_DESC(all_slaves_active, "Keep all frames received on an interface"
> module_param(resend_igmp, int, 0);
> MODULE_PARM_DESC(resend_igmp, "Number of IGMP membership reports to send on "
> 			      "link failure");
>+module_param(packets_per_slave, int, 0);
>+MODULE_PARM_DESC(packets_per_slave, "Packets to send per slave in balance-rr "
>+				    "mode; 0 for a random slave, 1 packet per "
>+				    "slave (default), >1 packets per slave.");
>
> /*----------------------------- Global variables ----------------------------*/
>
>@@ -3574,14 +3580,44 @@ void bond_xmit_slave_id(struct bonding *bond, struct sk_buff *skb, int slave_id)
> 	kfree_skb(skb);
> }
>
>+/**
>+ * bond_rr_gen_slave_id - generate slave id based on packets_per_slave
>+ * @bond: bonding device to use
>+ *
>+ * Based on the value of the bonding device's packets_per_slave parameter
>+ * this function generates a slave id, which is usually used as the next
>+ * slave to transmit through.
>+ */
>+static u32 bond_rr_gen_slave_id(struct bonding *bond)
>+{
>+	int packets_per_slave = bond->params.packets_per_slave;
>+	u32 slave_id;
>+
>+	switch (packets_per_slave) {
>+	case 0:
>+		slave_id = prandom_u32();
>+		break;
>+	case 1:
>+		slave_id = bond->rr_tx_counter;
>+		break;
>+	default:
>+		slave_id = reciprocal_divide(bond->rr_tx_counter,
>+					     packets_per_slave);
>+		break;
>+	}
>+	bond->rr_tx_counter++;
>+
>+	return slave_id;
>+}
>+
> static int bond_xmit_roundrobin(struct sk_buff *skb, struct net_device *bond_dev)
> {
> 	struct bonding *bond = netdev_priv(bond_dev);
> 	struct iphdr *iph = ip_hdr(skb);
> 	struct slave *slave;
>+	u32 slave_id;
>
>-	/*
>-	 * Start with the curr_active_slave that joined the bond as the
>+	/* Start with the curr_active_slave that joined the bond as the
> 	 * default for sending IGMP traffic.  For failover purposes one
> 	 * needs to maintain some consistency for the interface that will
> 	 * send the join/membership reports.  The curr_active_slave found
>@@ -3594,8 +3630,8 @@ static int bond_xmit_roundrobin(struct sk_buff *skb, struct net_device *bond_dev
> 		else
> 			bond_xmit_slave_id(bond, skb, 0);
> 	} else {
>-		bond_xmit_slave_id(bond, skb,
>-				   bond->rr_tx_counter++ % bond->slave_cnt);
>+		slave_id = bond_rr_gen_slave_id(bond);
>+		bond_xmit_slave_id(bond, skb, slave_id % bond->slave_cnt);
> 	}
>
> 	return NETDEV_TX_OK;
>@@ -4099,6 +4135,12 @@ static int bond_check_params(struct bond_params *params)
> 		resend_igmp = BOND_DEFAULT_RESEND_IGMP;
> 	}
>
>+	if (packets_per_slave < 0 || packets_per_slave > USHRT_MAX) {
>+		pr_warn("Warning: packets_per_slave (%d) should be between 0 and %u resetting to 1\n",
>+			packets_per_slave, USHRT_MAX);
>+		packets_per_slave = 1;
>+	}
>+
> 	/* reset values for TLB/ALB */
> 	if ((bond_mode == BOND_MODE_TLB) ||
> 	    (bond_mode == BOND_MODE_ALB)) {
>@@ -4288,7 +4330,10 @@ static int bond_check_params(struct bond_params *params)
> 	params->resend_igmp = resend_igmp;
> 	params->min_links = min_links;
> 	params->lp_interval = BOND_ALB_DEFAULT_LP_INTERVAL;
>-
>+	if (packets_per_slave > 1)
>+		params->packets_per_slave = reciprocal_value(packets_per_slave);
>+	else
>+		params->packets_per_slave = packets_per_slave;
> 	if (primary) {
> 		strncpy(params->primary, primary, IFNAMSIZ);
> 		params->primary[IFNAMSIZ - 1] = 0;
>diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
>index 47749c9..75dc4d0 100644
>--- a/drivers/net/bonding/bond_sysfs.c
>+++ b/drivers/net/bonding/bond_sysfs.c
>@@ -40,6 +40,7 @@
> #include <net/net_namespace.h>
> #include <net/netns/generic.h>
> #include <linux/nsproxy.h>
>+#include <linux/reciprocal_div.h>
>
> #include "bonding.h"
>
>@@ -1640,6 +1641,53 @@ out:
> static DEVICE_ATTR(lp_interval, S_IRUGO | S_IWUSR,
> 		   bonding_show_lp_interval, bonding_store_lp_interval);
>
>+static ssize_t bonding_show_packets_per_slave(struct device *d,
>+					      struct device_attribute *attr,
>+					      char *buf)
>+{
>+	struct bonding *bond = to_bond(d);
>+	int packets_per_slave = bond->params.packets_per_slave;
>+
>+	if (packets_per_slave > 1)
>+		packets_per_slave = reciprocal_value(packets_per_slave);
>+
>+	return sprintf(buf, "%d\n", packets_per_slave);
>+}
>+
>+static ssize_t bonding_store_packets_per_slave(struct device *d,
>+					       struct device_attribute *attr,
>+					       const char *buf, size_t count)
>+{
>+	struct bonding *bond = to_bond(d);
>+	int new_value, ret = count;
>+
>+	if (sscanf(buf, "%d", &new_value) != 1) {
>+		pr_err("%s: no packets_per_slave value specified.\n",
>+		       bond->dev->name);
>+		ret = -EINVAL;
>+		goto out;
>+	}
>+	if (new_value < 0 || new_value > USHRT_MAX) {
>+		pr_err("%s: packets_per_slave must be between 0 and %u\n",
>+		       bond->dev->name, USHRT_MAX);
>+		ret = -EINVAL;
>+		goto out;
>+	}
>+	if (bond->params.mode != BOND_MODE_ROUNDROBIN)
>+		pr_warn("%s: Warning: packets_per_slave has effect only in balance-rr mode\n",
>+			bond->dev->name);
>+	if (new_value > 1)
>+		bond->params.packets_per_slave = reciprocal_value(new_value);
>+	else
>+		bond->params.packets_per_slave = new_value;
>+out:
>+	return ret;
>+}
>+
>+static DEVICE_ATTR(packets_per_slave, S_IRUGO | S_IWUSR,
>+		   bonding_show_packets_per_slave,
>+		   bonding_store_packets_per_slave);
>+
> static struct attribute *per_bond_attrs[] = {
> 	&dev_attr_slaves.attr,
> 	&dev_attr_mode.attr,
>@@ -1671,6 +1719,7 @@ static struct attribute *per_bond_attrs[] = {
> 	&dev_attr_resend_igmp.attr,
> 	&dev_attr_min_links.attr,
> 	&dev_attr_lp_interval.attr,
>+	&dev_attr_packets_per_slave.attr,
> 	NULL,
> };
>
>diff --git a/drivers/net/bonding/bonding.h b/drivers/net/bonding/bonding.h
>index 046a605..77a07a1 100644
>--- a/drivers/net/bonding/bonding.h
>+++ b/drivers/net/bonding/bonding.h
>@@ -156,6 +156,7 @@ struct bond_params {
> 	int all_slaves_active;
> 	int resend_igmp;
> 	int lp_interval;
>+	int packets_per_slave;
> };
>
> struct bond_parm_tbl {
>@@ -222,7 +223,7 @@ struct bonding {
> 	char     proc_file_name[IFNAMSIZ];
> #endif /* CONFIG_PROC_FS */
> 	struct   list_head bond_list;
>-	u16      rr_tx_counter;
>+	u32      rr_tx_counter;
> 	struct   ad_bond_info ad_info;
> 	struct   alb_bond_info alb_info;
> 	struct   bond_params params;
>-- 
>1.8.1.4
>

[PATCH net-next 2/2] bonding: document the new packets_per_slave option

[Nikolay Aleksandrov]

Add new documentation for the packets_per_slave option available
for balance-rr mode.

Signed-off-by: Nikolay Aleksandrov <nikolay@redhat.com>
---
 Documentation/networking/bonding.txt | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Documentation/networking/bonding.txt b/Documentation/networking/bonding.txt
index 3856ed2..2cdb8b6 100644
--- a/Documentation/networking/bonding.txt
+++ b/Documentation/networking/bonding.txt
@@ -639,6 +639,15 @@ num_unsol_na
 	are generated by the ipv4 and ipv6 code and the numbers of
 	repetitions cannot be set independently.
 
+packets_per_slave
+
+	Specify the number of packets to transmit through a slave before
+	moving to the next one. When set to 0 then a slave is chosen at
+	random.
+
+	The valid range is 0 - 65535; the default value is 1. This option
+	has effect only in balance-rr mode.
+
 primary
 
 	A string (eth0, eth2, etc) specifying which slave is the
-- 
1.8.1.4

Re: [PATCH net-next 0/2] bonding: extend round-robin mode

[Jay Vosburgh]

Nikolay Aleksandrov <nikolay@redhat.com> wrote:

>This small patchset adds a new option called packets_per_slave to the
>bonding which aims to extend round-robin mode with the following effects:
>0 - choose the slave id at random
>1 - packet per slave (standard round-robin, default option value)
> >1 - transmit >1 packets per slave, switch the slaves in round-robin
>Patch02 adds a description for the new option to the bonding documentation.

	Could you explain why this is useful?  My guess is that you're
trying to synchronize with the packet receive processing of a peer
(perhaps for GRO?), but I think it would be useful to explain the
utility of this.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

Re: [PATCH net-next 0/2] bonding: extend round-robin mode

[Nikolay Aleksandrov]

On 11/06/2013 05:55 PM, Jay Vosburgh wrote:
> Nikolay Aleksandrov <nikolay@redhat.com> wrote:
> 
>> This small patchset adds a new option called packets_per_slave to the
>> bonding which aims to extend round-robin mode with the following effects:
>> 0 - choose the slave id at random
>> 1 - packet per slave (standard round-robin, default option value)
>>> 1 - transmit >1 packets per slave, switch the slaves in round-robin
>> Patch02 adds a description for the new option to the bonding documentation.
> 
> 	Could you explain why this is useful?  My guess is that you're
> trying to synchronize with the packet receive processing of a peer
> (perhaps for GRO?), but I think it would be useful to explain the
> utility of this.
> 
> 	-J
> 
> ---
> 	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
> 
Hi Jay,
Yes, that is one good use case, I'm also experimenting with a user-space
software that uses various heuristics and tunes this option (e.g., TCP-RR
case). I've been playing with this option for the past 3 weeks or so and
have to move on to some real-world tests, since my current environment
consists only of VMs and that's nowhere near the real world :-)
If your intention is to include such information in the bonding
documentation then I'll need some more time to gather it, and can do it as
either a follow-up or we can just drop this now and I'll re-post once I've
some definite real(lab)-world results.

Nik

Re: [PATCH net-next 0/2] bonding: extend round-robin mode

[David Miller]

From: Nikolay Aleksandrov <nikolay@redhat.com>
Date: Tue,  5 Nov 2013 13:51:40 +0100

> This small patchset adds a new option called packets_per_slave to the
> bonding which aims to extend round-robin mode with the following effects:
> 0 - choose the slave id at random
> 1 - packet per slave (standard round-robin, default option value)
>  >1 - transmit >1 packets per slave, switch the slaves in round-robin
> Patch02 adds a description for the new option to the bonding documentation.

Series applied, thanks.