* [PATCH net v6 1/2] bonding: convert unbalanced_load to per-cpu state
2026-08-31 1:50 [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs Hangbin Liu
@ 2026-08-31 1:50 ` Hangbin Liu
2026-08-31 1:51 ` [PATCH net v6 2/2] bonding: fix u32 overflow in compute_gap() Hangbin Liu
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Hangbin Liu @ 2026-08-31 1:50 UTC (permalink / raw)
To: Jay Vosburgh, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Simon Horman, Nikolay Aleksandrov
Cc: Hangbin Liu, netdev, linux-kernel, Hangbin Liu
From: Hangbin Liu <liuhangbin@kylinos.cn>
A later patch widens the bonding TLB tx counters from u32 to u64. The
unbalanced_load counter sits in the transmit hot path, and cross-CPU
synchronization of a u64 would introduce measurable overhead. Convert
unbalanced_load to a per-cpu counter first so that the subsequent
widening only touches per-cpu data local to each CPU.
Introduce struct unbalanced_load_stats to hold the per-cpu counter,
and move the aggregation into a helper, reset_unbalanced_load(), which
sums all per-cpu instances. Use the delta of current total load vs
variable prev_total_unbalanced to calculate the loading.
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Signed-off-by: Hangbin Liu <liuhangbin@kylinos.cn>
---
drivers/net/bonding/bond_alb.c | 37 ++++++++++++++++++++++++++++++-------
include/net/bond_alb.h | 7 ++++++-
2 files changed, 36 insertions(+), 8 deletions(-)
diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c
index 839f7482dc18..0afed2c39231 100644
--- a/drivers/net/bonding/bond_alb.c
+++ b/drivers/net/bonding/bond_alb.c
@@ -133,6 +133,10 @@ static int tlb_initialize(struct bonding *bond)
if (!new_hashtbl)
return -ENOMEM;
+ bond_info->unbalanced_load = alloc_percpu(struct unbalanced_load_stats);
+ if (!bond_info->unbalanced_load)
+ goto out;
+
spin_lock_bh(&bond->mode_lock);
bond_info->tx_hashtbl = new_hashtbl;
@@ -143,6 +147,10 @@ static int tlb_initialize(struct bonding *bond)
spin_unlock_bh(&bond->mode_lock);
return 0;
+
+out:
+ kfree(new_hashtbl);
+ return -ENOMEM;
}
/* Must be called only after all slaves have been released */
@@ -154,6 +162,8 @@ static void tlb_deinitialize(struct bonding *bond)
kfree(bond_info->tx_hashtbl);
bond_info->tx_hashtbl = NULL;
+ free_percpu(bond_info->unbalanced_load);
+ bond_info->prev_total_unbalanced = 0;
spin_unlock_bh(&bond->mode_lock);
}
@@ -1345,7 +1355,7 @@ static netdev_tx_t bond_do_alb_xmit(struct sk_buff *skb, struct bonding *bond,
/* unbalanced or unassigned, send through primary */
tx_slave = rcu_dereference(bond->curr_active_slave);
if (bond->params.tlb_dynamic_lb)
- bond_info->unbalanced_load += skb->len;
+ this_cpu_add(bond_info->unbalanced_load->tx_bytes, skb->len);
}
if (tx_slave && bond_slave_can_tx(tx_slave)) {
@@ -1529,6 +1539,23 @@ netdev_tx_t bond_alb_xmit(struct sk_buff *skb, struct net_device *bond_dev)
return bond_do_alb_xmit(skb, bond, tx_slave);
}
+static u32 reset_unbalanced_load(struct alb_bond_info *bond_info)
+{
+ struct unbalanced_load_stats *p;
+ u32 delta, total_bytes = 0;
+ int i;
+
+ for_each_possible_cpu(i) {
+ p = per_cpu_ptr(bond_info->unbalanced_load, i);
+ total_bytes += READ_ONCE(p->tx_bytes);
+ }
+
+ delta = total_bytes - bond_info->prev_total_unbalanced;
+ bond_info->prev_total_unbalanced = total_bytes;
+
+ return delta / BOND_TLB_REBALANCE_INTERVAL;
+}
+
void bond_alb_monitor(struct work_struct *work)
{
struct bonding *bond = container_of(work, struct bonding,
@@ -1570,12 +1597,8 @@ void bond_alb_monitor(struct work_struct *work)
if (atomic_read(&bond_info->tx_rebalance_counter) >= BOND_TLB_REBALANCE_TICKS) {
bond_for_each_slave_rcu(bond, slave, iter) {
tlb_clear_slave(bond, slave, 1);
- if (slave == rcu_access_pointer(bond->curr_active_slave)) {
- SLAVE_TLB_INFO(slave).load =
- bond_info->unbalanced_load /
- BOND_TLB_REBALANCE_INTERVAL;
- bond_info->unbalanced_load = 0;
- }
+ if (slave == rcu_access_pointer(bond->curr_active_slave))
+ SLAVE_TLB_INFO(slave).load = reset_unbalanced_load(bond_info);
}
atomic_set(&bond_info->tx_rebalance_counter, 0);
}
diff --git a/include/net/bond_alb.h b/include/net/bond_alb.h
index e5945427f38d..6fb09b4fc7e2 100644
--- a/include/net/bond_alb.h
+++ b/include/net/bond_alb.h
@@ -123,9 +123,14 @@ struct tlb_slave_info {
*/
};
+struct unbalanced_load_stats {
+ u32 tx_bytes;
+};
+
struct alb_bond_info {
struct tlb_client_info *tx_hashtbl; /* Dynamically allocated */
- u32 unbalanced_load;
+ struct unbalanced_load_stats __percpu *unbalanced_load;
+ u32 prev_total_unbalanced;
atomic_t tx_rebalance_counter;
int lp_counter;
/* -------- rlb parameters -------- */
--
2.55.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH net v6 2/2] bonding: fix u32 overflow in compute_gap()
2026-08-31 1:50 [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs Hangbin Liu
2026-08-31 1:50 ` [PATCH net v6 1/2] bonding: convert unbalanced_load to per-cpu state Hangbin Liu
@ 2026-08-31 1:51 ` Hangbin Liu
2026-09-02 1:48 ` [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs Jay Vosburgh
2026-09-03 1:40 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 5+ messages in thread
From: Hangbin Liu @ 2026-08-31 1:51 UTC (permalink / raw)
To: Jay Vosburgh, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Simon Horman, Nikolay Aleksandrov
Cc: Hangbin Liu, netdev, linux-kernel, Hangbin Liu
From: Hangbin Liu <liuhangbin@kylinos.cn>
The TLB load-tracking fields tx_bytes, load_history, load, and
unbalanced_load are all u32. At sustained throughput above ~3.2 Gbit/s
over the 10-second rebalance interval the byte counters wrap, causing
compute_gap() to produce incorrect gap values and mis-select slaves.
Such speeds are common on modern NICs under heavy traffic.
Widen these fields to u64. Use u64_stats_sync to protect the per-cpu
unbalanced_load_stats against tearing on 32-bit architectures, and
div_u64() for the 64-bit divisions. The tx_bytes and load_history
are protected in spin_lock. Also protect the slave load writing in
bond_alb_monitor() with spin_lock in case of tear on 32-bit.
For compute_gap(), we still use s64 arithmetic throughout, so we can
preserve the existing calculation logic. This order of magnitude is
sufficiently large, there is no need to worry about overflow for now.
Detected by AI code review.
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Reviewed-by: Nikolay Aleksandrov <razor@blackwall.org>
Signed-off-by: Hangbin Liu <liuhangbin@kylinos.cn>
---
drivers/net/bonding/bond_alb.c | 46 ++++++++++++++++++++++++++++++------------
include/net/bond_alb.h | 11 +++++-----
2 files changed, 39 insertions(+), 18 deletions(-)
diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c
index 0afed2c39231..bbe9a6985f90 100644
--- a/drivers/net/bonding/bond_alb.c
+++ b/drivers/net/bonding/bond_alb.c
@@ -6,6 +6,7 @@
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
#include <linux/pkt_sched.h>
#include <linux/spinlock.h>
#include <linux/slab.h>
@@ -74,8 +75,8 @@ static inline u8 _simple_hash(const u8 *hash_start, int hash_size)
static inline void tlb_init_table_entry(struct tlb_client_info *entry, int save_load)
{
if (save_load) {
- entry->load_history = 1 + entry->tx_bytes /
- BOND_TLB_REBALANCE_INTERVAL;
+ entry->load_history = 1 + div_u64(entry->tx_bytes,
+ BOND_TLB_REBALANCE_INTERVAL);
entry->tx_bytes = 0;
}
@@ -133,7 +134,7 @@ static int tlb_initialize(struct bonding *bond)
if (!new_hashtbl)
return -ENOMEM;
- bond_info->unbalanced_load = alloc_percpu(struct unbalanced_load_stats);
+ bond_info->unbalanced_load = netdev_alloc_pcpu_stats(struct unbalanced_load_stats);
if (!bond_info->unbalanced_load)
goto out;
@@ -170,8 +171,10 @@ static void tlb_deinitialize(struct bonding *bond)
static long long compute_gap(struct slave *slave)
{
- return (s64) (slave->speed << 20) - /* Convert to Megabit per sec */
- (s64) (SLAVE_TLB_INFO(slave).load << 3); /* Bytes to bits */
+ u32 raw_speed = READ_ONCE(slave->speed);
+
+ return ((s64)raw_speed << 20) - /* Convert to bits per sec */
+ ((s64)SLAVE_TLB_INFO(slave).load << 3); /* Bytes to bits */
}
static struct slave *tlb_get_least_loaded_slave(struct bonding *bond)
@@ -1354,8 +1357,14 @@ static netdev_tx_t bond_do_alb_xmit(struct sk_buff *skb, struct bonding *bond,
if (!tx_slave) {
/* unbalanced or unassigned, send through primary */
tx_slave = rcu_dereference(bond->curr_active_slave);
- if (bond->params.tlb_dynamic_lb)
- this_cpu_add(bond_info->unbalanced_load->tx_bytes, skb->len);
+ if (bond->params.tlb_dynamic_lb) {
+ struct unbalanced_load_stats *pcpu_load;
+
+ pcpu_load = this_cpu_ptr(bond_info->unbalanced_load);
+ u64_stats_update_begin(&pcpu_load->syncp);
+ u64_stats_add(&pcpu_load->tx_bytes, skb->len);
+ u64_stats_update_end(&pcpu_load->syncp);
+ }
}
if (tx_slave && bond_slave_can_tx(tx_slave)) {
@@ -1539,21 +1548,27 @@ netdev_tx_t bond_alb_xmit(struct sk_buff *skb, struct net_device *bond_dev)
return bond_do_alb_xmit(skb, bond, tx_slave);
}
-static u32 reset_unbalanced_load(struct alb_bond_info *bond_info)
+static u64 reset_unbalanced_load(struct alb_bond_info *bond_info)
{
+ u64 delta, tx_bytes, total_bytes = 0;
struct unbalanced_load_stats *p;
- u32 delta, total_bytes = 0;
+ unsigned int start;
int i;
for_each_possible_cpu(i) {
p = per_cpu_ptr(bond_info->unbalanced_load, i);
- total_bytes += READ_ONCE(p->tx_bytes);
+ do {
+ start = u64_stats_fetch_begin(&p->syncp);
+ tx_bytes = u64_stats_read(&p->tx_bytes);
+ } while (u64_stats_fetch_retry(&p->syncp, start));
+
+ total_bytes += tx_bytes;
}
delta = total_bytes - bond_info->prev_total_unbalanced;
bond_info->prev_total_unbalanced = total_bytes;
- return delta / BOND_TLB_REBALANCE_INTERVAL;
+ return div_u64(delta, BOND_TLB_REBALANCE_INTERVAL);
}
void bond_alb_monitor(struct work_struct *work)
@@ -1597,8 +1612,13 @@ void bond_alb_monitor(struct work_struct *work)
if (atomic_read(&bond_info->tx_rebalance_counter) >= BOND_TLB_REBALANCE_TICKS) {
bond_for_each_slave_rcu(bond, slave, iter) {
tlb_clear_slave(bond, slave, 1);
- if (slave == rcu_access_pointer(bond->curr_active_slave))
- SLAVE_TLB_INFO(slave).load = reset_unbalanced_load(bond_info);
+ if (slave == rcu_access_pointer(bond->curr_active_slave)) {
+ u64 new_load = reset_unbalanced_load(bond_info);
+
+ spin_lock_bh(&bond->mode_lock);
+ SLAVE_TLB_INFO(slave).load = new_load;
+ spin_unlock_bh(&bond->mode_lock);
+ }
}
atomic_set(&bond_info->tx_rebalance_counter, 0);
}
diff --git a/include/net/bond_alb.h b/include/net/bond_alb.h
index 6fb09b4fc7e2..32f1981033e4 100644
--- a/include/net/bond_alb.h
+++ b/include/net/bond_alb.h
@@ -57,12 +57,12 @@ struct tlb_client_info {
* packets to a Client that the Hash function
* gave this entry index.
*/
- u32 tx_bytes; /* Each Client accumulates the BytesTx that
+ u64 tx_bytes; /* Each Client accumulates the BytesTx that
* were transmitted to it, and after each
* CallBack the LoadHistory is divided
* by the balance interval
*/
- u32 load_history; /* This field contains the amount of Bytes
+ u64 load_history; /* This field contains the amount of Bytes
* that were transmitted to this client by
* the server on the previous balance
* interval in Bps.
@@ -118,19 +118,20 @@ struct tlb_slave_info {
* are the entries that were assigned to use this
* slave for transmit.
*/
- u32 load; /* Each slave sums the loadHistory of all clients
+ u64 load; /* Each slave sums the loadHistory of all clients
* assigned to it
*/
};
struct unbalanced_load_stats {
- u32 tx_bytes;
+ u64_stats_t tx_bytes;
+ struct u64_stats_sync syncp;
};
struct alb_bond_info {
struct tlb_client_info *tx_hashtbl; /* Dynamically allocated */
struct unbalanced_load_stats __percpu *unbalanced_load;
- u32 prev_total_unbalanced;
+ u64 prev_total_unbalanced;
atomic_t tx_rebalance_counter;
int lp_counter;
/* -------- rlb parameters -------- */
--
2.55.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs
2026-08-31 1:50 [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs Hangbin Liu
2026-08-31 1:50 ` [PATCH net v6 1/2] bonding: convert unbalanced_load to per-cpu state Hangbin Liu
2026-08-31 1:51 ` [PATCH net v6 2/2] bonding: fix u32 overflow in compute_gap() Hangbin Liu
@ 2026-09-02 1:48 ` Jay Vosburgh
2026-09-03 1:40 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 5+ messages in thread
From: Jay Vosburgh @ 2026-09-02 1:48 UTC (permalink / raw)
To: Hangbin Liu
Cc: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Simon Horman, Nikolay Aleksandrov, netdev,
linux-kernel, Hangbin Liu
Hangbin Liu <hangbin.liu@linux.dev> wrote:
>The bonding TLB (Transmit Load Balancing) mode tracks per-slave and
>per-client transmit byte counts in u32 fields. At sustained throughput
>above ~3.2 Gbit/s over the 10-second rebalance interval these counters
>wrap, causing compute_gap() to produce incorrect gap values and
>mis-select transmit slaves. Such speeds are routine on modern NICs
>under heavy traffic.
>
>This two-patch series fixes the overflow by widening the relevant
>fields to u64.
>
>Patch 1 converts the unbalanced_load counter to per-cpu state as a
>preparatory step. The counter sits in the transmit hot path, so
>converting it to per-cpu before widening avoids introducing cross-CPU
>synchronization overhead for a u64. Also use a prev_total_unbalanced
>to store the previous total load to avoid reset per-cpu data.
>
>Patch 2 widens tx_bytes, load_history, load, and the per-cpu
>unbalanced_load tx_bytes from u32 to u64. It adds u64_stats_sync
>protection for the per-cpu counter to prevent tearing on 32-bit
>architectures, and reworks compute_gap() to use s64 arithmetic with
>READ_ONCE() on slave->speed.
>
>Signed-off-by: Hangbin Liu <liuhangbin@kylinos.cn>
For the series, looks good to me.
My only nit is that you don't state why you use READ_ONCE for
the access of ->speed, and I didn't see it in the prior discussion
(perhaps I missed it). Regardless, I presume it's because the access
could race with a concurrent update in bond_update_speed_duplex.
Acked-by: Jay Vosburgh <jv@jvosburgh.net>
-J
>---
>Changes in v6:
>- Do not return fixed value for unknown speed logic, we need load balance
> even NIC speed is UNKNOWN (sashiko)
>- Link to v5: https://lore.kernel.org/r/20260825-bond_overflow-v5-0-7a800de133f1@kylinos.cn
>
>Changes in v5:
>- use s64 as the return value for compute_gap, so we can compute the
> difference when NICs are overload. (Nikolay Aleksandrov)
>- Link to v4: https://lore.kernel.org/r/20260820-bond_overflow-v4-0-805ba0d3efb6@kylinos.cn
>
>Changes in v4:
>- move per-cpu allocation to tlb_initialize/tlb_deinitialize (Nikolay Aleksandrov)
>- use an extra prev_total_unbalanced to avoid reset per-cpu data (Nikolay Aleksandrov)
>- Link to v3: https://lore.kernel.org/r/20260818-bond_overflow-v3-0-e05d4dbc2fd8@kylinos.cn
>
>Changes in v3:
>- Add a preparatory patch to convert unbalanced_load to per-cpu first
>- widens tlb counters to u64 and add helpers to prevent tearing on 32-bit
>- Link to v2: https://lore.kernel.org/r/20260814-bond_overflow-v2-1-d3fe588ad167@kylinos.cn
>
>Changes in v2:
>- update comment description, including AI-detected info.
>- fix tx_bytes/load type detected by sashiko
>- cast SPEED_UNKNOWN to 0 before shift, detected by sashiko
>- Link to v1: https://lore.kernel.org/r/20260810-bond_overflow-v1-1-c9ff29d76770@kylinos.cn
>
>---
>Hangbin Liu (2):
> bonding: convert unbalanced_load to per-cpu state
> bonding: fix u32 overflow in compute_gap()
>
> drivers/net/bonding/bond_alb.c | 63 +++++++++++++++++++++++++++++++++++-------
> include/net/bond_alb.h | 14 +++++++---
> 2 files changed, 63 insertions(+), 14 deletions(-)
>---
>base-commit: a8455260b2e9c024d1872ac1c094793d55a7e537
>change-id: 20260806-bond_overflow-ac6a6a78d6a0
>
>Best regards,
>--
>Hangbin Liu <liuhangbin@kylinos.cn>
>
---
-Jay Vosburgh, jv@jvosburgh.net
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs
2026-08-31 1:50 [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs Hangbin Liu
` (2 preceding siblings ...)
2026-09-02 1:48 ` [PATCH net v6 0/2] bonding: fix TLB load-tracking overflow on high-speed NICs Jay Vosburgh
@ 2026-09-03 1:40 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 5+ messages in thread
From: patchwork-bot+netdevbpf @ 2026-09-03 1:40 UTC (permalink / raw)
To: Hangbin Liu
Cc: jv, andrew+netdev, davem, edumazet, kuba, pabeni, horms, razor,
netdev, linux-kernel, liuhangbin
Hello:
This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:
On Mon, 31 Aug 2026 09:50:58 +0800 you wrote:
> The bonding TLB (Transmit Load Balancing) mode tracks per-slave and
> per-client transmit byte counts in u32 fields. At sustained throughput
> above ~3.2 Gbit/s over the 10-second rebalance interval these counters
> wrap, causing compute_gap() to produce incorrect gap values and
> mis-select transmit slaves. Such speeds are routine on modern NICs
> under heavy traffic.
>
> [...]
Here is the summary with links:
- [net,v6,1/2] bonding: convert unbalanced_load to per-cpu state
https://git.kernel.org/netdev/net-next/c/e14761563588
- [net,v6,2/2] bonding: fix u32 overflow in compute_gap()
https://git.kernel.org/netdev/net-next/c/4cef95f72bbd
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 5+ messages in thread