From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C59425C6EE for ; Thu, 11 Jun 2026 11:21:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781176914; cv=none; b=u2Y15u06Uk7qGJn3chLSW0PClOJLOmyDq8W5u3bXzFc0Ml/kLCz+Ym6bleIakwUiuoOzl9WZzG1h7basQPiH/dxYAwrjJiChPaFEPAEKTpF87i6oIVgMffmc4X85RAglZJVMobGcMwALhf9O93f/QkmgTMxRcWPMXGoTG4B50tY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781176914; c=relaxed/simple; bh=BnzYBUOnLSiIWSs8RNklNX8J/RRI7TV+qjSBhPXGbfA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=e3B7KCdlibkkDgElMb+SNXDhBdncQ8qHmtZTWyoVE3roMp2YwlEM65cu8lj3RKMHX9qAYRIf5IEa/bVNNSU4b8XIF7ZHASws581dNxlKU25nW1TwjPczgS5uLGQQNrkkTeEZXNIguX39QIEhLP758SqlpgG4hcKCPFf7mi/4d7c= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DDHghERq; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DDHghERq" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-36b9ec98144so6554545a91.1 for ; Thu, 11 Jun 2026 04:21:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781176913; x=1781781713; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=0p9/X+E7tl8Yx7noYo8cCpp7nCnHbvyUjv3ehBjdSfw=; b=DDHghERqIh+Jr6+lCwe5PsBN8eeFKNAz/Gnqt04Y19eHUnE5hL3c1wNH+MtE01vB3+ Cv1PBOwv82j2Ii+KpqmpSjEyP79/Kr/xNgJeEOJB53ilMDgOC/s599ggvvHXJPQDgrM7 ccrbWbccTf31XDxRFLsG9/t0nUpNICxg4OavdtcoX/IM4RkS6wv6ly/6xeT4rLJBzafK NKKZxLGhKXSpfWPOcr+Rkbzi24ZfUh74Bj2BOTzmSextDzrHtib3B3g99hxuiYmkNlK2 k10VyNqKo5QRhtXsJGtot49jnuqam8Kj8AwqawIcphqj9wgpR+KMW/CVAkl3mAUsn/Iu PbkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781176913; x=1781781713; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0p9/X+E7tl8Yx7noYo8cCpp7nCnHbvyUjv3ehBjdSfw=; b=cQVYrh242fCRYmR3PlQ2MMugcCpyqEuZTu/5+XqoPGVwbEJLFRgTz8QgGipRkK0UeU 4TJ+4UQ1R2jQ5volPC6FpbmFiTbzNBpWiqBX2EUKGcSqgVYbI6K1rG3z4sIJs7KQwLgI AKsI+wGLI++jeN5UHzE03SvcOOuDgj8UxFZJ8gCAX0dNHWl6JGQr+C39x+iS00lS5wVe IqXAq+c1e2Rw/BSb6WhRSTAjf25p6XRsBPVqdbaR+JrqNVqzo6i+LWnL+Jt64tWJzttF lb0T8bbeV6plZ0sW7FKkiqH6n7/wOQ65Yxda0/PBKns4omJJHp9YoJd2MYbcILL1QlW1 ni7Q== X-Forwarded-Encrypted: i=1; AFNElJ+4QEekLgo8rr7O8UaoJq+X6EErsBk7Vl58Mslz/KXRpFTy1bFGgoaKg8RWJ09ySAAm6J6iXuxnqfTXudw=@vger.kernel.org X-Gm-Message-State: AOJu0YwHS1Yxs9eTUQFjfe3LXsx9uKNc9LUxJ7MQjRpwtrnppKPIUILV be3zDJ0q4WDw01BMzAyQBVGpQFV0Y/rU1e6MnVHDMts4mEyNd+zFDFQV X-Gm-Gg: Acq92OF9q41/XNOZKNKXmCve6JeteYHJLBYOfvKA54Rlms5rEL0sFNoH5rGQWzbERdk fm2IbBCJT1PR8+t59n+HqEILOfgYtt5OmJDHgLjrQ5rtyBKrnsggm4Bj+RamnE9LdQpgAxUWOHJ P0TmpKZ4Oiw72kdxtJt3pijqcb019e0dJEZ03QpYsJwx3EPrtqvKikEmIb/j1CNBNckG2Tq674N BIHj5X2G5jDz3wpT7jYs1QZy+1KfywNhFqAJC9kDo9+R1WOaCVM2RdgFy6COL4cwQq6WDL+Bp2g Ay0QB7WPsPvEZL351fjeZHxvYjfwdpQGKh9lADA3ZNRD6/OVbXzQJTSeBIBb1C+kJFcdR7DQZ4s u8Ym+ftOvJacb6D1CTOnYltX/zv9HyoVHVoPIlS7Zrv0q22HnSNMzuXj+XieRnZwYUNZwMg5oMz qUPRq5d7JclWmrvLbFS47ncmwuNjgRgs+gPZpmInOA+1I27/zbbKSK X-Received: by 2002:a17:90b:3f50:b0:369:c5f4:9681 with SMTP id 98e67ed59e1d1-377a870982emr2730988a91.22.1781176912529; Thu, 11 Jun 2026 04:21:52 -0700 (PDT) Received: from [192.168.89.2] ([27.232.220.71]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-377522a19eesm2163098a91.1.2026.06.11.04.21.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 11 Jun 2026 04:21:52 -0700 (PDT) Message-ID: <3a89008a-e4ff-4249-80b5-e2caface5642@gmail.com> Date: Thu, 11 Jun 2026 20:21:48 +0900 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH RFC net-next 0/4] bonding: support LAG IPsec offload with replicated SAs To: Leon Romanovsky Cc: netdev@vger.kernel.org, Jay Vosburgh , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Steffen Klassert , Herbert Xu , linux-kernel@vger.kernel.org References: <20260520081004.2232091-1-hurryman2212@gmail.com> <20260610141843.GI327369@unreal> <20260611100632.GL327369@unreal> Content-Language: en-US From: Jihong Min In-Reply-To: <20260611100632.GL327369@unreal> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 6/11/26 19:06, Leon Romanovsky wrote: > On Thu, Jun 11, 2026 at 05:56:17PM +0900, Jihong Min wrote: >> Hi, >> >> On 6/10/26 23:18, Leon Romanovsky wrote: >>> On Wed, May 20, 2026 at 05:10:00PM +0900, Jihong Min wrote: >>>> This RFC adds a bonding model for IPsec/XFRM hardware offload on >>>> 802.3ad and balance-xor LAG devices when the transmit hash policy is >>>> layer3+4. This is an intentional scope limit rather than a hard limit, >>>> as this is the configuration I can test with my gear. >>>> >>>> The main idea is to leave the existing upstream single-lower-device XFRM >>>> offload path for active-backup intentionally untouched, while adding a >>>> replicated state model for LAG. >>>> >>>> For LAG bonds, the bonding driver installs the same XFRM state on every >>>> eligible running slave and stores the per-slave hardware handles in >>>> bonding-private state. Lower drivers that support this model can then >>>> resolve the handle for the concrete lower netdev used by the datapath. >>>> >>>> LAG IPsec features are user controlled. Newly eligible LAG bonds start >>>> with the ESP/XFRM features disabled, but advertise supported mutable >>>> features when all running eligible slaves can support them. Users can >>>> then opt in with ethtool. Feature enable is propagated to the lower >>>> devices and rolled back if a lower device cannot enable the requested >>>> features. >>>> >>>> The series also handles LAG membership and eligibility changes by adding >>>> replicated SAs to newly usable slaves, removing the departing lower >>>> instance on down/remove, and flushing bond-owned XFRM offload state when >>>> the bond leaves the supported mode or hash-policy configuration. >>>> >>>> This series does not convert any physical NIC driver. A lower driver >>>> must explicitly opt in to the replicated-upper-device model before it can >>>> use these bond-owned states in its datapath. >>>> >>>> For example, a driver such as mlx5 would opt in by marking its >>>> xfrmdev_ops and by resolving datapath handles through the helper: >>>> >>>> static const struct xfrmdev_ops mlx5e_ipsec_xfrmdev_ops = { >>>> ... >>>> .xdo_dev_state_lower_handle = NULL, >>>> .flags = XFRMDEV_OPS_F_LOWER_HANDLE, >>>> }; >>>> >>>> handle = xfrm_dev_state_lower_handle(x, netdev); >>>> if (!handle) >>>> goto drop; >>>> >>>> sa_entry = (struct mlx5e_ipsec_sa_entry *)handle; >>> >>> I’m curious how you replicate and maintain the hardware state across these >>> devices. How are you handling the anti-replay window? >>> >>> Thanks >>> >> >> The short answer is that the RFC I sent was not complete enough in this >> area. > > The issue is that you are adding support for crypto offload where the > anti-replay window is managed in software. You must ensure that all SAs can > safely share and update this state. > > Thanks I agree. Generic XFRM crypto/packet offload with bonding enabled is not impossible, but crypto offload needs a different contract from packet offload. If the sequence number and replay window are still owned by the single software xfrm_state, and the lower devices only hold crypto context, then replicated lower handles may be valid. But the RFC did not prove or document that, and it used the same weak opt-in for both CRYPTO and PACKET. For a respin I should either leave XFRM_DEV_OFFLOAD_CRYPTO out of the replicated LAG path at first, or add a separate capability/contract for crypto offload which states that the software xfrm_state safely owns and updates the shared sequence/replay state across all lower instances.