Re: [PATCH net 0/2] bond: fix xfrm offload feature during init

Jianbo Liu
2025-01-09T09:51:38Z

On 1/9/2025 4:37 PM, Hangbin Liu wrote:
> On Thu, Jan 09, 2025 at 09:26:38AM +0800, Jianbo Liu wrote:
>>
>>
>> On 1/8/2025 3:14 PM, Hangbin Liu wrote:
>>> On Wed, Jan 08, 2025 at 11:40:05AM +0800, Jianbo Liu wrote:
>>>>
>>>>
>>>> On 1/8/2025 10:46 AM, Hangbin Liu wrote:
>>>>> On Mon, Jan 06, 2025 at 10:47:16AM +0000, Hangbin Liu wrote:
>>>>>> On Thu, Jan 02, 2025 at 11:33:34AM +0800, Jianbo Liu wrote:
>>>>>>>>> Re-locking doesn't look great, glancing at the code I don't see any
>>>>>>>>> obvious better workarounds. Easiest fix would be to don't let the
>>>>>>>>> drivers sleep in the callbacks and then we can go back to a spin lock.
>>>>>>>>> Maybe nvidia people have better ideas, I'm not familiar with this
>>>>>>>>> offload.
>>>>>>>>
>>>>>>>> I don't know how to disable bonding sleeping since we use mutex_lock now.
>>>>>>>> Hi Jianbo, do you have any idea?
>>>>>>>>
>>>>>>>
>>>>>>> I think we should allow drivers to sleep in the callbacks. So, maybe it's
>>>>>>> better to move driver's xdo_dev_state_delete out of state's spin lock.
>>>>>>
>>>>>> I just check the code, xfrm_dev_state_delete() and later
>>>>>> dev->xfrmdev_ops->xdo_dev_state_delete(x) have too many xfrm_state x
>>>>>> checks. Can we really move it out of spin lock from xfrm_state_delete()
>>>>>
>>>>> I tried to move the mutex lock code to a work queue, but found we need to
>>>>> check (ipsec->xs == xs) in bonding. So we still need xfrm_state x during bond
>>>>
>>>> Maybe I miss something, but why need to hold spin lock. You can keep xfrm
>>>> state by its refcnt.
>>>
>>> Do you mean move the xfrm_dev_state_delete() out of spin lock directly like:
>>>
>>
>> Yes. Not feasible?
>>
>>> diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
>>> index 67ca7ac955a3..6881ddeb4360 100644
>>> --- a/net/xfrm/xfrm_state.c
>>> +++ b/net/xfrm/xfrm_state.c
>>> @@ -766,13 +766,6 @@ int __xfrm_state_delete(struct xfrm_state *x)
>>> if (x->encap_sk)
>>> sock_put(rcu_dereference_raw(x->encap_sk));
>>> - xfrm_dev_state_delete(x);
>>> -
>>> - /* All xfrm_state objects are created by xfrm_state_alloc.
>>> - * The xfrm_state_alloc call gives a reference, and that
>>> - * is what we are dropping here.
>>> - */
>>> - xfrm_state_put(x);
>>> err = 0;
>>> }
>>> @@ -787,8 +780,20 @@ int xfrm_state_delete(struct xfrm_state *x)
>>> spin_lock_bh(&x->lock);
>>> err = __xfrm_state_delete(x);
>>> spin_unlock_bh(&x->lock);
>>> + if (err)
>>> + return err;
>>> - return err;
>>> + if (x->km.state == XFRM_STATE_DEAD) {
>>> + xfrm_dev_state_delete(x);
>>> +
>>> + /* All xfrm_state objects are created by xfrm_state_alloc.
>>> + * The xfrm_state_alloc call gives a reference, and that
>>> + * is what we are dropping here.
>>> + */
>>> + xfrm_state_put(x);
>>> + }
>>> +
>>> + return 0;
>>> }
>>> EXPORT_SYMBOL(xfrm_state_delete);
>>>
>>> Then why we need the spin lock in xfrm_state_delete?
>>>
>>
>> No, we don't need. But I am trying to understand what you said in your last
>> email about adding a new lock, or unlocking spin lock in
>
> I *thought* we need the spin lock in xfrm_state_delete(). So to protect xfrm_state,

But not need in bond_ipsec_del_sa() because the state still hold by
xfrm_state_hold(), right?

> we need a new lock. Although it looks redundant. e.g.
>
> int xfrm_state_delete(struct xfrm_state *x)
> {
> int err;
>
> spin_lock_bh(&x->lock);
> err = __xfrm_state_delete(x);
> spin_unlock_bh(&x->lock);
> if (err)
> return err;
>
> another_lock(&x->other_lock)
> if (x->km.state == XFRM_STATE_DEAD) {
> xfrm_dev_state_delete(x);
> xfrm_state_put(x);
> }
> another_unlock(&x->other_lock)
>
> return 0;
> }
>> bond_ipsec_del_sa(). Anything I missed?
>
> The unlock spin lock in bond_ipsec_del_sa looks like
> https://lore.kernel.org/netdev/Z1vfsAyuxcohT7th@fedora/
>
> Thanks
> Hangbin