Re: [PATCH net 0/2] bond: fix xfrm offload feature during init

[Hangbin Liu <liuhangbin@gmail.com>]

On Mon, Jan 20, 2025 at 04:16:49PM +0000, Cosmin Ratiu wrote:
> On Fri, 2025-01-17 at 08:54 +0100, Steffen Klassert wrote:
> > > 
> > > Hi Jianbo,
> > > 
> > > I talked with Sabrina and it looks we can't simply do this. Because
> > > both
> > > xfrm_add_sa_expire() and xfrm_timer_handler() calling
> > > __xfrm_state_delete() under
> > > spin lock. If we move the xfrm_dev_state_delete() out of
> > > __xfrm_state_delete(),
> > > all the places need to be handled correctly.
> > > 
> > > At the same time xfrm_timer_handler() calling
> > > xfrm_dev_state_update_stats before
> > > __xfrm_state_delete(). Should we also take care of it to make sure
> > > the state
> > > change and delete are called at the same time?
> > > 
> > > Hi Steffen, do you have any comments?
> > 
> > Can't you just fix this in bonding? xfrm_timer_handler() can't sleep
> > anyway, even if you remove the spinlock, it is a timer function.
> > 
> 
> I am not sure this can be fixed in bonding given that the
> xdo_dev_state_delete op could, in the general case, sleep while talking
> to the hardware. I don't think it's reasonable to expect devices to
> offload xfrm while the kernel holds a spinlock.
> Bonding just exposed this assumption mismatch because of the mutex that
> was added to replace a spinlock which exhibited the same problem we are
> talking about here.
> 
> Do the dev offload operations need to be synchronous? Couldn't
> __xfrm_state_delete instead schedule a wq to do the dev offload? I saw
> there's already an xfrm_state_gc_task that's invoked to call
> xfrm_dev_state_free, perhaps that could be used to do the delete as
> well?

Yes, I have tried to move the bonding gc work in bond_ipsec_del_sa() to a wq
in https://lore.kernel.org/netdev/Z33nEKg4PxwReUu_@fedora/. i.e. move the
following part out of spin lock via wq.

        mutex_lock(&bond->ipsec_lock);
        list_for_each_entry(ipsec, &bond->ipsec_list, list) {
                if (ipsec->xs == xs) {
                        list_del(&ipsec->list);
                        kfree(ipsec);
                        break;
                }
        }
        mutex_unlock(&bond->ipsec_lock);

But we can see there is an (ipsec->xs == xs). So we still need to make
sure the xs is not released. Can we add a xs reference in bond_ipsec_del_sa()
to achieve this?

Thanks
Hangbin