[PATCH xfrm 0/2] Couple of error unwind fixes to packet offload

[PATCH xfrm 0/2] Couple of error unwind fixes to packet offload

[Leon Romanovsky]

From: Leon Romanovsky <leonro@nvidia.com>

Hi Steffen,

There are two straightforward fixes to XFRM.

Thanks

Leon Romanovsky (2):
  xfrm: release all offloaded policy memory
  xfrm: Fix leak of dev tracker

 net/xfrm/xfrm_device.c | 2 +-
 net/xfrm/xfrm_user.c   | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

-- 
2.40.0

[PATCH xfrm 1/2] xfrm: release all offloaded policy memory

[Leon Romanovsky]

From: Leon Romanovsky <leonro@nvidia.com>

Failure to add offloaded policy will cause to the following
error once user will try to reload driver.

Unregister_netdevice: waiting for eth3 to become free. Usage count = 2

This was caused by xfrm_dev_policy_add() which increments reference
to net_device. That reference was supposed to be decremented
in xfrm_dev_policy_free(). However the latter wasn't called.

 unregister_netdevice: waiting for eth3 to become free. Usage count = 2
 leaked reference.
  xfrm_dev_policy_add+0xff/0x3d0
  xfrm_policy_construct+0x352/0x420
  xfrm_add_policy+0x179/0x320
  xfrm_user_rcv_msg+0x1d2/0x3d0
  netlink_rcv_skb+0xe0/0x210
  xfrm_netlink_rcv+0x45/0x50
  netlink_unicast+0x346/0x490
  netlink_sendmsg+0x3b0/0x6c0
  sock_sendmsg+0x73/0xc0
  sock_write_iter+0x13b/0x1f0
  vfs_write+0x528/0x5d0
  ksys_write+0x120/0x150
  do_syscall_64+0x3d/0x90
  entry_SYSCALL_64_after_hwframe+0x46/0xb0

Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 net/xfrm/xfrm_user.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index d720e163ae6e..0e398a589536 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -1980,6 +1980,7 @@ static int xfrm_add_policy(struct sk_buff *skb, struct nlmsghdr *nlh,
 
 	if (err) {
 		xfrm_dev_policy_delete(xp);
+		xfrm_dev_policy_free(xp);
 		security_xfrm_policy_free(xp->security);
 		kfree(xp);
 		return err;
-- 
2.40.0

Re: [PATCH xfrm 1/2] xfrm: release all offloaded policy memory

[Simon Horman]

On Wed, Apr 19, 2023 at 03:19:07PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
> 
> Failure to add offloaded policy will cause to the following
> error once user will try to reload driver.
> 
> Unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> 
> This was caused by xfrm_dev_policy_add() which increments reference
> to net_device. That reference was supposed to be decremented
> in xfrm_dev_policy_free(). However the latter wasn't called.
> 
>  unregister_netdevice: waiting for eth3 to become free. Usage count = 2
>  leaked reference.
>   xfrm_dev_policy_add+0xff/0x3d0
>   xfrm_policy_construct+0x352/0x420
>   xfrm_add_policy+0x179/0x320
>   xfrm_user_rcv_msg+0x1d2/0x3d0
>   netlink_rcv_skb+0xe0/0x210
>   xfrm_netlink_rcv+0x45/0x50
>   netlink_unicast+0x346/0x490
>   netlink_sendmsg+0x3b0/0x6c0
>   sock_sendmsg+0x73/0xc0
>   sock_write_iter+0x13b/0x1f0
>   vfs_write+0x528/0x5d0
>   ksys_write+0x120/0x150
>   do_syscall_64+0x3d/0x90
>   entry_SYSCALL_64_after_hwframe+0x46/0xb0
> 
> Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>

Reviewed-by: Simon Horman <simon.horman@corigine.com>

Re: [PATCH xfrm 1/2] xfrm: release all offloaded policy memory

[Eric Dumazet]

On Wed, Apr 19, 2023 at 2:19 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> From: Leon Romanovsky <leonro@nvidia.com>
>
> Failure to add offloaded policy will cause to the following
> error once user will try to reload driver.
>
> Unregister_netdevice: waiting for eth3 to become free. Usage count = 2
>
> This was caused by xfrm_dev_policy_add() which increments reference
> to net_device. That reference was supposed to be decremented
> in xfrm_dev_policy_free(). However the latter wasn't called.
>
>  unregister_netdevice: waiting for eth3 to become free. Usage count = 2
>  leaked reference.
>   xfrm_dev_policy_add+0xff/0x3d0
>   xfrm_policy_construct+0x352/0x420
>   xfrm_add_policy+0x179/0x320
>   xfrm_user_rcv_msg+0x1d2/0x3d0
>   netlink_rcv_skb+0xe0/0x210
>   xfrm_netlink_rcv+0x45/0x50
>   netlink_unicast+0x346/0x490
>   netlink_sendmsg+0x3b0/0x6c0
>   sock_sendmsg+0x73/0xc0
>   sock_write_iter+0x13b/0x1f0
>   vfs_write+0x528/0x5d0
>   ksys_write+0x120/0x150
>   do_syscall_64+0x3d/0x90
>   entry_SYSCALL_64_after_hwframe+0x46/0xb0
>
> Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---

Reviewed-by: Eric Dumazet <edumazet@google.com>

While reviewing this patch, I also saw xfrm_dev_policy_add() could use
GFP_KERNEL ?

diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
index bef28c6187ebdd0cfc34c8594aab96ac0b13dd24..508c96c90b3911eb88063ad680c77af2b317c95f
100644
--- a/net/xfrm/xfrm_device.c
+++ b/net/xfrm/xfrm_device.c
@@ -363,7 +363,7 @@ int xfrm_dev_policy_add(struct net *net, struct
xfrm_policy *xp,
        }

        xdo->dev = dev;
-       netdev_tracker_alloc(dev, &xdo->dev_tracker, GFP_ATOMIC);
+       netdev_tracker_alloc(dev, &xdo->dev_tracker, GFP_KERNEL);
        xdo->real_dev = dev;
        xdo->type = XFRM_DEV_OFFLOAD_PACKET;
        switch (dir) {

Re: [PATCH xfrm 1/2] xfrm: release all offloaded policy memory

[Leon Romanovsky]

On Thu, Apr 20, 2023 at 06:51:52PM +0200, Eric Dumazet wrote:
> On Wed, Apr 19, 2023 at 2:19 PM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > From: Leon Romanovsky <leonro@nvidia.com>
> >
> > Failure to add offloaded policy will cause to the following
> > error once user will try to reload driver.
> >
> > Unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> >
> > This was caused by xfrm_dev_policy_add() which increments reference
> > to net_device. That reference was supposed to be decremented
> > in xfrm_dev_policy_free(). However the latter wasn't called.
> >
> >  unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> >  leaked reference.
> >   xfrm_dev_policy_add+0xff/0x3d0
> >   xfrm_policy_construct+0x352/0x420
> >   xfrm_add_policy+0x179/0x320
> >   xfrm_user_rcv_msg+0x1d2/0x3d0
> >   netlink_rcv_skb+0xe0/0x210
> >   xfrm_netlink_rcv+0x45/0x50
> >   netlink_unicast+0x346/0x490
> >   netlink_sendmsg+0x3b0/0x6c0
> >   sock_sendmsg+0x73/0xc0
> >   sock_write_iter+0x13b/0x1f0
> >   vfs_write+0x528/0x5d0
> >   ksys_write+0x120/0x150
> >   do_syscall_64+0x3d/0x90
> >   entry_SYSCALL_64_after_hwframe+0x46/0xb0
> >
> > Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > ---
> 
> Reviewed-by: Eric Dumazet <edumazet@google.com>
> 
> While reviewing this patch, I also saw xfrm_dev_policy_add() could use
> GFP_KERNEL ?

netdev_tracker_alloc(...) line was copied from commit e1b539bd73a7
("xfrm: add net device refcount tracker to struct xfrm_state_offload")

Thanks

> 
> diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
> index bef28c6187ebdd0cfc34c8594aab96ac0b13dd24..508c96c90b3911eb88063ad680c77af2b317c95f
> 100644
> --- a/net/xfrm/xfrm_device.c
> +++ b/net/xfrm/xfrm_device.c
> @@ -363,7 +363,7 @@ int xfrm_dev_policy_add(struct net *net, struct
> xfrm_policy *xp,
>         }
> 
>         xdo->dev = dev;
> -       netdev_tracker_alloc(dev, &xdo->dev_tracker, GFP_ATOMIC);
> +       netdev_tracker_alloc(dev, &xdo->dev_tracker, GFP_KERNEL);
>         xdo->real_dev = dev;
>         xdo->type = XFRM_DEV_OFFLOAD_PACKET;
>         switch (dir) {

Re: [PATCH xfrm 1/2] xfrm: release all offloaded policy memory

[Eric Dumazet]

On Thu, Apr 20, 2023 at 7:05 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> On Thu, Apr 20, 2023 at 06:51:52PM +0200, Eric Dumazet wrote:
> > On Wed, Apr 19, 2023 at 2:19 PM Leon Romanovsky <leon@kernel.org> wrote:
> > >
> > > From: Leon Romanovsky <leonro@nvidia.com>
> > >
> > > Failure to add offloaded policy will cause to the following
> > > error once user will try to reload driver.
> > >
> > > Unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> > >
> > > This was caused by xfrm_dev_policy_add() which increments reference
> > > to net_device. That reference was supposed to be decremented
> > > in xfrm_dev_policy_free(). However the latter wasn't called.
> > >
> > >  unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> > >  leaked reference.
> > >   xfrm_dev_policy_add+0xff/0x3d0
> > >   xfrm_policy_construct+0x352/0x420
> > >   xfrm_add_policy+0x179/0x320
> > >   xfrm_user_rcv_msg+0x1d2/0x3d0
> > >   netlink_rcv_skb+0xe0/0x210
> > >   xfrm_netlink_rcv+0x45/0x50
> > >   netlink_unicast+0x346/0x490
> > >   netlink_sendmsg+0x3b0/0x6c0
> > >   sock_sendmsg+0x73/0xc0
> > >   sock_write_iter+0x13b/0x1f0
> > >   vfs_write+0x528/0x5d0
> > >   ksys_write+0x120/0x150
> > >   do_syscall_64+0x3d/0x90
> > >   entry_SYSCALL_64_after_hwframe+0x46/0xb0
> > >
> > > Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> > > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > > ---
> >
> > Reviewed-by: Eric Dumazet <edumazet@google.com>
> >
> > While reviewing this patch, I also saw xfrm_dev_policy_add() could use
> > GFP_KERNEL ?
>
> netdev_tracker_alloc(...) line was copied from commit e1b539bd73a7
> ("xfrm: add net device refcount tracker to struct xfrm_state_offload")
>
> Thanks

Then I guess Steffen can fix both call sites...

Re: [PATCH xfrm 1/2] xfrm: release all offloaded policy memory

[Steffen Klassert]

On Thu, Apr 20, 2023 at 07:33:14PM +0200, Eric Dumazet wrote:
> On Thu, Apr 20, 2023 at 7:05 PM Leon Romanovsky <leon@kernel.org> wrote:
> >
> > On Thu, Apr 20, 2023 at 06:51:52PM +0200, Eric Dumazet wrote:
> > > On Wed, Apr 19, 2023 at 2:19 PM Leon Romanovsky <leon@kernel.org> wrote:
> > > >
> > > > From: Leon Romanovsky <leonro@nvidia.com>
> > > >
> > > > Failure to add offloaded policy will cause to the following
> > > > error once user will try to reload driver.
> > > >
> > > > Unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> > > >
> > > > This was caused by xfrm_dev_policy_add() which increments reference
> > > > to net_device. That reference was supposed to be decremented
> > > > in xfrm_dev_policy_free(). However the latter wasn't called.
> > > >
> > > >  unregister_netdevice: waiting for eth3 to become free. Usage count = 2
> > > >  leaked reference.
> > > >   xfrm_dev_policy_add+0xff/0x3d0
> > > >   xfrm_policy_construct+0x352/0x420
> > > >   xfrm_add_policy+0x179/0x320
> > > >   xfrm_user_rcv_msg+0x1d2/0x3d0
> > > >   netlink_rcv_skb+0xe0/0x210
> > > >   xfrm_netlink_rcv+0x45/0x50
> > > >   netlink_unicast+0x346/0x490
> > > >   netlink_sendmsg+0x3b0/0x6c0
> > > >   sock_sendmsg+0x73/0xc0
> > > >   sock_write_iter+0x13b/0x1f0
> > > >   vfs_write+0x528/0x5d0
> > > >   ksys_write+0x120/0x150
> > > >   do_syscall_64+0x3d/0x90
> > > >   entry_SYSCALL_64_after_hwframe+0x46/0xb0
> > > >
> > > > Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> > > > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > > > ---
> > >
> > > Reviewed-by: Eric Dumazet <edumazet@google.com>
> > >
> > > While reviewing this patch, I also saw xfrm_dev_policy_add() could use
> > > GFP_KERNEL ?
> >
> > netdev_tracker_alloc(...) line was copied from commit e1b539bd73a7
> > ("xfrm: add net device refcount tracker to struct xfrm_state_offload")
> >
> > Thanks
> 
> Then I guess Steffen can fix both call sites...

Sure, will do a patch.

Thanks!

[PATCH xfrm 2/2] xfrm: Fix leak of dev tracker

[Leon Romanovsky]

From: Leon Romanovsky <leonro@nvidia.com>

At the stage of direction checks, the netdev reference tracker is
already initialized, but released with wrong *_put() call.

Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
 net/xfrm/xfrm_device.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/xfrm/xfrm_device.c b/net/xfrm/xfrm_device.c
index bef28c6187eb..408f5e55744e 100644
--- a/net/xfrm/xfrm_device.c
+++ b/net/xfrm/xfrm_device.c
@@ -378,7 +378,7 @@ int xfrm_dev_policy_add(struct net *net, struct xfrm_policy *xp,
 		break;
 	default:
 		xdo->dev = NULL;
-		dev_put(dev);
+		netdev_put(dev, &xdo->dev_tracker);
 		NL_SET_ERR_MSG(extack, "Unrecognized offload direction");
 		return -EINVAL;
 	}
-- 
2.40.0

Re: [PATCH xfrm 2/2] xfrm: Fix leak of dev tracker

[Simon Horman]

On Wed, Apr 19, 2023 at 03:19:08PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
> 
> At the stage of direction checks, the netdev reference tracker is
> already initialized, but released with wrong *_put() call.
> 
> Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>

Reviewed-by: Simon Horman <simon.horman@corigine.com>

Re: [PATCH xfrm 2/2] xfrm: Fix leak of dev tracker

[Eric Dumazet]

On Wed, Apr 19, 2023 at 2:19 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> From: Leon Romanovsky <leonro@nvidia.com>
>
> At the stage of direction checks, the netdev reference tracker is
> already initialized, but released with wrong *_put() call.
>
> Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>

Reviewed-by: Eric Dumazet <edumazet@google.com>

Re: [PATCH xfrm 0/2] Couple of error unwind fixes to packet offload

[Steffen Klassert]

On Wed, Apr 19, 2023 at 03:19:06PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
> 
> Hi Steffen,
> 
> There are two straightforward fixes to XFRM.
> 
> Thanks
> 
> Leon Romanovsky (2):
>   xfrm: release all offloaded policy memory
>   xfrm: Fix leak of dev tracker

Applied, thanks Leon!