sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

Hi,

When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with 
suitable arguments, the local loopback device in the netns is not 
getting destroyed after deleting the netns.

For example:

ip netns add TEST
ip netns exec TEST ip link set lo up
ip link add dummy0 type dummy
ip link add dummy1 type dummy
ip link add dummy2 type dummy
ip link set dev dummy0 netns TEST
ip link set dev dummy1 netns TEST
ip link set dev dummy2 netns TEST
ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
ip netns exec TEST ip link set dummy0 up
ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
ip netns exec TEST ip link set dummy1 up
ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
ip netns exec TEST ip link set dummy2 up
ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 
20000 -s -B 192.168.1.3
ip netns del TEST

Results to:

[  354.179591] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  364.419674] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  374.663664] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  384.903717] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  395.143724] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  405.383645] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
...

Based on a quick test, 4.14 and 4.15 does not suffer from this, but its 
reproducible e.g. in 4.4.113 and 4.9.75

Any ideas?

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Neil Horman]

On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> Hi,
> 
> When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with
> suitable arguments, the local loopback device in the netns is not getting
> destroyed after deleting the netns.
> 
> For example:
> 
> ip netns add TEST
> ip netns exec TEST ip link set lo up
> ip link add dummy0 type dummy
> ip link add dummy1 type dummy
> ip link add dummy2 type dummy
> ip link set dev dummy0 netns TEST
> ip link set dev dummy1 netns TEST
> ip link set dev dummy2 netns TEST
> ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
> ip netns exec TEST ip link set dummy0 up
> ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
> ip netns exec TEST ip link set dummy1 up
> ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
> ip netns exec TEST ip link set dummy2 up
> ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 20000
> -s -B 192.168.1.3
> ip netns del TEST
> 
> Results to:
> 
> [  354.179591] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  364.419674] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  374.663664] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  384.903717] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  395.143724] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  405.383645] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> ...
> 
> Based on a quick test, 4.14 and 4.15 does not suffer from this, but its
> reproducible e.g. in 4.4.113 and 4.9.75
> 
> Any ideas?
> 
> Tommi
> 
Does the problem occur if you don't set lo up?

Neil

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

On 30.01.2018 17:59, Neil Horman wrote:
> On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
>>
>> ip netns add TEST
>> ip netns exec TEST ip link set lo up
>> ip link add dummy0 type dummy
>> ip link add dummy1 type dummy
>> ip link add dummy2 type dummy
>> ip link set dev dummy0 netns TEST
>> ip link set dev dummy1 netns TEST
>> ip link set dev dummy2 netns TEST
>> ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
>> ip netns exec TEST ip link set dummy0 up
>> ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
>> ip netns exec TEST ip link set dummy1 up
>> ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
>> ip netns exec TEST ip link set dummy2 up
>> ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 20000
>> -s -B 192.168.1.3
>> ip netns del TEST
>>
> Does the problem occur if you don't set lo up?

Still happens after dropping "ip netns exec TEST ip link set lo up".

Omitting "-B 192.168.1.3" from the sctp_test args helps.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Neil Horman]

On Tue, Jan 30, 2018 at 09:24:17PM +0200, Tommi Rantala wrote:
> On 30.01.2018 17:59, Neil Horman wrote:
> > On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> > > 
> > > ip netns add TEST
> > > ip netns exec TEST ip link set lo up
> > > ip link add dummy0 type dummy
> > > ip link add dummy1 type dummy
> > > ip link add dummy2 type dummy
> > > ip link set dev dummy0 netns TEST
> > > ip link set dev dummy1 netns TEST
> > > ip link set dev dummy2 netns TEST
> > > ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
> > > ip netns exec TEST ip link set dummy0 up
> > > ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
> > > ip netns exec TEST ip link set dummy1 up
> > > ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
> > > ip netns exec TEST ip link set dummy2 up
> > > ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 20000
> > > -s -B 192.168.1.3
> > > ip netns del TEST
> > > 
> > Does the problem occur if you don't set lo up?
> 
> Still happens after dropping "ip netns exec TEST ip link set lo up".
> 
> Omitting "-B 192.168.1.3" from the sctp_test args helps.
> 
Thats....wierd.  I'll look at the sctp_test code in the AM
Neil

> Tommi
> 

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

On 30.01.2018 23:03, Neil Horman wrote:
> On Tue, Jan 30, 2018 at 09:24:17PM +0200, Tommi Rantala wrote:
>> On 30.01.2018 17:59, Neil Horman wrote:
>>> On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
>>>>
>>>> ip netns add TEST
>>>> ip netns exec TEST ip link set lo up
>>>> ip link add dummy0 type dummy
>>>> ip link add dummy1 type dummy
>>>> ip link add dummy2 type dummy
>>>> ip link set dev dummy0 netns TEST
>>>> ip link set dev dummy1 netns TEST
>>>> ip link set dev dummy2 netns TEST
>>>> ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
>>>> ip netns exec TEST ip link set dummy0 up
>>>> ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
>>>> ip netns exec TEST ip link set dummy1 up
>>>> ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
>>>> ip netns exec TEST ip link set dummy2 up
>>>> ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 20000
>>>> -s -B 192.168.1.3
>>>> ip netns del TEST
>>>>
>>> Does the problem occur if you don't set lo up?
>>
>> Still happens after dropping "ip netns exec TEST ip link set lo up".
>>
>> Omitting "-B 192.168.1.3" from the sctp_test args helps.
>>
> Thats....wierd.  I'll look at the sctp_test code in the AM

I think there's a problem in the dst refcounting in sctp_v4_get_dst()

There's a dst_entry struct that has >0 refcnt after running the 
testcase, which makes it impossible to delete the loopback device, as 
that dst is never freed.

I'll try to make a patch.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

On 31.01.2018 11:42, Tommi Rantala wrote:
> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
> 
> There's a dst_entry struct that has >0 refcnt after running the 
> testcase, which makes it impossible to delete the loopback device, as 
> that dst is never freed.
> 
> I'll try to make a patch.
> 
> Tommi
> 

Hi,

This fixes the problem for me, does it look good?

Tested it briefly on 4.4.113 and 4.15-rc9.


 From b94c037d27e36a3053e6862b7e7da5f07f2f5597 Mon Sep 17 00:00:00 2001
From: Tommi Rantala <tommi.t.rantala@nokia.com>
Date: Wed, 31 Jan 2018 11:24:31 +0000
Subject: [PATCH] sctp: fix dst leak in sctp_v4_get_dst

Fix dst reference leak in sctp_v4_get_dst() introduced in commit
410f03831 ("sctp: add routing output fallback"):

When walking the address_list, successive ip_route_output_key() calls
may return the same rt->dst with the reference incremented on each call.

The code would not decrement the dst refcount when the dst pointer was
identical from the previous iteration, causing the dst leak.

Testcase:
   ip netns add TEST
   ip netns exec TEST ip link set lo up
   ip link add dummy0 type dummy
   ip link add dummy1 type dummy
   ip link add dummy2 type dummy
   ip link set dev dummy0 netns TEST
   ip link set dev dummy1 netns TEST
   ip link set dev dummy2 netns TEST
   ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
   ip netns exec TEST ip link set dummy0 up
   ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
   ip netns exec TEST ip link set dummy1 up
   ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
   ip netns exec TEST ip link set dummy2 up
   ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 
-p 20000 -s -B 192.168.1.3
   ip netns del TEST

In 4.4 and 4.9 kernels this results to:
   [  354.179591] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
   [  364.419674] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
   [  374.663664] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
   [  384.903717] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
   [  395.143724] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
   [  405.383645] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
   ...

Fixes: 410f03831 ("sctp: add routing output fallback")
Signed-off-by: Tommi Rantala <tommi.t.rantala@nokia.com>
---
  net/sctp/protocol.c | 9 ++++-----
  1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c
index 8b4ff315695e..aadb9e7f60e2 100644
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -508,21 +508,20 @@ static void sctp_v4_get_dst(struct sctp_transport 
*t, union sctp_addr *saddr,
  		if (IS_ERR(rt))
  			continue;

-		if (!dst)
-			dst = &rt->dst;
-
  		/* Ensure the src address belongs to the output
  		 * interface.
  		 */
  		odev = __ip_dev_find(sock_net(sk), laddr->a.v4.sin_addr.s_addr,
  				     false);
  		if (!odev || odev->ifindex != fl4->flowi4_oif) {
-			if (&rt->dst != dst)
+			if (!dst)
+				dst = &rt->dst;
+			else
  				dst_release(&rt->dst);
  			continue;
  		}

-		if (dst != &rt->dst)
+		if (dst && dst != &rt->dst)
  			dst_release(dst);
  		dst = &rt->dst;
  		break;
-- 
2.15.1

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Neil Horman]

On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
> On 30.01.2018 23:03, Neil Horman wrote:
> > On Tue, Jan 30, 2018 at 09:24:17PM +0200, Tommi Rantala wrote:
> > > On 30.01.2018 17:59, Neil Horman wrote:
> > > > On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> > > > > 
> > > > > ip netns add TEST
> > > > > ip netns exec TEST ip link set lo up
> > > > > ip link add dummy0 type dummy
> > > > > ip link add dummy1 type dummy
> > > > > ip link add dummy2 type dummy
> > > > > ip link set dev dummy0 netns TEST
> > > > > ip link set dev dummy1 netns TEST
> > > > > ip link set dev dummy2 netns TEST
> > > > > ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
> > > > > ip netns exec TEST ip link set dummy0 up
> > > > > ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
> > > > > ip netns exec TEST ip link set dummy1 up
> > > > > ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
> > > > > ip netns exec TEST ip link set dummy2 up
> > > > > ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 20000
> > > > > -s -B 192.168.1.3
> > > > > ip netns del TEST
> > > > > 
> > > > Does the problem occur if you don't set lo up?
> > > 
> > > Still happens after dropping "ip netns exec TEST ip link set lo up".
> > > 
> > > Omitting "-B 192.168.1.3" from the sctp_test args helps.
> > > 
> > Thats....wierd.  I'll look at the sctp_test code in the AM
> 
> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
> 
> There's a dst_entry struct that has >0 refcnt after running the testcase,
> which makes it impossible to delete the loopback device, as that dst is
> never freed.
> 
> I'll try to make a patch.
> 
Are you looking at the second for loop there, which uses ip_route_output_key,
but discards the result if dst is already set?  That does look a bit wonky, and
the same problem may exist in the ipv6 path.  Let me know what the result is.
Neil

> Tommi
> 
> 

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

On 31.01.2018 14:31, Neil Horman wrote:
> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
>> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
>>
>> There's a dst_entry struct that has >0 refcnt after running the testcase,
>> which makes it impossible to delete the loopback device, as that dst is
>> never freed.
>>
>> I'll try to make a patch.
>>
> Are you looking at the second for loop there, which uses ip_route_output_key,
> but discards the result if dst is already set?  That does look a bit wonky, and
> the same problem may exist in the ipv6 path.  Let me know what the result is.

Yes, that was it. Did you receive the email I sent with the patch?
(I'm not seeing that message e.g. at spinics.net linux-sctp archive, so 
just wondering if that email got lost somehow...)

I'll check the ipv6 case, did not try it yet.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

2018-01-31 19:51 GMT+02:00 Tommi Rantala <tommi.t.rantala@nokia.com>:
> On 31.01.2018 14:31, Neil Horman wrote:
>>
>> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
>>>
>>> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
>>>
>>> There's a dst_entry struct that has >0 refcnt after running the testcase,
>>> which makes it impossible to delete the loopback device, as that dst is
>>> never freed.
>>>
>>> I'll try to make a patch.
>>>
>> Are you looking at the second for loop there, which uses
>> ip_route_output_key,
>> but discards the result if dst is already set?  That does look a bit
>> wonky, and
>> the same problem may exist in the ipv6 path.  Let me know what the result
>> is.
>
>
> Yes, that was it. Did you receive the email I sent with the patch?
> (I'm not seeing that message e.g. at spinics.net linux-sctp archive, so just
> wondering if that email got lost somehow...)
>
> I'll check the ipv6 case, did not try it yet.
>
> Tommi

As far as I can tell, the ipv6 code does not suffer from this.
The dst handling there looks good to me.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Neil Horman]

On Thu, Feb 01, 2018 at 08:02:07PM +0200, Tommi Rantala wrote:
> 2018-01-31 19:51 GMT+02:00 Tommi Rantala <tommi.t.rantala@nokia.com>:
> > On 31.01.2018 14:31, Neil Horman wrote:
> >>
> >> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
> >>>
> >>> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
> >>>
> >>> There's a dst_entry struct that has >0 refcnt after running the testcase,
> >>> which makes it impossible to delete the loopback device, as that dst is
> >>> never freed.
> >>>
> >>> I'll try to make a patch.
> >>>
> >> Are you looking at the second for loop there, which uses
> >> ip_route_output_key,
> >> but discards the result if dst is already set?  That does look a bit
> >> wonky, and
> >> the same problem may exist in the ipv6 path.  Let me know what the result
> >> is.
> >
> >
> > Yes, that was it. Did you receive the email I sent with the patch?
> > (I'm not seeing that message e.g. at spinics.net linux-sctp archive, so just
> > wondering if that email got lost somehow...)
> >
No, I can't say I saw the patch on the list.  Can you resend it?

Neil

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

2018-02-01 21:23 GMT+02:00 Neil Horman <nhorman@tuxdriver.com>:
> No, I can't say I saw the patch on the list.  Can you resend it?

Here's the patch again, sending from gmail this time.

The previous mail is now also at spinics, dunno what happened.
https://www.spinics.net/lists/linux-sctp/msg07005.html


>From b94c037d27e36a3053e6862b7e7da5f07f2f5597 Mon Sep 17 00:00:00 2001
From: Tommi Rantala <tommi.t.rantala@nokia.com>
Date: Wed, 31 Jan 2018 11:24:31 +0000
Subject: [PATCH] sctp: fix dst leak in sctp_v4_get_dst

Fix dst reference leak in sctp_v4_get_dst() introduced in commit
410f03831 ("sctp: add routing output fallback"):

When walking the address_list, successive ip_route_output_key() calls
may return the same rt->dst with the reference incremented on each call.

The code would not decrement the dst refcount when the dst pointer was
identical from the previous iteration, causing the dst leak.

Testcase:
  ip netns add TEST
  ip netns exec TEST ip link set lo up
  ip link add dummy0 type dummy
  ip link add dummy1 type dummy
  ip link add dummy2 type dummy
  ip link set dev dummy0 netns TEST
  ip link set dev dummy1 netns TEST
  ip link set dev dummy2 netns TEST
  ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
  ip netns exec TEST ip link set dummy0 up
  ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
  ip netns exec TEST ip link set dummy1 up
  ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
  ip netns exec TEST ip link set dummy2 up
  ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1
-p 20000 -s -B 192.168.1.3
  ip netns del TEST

In 4.4 and 4.9 kernels this results to:
  [  354.179591] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  364.419674] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  374.663664] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  384.903717] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  395.143724] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  405.383645] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  ...

Fixes: 410f03831 ("sctp: add routing output fallback")
Signed-off-by: Tommi Rantala <tommi.t.rantala@nokia.com>
---
 net/sctp/protocol.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c
index 8b4ff315695e..aadb9e7f60e2 100644
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -508,21 +508,20 @@ static void sctp_v4_get_dst(struct
sctp_transport *t, union sctp_addr *saddr,
  if (IS_ERR(rt))
  continue;

- if (!dst)
- dst = &rt->dst;
-
  /* Ensure the src address belongs to the output
  * interface.
  */
  odev = __ip_dev_find(sock_net(sk), laddr->a.v4.sin_addr.s_addr,
       false);
  if (!odev || odev->ifindex != fl4->flowi4_oif) {
- if (&rt->dst != dst)
+ if (!dst)
+ dst = &rt->dst;
+ else
  dst_release(&rt->dst);
  continue;
  }

- if (dst != &rt->dst)
+ if (dst && dst != &rt->dst)
  dst_release(dst);
  dst = &rt->dst;
  break;
-- 
2.15.1

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Neil Horman]

On Thu, Feb 01, 2018 at 10:41:27PM +0200, Tommi Rantala wrote:
> 2018-02-01 21:23 GMT+02:00 Neil Horman <nhorman@tuxdriver.com>:
> > No, I can't say I saw the patch on the list.  Can you resend it?
> 
> Here's the patch again, sending from gmail this time.
> 
> The previous mail is now also at spinics, dunno what happened.
> https://www.spinics.net/lists/linux-sctp/msg07005.html
> 
> 
> From b94c037d27e36a3053e6862b7e7da5f07f2f5597 Mon Sep 17 00:00:00 2001
> From: Tommi Rantala <tommi.t.rantala@nokia.com>
> Date: Wed, 31 Jan 2018 11:24:31 +0000
> Subject: [PATCH] sctp: fix dst leak in sctp_v4_get_dst
> 
> Fix dst reference leak in sctp_v4_get_dst() introduced in commit
> 410f03831 ("sctp: add routing output fallback"):
> 
> When walking the address_list, successive ip_route_output_key() calls
> may return the same rt->dst with the reference incremented on each call.
> 
> The code would not decrement the dst refcount when the dst pointer was
> identical from the previous iteration, causing the dst leak.
> 
> Testcase:
>   ip netns add TEST
>   ip netns exec TEST ip link set lo up
>   ip link add dummy0 type dummy
>   ip link add dummy1 type dummy
>   ip link add dummy2 type dummy
>   ip link set dev dummy0 netns TEST
>   ip link set dev dummy1 netns TEST
>   ip link set dev dummy2 netns TEST
>   ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
>   ip netns exec TEST ip link set dummy0 up
>   ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
>   ip netns exec TEST ip link set dummy1 up
>   ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
>   ip netns exec TEST ip link set dummy2 up
>   ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1
> -p 20000 -s -B 192.168.1.3
>   ip netns del TEST
> 
Patch looks good, but if you could please submit it with the proper title in a
separate thread so it gets davem's attention properly, I'd appreciate it.
Additional points if you update it to include the ipv6 fixes :)
Neil

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

On 02.02.2018 14:34, Neil Horman wrote:
> Patch looks good, but if you could please submit it with the proper title in a
> separate thread so it gets davem's attention properly, I'd appreciate it.
> Additional points if you update it to include the ipv6 fixes :)

Thanks, I'll send this patch properly.

I think I will let Alexey post the ipv6 fixes separately, as he's also 
planning another fix to sctp_v6_get_dst.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Alexey Kodanev]

On 01.02.2018 21:02, Tommi Rantala wrote:
> 2018-01-31 19:51 GMT+02:00 Tommi Rantala <tommi.t.rantala@nokia.com>:
>> On 31.01.2018 14:31, Neil Horman wrote:
>>>
>>> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
>>>>
>>>> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
>>>>
>>>> There's a dst_entry struct that has >0 refcnt after running the testcase,
>>>> which makes it impossible to delete the loopback device, as that dst is
>>>> never freed.
>>>>
>>>> I'll try to make a patch.
>>>>
>>> Are you looking at the second for loop there, which uses
>>> ip_route_output_key,
>>> but discards the result if dst is already set?  That does look a bit
>>> wonky, and
>>> the same problem may exist in the ipv6 path.  Let me know what the result
>>> is.
>>
>>
>> Yes, that was it. Did you receive the email I sent with the patch?
>> (I'm not seeing that message e.g. at spinics.net linux-sctp archive, so just
>> wondering if that email got lost somehow...)
>>
>> I'll check the ipv6 case, did not try it yet.
>>
>> Tommi
> 
> As far as I can tell, the ipv6 code does not suffer from this.
> The dst handling there looks good to me.
> 

For ipv6 part, shouldn't we release 'bdst' there if the previous address
match is better and we continue to the next iteration?

diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
index 5d4c15b..a044096 100644
--- a/net/sctp/ipv6.c
+++ b/net/sctp/ipv6.c
@@ -336,8 +336,11 @@ static void sctp_v6_get_dst(struct sctp_transport *t, union sctp_addr *saddr,
                }
 
                bmatchlen = sctp_v6_addr_match_len(daddr, &laddr->a);
-               if (matchlen > bmatchlen)
+               if (matchlen > bmatchlen) {
+                       if (!IS_ERR(bdst))
+                               dst_release(bdst);
                        continue;
+               }
 
                if (!IS_ERR_OR_NULL(dst))
                        dst_release(dst);

Thanks,
Alexey

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

2018-02-02 1:57 GMT+02:00 Alexey Kodanev <alexey.kodanev@oracle.com>:
> For ipv6 part, shouldn't we release 'bdst' there if the previous address
> match is better and we continue to the next iteration?

Good catch!

Didn't see that one.

Tommi

> diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
> index 5d4c15b..a044096 100644
> --- a/net/sctp/ipv6.c
> +++ b/net/sctp/ipv6.c
> @@ -336,8 +336,11 @@ static void sctp_v6_get_dst(struct sctp_transport *t, union sctp_addr *saddr,
>                 }
>
>                 bmatchlen = sctp_v6_addr_match_len(daddr, &laddr->a);
> -               if (matchlen > bmatchlen)
> +               if (matchlen > bmatchlen) {
> +                       if (!IS_ERR(bdst))
> +                               dst_release(bdst);
>                         continue;
> +               }
>
>                 if (!IS_ERR_OR_NULL(dst))
>                         dst_release(dst);
>
> Thanks,
> Alexey

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Alexey Kodanev]

On 02.02.2018 11:27, Tommi Rantala wrote:
> 2018-02-02 1:57 GMT+02:00 Alexey Kodanev <alexey.kodanev@oracle.com>:
>> For ipv6 part, shouldn't we release 'bdst' there if the previous address
>> match is better and we continue to the next iteration?
> 
> Good catch!
> 

On the second thought, I think, we should also check 'bdst' ptr for
the error earlier, i.e. right after 'bdst = ip6_dst_lookup_flow(...)'.
I'll prepare the patch.

Thanks,
Alexey


> Didn't see that one.
> 
> Tommi
> 
>> diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
>> index 5d4c15b..a044096 100644
>> --- a/net/sctp/ipv6.c
>> +++ b/net/sctp/ipv6.c
>> @@ -336,8 +336,11 @@ static void sctp_v6_get_dst(struct sctp_transport *t, union sctp_addr *saddr,
>>                 }
>>
>>                 bmatchlen = sctp_v6_addr_match_len(daddr, &laddr->a);
>> -               if (matchlen > bmatchlen)
>> +               if (matchlen > bmatchlen) {
>> +                       if (!IS_ERR(bdst))
>> +                               dst_release(bdst);
>>                         continue;
>> +               }
>>
>>                 if (!IS_ERR_OR_NULL(dst))
>>                         dst_release(dst);
>>
>> Thanks,
>> Alexey

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Marcelo Ricardo Leitner]

On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> Hi,
> 
> When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with
> suitable arguments, the local loopback device in the netns is not getting
> destroyed after deleting the netns.
> 
...
> 
> Based on a quick test, 4.14 and 4.15 does not suffer from this, but its
> reproducible e.g. in 4.4.113 and 4.9.75
> 
> Any ideas?

By the versions you mentioned and report, maybe f186ce61bb82 ("Fix an
intermittent pr_emerg warning about lo becoming free.") fixed it.

  Marcelo

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

[Tommi Rantala]

On 31.01.2018 00:52, Marcelo Ricardo Leitner wrote:
> On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
>> Hi,
>>
>> When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with
>> suitable arguments, the local loopback device in the netns is not getting
>> destroyed after deleting the netns.
>>
> ...
>>
>> Based on a quick test, 4.14 and 4.15 does not suffer from this, but its
>> reproducible e.g. in 4.4.113 and 4.9.75
>>
>> Any ideas?
> 
> By the versions you mentioned and report, maybe f186ce61bb82 ("Fix an
> intermittent pr_emerg warning about lo becoming free.") fixed it.

Thanks, found that commit too, but it does not help. It has been 
included 4.4.76 already.

Also the warning is not intermittent, but permanent...

It makes it impossible to add another netns, in 4.4 "ip netns add" gets 
stuck in uninterruptible sleep:

$ grep State /proc/$(pidof ip)/status
State:  D (disk sleep)

$ cat /proc/$(pidof ip)/stack
[<ffffffff8249c342>] copy_net_ns+0x72/0x100
[<ffffffff82075ee9>] create_new_namespaces+0xf9/0x1b0
[<ffffffff82076141>] unshare_nsproxy_namespaces+0x61/0xa0
[<ffffffff82056c1b>] SyS_unshare+0x1ab/0x2e0
[<ffffffff825a266e>] entry_SYSCALL_64_fastpath+0x12/0x6d
[<ffffffffffffffff>] 0xffffffffffffffff


Tommi