[PATCH net v2 0/2] af_unix: Fix GC and improve selftest

[PATCH net v2 0/2] af_unix: Fix GC and improve selftest

[Michal Luczaj]

Series deals with AF_UNIX garbage collector mishandling some in-flight
graph cycles. Embryos carrying OOB packets with SCM_RIGHTS cause issues.

Patch 1/2 fixes the memory leak.
Patch 2/2 tweaks the selftest for a better OOB coverage.

v2:
  - Patch 1/2: remove WARN_ON_ONCE() (Kuniyuki)
  - Combine both patches into a series (Kuniyuki)

v1: https://lore.kernel.org/netdev/20240516103049.1132040-1-mhal@rbox.co/

Kuniyuki Iwashima (1):
  selftest: af_unix: Make SCM_RIGHTS into OOB data.

Michal Luczaj (1):
  af_unix: Fix garbage collection of embryos carrying OOB with
    SCM_RIGHTS

 net/unix/garbage.c                            | 23 +++++++++++--------
 .../selftests/net/af_unix/scm_rights.c        |  4 ++--
 2 files changed, 16 insertions(+), 11 deletions(-)

-- 
2.45.0

[PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Michal Luczaj]

GC attempts to explicitly drop oob_skb before purging the hit list.

The problem is with embryos: kfree_skb(u->oob_skb) is never called on an
embryo socket, as those sockets are not directly stacked by the SCC walk.

The python script below [0] sends a listener's fd to its embryo as OOB
data.  While GC does collect the embryo's queue, it fails to drop the OOB
skb's refcount.  The skb which was in embryo's receive queue stays as
unix_sk(sk)->oob_skb and keeps the listener's refcount [1].

Tell GC to dispose embryo's oob_skb.

[0]:
from array import array
from socket import *

addr = '\x00unix-oob'
lis = socket(AF_UNIX, SOCK_STREAM)
lis.bind(addr)
lis.listen(1)

s = socket(AF_UNIX, SOCK_STREAM)
s.connect(addr)
scm = (SOL_SOCKET, SCM_RIGHTS, array('i', [lis.fileno()]))
s.sendmsg([b'x'], [scm], MSG_OOB)
lis.close()

[1]
$ grep unix-oob /proc/net/unix
$ ./unix-oob.py
$ grep unix-oob /proc/net/unix
0000000000000000: 00000002 00000000 00000000 0001 02     0 @unix-oob
0000000000000000: 00000002 00000000 00010000 0001 01  6072 @unix-oob

Fixes: 4090fa373f0e ("af_unix: Replace garbage collection algorithm.")
Signed-off-by: Michal Luczaj <mhal@rbox.co>
---
 net/unix/garbage.c | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/net/unix/garbage.c b/net/unix/garbage.c
index 1f8b8cdfcdc8..dfe94a90ece4 100644
--- a/net/unix/garbage.c
+++ b/net/unix/garbage.c
@@ -342,6 +342,18 @@ enum unix_recv_queue_lock_class {
 	U_RECVQ_LOCK_EMBRYO,
 };
 
+static void unix_collect_queue(struct unix_sock *u, struct sk_buff_head *hitlist)
+{
+	skb_queue_splice_init(&u->sk.sk_receive_queue, hitlist);
+
+#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
+	if (u->oob_skb) {
+		WARN_ON_ONCE(skb_unref(u->oob_skb));
+		u->oob_skb = NULL;
+	}
+#endif
+}
+
 static void unix_collect_skb(struct list_head *scc, struct sk_buff_head *hitlist)
 {
 	struct unix_vertex *vertex;
@@ -365,18 +377,11 @@ static void unix_collect_skb(struct list_head *scc, struct sk_buff_head *hitlist
 
 				/* listener -> embryo order, the inversion never happens. */
 				spin_lock_nested(&embryo_queue->lock, U_RECVQ_LOCK_EMBRYO);
-				skb_queue_splice_init(embryo_queue, hitlist);
+				unix_collect_queue(unix_sk(skb->sk), hitlist);
 				spin_unlock(&embryo_queue->lock);
 			}
 		} else {
-			skb_queue_splice_init(queue, hitlist);
-
-#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
-			if (u->oob_skb) {
-				kfree_skb(u->oob_skb);
-				u->oob_skb = NULL;
-			}
-#endif
+			unix_collect_queue(u, hitlist);
 		}
 
 		spin_unlock(&queue->lock);
-- 
2.45.0

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Kuniyuki Iwashima]

From: Michal Luczaj <mhal@rbox.co>
Date: Thu, 16 May 2024 16:50:09 +0200
> GC attempts to explicitly drop oob_skb before purging the hit list.

Sorry for not catching these in v1,

nit: s/oob_skb/oob_skb's reference/

> 
> The problem is with embryos: kfree_skb(u->oob_skb) is never called on an
> embryo socket, as those sockets are not directly stacked by the SCC walk.

", as ..." is not correct and can be just removed.  Here we walk
through embryos as written in the next paragraph but we forget
dropping oob_skb's refcnt.

> 
> The python script below [0] sends a listener's fd to its embryo as OOB
> data.  While GC does collect the embryo's queue, it fails to drop the OOB
> skb's refcount.  The skb which was in embryo's receive queue stays as
> unix_sk(sk)->oob_skb and keeps the listener's refcount [1].
> 
> Tell GC to dispose embryo's oob_skb.
> 
> [0]:
> from array import array
> from socket import *
> 
> addr = '\x00unix-oob'
> lis = socket(AF_UNIX, SOCK_STREAM)
> lis.bind(addr)
> lis.listen(1)
> 
> s = socket(AF_UNIX, SOCK_STREAM)
> s.connect(addr)
> scm = (SOL_SOCKET, SCM_RIGHTS, array('i', [lis.fileno()]))
> s.sendmsg([b'x'], [scm], MSG_OOB)
> lis.close()
> 
> [1]
> $ grep unix-oob /proc/net/unix
> $ ./unix-oob.py
> $ grep unix-oob /proc/net/unix
> 0000000000000000: 00000002 00000000 00000000 0001 02     0 @unix-oob
> 0000000000000000: 00000002 00000000 00010000 0001 01  6072 @unix-oob
> 
> Fixes: 4090fa373f0e ("af_unix: Replace garbage collection algorithm.")
> Signed-off-by: Michal Luczaj <mhal@rbox.co>

with the above corrected, you can add

Reviewed-by: Kuniyuki Iwashima <kuniyu@amazon.com>

Thanks!


> ---
>  net/unix/garbage.c | 23 ++++++++++++++---------
>  1 file changed, 14 insertions(+), 9 deletions(-)
> 
> diff --git a/net/unix/garbage.c b/net/unix/garbage.c
> index 1f8b8cdfcdc8..dfe94a90ece4 100644
> --- a/net/unix/garbage.c
> +++ b/net/unix/garbage.c
> @@ -342,6 +342,18 @@ enum unix_recv_queue_lock_class {
>  	U_RECVQ_LOCK_EMBRYO,
>  };
>  
> +static void unix_collect_queue(struct unix_sock *u, struct sk_buff_head *hitlist)
> +{
> +	skb_queue_splice_init(&u->sk.sk_receive_queue, hitlist);
> +
> +#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
> +	if (u->oob_skb) {
> +		WARN_ON_ONCE(skb_unref(u->oob_skb));
> +		u->oob_skb = NULL;
> +	}
> +#endif
> +}
> +
>  static void unix_collect_skb(struct list_head *scc, struct sk_buff_head *hitlist)
>  {
>  	struct unix_vertex *vertex;
> @@ -365,18 +377,11 @@ static void unix_collect_skb(struct list_head *scc, struct sk_buff_head *hitlist
>  
>  				/* listener -> embryo order, the inversion never happens. */
>  				spin_lock_nested(&embryo_queue->lock, U_RECVQ_LOCK_EMBRYO);
> -				skb_queue_splice_init(embryo_queue, hitlist);
> +				unix_collect_queue(unix_sk(skb->sk), hitlist);
>  				spin_unlock(&embryo_queue->lock);
>  			}
>  		} else {
> -			skb_queue_splice_init(queue, hitlist);
> -
> -#if IS_ENABLED(CONFIG_AF_UNIX_OOB)
> -			if (u->oob_skb) {
> -				kfree_skb(u->oob_skb);
> -				u->oob_skb = NULL;
> -			}
> -#endif
> +			unix_collect_queue(u, hitlist);
>  		}
>  
>  		spin_unlock(&queue->lock);
> -- 
> 2.45.0
> 

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Michal Luczaj]

On 5/17/24 03:45, Kuniyuki Iwashima wrote:
> From: Michal Luczaj <mhal@rbox.co>
> Date: Thu, 16 May 2024 16:50:09 +0200
>> GC attempts to explicitly drop oob_skb before purging the hit list.
> 
> Sorry for not catching these in v1,
> 
> nit: s/oob_skb/oob_skb's reference/

Argh, sorry, I've copy-pasted my own misformulation.

>> The problem is with embryos: kfree_skb(u->oob_skb) is never called on an
>> embryo socket, as those sockets are not directly stacked by the SCC walk.
> 
> ", as ..." is not correct and can be just removed.  Here we walk
> through embryos as written in the next paragraph but we forget
> dropping oob_skb's refcnt.

Oh, I agree we walk through embryos. I wrote that embryos are not _stacked_
by the SCC walk, i.e. embryos don't appear on the `vertex_stack`. But I
think you're right, such comment of mine would be incorrect anyway. So,
removing and resending.

>> The python script below [0] sends a listener's fd to its embryo as OOB
>> data.  While GC does collect the embryo's queue, it fails to drop the OOB
>> skb's refcount.  The skb which was in embryo's receive queue stays as
>> unix_sk(sk)->oob_skb and keeps the listener's refcount [1].
>>
>> Tell GC to dispose embryo's oob_skb.
>>
>> [0]:
>> from array import array
>> from socket import *
>>
>> addr = '\x00unix-oob'
>> lis = socket(AF_UNIX, SOCK_STREAM)
>> lis.bind(addr)
>> lis.listen(1)
>>
>> s = socket(AF_UNIX, SOCK_STREAM)
>> s.connect(addr)
>> scm = (SOL_SOCKET, SCM_RIGHTS, array('i', [lis.fileno()]))
>> s.sendmsg([b'x'], [scm], MSG_OOB)
>> lis.close()
>>
>> [1]
>> $ grep unix-oob /proc/net/unix
>> $ ./unix-oob.py
>> $ grep unix-oob /proc/net/unix
>> 0000000000000000: 00000002 00000000 00000000 0001 02     0 @unix-oob
>> 0000000000000000: 00000002 00000000 00010000 0001 01  6072 @unix-oob
>>
>> Fixes: 4090fa373f0e ("af_unix: Replace garbage collection algorithm.")
>> Signed-off-by: Michal Luczaj <mhal@rbox.co>
> 
> with the above corrected, you can add
> 
> Reviewed-by: Kuniyuki Iwashima <kuniyu@amazon.com>
> 
> Thanks!

All right, thank you.

One question: git send-email automatically adds my Signed-off-by to your
patch (patch 2/2 in this series). Should I leave it that way?

>> ...

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Kuniyuki Iwashima]

From: Michal Luczaj <mhal@rbox.co>
Date: Fri, 17 May 2024 07:59:16 +0200
> One question: git send-email automatically adds my Signed-off-by to your
> patch (patch 2/2 in this series). Should I leave it that way?

SOB is usually added by someone who changed the diff or merged it.

I think it would be better not to add it if not intended.  At least
on my laptop, it does not add SOB automatically.

FWIW, my command is like

  git send-email --annotate --cover-letter --thread --no-chain-reply-to \
  --subject-prefix "PATCH v1 net-next" \
  --to "" \
  --cc "" \
  --cc netdev@vger.kernel.org \
  --batch-size 1 --relogin-delay 15 --dry-run HEAD~10

Thanks!

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Michal Luczaj]

On 5/17/24 09:47, Kuniyuki Iwashima wrote:
> From: Michal Luczaj <mhal@rbox.co>
> Date: Fri, 17 May 2024 07:59:16 +0200
>> One question: git send-email automatically adds my Signed-off-by to your
>> patch (patch 2/2 in this series). Should I leave it that way?
> 
> SOB is usually added by someone who changed the diff or merged it.
> 
> I think it would be better not to add it if not intended.  At least
> on my laptop, it does not add SOB automatically.

Sure, I understand. And the problem was that I had format.signOff = true in
.gitconfig. Fixed.

> FWIW, my command is like
> 
>   git send-email --annotate --cover-letter --thread --no-chain-reply-to \
>   --subject-prefix "PATCH v1 net-next" \

maintainer-netdev.rst shows an example with a slightly different order:
"[PATCH net-next v3]". But I guess it doesn't matter?

>   --to "" \
>   --cc "" \
>   --cc netdev@vger.kernel.org \
>   --batch-size 1 --relogin-delay 15 --dry-run HEAD~10
> 
> Thanks!

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Kuniyuki Iwashima]

From: Michal Luczaj <mhal@rbox.co>
Date: Fri, 17 May 2024 10:55:53 +0200
> On 5/17/24 09:47, Kuniyuki Iwashima wrote:
> > From: Michal Luczaj <mhal@rbox.co>
> > Date: Fri, 17 May 2024 07:59:16 +0200
> >> One question: git send-email automatically adds my Signed-off-by to your
> >> patch (patch 2/2 in this series). Should I leave it that way?
> > 
> > SOB is usually added by someone who changed the diff or merged it.
> > 
> > I think it would be better not to add it if not intended.  At least
> > on my laptop, it does not add SOB automatically.
> 
> Sure, I understand. And the problem was that I had format.signOff = true in
> .gitconfig. Fixed.
> 
> > FWIW, my command is like
> > 
> >   git send-email --annotate --cover-letter --thread --no-chain-reply-to \
> >   --subject-prefix "PATCH v1 net-next" \
> 
> maintainer-netdev.rst shows an example with a slightly different order:
> "[PATCH net-next v3]". But I guess it doesn't matter?

It seems patchwork can parse either format. ("net,vX" and "vX,net")

https://patchwork.kernel.org/project/netdevbpf/list/

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Michal Luczaj]

On 5/17/24 11:22, Kuniyuki Iwashima wrote:
> From: Michal Luczaj <mhal@rbox.co>
> Date: Fri, 17 May 2024 10:55:53 +0200
>> On 5/17/24 09:47, Kuniyuki Iwashima wrote:
>>> From: Michal Luczaj <mhal@rbox.co>
>>> Date: Fri, 17 May 2024 07:59:16 +0200
>>>> One question: git send-email automatically adds my Signed-off-by to your
>>>> patch (patch 2/2 in this series). Should I leave it that way?
>>>
>>> SOB is usually added by someone who changed the diff or merged it.
>>>
>>> I think it would be better not to add it if not intended.  At least
>>> on my laptop, it does not add SOB automatically.
>>
>> Sure, I understand. And the problem was that I had format.signOff = true in
>> .gitconfig. Fixed.
>>
>>> FWIW, my command is like
>>>
>>>   git send-email --annotate --cover-letter --thread --no-chain-reply-to \
>>>   --subject-prefix "PATCH v1 net-next" \
>>
>> maintainer-netdev.rst shows an example with a slightly different order:
>> "[PATCH net-next v3]". But I guess it doesn't matter?
> 
> It seems patchwork can parse either format. ("net,vX" and "vX,net")
> 
> https://patchwork.kernel.org/project/netdevbpf/list/

OK, v3 sent:
https://lore.kernel.org/netdev/20240517093138.1436323-1-mhal@rbox.co/

Thanks for help and review,
Michal

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Jakub Kicinski]

On Fri, 17 May 2024 16:47:42 +0900 Kuniyuki Iwashima wrote:
> From: Michal Luczaj <mhal@rbox.co>
> Date: Fri, 17 May 2024 07:59:16 +0200
> > One question: git send-email automatically adds my Signed-off-by to your
> > patch (patch 2/2 in this series). Should I leave it that way?  
> 
> SOB is usually added by someone who changed the diff or merged it.
> 
> I think it would be better not to add it if not intended. 

My understanding is that Michal should indeed add his SOB, as he is 
the one submitting the patch now, and from the "certificate of origin"
we need his assurance that the code is indeed released under the
appropriate license.

If you could reply to your own posting with the SoB, Michal, that'd be
enough.

Re: [PATCH net v2 1/2] af_unix: Fix garbage collection of embryos carrying OOB with SCM_RIGHTS

[Michal Luczaj]

On 5/17/24 21:24, Jakub Kicinski wrote:
> On Fri, 17 May 2024 16:47:42 +0900 Kuniyuki Iwashima wrote:
>> From: Michal Luczaj <mhal@rbox.co>
>> Date: Fri, 17 May 2024 07:59:16 +0200
>>> One question: git send-email automatically adds my Signed-off-by to your
>>> patch (patch 2/2 in this series). Should I leave it that way?  
>>
>> SOB is usually added by someone who changed the diff or merged it.
>>
>> I think it would be better not to add it if not intended. 
> 
> My understanding is that Michal should indeed add his SOB, as he is 
> the one submitting the patch now, and from the "certificate of origin"
> we need his assurance that the code is indeed released under the
> appropriate license.
> 
> If you could reply to your own posting with the SoB, Michal, that'd be
> enough.

And there it is:
https://lore.kernel.org/netdev/71ceafc1-269c-44e0-80f0-6f16a1f35f0b@rbox.co/

Thanks,
Michal

[PATCH net v2 2/2] selftest: af_unix: Make SCM_RIGHTS into OOB data.

[Michal Luczaj]

From: Kuniyuki Iwashima <kuniyu@amazon.com>

scm_rights.c covers various test cases for inflight file descriptors
and garbage collector for AF_UNIX sockets.

Currently, SCM_RIGHTS messages are sent with 3-bytes string, and it's
not good for MSG_OOB cases, as SCM_RIGTS cmsg goes with the first 2-bytes,
which is non-OOB data.

Let's send SCM_RIGHTS messages with 1-byte character to pack SCM_RIGHTS
into OOB data.

Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
---
 tools/testing/selftests/net/af_unix/scm_rights.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/af_unix/scm_rights.c b/tools/testing/selftests/net/af_unix/scm_rights.c
index bab606c9f1eb..2bfed46e0b19 100644
--- a/tools/testing/selftests/net/af_unix/scm_rights.c
+++ b/tools/testing/selftests/net/af_unix/scm_rights.c
@@ -197,8 +197,8 @@ void __send_fd(struct __test_metadata *_metadata,
 	       const FIXTURE_VARIANT(scm_rights) *variant,
 	       int inflight, int receiver)
 {
-#define MSG "nop"
-#define MSGLEN 3
+#define MSG "x"
+#define MSGLEN 1
 	struct {
 		struct cmsghdr cmsghdr;
 		int fd[2];
-- 
2.45.0