[PATCH net 1/2] net: tls: explicitly disallow disconnect

[PATCH net 1/2] net: tls: explicitly disallow disconnect

[Jakub Kicinski]

syzbot discovered that it can disconnect a TLS socket and then
run into all sort of unexpected corner cases. I have a vague
recollection of Eric pointing this out to us a long time ago.
Supporting disconnect is really hard, for one thing if offload
is enabled we'd need to wait for all packets to be _acked_.
Disconnect is not commonly used, disallow it.

The immediate problem syzbot run into is the warning in the strp,
but that's just the easiest bug to trigger:

  WARNING: CPU: 0 PID: 5834 at net/tls/tls_strp.c:486 tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
  RIP: 0010:tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
  Call Trace:
   <TASK>
   tls_rx_rec_wait+0x280/0xa60 net/tls/tls_sw.c:1363
   tls_sw_recvmsg+0x85c/0x1c30 net/tls/tls_sw.c:2043
   inet6_recvmsg+0x2c9/0x730 net/ipv6/af_inet6.c:678
   sock_recvmsg_nosec net/socket.c:1023 [inline]
   sock_recvmsg+0x109/0x280 net/socket.c:1045
   __sys_recvfrom+0x202/0x380 net/socket.c:2237

Fixes: 3c4d7559159b ("tls: kernel TLS support")
Reported-by: syzbot+b4cd76826045a1eb93c1@syzkaller.appspotmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 net/tls/tls_main.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index cb86b0bf9a53..a3ccb3135e51 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -852,6 +852,11 @@ static int tls_setsockopt(struct sock *sk, int level, int optname,
 	return do_tls_setsockopt(sk, optname, optval, optlen);
 }
 
+static int tls_disconnect(struct sock *sk, int flags)
+{
+	return -EOPNOTSUPP;
+}
+
 struct tls_context *tls_ctx_create(struct sock *sk)
 {
 	struct inet_connection_sock *icsk = inet_csk(sk);
@@ -947,6 +952,7 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG],
 	prot[TLS_BASE][TLS_BASE] = *base;
 	prot[TLS_BASE][TLS_BASE].setsockopt	= tls_setsockopt;
 	prot[TLS_BASE][TLS_BASE].getsockopt	= tls_getsockopt;
+	prot[TLS_BASE][TLS_BASE].disconnect	= tls_disconnect;
 	prot[TLS_BASE][TLS_BASE].close		= tls_sk_proto_close;
 
 	prot[TLS_SW][TLS_BASE] = prot[TLS_BASE][TLS_BASE];
-- 
2.49.0

[PATCH net 2/2] selftests: tls: check that disconnect does nothing

[Jakub Kicinski]

"Inspired" by syzbot test, pre-queue some data, disconnect()
and try to receive(). This used to trigger a warning in TLS's strp.
Now we expect the disconnect() to have almost no effect.

Link: https://lore.kernel.org/67e6be74.050a0220.2f068f.007e.GAE@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 tools/testing/selftests/net/tls.c | 36 +++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c
index 9a85f93c33d8..5ded3b3a7538 100644
--- a/tools/testing/selftests/net/tls.c
+++ b/tools/testing/selftests/net/tls.c
@@ -1753,6 +1753,42 @@ TEST_F(tls_basic, rekey_tx)
 	EXPECT_EQ(memcmp(buf, test_str, send_len), 0);
 }
 
+TEST_F(tls_basic, disconnect)
+{
+	char const *test_str = "test_message";
+	int send_len = strlen(test_str) + 1;
+	struct tls_crypto_info_keys key;
+	struct sockaddr_in addr;
+	char buf[20];
+	int ret;
+
+	if (self->notls)
+		return;
+
+	tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128,
+			     &key, 0);
+
+	ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &key, key.len);
+	ASSERT_EQ(ret, 0);
+
+	/* Pre-queue the data so that setsockopt parses it but doesn't
+	 * dequeue it from the TCP socket. recvmsg would dequeue.
+	 */
+	EXPECT_EQ(send(self->fd, test_str, send_len, 0), send_len);
+
+	ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &key, key.len);
+	ASSERT_EQ(ret, 0);
+
+	addr.sin_family = AF_UNSPEC;
+	addr.sin_addr.s_addr = htonl(INADDR_ANY);
+	addr.sin_port = 0;
+	ret = connect(self->cfd, &addr, sizeof(addr));
+	EXPECT_EQ(ret, -1);
+	EXPECT_EQ(errno, EOPNOTSUPP);
+
+	EXPECT_EQ(recv(self->cfd, buf, send_len, 0), send_len);
+}
+
 TEST_F(tls, rekey)
 {
 	char const *test_str_1 = "test_message_before_rekey";
-- 
2.49.0

Re: [PATCH net 1/2] net: tls: explicitly disallow disconnect

[Eric Dumazet]

On Fri, Apr 4, 2025 at 8:03 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> syzbot discovered that it can disconnect a TLS socket and then
> run into all sort of unexpected corner cases. I have a vague
> recollection of Eric pointing this out to us a long time ago.
> Supporting disconnect is really hard, for one thing if offload
> is enabled we'd need to wait for all packets to be _acked_.
> Disconnect is not commonly used, disallow it.

Indeed.

We have sk->sk_disconnects for protocols that have/want to support disconnect.

Anyone interested would have to look at commit 419ce133ab928ab5 ("tcp:
allow again tcp_disconnect() when threads are waiting")

>
> The immediate problem syzbot run into is the warning in the strp,
> but that's just the easiest bug to trigger:
>
>   WARNING: CPU: 0 PID: 5834 at net/tls/tls_strp.c:486 tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>   RIP: 0010:tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>   Call Trace:
>    <TASK>
>    tls_rx_rec_wait+0x280/0xa60 net/tls/tls_sw.c:1363
>    tls_sw_recvmsg+0x85c/0x1c30 net/tls/tls_sw.c:2043
>    inet6_recvmsg+0x2c9/0x730 net/ipv6/af_inet6.c:678
>    sock_recvmsg_nosec net/socket.c:1023 [inline]
>    sock_recvmsg+0x109/0x280 net/socket.c:1045
>    __sys_recvfrom+0x202/0x380 net/socket.c:2237
>
> Fixes: 3c4d7559159b ("tls: kernel TLS support")
> Reported-by: syzbot+b4cd76826045a1eb93c1@syzkaller.appspotmail.com
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Reviewed-by: Eric Dumazet <edumazet@google.com>

Re: [PATCH net 1/2] net: tls: explicitly disallow disconnect

[Sabrina Dubroca]

2025-04-04, 11:03:33 -0700, Jakub Kicinski wrote:
> syzbot discovered that it can disconnect a TLS socket and then
> run into all sort of unexpected corner cases. I have a vague
> recollection of Eric pointing this out to us a long time ago.
> Supporting disconnect is really hard, for one thing if offload
> is enabled we'd need to wait for all packets to be _acked_.
> Disconnect is not commonly used, disallow it.
> 
> The immediate problem syzbot run into is the warning in the strp,
> but that's just the easiest bug to trigger:
> 
>   WARNING: CPU: 0 PID: 5834 at net/tls/tls_strp.c:486 tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>   RIP: 0010:tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>   Call Trace:
>    <TASK>
>    tls_rx_rec_wait+0x280/0xa60 net/tls/tls_sw.c:1363
>    tls_sw_recvmsg+0x85c/0x1c30 net/tls/tls_sw.c:2043
>    inet6_recvmsg+0x2c9/0x730 net/ipv6/af_inet6.c:678
>    sock_recvmsg_nosec net/socket.c:1023 [inline]
>    sock_recvmsg+0x109/0x280 net/socket.c:1045
>    __sys_recvfrom+0x202/0x380 net/socket.c:2237
> 
> Fixes: 3c4d7559159b ("tls: kernel TLS support")
> Reported-by: syzbot+b4cd76826045a1eb93c1@syzkaller.appspotmail.com
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Reviewed-by: Sabrina Dubroca <sd@queasysnail.net>

(hopefully nobody complains about this. but since it was broken
anyway...)

-- 
Sabrina

Re: [PATCH net 2/2] selftests: tls: check that disconnect does nothing

[Sabrina Dubroca]

2025-04-04, 11:03:34 -0700, Jakub Kicinski wrote:
> "Inspired" by syzbot test, pre-queue some data, disconnect()
> and try to receive(). This used to trigger a warning in TLS's strp.
> Now we expect the disconnect() to have almost no effect.
> 
> Link: https://lore.kernel.org/67e6be74.050a0220.2f068f.007e.GAE@google.com
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Reviewed-by: Sabrina Dubroca <sd@queasysnail.net>

-- 
Sabrina

Re: [PATCH net 1/2] net: tls: explicitly disallow disconnect

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (main)
by Paolo Abeni <pabeni@redhat.com>:

On Fri,  4 Apr 2025 11:03:33 -0700 you wrote:
> syzbot discovered that it can disconnect a TLS socket and then
> run into all sort of unexpected corner cases. I have a vague
> recollection of Eric pointing this out to us a long time ago.
> Supporting disconnect is really hard, for one thing if offload
> is enabled we'd need to wait for all packets to be _acked_.
> Disconnect is not commonly used, disallow it.
> 
> [...]

Here is the summary with links:
  - [net,1/2] net: tls: explicitly disallow disconnect
    https://git.kernel.org/netdev/net/c/5071a1e606b3
  - [net,2/2] selftests: tls: check that disconnect does nothing
    https://git.kernel.org/netdev/net/c/a1328a671e1c

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH net 1/2] net: tls: explicitly disallow disconnect

[Ihor Solodrai]

On 4/4/25 11:03 AM, Jakub Kicinski wrote:
> syzbot discovered that it can disconnect a TLS socket and then
> run into all sort of unexpected corner cases. I have a vague
> recollection of Eric pointing this out to us a long time ago.
> Supporting disconnect is really hard, for one thing if offload
> is enabled we'd need to wait for all packets to be _acked_.
> Disconnect is not commonly used, disallow it.
>
> The immediate problem syzbot run into is the warning in the strp,
> but that's just the easiest bug to trigger:
>
>   WARNING: CPU: 0 PID: 5834 at net/tls/tls_strp.c:486 tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>   RIP: 0010:tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>   Call Trace:
>    <TASK>
>    tls_rx_rec_wait+0x280/0xa60 net/tls/tls_sw.c:1363
>    tls_sw_recvmsg+0x85c/0x1c30 net/tls/tls_sw.c:2043
>    inet6_recvmsg+0x2c9/0x730 net/ipv6/af_inet6.c:678
>    sock_recvmsg_nosec net/socket.c:1023 [inline]
>    sock_recvmsg+0x109/0x280 net/socket.c:1045
>    __sys_recvfrom+0x202/0x380 net/socket.c:2237
>
> Fixes: 3c4d7559159b ("tls: kernel TLS support")
> Reported-by: syzbot+b4cd76826045a1eb93c1@syzkaller.appspotmail.com
> Signed-off-by: Jakub Kicinski <kuba@kernel.org>

Hi everyone.

This patch has broken a BPF selftest and as a result BPF CI:
* https://github.com/kernel-patches/bpf/actions/runs/14458537639
* https://github.com/kernel-patches/bpf/actions/runs/14457178732

The test in question is test_sockmap_ktls_disconnect_after_delete
(tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c) [1].

Since the test is about disconnect use-case, and the patch disallows
it, I assume it's appropriate to simply remove the test?

Please let me know. Thanks.

[1] https://web.git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c#n28

> ---
>  net/tls/tls_main.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
> index cb86b0bf9a53..a3ccb3135e51 100644
> --- a/net/tls/tls_main.c
> +++ b/net/tls/tls_main.c
> @@ -852,6 +852,11 @@ static int tls_setsockopt(struct sock *sk, int level, int optname,
>  	return do_tls_setsockopt(sk, optname, optval, optlen);
>  }
>  
> +static int tls_disconnect(struct sock *sk, int flags)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
>  struct tls_context *tls_ctx_create(struct sock *sk)
>  {
>  	struct inet_connection_sock *icsk = inet_csk(sk);
> @@ -947,6 +952,7 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG],
>  	prot[TLS_BASE][TLS_BASE] = *base;
>  	prot[TLS_BASE][TLS_BASE].setsockopt	= tls_setsockopt;
>  	prot[TLS_BASE][TLS_BASE].getsockopt	= tls_getsockopt;
> +	prot[TLS_BASE][TLS_BASE].disconnect	= tls_disconnect;
>  	prot[TLS_BASE][TLS_BASE].close		= tls_sk_proto_close;
>  
>  	prot[TLS_SW][TLS_BASE] = prot[TLS_BASE][TLS_BASE];

Re: [PATCH net 1/2] net: tls: explicitly disallow disconnect

[Paolo Abeni]

On 4/15/25 5:16 AM, Ihor Solodrai wrote:
> On 4/4/25 11:03 AM, Jakub Kicinski wrote:
>> syzbot discovered that it can disconnect a TLS socket and then
>> run into all sort of unexpected corner cases. I have a vague
>> recollection of Eric pointing this out to us a long time ago.
>> Supporting disconnect is really hard, for one thing if offload
>> is enabled we'd need to wait for all packets to be _acked_.
>> Disconnect is not commonly used, disallow it.
>>
>> The immediate problem syzbot run into is the warning in the strp,
>> but that's just the easiest bug to trigger:
>>
>>   WARNING: CPU: 0 PID: 5834 at net/tls/tls_strp.c:486 tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>>   RIP: 0010:tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
>>   Call Trace:
>>    <TASK>
>>    tls_rx_rec_wait+0x280/0xa60 net/tls/tls_sw.c:1363
>>    tls_sw_recvmsg+0x85c/0x1c30 net/tls/tls_sw.c:2043
>>    inet6_recvmsg+0x2c9/0x730 net/ipv6/af_inet6.c:678
>>    sock_recvmsg_nosec net/socket.c:1023 [inline]
>>    sock_recvmsg+0x109/0x280 net/socket.c:1045
>>    __sys_recvfrom+0x202/0x380 net/socket.c:2237
>>
>> Fixes: 3c4d7559159b ("tls: kernel TLS support")
>> Reported-by: syzbot+b4cd76826045a1eb93c1@syzkaller.appspotmail.com
>> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> 
> Hi everyone.
> 
> This patch has broken a BPF selftest and as a result BPF CI:
> * https://github.com/kernel-patches/bpf/actions/runs/14458537639
> * https://github.com/kernel-patches/bpf/actions/runs/14457178732
> 
> The test in question is test_sockmap_ktls_disconnect_after_delete
> (tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c) [1].
> 
> Since the test is about disconnect use-case, and the patch disallows
> it, I assume it's appropriate to simply remove the test?

Ideally, yes. disconnect() implementation by its own nature error and
race prone, I guess  TLS adds some more spice to it. Unless there is a
real end-user scenario behind it, removing the disconnect()
implementation is by far the best option.

Still the test presence hints at some possible use-case[???]. Was it
created using the plain tcp test cases as a template?

Thanks,

Paolo

Re: [PATCH net 1/2] net: tls: explicitly disallow disconnect

[Jiayuan Chen]

April 15, 2025 at 11:16, "Ihor Solodrai" <ihor.solodrai@linux.dev> wrote:



> 
> On 4/4/25 11:03 AM, Jakub Kicinski wrote:
> 
> > 
> > syzbot discovered that it can disconnect a TLS socket and then
> > 
> >  run into all sort of unexpected corner cases. I have a vague
> > 
> >  recollection of Eric pointing this out to us a long time ago.
> > 
> >  Supporting disconnect is really hard, for one thing if offload
> > 
> >  is enabled we'd need to wait for all packets to be _acked_.
> > 
> >  Disconnect is not commonly used, disallow it.
> > 
> >  The immediate problem syzbot run into is the warning in the strp,
> > 
> >  but that's just the easiest bug to trigger:
> > 
> >  WARNING: CPU: 0 PID: 5834 at net/tls/tls_strp.c:486 tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
> > 
> >  RIP: 0010:tls_strp_msg_load+0x72e/0xa80 net/tls/tls_strp.c:486
> > 
> >  Call Trace:
> > 
> >  <TASK>
> > 
> >  tls_rx_rec_wait+0x280/0xa60 net/tls/tls_sw.c:1363
> > 
> >  tls_sw_recvmsg+0x85c/0x1c30 net/tls/tls_sw.c:2043
> > 
> >  inet6_recvmsg+0x2c9/0x730 net/ipv6/af_inet6.c:678
> > 
> >  sock_recvmsg_nosec net/socket.c:1023 [inline]
> > 
> >  sock_recvmsg+0x109/0x280 net/socket.c:1045
> > 
> >  __sys_recvfrom+0x202/0x380 net/socket.c:2237
> > 
> >  Fixes: 3c4d7559159b ("tls: kernel TLS support")
> > 
> >  Reported-by: syzbot+b4cd76826045a1eb93c1@syzkaller.appspotmail.com
> > 
> >  Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> > 
> 
> Hi everyone.
> 
> This patch has broken a BPF selftest and as a result BPF CI:
> 
> * https://github.com/kernel-patches/bpf/actions/runs/14458537639
> 
> * https://github.com/kernel-patches/bpf/actions/runs/14457178732
> 
> The test in question is test_sockmap_ktls_disconnect_after_delete
> 
> (tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c) [1].
> 
> Since the test is about disconnect use-case, and the patch disallows
> 
> it, I assume it's appropriate to simply remove the test?
> 
> Please let me know. Thanks.
> 
> [1] https://web.git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/tools/testing/selftests/bpf/prog_tests/sockmap_ktls.c#n28
> 
> > 
> > ---
> >  net/tls/tls_main.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >  diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
> >  index cb86b0bf9a53..a3ccb3135e51 100644
> >  --- a/net/tls/tls_main.c
> >  +++ b/net/tls/tls_main.c
> >  @@ -852,6 +852,11 @@ static int tls_setsockopt(struct sock *sk, int level, int optname,
> >  return do_tls_setsockopt(sk, optname, optval, optlen);
> >  }
> > 
> >  
> >  +static int tls_disconnect(struct sock *sk, int flags)
> > 
> >  +{
> >  + return -EOPNOTSUPP;
> >  +}
> >  +
> >  struct tls_context *tls_ctx_create(struct sock *sk)
> >  {
> >  struct inet_connection_sock *icsk = inet_csk(sk);
> > 
> >  @@ -947,6 +952,7 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG],
> > 
> >  prot[TLS_BASE][TLS_BASE] = *base;
> >  prot[TLS_BASE][TLS_BASE].setsockopt = tls_setsockopt;
> >  prot[TLS_BASE][TLS_BASE].getsockopt = tls_getsockopt;
> >  + prot[TLS_BASE][TLS_BASE].disconnect = tls_disconnect;
> >  prot[TLS_BASE][TLS_BASE].close = tls_sk_proto_close;
> > 
> > 
> >  prot[TLS_SW][TLS_BASE] = prot[TLS_BASE][TLS_BASE];
> >
>

The original selftest patch d1ba1204f2ee was to re-produce the endless
loop fiexed by 4da6a196f93b.

sk->sk_prot->unhash
        tcp_bpf_unhash
            sk->sk_prot->unhash
                ...
It's try to use disconnect to trigger unhash handler.
I believe we can remove it and use another selftest
instead later.