[PATCH net v3 0/2] kcm: reject the combination of KCM and TLS

[PATCH net v3 0/2] kcm: reject the combination of KCM and TLS

[Rishikesh Jethwani]

KCM's data_ready callback calls the lower socket's read_sock() inline,
from softirq. With kTLS RX that is tls_sw_read_sock(), which can sleep.
v2 deferred KCM parsing to a workqueue whenever read_sock() is not
tcp_read_sock(), and reworked the strparser workqueue to go with it. As
suggested in review [1], v3 instead rejects the combination, as was
done for sockmap in commit 460e6486617c ("tls: reject the combination
of TLS and sockmap"). It has never worked with kTLS RX: before commit
662fbcec32f4 ("net/tls: implement ->read_sock()") KCM parsed TLS
ciphertext as messages, and since then it sleeps in atomic context.

Patch 1 makes kcm_attach() refuse sockets that have any ULP, and
tls_init() refuse sockets attached to KCM. Patch 2 adds tls selftests
for both orders.

Patch 1 changes user-visible behaviour and is tagged for stable. Its
message calls out the errno change and the one refused setup that
previously passed data: TLS TX-only, in either order.

On the new sock flag: patch 1 adds SOCK_KCM to enum sock_flags, which
leaves one free sk_flags bit on 32-bit. I looked at the alternatives
first:

 - sk_user_data alone cannot be the marker. sunrpc and nvme-tcp set it
   on sockets that tlshd later upgrades to TLS, so it is already set
   when tls_init() runs and tls_init() cannot reject on it.

 - A new SK_USER_DATA_KCM tag, alongside SK_USER_DATA_PSOCK, is not
   safe. The tags live in the low bits of the stored pointer, and bit 3
   is not free: reuseport_array stores &array->ptrs[index] | NOCOPY |
   BPF, where ptrs[] holds 8-byte pointers, so every odd index has bit 3
   set and widening SK_USER_DATA_PTRMASK would corrupt it. It would also
   need masking at each of KCM's raw sk_user_data reads.

 - There is precedent for a flag owned by one subsystem: SOCK_XDP
   (commit e4a2a3048ed9) is set only by tun.

If you'd rather not spend an sk_flags bit on this, the alternative I
would implement is a bool in struct tcp_sock under
IS_ENABLED(CONFIG_AF_KCM), like is_mptcp and syn_smc. KCM only attaches
TCP sockets and the TLS ULP is TCP-only, so both checks can use
tcp_sk(). Happy to respin that way.

Changes in v3:
 - Reject KCM and ULPs in both orders instead of deferring KCM parsing
   to a workqueue.
 - Drop the strparser workqueue patch; it is no longer needed for this
   fix and may be posted to net-next separately.
 - Add selftests.

v1: https://lore.kernel.org/netdev/20261013033920.2069285-1-rjethwani@purestorage.com/
v2: https://lore.kernel.org/netdev/20261016025059.289202-1-rjethwani@purestorage.com/

[1] https://lore.kernel.org/netdev/3108e934-ec00-458b-8720-450554b35b55@linux.dev/

Rishikesh Jethwani (2):
  kcm: reject attaching sockets with a ULP, and TLS on KCM sockets
  selftests: tls: test that TLS and KCM are mutually exclusive

 include/net/sock.h                 |   1 +
 net/kcm/kcmsock.c                  |  12 +++
 net/tls/tls_main.c                 |   6 ++
 tools/testing/selftests/net/config |   1 +
 tools/testing/selftests/net/tls.c  | 144 +++++++++++++++++++++++++++++
 5 files changed, 164 insertions(+)

-- 
2.50.1

[PATCH net v3 1/2] kcm: reject attaching sockets with a ULP, and TLS on KCM sockets

[Rishikesh Jethwani]

KCM and kTLS RX do not work together, in either order.

KCM's psock_data_ready() is the lower socket's data_ready callback. It
can run in softirq context under read_lock_bh(&sk->sk_callback_lock),
and calls strp_data_ready(), which invokes sock->ops->read_sock()
inline. With kTLS RX, that is tls_sw_read_sock(), which can sleep: it
allocates with sk->sk_allocation, waits for async crypto, flushes the
socket backlog, and may wait in tls_rx_reader_acquire().

Found by code review. On an unmodified kernel with
CONFIG_DEBUG_ATOMIC_SLEEP, sending TLS records over loopback to a socket
with software kTLS RX attached to a KCM mux reports:

  BUG: sleeping function called from invalid context at include/linux/sched/mm.h:321
  in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 189, name: ksoftirqd/34
  Call Trace:
   __might_resched
   kmem_cache_alloc_node_noprof
   __alloc_skb
   alloc_skb_with_frags
   tls_rx_one_record
   tls_sw_read_sock
   strp_read_sock
   strp_data_ready
   psock_data_ready [kcm]

Before commit 662fbcec32f4 ("net/tls: implement ->read_sock()"), TLS
inherited tcp_read_sock(), so KCM parsed the ciphertext in the TCP
receive queue as messages. HW TLS RX uses the same proto_ops as SW
RX, so it is affected the same way.

The two also stack socket callbacks and neither restores the other's:
with TLS RX configured after the attach, kcm_unattach() overwrites
tls_data_ready(), and releasing TLS RX later reinstalls
psock_data_ready() on a socket no longer attached to KCM. On TX,
kcm_unattach() restores its saved sk_write_space, dropping
tls_write_space().

There is no known user of this combination, so reject it, as was done
for sockmap in commit 460e6486617c ("tls: reject the combination of TLS
and sockmap"):

 - kcm_attach() refuses sockets that already have any ULP, not only
   TLS, as sk_psock_init() does for sockmap, with -EOPNOTSUPP like its
   other refusals;
 - tls_init() refuses sockets attached to KCM with -EINVAL, like
   __tcp_set_ulp() and the sockmap check, tracked by a new SOCK_KCM
   flag set on attach and cleared on unattach.

sk_user_data cannot serve as the marker: sunrpc and nvme-tcp set it on
sockets that tlshd later upgrades to TLS, so it is already set when
tls_init() runs. Unlike sockmap, both sides run under the lower
socket's lock (tcp_set_ulp() asserts this), so the check is made at ULP
install rather than at key configuration.

This is a user-visible change that will also reach stable kernels:
attaching a socket with a ULP now fails with -EOPNOTSUPP, and installing
TLS on an attached socket fails with -EINVAL. Every refused setup was
already broken as described above, except one where only kTLS TX is
configured. Such a socket still passes data, because read_sock() stays
tcp_read_sock() until kTLS RX is configured, and KCM transmits with
sock_sendmsg(), so records reach tls_sw_sendmsg(). It is still wrong: if
TLS was armed before the attach, psock_write_space() shadows
tls_write_space() and does not chain to it, so a record deferred on a
full send buffer waits for the next send; if TLS was armed after the
attach, the chain works until kcm_unattach() drops tls_write_space(), as
above. The direction is not known at ULP install, so refusing it keeps
one rule for both orders and matches sk_psock_init().

Tested with CONFIG_DEBUG_ATOMIC_SLEEP: both orders are now rejected,
plain KCM still delivers messages, and the TLS ULP can be installed
after SIOCKCMUNATTACH.

Fixes: 662fbcec32f4 ("net/tls: implement ->read_sock()")
Cc: stable@vger.kernel.org
Suggested-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/netdev/3108e934-ec00-458b-8720-450554b35b55@linux.dev/
Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
---
 include/net/sock.h |  1 +
 net/kcm/kcmsock.c  | 12 ++++++++++++
 net/tls/tls_main.c |  6 ++++++
 3 files changed, 19 insertions(+)

diff --git a/include/net/sock.h b/include/net/sock.h
index 173b37b2b8bb..e579a4a96c8e 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1024,6 +1024,7 @@ enum sock_flags {
 	SOCK_RCVMARK, /* Receive SO_MARK  ancillary data with packet */
 	SOCK_RCVPRIORITY, /* Receive SO_PRIORITY ancillary data with packet */
 	SOCK_TIMESTAMPING_ANY, /* Copy of sk_tsflags & TSFLAGS_ANY */
+	SOCK_KCM, /* Attached to a KCM mux as a lower socket */
 };
 
 #define SK_FLAGS_TIMESTAMP ((1UL << SOCK_TIMESTAMP) | (1UL << SOCK_TIMESTAMPING_RX_SOFTWARE))
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index a7f94d42036f..546d5b563382 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -27,6 +27,7 @@
 #include <linux/sched/signal.h>
 #include <linux/uio.h>
 
+#include <net/inet_connection_sock.h>
 #include <net/kcm.h>
 #include <net/netns/generic.h>
 #include <net/sock.h>
@@ -1272,6 +1273,15 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
 		goto out;
 	}
 
+	/* KCM and ULPs are mutually exclusive. A ULP such as kTLS replaces
+	 * read_sock() and stacks its own socket callbacks with ours. tls_init()
+	 * rejects sockets already attached to KCM, see SOCK_KCM.
+	 */
+	if (inet_csk_has_ulp(csk)) {
+		err = -EOPNOTSUPP;
+		goto out;
+	}
+
 	psock = kmem_cache_zalloc(kcm_psockp, GFP_KERNEL);
 	if (!psock) {
 		err = -ENOMEM;
@@ -1305,6 +1315,7 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
 	psock->save_write_space = csk->sk_write_space;
 	psock->save_state_change = csk->sk_state_change;
 	csk->sk_user_data = psock;
+	sock_set_flag(csk, SOCK_KCM);
 	WRITE_ONCE(csk->sk_data_ready, psock_data_ready);
 	WRITE_ONCE(csk->sk_write_space, psock_write_space);
 	csk->sk_state_change = psock_state_change;
@@ -1382,6 +1393,7 @@ static void kcm_unattach(struct kcm_psock *psock)
 	 */
 	write_lock_bh(&csk->sk_callback_lock);
 	csk->sk_user_data = NULL;
+	sock_reset_flag(csk, SOCK_KCM);
 	WRITE_ONCE(csk->sk_data_ready, psock->save_data_ready);
 	WRITE_ONCE(csk->sk_write_space, psock->save_write_space);
 	csk->sk_state_change = psock->save_state_change;
diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index b8d5a843f761..e45062841950 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -1067,6 +1067,12 @@ static int tls_init(struct sock *sk)
 	if (sk->sk_state != TCP_ESTABLISHED)
 		return -ENOTCONN;
 
+	/* TLS and KCM are mutually exclusive. kcm_attach() rejects sockets
+	 * with a ULP; both run under the socket lock, so this cannot race.
+	 */
+	if (sock_flag(sk, SOCK_KCM))
+		return -EINVAL;
+
 	/* allocate tls context */
 	write_lock_bh(&sk->sk_callback_lock);
 	ctx = tls_ctx_create(sk);
-- 
2.50.1

[PATCH net v3 2/2] selftests: tls: test that TLS and KCM are mutually exclusive

[Rishikesh Jethwani]

Cover both orders, with TLS RX and TLS TX on the attach side, and check
that a socket detached from KCM with SIOCKCMUNATTACH can still be
upgraded to TLS.

KCM needs a BPF parser program, so the tests are skipped without
AF_KCM or when the program can't be loaded. Add CONFIG_AF_KCM to the
net selftest config.

Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
---
 tools/testing/selftests/net/config |   1 +
 tools/testing/selftests/net/tls.c  | 144 +++++++++++++++++++++++++++++
 2 files changed, 145 insertions(+)

diff --git a/tools/testing/selftests/net/config b/tools/testing/selftests/net/config
index cb8cb4696df7..1619478fffc4 100644
--- a/tools/testing/selftests/net/config
+++ b/tools/testing/selftests/net/config
@@ -1,3 +1,4 @@
+CONFIG_AF_KCM=m
 CONFIG_AMT=m
 CONFIG_BAREUDP=m
 CONFIG_BONDING=m
diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c
index 9d3cd4fff062..2e4128e0676a 100644
--- a/tools/testing/selftests/net/tls.c
+++ b/tools/testing/selftests/net/tls.c
@@ -7,19 +7,24 @@
 #include <error.h>
 #include <fcntl.h>
 #include <poll.h>
+#include <stddef.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
 
+#include <linux/bpf.h>
+#include <linux/kcm.h>
 #include <linux/tls.h>
 #include <linux/tcp.h>
 #include <linux/socket.h>
 
 #include <sys/epoll.h>
+#include <sys/ioctl.h>
 #include <sys/types.h>
 #include <sys/sendfile.h>
 #include <sys/socket.h>
 #include <sys/stat.h>
+#include <sys/syscall.h>
 
 #include "kselftest_harness.h"
 
@@ -3492,6 +3497,145 @@ TEST(data_steal) {
 	close(cfd);
 }
 
+#ifndef AF_KCM
+#define AF_KCM 41
+#endif
+
+/* KCM and TLS are mutually exclusive, in either order. */
+FIXTURE(kcm)
+{
+	int fd, cfd, kcm, bpf;
+};
+
+FIXTURE_SETUP(kcm)
+{
+	/* KCM parser: each message is everything queued (skb->len) */
+	struct bpf_insn insns[] = {
+		{ .code = BPF_LDX | BPF_MEM | BPF_W, .dst_reg = BPF_REG_0,
+		  .src_reg = BPF_REG_1,
+		  .off = offsetof(struct __sk_buff, len) },
+		{ .code = BPF_JMP | BPF_EXIT },
+	};
+	union bpf_attr attr = {};
+	struct sockaddr_in addr;
+	socklen_t len;
+	int sfd, ret;
+
+	self->fd = -1;
+	self->cfd = -1;
+	self->bpf = -1;
+
+	self->kcm = socket(AF_KCM, SOCK_DGRAM, KCMPROTO_CONNECTED);
+	if (self->kcm < 0)
+		SKIP(return, "no KCM support");
+
+	attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
+	attr.insns = (unsigned long)insns;
+	attr.insn_cnt = ARRAY_SIZE(insns);
+	attr.license = (unsigned long)"GPL";
+	self->bpf = syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
+	if (self->bpf < 0)
+		SKIP(return, "can't load KCM parser BPF program");
+
+	len = sizeof(addr);
+	addr.sin_family = AF_INET;
+	addr.sin_addr.s_addr = htonl(INADDR_ANY);
+	addr.sin_port = 0;
+
+	self->fd = socket(AF_INET, SOCK_STREAM, 0);
+	sfd = socket(AF_INET, SOCK_STREAM, 0);
+
+	ret = bind(sfd, &addr, sizeof(addr));
+	ASSERT_EQ(ret, 0);
+	ret = listen(sfd, 10);
+	ASSERT_EQ(ret, 0);
+	ret = getsockname(sfd, &addr, &len);
+	ASSERT_EQ(ret, 0);
+	ret = connect(self->fd, &addr, sizeof(addr));
+	ASSERT_EQ(ret, 0);
+	self->cfd = accept(sfd, &addr, &len);
+	ASSERT_GE(self->cfd, 0);
+
+	close(sfd);
+}
+
+FIXTURE_TEARDOWN(kcm)
+{
+	close(self->kcm);
+	close(self->bpf);
+	close(self->fd);
+	close(self->cfd);
+}
+
+static int kcm_attach(struct _test_data_kcm *self)
+{
+	struct kcm_attach attach = { .fd = self->cfd, .bpf_fd = self->bpf };
+
+	return ioctl(self->kcm, SIOCKCMATTACH, &attach);
+}
+
+static int kcm_unattach(struct _test_data_kcm *self)
+{
+	struct kcm_unattach unattach = { .fd = self->cfd };
+
+	return ioctl(self->kcm, SIOCKCMUNATTACH, &unattach);
+}
+
+static void kcm_tls_then_attach(struct __test_metadata *_metadata,
+				struct _test_data_kcm *self, int optname)
+{
+	struct tls_crypto_info_keys tls;
+	int ret;
+
+	ret = setsockopt(self->cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
+	if (ret && errno == ENOENT)
+		SKIP(return, "no TLS support");
+	ASSERT_EQ(ret, 0);
+
+	tls_crypto_info_init(TLS_1_2_VERSION, TLS_CIPHER_AES_GCM_128, &tls, 0);
+	ret = setsockopt(self->cfd, SOL_TLS, optname, &tls, tls.len);
+	ASSERT_EQ(ret, 0);
+
+	EXPECT_EQ(kcm_attach(self), -1);
+	EXPECT_EQ(errno, EOPNOTSUPP);
+}
+
+TEST_F(kcm, tls_rx_then_attach)
+{
+	kcm_tls_then_attach(_metadata, self, TLS_RX);
+}
+
+TEST_F(kcm, tls_tx_then_attach)
+{
+	kcm_tls_then_attach(_metadata, self, TLS_TX);
+}
+
+TEST_F(kcm, attach_then_tls)
+{
+	int ret;
+
+	ASSERT_EQ(kcm_attach(self), 0);
+
+	ret = setsockopt(self->cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
+	if (ret && errno == ENOENT)
+		SKIP(return, "no TLS support");
+	EXPECT_EQ(ret, -1);
+	EXPECT_EQ(errno, EINVAL);
+}
+
+TEST_F(kcm, tls_after_unattach)
+{
+	int ret;
+
+	ASSERT_EQ(kcm_attach(self), 0);
+	ASSERT_EQ(kcm_unattach(self), 0);
+
+	ret = setsockopt(self->cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
+	if (ret && errno == ENOENT)
+		SKIP(return, "no TLS support");
+	EXPECT_EQ(ret, 0);
+}
+
 static void __attribute__((constructor)) fips_check(void) {
 	int res;
 	FILE *f;
-- 
2.50.1

Re: [PATCH net v3 1/2] kcm: reject attaching sockets with a ULP, and TLS on KCM sockets

[Kuniyuki Iwashima]

On Wed, 28 Oct 2026 11:35:32 -0600, Rishikesh Jethwani wrote:
> KCM and kTLS RX do not work together, in either order.
> 
> KCM's psock_data_ready() is the lower socket's data_ready callback. It
> can run in softirq context under read_lock_bh(&sk->sk_callback_lock),
> and calls strp_data_ready(), which invokes sock->ops->read_sock()
> inline. With kTLS RX, that is tls_sw_read_sock(), which can sleep: it
> allocates with sk->sk_allocation, waits for async crypto, flushes the
> socket backlog, and may wait in tls_rx_reader_acquire().
> 
> Found by code review. On an unmodified kernel with
> CONFIG_DEBUG_ATOMIC_SLEEP, sending TLS records over loopback to a socket
> with software kTLS RX attached to a KCM mux reports:
> 
>   BUG: sleeping function called from invalid context at include/linux/sched/mm.h:321
>   in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 189, name: ksoftirqd/34
>   Call Trace:
>    __might_resched
>    kmem_cache_alloc_node_noprof
>    __alloc_skb
>    alloc_skb_with_frags
>    tls_rx_one_record
>    tls_sw_read_sock
>    strp_read_sock
>    strp_data_ready
>    psock_data_ready [kcm]
> 
> Before commit 662fbcec32f4 ("net/tls: implement ->read_sock()"), TLS
> inherited tcp_read_sock(), so KCM parsed the ciphertext in the TCP
> receive queue as messages. HW TLS RX uses the same proto_ops as SW
> RX, so it is affected the same way.
> 
> The two also stack socket callbacks and neither restores the other's:
> with TLS RX configured after the attach, kcm_unattach() overwrites
> tls_data_ready(), and releasing TLS RX later reinstalls
> psock_data_ready() on a socket no longer attached to KCM. On TX,
> kcm_unattach() restores its saved sk_write_space, dropping
> tls_write_space().
> 
> There is no known user of this combination, so reject it, as was done
> for sockmap in commit 460e6486617c ("tls: reject the combination of TLS
> and sockmap"):
> 
>  - kcm_attach() refuses sockets that already have any ULP, not only
>    TLS, as sk_psock_init() does for sockmap, with -EOPNOTSUPP like its
>    other refusals;
>  - tls_init() refuses sockets attached to KCM with -EINVAL, like
>    __tcp_set_ulp() and the sockmap check, tracked by a new SOCK_KCM
>    flag set on attach and cleared on unattach.
> 
> sk_user_data cannot serve as the marker: sunrpc and nvme-tcp set it on
> sockets that tlshd later upgrades to TLS, so it is already set when
> tls_init() runs. Unlike sockmap, both sides run under the lower
> socket's lock (tcp_set_ulp() asserts this), so the check is made at ULP
> install rather than at key configuration.
> 
> This is a user-visible change that will also reach stable kernels:
> attaching a socket with a ULP now fails with -EOPNOTSUPP, and installing
> TLS on an attached socket fails with -EINVAL. Every refused setup was
> already broken as described above, except one where only kTLS TX is
> configured. Such a socket still passes data, because read_sock() stays
> tcp_read_sock() until kTLS RX is configured, and KCM transmits with
> sock_sendmsg(), so records reach tls_sw_sendmsg(). It is still wrong: if
> TLS was armed before the attach, psock_write_space() shadows
> tls_write_space() and does not chain to it, so a record deferred on a
> full send buffer waits for the next send; if TLS was armed after the
> attach, the chain works until kcm_unattach() drops tls_write_space(), as
> above. The direction is not known at ULP install, so refusing it keeps
> one rule for both orders and matches sk_psock_init().
> 
> Tested with CONFIG_DEBUG_ATOMIC_SLEEP: both orders are now rejected,
> plain KCM still delivers messages, and the TLS ULP can be installed
> after SIOCKCMUNATTACH.
> 
> Fixes: 662fbcec32f4 ("net/tls: implement ->read_sock()")
> Cc: stable@vger.kernel.org
> Suggested-by: Jiayuan Chen <jiayuan.chen@linux.dev>
> Link: https://lore.kernel.org/netdev/3108e934-ec00-458b-8720-450554b35b55@linux.dev/
> Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
> ---
>  include/net/sock.h |  1 +
>  net/kcm/kcmsock.c  | 12 ++++++++++++
>  net/tls/tls_main.c |  6 ++++++
>  3 files changed, 19 insertions(+)
> 
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 173b37b2b8bb..e579a4a96c8e 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -1024,6 +1024,7 @@ enum sock_flags {
>  	SOCK_RCVMARK, /* Receive SO_MARK  ancillary data with packet */
>  	SOCK_RCVPRIORITY, /* Receive SO_PRIORITY ancillary data with packet */
>  	SOCK_TIMESTAMPING_ANY, /* Copy of sk_tsflags & TSFLAGS_ANY */
> +	SOCK_KCM, /* Attached to a KCM mux as a lower socket */

I guess this does not compile for 32-bit ?


>  };
>  
>  #define SK_FLAGS_TIMESTAMP ((1UL << SOCK_TIMESTAMP) | (1UL << SOCK_TIMESTAMPING_RX_SOFTWARE))
> diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
> index a7f94d42036f..546d5b563382 100644
> --- a/net/kcm/kcmsock.c
> +++ b/net/kcm/kcmsock.c
> @@ -27,6 +27,7 @@
>  #include <linux/sched/signal.h>
>  #include <linux/uio.h>
>  
> +#include <net/inet_connection_sock.h>
>  #include <net/kcm.h>
>  #include <net/netns/generic.h>
>  #include <net/sock.h>
> @@ -1272,6 +1273,15 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
>  		goto out;
>  	}
>  
> +	/* KCM and ULPs are mutually exclusive. A ULP such as kTLS replaces
> +	 * read_sock() and stacks its own socket callbacks with ours. tls_init()
> +	 * rejects sockets already attached to KCM, see SOCK_KCM.
> +	 */
> +	if (inet_csk_has_ulp(csk)) {
> +		err = -EOPNOTSUPP;
> +		goto out;
> +	}
> +
>  	psock = kmem_cache_zalloc(kcm_psockp, GFP_KERNEL);
>  	if (!psock) {
>  		err = -ENOMEM;
> @@ -1305,6 +1315,7 @@ static int kcm_attach(struct socket *sock, struct socket *csock,
>  	psock->save_write_space = csk->sk_write_space;
>  	psock->save_state_change = csk->sk_state_change;
>  	csk->sk_user_data = psock;
> +	sock_set_flag(csk, SOCK_KCM);
>  	WRITE_ONCE(csk->sk_data_ready, psock_data_ready);
>  	WRITE_ONCE(csk->sk_write_space, psock_write_space);
>  	csk->sk_state_change = psock_state_change;
> @@ -1382,6 +1393,7 @@ static void kcm_unattach(struct kcm_psock *psock)
>  	 */
>  	write_lock_bh(&csk->sk_callback_lock);
>  	csk->sk_user_data = NULL;
> +	sock_reset_flag(csk, SOCK_KCM);
>  	WRITE_ONCE(csk->sk_data_ready, psock->save_data_ready);
>  	WRITE_ONCE(csk->sk_write_space, psock->save_write_space);
>  	csk->sk_state_change = psock->save_state_change;
> diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
> index b8d5a843f761..e45062841950 100644
> --- a/net/tls/tls_main.c
> +++ b/net/tls/tls_main.c
> @@ -1067,6 +1067,12 @@ static int tls_init(struct sock *sk)
>  	if (sk->sk_state != TCP_ESTABLISHED)
>  		return -ENOTCONN;
>  
> +	/* TLS and KCM are mutually exclusive. kcm_attach() rejects sockets
> +	 * with a ULP; both run under the socket lock, so this cannot race.
> +	 */
> +	if (sock_flag(sk, SOCK_KCM))
> +		return -EINVAL;
> +
>  	/* allocate tls context */
>  	write_lock_bh(&sk->sk_callback_lock);

Perhaps move sk_psock() check in do_tls_setsockopt_conf() here.



>  	ctx = tls_ctx_create(sk);
> -- 
> 2.50.1
> 

Re: [PATCH net v3 2/2] selftests: tls: test that TLS and KCM are mutually exclusive

[Kuniyuki Iwashima]

On Wed, 28 Oct 2026 11:35:33 -0600, Rishikesh Jethwani wrote:
> Cover both orders, with TLS RX and TLS TX on the attach side, and check
> that a socket detached from KCM with SIOCKCMUNATTACH can still be
> upgraded to TLS.

I don't think we want a test.

This will just waste testing budgets unless someone tries
to restore the combination, which is unlikely to happen.


> 
> KCM needs a BPF parser program, so the tests are skipped without
> AF_KCM or when the program can't be loaded. Add CONFIG_AF_KCM to the
> net selftest config.
> 
> Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
> ---
>  tools/testing/selftests/net/config |   1 +
>  tools/testing/selftests/net/tls.c  | 144 +++++++++++++++++++++++++++++
>  2 files changed, 145 insertions(+)
> 
> diff --git a/tools/testing/selftests/net/config b/tools/testing/selftests/net/config
> index cb8cb4696df7..1619478fffc4 100644
> --- a/tools/testing/selftests/net/config
> +++ b/tools/testing/selftests/net/config
> @@ -1,3 +1,4 @@
> +CONFIG_AF_KCM=m
>  CONFIG_AMT=m
>  CONFIG_BAREUDP=m
>  CONFIG_BONDING=m
> diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c
> index 9d3cd4fff062..2e4128e0676a 100644
> --- a/tools/testing/selftests/net/tls.c
> +++ b/tools/testing/selftests/net/tls.c
> @@ -7,19 +7,24 @@
>  #include <error.h>
>  #include <fcntl.h>
>  #include <poll.h>
> +#include <stddef.h>
>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <unistd.h>
>  
> +#include <linux/bpf.h>
> +#include <linux/kcm.h>
>  #include <linux/tls.h>
>  #include <linux/tcp.h>
>  #include <linux/socket.h>
>  
>  #include <sys/epoll.h>
> +#include <sys/ioctl.h>
>  #include <sys/types.h>
>  #include <sys/sendfile.h>
>  #include <sys/socket.h>
>  #include <sys/stat.h>
> +#include <sys/syscall.h>
>  
>  #include "kselftest_harness.h"
>  
> @@ -3492,6 +3497,145 @@ TEST(data_steal) {
>  	close(cfd);
>  }
>  
> +#ifndef AF_KCM
> +#define AF_KCM 41
> +#endif
> +
> +/* KCM and TLS are mutually exclusive, in either order. */
> +FIXTURE(kcm)
> +{
> +	int fd, cfd, kcm, bpf;
> +};
> +
> +FIXTURE_SETUP(kcm)
> +{
> +	/* KCM parser: each message is everything queued (skb->len) */
> +	struct bpf_insn insns[] = {
> +		{ .code = BPF_LDX | BPF_MEM | BPF_W, .dst_reg = BPF_REG_0,
> +		  .src_reg = BPF_REG_1,
> +		  .off = offsetof(struct __sk_buff, len) },
> +		{ .code = BPF_JMP | BPF_EXIT },
> +	};
> +	union bpf_attr attr = {};
> +	struct sockaddr_in addr;
> +	socklen_t len;
> +	int sfd, ret;
> +
> +	self->fd = -1;
> +	self->cfd = -1;
> +	self->bpf = -1;
> +
> +	self->kcm = socket(AF_KCM, SOCK_DGRAM, KCMPROTO_CONNECTED);
> +	if (self->kcm < 0)
> +		SKIP(return, "no KCM support");
> +
> +	attr.prog_type = BPF_PROG_TYPE_SOCKET_FILTER;
> +	attr.insns = (unsigned long)insns;
> +	attr.insn_cnt = ARRAY_SIZE(insns);
> +	attr.license = (unsigned long)"GPL";
> +	self->bpf = syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
> +	if (self->bpf < 0)
> +		SKIP(return, "can't load KCM parser BPF program");
> +
> +	len = sizeof(addr);
> +	addr.sin_family = AF_INET;
> +	addr.sin_addr.s_addr = htonl(INADDR_ANY);
> +	addr.sin_port = 0;
> +
> +	self->fd = socket(AF_INET, SOCK_STREAM, 0);
> +	sfd = socket(AF_INET, SOCK_STREAM, 0);
> +
> +	ret = bind(sfd, &addr, sizeof(addr));
> +	ASSERT_EQ(ret, 0);
> +	ret = listen(sfd, 10);
> +	ASSERT_EQ(ret, 0);
> +	ret = getsockname(sfd, &addr, &len);
> +	ASSERT_EQ(ret, 0);
> +	ret = connect(self->fd, &addr, sizeof(addr));
> +	ASSERT_EQ(ret, 0);
> +	self->cfd = accept(sfd, &addr, &len);
> +	ASSERT_GE(self->cfd, 0);
> +
> +	close(sfd);
> +}
> +
> +FIXTURE_TEARDOWN(kcm)
> +{
> +	close(self->kcm);
> +	close(self->bpf);
> +	close(self->fd);
> +	close(self->cfd);
> +}
> +
> +static int kcm_attach(struct _test_data_kcm *self)
> +{
> +	struct kcm_attach attach = { .fd = self->cfd, .bpf_fd = self->bpf };
> +
> +	return ioctl(self->kcm, SIOCKCMATTACH, &attach);
> +}
> +
> +static int kcm_unattach(struct _test_data_kcm *self)
> +{
> +	struct kcm_unattach unattach = { .fd = self->cfd };
> +
> +	return ioctl(self->kcm, SIOCKCMUNATTACH, &unattach);
> +}
> +
> +static void kcm_tls_then_attach(struct __test_metadata *_metadata,
> +				struct _test_data_kcm *self, int optname)
> +{
> +	struct tls_crypto_info_keys tls;
> +	int ret;
> +
> +	ret = setsockopt(self->cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
> +	if (ret && errno == ENOENT)
> +		SKIP(return, "no TLS support");
> +	ASSERT_EQ(ret, 0);
> +
> +	tls_crypto_info_init(TLS_1_2_VERSION, TLS_CIPHER_AES_GCM_128, &tls, 0);
> +	ret = setsockopt(self->cfd, SOL_TLS, optname, &tls, tls.len);
> +	ASSERT_EQ(ret, 0);
> +
> +	EXPECT_EQ(kcm_attach(self), -1);
> +	EXPECT_EQ(errno, EOPNOTSUPP);
> +}
> +
> +TEST_F(kcm, tls_rx_then_attach)
> +{
> +	kcm_tls_then_attach(_metadata, self, TLS_RX);
> +}
> +
> +TEST_F(kcm, tls_tx_then_attach)
> +{
> +	kcm_tls_then_attach(_metadata, self, TLS_TX);
> +}
> +
> +TEST_F(kcm, attach_then_tls)
> +{
> +	int ret;
> +
> +	ASSERT_EQ(kcm_attach(self), 0);
> +
> +	ret = setsockopt(self->cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
> +	if (ret && errno == ENOENT)
> +		SKIP(return, "no TLS support");
> +	EXPECT_EQ(ret, -1);
> +	EXPECT_EQ(errno, EINVAL);
> +}
> +
> +TEST_F(kcm, tls_after_unattach)
> +{
> +	int ret;
> +
> +	ASSERT_EQ(kcm_attach(self), 0);
> +	ASSERT_EQ(kcm_unattach(self), 0);
> +
> +	ret = setsockopt(self->cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
> +	if (ret && errno == ENOENT)
> +		SKIP(return, "no TLS support");
> +	EXPECT_EQ(ret, 0);
> +}
> +
>  static void __attribute__((constructor)) fips_check(void) {
>  	int res;
>  	FILE *f;
> -- 
> 2.50.1
> 

Re: [PATCH net v3 1/2] kcm: reject attaching sockets with a ULP, and TLS on KCM sockets

[Jiayuan Chen]

On 10/29/26 4:41 AM, Kuniyuki Iwashima wrote:
> On Wed, 28 Oct 2026 11:35:32 -0600, Rishikesh Jethwani wrote:
>> KCM and kTLS RX do not work together, in either order.
>>
>> KCM's psock_data_ready() is the lower socket's data_ready callback. It
>> can run in softirq context under read_lock_bh(&sk->sk_callback_lock),
>> and calls strp_data_ready(), which invokes sock->ops->read_sock()
>> inline. With kTLS RX, that is tls_sw_read_sock(), which can sleep: it
>> allocates with sk->sk_allocation, waits for async crypto, flushes the
>> socket backlog, and may wait in tls_rx_reader_acquire().
>>
>> Found by code review. On an unmodified kernel with
>> CONFIG_DEBUG_ATOMIC_SLEEP, sending TLS records over loopback to a socket
>> with software kTLS RX attached to a KCM mux reports:
>>
>>    BUG: sleeping function called from invalid context at include/linux/sched/mm.h:321
>>    in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 189, name: ksoftirqd/34
>>    Call Trace:
>>     __might_resched
>>     kmem_cache_alloc_node_noprof
>>     __alloc_skb
>>     alloc_skb_with_frags
>>     tls_rx_one_record
>>     tls_sw_read_sock
>>     strp_read_sock
>>     strp_data_ready
>>     psock_data_ready [kcm]
>>
>> Before commit 662fbcec32f4 ("net/tls: implement ->read_sock()"), TLS
>> inherited tcp_read_sock(), so KCM parsed the ciphertext in the TCP
>> receive queue as messages. HW TLS RX uses the same proto_ops as SW
>> RX, so it is affected the same way.
>>
>> The two also stack socket callbacks and neither restores the other's:
>> with TLS RX configured after the attach, kcm_unattach() overwrites
>> tls_data_ready(), and releasing TLS RX later reinstalls
>> psock_data_ready() on a socket no longer attached to KCM. On TX,
>> kcm_unattach() restores its saved sk_write_space, dropping
>> tls_write_space().
>>
>> There is no known user of this combination, so reject it, as was done
>> for sockmap in commit 460e6486617c ("tls: reject the combination of TLS
>> and sockmap"):
>>
>>   - kcm_attach() refuses sockets that already have any ULP, not only
>>     TLS, as sk_psock_init() does for sockmap, with -EOPNOTSUPP like its
>>     other refusals;
>>   - tls_init() refuses sockets attached to KCM with -EINVAL, like
>>     __tcp_set_ulp() and the sockmap check, tracked by a new SOCK_KCM
>>     flag set on attach and cleared on unattach.
>>
>> sk_user_data cannot serve as the marker: sunrpc and nvme-tcp set it on
>> sockets that tlshd later upgrades to TLS, so it is already set when
>> tls_init() runs. Unlike sockmap, both sides run under the lower
>> socket's lock (tcp_set_ulp() asserts this), so the check is made at ULP
>> install rather than at key configuration.
>>
>> This is a user-visible change that will also reach stable kernels:
>> attaching a socket with a ULP now fails with -EOPNOTSUPP, and installing
>> TLS on an attached socket fails with -EINVAL. Every refused setup was
>> already broken as described above, except one where only kTLS TX is
>> configured. Such a socket still passes data, because read_sock() stays
>> tcp_read_sock() until kTLS RX is configured, and KCM transmits with
>> sock_sendmsg(), so records reach tls_sw_sendmsg(). It is still wrong: if
>> TLS was armed before the attach, psock_write_space() shadows
>> tls_write_space() and does not chain to it, so a record deferred on a
>> full send buffer waits for the next send; if TLS was armed after the
>> attach, the chain works until kcm_unattach() drops tls_write_space(), as
>> above. The direction is not known at ULP install, so refusing it keeps
>> one rule for both orders and matches sk_psock_init().
>>
>> Tested with CONFIG_DEBUG_ATOMIC_SLEEP: both orders are now rejected,
>> plain KCM still delivers messages, and the TLS ULP can be installed
>> after SIOCKCMUNATTACH.
>>
>> Fixes: 662fbcec32f4 ("net/tls: implement ->read_sock()")
>> Cc: stable@vger.kernel.org
>> Suggested-by: Jiayuan Chen <jiayuan.chen@linux.dev>
>> Link: https://lore.kernel.org/netdev/3108e934-ec00-458b-8720-450554b35b55@linux.dev/
>> Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
>> ---
>>   include/net/sock.h |  1 +
>>   net/kcm/kcmsock.c  | 12 ++++++++++++
>>   net/tls/tls_main.c |  6 ++++++
>>   3 files changed, 19 insertions(+)
>>
>> diff --git a/include/net/sock.h b/include/net/sock.h
>> index 173b37b2b8bb..e579a4a96c8e 100644
>> --- a/include/net/sock.h
>> +++ b/include/net/sock.h
>> @@ -1024,6 +1024,7 @@ enum sock_flags {
>>   	SOCK_RCVMARK, /* Receive SO_MARK  ancillary data with packet */
>>   	SOCK_RCVPRIORITY, /* Receive SO_PRIORITY ancillary data with packet */
>>   	SOCK_TIMESTAMPING_ANY, /* Copy of sk_tsflags & TSFLAGS_ANY */
>> +	SOCK_KCM, /* Attached to a KCM mux as a lower socket */
> I guess this does not compile for 32-bit ?
>


I think it is a waste of sock_flags. Can we use a dummy ULP instead?
KCM takes over the socket like a ULP does, so once it holds icsk_ulp_ops,

tcp_set_ulp() refuses TLS with -EEXIST on its own and no change is 
needed on the TLS side.


Something like (not tested):


--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -30,11 +30,19 @@
#include <net/kcm.h>
#include <net/netns/generic.h>
#include <net/sock.h>
+#include <net/tcp.h>
#include <uapi/linux/kcm.h>
#include <trace/events/sock.h>

unsigned int kcm_net_id;

+/* Not registered, so TCP_ULP can't select it. It only holds
+ * icsk_ulp_ops so tcp_set_ulp() refuses other ULPs like kTLS.
+ */
+static const struct tcp_ulp_ops kcm_ulp_ops = {
+       .name = "kcm",
+};
+
static struct kmem_cache *kcm_psockp __read_mostly;
static struct kmem_cache *kcm_muxp __read_mostly;
static struct workqueue_struct *kcm_wq;
@@ -1272,6 +1280,14 @@
               goto out;
       }

+       /* KCM takes over read_sock() and the socket callbacks like a ULP
+        * does, the two can't stack.
+        */
+       if (inet_csk_has_ulp(csk)) {
+               err = -EOPNOTSUPP;
+               goto out;
+       }
+
       psock = kmem_cache_zalloc(kcm_psockp, GFP_KERNEL);
       if (!psock) {
               err = -ENOMEM;
@@ -1305,6 +1321,7 @@
       psock->save_write_space = csk->sk_write_space;
       psock->save_state_change = csk->sk_state_change;
       csk->sk_user_data = psock;
+       inet_csk(csk)->icsk_ulp_ops = &kcm_ulp_ops;
       WRITE_ONCE(csk->sk_data_ready, psock_data_ready);
       WRITE_ONCE(csk->sk_write_space, psock_write_space);
       csk->sk_state_change = psock_state_change;
@@ -1382,6 +1399,7 @@
        */
       write_lock_bh(&csk->sk_callback_lock);
       csk->sk_user_data = NULL;
+       inet_csk(csk)->icsk_ulp_ops = NULL;
       WRITE_ONCE(csk->sk_data_ready, psock->save_data_ready);
       WRITE_ONCE(csk->sk_write_space, psock->save_write_space);
       csk->sk_state_change = psock->save_state_change;