[PATCH v3 0/4] nvme-tcp: add IPv6 traffic class support

[PATCH v3 0/4] nvme-tcp: add IPv6 traffic class support

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

This series adds IPv6 traffic class (tclass) support to the NVMe/TCP
fabrics stack, mirroring the existing IPv4 TOS configuration.

The current code path only handles IPv4: nvme-fabrics exposes a 'tos'
option, the host (nvme-tcp) applies IP_TOS to each queue socket, and
the target (nvmet-tcp) reflects the value received on the wire. This
leaves no way to control or observe the IPv6 traffic class for NVMe/TCP
connections, which is a real gap for deployments running over IPv6 that
need a non-default tclass.

The series also refactors the socket option configuration on both host
and target sides to use the generic do_sock_setsockopt() helper. This
change eliminates the need for protocol-specific helpers and decouples
the configuration from the underlying transport protocol, laying the
groundwork for future protocol extensions such as MPTCP.

The series is split into two logical parts: the do_sock_setsockopt
refactoring (patches 1-2) and the IPv6 traffic class support (patches
3-4), following the natural host/target layering.

This was tested with the selftests mptcp_nvme.sh [1] to pass "--tos"
on the host side and validate that the value is reflected on the wire
by the target.

[1]
https://patchwork.kernel.org/project/linux-nvme/cover/cover.1779934709.git.tanggeliang@kylinos.cn/

v3:
 - add two patches to unify socket option configuration using
   do_sock_setsockopt on both host and target sides.
 - use do_sock_setsockopt for IPV6_TCLASS rather than exporting and
   calling ip6_sock_set_tclass, as suggested by Jakub.
 - reuse the existing NVMF_OPT_TOS for IPv6 sockets instead of adding
   a new NVMF_OPT_TCLASS option, as suggested by Stanislav.

v2:
 - Export and use the ip6_sock_set_tclass() helper in both target and host
   paths, aligning with the existing ip_sock_set_tos() usage.
 - Add CONFIG_IPV6 and AF_INET6 guards to avoid build failures when IPv6
   is disabled.
 - Use rcv_flowinfo from the target socket instead of np->tclass.
 - https://patchwork.kernel.org/project/linux-nvme/cover/cover.1786171863.git.tanggeliang@kylinos.cn/

v1:
 - https://patchwork.kernel.org/project/linux-nvme/cover/cover.1785122120.git.tanggeliang@kylinos.cn/

Geliang Tang (4):
  nvmet-tcp: unify sockopt with do_sock_setsockopt
  nvme-tcp: unify sockopt with do_sock_setsockopt
  nvmet-tcp: support IPv6 traffic class
  nvme-tcp: support IPv6 traffic class

 drivers/nvme/host/tcp.c   | 68 +++++++++++++++++++++++++++++++++------
 drivers/nvme/target/tcp.c | 68 ++++++++++++++++++++++++++++++++++-----
 2 files changed, 119 insertions(+), 17 deletions(-)

-- 
2.53.0

[PATCH v3 1/4] nvmet-tcp: unify sockopt with do_sock_setsockopt

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

This patch consolidates socket option settings in nvmet-tcp by utilizing
the generic do_sock_setsockopt() helper for options including SO_LINGER,
SO_PRIORITY, SO_REUSEADDR, TCP_NODELAY, and IP_TOS. This change eliminates
the need to export and use specialized helpers for each individual socket
option.

A key benefit of this refactoring is that it decouples the socket option
configuration from the underlying transport protocol. This makes it
easier to extend nvmet-tcp to support other protocols, such as MPTCP, in
the future, as do_sock_setsockopt() abstracts away protocol-specific
differences without requiring per-option protocol-specific wrappers.

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 drivers/nvme/target/tcp.c | 53 +++++++++++++++++++++++++++++++++------
 1 file changed, 45 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 75a276d73be3..e64592b79257 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1696,10 +1696,48 @@ static void nvmet_tcp_state_change(struct sock *sk)
 	read_unlock_bh(&sk->sk_callback_lock);
 }
 
+static void nvmet_tcp_sock_no_linger(struct sock *sk)
+{
+	struct linger ling = { .l_onoff = 1, .l_linger = 0 };
+
+	do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_LINGER,
+			   KERNEL_SOCKPTR(&ling), sizeof(ling));
+}
+
+static void nvmet_tcp_sock_set_priority(struct sock *sk, u32 priority)
+{
+	do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_PRIORITY,
+			   KERNEL_SOCKPTR(&priority), sizeof(priority));
+}
+
+static void nvmet_tcp_sock_set_reuseaddr(struct sock *sk)
+{
+	int val = SK_CAN_REUSE;
+
+	do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_REUSEADDR,
+			   KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static void nvmet_tcp_sock_set_nodelay(struct sock *sk)
+{
+	int val = 1;
+
+	do_sock_setsockopt(sk->sk_socket, false, SOL_TCP, TCP_NODELAY,
+			   KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static void nvmet_tcp_sock_set_tos(struct sock *sk)
+{
+	u8 tos = inet_sk(sk)->rcv_tos;
+
+	if (tos > 0)
+		do_sock_setsockopt(sk->sk_socket, false, SOL_IP, IP_TOS,
+				   KERNEL_SOCKPTR(&tos), sizeof(tos));
+}
+
 static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 {
 	struct socket *sock = queue->sock;
-	struct inet_sock *inet = inet_sk(sock->sk);
 	int ret;
 
 	ret = kernel_getsockname(sock,
@@ -1717,14 +1755,13 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 	 * close. This is done to prevent stale data from being sent should
 	 * the network connection be restored before TCP times out.
 	 */
-	sock_no_linger(sock->sk);
+	nvmet_tcp_sock_no_linger(sock->sk);
 
 	if (so_priority > 0)
-		sock_set_priority(sock->sk, so_priority);
+		nvmet_tcp_sock_set_priority(sock->sk, so_priority);
 
 	/* Set socket type of service */
-	if (inet->rcv_tos > 0)
-		ip_sock_set_tos(sock->sk, inet->rcv_tos);
+	nvmet_tcp_sock_set_tos(sock->sk);
 
 	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
@@ -2098,10 +2135,10 @@ static int nvmet_tcp_add_port(struct nvmet_port *nport)
 	port->sock->sk->sk_user_data = port;
 	port->data_ready = port->sock->sk->sk_data_ready;
 	port->sock->sk->sk_data_ready = nvmet_tcp_listen_data_ready;
-	sock_set_reuseaddr(port->sock->sk);
-	tcp_sock_set_nodelay(port->sock->sk);
+	nvmet_tcp_sock_set_reuseaddr(port->sock->sk);
+	nvmet_tcp_sock_set_nodelay(port->sock->sk);
 	if (so_priority > 0)
-		sock_set_priority(port->sock->sk, so_priority);
+		nvmet_tcp_sock_set_priority(port->sock->sk, so_priority);
 
 	ret = kernel_bind(port->sock, (struct sockaddr_unsized *)&port->addr,
 			sizeof(port->addr));
-- 
2.53.0

[PATCH v3 2/4] nvme-tcp: unify sockopt with do_sock_setsockopt

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

This patch consolidates socket option settings in nvme-tcp by utilizing
the generic do_sock_setsockopt() helper for options including SO_LINGER,
SO_PRIORITY, TCP_NODELAY, IP_TOS, SO_BINDTODEVICE, and TCP_SYNCNT.

Compared to the target-side implementation, this patch additionally
converts SO_BINDTODEVICE and TCP_SYNCNT to use the same unified mechanism.
This change eliminates the need to export and use specialized helpers for
each individual socket option.

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 drivers/nvme/host/tcp.c | 55 +++++++++++++++++++++++++++++++++++------
 1 file changed, 47 insertions(+), 8 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ba5c7b3e2a7c..89a82e8248d2 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1774,6 +1774,47 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
 	return ret;
 }
 
+static void nvme_tcp_sock_no_linger(struct sock *sk)
+{
+	struct linger ling = { .l_onoff = 1, .l_linger = 0 };
+
+	do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_LINGER,
+			   KERNEL_SOCKPTR(&ling), sizeof(ling));
+}
+
+static void nvme_tcp_sock_set_priority(struct sock *sk, u32 priority)
+{
+	do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET, SO_PRIORITY,
+			   KERNEL_SOCKPTR(&priority), sizeof(priority));
+}
+
+static int nvme_tcp_sock_set_bindtodevice(struct sock *sk, char *iface)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_SOCKET,
+				  SO_BINDTODEVICE, KERNEL_SOCKPTR(iface),
+				  strlen(iface));
+}
+
+static void nvme_tcp_sock_set_nodelay(struct sock *sk)
+{
+	int val = 1;
+
+	do_sock_setsockopt(sk->sk_socket, false, SOL_TCP, TCP_NODELAY,
+			   KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static int nvme_tcp_sock_set_syncnt(struct sock *sk, int val)
+{
+	return do_sock_setsockopt(sk->sk_socket, false, SOL_TCP, TCP_SYNCNT,
+				  KERNEL_SOCKPTR(&val), sizeof(val));
+}
+
+static void nvme_tcp_sock_set_tos(struct sock *sk, int tos)
+{
+	do_sock_setsockopt(sk->sk_socket, false, SOL_IP, IP_TOS,
+			   KERNEL_SOCKPTR(&tos), sizeof(tos));
+}
+
 static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 				key_serial_t pskid)
 {
@@ -1819,24 +1860,24 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 #endif
 
 	/* Single syn retry */
-	tcp_sock_set_syncnt(queue->sock->sk, 1);
+	nvme_tcp_sock_set_syncnt(queue->sock->sk, 1);
 
 	/* Set TCP no delay */
-	tcp_sock_set_nodelay(queue->sock->sk);
+	nvme_tcp_sock_set_nodelay(queue->sock->sk);
 
 	/*
 	 * Cleanup whatever is sitting in the TCP transmit queue on socket
 	 * close. This is done to prevent stale data from being sent should
 	 * the network connection be restored before TCP times out.
 	 */
-	sock_no_linger(queue->sock->sk);
+	nvme_tcp_sock_no_linger(queue->sock->sk);
 
 	if (so_priority > 0)
-		sock_set_priority(queue->sock->sk, so_priority);
+		nvme_tcp_sock_set_priority(queue->sock->sk, so_priority);
 
 	/* Set socket type of service */
 	if (nctrl->opts->tos >= 0)
-		ip_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
+		nvme_tcp_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
 
 	/* Set 10 seconds timeout for icresp recvmsg */
 	queue->sock->sk->sk_rcvtimeo = 10 * HZ;
@@ -1864,10 +1905,8 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 
 	if (nctrl->opts->mask & NVMF_OPT_HOST_IFACE) {
 		char *iface = nctrl->opts->host_iface;
-		sockptr_t optval = KERNEL_SOCKPTR(iface);
 
-		ret = sock_setsockopt(queue->sock, SOL_SOCKET, SO_BINDTODEVICE,
-				      optval, strlen(iface));
+		ret = nvme_tcp_sock_set_bindtodevice(queue->sock->sk, iface);
 		if (ret) {
 			dev_err(nctrl->device,
 			  "failed to bind to interface %s queue %d err %d\n",
-- 
2.53.0

[PATCH v3 3/4] nvmet-tcp: support IPv6 traffic class

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

Currently, nvmet-tcp only applies the received IPv4 TOS value when setting
up a queue socket, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets. Obtain the
traffic class from the IPv6 socket's rcv_flowinfo and apply it through
IPV6_TCLASS using do_sock_setsockopt().

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 drivers/nvme/target/tcp.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index e64592b79257..d34249cf3756 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1735,6 +1735,20 @@ static void nvmet_tcp_sock_set_tos(struct sock *sk)
 				   KERNEL_SOCKPTR(&tos), sizeof(tos));
 }
 
+static void nvmet_tcp_sock_set_tclass(struct sock *sk)
+{
+#if IS_ENABLED(CONFIG_IPV6)
+	if (sk->sk_family == AF_INET6) {
+		u8 tclass = ip6_tclass(inet6_sk(sk)->rcv_flowinfo);
+
+		if (tclass > 0)
+			do_sock_setsockopt(sk->sk_socket, false, SOL_IPV6,
+					   IPV6_TCLASS, KERNEL_SOCKPTR(&tclass),
+					   sizeof(tclass));
+	}
+#endif
+}
+
 static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 {
 	struct socket *sock = queue->sock;
@@ -1762,6 +1776,7 @@ static int nvmet_tcp_set_queue_sock(struct nvmet_tcp_queue *queue)
 
 	/* Set socket type of service */
 	nvmet_tcp_sock_set_tos(sock->sk);
+	nvmet_tcp_sock_set_tclass(sock->sk);
 
 	ret = 0;
 	write_lock_bh(&sock->sk->sk_callback_lock);
-- 
2.53.0

[PATCH v3 4/4] nvme-tcp: support IPv6 traffic class

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

Currently, nvme-tcp host only supports setting the IPv4 TOS value when a
TOS is specified, but does not handle the IPv6 traffic class.

Extend the queue socket setup to handle AF_INET6 sockets by applying the
TOS value to both IPv4 and IPv6 sockets. For IPv6, the TOS value is set
as the IPv6 traffic class via IPV6_TCLASS using do_sock_setsockopt().

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 drivers/nvme/host/tcp.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 89a82e8248d2..657acae9e4a4 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1815,6 +1815,15 @@ static void nvme_tcp_sock_set_tos(struct sock *sk, int tos)
 			   KERNEL_SOCKPTR(&tos), sizeof(tos));
 }
 
+static void nvme_tcp_sock_set_tclass(struct sock *sk, int tclass)
+{
+#if IS_ENABLED(CONFIG_IPV6)
+	if (sk->sk_family == AF_INET6)
+		do_sock_setsockopt(sk->sk_socket, false, SOL_IPV6, IPV6_TCLASS,
+				   KERNEL_SOCKPTR(&tclass), sizeof(tclass));
+#endif
+}
+
 static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 				key_serial_t pskid)
 {
@@ -1876,8 +1885,10 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 		nvme_tcp_sock_set_priority(queue->sock->sk, so_priority);
 
 	/* Set socket type of service */
-	if (nctrl->opts->tos >= 0)
+	if (nctrl->opts->tos >= 0) {
 		nvme_tcp_sock_set_tos(queue->sock->sk, nctrl->opts->tos);
+		nvme_tcp_sock_set_tclass(queue->sock->sk, nctrl->opts->tos);
+	}
 
 	/* Set 10 seconds timeout for icresp recvmsg */
 	queue->sock->sk->sk_rcvtimeo = 10 * HZ;
-- 
2.53.0