[PATCH bpf-next v3 0/2] Enable bpf_setsockopt() on ktls enabled sockets.

[PATCH bpf-next v3 0/2] Enable bpf_setsockopt() on ktls enabled sockets.

[Kui-Feng Lee]

This patchset implements a change to bpf_setsockopt() which allows
ktls enabled sockets to be used with the SOL_TCP level. This is
necessary as when ktls is enabled, it changes the function pointer of
setsockopt of the socket, which bpf_setsockopt() checks in order to
make sure that the socket is a TCP socket. Checking sk_protocol
instead of the function pointer will ensure that bpf_setsockopt() with
the SOL_TCP level still works on sockets with ktls enabled.

The major differences form v2 are:
 - Add a read() call to make sure that the FIN has arrived.
 - Remove the dependency on other test's header.

The major differences from v1 are:
 - Test with a IPv6 connect as well.
 - Use ASSERT_OK()

v2: https://lore.kernel.org/bpf/20230124181220.2871611-1-kuifeng@meta.com/
v1: https://lore.kernel.org/bpf/20230121025716.3039933-1-kuifeng@meta.com/

Kui-Feng Lee (2):
  bpf: Check the protocol of a sock to agree the calls to
    bpf_setsockopt().
  selftests/bpf: Calls bpf_setsockopt() on a ktls enabled socket.

 net/core/filter.c                             |  2 +-
 .../selftests/bpf/prog_tests/setget_sockopt.c | 73 +++++++++++++++++++
 .../selftests/bpf/progs/setget_sockopt.c      |  8 ++
 3 files changed, 82 insertions(+), 1 deletion(-)

-- 
2.30.2

[PATCH bpf-next v3 1/2] bpf: Check the protocol of a sock to agree the calls to bpf_setsockopt().

[Kui-Feng Lee]

Resolve an issue when calling sol_tcp_sockopt() on a socket with ktls
enabled. Prior to this patch, sol_tcp_sockopt() would only allow calls
if the function pointer of setsockopt of the socket was set to
tcp_setsockopt(). However, any socket with ktls enabled would have its
function pointer set to tls_setsockopt(). To resolve this issue, the
patch adds a check of the protocol of the linux socket and allows
bpf_setsockopt() to be called if ktls is initialized on the linux
socket. This ensures that calls to sol_tcp_sockopt() will succeed on
sockets with ktls enabled.

Signed-off-by: Kui-Feng Lee <kuifeng@meta.com>
---
 net/core/filter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/core/filter.c b/net/core/filter.c
index b4547a2c02f4..890384cbdeb2 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5204,7 +5204,7 @@ static int sol_tcp_sockopt(struct sock *sk, int optname,
 			   char *optval, int *optlen,
 			   bool getopt)
 {
-	if (sk->sk_prot->setsockopt != tcp_setsockopt)
+	if (sk->sk_protocol != IPPROTO_TCP)
 		return -EINVAL;
 
 	switch (optname) {
-- 
2.30.2

[PATCH bpf-next v3 2/2] selftests/bpf: Calls bpf_setsockopt() on a ktls enabled socket.

[Kui-Feng Lee]

Ensures that whenever bpf_setsockopt() is called with the SOL_TCP
option on a ktls enabled socket, the call will be accepted by the
system. The provided test makes sure of this by performing an
examination when the server side socket is in the CLOSE_WAIT state. At
this stage, ktls is still enabled on the server socket and can be used
to test if bpf_setsockopt() works correctly with linux.

Signed-off-by: Kui-Feng Lee <kuifeng@meta.com>
---
 .../selftests/bpf/prog_tests/setget_sockopt.c | 73 +++++++++++++++++++
 .../selftests/bpf/progs/setget_sockopt.c      |  8 ++
 2 files changed, 81 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/setget_sockopt.c b/tools/testing/selftests/bpf/prog_tests/setget_sockopt.c
index 018611e6b248..8591a3a780c9 100644
--- a/tools/testing/selftests/bpf/prog_tests/setget_sockopt.c
+++ b/tools/testing/selftests/bpf/prog_tests/setget_sockopt.c
@@ -4,6 +4,7 @@
 #define _GNU_SOURCE
 #include <sched.h>
 #include <linux/socket.h>
+#include <linux/tls.h>
 #include <net/if.h>
 
 #include "test_progs.h"
@@ -83,6 +84,76 @@ static void test_udp(int family)
 	ASSERT_EQ(bss->nr_binddev, 1, "nr_bind");
 }
 
+static void test_ktls(int family)
+{
+	struct tls12_crypto_info_aes_gcm_128 aes128;
+	struct setget_sockopt__bss *bss = skel->bss;
+	int cfd = -1, sfd = -1, fd = -1, ret;
+
+	memset(bss, 0, sizeof(*bss));
+
+	sfd = start_server(family, SOCK_STREAM,
+			   family == AF_INET6 ? addr6_str : addr4_str, 0, 0);
+	if (!ASSERT_GE(sfd, 0, "start_server"))
+		return;
+	fd = connect_to_fd(sfd, 0);
+	if (!ASSERT_GE(fd, 0, "connect_to_fd"))
+		goto err_out;
+
+	cfd = accept(sfd, NULL, 0);
+	if (!ASSERT_GE(cfd, 0, "accept"))
+		goto err_out;
+
+	close(sfd);
+	sfd = -1;
+
+	/* Setup KTLS */
+	ret = setsockopt(fd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
+	if (!ASSERT_OK(ret, "setsockopt"))
+		goto err_out;
+	ret = setsockopt(cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
+	if (!ASSERT_OK(ret, "setsockopt"))
+		goto err_out;
+
+	memset(&aes128, 0, sizeof(aes128));
+	aes128.info.version = TLS_1_2_VERSION;
+	aes128.info.cipher_type = TLS_CIPHER_AES_GCM_128;
+
+	ret = setsockopt(fd, SOL_TLS, TLS_TX, &aes128, sizeof(aes128));
+	if (!ASSERT_OK(ret, "setsockopt"))
+		goto err_out;
+
+	ret = setsockopt(cfd, SOL_TLS, TLS_RX, &aes128, sizeof(aes128));
+	if (!ASSERT_OK(ret, "setsockopt"))
+		goto err_out;
+
+	/* KTLS is enabled */
+
+	close(fd);
+	/* At this point, the cfd socket is at the CLOSE_WAIT state
+	 * and still run TLS protocol.  The test for
+	 * BPF_TCP_CLOSE_WAIT should be run at this point.
+	 */
+	char buf[1];
+	ret = read(cfd, buf, 1);
+	ASSERT_EQ(ret, 0, "read");
+	close(cfd);
+
+	ASSERT_EQ(bss->nr_listen, 1, "nr_listen");
+	ASSERT_EQ(bss->nr_connect, 1, "nr_connect");
+	ASSERT_EQ(bss->nr_active, 1, "nr_active");
+	ASSERT_EQ(bss->nr_passive, 1, "nr_passive");
+	ASSERT_EQ(bss->nr_socket_post_create, 2, "nr_socket_post_create");
+	ASSERT_EQ(bss->nr_binddev, 2, "nr_bind");
+	ASSERT_EQ(bss->nr_fin_wait1, 1, "nr_fin_wait1");
+	return;
+
+err_out:
+	close(fd);
+	close(cfd);
+	close(sfd);
+}
+
 void test_setget_sockopt(void)
 {
 	cg_fd = test__join_cgroup(CG_NAME);
@@ -118,6 +189,8 @@ void test_setget_sockopt(void)
 	test_tcp(AF_INET);
 	test_udp(AF_INET6);
 	test_udp(AF_INET);
+	test_ktls(AF_INET6);
+	test_ktls(AF_INET);
 
 done:
 	setget_sockopt__destroy(skel);
diff --git a/tools/testing/selftests/bpf/progs/setget_sockopt.c b/tools/testing/selftests/bpf/progs/setget_sockopt.c
index 9523333b8905..7a438600ae98 100644
--- a/tools/testing/selftests/bpf/progs/setget_sockopt.c
+++ b/tools/testing/selftests/bpf/progs/setget_sockopt.c
@@ -22,6 +22,7 @@ int nr_active;
 int nr_connect;
 int nr_binddev;
 int nr_socket_post_create;
+int nr_fin_wait1;
 
 struct sockopt_test {
 	int opt;
@@ -386,6 +387,13 @@ int skops_sockopt(struct bpf_sock_ops *skops)
 		nr_passive += !(bpf_test_sockopt(skops, sk) ||
 				test_tcp_maxseg(skops, sk) ||
 				test_tcp_saved_syn(skops, sk));
+		bpf_sock_ops_cb_flags_set(skops,
+					  skops->bpf_sock_ops_cb_flags |
+					  BPF_SOCK_OPS_STATE_CB_FLAG);
+		break;
+	case BPF_SOCK_OPS_STATE_CB:
+		if (skops->args[1] == BPF_TCP_CLOSE_WAIT)
+			nr_fin_wait1 += !bpf_test_sockopt(skops, sk);
 		break;
 	}
 
-- 
2.30.2

Re: [PATCH bpf-next v3 2/2] selftests/bpf: Calls bpf_setsockopt() on a ktls enabled socket.

[Martin KaFai Lau]

On 1/25/23 12:16 PM, Kui-Feng Lee wrote:
> +static void test_ktls(int family)
> +{
> +	struct tls12_crypto_info_aes_gcm_128 aes128;
> +	struct setget_sockopt__bss *bss = skel->bss;
> +	int cfd = -1, sfd = -1, fd = -1, ret;
> +
> +	memset(bss, 0, sizeof(*bss));
> +
> +	sfd = start_server(family, SOCK_STREAM,
> +			   family == AF_INET6 ? addr6_str : addr4_str, 0, 0);
> +	if (!ASSERT_GE(sfd, 0, "start_server"))
> +		return;
> +	fd = connect_to_fd(sfd, 0);
> +	if (!ASSERT_GE(fd, 0, "connect_to_fd"))
> +		goto err_out;
> +
> +	cfd = accept(sfd, NULL, 0);
> +	if (!ASSERT_GE(cfd, 0, "accept"))
> +		goto err_out;
> +
> +	close(sfd);
> +	sfd = -1;
> +
> +	/* Setup KTLS */
> +	ret = setsockopt(fd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
> +	if (!ASSERT_OK(ret, "setsockopt"))
> +		goto err_out;
> +	ret = setsockopt(cfd, IPPROTO_TCP, TCP_ULP, "tls", sizeof("tls"));
> +	if (!ASSERT_OK(ret, "setsockopt"))
> +		goto err_out;
> +
> +	memset(&aes128, 0, sizeof(aes128));
> +	aes128.info.version = TLS_1_2_VERSION;
> +	aes128.info.cipher_type = TLS_CIPHER_AES_GCM_128;
> +
> +	ret = setsockopt(fd, SOL_TLS, TLS_TX, &aes128, sizeof(aes128));
> +	if (!ASSERT_OK(ret, "setsockopt"))
> +		goto err_out;
> +
> +	ret = setsockopt(cfd, SOL_TLS, TLS_RX, &aes128, sizeof(aes128));
> +	if (!ASSERT_OK(ret, "setsockopt"))
> +		goto err_out;
> +
> +	/* KTLS is enabled */
> +
> +	close(fd);
> +	/* At this point, the cfd socket is at the CLOSE_WAIT state
> +	 * and still run TLS protocol.  The test for
> +	 * BPF_TCP_CLOSE_WAIT should be run at this point.
> +	 */
> +	char buf[1];

This checkpatch warning is reasonable:

https://patchwork.kernel.org/project/netdevbpf/patch/20230125201608.908230-3-kuifeng@meta.com/

WARNING: Missing a blank line after declarations
#88: FILE: tools/testing/selftests/bpf/prog_tests/setget_sockopt.c:138:
+	char buf[1];
+	ret = read(cfd, buf, 1);

I fixed it up and take this chance to move it to the beginning of the function. 
Applied. Thanks.


> +	ret = read(cfd, buf, 1);
> +	ASSERT_EQ(ret, 0, "read");
> +	close(cfd);

Re: [PATCH bpf-next v3 2/2] selftests/bpf: Calls bpf_setsockopt() on a ktls enabled socket.

[Kui-Feng Lee]

On 1/25/23 15:17, Martin KaFai Lau wrote:
> On 1/25/23 12:16 PM, Kui-Feng Lee wrote:
>> +    close(fd);
>> +    /* At this point, the cfd socket is at the CLOSE_WAIT state
>> +     * and still run TLS protocol.  The test for
>> +     * BPF_TCP_CLOSE_WAIT should be run at this point.
>> +     */
>> +    char buf[1];
>
> This checkpatch warning is reasonable:
>
> https://patchwork.kernel.org/project/netdevbpf/patch/20230125201608.908230-3-kuifeng@meta.com/ 
>
>
> WARNING: Missing a blank line after declarations
> #88: FILE: tools/testing/selftests/bpf/prog_tests/setget_sockopt.c:138:
> +    char buf[1];
> +    ret = read(cfd, buf, 1);
>
> I fixed it up and take this chance to move it to the beginning of the 
> function. Applied. Thanks.


Thank you for fixing that and reviewing!


>
>
>> +    ret = read(cfd, buf, 1);
>> +    ASSERT_EQ(ret, 0, "read");
>> +    close(cfd);
>

Re: [PATCH bpf-next v3 0/2] Enable bpf_setsockopt() on ktls enabled sockets.

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (master)
by Martin KaFai Lau <martin.lau@kernel.org>:

On Wed, 25 Jan 2023 12:16:06 -0800 you wrote:
> This patchset implements a change to bpf_setsockopt() which allows
> ktls enabled sockets to be used with the SOL_TCP level. This is
> necessary as when ktls is enabled, it changes the function pointer of
> setsockopt of the socket, which bpf_setsockopt() checks in order to
> make sure that the socket is a TCP socket. Checking sk_protocol
> instead of the function pointer will ensure that bpf_setsockopt() with
> the SOL_TCP level still works on sockets with ktls enabled.
> 
> [...]

Here is the summary with links:
  - [bpf-next,v3,1/2] bpf: Check the protocol of a sock to agree the calls to bpf_setsockopt().
    https://git.kernel.org/bpf/bpf-next/c/2ab42c7b871f
  - [bpf-next,v3,2/2] selftests/bpf: Calls bpf_setsockopt() on a ktls enabled socket.
    https://git.kernel.org/bpf/bpf-next/c/d1246f936023

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html