[RFC net-next 0/3] selftests: rds selftest

[RFC net-next 0/3] selftests: rds selftest

[allison.henderson]

From: Allison Henderson <allison.henderson@oracle.com>

Hi All,

This series is a new selftest that Vegard, Chuck and myself have been working on to provide some test coverage for rds.  It still has a few bugs to work out, so it's not quite ready for submission yet.  But we thought an RFC would be a good idea just to collect some feed back to see what people think.  Some things to be aware of that we are still working through:

  Occasionally we see intermittent hangs during the send and recv.
  So we still need to track down and address the cause

  We may still add a time out to handle unexpected hangs

  We've been having some trouble getting gcov to generate a report if
  the version of gcov doesn't match the version of gcc installed.  So
  we may further adapt the test to omit the coverage report if the
  required dependencies are not met

  Some distros are still not collecting meaningfull gcda data and
  generate an empty report, so we are still trying to figure out the
  cause of that

  May still add more exit codes for PASS/FAIL/BROKEN/CONFIG conditions

Questions and comments appreciated.  Thanks everyone!

Allison

Vegard Nossum (3):
  .gitignore: add .gcda files
  net: rds: add option for GCOV profiling
  selftests: rds: add testing infrastructure

 .gitignore                                 |   1 +
 MAINTAINERS                                |   1 +
 net/rds/Kconfig                            |   9 +
 net/rds/Makefile                           |   5 +
 tools/testing/selftests/Makefile           |   1 +
 tools/testing/selftests/net/rds/Makefile   |  13 ++
 tools/testing/selftests/net/rds/README.txt |  15 ++
 tools/testing/selftests/net/rds/config.sh  |  33 +++
 tools/testing/selftests/net/rds/init.sh    |  49 +++++
 tools/testing/selftests/net/rds/run.sh     | 168 ++++++++++++++
 tools/testing/selftests/net/rds/test.py    | 244 +++++++++++++++++++++
 11 files changed, 539 insertions(+)
 create mode 100644 tools/testing/selftests/net/rds/Makefile
 create mode 100644 tools/testing/selftests/net/rds/README.txt
 create mode 100755 tools/testing/selftests/net/rds/config.sh
 create mode 100755 tools/testing/selftests/net/rds/init.sh
 create mode 100755 tools/testing/selftests/net/rds/run.sh
 create mode 100644 tools/testing/selftests/net/rds/test.py

-- 
2.25.1

[RFC net-next 1/3] .gitignore: add .gcda files

[allison.henderson]

From: Vegard Nossum <vegard.nossum@oracle.com>

These files contain the runtime coverage data generated by gcov.

Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index c59dc60ba62e..8ff1b4138c56 100644
--- a/.gitignore
+++ b/.gitignore
@@ -24,6 +24,7 @@
 *.dwo
 *.elf
 *.gcno
+*.gcda
 *.gz
 *.i
 *.ko
-- 
2.25.1

[RFC net-next 2/3] net: rds: add option for GCOV profiling

[allison.henderson]

From: Vegard Nossum <vegard.nossum@oracle.com>

This commit basically implements what ftrace does with
CONFIG_GCOV_PROFILE_FTRACE for RDS.

We need this in order to better be able to understand what code paths
our unit tests are hitting (or not hitting).

Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
---
 net/rds/Kconfig  | 9 +++++++++
 net/rds/Makefile | 5 +++++
 2 files changed, 14 insertions(+)

diff --git a/net/rds/Kconfig b/net/rds/Kconfig
index 75cd696963b2..f007730aa2bb 100644
--- a/net/rds/Kconfig
+++ b/net/rds/Kconfig
@@ -26,3 +26,12 @@ config RDS_DEBUG
 	bool "RDS debugging messages"
 	depends on RDS
 	default n
+
+config GCOV_PROFILE_RDS
+	bool "Enable GCOV profiling on RDS"
+	depends on GCOV_KERNEL
+	help
+	  Enable GCOV profiling on RDS for checking which functions/lines
+	  are executed.
+
+	  If unsure, say N.
diff --git a/net/rds/Makefile b/net/rds/Makefile
index 8fdc118e2927..3af1ca1d965c 100644
--- a/net/rds/Makefile
+++ b/net/rds/Makefile
@@ -15,3 +15,8 @@ rds_tcp-y :=		tcp.o tcp_connect.o tcp_listen.o tcp_recv.o \
 			tcp_send.o tcp_stats.o
 
 ccflags-$(CONFIG_RDS_DEBUG)	:=	-DRDS_DEBUG
+
+# for GCOV coverage profiling
+ifdef CONFIG_GCOV_PROFILE_RDS
+GCOV_PROFILE := y
+endif
-- 
2.25.1

[RFC net-next 3/3] selftests: rds: add testing infrastructure

[allison.henderson]

From: Vegard Nossum <vegard.nossum@oracle.com>

This adds some basic self-testing infrastructure for RDS.

Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 MAINTAINERS                                |   1 +
 tools/testing/selftests/Makefile           |   1 +
 tools/testing/selftests/net/rds/Makefile   |  13 ++
 tools/testing/selftests/net/rds/README.txt |  15 ++
 tools/testing/selftests/net/rds/config.sh  |  33 +++
 tools/testing/selftests/net/rds/init.sh    |  49 +++++
 tools/testing/selftests/net/rds/run.sh     | 168 ++++++++++++++
 tools/testing/selftests/net/rds/test.py    | 244 +++++++++++++++++++++
 8 files changed, 524 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7538152be2f1..009766b77b7e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18851,6 +18851,7 @@ S:	Supported
 W:	https://oss.oracle.com/projects/rds/
 F:	Documentation/networking/rds.rst
 F:	net/rds/
+F:	tools/testing/selftests/net/rds/
 
 RDT - RESOURCE ALLOCATION
 M:	Fenghua Yu <fenghua.yu@intel.com>
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 9039f3709aff..5b01fe3277e2 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -66,6 +66,7 @@ TARGETS += net/mptcp
 TARGETS += net/openvswitch
 TARGETS += net/tcp_ao
 TARGETS += net/netfilter
+TARGETS += net/rds
 TARGETS += nsfs
 TARGETS += perf_events
 TARGETS += pidfd
diff --git a/tools/testing/selftests/net/rds/Makefile b/tools/testing/selftests/net/rds/Makefile
new file mode 100644
index 000000000000..52fe54006eba
--- /dev/null
+++ b/tools/testing/selftests/net/rds/Makefile
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0
+
+all:
+	@echo mk_build_dir="$(shell pwd)" > include.sh
+
+TEST_PROGS := run.sh \
+	include.sh \
+	test.py \
+	init.sh
+
+EXTRA_CLEAN := /tmp/rds_logs
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/net/rds/README.txt b/tools/testing/selftests/net/rds/README.txt
new file mode 100644
index 000000000000..e46f980adb3e
--- /dev/null
+++ b/tools/testing/selftests/net/rds/README.txt
@@ -0,0 +1,15 @@
+RDS self-tests
+==============
+
+Usage:
+
+    # create a suitable .config
+    tools/testing/selftests/net/rds/config.sh
+
+    # build the kernel
+    make -j128
+
+    # launch the tests in a VM
+    tools/testing/selftests/net/rds/run.sh
+
+An HTML coverage report will be output in /tmp/rds_logs/coverage/.
diff --git a/tools/testing/selftests/net/rds/config.sh b/tools/testing/selftests/net/rds/config.sh
new file mode 100755
index 000000000000..c2c36756ba1f
--- /dev/null
+++ b/tools/testing/selftests/net/rds/config.sh
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0
+#! /bin/bash
+
+set -e
+set -u
+set -x
+
+unset KBUILD_OUTPUT
+
+# start with a default config
+make defconfig
+
+# no modules
+scripts/config --disable CONFIG_MODULES
+
+# enable RDS
+scripts/config --enable CONFIG_RDS
+scripts/config --enable CONFIG_RDS_TCP
+
+# instrument RDS and only RDS
+scripts/config --enable CONFIG_GCOV_KERNEL
+scripts/config --disable GCOV_PROFILE_ALL
+scripts/config --enable GCOV_PROFILE_RDS
+
+# need network namespaces to run tests with veth network interfaces
+scripts/config --enable CONFIG_NET_NS
+scripts/config --enable CONFIG_VETH
+
+# simulate packet loss
+scripts/config --enable CONFIG_NET_SCH_NETEM
+
+# generate real .config without asking any questions
+make olddefconfig
diff --git a/tools/testing/selftests/net/rds/init.sh b/tools/testing/selftests/net/rds/init.sh
new file mode 100755
index 000000000000..a29e3de81ed5
--- /dev/null
+++ b/tools/testing/selftests/net/rds/init.sh
@@ -0,0 +1,49 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+set -u
+
+LOG_DIR=/tmp
+PY_CMD="/usr/bin/python3"
+while getopts "d:p:" opt; do
+  case ${opt} in
+    d)
+      LOG_DIR=${OPTARG}
+      ;;
+    p)
+      PY_CMD=${OPTARG}
+      ;;
+    :)
+      echo "USAGE: init.sh [-d logdir] [-p python_cmd]"
+      exit 1
+      ;;
+    ?)
+      echo "Invalid option: -${OPTARG}."
+      exit 1
+      ;;
+  esac
+done
+
+LOG_FILE=$LOG_DIR/rds-strace.txt
+
+mount -t proc none /proc
+mount -t sysfs none /sys
+mount -t tmpfs none /var/run
+mount -t debugfs none /sys/kernel/debug
+
+echo running RDS tests...
+echo Traces will be logged to $LOG_FILE
+rm -f $LOG_FILE
+strace -T -tt -o "$LOG_FILE" $PY_CMD $(dirname "$0")/test.py -d "$LOG_DIR" || true
+
+echo saving coverage data...
+(set +x; cd /sys/kernel/debug/gcov; find * -name '*.gcda' | \
+while read f
+do
+	cat < /sys/kernel/debug/gcov/$f > /$f
+done)
+
+dmesg > $LOG_DIR/dmesg.out
+
+/usr/sbin/poweroff --no-wtmp --force
diff --git a/tools/testing/selftests/net/rds/run.sh b/tools/testing/selftests/net/rds/run.sh
new file mode 100755
index 000000000000..d77ded7c8e98
--- /dev/null
+++ b/tools/testing/selftests/net/rds/run.sh
@@ -0,0 +1,168 @@
+# SPDX-License-Identifier: GPL-2.0
+#! /bin/bash
+
+set -e
+set -u
+
+unset KBUILD_OUTPUT
+
+current_dir="$(realpath "$(dirname "$0")")"
+build_dir=$current_dir
+
+build_include="$current_dir/include.sh"
+if test -f "$build_include"; then
+	# this include will define "$mk_build_dir" as the location the test was
+	# built.  We will need this if the tests are installed in a location
+	# other than the kernel source
+
+	source $build_include
+	build_dir=$mk_build_dir
+fi
+
+# This test requires kernel source and the *.gcda data therein
+# Locate the top level of the kernel source, and the net/rds
+# subfolder with the appropriate *.gcno object files
+ksrc_dir="$(realpath $build_dir/../../../../../)"
+kconfig="$ksrc_dir/.config"
+obj_dir="$ksrc_dir/net/rds"
+
+GCOV_CMD=gcov
+
+# This script currently only works for x86_64
+ARCH="$(uname -m)"
+case "${ARCH}" in
+x86_64)
+	QEMU_BINARY=qemu-system-x86_64
+	;;
+*)
+	echo "selftests: [SKIP] Unsupported architecture"
+	exit 4
+	;;
+esac
+
+# Kselftest framework requirement - SKIP code is 4.
+check_conf_enabled() {
+	if ! grep -x "$1=y" $kconfig > /dev/null 2>&1; then
+		echo selftests: [SKIP] This test requires $1 enabled
+		echo Please run tools/testing/selftests/net/rds/config.sh and rebuild the kernel
+		exit 4
+	fi
+}
+check_conf_disabled() {
+	if grep -x "$1=y" $kconfig > /dev/null 2>&1; then
+		echo selftests: [SKIP] This test requires $1 disabled
+		echo Please run tools/testing/selftests/net/rds/config.sh and rebuild the kernel
+		exit 4
+	fi
+}
+check_conf() {
+	check_conf_enabled CONFIG_NET_SCH_NETEM
+	check_conf_enabled CONFIG_VETH
+	check_conf_enabled CONFIG_NET_NS
+	check_conf_enabled CONFIG_GCOV_PROFILE_RDS
+	check_conf_enabled CONFIG_GCOV_KERNEL
+	check_conf_enabled CONFIG_RDS_TCP
+	check_conf_enabled CONFIG_RDS
+	check_conf_disabled CONFIG_MODULES
+	check_conf_disabled CONFIG_GCOV_PROFILE_ALL
+}
+
+check_env()
+{
+	if ! test -d $obj_dir; then
+		echo "selftests: [SKIP] This test requires a kernel source tree"
+		exit 4
+	fi
+	if ! test -e $kconfig; then
+		echo "selftests: [SKIP] This test requires a configured kernel source tree"
+		exit 4
+	fi
+	if ! which strace > /dev/null 2>&1; then
+		echo "selftests: [SKIP] Could not run test without strace"
+		exit 4
+	fi
+	if ! which tcpdump > /dev/null 2>&1; then
+		echo "selftests: [SKIP] Could not run test without tcpdump"
+		exit 4
+	fi
+	if ! which $GCOV_CMD > /dev/null 2>&1; then
+		echo "selftests: [SKIP] Could not run with out gcov. "
+		exit 4
+	fi
+
+	# the gcov version much match the gcc version
+	GCC_VER=`gcc -dumpfullversion`
+	GCOV_VER=`$GCOV_CMD -v | grep gcov | awk '{print $3}'| awk 'BEGIN {FS="-"}{print $1}'`
+	if [ "$GCOV_VER" != "$GCC_VER" ]; then
+		#attempt to find a matching gcov version
+		GCOV_CMD=gcov-`gcc -dumpversion`
+
+		if ! which $GCOV_CMD > /dev/null 2>&1; then
+			echo "selftests: [SKIP] gcov version must match gcc version"
+			exit 4
+		fi
+
+		#recheck version number of found gcov executable
+		GCOV_VER=`$GCOV_CMD -v | grep gcov | awk '{print $3}'| \
+			  awk 'BEGIN {FS="-"}{print $1}'`
+		if [ "$GCOV_VER" != "$GCC_VER" ]; then
+			echo "selftests: [SKIP] gcov version must match gcc version"
+			exit 4
+		fi
+	fi
+
+	if ! which gcovr > /dev/null 2>&1; then
+		echo "selftests: [SKIP] Could not run test without gcovr"
+		exit 4
+	fi
+
+	if ! which $QEMU_BINARY > /dev/null 2>&1; then
+		echo "selftests: [SKIP] Could not run test without qemu"
+		exit 4
+	fi
+
+	if ! which python3 > /dev/null 2>&1; then
+		echo "selftests: [SKIP] Could not run test without python3"
+		exit 4
+	fi
+
+	python_major=`python3 -c "import sys; print(sys.version_info[0])"`
+	python_minor=`python3 -c "import sys; print(sys.version_info[1])"`
+	if [[ python_major -lt 3 || ( python_major -eq 3 && python_minor -lt 9 ) ]] ; then
+		echo "selftests: [SKIP] Could not run test without at least python3.9"
+		python3 -V
+		exit 4
+	fi
+}
+
+check_env
+check_conf
+
+#if we are running in a python environment, we need to capture that
+#python bin so we can use the same python environment in the vm
+PY_CMD=`which python3`
+
+LOG_DIR=/tmp/rds_logs
+mkdir -p  $LOG_DIR
+
+# start a VM using a 9P root filesystem that maps to the host's /
+# we pass ./init.sh from the same directory as we are in as the
+# guest's init, which will run the tests and copy the coverage
+# data back to the host filesystem.
+$QEMU_BINARY \
+	-enable-kvm \
+	-cpu host \
+	-smp 4 \
+	-kernel ${ksrc_dir}/arch/x86/boot/bzImage \
+	-append "rootfstype=9p root=/dev/root rootflags=trans=virtio,version=9p2000.L rw \
+		console=ttyS0 init=${current_dir}/init.sh -d ${LOG_DIR} -p ${PY_CMD}" \
+	-display none \
+	-serial stdio \
+	-fsdev local,id=fsdev0,path=/,security_model=none,multidevs=remap \
+	-device virtio-9p-pci,fsdev=fsdev0,mount_tag=/dev/root \
+	-no-reboot
+
+# generate a nice HTML coverage report
+echo running gcovr...
+gcovr -v -s --html-details --gcov-executable $GCOV_CMD --gcov-ignore-parse-errors \
+	-o $LOG_DIR/coverage/ "${ksrc_dir}/net/rds/"
diff --git a/tools/testing/selftests/net/rds/test.py b/tools/testing/selftests/net/rds/test.py
new file mode 100644
index 000000000000..8a06e871e368
--- /dev/null
+++ b/tools/testing/selftests/net/rds/test.py
@@ -0,0 +1,244 @@
+# SPDX-License-Identifier: GPL-2.0
+#! /usr/bin/env python3
+
+import argparse
+import ctypes
+import errno
+import hashlib
+import os
+import select
+import signal
+import socket
+import subprocess
+import sys
+import atexit
+from pwd import getpwuid
+from os import stat
+
+libc = ctypes.cdll.LoadLibrary('libc.so.6')
+setns = libc.setns
+
+net0 = 'net0'
+net1 = 'net1'
+
+veth0 = 'veth0'
+veth1 = 'veth1'
+
+# Convenience wrapper function for calling the subsystem ip command.
+def ip(*args):
+    subprocess.check_call(['/usr/sbin/ip'] + list(args))
+
+# Helper function for creating a socket inside a network namespace.
+# We need this because otherwise RDS will detect that the two TCP
+# sockets are on the same interface and use the loop transport instead
+# of the TCP transport.
+def netns_socket(netns, *args):
+    u0, u1 = socket.socketpair(socket.AF_UNIX, socket.SOCK_SEQPACKET)
+
+    child = os.fork()
+    if child == 0:
+        # change network namespace
+        with open(f'/var/run/netns/{netns}') as f:
+            try:
+                ret = setns(f.fileno(), 0)
+            except IOError as e:
+                print(e.errno)
+                print(e)
+
+        # create socket in target namespace
+        s = socket.socket(*args)
+
+        # send resulting socket to parent
+        socket.send_fds(u0, [], [s.fileno()])
+
+        sys.exit(0)
+
+    # receive socket from child
+    _, s, _, _ = socket.recv_fds(u1, 0, 1)
+    os.waitpid(child, 0)
+    u0.close()
+    u1.close()
+    return socket.fromfd(s[0], *args)
+
+#Parse out command line arguments.  We take an optional
+# timeout parameter and an optional log output folder
+parser = argparse.ArgumentParser(description="init script args",
+                                 formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+parser.add_argument("-d", "--logdir", action="store", help="directory to store logs", default="/tmp")
+parser.add_argument('--timeout', type=int, default=0)
+args = parser.parse_args()
+logdir=args.logdir
+
+ip('netns', 'add', net0)
+ip('netns', 'add', net1)
+ip('link', 'add', 'type', 'veth')
+
+addrs = [
+    # we technically don't need different port numbers, but this will
+    # help identify traffic in the network analyzer
+    ('10.0.0.1', 10000),
+    ('10.0.0.2', 20000),
+]
+
+# move interfaces to separate namespaces so they can no longer be
+# bound directly; this prevents rds from switching over from the tcp
+# transport to the loop transport.
+ip('link', 'set', veth0, 'netns', net0, 'up')
+ip('link', 'set', veth1, 'netns', net1, 'up')
+
+# add addresses
+ip('-n', net0, 'addr', 'add', addrs[0][0] + '/32', 'dev', veth0)
+ip('-n', net1, 'addr', 'add', addrs[1][0] + '/32', 'dev', veth1)
+
+# add routes
+ip('-n', net0, 'route', 'add', addrs[1][0] + '/32', 'dev', veth0)
+ip('-n', net1, 'route', 'add', addrs[0][0] + '/32', 'dev', veth1)
+
+# sanity check that our two interfaces/addresses are correctly set up
+# and communicating by doing a single ping
+ip('netns', 'exec', net0, 'ping', '-c', '1', addrs[1][0])
+
+# Start a packet capture on each network
+for net in [net0, net1]:
+    tcpdump_pid = os.fork()
+    if tcpdump_pid == 0:
+        pcap = logdir+'/'+net+'.pcap'
+        subprocess.check_call(['touch', pcap])
+        user = getpwuid(stat(pcap).st_uid).pw_name
+        ip('netns', 'exec', net, '/usr/sbin/tcpdump', '-Z', user, '-i', 'any', '-w', pcap)
+        sys.exit(0)
+
+# simulate packet loss, duplication and corruption
+for net, iface in [(net0, veth0), (net1, veth1)]:
+    ip('netns', 'exec', net,
+        '/usr/sbin/tc', 'qdisc', 'add', 'dev', iface, 'root', 'netem',
+        'corrupt', '5%',
+        'loss', '5%',
+        'duplicate', '5%',
+    )
+
+# add a timeout
+if args.timeout > 0:
+    signal.alarm(args.timeout)
+
+sockets = [
+    netns_socket(net0, socket.AF_RDS, socket.SOCK_SEQPACKET),
+    netns_socket(net1, socket.AF_RDS, socket.SOCK_SEQPACKET),
+]
+
+for s, addr in zip(sockets, addrs):
+    s.bind(addr)
+    s.setblocking(0)
+
+fileno_to_socket = {
+    s.fileno(): s for s in sockets
+}
+
+addr_to_socket = {
+    addr: s for addr, s in zip(addrs, sockets)
+}
+
+socket_to_addr = {
+    s: addr for addr, s in zip(addrs, sockets)
+}
+
+send_hashes = {}
+recv_hashes = {}
+
+ep = select.epoll()
+
+for s in sockets:
+    ep.register(s, select.EPOLLRDNORM)
+
+n = 50000
+nr_send = 0
+nr_recv = 0
+
+while nr_send < n:
+    # Send as much as we can without blocking
+    print("sending...", nr_send, nr_recv)
+    while nr_send < n:
+        send_data = hashlib.sha256(f'packet {nr_send}'.encode('utf-8')).hexdigest().encode('utf-8')
+
+        # pseudo-random send/receive pattern
+        sender = sockets[nr_send % 2]
+        receiver = sockets[1 - (nr_send % 3) % 2]
+
+        try:
+            sender.sendto(send_data, socket_to_addr[receiver])
+            send_hashes.setdefault((sender.fileno(), receiver.fileno()), hashlib.sha256()).update(f'<{send_data}>'.encode('utf-8'))
+            nr_send = nr_send + 1
+        except BlockingIOError as e:
+            break
+        except OSError as e:
+            if e.errno in [errno.ENOBUFS, errno.ECONNRESET, errno.EPIPE]:
+                break
+            raise
+
+    # Receive as much as we can without blocking
+    print("receiving...", nr_send, nr_recv)
+    while nr_recv < nr_send:
+        for fileno, eventmask in ep.poll():
+            receiver = fileno_to_socket[fileno]
+
+            if eventmask & select.EPOLLRDNORM:
+                while True:
+                    try:
+                        recv_data, address = receiver.recvfrom(1024)
+                        sender = addr_to_socket[address]
+                        recv_hashes.setdefault((sender.fileno(), receiver.fileno()), hashlib.sha256()).update(f'<{recv_data}>'.encode('utf-8'))
+                        nr_recv = nr_recv + 1
+                    except BlockingIOError as e:
+                        break
+
+    # exercise net/rds/tcp.c:rds_tcp_sysctl_reset()
+    for net in [net0, net1]:
+        ip('netns', 'exec', net, '/usr/sbin/sysctl', 'net.rds.tcp.rds_tcp_rcvbuf=10000')
+        ip('netns', 'exec', net, '/usr/sbin/sysctl', 'net.rds.tcp.rds_tcp_sndbuf=10000')
+
+print("done", nr_send, nr_recv)
+
+# the Python socket module doesn't know these
+RDS_INFO_FIRST = 10000
+RDS_INFO_LAST = 10017
+
+nr_success = 0
+nr_error = 0
+
+for s in sockets:
+    for optname in range(RDS_INFO_FIRST, RDS_INFO_LAST + 1):
+        # Sigh, the Python socket module doesn't allow us to pass
+        # buffer lengths greater than 1024 for some reason. RDS
+        # wants multiple pages.
+        try:
+            s.getsockopt(socket.SOL_RDS, optname, 1024)
+            nr_success = nr_success + 1
+        except OSError as e:
+            nr_error = nr_error + 1
+            if e.errno == errno.ENOSPC:
+                # ignore
+                pass
+
+print(f"getsockopt(): {nr_success}/{nr_error}")
+
+print("Stopping network packet captures")
+subprocess.check_call(['killall', '-q', 'tcpdump'])
+
+# We're done sending and receiving stuff, now let's check if what
+# we received is what we sent.
+for (sender, receiver), send_hash in send_hashes.items():
+    recv_hash = recv_hashes.get((sender, receiver))
+
+    if recv_hash is None:
+        print("FAIL: No data received")
+        sys.exit(1)
+
+    if send_hash.hexdigest() != recv_hash.hexdigest():
+        print("FAIL: Send/recv mismatch")
+        print("hash expected:", send_hash.hexdigest())
+        print("hash received:", recv_hash.hexdigest())
+        sys.exit(1)
+
+    print(f"{sender}/{receiver}: ok")
+
+print("Success")
-- 
2.25.1

Re: [RFC net-next 3/3] selftests: rds: add testing infrastructure

[Simon Horman]

On Thu, Jun 06, 2024 at 05:36:31PM -0700, allison.henderson@oracle.com wrote:
> From: Vegard Nossum <vegard.nossum@oracle.com>
> 
> This adds some basic self-testing infrastructure for RDS.
> 
> Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Hi Allison,

Some minor nits from my side.

> diff --git a/tools/testing/selftests/net/rds/config.sh b/tools/testing/selftests/net/rds/config.sh
> new file mode 100755
> index 000000000000..c2c36756ba1f
> --- /dev/null
> +++ b/tools/testing/selftests/net/rds/config.sh
> @@ -0,0 +1,33 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#! /bin/bash

#! line needs to be the first line for it to take effect

Flagged by shellcheck.

https://www.shellcheck.net/wiki/SC1128

> +
> +set -e
> +set -u
> +set -x
> +
> +unset KBUILD_OUTPUT
> +
> +# start with a default config
> +make defconfig
> +
> +# no modules
> +scripts/config --disable CONFIG_MODULES
> +
> +# enable RDS
> +scripts/config --enable CONFIG_RDS
> +scripts/config --enable CONFIG_RDS_TCP
> +
> +# instrument RDS and only RDS
> +scripts/config --enable CONFIG_GCOV_KERNEL
> +scripts/config --disable GCOV_PROFILE_ALL
> +scripts/config --enable GCOV_PROFILE_RDS
> +
> +# need network namespaces to run tests with veth network interfaces
> +scripts/config --enable CONFIG_NET_NS
> +scripts/config --enable CONFIG_VETH
> +
> +# simulate packet loss
> +scripts/config --enable CONFIG_NET_SCH_NETEM
> +
> +# generate real .config without asking any questions
> +make olddefconfig
> diff --git a/tools/testing/selftests/net/rds/init.sh b/tools/testing/selftests/net/rds/init.sh
> new file mode 100755
> index 000000000000..a29e3de81ed5
> --- /dev/null
> +++ b/tools/testing/selftests/net/rds/init.sh
> @@ -0,0 +1,49 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +
> +set -e
> +set -u
> +
> +LOG_DIR=/tmp
> +PY_CMD="/usr/bin/python3"
> +while getopts "d:p:" opt; do
> +  case ${opt} in
> +    d)
> +      LOG_DIR=${OPTARG}
> +      ;;
> +    p)
> +      PY_CMD=${OPTARG}
> +      ;;
> +    :)
> +      echo "USAGE: init.sh [-d logdir] [-p python_cmd]"
> +      exit 1
> +      ;;
> +    ?)
> +      echo "Invalid option: -${OPTARG}."
> +      exit 1
> +      ;;
> +  esac
> +done
> +
> +LOG_FILE=$LOG_DIR/rds-strace.txt
> +
> +mount -t proc none /proc
> +mount -t sysfs none /sys
> +mount -t tmpfs none /var/run
> +mount -t debugfs none /sys/kernel/debug
> +
> +echo running RDS tests...
> +echo Traces will be logged to $LOG_FILE
> +rm -f $LOG_FILE
> +strace -T -tt -o "$LOG_FILE" $PY_CMD $(dirname "$0")/test.py -d "$LOG_DIR" || true

Perhaps it can't occur, but I don't think this will behave as
expected if the out put of dirname includes spaces.

Flagged by shellecheck.
https://www.shellcheck.net/wiki/SC2046

Also, $LOG_DIR is quoted here, but not elsewhere, which seems inconsistent.

> +
> +echo saving coverage data...
> +(set +x; cd /sys/kernel/debug/gcov; find * -name '*.gcda' | \

shellcheck warns that:

SC2035 (info): Use ./*glob* or -- *glob* so names with dashes won't become options. 

https://www.shellcheck.net/wiki/SC2035

Although I guess in practice there are no filenames with dashes in
that directory.

> +while read f
> +do
> +	cat < /sys/kernel/debug/gcov/$f > /$f

Again, I guess it doesn't occur in practice.
But should $f be quoted in case it includes whitespace?

> +done)
> +
> +dmesg > $LOG_DIR/dmesg.out
> +
> +/usr/sbin/poweroff --no-wtmp --force

...

Re: [RFC net-next 0/3] selftests: rds selftest

[Allison Henderson]

On Thu, 2024-06-06 at 17:36 -0700, allison.henderson@oracle.com wrote:
+cc Chuck Lever, Peter Oberparleiter, Vegard Nossum, rds-devel

> From: Allison Henderson <allison.henderson@oracle.com>
> 
> Hi All,
> 
> This series is a new selftest that Vegard, Chuck and myself have been
> working on to provide some test coverage for rds.  It still has a few
> bugs to work out, so it's not quite ready for submission yet.  But we
> thought an RFC would be a good idea just to collect some feed back to
> see what people think.  Some things to be aware of that we are still
> working through:
> 
>   Occasionally we see intermittent hangs during the send and recv.
>   So we still need to track down and address the cause
> 
>   We may still add a time out to handle unexpected hangs
> 
>   We've been having some trouble getting gcov to generate a report if
>   the version of gcov doesn't match the version of gcc installed.  So
>   we may further adapt the test to omit the coverage report if the
>   required dependencies are not met
> 
>   Some distros are still not collecting meaningfull gcda data and
>   generate an empty report, so we are still trying to figure out the
>   cause of that
> 
>   May still add more exit codes for PASS/FAIL/BROKEN/CONFIG
> conditions
> 
> Questions and comments appreciated.  Thanks everyone!
> 
> Allison
> 
> Vegard Nossum (3):
>   .gitignore: add .gcda files
>   net: rds: add option for GCOV profiling
>   selftests: rds: add testing infrastructure
> 
>  .gitignore                                 |   1 +
>  MAINTAINERS                                |   1 +
>  net/rds/Kconfig                            |   9 +
>  net/rds/Makefile                           |   5 +
>  tools/testing/selftests/Makefile           |   1 +
>  tools/testing/selftests/net/rds/Makefile   |  13 ++
>  tools/testing/selftests/net/rds/README.txt |  15 ++
>  tools/testing/selftests/net/rds/config.sh  |  33 +++
>  tools/testing/selftests/net/rds/init.sh    |  49 +++++
>  tools/testing/selftests/net/rds/run.sh     | 168 ++++++++++++++
>  tools/testing/selftests/net/rds/test.py    | 244
> +++++++++++++++++++++
>  11 files changed, 539 insertions(+)
>  create mode 100644 tools/testing/selftests/net/rds/Makefile
>  create mode 100644 tools/testing/selftests/net/rds/README.txt
>  create mode 100755 tools/testing/selftests/net/rds/config.sh
>  create mode 100755 tools/testing/selftests/net/rds/init.sh
>  create mode 100755 tools/testing/selftests/net/rds/run.sh
>  create mode 100644 tools/testing/selftests/net/rds/test.py
> 

Re: [RFC net-next 1/3] .gitignore: add .gcda files

[Allison Henderson]

On Thu, 2024-06-06 at 17:36 -0700, allison.henderson@oracle.com wrote:
+cc Chuck Lever, Peter Oberparleiter, Vegard Nossum, rds-devel
> From: Vegard Nossum <vegard.nossum@oracle.com>
> 
> These files contain the runtime coverage data generated by gcov.
> 
> Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
> Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
> Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
> ---
>  .gitignore | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/.gitignore b/.gitignore
> index c59dc60ba62e..8ff1b4138c56 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -24,6 +24,7 @@
>  *.dwo
>  *.elf
>  *.gcno
> +*.gcda
>  *.gz
>  *.i
>  *.ko

Re: [RFC net-next 2/3] net: rds: add option for GCOV profiling

[Allison Henderson]

On Thu, 2024-06-06 at 17:36 -0700, allison.henderson@oracle.com wrote:
+cc Chuck Lever, Peter Oberparleiter, Vegard Nossum, rds-devel
> From: Vegard Nossum <vegard.nossum@oracle.com>
> 
> This commit basically implements what ftrace does with
> CONFIG_GCOV_PROFILE_FTRACE for RDS.
> 
> We need this in order to better be able to understand what code paths
> our unit tests are hitting (or not hitting).
> 
> Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> Reviewed-by: Allison Henderson <allison.henderson@oracle.com>
> Reviewed-by: Chuck Lever <chuck.lever@oracle.com>
> Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
> ---
>  net/rds/Kconfig  | 9 +++++++++
>  net/rds/Makefile | 5 +++++
>  2 files changed, 14 insertions(+)
> 
> diff --git a/net/rds/Kconfig b/net/rds/Kconfig
> index 75cd696963b2..f007730aa2bb 100644
> --- a/net/rds/Kconfig
> +++ b/net/rds/Kconfig
> @@ -26,3 +26,12 @@ config RDS_DEBUG
>         bool "RDS debugging messages"
>         depends on RDS
>         default n
> +
> +config GCOV_PROFILE_RDS
> +       bool "Enable GCOV profiling on RDS"
> +       depends on GCOV_KERNEL
> +       help
> +         Enable GCOV profiling on RDS for checking which
> functions/lines
> +         are executed.
> +
> +         If unsure, say N.
> diff --git a/net/rds/Makefile b/net/rds/Makefile
> index 8fdc118e2927..3af1ca1d965c 100644
> --- a/net/rds/Makefile
> +++ b/net/rds/Makefile
> @@ -15,3 +15,8 @@ rds_tcp-y :=          tcp.o tcp_connect.o
> tcp_listen.o tcp_recv.o \
>                         tcp_send.o tcp_stats.o
>  
>  ccflags-$(CONFIG_RDS_DEBUG)    :=      -DRDS_DEBUG
> +
> +# for GCOV coverage profiling
> +ifdef CONFIG_GCOV_PROFILE_RDS
> +GCOV_PROFILE := y
> +endif

Re: [RFC net-next 3/3] selftests: rds: add testing infrastructure

[Allison Henderson]

On Fri, 2024-06-07 at 21:24 +0100, Simon Horman wrote:
+cc Chuck Lever, Peter Oberparleiter, Vegard Nossum, rds-devel
> On Thu, Jun 06, 2024 at 05:36:31PM -0700,
> allison.henderson@oracle.com wrote:
> > From: Vegard Nossum <vegard.nossum@oracle.com>
> > 
> > This adds some basic self-testing infrastructure for RDS.
> > 
> > Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
> > Signed-off-by: Allison Henderson <allison.henderson@oracle.com>
> > Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
> 
> Hi Allison,
> 
> Some minor nits from my side.
> 
> > diff --git a/tools/testing/selftests/net/rds/config.sh
> > b/tools/testing/selftests/net/rds/config.sh
> > new file mode 100755
> > index 000000000000..c2c36756ba1f
> > --- /dev/null
> > +++ b/tools/testing/selftests/net/rds/config.sh
> > @@ -0,0 +1,33 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +#! /bin/bash
> 
> #! line needs to be the first line for it to take effect
> 
> Flagged by shellcheck.
> 
> https://urldefense.com/v3/__https://www.shellcheck.net/wiki/SC1128__;!!ACWV5N9M2RV99hQ!OjLySFP6wNEm6wWK90hVJHa9RXAP8310v8V0uKLQ9KvODoyD8hUHX0CRLZpKBl1jQGwfR-vYcBI5w8wRwAI$
>  
> 

Ah, alrighty I will go through and fix all the shell check nits

> > +
> > +set -e
> > +set -u
> > +set -x
> > +
> > +unset KBUILD_OUTPUT
> > +
> > +# start with a default config
> > +make defconfig
> > +
> > +# no modules
> > +scripts/config --disable CONFIG_MODULES
> > +
> > +# enable RDS
> > +scripts/config --enable CONFIG_RDS
> > +scripts/config --enable CONFIG_RDS_TCP
> > +
> > +# instrument RDS and only RDS
> > +scripts/config --enable CONFIG_GCOV_KERNEL
> > +scripts/config --disable GCOV_PROFILE_ALL
> > +scripts/config --enable GCOV_PROFILE_RDS
> > +
> > +# need network namespaces to run tests with veth network
> > interfaces
> > +scripts/config --enable CONFIG_NET_NS
> > +scripts/config --enable CONFIG_VETH
> > +
> > +# simulate packet loss
> > +scripts/config --enable CONFIG_NET_SCH_NETEM
> > +
> > +# generate real .config without asking any questions
> > +make olddefconfig
> > diff --git a/tools/testing/selftests/net/rds/init.sh
> > b/tools/testing/selftests/net/rds/init.sh
> > new file mode 100755
> > index 000000000000..a29e3de81ed5
> > --- /dev/null
> > +++ b/tools/testing/selftests/net/rds/init.sh
> > @@ -0,0 +1,49 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +set -e
> > +set -u
> > +
> > +LOG_DIR=/tmp
> > +PY_CMD="/usr/bin/python3"
> > +while getopts "d:p:" opt; do
> > +  case ${opt} in
> > +    d)
> > +      LOG_DIR=${OPTARG}
> > +      ;;
> > +    p)
> > +      PY_CMD=${OPTARG}
> > +      ;;
> > +    :)
> > +      echo "USAGE: init.sh [-d logdir] [-p python_cmd]"
> > +      exit 1
> > +      ;;
> > +    ?)
> > +      echo "Invalid option: -${OPTARG}."
> > +      exit 1
> > +      ;;
> > +  esac
> > +done
> > +
> > +LOG_FILE=$LOG_DIR/rds-strace.txt
> > +
> > +mount -t proc none /proc
> > +mount -t sysfs none /sys
> > +mount -t tmpfs none /var/run
> > +mount -t debugfs none /sys/kernel/debug
> > +
> > +echo running RDS tests...
> > +echo Traces will be logged to $LOG_FILE
> > +rm -f $LOG_FILE
> > +strace -T -tt -o "$LOG_FILE" $PY_CMD $(dirname "$0")/test.py -d
> > "$LOG_DIR" || true
> 
> Perhaps it can't occur, but I don't think this will behave as
> expected if the out put of dirname includes spaces.
> 
> Flagged by shellecheck.
> https://urldefense.com/v3/__https://www.shellcheck.net/wiki/SC2046__;!!ACWV5N9M2RV99hQ!OjLySFP6wNEm6wWK90hVJHa9RXAP8310v8V0uKLQ9KvODoyD8hUHX0CRLZpKBl1jQGwfR-vYcBI5005wM3k$
>  
> 
> Also, $LOG_DIR is quoted here, but not elsewhere, which seems
> inconsistent.

Noted, yes it should probably be quoted
> 
> > +
> > +echo saving coverage data...
> > +(set +x; cd /sys/kernel/debug/gcov; find * -name '*.gcda' | \
> 
> shellcheck warns that:
> 
> SC2035 (info): Use ./*glob* or -- *glob* so names with dashes won't
> become options. 
> 
> https://urldefense.com/v3/__https://www.shellcheck.net/wiki/SC2035__;!!ACWV5N9M2RV99hQ!OjLySFP6wNEm6wWK90hVJHa9RXAP8310v8V0uKLQ9KvODoyD8hUHX0CRLZpKBl1jQGwfR-vYcBI5Es_quwA$
>  
> 
> Although I guess in practice there are no filenames with dashes in
> that directory.
Not at the moment, but its always possible that such a file could be
introduced later.  Will fix :-)

> 
> > +while read f
> > +do
> > +       cat < /sys/kernel/debug/gcov/$f > /$f
> 
> Again, I guess it doesn't occur in practice.
> But should $f be quoted in case it includes whitespace?
Yes, I think so.  Even if it's not an issue now, it would be nice to
not deal will testcase breakage if file name were to change in the
future.  Will fix this too.

Thanks for the review! :-)
Allison
> 
> > +done)
> > +
> > +dmesg > $LOG_DIR/dmesg.out
> > +
> > +/usr/sbin/poweroff --no-wtmp --force
> 
> ...