* PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh
@ 2023-01-03 14:03 Mirsad Goran Todorovac
2023-01-03 14:28 ` Florian Westphal
0 siblings, 1 reply; 5+ messages in thread
From: Mirsad Goran Todorovac @ 2023-01-03 14:03 UTC (permalink / raw)
To: linux-kselftest; +Cc: LKML, Shuah Khan, Florian Westphal, Jakub Kicinski
[-- Attachment #1: Type: text/plain, Size: 8650 bytes --]
Hi all,
The `make kselftest` hangs in netfilter/nft_trans_stress.sh test when testing 6.2.0-rc2
(attempted 2 times, waiting half an hour at least):
make[2]: Leaving directory '/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/net/openvswitch'
make[2]: Entering directory '/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter'
TAP version 13
1..17
# selftests: netfilter: nft_trans_stress.sh
# PASS: nft add/delete test returned 0
# PASS: nft reload test returned 0
The test script seems to be stuck in ping flood:
%Cpu(s): 0.9 us, 21.5 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 77.6 si, 0.0 st
MiB Mem : 7734.0 total, 4624.5 free, 1966.2 used, 1143.2 buff/cache
MiB Swap: 10240.0 total, 9422.2 free, 817.8 used. 4946.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
602866 root 20 0 13048 2944 2816 R 51.4 0.0 6:31.68 ping -6 ::1 -fq
602869 root 20 0 13048 2944 2816 R 51.4 0.0 6:56.59 ping -6 ::1 -fq
602871 root 20 0 13048 2944 2816 R 51.4 0.0 6:47.24 ping -4 127.0.0.1 -fq
602872 root 20 0 13048 2944 2816 R 50.5 0.0 6:47.24 ping -6 ::1 -fq
602868 root 20 0 13048 2944 2816 R 49.5 0.0 6:56.60 ping -4 127.0.0.1 -fq
602856 root 20 0 13048 2944 2816 R 48.6 0.0 6:53.21 ping -4 127.0.0.1 -fq
602857 root 20 0 13048 2944 2816 R 48.6 0.0 6:53.21 ping -6 ::1 -fq
602865 root 20 0 13048 2944 2816 R 48.6 0.0 6:31.64 ping -4 127.0.0.1 -fq
602853 root 20 0 13048 2944 2816 R 47.7 0.0 6:42.07 ping -4 127.0.0.1 -fq
602863 root 20 0 13048 2944 2816 R 47.7 0.0 6:41.97 ping -6 ::1 -fq
602850 root 20 0 13048 2944 2816 R 46.7 0.0 6:45.15 ping -4 127.0.0.1 -fq
602851 root 20 0 13048 2944 2816 R 46.7 0.0 6:45.16 ping -6 ::1 -fq
602862 root 20 0 13048 2944 2816 R 46.7 0.0 6:41.96 ping -4 127.0.0.1 -fq
602854 root 20 0 13048 2944 2816 R 45.8 0.0 6:42.05 ping -6 ::1 -fq
602859 root 20 0 13048 2944 2816 R 44.9 0.0 6:43.49 ping -4 127.0.0.1 -fq
602860 root 20 0 13048 2944 2816 R 43.9 0.0 6:43.53 ping -6 ::1 -fq
15 root 20 0 0 0 0 R 9.3 0.0 14:17.74 [rcu_preempt]
Here is the process list [edited]:
root 455273 156779 0 12:54 pts/2 00:00:00 make kselftest
root 455274 156779 0 12:54 pts/2 00:00:02 tee ../kselftest-6.2.0-rc2-2.log
root 455556 455273 0 12:54 pts/2 00:00:00 make -C ./tools/testing/selftests run_tests
root 455864 455556 0 12:54 pts/2 00:00:00 /bin/sh -c for TARGET in alsa amd-pstate arm64 breakpoints capabilities cgroup
clone3 core cpufreq cpu-hotplug damon drivers/dma-buf drivers/s390x/uvdevice drivers/net/bonding drivers/net/team efivarfs exec
filesystems filesystems/binderfs filesystems/epoll filesystems/fat firmware fpu ftrace futex gpio intel_pstate iommu ipc ir kcmp
kexec kvm landlock lib livepatch lkdtm membarrier memfd memory-hotplug mincore mount mount_setattr move_mount_set_group mqueue nci
net net/af_unix net/forwarding net/hsr net/mptcp net/openvswitch netfilter nsfs pidfd pid_namespace powerpc proc pstore ptrace
openat2 resctrl rlimits rseq rtc seccomp sgx sigaltstack size sparc64 splice static_keys sync syscall_user_dispatch sysctl
tc-testing tdx timens timers tmpfs tpm2 user vDSO vm x86 zram; do \ ?BUILD_TARGET=$BUILD/$TARGET;?\ ?make OUTPUT=$BUILD_TARGET -C
$TARGET run_tests \ ???O=/home/marvin/linux/kernel/linux_torvalds;?? \ done;
/netfilter -C netfilter run_tests O=/home/marvin/linux/kernel/linux_torvalds
root 599609 599608 0 14:11 pts/2 00:00:00 /bin/sh -c
BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; .
/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1;
fi; run_many nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh conntrack_icmp_related.sh nft_flowtable.sh ipvs.sh
nft_concat_range.sh nft_conntrack_helper.sh nft_queue.sh nft_meta.sh nf_nat_edemux.sh ipip-conntrack-mtu.sh
conntrack_tcp_unreplied.sh conntrack_vrf.sh nft_synproxy.sh rpath.sh
root 599622 599609 0 14:11 pts/2 00:00:00 /bin/sh -c
BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; .
/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1;
fi; run_many nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh conntrack_icmp_related.sh nft_flowtable.sh ipvs.sh
nft_concat_range.sh nft_conntrack_helper.sh nft_queue.sh nft_meta.sh nf_nat_edemux.sh ipip-conntrack-mtu.sh
conntrack_tcp_unreplied.sh conntrack_vrf.sh nft_synproxy.sh rpath.sh
root 599623 599622 0 14:11 pts/2 00:00:00 /bin/sh -c
BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; .
/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1;
fi; run_many nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh conntrack_icmp_related.sh nft_flowtable.sh ipvs.sh
nft_concat_range.sh nft_conntrack_helper.sh nft_queue.sh nft_meta.sh nf_nat_edemux.sh ipip-conntrack-mtu.sh
conntrack_tcp_unreplied.sh conntrack_vrf.sh nft_synproxy.sh rpath.sh
root 599624 599623 0 14:11 pts/2 00:00:00 /bin/sh -c
BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; .
/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1;
fi; run_many nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh conntrack_icmp_related.sh nft_flowtable.sh ipvs.sh
nft_concat_range.sh nft_conntrack_helper.sh nft_queue.sh nft_meta.sh nf_nat_edemux.sh ipip-conntrack-mtu.sh
conntrack_tcp_unreplied.sh conntrack_vrf.sh nft_synproxy.sh rpath.sh
root 599628 599624 0 14:11 pts/2 00:00:00 /bin/sh -c
BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; .
/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1;
fi; run_many nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh conntrack_icmp_related.sh nft_flowtable.sh ipvs.sh
nft_concat_range.sh nft_conntrack_helper.sh nft_queue.sh nft_meta.sh nf_nat_edemux.sh ipip-conntrack-mtu.sh
conntrack_tcp_unreplied.sh conntrack_vrf.sh nft_synproxy.sh rpath.sh
root 599629 599628 0 14:11 pts/2 00:00:00 perl
/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/prefix.pl
root 602850 2528 46 14:11 pts/2 00:01:24 ping -4 127.0.0.1 -fq
root 602851 2528 46 14:11 pts/2 00:01:24 ping -6 ::1 -fq
root 602853 2528 46 14:11 pts/2 00:01:25 ping -4 127.0.0.1 -fq
root 602854 2528 46 14:11 pts/2 00:01:25 ping -6 ::1 -fq
root 602856 2528 47 14:11 pts/2 00:01:27 ping -4 127.0.0.1 -fq
root 602857 2528 47 14:11 pts/2 00:01:27 ping -6 ::1 -fq
root 602859 2528 45 14:11 pts/2 00:01:23 ping -4 127.0.0.1 -fq
root 602860 2528 45 14:11 pts/2 00:01:23 ping -6 ::1 -fq
root 602862 2528 47 14:11 pts/2 00:01:26 ping -4 127.0.0.1 -fq
root 602863 2528 47 14:11 pts/2 00:01:26 ping -6 ::1 -fq
root 602865 2528 46 14:11 pts/2 00:01:25 ping -4 127.0.0.1 -fq
root 602866 2528 46 14:11 pts/2 00:01:25 ping -6 ::1 -fq
root 602868 2528 48 14:11 pts/2 00:01:28 ping -4 127.0.0.1 -fq
root 602869 2528 48 14:11 pts/2 00:01:28 ping -6 ::1 -fq
root 602871 2528 46 14:11 pts/2 00:01:25 ping -4 127.0.0.1 -fq
root 602872 2528 46 14:11 pts/2 00:01:25 ping -6 ::1 -fq
The platform is Lenovo Ideapad 3 15ITL6 BIOS GGCN50WW running Ubuntu 22.10 kinetic kudu
with XWayland and vanilla mainline Torvalds tree kernel with MGLRU|KMEMLEAK enabled in config.
I am available for additional diagnostics.
Please find the dmesg log, config and the hardware list attached.
NOTE: NMI backtrace reports the kernel as "Tainted", but I am not aware of any modules
force-loaded or proprietary modules.
Thanks,
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
--
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
[-- Attachment #2: config-6.2.0-rc2-mglru-kmemlk-nokasan+.xz --]
[-- Type: application/x-xz, Size: 57112 bytes --]
[-- Attachment #3: lshw.txt.xz --]
[-- Type: application/x-xz, Size: 5184 bytes --]
[-- Attachment #4: dmesg.log.xz --]
[-- Type: application/x-xz, Size: 26936 bytes --]
[-- Attachment #5: lsmod.txt.xz --]
[-- Type: application/x-xz, Size: 3412 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh
2023-01-03 14:03 PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh Mirsad Goran Todorovac
@ 2023-01-03 14:28 ` Florian Westphal
2023-01-03 15:35 ` Mirsad Goran Todorovac
0 siblings, 1 reply; 5+ messages in thread
From: Florian Westphal @ 2023-01-03 14:28 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: linux-kselftest, LKML, Shuah Khan, Florian Westphal,
Jakub Kicinski
Mirsad Goran Todorovac <mirsad.todorovac@alu.unizg.hr> wrote:
> Hi all,
>
> The `make kselftest` hangs in netfilter/nft_trans_stress.sh test when testing 6.2.0-rc2
> (attempted 2 times, waiting half an hour at least):
This script completes in 49 seconds for me.
> # selftests: netfilter: nft_trans_stress.sh
> # PASS: nft add/delete test returned 0
> # PASS: nft reload test returned 0
>
> The test script seems to be stuck in ping flood:
No, it gets stuck in the sub-test after 'PASS: nft reload test returned 0'
Can you send me the output of 'bash -x nft_trans_stress.sh'?
It should tell which command/program isn't making progress.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh
2023-01-03 14:28 ` Florian Westphal
@ 2023-01-03 15:35 ` Mirsad Goran Todorovac
2023-01-03 16:11 ` Florian Westphal
0 siblings, 1 reply; 5+ messages in thread
From: Mirsad Goran Todorovac @ 2023-01-03 15:35 UTC (permalink / raw)
To: Florian Westphal; +Cc: linux-kselftest, LKML, Shuah Khan, Jakub Kicinski
On 03. 01. 2023. 15:28, Florian Westphal wrote:
> Mirsad Goran Todorovac <mirsad.todorovac@alu.unizg.hr> wrote:
>> Hi all,
>>
>> The `make kselftest` hangs in netfilter/nft_trans_stress.sh test when testing 6.2.0-rc2
>> (attempted 2 times, waiting half an hour at least):
>
> This script completes in 49 seconds for me.
>
>> # selftests: netfilter: nft_trans_stress.sh
>> # PASS: nft add/delete test returned 0
>> # PASS: nft reload test returned 0
>>
>> The test script seems to be stuck in ping flood:
>
> No, it gets stuck in the sub-test after 'PASS: nft reload test returned 0'
>
> Can you send me the output of 'bash -x nft_trans_stress.sh'?
> It should tell which command/program isn't making progress.
Hi, Florian!
Well, when ran alone, the script ended successfully:
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter# bash nft_trans_stress.sh
PASS: nft add/delete test returned 0
PASS: nft reload test returned 0
PASS: nft add/delete with nftrace enabled test returned 0
PASS: nft add/delete with nftrace enabled test returned 0
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter#
There was no stall after "nft reload test" in a standalone superuser run.
Hope this helps.
Kind regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
--
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh
2023-01-03 15:35 ` Mirsad Goran Todorovac
@ 2023-01-03 16:11 ` Florian Westphal
2023-01-03 21:17 ` Mirsad Goran Todorovac
0 siblings, 1 reply; 5+ messages in thread
From: Florian Westphal @ 2023-01-03 16:11 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: Florian Westphal, linux-kselftest, LKML, Shuah Khan,
Jakub Kicinski
Mirsad Goran Todorovac <mirsad.todorovac@alu.unizg.hr> wrote:
> > Can you send me the output of 'bash -x nft_trans_stress.sh'?
> > It should tell which command/program isn't making progress.
>
> Hi, Florian!
>
> Well, when ran alone, the script ended successfully:
>
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter# bash nft_trans_stress.sh
> PASS: nft add/delete test returned 0
> PASS: nft reload test returned 0
> PASS: nft add/delete with nftrace enabled test returned 0
> PASS: nft add/delete with nftrace enabled test returned 0
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter#
>
> There was no stall after "nft reload test" in a standalone superuser run.
Hmm. Does this patch make it work when running via 'make kselftest'?
diff --git a/tools/testing/selftests/netfilter/nft_trans_stress.sh b/tools/testing/selftests/netfilter/nft_trans_stress.sh
--- a/tools/testing/selftests/netfilter/nft_trans_stress.sh
+++ b/tools/testing/selftests/netfilter/nft_trans_stress.sh
@@ -10,12 +10,20 @@
ksft_skip=4
testns=testns-$(mktemp -u "XXXXXXXX")
+tmp=""
tables="foo bar baz quux"
global_ret=0
eret=0
lret=0
+cleanup() {
+ ip netns pids "$testns" | xargs kill 2>/dev/null
+ ip netns del "$testns"
+
+ rm -f "$tmp"
+}
+
check_result()
{
local r=$1
@@ -43,6 +51,7 @@ if [ $? -ne 0 ];then
exit $ksft_skip
fi
+trap cleanup EXIT
tmp=$(mktemp)
for table in $tables; do
@@ -139,11 +148,4 @@ done
check_result $lret "add/delete with nftrace enabled"
-pkill -9 ping
-
-wait
-
-rm -f "$tmp"
-ip netns del "$testns"
-
exit $global_ret
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh
2023-01-03 16:11 ` Florian Westphal
@ 2023-01-03 21:17 ` Mirsad Goran Todorovac
0 siblings, 0 replies; 5+ messages in thread
From: Mirsad Goran Todorovac @ 2023-01-03 21:17 UTC (permalink / raw)
To: Florian Westphal; +Cc: linux-kselftest, LKML, Shuah Khan, Jakub Kicinski
On 03. 01. 2023. 17:11, Florian Westphal wrote:
> Mirsad Goran Todorovac <mirsad.todorovac@alu.unizg.hr> wrote:
>>> Can you send me the output of 'bash -x nft_trans_stress.sh'?
>>> It should tell which command/program isn't making progress.
>>
>> Hi, Florian!
>>
>> Well, when ran alone, the script ended successfully:
>>
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter# bash nft_trans_stress.sh
>> PASS: nft add/delete test returned 0
>> PASS: nft reload test returned 0
>> PASS: nft add/delete with nftrace enabled test returned 0
>> PASS: nft add/delete with nftrace enabled test returned 0
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter#
>>
>> There was no stall after "nft reload test" in a standalone superuser run.
>
> Hmm. Does this patch make it work when running via 'make kselftest'?
>
> diff --git a/tools/testing/selftests/netfilter/nft_trans_stress.sh b/tools/testing/selftests/netfilter/nft_trans_stress.sh
> --- a/tools/testing/selftests/netfilter/nft_trans_stress.sh
> +++ b/tools/testing/selftests/netfilter/nft_trans_stress.sh
> @@ -10,12 +10,20 @@
> ksft_skip=4
>
> testns=testns-$(mktemp -u "XXXXXXXX")
> +tmp=""
>
> tables="foo bar baz quux"
> global_ret=0
> eret=0
> lret=0
>
> +cleanup() {
> + ip netns pids "$testns" | xargs kill 2>/dev/null
> + ip netns del "$testns"
> +
> + rm -f "$tmp"
> +}
> +
> check_result()
> {
> local r=$1
> @@ -43,6 +51,7 @@ if [ $? -ne 0 ];then
> exit $ksft_skip
> fi
>
> +trap cleanup EXIT
> tmp=$(mktemp)
>
> for table in $tables; do
> @@ -139,11 +148,4 @@ done
>
> check_result $lret "add/delete with nftrace enabled"
>
> -pkill -9 ping
> -
> -wait
> -
> -rm -f "$tmp"
> -ip netns del "$testns"
> -
> exit $global_ret
I've got the following output:
make[2]: Entering directory '/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter'
TAP version 13
1..17
# selftests: netfilter: nft_trans_stress.sh
# PASS: nft add/delete test returned 0
# PASS: nft reload test returned 0
#
not ok 1 selftests: netfilter: nft_trans_stress.sh # TIMEOUT 45 seconds
# selftests: netfilter: nft_fib.sh
# PASS: fib expression did not cause unwanted packet drops
It did not hang, but still did not finish all 4 tests correctly like when ran standalone:
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter# time ./nft_trans_stress.sh
PASS: nft add/delete test returned 0
PASS: nft reload test returned 0
PASS: nft add/delete with nftrace enabled test returned 0
PASS: nft add/delete with nftrace enabled test returned 0
real 1m11.014s
user 0m6.470s
sys 0m20.344s
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/netfilter#
I hope we're closer to the solution.
Kind regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
--
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-01-03 21:18 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-03 14:03 PROBLEM: Hang in selftests/netfilter/nft_trans_stress.sh Mirsad Goran Todorovac
2023-01-03 14:28 ` Florian Westphal
2023-01-03 15:35 ` Mirsad Goran Todorovac
2023-01-03 16:11 ` Florian Westphal
2023-01-03 21:17 ` Mirsad Goran Todorovac
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox