* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 [not found] <1608957.xtEoauR4Ya@merkaba> @ 2019-01-15 10:10 ` Michal Kubecek 2019-01-15 10:10 ` Michal Kubecek 2019-01-15 10:15 ` Florian Westphal 0 siblings, 2 replies; 6+ messages in thread From: Michal Kubecek @ 2019-01-15 10:10 UTC (permalink / raw) To: netfilter-devel; +Cc: netdev, linux-kernel, Martin Steigerwald (CC netfilter-devel and netdev) On Mon, Jan 14, 2019 at 11:53:10PM +0100, Martin Steigerwald wrote: > Hi! > > Does that ring a bell with someone? For now I just downgraded, no time > for detailed analysis. > > Debian bug report at: > > iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 > https://bugs.debian.org/919325 > > 4.20 works, 5.0-rc2 showed this issue with iptables. Configurations attached. > > Excerpt from Debian bug report follows: > > I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow > after startup. I saw iptables consuming 100% CPU, it only responded to > SIGKILL. It got restarted several times, probably by some systemd service. > > Then I started 'iptables -nvL' manually. And I got this: > > % strace -p 5748 > [… tons more, in what appeared an endless loop …] > recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > recvmsg(3, ^C{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > strace: Process 5748 detached > > and this (output from atop): > > PID TID MINFLT MAJFLT VSTEXT VSLIBS VDATA VSTACK VSIZE RSIZE PSIZE VGROW RGROW SWAPSZ RUID EUID MEM CMD 1/16 > 11575 - 61552 0 152K 2324K 5.0G 132K 5.1G 5.1G 0K 240.4M 240.5M 0K root root 33% iptables > > I had it growing till 10 GiB before I stopped it by SIGKILL to prevent > excessive swapping. > > I will attach kernel configuration. > > That is all I am willing to spend time on for now before going to sleep. > I will however reboot with older 4.20 kernel to see whether it is kernel > related. > > […] > > -- System Information: > Debian Release: buster/sid > […] > Kernel: Linux 5.0.0-rc2-tp520 (SMP w/4 CPU cores; PREEMPT) > > Thanks, > -- > Martin ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 2019-01-15 10:10 ` [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 Michal Kubecek @ 2019-01-15 10:10 ` Michal Kubecek 2019-01-15 10:15 ` Florian Westphal 1 sibling, 0 replies; 6+ messages in thread From: Michal Kubecek @ 2019-01-15 10:10 UTC (permalink / raw) To: netfilter-devel; +Cc: netdev, linux-kernel, Martin Steigerwald (CC netfilter-devel and netdev) On Mon, Jan 14, 2019 at 11:53:10PM +0100, Martin Steigerwald wrote: > Hi! > > Does that ring a bell with someone? For now I just downgraded, no time > for detailed analysis. > > Debian bug report at: > > iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 > https://bugs.debian.org/919325 > > 4.20 works, 5.0-rc2 showed this issue with iptables. Configurations attached. > > Excerpt from Debian bug report follows: > > I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow > after startup. I saw iptables consuming 100% CPU, it only responded to > SIGKILL. It got restarted several times, probably by some systemd service. > > Then I started 'iptables -nvL' manually. And I got this: > > % strace -p 5748 > [… tons more, in what appeared an endless loop …] > recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > recvmsg(3, ^C{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=372, type=0xa06 /* NLMSG_??? */, flags=NLM_F_MULTI|0x800, seq=0, pid=5748}, "\x02\x00\x00\x11\x0b\x00\x01\x00\x66\x69\x6c\x74\x65\x72\x00\x00\x0b\x00\x02\x00\x4f\x55\x54\x50\x55\x54\x00\x00\x0c\x00\x03\x00"...}, iov_len=16536}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 372 > strace: Process 5748 detached > > and this (output from atop): > > PID TID MINFLT MAJFLT VSTEXT VSLIBS VDATA VSTACK VSIZE RSIZE PSIZE VGROW RGROW SWAPSZ RUID EUID MEM CMD 1/16 > 11575 - 61552 0 152K 2324K 5.0G 132K 5.1G 5.1G 0K 240.4M 240.5M 0K root root 33% iptables > > I had it growing till 10 GiB before I stopped it by SIGKILL to prevent > excessive swapping. > > I will attach kernel configuration. > > That is all I am willing to spend time on for now before going to sleep. > I will however reboot with older 4.20 kernel to see whether it is kernel > related. > > […] > > -- System Information: > Debian Release: buster/sid > […] > Kernel: Linux 5.0.0-rc2-tp520 (SMP w/4 CPU cores; PREEMPT) > > Thanks, > -- > Martin ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 2019-01-15 10:10 ` [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 Michal Kubecek 2019-01-15 10:10 ` Michal Kubecek @ 2019-01-15 10:15 ` Florian Westphal 2019-01-15 10:15 ` Florian Westphal 2019-01-15 11:52 ` Martin Steigerwald 1 sibling, 2 replies; 6+ messages in thread From: Florian Westphal @ 2019-01-15 10:15 UTC (permalink / raw) To: Michal Kubecek; +Cc: netfilter-devel, netdev, linux-kernel, Martin Steigerwald Michal Kubecek <mkubecek@suse.cz> wrote: > > I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow > > after startup. I saw iptables consuming 100% CPU, it only responded to > > SIGKILL. It got restarted several times, probably by some systemd service. > > > > Then I started 'iptables -nvL' manually. And I got this: > > > > % strace -p 5748 > > [… tons more, in what appeared an endless loop …] This is fixed by: http://patchwork.ozlabs.org/patch/1024772/ ("netfilter: nf_tables: Fix for endless loop when dumping ruleset"). ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 2019-01-15 10:15 ` Florian Westphal @ 2019-01-15 10:15 ` Florian Westphal 2019-01-15 11:52 ` Martin Steigerwald 1 sibling, 0 replies; 6+ messages in thread From: Florian Westphal @ 2019-01-15 10:15 UTC (permalink / raw) To: Michal Kubecek; +Cc: netfilter-devel, netdev, linux-kernel, Martin Steigerwald Michal Kubecek <mkubecek@suse.cz> wrote: > > I upgraded to self-compiled 5.0-rc2 today and found the machine to be slow > > after startup. I saw iptables consuming 100% CPU, it only responded to > > SIGKILL. It got restarted several times, probably by some systemd service. > > > > Then I started 'iptables -nvL' manually. And I got this: > > > > % strace -p 5748 > > [… tons more, in what appeared an endless loop …] This is fixed by: http://patchwork.ozlabs.org/patch/1024772/ ("netfilter: nf_tables: Fix for endless loop when dumping ruleset"). ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 2019-01-15 10:15 ` Florian Westphal 2019-01-15 10:15 ` Florian Westphal @ 2019-01-15 11:52 ` Martin Steigerwald 2019-01-15 11:52 ` Martin Steigerwald 1 sibling, 1 reply; 6+ messages in thread From: Martin Steigerwald @ 2019-01-15 11:52 UTC (permalink / raw) To: Florian Westphal; +Cc: Michal Kubecek, netfilter-devel, netdev, linux-kernel Florian Westphal - 15.01.19, 11:15: > Michal Kubecek <mkubecek@suse.cz> wrote: > > > I upgraded to self-compiled 5.0-rc2 today and found the machine to > > > be slow after startup. I saw iptables consuming 100% CPU, it only > > > responded to SIGKILL. It got restarted several times, probably by > > > some systemd service. > > > > > > Then I started 'iptables -nvL' manually. And I got this: > > > > > > % strace -p 5748 > > > [… tons more, in what appeared an endless loop …] > > This is fixed by: > > http://patchwork.ozlabs.org/patch/1024772/ > ("netfilter: nf_tables: Fix for endless loop when dumping ruleset"). Thanks, Florian. Will wait for first 5.0-rcx with x=>2 that contains the fix. Bug closed on Debian side already, was premature to report it there. Ciao, -- Martin ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 2019-01-15 11:52 ` Martin Steigerwald @ 2019-01-15 11:52 ` Martin Steigerwald 0 siblings, 0 replies; 6+ messages in thread From: Martin Steigerwald @ 2019-01-15 11:52 UTC (permalink / raw) To: Florian Westphal; +Cc: Michal Kubecek, netfilter-devel, netdev, linux-kernel Florian Westphal - 15.01.19, 11:15: > Michal Kubecek <mkubecek@suse.cz> wrote: > > > I upgraded to self-compiled 5.0-rc2 today and found the machine to > > > be slow after startup. I saw iptables consuming 100% CPU, it only > > > responded to SIGKILL. It got restarted several times, probably by > > > some systemd service. > > > > > > Then I started 'iptables -nvL' manually. And I got this: > > > > > > % strace -p 5748 > > > [… tons more, in what appeared an endless loop …] > > This is fixed by: > > http://patchwork.ozlabs.org/patch/1024772/ > ("netfilter: nf_tables: Fix for endless loop when dumping ruleset"). Thanks, Florian. Will wait for first 5.0-rcx with x=>2 that contains the fix. Bug closed on Debian side already, was premature to report it there. Ciao, -- Martin ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-01-15 12:00 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1608957.xtEoauR4Ya@merkaba>
2019-01-15 10:10 ` [REGRESSION] 5.0-rc2: iptables -nvL consumes 100% of CPU and hogs memory with kernel 5.0-rc2 Michal Kubecek
2019-01-15 10:10 ` Michal Kubecek
2019-01-15 10:15 ` Florian Westphal
2019-01-15 10:15 ` Florian Westphal
2019-01-15 11:52 ` Martin Steigerwald
2019-01-15 11:52 ` Martin Steigerwald
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).