From mboxrd@z Thu Jan 1 00:00:00 1970 From: Florian Westphal Subject: Re: Memory leaks in conntrack Date: Wed, 13 Sep 2017 17:37:34 +0200 Message-ID: <20170913153734.GC2453@breakpoint.cc> References: <20170913080516.GD25977@breakpoint.cc> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Cong Wang , netfilter-devel@vger.kernel.org, Linux Kernel Network Developers To: Florian Westphal Return-path: Content-Disposition: inline In-Reply-To: <20170913080516.GD25977@breakpoint.cc> Sender: netdev-owner@vger.kernel.org List-Id: netfilter-devel.vger.kernel.org Florian Westphal wrote: > Cong Wang wrote: > > While testing my TC filter patches (so not related to conntrack), the > > following memory leaks are shown up: > > > > unreferenced object 0xffff9b19ba551228 (size 128): > > comm "chronyd", pid 338, jiffies 4294910829 (age 53.188s) > > hex dump (first 32 bytes): > > 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk > > 00 00 00 00 18 00 00 30 00 00 00 00 00 00 00 00 .......0........ > > backtrace: > > [] create_object+0x169/0x2aa > > [] kmemleak_alloc+0x25/0x41 > > [] slab_post_alloc_hook+0x44/0x65 > > [] __kmalloc_track_caller+0x113/0x146 > > [] __krealloc+0x4a/0x69 > > [] nf_ct_ext_add+0xe1/0x145 > > [] init_conntrack+0x1f7/0x36e > > [] nf_conntrack_in+0x1d3/0x326 > > [] ipv4_conntrack_local+0x4d/0x50 > > [] nf_hook_slow+0x3c/0x9b > > [] nf_hook.constprop.40+0xbe/0xd8 > > [] __ip_local_out+0xb3/0xbf > > [] ip_local_out+0x1c/0x36 > > [] ip_send_skb+0x19/0x3d > > [] udp_send_skb+0x17e/0x1df > > [] udp_sendmsg+0x5a2/0x77c > > unreferenced object 0xffff9b19a69b3340 (size 336): > > comm "chronyd", pid 338, jiffies 4294910868 (age 53.032s) > > hex dump (first 32 bytes): > > 01 00 00 00 5a 5a 5a 5a 00 00 00 00 ad 4e ad de ....ZZZZ.....N.. > > ff ff ff ff 5a 5a 5a 5a ff ff ff ff ff ff ff ff ....ZZZZ........ > > backtrace: > > [] create_object+0x169/0x2aa > > [] kmemleak_alloc+0x25/0x41 > > [] slab_post_alloc_hook+0x44/0x65 > > [] kmem_cache_alloc+0xd7/0x1f1 > > [] __nf_conntrack_alloc+0xa2/0x146 > > [] init_conntrack+0xb2/0x36e > > [] nf_conntrack_in+0x1d3/0x326 > > [] ipv4_conntrack_local+0x4d/0x50 > > [] nf_hook_slow+0x3c/0x9b > > [] nf_hook.constprop.40+0xbe/0xd8 > > [] __ip_local_out+0xb3/0xbf > > [] ip_local_out+0x1c/0x36 > > [] ip_send_skb+0x19/0x3d > > [] udp_send_skb+0x17e/0x1df > > [] udp_sendmsg+0x5a2/0x77c > > [] inet_sendmsg+0x37/0x5e > > > > I don't touch chronyd in my VM, so I have no idea why it sends out UDP > > packets, my guess is it is some periodical packet. > > > > I don't think I use conntrack either, since /proc/net/ip_conntrack > > does not exist. > > You probably do, can you try "cat /proc/net/nf_conntrack" instead? > > (otherwise there should be no ipv4_conntrack_local() invocation > since we would not register this hook at all). > > I tried to reproduce this but so far I had no success. > If you can identify something that could give a hint when this > is happening (only once after boot, periodically, only with udp, etc) > please let us know. FWIW i managed to obtain a similar backtrace, but in that case it was a false positive (peeking at the address content showed it was my ssh connection to the vm and timeout and tcp conntrackk struct fields were changing; i.e. the nf_conn reported was still in the conntrack hash. Why this address was reported i do not know, afaik kmemleak does scan for addresses anywhere in the object (we use container_of() to get back nf_conn from the hlist_node), so it should have found the address linked via the main conntrack hash table. Right now I don't have enough info to dig any further, sorry :-/