Re: [B.A.T.M.A.N.] [PATCH maint] batman-adv: release vlan object after checking the CRC

Russell Senior
2014-01-27T18:43:19Z

>>>>> "Antonio" == Antonio Quartulli <antonio@meshcoding.com> writes:

Antonio> There is a refcounter unbalance in the CRC checking routine
Antonio> invoked on OGM reception. A vlan object is retrieved (thus
Antonio> its refcounter is increased by one) but it is never properly
Antonio> released. This leads to a memleak because the vlan object
Antonio> will never be free'd.

Antonio> Fix this by releasing the vlan object after having read the
Antonio> CRC.

Antonio> Reported-by: Russell Senior <russell@personaltelco.net>
Antonio> [...]

I am still seeing a kernel memory leak, even with this patch.

My test configuration is a Ubiquiti AirRouter (ar71xx) on OpenWrt r39397 with
this patch and the MTU patch on one end, and a Soekris net4826 (x86)
on the same OpenWrt r39397 revision and patchs on the other end
(kmemleak also enabled on the x86 end, not on the ar71xx end).

The airrouter, with the bat0 interface disabled, has a /proc/meminfo
that looks like this right after boot:

root@ar0:/# cat /proc/meminfo
MemTotal: 29044 kB
MemFree: 8220 kB
Buffers: 2164 kB
Cached: 6176 kB
SwapCached: 0 kB
Active: 4624 kB
Inactive: 5672 kB
Active(anon): 1984 kB
Inactive(anon): 20 kB
Active(file): 2640 kB
Inactive(file): 5652 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 1972 kB
Mapped: 1872 kB
Shmem: 48 kB
Slab: 4928 kB
SReclaimable: 932 kB
SUnreclaim: 3996 kB
KernelStack: 272 kB
PageTables: 188 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 14520 kB
Committed_AS: 3992 kB
VmallocTotal: 1048372 kB
VmallocUsed: 1476 kB
VmallocChunk: 1043448 kB

With bat0 enabled, however, the AirRouter will begin to become
unresponsive at around 7m15s after boot and then OOM and reboot within
9 minutes of uptime. Using "watch -d cat /proc/meminfo", I see
MemFree first erode, then it begin to chew into Buffers and Cached
until supreme unhappiness ensues.

On the x86 side, I see some activity from kmemleak, which
substantially goes away when batman-adv does not have a peer. The
unreferenced objects that show up in /sys/kernel/debug/kmemleak look
like this (only a sample):

unreferenced object 0xc2eab000 (size 184):
comm "softirq", pid 0, jiffies 33396 (age 692.960s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 3c c2 00 00 00 00 00 00 00 00 ......<.........
backtrace:
[<c10039b9>] print_context_stack+0x99/0xb0
[<c108cf00>] kmem_cache_alloc+0xd0/0xf0
[<c12eedc9>] build_skb+0x29/0x140
[<c12eedc9>] build_skb+0x29/0x140
[<c12f1a86>] __netdev_alloc_skb+0x56/0xd0
[<c4c020b2>] ath_rxbuf_alloc+0x22/0x80 [ath]
[<c1372619>] free_debug_processing+0x14c/0x190
[<c4c42204>] ath5k_stop+0xd4/0xc90 [ath5k]
[<c4c43773>] ath5k_beacon_update_timers+0x9b3/0x9d0 [ath5k]
[<c4c02a8f>] ath_hw_cycle_counters_update+0xcf/0x130 [ath]
[<c108c727>] kmem_cache_free+0xe7/0x100
[<c105ccfa>] __rcu_process_callbacks+0x5a/0x70
[<c102d62e>] tasklet_action+0x3e/0x70
[<c102dac5>] __do_softirq+0x85/0x140
[<c102da40>] __do_softirq+0x0/0x140
[<c102dc42>] irq_exit+0x32/0x50
unreferenced object 0xc0c1c600 (size 184):
comm "softirq", pid 0, jiffies 33490 (age 692.020s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 3c c2 00 00 00 00 00 00 00 00 ......<.........
backtrace:
[<c10039b9>] print_context_stack+0x99/0xb0
[<c108cf00>] kmem_cache_alloc+0xd0/0xf0
[<c12eedc9>] build_skb+0x29/0x140
[<c12eedc9>] build_skb+0x29/0x140
[<c12f1a86>] __netdev_alloc_skb+0x56/0xd0
[<c4c020b2>] ath_rxbuf_alloc+0x22/0x80 [ath]
[<c1372619>] free_debug_processing+0x14c/0x190
[<c4c42204>] ath5k_stop+0xd4/0xc90 [ath5k]
[<c4c43773>] ath5k_beacon_update_timers+0x9b3/0x9d0 [ath5k]
[<c108c727>] kmem_cache_free+0xe7/0x100
[<c105ccfa>] __rcu_process_callbacks+0x5a/0x70
[<c102d62e>] tasklet_action+0x3e/0x70
[<c102dac5>] __do_softirq+0x85/0x140
[<c102da40>] __do_softirq+0x0/0x140
[<c102dc42>] irq_exit+0x32/0x50
[<c1002b4d>] do_IRQ+0x8d/0xb0

Memory is disappearing at a faster rate than is accounted for here
(size 184 bytes at a time). The OOM-reboot rate indicates memory is
being consumed at a rate on the order of 30-40 kB per second.

My OpenWrt diffconfig is as follows:

CONFIG_TARGET_x86=y
CONFIG_TARGET_x86_generic=y
CONFIG_TARGET_x86_generic_Soekris48xx=y
CONFIG_DEVEL=y
CONFIG_ALFRED_NEEDS_lua=y
CONFIG_BUILD_LOG=y
CONFIG_DOWNLOAD_FOLDER="/usr/portage/distfiles"
CONFIG_KMOD_BATMAN_ADV_BATCTL=y
CONFIG_KMOD_BATMAN_ADV_BLA=y
CONFIG_KMOD_BATMAN_ADV_DAT=y
CONFIG_KMOD_BATMAN_ADV_DEBUG_LOG=y
CONFIG_LIBCURL_FILE=y
CONFIG_LIBCURL_FTP=y
CONFIG_LIBCURL_HTTP=y
CONFIG_LIBCURL_POLARSSL=y
CONFIG_OPENSSL_WITH_EC=y
CONFIG_PACKAGE_ALFRED_BATHOSTS=y
CONFIG_PACKAGE_ALFRED_VIS=y
CONFIG_PACKAGE_MAC80211_DEBUGFS=y
CONFIG_PACKAGE_MAC80211_MESH=y
CONFIG_PACKAGE_alfred=y
CONFIG_PACKAGE_bridge=y
CONFIG_PACKAGE_crda=y
CONFIG_PACKAGE_curl=y
CONFIG_PACKAGE_diffutils=y
# CONFIG_PACKAGE_dnsmasq is not set
# CONFIG_PACKAGE_firewall is not set
CONFIG_PACKAGE_horst=y
CONFIG_PACKAGE_hostapd-common=y
CONFIG_PACKAGE_iftop=y
CONFIG_PACKAGE_ip=y
CONFIG_PACKAGE_iw=y
CONFIG_PACKAGE_iwinfo=y
CONFIG_PACKAGE_kmod-ath=y
CONFIG_PACKAGE_kmod-ath5k=y
CONFIG_PACKAGE_kmod-batman-adv=y
CONFIG_PACKAGE_kmod-bridge=y
CONFIG_PACKAGE_kmod-cfg80211=y
CONFIG_PACKAGE_kmod-crypto-aes=y
CONFIG_PACKAGE_kmod-crypto-arc4=y
CONFIG_PACKAGE_kmod-crypto-core=y
CONFIG_PACKAGE_kmod-crypto-crc32c=y
CONFIG_PACKAGE_kmod-crypto-hash=y
# CONFIG_PACKAGE_kmod-lib-crc-ccitt is not set
CONFIG_PACKAGE_kmod-lib-crc16=y
CONFIG_PACKAGE_kmod-lib-crc32c=y
CONFIG_PACKAGE_kmod-llc=y
CONFIG_PACKAGE_kmod-mac80211=y
# CONFIG_PACKAGE_kmod-ppp is not set
CONFIG_PACKAGE_kmod-stp=y
CONFIG_PACKAGE_libcurl=y
CONFIG_PACKAGE_libelf1=y
CONFIG_PACKAGE_libiwinfo=y
CONFIG_PACKAGE_liblua=y
CONFIG_PACKAGE_libncurses=y
CONFIG_PACKAGE_libnetsnmp=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libpcap=y
CONFIG_PACKAGE_libpcre=y
CONFIG_PACKAGE_libpolarssl=y
CONFIG_PACKAGE_libpopt=y
CONFIG_PACKAGE_libpthread=y
CONFIG_PACKAGE_librt=y
CONFIG_PACKAGE_lua=y
CONFIG_PACKAGE_missnet-node=y
# CONFIG_PACKAGE_odhcp6c is not set
# CONFIG_PACKAGE_ppp is not set
CONFIG_PACKAGE_procps=y
CONFIG_PACKAGE_procps-free=y
CONFIG_PACKAGE_procps-pgrep=y
CONFIG_PACKAGE_procps-pkill=y
CONFIG_PACKAGE_procps-pmap=y
CONFIG_PACKAGE_procps-ps=y
CONFIG_PACKAGE_procps-pwdx=y
CONFIG_PACKAGE_procps-skill=y
CONFIG_PACKAGE_procps-slabtop=y
CONFIG_PACKAGE_procps-snice=y
CONFIG_PACKAGE_procps-tload=y
CONFIG_PACKAGE_procps-top=y
CONFIG_PACKAGE_procps-vmstat=y
CONFIG_PACKAGE_procps-w=y
CONFIG_PACKAGE_procps-watch=y
CONFIG_PACKAGE_rsync=y
CONFIG_PACKAGE_snmpd=y
CONFIG_PACKAGE_tcpdump=y
CONFIG_PACKAGE_terminfo=y
CONFIG_PACKAGE_traceroute6=y
CONFIG_PACKAGE_wget=y
CONFIG_PACKAGE_wireless-tools=y
CONFIG_PACKAGE_wpad-mini=y
CONFIG_PACKAGE_zlib=y
# CONFIG_TARGET_ROOTFS_EXT4FS is not set
# CONFIG_TARGET_ROOTFS_TARGZ is not set

Anything you are interested in that I left out, I'm happy to provide
on request.

--
Russell Senior, President
russell@personaltelco.net