From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrey Konovalov Subject: Potential use-after-free in e1000_clean_tx_irq Date: Wed, 21 Aug 2013 20:10:15 +0400 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4181752635677439983==" Cc: netdev@vger.kernel.org, Alexander Potapenko , Evgeniy Stepanov , Dmitry Vyukov , Kostya Serebryany To: e1000-devel@lists.sourceforge.net Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: e1000-devel-bounces@lists.sourceforge.net List-Id: netdev.vger.kernel.org --===============4181752635677439983== Content-Type: multipart/alternative; boundary=089e013cc028bab52204e4776829 --089e013cc028bab52204e4776829 Content-Type: text/plain; charset=ISO-8859-1 Hi, I'm working on a memory error detector AddressSanitizer for Linux kernel ( https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel), which can detect use-after-free and buffer-overflow errors. Currently the tool is in very early stage and it can contain bugs. I was running a system call fuzzer and got some reports: [ 64.143848] ========================================================================= [ 64.144763] ERROR: AddressSanitizer: heap-use-after-free on address ffff88002a3dae60 [ 64.145945] Stack trace: [ 64.146302] [] asan_report_error+0x85/0x2c0 [ 64.147112] [] asan_check_region+0x30/0x40 [ 64.147966] [] __tsan_read4+0x13/0x20 [ 64.148808] [] e1000_clean+0x1d0/0x11b0 [e1000] [ 64.149742] [] net_rx_action+0x1aa/0x380 [ 64.150574] [] __do_softirq+0x182/0x3a0 [ 64.151391] [] call_softirq+0x1c/0x30 [ 64.152179] [] do_softirq+0x5d/0xc0 [ 64.152926] [] local_bh_enable+0x127/0x130 [ 64.153717] [] ip_finish_output+0x365/0x640 [ 64.154653] [] ip_output+0xb9/0x100 [ 64.155423] [] ip_local_out+0x4c/0x60 [ 64.156216] [] ip_send_skb+0x23/0x70 [ 64.156951] [] udp_send_skb+0x584/0x6e0 [ 64.157761] [] udp_sendmsg+0x4dc/0xfd0 [ 64.158581] [] inet_sendmsg+0x108/0x160 [ 64.159401] [] sock_sendmsg+0x133/0x170 [ 64.160143] [] SYSC_sendto+0x1e9/0x2d0 [ 64.160932] [] SyS_sendto+0x49/0x70 [ 64.161710] [] compat_sys_socketcall+0x305/0x530 [ 64.162634] [] sysenter_dispatch+0x7/0x1a [ 64.163471] [] 0xffffffffffffffff [ 64.164213] Free stack trace: [ 64.164660] [] asan_slab_free+0x61/0xb0 [ 64.165466] [] kmem_cache_free+0x55/0x2e0 [ 64.166261] [] kfree_skbmem+0x5b/0xd0 [ 64.167046] [] consume_skb+0x4c/0xd0 [ 64.167811] [] dev_kfree_skb_any+0x60/0x70 [ 64.168710] [] e1000_unmap_and_free_tx_resource.isra.45+0xda/0x130 [e1000] [ 64.169954] [] e1000_clean+0x1e9/0x11b0 [e1000] [ 64.170859] [] net_rx_action+0x1aa/0x380 [ 64.171685] [] __do_softirq+0x182/0x3a0 [ 64.172491] [] call_softirq+0x1c/0x30 [ 64.173276] [] do_softirq+0x5d/0xc0 [ 64.174041] [] local_bh_enable+0x127/0x130 [ 64.174868] [] ip_finish_output+0x365/0x640 [ 64.175734] [] ip_output+0xb9/0x100 [ 64.176518] [] ip_local_out+0x4c/0x60 [ 64.177315] [] ip_send_skb+0x23/0x70 [ 64.178087] [] udp_send_skb+0x584/0x6e0 [ 64.178910] [] udp_sendmsg+0x4dc/0xfd0 [ 64.179713] [] inet_sendmsg+0x108/0x160 [ 64.180524] [] sock_sendmsg+0x133/0x170 [ 64.181332] [] SYSC_sendto+0x1e9/0x2d0 [ 64.182001] [] SyS_sendto+0x49/0x70 [ 64.182771] [] SyS_send+0x3b/0x50 [ 64.183508] Shadow bytes around the buggy address: [ 64.184258] ffff88003ba7b570: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.185109] ffff88003ba7b580: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd [ 64.186184] ffff88003ba7b590: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd [ 64.187257] ffff88003ba7b5a0: fd fd fd fd fa fa fa fa fa fa fa fa fa fa fa fa [ 64.188442] ffff88003ba7b5b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.189520] =>ffff88003ba7b5c0: fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd [ 64.190600] ffff88003ba7b5d0: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa [ 64.191676] ffff88003ba7b5e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.192730] ffff88003ba7b5f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.193822] ffff88003ba7b600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.194903] ffff88003ba7b610: fa fa fa fa fa fa fa fa 00 00 00 00 fa fa fa fa [ 64.195954] Shadow byte legend (one shadow byte represents 8 application bytes): [ 64.197039] Addressable: 00 [ 64.197585] Partially addressable: 01 02 03 04 05 06 07 [ 64.198407] Heap redzone: fa [ 64.198955] Freed heap region: fd [ 64.199519] ========================================================================= [ 64.200424] ========================================================================= [ 64.201539] ERROR: AddressSanitizer: heap-use-after-free on address ffff88002a3daedc [ 64.202432] Stack trace: [ 64.202814] [] asan_report_error+0x85/0x2c0 [ 64.203636] [] asan_check_region+0x30/0x40 [ 64.204481] [] __tsan_read4+0x13/0x20 [ 64.205267] [] consume_skb+0x2b/0xd0 [ 64.206066] [] dev_kfree_skb_any+0x60/0x70 [ 64.206923] [] e1000_unmap_and_free_tx_resource.isra.45+0xda/0x130 [e1000] [ 64.208144] [] e1000_clean+0x1e9/0x11b0 [e1000] [ 64.209067] [] net_rx_action+0x1aa/0x380 [ 64.209866] [] __do_softirq+0x182/0x3a0 [ 64.210671] [] call_softirq+0x1c/0x30 [ 64.211463] [] do_softirq+0x5d/0xc0 [ 64.212227] [] local_bh_enable+0x127/0x130 [ 64.213074] [] ip_finish_output+0x365/0x640 [ 64.213910] [] ip_output+0xb9/0x100 [ 64.214661] [] ip_local_out+0x4c/0x60 [ 64.215451] [] ip_send_skb+0x23/0x70 [ 64.216249] [] udp_send_skb+0x584/0x6e0 [ 64.217064] [] udp_sendmsg+0x4dc/0xfd0 [ 64.217838] [] inet_sendmsg+0x108/0x160 [ 64.218687] [] sock_sendmsg+0x133/0x170 [ 64.219496] [] SYSC_sendto+0x1e9/0x2d0 [ 64.220297] [] SyS_sendto+0x49/0x70 [ 64.221069] [] compat_sys_socketcall+0x305/0x530 [ 64.222053] [] sysenter_dispatch+0x7/0x1a [ 64.222867] [] 0xffffffffffffffff [ 64.223592] Free stack trace: [ 64.224050] [] asan_slab_free+0x61/0xb0 [ 64.224841] [] kmem_cache_free+0x55/0x2e0 [ 64.225559] [] kfree_skbmem+0x5b/0xd0 [ 64.226342] [] consume_skb+0x4c/0xd0 [ 64.227121] [] dev_kfree_skb_any+0x60/0x70 [ 64.227958] [] e1000_unmap_and_free_tx_resource.isra.45+0xda/0x130 [e1000] [ 64.229240] [] e1000_clean+0x1e9/0x11b0 [e1000] [ 64.230149] [] net_rx_action+0x1aa/0x380 [ 64.230947] [] __do_softirq+0x182/0x3a0 [ 64.231706] [] call_softirq+0x1c/0x30 [ 64.232451] [] do_softirq+0x5d/0xc0 [ 64.233219] [] local_bh_enable+0x127/0x130 [ 64.234012] [] ip_finish_output+0x365/0x640 [ 64.234842] [] ip_output+0xb9/0x100 [ 64.235621] [] ip_local_out+0x4c/0x60 [ 64.236431] [] ip_send_skb+0x23/0x70 [ 64.237221] [] udp_send_skb+0x584/0x6e0 [ 64.237991] [] udp_sendmsg+0x4dc/0xfd0 [ 64.238788] [] inet_sendmsg+0x108/0x160 [ 64.239604] [] sock_sendmsg+0x133/0x170 [ 64.240413] [] SYSC_sendto+0x1e9/0x2d0 [ 64.241213] [] SyS_sendto+0x49/0x70 [ 64.241954] [] SyS_send+0x3b/0x50 [ 64.242681] Shadow bytes around the buggy address: [ 64.243422] ffff88003ba7b580: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd [ 64.244534] ffff88003ba7b590: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd [ 64.245615] ffff88003ba7b5a0: fd fd fd fd fa fa fa fa fa fa fa fa fa fa fa fa [ 64.246691] ffff88003ba7b5b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.247768] ffff88003ba7b5c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd [ 64.248886] =>ffff88003ba7b5d0: fd fd fd fd fd fd fd fd fd fd fd[fd]fa fa fa fa [ 64.250001] ffff88003ba7b5e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.251084] ffff88003ba7b5f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.252162] ffff88003ba7b600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.253236] ffff88003ba7b610: fa fa fa fa fa fa fa fa 00 00 00 00 fa fa fa fa [ 64.254312] ffff88003ba7b620: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa [ 64.255431] Shadow byte legend (one shadow byte represents 8 application bytes): [ 64.256500] Addressable: 00 [ 64.257083] Partially addressable: 01 02 03 04 05 06 07 [ 64.257851] Heap redzone: fa [ 64.258397] Freed heap region: fd [ 64.258942] ========================================================================= There were more use-after-free reports after these two. The first use-after-free was caused by accessing 'len' field in 'buffer_info->skb' in 'e1000_clean_tx_irq' (line 3835). Our guess is that 'buffer_info->skb' had been freed in another thread (the bottom frames of the stack traces are different) by 'e1000_unmap_and_free_tx_resource' (line 1972) but wasn't assigned to 'NULL' yet (line 1973). The kernel version is 3.11-rc4 (last commit: b7bc9e7d808ba55729bd263b0210cda36965be32). e100_clean_tx_irq: http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/e1000/e1000_main.c#L3835 e1000_unmap_and_free_tx_resource: http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/e1000/e1000_main.c#L1958 Since these reports were caused by a system call fuzzer I don't know how to reproduce them. Could you confirm if this is a real bug? Thanks! --089e013cc028bab52204e4776829-- --===============4181752635677439983== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk --===============4181752635677439983== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired --===============4181752635677439983==--