From: Willy Tarreau <w@1wt.eu>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Sabrina Dubroca <sd@queasysnail.net>,
Aaron Brown <aaron.f.brown@intel.com>,
Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
Ben Hutchings <ben@decadent.org.uk>, Willy Tarreau <w@1wt.eu>
Subject: [PATCH 2.6.32 16/62] e1000: add dummy allocator to fix race condition between mtu change and netpoll
Date: Sun, 13 Sep 2015 00:56:22 +0200 [thread overview]
Message-ID: <20150912225607.258086407@1wt.eu> (raw)
In-Reply-To: <08d3b586eb2e764308c3de9ee398a17c@local>
2.6.32-longterm review patch. If anyone has any objections, please let me know.
------------------
From: Sabrina Dubroca <sd@queasysnail.net>
commit 08e8331654d1d7b2c58045e549005bc356aa7810 upstream.
There is a race condition between e1000_change_mtu's cleanups and
netpoll, when we change the MTU across jumbo size:
Changing MTU frees all the rx buffers:
e1000_change_mtu -> e1000_down -> e1000_clean_all_rx_rings ->
e1000_clean_rx_ring
Then, close to the end of e1000_change_mtu:
pr_info -> ... -> netpoll_poll_dev -> e1000_clean ->
e1000_clean_rx_irq -> e1000_alloc_rx_buffers -> e1000_alloc_frag
And when we come back to do the rest of the MTU change:
e1000_up -> e1000_configure -> e1000_configure_rx ->
e1000_alloc_jumbo_rx_buffers
alloc_jumbo finds the buffers already != NULL, since data (shared with
page in e1000_rx_buffer->rxbuf) has been re-alloc'd, but it's garbage,
or at least not what is expected when in jumbo state.
This results in an unusable adapter (packets don't get through), and a
NULL pointer dereference on the next call to e1000_clean_rx_ring
(other mtu change, link down, shutdown):
BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [<ffffffff81194d6e>] put_compound_page+0x7e/0x330
[...]
Call Trace:
[<ffffffff81195445>] put_page+0x55/0x60
[<ffffffff815d9f44>] e1000_clean_rx_ring+0x134/0x200
[<ffffffff815da055>] e1000_clean_all_rx_rings+0x45/0x60
[<ffffffff815df5e0>] e1000_down+0x1c0/0x1d0
[<ffffffff811e2260>] ? deactivate_slab+0x7f0/0x840
[<ffffffff815e21bc>] e1000_change_mtu+0xdc/0x170
[<ffffffff81647050>] dev_set_mtu+0xa0/0x140
[<ffffffff81664218>] do_setlink+0x218/0xac0
[<ffffffff814459e9>] ? nla_parse+0xb9/0x120
[<ffffffff816652d0>] rtnl_newlink+0x6d0/0x890
[<ffffffff8104f000>] ? kvm_clock_read+0x20/0x40
[<ffffffff810a2068>] ? sched_clock_cpu+0xa8/0x100
[<ffffffff81663802>] rtnetlink_rcv_msg+0x92/0x260
By setting the allocator to a dummy version, netpoll can't mess up our
rx buffers. The allocator is set back to a sane value in
e1000_configure_rx.
Fixes: edbbb3ca1077 ("e1000: implement jumbo receive with partial descriptors")
Signed-off-by: Sabrina Dubroca <sd@queasysnail.net>
Tested-by: Aaron Brown <aaron.f.brown@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
(cherry picked from commit f655adbac3de7ce92b2b0f9ce2d426b55b600e38)
[wt: path is drivers/net/e1000/e1000_main.c in 2.6.32]
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
drivers/net/e1000/e1000_main.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index 4079a33..375d332 100644
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -140,6 +140,11 @@ static bool e1000_clean_rx_irq(struct e1000_adapter *adapter,
static bool e1000_clean_jumbo_rx_irq(struct e1000_adapter *adapter,
struct e1000_rx_ring *rx_ring,
int *work_done, int work_to_do);
+static void e1000_alloc_dummy_rx_buffers(struct e1000_adapter *adapter,
+ struct e1000_rx_ring *rx_ring,
+ int cleaned_count)
+{
+}
static void e1000_alloc_rx_buffers(struct e1000_adapter *adapter,
struct e1000_rx_ring *rx_ring,
int cleaned_count);
@@ -3154,8 +3159,11 @@ static int e1000_change_mtu(struct net_device *netdev, int new_mtu)
msleep(1);
/* e1000_down has a dependency on max_frame_size */
hw->max_frame_size = max_frame;
- if (netif_running(netdev))
+ if (netif_running(netdev)) {
+ /* prevent buffers from being reallocated */
+ adapter->alloc_rx_buf = e1000_alloc_dummy_rx_buffers;
e1000_down(adapter);
+ }
/* NOTE: netdev_alloc_skb reserves 16 bytes, and typically NET_IP_ALIGN
* means we reserve 2 more, this pushes us to allocate from the next
--
1.7.12.2.21.g234cd45.dirty
next prev parent reply other threads:[~2015-09-12 22:56 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <08d3b586eb2e764308c3de9ee398a17c@local>
2015-09-12 22:56 ` [PATCH 2.6.32 00/62] 2.6.32.68-longterm review Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 01/62] pipe: iovec: Fix memory corruption when retrying atomic copy as non-atomic Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 02/62] sg_start_req(): make sure that theres not too many elements in iovec Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 03/62] crypto: testmgr - update LZO compression test vectors Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 04/62] TTY: drop driver reference in tty_open fail path Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 05/62] netlink: fix possible spoofing from non-root processes Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 06/62] eCryptfs: Remove buggy and unnecessary write in file name decode routine Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 07/62] HID: fix a couple of off-by-ones Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 08/62] udf: Verify i_size when loading inode Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 09/62] udf: Verify symlink size before loading it Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 11/62] udf: Check path length when reading symlink Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 12/62] udf: Check component length before reading it Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 13/62] Remove repeated loads blocksize Willy Tarreau
2015-09-15 1:42 ` Ben Hutchings
2015-09-15 7:39 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 14/62] udf: Check length of extended attributes and allocation descriptors Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 15/62] udp: fix behavior of wrong checksums Willy Tarreau
2015-09-15 1:44 ` Ben Hutchings
2015-09-15 7:41 ` Willy Tarreau
2015-09-12 22:56 ` Willy Tarreau [this message]
2015-09-12 22:56 ` [PATCH 2.6.32 17/62] powerpc: Fix missing L2 cache size in /sys/devices/system/cpu Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 18/62] s390/hibernate: fix save and restore of kernel text section Willy Tarreau
2015-09-15 2:10 ` Ben Hutchings
2015-09-15 6:09 ` Heiko Carstens
2015-09-15 7:41 ` Willy Tarreau
2015-09-15 7:44 ` Heiko Carstens
2015-09-12 22:56 ` [PATCH 2.6.32 19/62] ptrace: fix race between ptrace_resume() and wait_task_stopped() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 20/62] memstick: mspro_block: add missing curly braces Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 21/62] md/raid5: dont record new size if resize_stripes fails Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 22/62] powerpc: Align TOC to 256 bytes Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 23/62] jbd2: fix r_count overflows leading to buffer overflow in journal recovery Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 24/62] sd: Disable support for 256 byte/sector disks Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 25/62] lguest: fix out-by-one error in address checking Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 26/62] fs, omfs: add NULL terminator in the end up the token list Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 27/62] x86_64: Fix strnlen_user() to not touch memory after specified maximum Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 28/62] tracing: Have filter check for balanced ops Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 29/62] sctp: Fix race between OOTB responce and route removal Willy Tarreau
2015-09-15 2:26 ` Ben Hutchings
2015-09-15 7:43 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 30/62] x86/reboot: Fix a warning message triggered by stop_other_cpus() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 31/62] include/linux/sched.h: dont use task->pid/tgid in same_thread_group/has_group_leader_pid Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 32/62] __ptrace_may_access() should not deny sub-threads Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 33/62] MIPS: Octeon: Remove udelay() causing huge IRQ latency Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 34/62] MIPS: Fix race condition in lazy cache flushing Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 35/62] MIPS: Fix cpu_has_mips_r2_exec_hazard Willy Tarreau
2015-09-15 3:02 ` Ben Hutchings
2015-09-15 7:43 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 36/62] MIPS: Octeon: Delete override of cpu_has_mips_r2_exec_hazard Willy Tarreau
2015-09-15 11:37 ` Ben Hutchings
2015-09-16 5:17 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 37/62] hrtimer: Allow concurrent hrtimer_start() for self restarting timers Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 38/62] s5h1420: fix a buffer overflow when checking userspace params Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 39/62] cx24116: " Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 40/62] pktgen: adjust spacing in proc file interface output Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 41/62] staging: vt6655: device_rx_srv check sk_buff is NULL Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 42/62] fixing infinite OPEN loop in 4.0 stateid recovery Willy Tarreau
2015-09-14 23:54 ` Kornievskaia, Olga
2015-09-15 5:37 ` Willy Tarreau
[not found] ` <7E69BA76-E3DF-4389-A8C4-F23C9E1FD5CC@netapp.com>
[not found] ` <55D3ECC9-EC69-469F-AD46-EE3818F3D138@netapp.com>
2015-09-16 5:33 ` Willy Tarreau
2015-11-15 0:53 ` [stable] Failing to send a CLOSE if file is opened WRONLY and server reboots on a 4.x mount Ben Hutchings
2015-11-15 7:20 ` Willy Tarreau
2015-11-18 23:20 ` Luis Henriques
2015-09-12 22:56 ` [PATCH 2.6.32 43/62] SUNRPC: Fix a memory leak in the backchannel code Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 44/62] dmaengine: mv_xor: bug fix for racing condition in descriptors cleanup Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 45/62] ext4: fix race between truncate and __ext4_journalled_writepage() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 46/62] Disable write buffering on Toshiba ToPIC95 Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 47/62] nfs: increase size of EXCHANGE_ID name string buffer Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 48/62] ext4: call sync_blockdev() before invalidate_bdev() in put_super() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 49/62] NET: ROSE: Dont dereference NULL neighbour pointer Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 50/62] mm: kmemleak: allow safe memory scanning during kmemleak disabling Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 51/62] tracing/filter: Do not WARN on operand count going below zero Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 52/62] tracing/filter: Do not allow infix to exceed end of string Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 53/62] fuse: initialize fc->release before calling it Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 54/62] mm: avoid setting up anonymous pages into file mapping Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 55/62] s390/process: fix sfpc inline assembly Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 56/62] libata: increase the timeout when setting transfer mode Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 57/62] x86/xen: Probe target addresses in set_aliased_prot() before the hypercall Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 58/62] dccp: fix auto-loading of dccp(_probe) Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 60/62] dccp: catch failed request_module call in dccp_probe init Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 61/62] dmaengine: fix missing cnt in ?: in dmatest Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 62/62] ipv6: Fix return of xfrm6_tunnel_rcv() Willy Tarreau
2015-09-12 23:18 ` [PATCH 2.6.32 00/62] 2.6.32.68-longterm review Willy Tarreau
2015-09-15 12:06 ` Ben Hutchings
2015-09-16 5:23 ` Willy Tarreau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150912225607.258086407@1wt.eu \
--to=w@1wt.eu \
--cc=aaron.f.brown@intel.com \
--cc=ben@decadent.org.uk \
--cc=jeffrey.t.kirsher@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=sd@queasysnail.net \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).