From: Willy Tarreau <w@1wt.eu>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Alexander Sverdlin <alexander.sverdlin@nokia.com>,
Neil Horman <nhorman@tuxdriver.com>,
Marcelo Ricardo Leitner <marcelo.leitner@gmail.com>,
Vlad Yasevich <vyasevich@gmail.com>,
"David S. Miller" <davem@davemloft.net>,
Ben Hutchings <ben@decadent.org.uk>, Willy Tarreau <w@1wt.eu>
Subject: [PATCH 2.6.32 29/62] sctp: Fix race between OOTB responce and route removal
Date: Sun, 13 Sep 2015 00:56:35 +0200 [thread overview]
Message-ID: <20150912225607.810043732@1wt.eu> (raw)
In-Reply-To: <08d3b586eb2e764308c3de9ee398a17c@local>
2.6.32-longterm review patch. If anyone has any objections, please let me know.
------------------
From: Alexander Sverdlin <alexander.sverdlin@nokia.com>
[ Upstream commit 29c4afc4e98f4dc0ea9df22c631841f9c220b944 ]
There is NULL pointer dereference possible during statistics update if the route
used for OOTB responce is removed at unfortunate time. If the route exists when
we receive OOTB packet and we finally jump into sctp_packet_transmit() to send
ABORT, but in the meantime route is removed under our feet, we take "no_route"
path and try to update stats with IP_INC_STATS(sock_net(asoc->base.sk), ...).
But sctp_ootb_pkt_new() used to prepare responce packet doesn't call
sctp_transport_set_owner() and therefore there is no asoc associated with this
packet. Probably temporary asoc just for OOTB responces is overkill, so just
introduce a check like in all other places in sctp_packet_transmit(), where
"asoc" is dereferenced.
To reproduce this, one needs to
0. ensure that sctp module is loaded (otherwise ABORT is not generated)
1. remove default route on the machine
2. while true; do
ip route del [interface-specific route]
ip route add [interface-specific route]
done
3. send enough OOTB packets (i.e. HB REQs) from another host to trigger ABORT
responce
On x86_64 the crash looks like this:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000020
IP: [<ffffffffa05ec9ac>] sctp_packet_transmit+0x63c/0x730 [sctp]
PGD 0
Oops: 0000 [#1] PREEMPT SMP
Modules linked in: ...
CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.0.5-1-ARCH #1
Hardware name: ...
task: ffffffff818124c0 ti: ffffffff81800000 task.ti: ffffffff81800000
RIP: 0010:[<ffffffffa05ec9ac>] [<ffffffffa05ec9ac>] sctp_packet_transmit+0x63c/0x730 [sctp]
RSP: 0018:ffff880127c037b8 EFLAGS: 00010296
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00000015ff66b480
RDX: 00000015ff66b400 RSI: ffff880127c17200 RDI: ffff880123403700
RBP: ffff880127c03888 R08: 0000000000017200 R09: ffffffff814625af
R10: ffffea00047e4680 R11: 00000000ffffff80 R12: ffff8800b0d38a28
R13: ffff8800b0d38a28 R14: ffff8800b3e88000 R15: ffffffffa05f24e0
FS: 0000000000000000(0000) GS:ffff880127c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000020 CR3: 00000000c855b000 CR4: 00000000000007f0
Stack:
ffff880127c03910 ffff8800b0d38a28 ffffffff8189d240 ffff88011f91b400
ffff880127c03828 ffffffffa05c94c5 0000000000000000 ffff8800baa1c520
0000000000000000 0000000000000001 0000000000000000 0000000000000000
Call Trace:
<IRQ>
[<ffffffffa05c94c5>] ? sctp_sf_tabort_8_4_8.isra.20+0x85/0x140 [sctp]
[<ffffffffa05d6b42>] ? sctp_transport_put+0x52/0x80 [sctp]
[<ffffffffa05d0bfc>] sctp_do_sm+0xb8c/0x19a0 [sctp]
[<ffffffff810b0e00>] ? trigger_load_balance+0x90/0x210
[<ffffffff810e0329>] ? update_process_times+0x59/0x60
[<ffffffff812c7a40>] ? timerqueue_add+0x60/0xb0
[<ffffffff810e0549>] ? enqueue_hrtimer+0x29/0xa0
[<ffffffff8101f599>] ? read_tsc+0x9/0x10
[<ffffffff8116d4b5>] ? put_page+0x55/0x60
[<ffffffff810ee1ad>] ? clockevents_program_event+0x6d/0x100
[<ffffffff81462b68>] ? skb_free_head+0x58/0x80
[<ffffffffa029a10b>] ? chksum_update+0x1b/0x27 [crc32c_generic]
[<ffffffff81283f3e>] ? crypto_shash_update+0xce/0xf0
[<ffffffffa05d3993>] sctp_endpoint_bh_rcv+0x113/0x280 [sctp]
[<ffffffffa05dd4e6>] sctp_inq_push+0x46/0x60 [sctp]
[<ffffffffa05ed7a0>] sctp_rcv+0x880/0x910 [sctp]
[<ffffffffa05ecb50>] ? sctp_packet_transmit_chunk+0xb0/0xb0 [sctp]
[<ffffffffa05ecb70>] ? sctp_csum_update+0x20/0x20 [sctp]
[<ffffffff814b05a5>] ? ip_route_input_noref+0x235/0xd30
[<ffffffff81051d6b>] ? ack_ioapic_level+0x7b/0x150
[<ffffffff814b27be>] ip_local_deliver_finish+0xae/0x210
[<ffffffff814b2e15>] ip_local_deliver+0x35/0x90
[<ffffffff814b2a15>] ip_rcv_finish+0xf5/0x370
[<ffffffff814b3128>] ip_rcv+0x2b8/0x3a0
[<ffffffff81474193>] __netif_receive_skb_core+0x763/0xa50
[<ffffffff81476c28>] __netif_receive_skb+0x18/0x60
[<ffffffff81476cb0>] netif_receive_skb_internal+0x40/0xd0
[<ffffffff814776c8>] napi_gro_receive+0xe8/0x120
[<ffffffffa03946aa>] rtl8169_poll+0x2da/0x660 [r8169]
[<ffffffff8147896a>] net_rx_action+0x21a/0x360
[<ffffffff81078dc1>] __do_softirq+0xe1/0x2d0
[<ffffffff8107912d>] irq_exit+0xad/0xb0
[<ffffffff8157d158>] do_IRQ+0x58/0xf0
[<ffffffff8157b06d>] common_interrupt+0x6d/0x6d
<EOI>
[<ffffffff810e1218>] ? hrtimer_start+0x18/0x20
[<ffffffffa05d65f9>] ? sctp_transport_destroy_rcu+0x29/0x30 [sctp]
[<ffffffff81020c50>] ? mwait_idle+0x60/0xa0
[<ffffffff810216ef>] arch_cpu_idle+0xf/0x20
[<ffffffff810b731c>] cpu_startup_entry+0x3ec/0x480
[<ffffffff8156b365>] rest_init+0x85/0x90
[<ffffffff818eb035>] start_kernel+0x48b/0x4ac
[<ffffffff818ea120>] ? early_idt_handlers+0x120/0x120
[<ffffffff818ea339>] x86_64_start_reservations+0x2a/0x2c
[<ffffffff818ea49c>] x86_64_start_kernel+0x161/0x184
Code: 90 48 8b 80 b8 00 00 00 48 89 85 70 ff ff ff 48 83 bd 70 ff ff ff 00 0f 85 cd fa ff ff 48 89 df 31 db e8 18 63 e7 e0 48 8b 45 80 <48> 8b 40 20 48 8b 40 30 48 8b 80 68 01 00 00 65 48 ff 40 78 e9
RIP [<ffffffffa05ec9ac>] sctp_packet_transmit+0x63c/0x730 [sctp]
RSP <ffff880127c037b8>
CR2: 0000000000000020
---[ end trace 5aec7fd2dc983574 ]---
Kernel panic - not syncing: Fatal exception in interrupt
Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
drm_kms_helper: panic occurred, switching back to text console
---[ end Kernel panic - not syncing: Fatal exception in interrupt
Signed-off-by: Alexander Sverdlin <alexander.sverdlin@nokia.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Marcelo Ricardo Leitner <marcelo.leitner@gmail.com>
Acked-by: Vlad Yasevich <vyasevich@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
[bwh: Backported to 3.2: sctp alway uses init_net]
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
(cherry picked from commit 117b8a10fe0c434d9043267efd51f3ba3f3d359a)
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
net/sctp/output.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/net/sctp/output.c b/net/sctp/output.c
index 432361b..88ed1f9 100644
--- a/net/sctp/output.c
+++ b/net/sctp/output.c
@@ -580,7 +580,9 @@ out:
return err;
no_route:
kfree_skb(nskb);
- IP_INC_STATS(&init_net, IPSTATS_MIB_OUTNOROUTES);
+
+ if (asoc)
+ IP_INC_STATS(&init_net, IPSTATS_MIB_OUTNOROUTES);
/* FIXME: Returning the 'err' will effect all the associations
* associated with a socket, although only one of the paths of the
--
1.7.12.2.21.g234cd45.dirty
next prev parent reply other threads:[~2015-09-12 23:12 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-12 22:56 [PATCH 2.6.32 00/62] 2.6.32.68-longterm review Willy Tarreau
2015-09-12 22:56 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 01/62] pipe: iovec: Fix memory corruption when retrying atomic copy as non-atomic Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 02/62] sg_start_req(): make sure that theres not too many elements in iovec Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 03/62] crypto: testmgr - update LZO compression test vectors Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 04/62] TTY: drop driver reference in tty_open fail path Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 05/62] netlink: fix possible spoofing from non-root processes Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 06/62] eCryptfs: Remove buggy and unnecessary write in file name decode routine Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 07/62] HID: fix a couple of off-by-ones Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 08/62] udf: Verify i_size when loading inode Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 09/62] udf: Verify symlink size before loading it Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 11/62] udf: Check path length when reading symlink Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 12/62] udf: Check component length before reading it Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 13/62] Remove repeated loads blocksize Willy Tarreau
2015-09-15 1:42 ` Ben Hutchings
2015-09-15 7:39 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 14/62] udf: Check length of extended attributes and allocation descriptors Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 15/62] udp: fix behavior of wrong checksums Willy Tarreau
2015-09-15 1:44 ` Ben Hutchings
2015-09-15 7:41 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 16/62] e1000: add dummy allocator to fix race condition between mtu change and netpoll Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 17/62] powerpc: Fix missing L2 cache size in /sys/devices/system/cpu Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 18/62] s390/hibernate: fix save and restore of kernel text section Willy Tarreau
2015-09-15 2:10 ` Ben Hutchings
2015-09-15 6:09 ` Heiko Carstens
2015-09-15 7:41 ` Willy Tarreau
2015-09-15 7:44 ` Heiko Carstens
2015-09-12 22:56 ` [PATCH 2.6.32 19/62] ptrace: fix race between ptrace_resume() and wait_task_stopped() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 20/62] memstick: mspro_block: add missing curly braces Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 21/62] md/raid5: dont record new size if resize_stripes fails Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 22/62] powerpc: Align TOC to 256 bytes Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 23/62] jbd2: fix r_count overflows leading to buffer overflow in journal recovery Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 24/62] sd: Disable support for 256 byte/sector disks Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 25/62] lguest: fix out-by-one error in address checking Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 26/62] fs, omfs: add NULL terminator in the end up the token list Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 27/62] x86_64: Fix strnlen_user() to not touch memory after specified maximum Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 28/62] tracing: Have filter check for balanced ops Willy Tarreau
2015-09-12 22:56 ` Willy Tarreau [this message]
2015-09-15 2:26 ` [PATCH 2.6.32 29/62] sctp: Fix race between OOTB responce and route removal Ben Hutchings
2015-09-15 7:43 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 30/62] x86/reboot: Fix a warning message triggered by stop_other_cpus() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 31/62] include/linux/sched.h: dont use task->pid/tgid in same_thread_group/has_group_leader_pid Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 32/62] __ptrace_may_access() should not deny sub-threads Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 33/62] MIPS: Octeon: Remove udelay() causing huge IRQ latency Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 34/62] MIPS: Fix race condition in lazy cache flushing Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 35/62] MIPS: Fix cpu_has_mips_r2_exec_hazard Willy Tarreau
2015-09-15 3:02 ` Ben Hutchings
2015-09-15 7:43 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 36/62] MIPS: Octeon: Delete override of cpu_has_mips_r2_exec_hazard Willy Tarreau
2015-09-15 11:37 ` Ben Hutchings
2015-09-16 5:17 ` Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 37/62] hrtimer: Allow concurrent hrtimer_start() for self restarting timers Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 38/62] s5h1420: fix a buffer overflow when checking userspace params Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 39/62] cx24116: " Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 40/62] pktgen: adjust spacing in proc file interface output Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 41/62] staging: vt6655: device_rx_srv check sk_buff is NULL Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 42/62] fixing infinite OPEN loop in 4.0 stateid recovery Willy Tarreau
2015-09-14 23:54 ` Kornievskaia, Olga
2015-09-15 5:37 ` Willy Tarreau
[not found] ` <7E69BA76-E3DF-4389-A8C4-F23C9E1FD5CC@netapp.com>
[not found] ` <55D3ECC9-EC69-469F-AD46-EE3818F3D138@netapp.com>
2015-09-16 5:33 ` Willy Tarreau
2015-11-15 0:53 ` [stable] Failing to send a CLOSE if file is opened WRONLY and server reboots on a 4.x mount Ben Hutchings
2015-11-15 7:20 ` Willy Tarreau
2015-11-18 23:20 ` Luis Henriques
2015-11-18 23:20 ` Luis Henriques
2015-09-12 22:56 ` [PATCH 2.6.32 43/62] SUNRPC: Fix a memory leak in the backchannel code Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 44/62] dmaengine: mv_xor: bug fix for racing condition in descriptors cleanup Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 45/62] ext4: fix race between truncate and __ext4_journalled_writepage() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 46/62] Disable write buffering on Toshiba ToPIC95 Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 47/62] nfs: increase size of EXCHANGE_ID name string buffer Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 48/62] ext4: call sync_blockdev() before invalidate_bdev() in put_super() Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 49/62] NET: ROSE: Dont dereference NULL neighbour pointer Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 50/62] mm: kmemleak: allow safe memory scanning during kmemleak disabling Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 51/62] tracing/filter: Do not WARN on operand count going below zero Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 52/62] tracing/filter: Do not allow infix to exceed end of string Willy Tarreau
2015-09-12 22:56 ` [PATCH 2.6.32 53/62] fuse: initialize fc->release before calling it Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 54/62] mm: avoid setting up anonymous pages into file mapping Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 55/62] s390/process: fix sfpc inline assembly Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 56/62] libata: increase the timeout when setting transfer mode Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 57/62] x86/xen: Probe target addresses in set_aliased_prot() before the hypercall Willy Tarreau
2015-09-12 22:57 ` Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 58/62] dccp: fix auto-loading of dccp(_probe) Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 60/62] dccp: catch failed request_module call in dccp_probe init Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 61/62] dmaengine: fix missing cnt in ?: in dmatest Willy Tarreau
2015-09-12 22:57 ` [PATCH 2.6.32 62/62] ipv6: Fix return of xfrm6_tunnel_rcv() Willy Tarreau
2015-09-12 23:18 ` [PATCH 2.6.32 00/62] 2.6.32.68-longterm review Willy Tarreau
2015-09-15 12:06 ` Ben Hutchings
2015-09-16 5:23 ` Willy Tarreau
2015-09-16 5:23 ` Willy Tarreau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150912225607.810043732@1wt.eu \
--to=w@1wt.eu \
--cc=alexander.sverdlin@nokia.com \
--cc=ben@decadent.org.uk \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=marcelo.leitner@gmail.com \
--cc=nhorman@tuxdriver.com \
--cc=stable@vger.kernel.org \
--cc=vyasevich@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.