From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: Kernel panic when using bridge Date: Tue, 12 Apr 2011 13:49:11 +0200 Message-ID: <1302608951.3233.33.camel@edumazet-laptop> References: <4D9E62D9.5010400@scotdoyle.com> <20110408121700.0aad53fe@nehalam> <4D9FE5BE.6060600@scotdoyle.com> <20110409161908.a2aca120.shimoda.hiroaki@gmail.com> <4DA39330.2030102@scotdoyle.com> <20110411183105.46e86684@nehalam> <4DA3CB4B.9090506@scotdoyle.com> <1302581384.3603.14.camel@edumazet-laptop> <1302582172.3603.18.camel@edumazet-laptop> <4DA3E074.5090603@scotdoyle.com> <1302587490.3603.22.camel@edumazet-laptop> <4DA3F909.5020609@scotdoyle.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Stephen Hemminger , Hiroaki SHIMODA , netdev@vger.kernel.org, Jan Luebbe To: Scot Doyle Return-path: Received: from mail-ww0-f44.google.com ([74.125.82.44]:62753 "EHLO mail-ww0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751028Ab1DLLxK (ORCPT ); Tue, 12 Apr 2011 07:53:10 -0400 Received: by wwa36 with SMTP id 36so7795022wwa.1 for ; Tue, 12 Apr 2011 04:53:09 -0700 (PDT) In-Reply-To: <4DA3F909.5020609@scotdoyle.com> Sender: netdev-owner@vger.kernel.org List-ID: Le mardi 12 avril 2011 =C3=A0 02:02 -0500, Scot Doyle a =C3=A9crit : > On 04/12/2011 12:51 AM, Eric Dumazet wrote: > > > > Oh well, sorry (not enough time these days to even test patches) > > > > if (!skb_dst(skb)) { >=20 > --- br_netfilter.c.a 2011-04-01 02:37:53.000000000 -0500 > +++ br_netfilter.c.b 2011-04-12 00:29:00.000000000 -0500 > @@ -221,6 +221,7 @@ static int br_parse_ip_options(struct sk > struct ip_options *opt; > struct iphdr *iph; > struct net_device *dev =3D skb->dev; > + struct rtable *rt; > u32 len; >=20 > iph =3D ip_hdr(skb); > @@ -255,6 +256,16 @@ static int br_parse_ip_options(struct sk > return 0; > } >=20 > + /* Associate bogus bridge route table */ > + if (!skb_dst(skb)) { > + rt =3D bridge_parent_rtable(dev); > + if (!rt) { > + kfree_skb(skb); > + return 0; > + } > + skb_dst_set_noref(skb,&rt->dst); > + } > + > opt->optlen =3D iph->ihl*4 - sizeof(struct iphdr); > if (ip_options_compile(dev_net(dev), opt, skb)) > goto inhdr_error; >=20 >=20 > Now we are making progress! With the patch above from Stephen and Eri= c,=20 > I cannot make the kernel panic when sending packets to the IP address= of=20 > the bridge. >=20 > However, if a guest virtual machine is sharing the bridge with the ho= st=20 > via a tap device, I can cause a host panic by targeting the IP addres= s=20 > of the guest. Is this an unrelated problem? >=20 > Here are two kernel panics. The guest virtual machine was pingable=20 > before being attacked with IP Stack Checker's tcpsic command. Spannin= g=20 > Tree Protocol was off during the first panic and on during the second= =2E >=20 > ------------ >=20 > [ 606.921739] br0: port 2(tap0) entering forwarding state > [ 636.058941] Kernel panic - not syncing: stack-protector: Kernel st= ack=20 > is corrupted in: ffffffff812c2781 > [ 636.058942] > [ 636.069789] Pid: 2261, comm: kvm Tainted: G W 2.6.39-rc2+= #11 > [ 636.076292] Call Trace: > [ 636.078725] [] ? panic+0x92/0x1a1 > [ 636.084287] [] ? _local_bh_enable_ip.clone.8+0x= 20/0x8c > [ 636.091044] [] ? icmp_send+0x337/0x349 > [ 636.096418] [] ? __stack_chk_fail+0x17/0x17 > [ 636.102221] [] ? icmp_send+0x337/0x349 > [ 636.107595] [] ? nf_iterate+0x41/0x7e > [ 636.112883] [] ? nf_iterate+0x41/0x7e > [ 636.118172] [] ? br_flood+0xc8/0xc8 [bridge] > [ 636.124065] [] ? __br_deliver+0xb0/0xb0 [bridge= ] > [ 636.130302] [] ? nf_hook_slow+0x73/0x114 > [ 636.135850] [] ? __br_deliver+0xb0/0xb0 [bridge= ] > [ 636.142089] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 636.148586] [] ? __br_deliver+0xb0/0xb0 [bridge= ] > [ 636.154826] [] ? NF_HOOK.clone.5+0x3c/0x56 [bri= dge] > [ 636.161323] [] ?=20 > br_handle_frame_finish+0x158/0x1c7 [bridge] > [ 636.168601] [] ?=20 > br_nf_pre_routing_finish+0x1d4/0x1e1 [bridge] > [ 636.176052] [] ? NF_HOOK_THRESH+0x3b/0x55 [brid= ge] > [ 636.182463] [] ? br_nf_pre_routing+0x3be/0x3cb=20 > [bridge] > [ 636.189307] [] ? nf_hook_slow+0x73/0x114 > [ 636.194852] [] ? nf_iterate+0x41/0x7e > [ 636.200139] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 636.206637] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 636.213133] [] ? nf_hook_slow+0x73/0x114 > [ 636.218679] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 636.225177] [] ?=20 > br_handle_frame_finish+0x158/0x1c7 [bridge] > [ 636.232455] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 636.238954] [] ? NF_HOOK.clone.4+0x3c/0x56 [bri= dge] > [ 636.245452] [] ? tcp_gro_receive+0xa1/0x204 > [ 636.251258] [] ? br_handle_frame+0x195/0x1ac [b= ridge] > [ 636.257928] [] ?=20 > br_handle_frame_finish+0x1c7/0x1c7 [bridge] > [ 636.265204] [] ? __netif_receive_skb+0x2a7/0x45= 0 > [ 636.271443] [] ? netif_receive_skb+0x52/0x58 > [ 636.277335] [] ? napi_gro_receive+0x1f/0x2f > [ 636.283139] [] ? napi_skb_finish+0x1c/0x31 > [ 636.288865] [] ? igb_poll+0x6d9/0x9ee [igb] > [ 636.294673] [] ? scsi_run_queue+0x2ce/0x30a [sc= si_mod] > [ 636.301431] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 636.307930] [] ? __netif_receive_skb+0x2a7/0x45= 0 > [ 636.314168] [] ? net_rx_action+0xa4/0x1b1 > [ 636.319800] [] ? __do_softirq+0xb8/0x176 > [ 636.325346] [] ? call_softirq+0x1c/0x30 > [ 636.330807] [] ? do_softirq+0x3f/0x84 > [ 636.336092] [] ? irq_exit+0x3f/0x8f > [ 636.341204] [] ? do_IRQ+0x85/0x9e > [ 636.346146] [] ? common_interrupt+0x13/0x13 > [ 636.351949] [] ? arch_local_irq_save+0x12= /0x1b > [ 636.358629] [] ? arch_local_irq_restore+0x2/0x8 > [ 636.364781] [] ? netif_rx_ni+0x1e/0x27 > [ 636.370154] [] ? tun_get_user+0x3a3/0x3cb [tun] > [ 636.376305] [] ? tun_get_socket+0x3b/0x3b [tun] > [ 636.382457] [] ? tun_chr_aio_write+0x5e/0x79 [t= un] > [ 636.388869] [] ? do_sync_readv_writev+0x9a/0xd5 > [ 636.395021] [] ? need_resched+0x1a/0x23 > [ 636.400481] [] ? _cond_resched+0x9/0x20 > [ 636.405941] [] ? copy_from_user+0x18/0x30 > [ 636.411573] [] ? security_file_permission+0x18/= 0x33 > [ 636.418068] [] ? do_readv_writev+0xa4/0x11a > [ 636.423873] [] ? fput+0x1a/0x1a2 > [ 636.428726] [] ? sys_writev+0x45/0x90 > [ 636.434012] [] ? system_call_fastpath+0x16/0x1b >=20 > ------------ >=20 > [ 110.442839] br0: port 2(tap0) entering forwarding state > [ 136.948700] Kernel panic - not syncing: stack-protector: Kernel st= ack=20 > is corrupted in: ffffffff812c2781 > [ 136.948702] > [ 136.959561] Pid: 1093, comm: md123_resync Not tainted 2.6.39-rc2+ = #11 > [ 136.965977] Call Trace: > [ 136.968408] [] ? panic+0x92/0x1a1 > [ 136.973970] [] ? _local_bh_enable_ip.clone.8+0x= 20/0x8c > [ 136.980727] [] ? icmp_send+0x337/0x349 > [ 136.986102] [] ? __stack_chk_fail+0x17/0x17 > [ 136.991906] [] ? icmp_send+0x337/0x349 > [ 136.997281] [] ? nf_iterate+0x41/0x7e > [ 137.002570] [] ?=20 > br_handle_frame_finish+0x158/0x1c7 [bridge] > [ 137.009847] [] ?=20 > br_nf_pre_routing_finish+0x1d4/0x1e1 [bridge] > [ 137.017297] [] ? NF_HOOK_THRESH+0x3b/0x55 [brid= ge] > [ 137.023707] [] ? br_nf_pre_routing+0x3be/0x3cb=20 > [bridge] > [ 137.030551] [] ? nf_iterate+0x41/0x7e > [ 137.035837] [] ? test_tsk_need_resched+0xe/0x17 > [ 137.041991] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 137.048488] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 137.054984] [] ? nf_hook_slow+0x73/0x114 > [ 137.060531] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 137.067028] [] ? NF_HOOK.clone.4+0x56/0x56 [bri= dge] > [ 137.073526] [] ? NF_HOOK.clone.4+0x3c/0x56 [bri= dge] > [ 137.080023] [] ? tcp_gro_receive+0xa1/0x204 > [ 137.085830] [] ? br_handle_frame+0x195/0x1ac [b= ridge] > [ 137.092500] [] ?=20 > br_handle_frame_finish+0x1c7/0x1c7 [bridge] > [ 137.099776] [] ? __netif_receive_skb+0x2a7/0x45= 0 > [ 137.106013] [] ? netif_receive_skb+0x52/0x58 > [ 137.111906] [] ? napi_gro_receive+0x1f/0x2f > [ 137.117713] [] ? napi_skb_finish+0x1c/0x31 > [ 137.123438] [] ? igb_poll+0x6d9/0x9ee [igb] > [ 137.129243] [] ? handle_irq_event+0x40/0x55 > [ 137.135049] [] ? common_interrupt+0x13/0x13 > [ 137.140854] [] ? net_rx_action+0xa4/0x1b1 > [ 137.146487] [] ? __do_softirq+0xb8/0x176 > [ 137.152034] [] ? call_softirq+0x1c/0x30 > [ 137.157494] [] ? do_softirq+0x3f/0x84 > [ 137.162779] [] ? irq_exit+0x3f/0x8f > [ 137.167893] [] ? do_IRQ+0x85/0x9e > [ 137.172833] [] ? common_interrupt+0x13/0x13 > [ 137.178636] [] ? arch_local_irq_restore+0= x2/0x8 > [ 137.185408] [] ? _scsih_qcmd+0x54f/0x561 [mpt2s= as] > [ 137.191823] [] ? scsi_dispatch_cmd+0x180/0x219=20 > [scsi_mod] > [ 137.198841] [] ? scsi_request_fn+0x3e6/0x413=20 > [scsi_mod] > [ 137.205683] [] ? elv_rqhash_add.clone.15+0x26/0= x4c > [ 137.212095] [] ? __blk_run_queue+0x5e/0x84 > [ 137.217814] [] ? __make_request+0x273/0x28f > [ 137.223619] [] ? generic_make_request+0x267/0x2= e1 > [ 137.229943] [] ? remove_wait_queue+0x11/0x4d > [ 137.235837] [] ? raise_barrier+0x162/0x16f [rai= d1] > [ 137.242246] [] ? try_to_wake_up+0x17c/0x17c > [ 137.248052] [] ? sync_request+0x567/0x583 [raid= 1] > [ 137.254379] [] ? md_do_sync+0x776/0xb8e [md_mod= ] > [ 137.260617] [] ? sched_clock+0x5/0x8 > [ 137.265819] [] ? md_thread+0xfa/0x118 [md_mod] > [ 137.271886] [] ? md_rdev_init+0x8f/0x8f [md_mod= ] > [ 137.278124] [] ? md_rdev_init+0x8f/0x8f [md_mod= ] > [ 137.284362] [] ? kthread+0x7a/0x82 > [ 137.289390] [] ? kernel_thread_helper+0x4/0x10 > [ 137.295454] [] ? kthread_worker_fn+0x149/0x149 > [ 137.301519] [] ? gs_change+0x13/0x13 >=20 Considering recent changes in ip_options_echo() I would suggest to add following patch and/or revert commit 8628bd8af7c4c14f40 (ipv4: Fix IP timestamp option (IPOPT_TS_PRESPEC) handling in ip_options_echo()) Thanks diff --git a/net/ipv4/ip_options.c b/net/ipv4/ip_options.c index 28a736f..35f2bf9 100644 --- a/net/ipv4/ip_options.c +++ b/net/ipv4/ip_options.c @@ -200,6 +200,11 @@ int ip_options_echo(struct ip_options * dopt, stru= ct sk_buff * skb) *dptr++ =3D IPOPT_END; dopt->optlen++; } + if (unlikely(dopt->optlen > 40)) { + pr_err("ip_options_echo() fatal error optlen=3D%u > 40\n", dopt->opt= len); + print_hex_dump(KERN_ERR, "ip options: ", DUMP_PREFIX_OFFSET, + 16, 1, dopt->__data, dopt->optlen, false); + } return 0; } =20