From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Parkin Subject: Re: LOCKDEP complaints in l2tp_xmit_skb() Date: Thu, 28 Jun 2012 16:20:11 +0100 Message-ID: <20120628152010.GC2507@raven> References: <20120627111152.GA2531@raven> <1340866572.26242.312.camel@edumazet-glaptop> <1340873862.13187.5.camel@edumazet-glaptop> <1340882551.13187.96.camel@edumazet-glaptop> <20120628143302.GB2507@raven> <1340895642.13187.107.camel@edumazet-glaptop> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="96YOpH+ONegL0A3E" Cc: David Miller , netdev To: Eric Dumazet Return-path: Received: from katalix.com ([82.103.140.233]:42777 "EHLO mail.katalix.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753921Ab2F1PUN (ORCPT ); Thu, 28 Jun 2012 11:20:13 -0400 Content-Disposition: inline In-Reply-To: <1340895642.13187.107.camel@edumazet-glaptop> Sender: netdev-owner@vger.kernel.org List-ID: --96YOpH+ONegL0A3E Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 28, 2012 at 05:00:42PM +0200, Eric Dumazet wrote: > On Thu, 2012-06-28 at 15:33 +0100, Tom Parkin wrote: > > On Thu, Jun 28, 2012 at 01:22:31PM +0200, Eric Dumazet wrote: > > > On Thu, 2012-06-28 at 10:57 +0200, Eric Dumazet wrote: > > > > On Thu, 2012-06-28 at 08:56 +0200, Eric Dumazet wrote: > > > >=20 > > > > > [PATCH] net: Qdisc busylock gets its own lockdep class > > > > >=20 > > > > > Tom Parkin reported following LOCKDEP splat : > > > > .. > > > > >=20 > > > > > Instruct lockdep that each Qdisc busylock is independant, or else > > > > > bonding or various tunnels can trigger a splat. > > > > >=20 > > > > > Reported-by: Tom Parkin > > > > > Signed-off-by: Eric Dumazet > > > > > --- > > > >=20 > > > > I reproduced the bug using a bond0 device, adding a qdisc on it, > > > > (one Qdisc on bond0, and a Qdisc on the slave too) > > > >=20 > > > > Problem with this patch is I have following message : > > > >=20 > > > > BUG: key ffff88..... not in .data! > > > >=20 > > > > No more LOCKDEP splat, but patch not good as is. > > >=20 > > > I tested the alternative following patch with my bonding setup, > > > could you test it with l2tp ? > > > > >=20 > > I've tested against my l2tp test configuration and I still see LOCKDEP > > splats: > >=20 > > 2tp_core: L2TP core driver, V2.0 > > 2tp_netlink: L2TP netlink interface > > 2tp_eth: L2TP ethernet pseudowire support (L2TPv3) > >=20 > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > INFO: possible recursive locking detected ] > > .5.0-rc2-net-lockdep-u64-sync-007-+ #1 Not tainted > > -------------------------------------------- > > wapper/0/0 is trying to acquire lock: > > (slock-AF_INET){+.-...}, at: [] l2tp_xmit_skb+0x13f/0x8e0 [l2= tp_core] > >=20 > > ut task is already holding lock: > > (slock-AF_INET){+.-...}, at: [] ip_send_reply+0x107/0x2b0 > >=20 > > ther info that might help us debug this: > > Possible unsafe locking scenario: > >=20 > > CPU0 > > ---- > > lock(slock-AF_INET); > > lock(slock-AF_INET); > >=20 > > *** DEADLOCK *** > >=20 > > May be due to missing lock nesting notation > >=20 > > locks held by swapper/0/0: > > #0: (rcu_read_lock){.+.+..}, at: [] __netif_receive_skb+0xe4= /0x8d0 > > #1: (rcu_read_lock){.+.+..}, at: [] ip_local_deliver_finish+= 0x3c/0x4c0 > > #2: (slock-AF_INET){+.-...}, at: [] ip_send_reply+0x107/0x2b0 > > #3: (rcu_read_lock){.+.+..}, at: [] ip_finish_output+0x106/0= x710 > > #4: (rcu_read_lock_bh){.+....}, at: [] dev_queue_xmit+0x0/0x= bd0 > >=20 > > tack backtrace: > > id: 0, comm: swapper/0 Not tainted 3.5.0-rc2-net-lockdep-u64-sync-007-+= #1 > > all Trace: > > [] __lock_acquire+0xd52/0x17d0 > > [] ? trace_hardirqs_off+0xb/0x10 > > [] lock_acquire+0x88/0x120 > > [] ? l2tp_xmit_skb+0x13f/0x8e0 [l2tp_core] > > [] _raw_spin_lock+0x3b/0x70 > > [] ? l2tp_xmit_skb+0x13f/0x8e0 [l2tp_core] > > [] l2tp_xmit_skb+0x13f/0x8e0 [l2tp_core] > > [] l2tp_eth_dev_xmit+0x2d/0x40 [l2tp_eth] > > [] dev_hard_start_xmit+0x49f/0x7e0 > > [] ? dev_hard_start_xmit+0x51/0x7e0 > > [] sch_direct_xmit+0xa9/0x250 > > [] ? _raw_spin_lock+0x61/0x70 > > [] dev_queue_xmit+0x1cf/0xbd0 > > [] ? dev_hard_start_xmit+0x7e0/0x7e0 > > [] ip_finish_output+0x1e7/0x710 > > [] ? ip_finish_output+0x106/0x710 > > [] ? ip_output+0x60/0x120 > > [] ? trace_hardirqs_on+0xb/0x10 > > [] ip_output+0x7b/0x120 > > [] ? __ip_make_skb+0x229/0x360 > > [] ip_local_out+0x25/0x80 > > [] ip_send_skb+0x17/0x70 > > [] ip_push_pending_frames+0x2b/0x40 > > [] ip_send_reply+0x1c5/0x2b0 > > [] ? sched_clock_cpu+0xcf/0x150 > > [] tcp_v4_send_ack+0x1a3/0x260 > > [] ? tcp_timewait_state_process+0x90/0x3c0 > > [] tcp_v4_rcv+0x3ff/0xc20 > > [] ip_local_deliver_finish+0xef/0x4c0 > > [] ? ip_local_deliver_finish+0x3c/0x4c0 > > [] ip_local_deliver+0x3f/0x80 > > [] ip_rcv_finish+0x174/0x640 > > [] ip_rcv+0x221/0x320 > > [] __netif_receive_skb+0x7d1/0x8d0 > > [] ? __netif_receive_skb+0xe4/0x8d0 > > [] process_backlog+0xa7/0x170 > > [] net_rx_action+0x11d/0x210 > > [] ? local_bh_enable_ip+0xd0/0xd0 > > [] __do_softirq+0x97/0x1f0 > > [] ? local_bh_enable_ip+0xd0/0xd0 > > [] ? irq_exit+0x7e/0xa0 > > [] ? do_IRQ+0x4b/0xc0 > > [] ? common_interrupt+0x35/0x3c > > [] ? native_safe_halt+0x5/0x10 > > [] ? default_idle+0x4f/0x1e0 > > [] ? amd_e400_idle+0x51/0x100 > > [] ? cpu_idle+0xb9/0xe0 > > [] ? rest_init+0x112/0x124 > > [] ? __read_lock_failed+0x14/0x14 > > [] ? start_kernel+0x376/0x37c > > [] ? repair_env_string+0x51/0x51 > > [] ? i386_start_kernel+0x9b/0xa2 > >=20 >=20 > Yes, but this is not the splat I fixed. >=20 > You reported two different splats, didnt you ? >=20 > I fixed a core network issue, I am sure guys who wrote l2tp can fix the > l2tp one ;) Ha, yes, fair enough ;-) FWIW, I'm no longer seeing splat #1 from my previous report, but I do still see splat #2. Since they both ended up tripping over in l2tp_xmit_skb() I had assumed they were symptoms of the same root cause -- my mistake. Tom --=20 Tom Parkin Katalix Systems Ltd http://www.katalix.com Catalysts for your Embedded Linux software development --96YOpH+ONegL0A3E Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJP7HYqAAoJEJSMBmUKuovQa/UIAIEast9Uvdq/C3mXbOHhpfbz os75wYQvsnHckApEVMMAs9bc8Xn8jgivDartpe79IcUg4haVw2tQOnLR2boATGKd XYBRM3CBX3u4sSA7U0DhM4c2f9bW6RJbGcV1ZOk+yleaJ6uLfMh0APMLCyAvtUpB Nl/CzHM4ErrkOJb4rh+kEYzMHm/Xc18IJTcoWpGIMJb2W17StzCwjwuv1XlCs4+A 0/F7q4gbI+I2PcFlQDQ/wLFT92h4JuV3xHi98IlSl+fs6pyN3OLhfa9PBU1Z9hMs ocPJLqTmX8bEtk4BU8YIpiFqSRhMoEsSKBzkZ/WoRsjh0YwPieXJC2wuGfyYikU= =rgl9 -----END PGP SIGNATURE----- --96YOpH+ONegL0A3E--