netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
       [not found] <bug-9543-10286@http.bugzilla.kernel.org/>
@ 2007-12-11 11:46 ` Andrew Morton
  2007-12-11 15:04   ` Krzysztof Oledzki
  2007-12-12 13:31   ` Herbert Xu
  0 siblings, 2 replies; 24+ messages in thread
From: Andrew Morton @ 2007-12-11 11:46 UTC (permalink / raw)
  To: olel; +Cc: bugme-daemon, Stephen Hemminger, David S. Miller, netdev

On Tue, 11 Dec 2007 03:20:48 -0800 (PST) bugme-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=9543
> 
>            Summary: RTNL: assertion failed at net/ipv6/addrconf.c
>                     (2164)/RTNL: assertion failed at net/ipv4/devinet.c
>                     (1055)
>            Product: Drivers
>            Version: 2.5
>      KernelVersion: 2.6.24-rc4-git7
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: Network
>         AssignedTo: jgarzik@pobox.com
>         ReportedBy: olel@ans.pl
> 
> 
> Most recent kernel where this bug did not occur: 2.6.23
> Distribution: Gentoo
> 
> Problem Description:
> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
> RTNL: assertion failed at net/ipv6/addrconf.c (2164)
> Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1
>  [<78402cfb>] addrconf_notify+0x5b4/0x7b7
>  [<7812203a>] finish_task_switch+0x0/0x8c
>  [<781346ff>] worker_thread+0x0/0x85
>  [<78438e23>] schedule+0x545/0x55f
>  [<781408d1>] print_lock_contention_bug+0x11/0xd2
>  [<783bfa72>] rt_run_flush+0x43/0x8b
>  [<783bfa93>] rt_run_flush+0x64/0x8b
>  [<7813ac54>] notifier_call_chain+0x2a/0x52
>  [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a
>  [<783a3471>] netdev_state_change+0x18/0x29
>  [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e
>  [<783ac6f4>] linkwatch_event+0x1d/0x22
>  [<78133cdf>] run_workqueue+0xdb/0x1b6
>  [<78133c8b>] run_workqueue+0x87/0x1b6
>  [<783ac6d7>] linkwatch_event+0x0/0x22
>  [<781346ff>] worker_thread+0x0/0x85
>  [<78134778>] worker_thread+0x79/0x85
>  [<781371ad>] autoremove_wake_function+0x0/0x35
>  [<781370f6>] kthread+0x38/0x5e
>  [<781370be>] kthread+0x0/0x5e
>  [<78104baf>] kernel_thread_helper+0x7/0x10
>  =======================
> RTNL: assertion failed at net/ipv6/addrconf.c (1610)


Hopefully this is due to the bug you reported in bug #9542.

Does this patch fix both issues?


From: Andrew Morton <akpm@linux-foundation.org>

Remove stray rtnl_unlock().

Addresses http://bugzilla.kernel.org/show_bug.cgi?id=9542

Cc: "David S. Miller" <davem@davemloft.net>
Cc: Stephen Hemminger <shemminger@linux-foundation.org>
Cc: Krzysztof Oledzki <olel@ans.pl>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 drivers/net/bonding/bond_sysfs.c |    2 --
 1 file changed, 2 deletions(-)

diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix drivers/net/bonding/bond_sysfs.c
--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
+++ a/drivers/net/bonding/bond_sysfs.c
@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
 out:
 	write_unlock_bh(&bond->lock);
 
-	rtnl_unlock();
-
 	return count;
 }
 static DEVICE_ATTR(primary, S_IRUGO | S_IWUSR, bonding_show_primary, bonding_store_primary);
_


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-11 11:46 ` [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055) Andrew Morton
@ 2007-12-11 15:04   ` Krzysztof Oledzki
  2007-12-11 20:30     ` Andrew Morton
  2007-12-12 13:31   ` Herbert Xu
  1 sibling, 1 reply; 24+ messages in thread
From: Krzysztof Oledzki @ 2007-12-11 15:04 UTC (permalink / raw)
  To: Andrew Morton; +Cc: bugme-daemon, Stephen Hemminger, David S. Miller, netdev

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2265 bytes --]



On Tue, 11 Dec 2007, Andrew Morton wrote:

> On Tue, 11 Dec 2007 03:20:48 -0800 (PST) bugme-daemon@bugzilla.kernel.org wrote:
>
>> http://bugzilla.kernel.org/show_bug.cgi?id=9543
>>
>>            Summary: RTNL: assertion failed at net/ipv6/addrconf.c
>>                     (2164)/RTNL: assertion failed at net/ipv4/devinet.c
>>                     (1055)
>>            Product: Drivers
>>            Version: 2.5
>>      KernelVersion: 2.6.24-rc4-git7
>>           Platform: All
>>         OS/Version: Linux
>>               Tree: Mainline
>>             Status: NEW
>>           Severity: normal
>>           Priority: P1
>>          Component: Network
>>         AssignedTo: jgarzik@pobox.com
>>         ReportedBy: olel@ans.pl
>>
>>
>> Most recent kernel where this bug did not occur: 2.6.23
>> Distribution: Gentoo
>>
>> Problem Description:
>> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
>> RTNL: assertion failed at net/ipv6/addrconf.c (2164)
>> Pid: 9, comm: events/0 Not tainted 2.6.24-rc4-git7 #1
>>  [<78402cfb>] addrconf_notify+0x5b4/0x7b7
>>  [<7812203a>] finish_task_switch+0x0/0x8c
>>  [<781346ff>] worker_thread+0x0/0x85
>>  [<78438e23>] schedule+0x545/0x55f
>>  [<781408d1>] print_lock_contention_bug+0x11/0xd2
>>  [<783bfa72>] rt_run_flush+0x43/0x8b
>>  [<783bfa93>] rt_run_flush+0x64/0x8b
>>  [<7813ac54>] notifier_call_chain+0x2a/0x52
>>  [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a
>>  [<783a3471>] netdev_state_change+0x18/0x29
>>  [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e
>>  [<783ac6f4>] linkwatch_event+0x1d/0x22
>>  [<78133cdf>] run_workqueue+0xdb/0x1b6
>>  [<78133c8b>] run_workqueue+0x87/0x1b6
>>  [<783ac6d7>] linkwatch_event+0x0/0x22
>>  [<781346ff>] worker_thread+0x0/0x85
>>  [<78134778>] worker_thread+0x79/0x85
>>  [<781371ad>] autoremove_wake_function+0x0/0x35
>>  [<781370f6>] kthread+0x38/0x5e
>>  [<781370be>] kthread+0x0/0x5e
>>  [<78104baf>] kernel_thread_helper+0x7/0x10
>>  =======================
>> RTNL: assertion failed at net/ipv6/addrconf.c (1610)
>
>
> Hopefully this is due to the bug you reported in bug #9542.
>
> Does this patch fix both issues?

Unfortunately not. I just updated bugzilla.

Best regards,

 				Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-11 15:04   ` Krzysztof Oledzki
@ 2007-12-11 20:30     ` Andrew Morton
  0 siblings, 0 replies; 24+ messages in thread
From: Andrew Morton @ 2007-12-11 20:30 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: bugme-daemon, Stephen Hemminger, David S. Miller, netdev,
	Rafael J. Wysocki

On Tue, 11 Dec 2007 16:04:27 +0100 (CET) Krzysztof Oledzki <olel@ans.pl> wrote:

> >>  [<783bfa72>] rt_run_flush+0x43/0x8b
> >>  [<783bfa93>] rt_run_flush+0x64/0x8b
> >>  [<7813ac54>] notifier_call_chain+0x2a/0x52
> >>  [<7813ac9e>] raw_notifier_call_chain+0x17/0x1a
> >>  [<783a3471>] netdev_state_change+0x18/0x29
> >>  [<783ac6a9>] __linkwatch_run_queue+0x150/0x17e
> >>  [<783ac6f4>] linkwatch_event+0x1d/0x22
> >>  [<78133cdf>] run_workqueue+0xdb/0x1b6
> >>  [<78133c8b>] run_workqueue+0x87/0x1b6
> >>  [<783ac6d7>] linkwatch_event+0x0/0x22
> >>  [<781346ff>] worker_thread+0x0/0x85
> >>  [<78134778>] worker_thread+0x79/0x85
> >>  [<781371ad>] autoremove_wake_function+0x0/0x35
> >>  [<781370f6>] kthread+0x38/0x5e
> >>  [<781370be>] kthread+0x0/0x5e
> >>  [<78104baf>] kernel_thread_helper+0x7/0x10
> >>  =======================
> >> RTNL: assertion failed at net/ipv6/addrconf.c (1610)
> >
> >
> > Hopefully this is due to the bug you reported in bug #9542.
> >
> > Does this patch fix both issues?
> 
> Unfortunately not. I just updated bugzilla.

argh.  Please conduct these discussions via emailed reply-to-all, not via
the bugzilla web interface.

I think my patch _did_ fix http://bugzilla.kernel.org/show_bug.cgi?id=9542
But then you hit a new bonding locking bug.

Rafael, please track 9542 and 9543 as post-2.6.23 regressions, thanks.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-11 11:46 ` [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055) Andrew Morton
  2007-12-11 15:04   ` Krzysztof Oledzki
@ 2007-12-12 13:31   ` Herbert Xu
  2007-12-12 17:46     ` Jay Vosburgh
  1 sibling, 1 reply; 24+ messages in thread
From: Herbert Xu @ 2007-12-12 13:31 UTC (permalink / raw)
  To: Andrew Morton; +Cc: olel, bugme-daemon, shemminger, davem, netdev, fubar

Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> From: Andrew Morton <akpm@linux-foundation.org>
> 
> Remove stray rtnl_unlock().
> 
> Addresses http://bugzilla.kernel.org/show_bug.cgi?id=9542

Adnrew, please cc Jay Vosburgh <fubar@us.ibm.com> on bonding
issues.

> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix drivers/net/bonding/bond_sysfs.c
> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> +++ a/drivers/net/bonding/bond_sysfs.c
> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> out:
>        write_unlock_bh(&bond->lock);
> 
> -       rtnl_unlock();
> -

Looking at the changeset that added this perhaps the intention
is to hold the lock? If so we should add an rtnl_lock to the start
of the function.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-12 13:31   ` Herbert Xu
@ 2007-12-12 17:46     ` Jay Vosburgh
  2007-12-12 19:07       ` Andy Gospodarek
  2007-12-14 16:14       ` Krzysztof Oledzki
  0 siblings, 2 replies; 24+ messages in thread
From: Jay Vosburgh @ 2007-12-12 17:46 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Andrew Morton, olel, bugme-daemon, shemminger, davem, netdev,
	andy

Herbert Xu <herbert@gondor.apana.org.au> wrote:

>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix drivers/net/bonding/bond_sysfs.c
>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>> +++ a/drivers/net/bonding/bond_sysfs.c
>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>> out:
>>        write_unlock_bh(&bond->lock);
>> 
>> -       rtnl_unlock();
>> -
>
>Looking at the changeset that added this perhaps the intention
>is to hold the lock? If so we should add an rtnl_lock to the start
>of the function.

	Yes, this function needs to hold locks, and more than just
what's there now.  I believe the following should be correct; I haven't
tested it, though (I'm supposedly on vacation right now).

	The following change should be correct for the
bonding_store_primary case discussed in this thread, and also corrects
the bonding_store_active case which performs similar functions.

	The bond_change_active_slave and bond_select_active_slave
functions both require rtnl, bond->lock for read and curr_slave_lock for
write_bh, and no other locks.  This is so that the lower level
mode-specific functions can release locks down to just rtnl in order to
call, e.g., dev_set_mac_address with the locks it expects (rtnl only).

Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>

diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
index 11b76b3..28a2d80 100644
--- a/drivers/net/bonding/bond_sysfs.c
+++ b/drivers/net/bonding/bond_sysfs.c
@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d,
 	struct slave *slave;
 	struct bonding *bond = to_bond(d);
 
-	write_lock_bh(&bond->lock);
+	rtnl_lock();
+	read_lock(&bond->lock);
+	write_lock_bh(&bond->curr_slave_lock);
+
 	if (!USES_PRIMARY(bond->params.mode)) {
 		printk(KERN_INFO DRV_NAME
 		       ": %s: Unable to set primary slave; %s is in mode %d\n",
@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d,
 		}
 	}
 out:
-	write_unlock_bh(&bond->lock);
-
+	write_unlock_bh(&bond->curr_slave_lock);
+	read_unlock(&bond->lock);
 	rtnl_unlock();
 
 	return count;
@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device *d,
 	struct bonding *bond = to_bond(d);
 
 	rtnl_lock();
-	write_lock_bh(&bond->lock);
+	read_lock(&bond->lock);
+	write_lock_bh(&bond->curr_slave_lock);
 
 	if (!USES_PRIMARY(bond->params.mode)) {
 		printk(KERN_INFO DRV_NAME
@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device *d,
 		}
 	}
 out:
-	write_unlock_bh(&bond->lock);
+	write_unlock_bh(&bond->curr_slave_lock);
+	read_unlock(&bond->lock);
 	rtnl_unlock();
 
 	return count;


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-12 17:46     ` Jay Vosburgh
@ 2007-12-12 19:07       ` Andy Gospodarek
  2007-12-14 16:14       ` Krzysztof Oledzki
  1 sibling, 0 replies; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-12 19:07 UTC (permalink / raw)
  To: Jay Vosburgh
  Cc: Herbert Xu, Andrew Morton, olel, bugme-daemon, shemminger, davem,
	netdev, andy

On Wed, Dec 12, 2007 at 09:46:55AM -0800, Jay Vosburgh wrote:
> Herbert Xu <herbert@gondor.apana.org.au> wrote:
> 
> >> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix drivers/net/bonding/bond_sysfs.c
> >> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >> +++ a/drivers/net/bonding/bond_sysfs.c
> >> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >> out:
> >>        write_unlock_bh(&bond->lock);
> >> 
> >> -       rtnl_unlock();
> >> -
> >
> >Looking at the changeset that added this perhaps the intention
> >is to hold the lock? If so we should add an rtnl_lock to the start
> >of the function.
> 
> 	Yes, this function needs to hold locks, and more than just
> what's there now.  I believe the following should be correct; I haven't
> tested it, though (I'm supposedly on vacation right now).
> 
> 	The following change should be correct for the
> bonding_store_primary case discussed in this thread, and also corrects
> the bonding_store_active case which performs similar functions.
> 
> 	The bond_change_active_slave and bond_select_active_slave
> functions both require rtnl, bond->lock for read and curr_slave_lock for
> write_bh, and no other locks.  This is so that the lower level
> mode-specific functions can release locks down to just rtnl in order to
> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
> 
> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>

This looks good to me as well....

Signed-off-by: Andy Gospodarek <andy@greyhouse.net>


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-12 17:46     ` Jay Vosburgh
  2007-12-12 19:07       ` Andy Gospodarek
@ 2007-12-14 16:14       ` Krzysztof Oledzki
  2007-12-14 18:26         ` Andy Gospodarek
  1 sibling, 1 reply; 24+ messages in thread
From: Krzysztof Oledzki @ 2007-12-14 16:14 UTC (permalink / raw)
  To: Jay Vosburgh
  Cc: Herbert Xu, Andrew Morton, bugme-daemon, shemminger, davem,
	netdev, andy

[-- Attachment #1: Type: TEXT/PLAIN, Size: 58246 bytes --]



On Wed, 12 Dec 2007, Jay Vosburgh wrote:

> Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix drivers/net/bonding/bond_sysfs.c
>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>> +++ a/drivers/net/bonding/bond_sysfs.c
>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>>> out:
>>>        write_unlock_bh(&bond->lock);
>>>
>>> -       rtnl_unlock();
>>> -
>>
>> Looking at the changeset that added this perhaps the intention
>> is to hold the lock? If so we should add an rtnl_lock to the start
>> of the function.
>
> 	Yes, this function needs to hold locks, and more than just
> what's there now.  I believe the following should be correct; I haven't
> tested it, though (I'm supposedly on vacation right now).
>
> 	The following change should be correct for the
> bonding_store_primary case discussed in this thread, and also corrects
> the bonding_store_active case which performs similar functions.
>
> 	The bond_change_active_slave and bond_select_active_slave
> functions both require rtnl, bond->lock for read and curr_slave_lock for
> write_bh, and no other locks.  This is so that the lower level
> mode-specific functions can release locks down to just rtnl in order to
> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
>
> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>
> diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
> index 11b76b3..28a2d80 100644
> --- a/drivers/net/bonding/bond_sysfs.c
> +++ b/drivers/net/bonding/bond_sysfs.c
> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d,
> 	struct slave *slave;
> 	struct bonding *bond = to_bond(d);
>
> -	write_lock_bh(&bond->lock);
> +	rtnl_lock();
> +	read_lock(&bond->lock);
> +	write_lock_bh(&bond->curr_slave_lock);
> +
> 	if (!USES_PRIMARY(bond->params.mode)) {
> 		printk(KERN_INFO DRV_NAME
> 		       ": %s: Unable to set primary slave; %s is in mode %d\n",
> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d,
> 		}
> 	}
> out:
> -	write_unlock_bh(&bond->lock);
> -
> +	write_unlock_bh(&bond->curr_slave_lock);
> +	read_unlock(&bond->lock);
> 	rtnl_unlock();
>
> 	return count;
> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device *d,
> 	struct bonding *bond = to_bond(d);
>
> 	rtnl_lock();
> -	write_lock_bh(&bond->lock);
> +	read_lock(&bond->lock);
> +	write_lock_bh(&bond->curr_slave_lock);
>
> 	if (!USES_PRIMARY(bond->params.mode)) {
> 		printk(KERN_INFO DRV_NAME
> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device *d,
> 		}
> 	}
> out:
> -	write_unlock_bh(&bond->lock);
> +	write_unlock_bh(&bond->curr_slave_lock);
> +	read_unlock(&bond->lock);
> 	rtnl_unlock();
>
> 	return count;

Vanilla 2.6.24-rc5 plus this patch:

=========================================================
[ INFO: possible irq lock inversion dependency detected ]
2.6.24-rc5 #1
---------------------------------------------------------
events/0/9 just changed the state of lock:
  (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
but this lock took another, soft-read-irq-unsafe lock in the past:
  (&bond->lock){-.--}

and interrupts could create inverse lock ordering between them.


other info that might help us debug this:
4 locks held by events/0/9:
  #0:  (events){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6
  #1:  ((linkwatch_work).work){--..}, at: [<c0133c57>] 
run_workqueue+0x87/0x1b6
  #2:  (rtnl_mutex){--..}, at: [<c03abd50>] linkwatch_event+0x5/0x22
  #3:  (&ndev->lock){-.-+}, at: [<c0411b61>] 
mld_ifc_timer_expire+0x17/0x1fb

the first lock's dependencies:
-> (&mc->mca_lock){-+..} ops: 10 {
    initial-use  at:
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c014289c>] __lock_acquire+0x4ba/0xc07
                         [<c0109ef2>] save_stack_trace+0x20/0x3a
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0439d62>] _spin_lock_bh+0x3b/0x64
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                         [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                         [<c0401834>] ipv6_add_dev+0x21c/0x24b
                         [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                         [<c05c5b40>] addrconf_init+0x13/0x193
                         [<c0199f63>] proc_net_fops_create+0x10/0x21
                         [<c0419b38>] ip6_flowlabel_init+0x1e/0x20
                         [<c05c5a20>] inet6_init+0x1f0/0x2ad
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    in-softirq-W at:
                         [<c0142822>] __lock_acquire+0x440/0xc07
                         [<c013e0f3>] clockevents_program_event+0xe0/0xee
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c0439d62>] _spin_lock_bh+0x3b/0x64
                         [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                         [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c012df52>] run_timer_softirq+0xfa/0x15d
                         [<c012a8a6>] __do_softirq+0x56/0xdb
                         [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                         [<c012a8b8>] __do_softirq+0x68/0xdb
                         [<c012a961>] do_softirq+0x36/0x51
                         [<c012ae4a>] local_bh_enable_ip+0xad/0xed
                         [<c03bf107>] rt_run_flush+0x64/0x8b
                         [<c03e9296>] fib_netdev_event+0x61/0x65
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a2ae5>] netdev_state_change+0x18/0x29
                         [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e
                         [<c03abd68>] linkwatch_event+0x1d/0x22
                         [<c0133cab>] run_workqueue+0xdb/0x1b6
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c03abd4b>] linkwatch_event+0x0/0x22
                         [<c01346cb>] worker_thread+0x0/0x85
                         [<c0134744>] worker_thread+0x79/0x85
                         [<c0137179>] autoremove_wake_function+0x0/0x35
                         [<c01370c2>] kthread+0x38/0x5e
                         [<c013708a>] kthread+0x0/0x5e
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-W at:
                         [<c01417ee>] find_usage_backwards+0xbb/0xe2
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c014286a>] __lock_acquire+0x488/0xc07
                         [<c0109ef2>] save_stack_trace+0x20/0x3a
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0439d62>] _spin_lock_bh+0x3b/0x64
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                         [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                         [<c0401834>] ipv6_add_dev+0x21c/0x24b
                         [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                         [<c05c5b40>] addrconf_init+0x13/0x193
                         [<c0199f63>] proc_net_fops_create+0x10/0x21
                         [<c0419b38>] ip6_flowlabel_init+0x1e/0x20
                         [<c05c5a20>] inet6_init+0x1f0/0x2ad
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
  }
  ... key      at: [<c087e2d8>] __key.30798+0x0/0x8
  -> (_xmit_ETHER){-...} ops: 8 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0412114>] igmp6_group_added+0x56/0x11d
    [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
    [<c0410100>] igmp6_mc_seq_start+0xde/0x138
    [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
    [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
    [<c0401834>] ipv6_add_dev+0x21c/0x24b
    [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
    [<c0401e17>] addrconf_notify+0x60/0x7b7
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c0439066>] mutex_lock_nested+0x286/0x2ac
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f88>] register_netdevice_notifier+0x49/0x126
    [<c05c5bda>] addrconf_init+0xad/0x193
    [<c05c5b48>] addrconf_init+0x1b/0x193
    [<c05c5a20>] inet6_init+0x1f0/0x2ad
    [<c05a9499>] kernel_init+0x150/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c0104baf>] kernel_thread_helper+0x7/0x10
    [<ffffffff>] 0xffffffff

  -> (&bonding_netdev_xmit_lock_key){-...} ops: 6 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c0877804>] bonding_netdev_xmit_lock_key+0x0/0x8
   -> (&bond->lock){-.--} ops: 99 {
      initial-use  at:
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c014289c>] __lock_acquire+0x4ba/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c0439eee>] _read_lock_bh+0x3b/0x64
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                             [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                             [<c03aaa26>] rtnetlink_event+0x3f/0x42
                             [<c013ac20>] notifier_call_chain+0x2a/0x52
                             [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                             [<c03a31de>] register_netdevice+0x2a7/0x2e7
                             [<c02ed862>] bond_create+0x1f2/0x26a
                             [<c05bedcd>] bonding_init+0x761/0x7ea
                             [<c05be635>] e1000_init_module+0x45/0x7c
                             [<c05a9499>] kernel_init+0x150/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
      hardirq-on-W at:
                             [<c014286a>] __lock_acquire+0x488/0xc07
                             [<c012093c>] try_to_wake_up+0x2ce/0x2d8
                             [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02eda75>] 
bond_set_multicast_list+0x1d/0x241
                             [<c0439e25>] _write_lock_bh+0x3b/0x64
                             [<c02eda75>] 
bond_set_multicast_list+0x1d/0x241
                             [<c02eda75>] 
bond_set_multicast_list+0x1d/0x241
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d
                             [<c03a1675>] dev_set_rx_mode+0x23/0x36
                             [<c03a3d50>] dev_open+0x5e/0x77
                             [<c03a2a1f>] dev_change_flags+0x9d/0x14b
                             [<c03a1823>] __dev_get_by_name+0x68/0x73
                             [<c03e3850>] devinet_ioctl+0x22b/0x536
                             [<c03a3b45>] dev_ioctl+0x46f/0x5b7
                             [<c0399c78>] sock_ioctl+0x167/0x18b
                             [<c0399b11>] sock_ioctl+0x0/0x18b
                             [<c01725f7>] do_ioctl+0x1f/0x62
                             [<c0172867>] vfs_ioctl+0x22d/0x23f
                             [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                             [<c01728ac>] sys_ioctl+0x33/0x4b
                             [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                             [<ffffffff>] 0xffffffff
      softirq-on-R at:
                             [<c0141986>] mark_lock+0x64/0x451
                             [<c013575e>] __kernel_text_address+0x5/0xe
                             [<c0104ee2>] dump_trace+0x83/0x8d
                             [<c0142889>] __lock_acquire+0x4a7/0xc07
                             [<c013fc76>] save_trace+0x37/0x89
                             [<c0133c57>] run_workqueue+0x87/0x1b6
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c0439f4d>] _read_lock+0x36/0x5f
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c0133cab>] run_workqueue+0xdb/0x1b6
                             [<c0133c57>] run_workqueue+0x87/0x1b6
                             [<c02eee44>] bond_mii_monitor+0x0/0x85
                             [<c01346cb>] worker_thread+0x0/0x85
                             [<c0134744>] worker_thread+0x79/0x85
                             [<c0137179>] autoremove_wake_function+0x0/0x35
                             [<c01370c2>] kthread+0x38/0x5e
                             [<c013708a>] kthread+0x0/0x5e
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
      hardirq-on-R at:
                             [<c013fe0a>] get_lock_stats+0xd/0x2e
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c0142844>] __lock_acquire+0x462/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c0439eee>] _read_lock_bh+0x3b/0x64
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                             [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                             [<c03aaa26>] rtnetlink_event+0x3f/0x42
                             [<c013ac20>] notifier_call_chain+0x2a/0x52
                             [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                             [<c03a31de>] register_netdevice+0x2a7/0x2e7
                             [<c02ed862>] bond_create+0x1f2/0x26a
                             [<c05bedcd>] bonding_init+0x761/0x7ea
                             [<c05be635>] e1000_init_module+0x45/0x7c
                             [<c05a9499>] kernel_init+0x150/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
    }
    ... key      at: [<c08777d0>] __key.32969+0x0/0x8
    -> (_xmit_ETHER){-...} ops: 8 {
       initial-use  at:
                               [<c014289c>] __lock_acquire+0x4ba/0xc07
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0439d62>] _spin_lock_bh+0x3b/0x64
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0412114>] igmp6_group_added+0x56/0x11d
                               [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                               [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                               [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                               [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                               [<c0401834>] ipv6_add_dev+0x21c/0x24b
                               [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                               [<c0401e17>] addrconf_notify+0x60/0x7b7
                               [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                               [<c0141dac>] mark_held_locks+0x39/0x53
                               [<c0439066>] mutex_lock_nested+0x286/0x2ac
                               [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                               [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                               [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                               [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                               [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                               [<c05c5bda>] addrconf_init+0xad/0x193
                               [<c05c5b48>] addrconf_init+0x1b/0x193
                               [<c05c5a20>] inet6_init+0x1f0/0x2ad
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
       hardirq-on-W at:
                               [<c0141986>] mark_lock+0x64/0x451
                               [<c014286a>] __lock_acquire+0x488/0xc07
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0439d62>] _spin_lock_bh+0x3b/0x64
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0412114>] igmp6_group_added+0x56/0x11d
                               [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                               [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                               [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                               [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                               [<c0401834>] ipv6_add_dev+0x21c/0x24b
                               [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                               [<c0401e17>] addrconf_notify+0x60/0x7b7
                               [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                               [<c0141dac>] mark_held_locks+0x39/0x53
                               [<c0439066>] mutex_lock_nested+0x286/0x2ac
                               [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                               [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                               [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                               [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                               [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                               [<c05c5bda>] addrconf_init+0xad/0x193
                               [<c05c5b48>] addrconf_init+0x1b/0x193
                               [<c05c5a20>] inet6_init+0x1f0/0x2ad
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
     }
     ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
    ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
    [<c0109ef2>] save_stack_trace+0x20/0x3a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf
    [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65
    [<c02ee9af>] bond_select_active_slave+0x95/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

    -> (lweventlist_lock){.+..} ops: 10 {
       initial-use  at:
                               [<c0141986>] mark_lock+0x64/0x451
                               [<c014289c>] __lock_acquire+0x4ba/0xc07
                               [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3
                               [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3
                               [<c024a7de>] delay_tsc+0x25/0x3b
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                               [<c02e1c43>] e1000_probe+0xad1/0xbe8
                               [<c0257f3f>] pci_device_probe+0x36/0x57
                               [<c02d0e5f>] driver_probe_device+0xe1/0x15f
                               [<c043a2f9>] _spin_unlock+0x25/0x3b
                               [<c04375b2>] klist_next+0x58/0x6d
                               [<c02d0f6f>] __driver_attach+0x0/0x7f
                               [<c02d0fb8>] __driver_attach+0x49/0x7f
                               [<c02d0403>] bus_for_each_dev+0x36/0x58
                               [<c02d0cb7>] driver_attach+0x16/0x18
                               [<c02d0f6f>] __driver_attach+0x0/0x7f
                               [<c02d06fa>] bus_add_driver+0x6d/0x18d
                               [<c0258089>] __pci_register_driver+0x53/0x7f
                               [<c05be635>] e1000_init_module+0x45/0x7c
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
       in-softirq-W at:
                               [<c011d20a>] __wake_up_common+0x32/0x5c
                               [<c0142822>] __lock_acquire+0x440/0xc07
                               [<c043a3c5>] 
_spin_unlock_irqrestore+0x40/0x58
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c02dff01>] e1000_watchdog+0x0/0x5c9
                               [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                               [<c03aeb42>] netif_carrier_on+0x16/0x27
                               [<c02e0156>] e1000_watchdog+0x255/0x5c9
                               [<c02dff01>] e1000_watchdog+0x0/0x5c9
                               [<c012df52>] run_timer_softirq+0xfa/0x15d
                               [<c012a8a6>] __do_softirq+0x56/0xdb
                               [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                               [<c012a8b8>] __do_softirq+0x68/0xdb
                               [<c012a961>] do_softirq+0x36/0x51
                               [<c012ab07>] irq_exit+0x43/0x4e
                               [<c0114122>] 
smp_apic_timer_interrupt+0x74/0x80
                               [<c0104a01>] apic_timer_interrupt+0x29/0x38
                               [<c0104a0b>] apic_timer_interrupt+0x33/0x38
                               [<c01600d8>] sys_swapon+0x29c/0x9aa
                               [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f
                               [<c0102447>] mwait_idle+0x0/0xf
                               [<c0102581>] cpu_idle+0x99/0xc6
                               [<c05a98c7>] start_kernel+0x2c7/0x2cf
                               [<c05a90e0>] unknown_bootoption+0x0/0x195
                               [<ffffffff>] 0xffffffff
     }
     ... key      at: [<c058a194>] lweventlist_lock+0x14/0x40
    ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abbbb>] linkwatch_fire_event+0x25/0x37
    [<c03aeb42>] netif_carrier_on+0x16/0x27
    [<c02ede2c>] bond_set_carrier+0x31/0x55
    [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

   ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c0143062>] lock_acquire+0x79/0x93
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c0439e25>] _write_lock_bh+0x3b/0x64
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c013fe35>] put_lock_stats+0xa/0x1e
    [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d
    [<c03a1675>] dev_set_rx_mode+0x23/0x36
    [<c03a3d50>] dev_open+0x5e/0x77
    [<c03a2a1f>] dev_change_flags+0x9d/0x14b
    [<c03a1823>] __dev_get_by_name+0x68/0x73
    [<c03e3850>] devinet_ioctl+0x22b/0x536
    [<c03a3b45>] dev_ioctl+0x46f/0x5b7
    [<c0399c78>] sock_ioctl+0x167/0x18b
    [<c0399b11>] sock_ioctl+0x0/0x18b
    [<c01725f7>] do_ioctl+0x1f/0x62
    [<c0172867>] vfs_ioctl+0x22d/0x23f
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c01728ac>] sys_ioctl+0x33/0x4b
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0412114>] igmp6_group_added+0x56/0x11d
    [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
    [<c0410100>] igmp6_mc_seq_start+0xde/0x138
    [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
    [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
    [<c0401834>] ipv6_add_dev+0x21c/0x24b
    [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
    [<c0401e17>] addrconf_notify+0x60/0x7b7
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c0439066>] mutex_lock_nested+0x286/0x2ac
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f88>] register_netdevice_notifier+0x49/0x126
    [<c05c5bda>] addrconf_init+0xad/0x193
    [<c05c5b48>] addrconf_init+0x1b/0x193
    [<c05c5a20>] inet6_init+0x1f0/0x2ad
    [<c05a9499>] kernel_init+0x150/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c0104baf>] kernel_thread_helper+0x7/0x10
    [<ffffffff>] 0xffffffff


the second lock's dependencies:
-> (&bond->lock){-.--} ops: 99 {
    initial-use  at:
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c014289c>] __lock_acquire+0x4ba/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c0439eee>] _read_lock_bh+0x3b/0x64
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                         [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                         [<c03aaa26>] rtnetlink_event+0x3f/0x42
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a31de>] register_netdevice+0x2a7/0x2e7
                         [<c02ed862>] bond_create+0x1f2/0x26a
                         [<c05bedcd>] bonding_init+0x761/0x7ea
                         [<c05be635>] e1000_init_module+0x45/0x7c
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-W at:
                         [<c014286a>] __lock_acquire+0x488/0xc07
                         [<c012093c>] try_to_wake_up+0x2ce/0x2d8
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c0439e25>] _write_lock_bh+0x3b/0x64
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d
                         [<c03a1675>] dev_set_rx_mode+0x23/0x36
                         [<c03a3d50>] dev_open+0x5e/0x77
                         [<c03a2a1f>] dev_change_flags+0x9d/0x14b
                         [<c03a1823>] __dev_get_by_name+0x68/0x73
                         [<c03e3850>] devinet_ioctl+0x22b/0x536
                         [<c03a3b45>] dev_ioctl+0x46f/0x5b7
                         [<c0399c78>] sock_ioctl+0x167/0x18b
                         [<c0399b11>] sock_ioctl+0x0/0x18b
                         [<c01725f7>] do_ioctl+0x1f/0x62
                         [<c0172867>] vfs_ioctl+0x22d/0x23f
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c01728ac>] sys_ioctl+0x33/0x4b
                         [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                         [<ffffffff>] 0xffffffff
    softirq-on-R at:
                         [<c0141986>] mark_lock+0x64/0x451
                         [<c013575e>] __kernel_text_address+0x5/0xe
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c0142889>] __lock_acquire+0x4a7/0xc07
                         [<c013fc76>] save_trace+0x37/0x89
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c0439f4d>] _read_lock+0x36/0x5f
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c0133cab>] run_workqueue+0xdb/0x1b6
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c02eee44>] bond_mii_monitor+0x0/0x85
                         [<c01346cb>] worker_thread+0x0/0x85
                         [<c0134744>] worker_thread+0x79/0x85
                         [<c0137179>] autoremove_wake_function+0x0/0x35
                         [<c01370c2>] kthread+0x38/0x5e
                         [<c013708a>] kthread+0x0/0x5e
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-R at:
                         [<c013fe0a>] get_lock_stats+0xd/0x2e
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c0142844>] __lock_acquire+0x462/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c0439eee>] _read_lock_bh+0x3b/0x64
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                         [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                         [<c03aaa26>] rtnetlink_event+0x3f/0x42
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a31de>] register_netdevice+0x2a7/0x2e7
                         [<c02ed862>] bond_create+0x1f2/0x26a
                         [<c05bedcd>] bonding_init+0x761/0x7ea
                         [<c05be635>] e1000_init_module+0x45/0x7c
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
  }
  ... key      at: [<c08777d0>] __key.32969+0x0/0x8
  -> (_xmit_ETHER){-...} ops: 8 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] 
register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] 
register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
    [<c0109ef2>] save_stack_trace+0x20/0x3a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf
    [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65
    [<c02ee9af>] bond_select_active_slave+0x95/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

  -> (lweventlist_lock){.+..} ops: 10 {
     initial-use  at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3
                           [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3
                           [<c024a7de>] delay_tsc+0x25/0x3b
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                           [<c02e1c43>] e1000_probe+0xad1/0xbe8
                           [<c0257f3f>] pci_device_probe+0x36/0x57
                           [<c02d0e5f>] driver_probe_device+0xe1/0x15f
                           [<c043a2f9>] _spin_unlock+0x25/0x3b
                           [<c04375b2>] klist_next+0x58/0x6d
                           [<c02d0f6f>] __driver_attach+0x0/0x7f
                           [<c02d0fb8>] __driver_attach+0x49/0x7f
                           [<c02d0403>] bus_for_each_dev+0x36/0x58
                           [<c02d0cb7>] driver_attach+0x16/0x18
                           [<c02d0f6f>] __driver_attach+0x0/0x7f
                           [<c02d06fa>] bus_add_driver+0x6d/0x18d
                           [<c0258089>] __pci_register_driver+0x53/0x7f
                           [<c05be635>] e1000_init_module+0x45/0x7c
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     in-softirq-W at:
                           [<c011d20a>] __wake_up_common+0x32/0x5c
                           [<c0142822>] __lock_acquire+0x440/0xc07
                           [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c02dff01>] e1000_watchdog+0x0/0x5c9
                           [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                           [<c03aeb42>] netif_carrier_on+0x16/0x27
                           [<c02e0156>] e1000_watchdog+0x255/0x5c9
                           [<c02dff01>] e1000_watchdog+0x0/0x5c9
                           [<c012df52>] run_timer_softirq+0xfa/0x15d
                           [<c012a8a6>] __do_softirq+0x56/0xdb
                           [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                           [<c012a8b8>] __do_softirq+0x68/0xdb
                           [<c012a961>] do_softirq+0x36/0x51
                           [<c012ab07>] irq_exit+0x43/0x4e
                           [<c0114122>] smp_apic_timer_interrupt+0x74/0x80
                           [<c0104a01>] apic_timer_interrupt+0x29/0x38
                           [<c0104a0b>] apic_timer_interrupt+0x33/0x38
                           [<c01600d8>] sys_swapon+0x29c/0x9aa
                           [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f
                           [<c0102447>] mwait_idle+0x0/0xf
                           [<c0102581>] cpu_idle+0x99/0xc6
                           [<c05a98c7>] start_kernel+0x2c7/0x2cf
                           [<c05a90e0>] unknown_bootoption+0x0/0x195
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c058a194>] lweventlist_lock+0x14/0x40
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abbbb>] linkwatch_fire_event+0x25/0x37
    [<c03aeb42>] netif_carrier_on+0x16/0x27
    [<c02ede2c>] bond_set_carrier+0x31/0x55
    [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff


stack backtrace:
Pid: 9, comm: events/0 Not tainted 2.6.24-rc5 #1
  [<c0140b38>] print_irq_inversion_bug+0x108/0x112
  [<c014191d>] check_usage_forwards+0x3c/0x41
  [<c0141b09>] mark_lock+0x1e7/0x451
  [<c0142822>] __lock_acquire+0x440/0xc07
  [<c013e0f3>] clockevents_program_event+0xe0/0xee
  [<c0143062>] lock_acquire+0x79/0x93
  [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
  [<c0439d62>] _spin_lock_bh+0x3b/0x64
  [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
  [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
  [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
  [<c012df52>] run_timer_softirq+0xfa/0x15d
  [<c012a8a6>] __do_softirq+0x56/0xdb
  [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
  [<c012a8b8>] __do_softirq+0x68/0xdb
  [<c012a961>] do_softirq+0x36/0x51
  [<c012ae4a>] local_bh_enable_ip+0xad/0xed
  [<c03bf107>] rt_run_flush+0x64/0x8b
  [<c03e9296>] fib_netdev_event+0x61/0x65
  [<c013ac20>] notifier_call_chain+0x2a/0x52
  [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
  [<c03a2ae5>] netdev_state_change+0x18/0x29
  [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e
  [<c03abd68>] linkwatch_event+0x1d/0x22
  [<c0133cab>] run_workqueue+0xdb/0x1b6
  [<c0133c57>] run_workqueue+0x87/0x1b6
  [<c03abd4b>] linkwatch_event+0x0/0x22
  [<c01346cb>] worker_thread+0x0/0x85
  [<c0134744>] worker_thread+0x79/0x85
  [<c0137179>] autoremove_wake_function+0x0/0x35
  [<c01370c2>] kthread+0x38/0x5e
  [<c013708a>] kthread+0x0/0x5e
  [<c0104baf>] kernel_thread_helper+0x7/0x10
  =======================



Best regards,

 				Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 16:14       ` Krzysztof Oledzki
@ 2007-12-14 18:26         ` Andy Gospodarek
  2007-12-14 18:57           ` Krzysztof Oledzki
  0 siblings, 1 reply; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-14 18:26 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev, andy

On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Wed, 12 Dec 2007, Jay Vosburgh wrote:
> 
> >Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >
> >>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix 
> >>>drivers/net/bonding/bond_sysfs.c
> >>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>+++ a/drivers/net/bonding/bond_sysfs.c
> >>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >>>out:
> >>>       write_unlock_bh(&bond->lock);
> >>>
> >>>-       rtnl_unlock();
> >>>-
> >>
> >>Looking at the changeset that added this perhaps the intention
> >>is to hold the lock? If so we should add an rtnl_lock to the start
> >>of the function.
> >
> >	Yes, this function needs to hold locks, and more than just
> >what's there now.  I believe the following should be correct; I haven't
> >tested it, though (I'm supposedly on vacation right now).
> >
> >	The following change should be correct for the
> >bonding_store_primary case discussed in this thread, and also corrects
> >the bonding_store_active case which performs similar functions.
> >
> >	The bond_change_active_slave and bond_select_active_slave
> >functions both require rtnl, bond->lock for read and curr_slave_lock for
> >write_bh, and no other locks.  This is so that the lower level
> >mode-specific functions can release locks down to just rtnl in order to
> >call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
> >
> >Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
> >
> >diff --git a/drivers/net/bonding/bond_sysfs.c 
> >b/drivers/net/bonding/bond_sysfs.c
> >index 11b76b3..28a2d80 100644
> >--- a/drivers/net/bonding/bond_sysfs.c
> >+++ b/drivers/net/bonding/bond_sysfs.c
> >@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device 
> >*d,
> >	struct slave *slave;
> >	struct bonding *bond = to_bond(d);
> >
> >-	write_lock_bh(&bond->lock);
> >+	rtnl_lock();
> >+	read_lock(&bond->lock);
> >+	write_lock_bh(&bond->curr_slave_lock);
> >+
> >	if (!USES_PRIMARY(bond->params.mode)) {
> >		printk(KERN_INFO DRV_NAME
> >		       ": %s: Unable to set primary slave; %s is in mode 
> >		       %d\n",
> >@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device 
> >*d,
> >		}
> >	}
> >out:
> >-	write_unlock_bh(&bond->lock);
> >-
> >+	write_unlock_bh(&bond->curr_slave_lock);
> >+	read_unlock(&bond->lock);
> >	rtnl_unlock();
> >
> >	return count;
> >@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct 
> >device *d,
> >	struct bonding *bond = to_bond(d);
> >
> >	rtnl_lock();
> >-	write_lock_bh(&bond->lock);
> >+	read_lock(&bond->lock);
> >+	write_lock_bh(&bond->curr_slave_lock);
> >
> >	if (!USES_PRIMARY(bond->params.mode)) {
> >		printk(KERN_INFO DRV_NAME
> >@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct 
> >device *d,
> >		}
> >	}
> >out:
> >-	write_unlock_bh(&bond->lock);
> >+	write_unlock_bh(&bond->curr_slave_lock);
> >+	read_unlock(&bond->lock);
> >	rtnl_unlock();
> >
> >	return count;
> 
> Vanilla 2.6.24-rc5 plus this patch:
> 
> =========================================================
> [ INFO: possible irq lock inversion dependency detected ]
> 2.6.24-rc5 #1
> ---------------------------------------------------------
> events/0/9 just changed the state of lock:
>  (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
> but this lock took another, soft-read-irq-unsafe lock in the past:
>  (&bond->lock){-.--}
> 
> and interrupts could create inverse lock ordering between them.
> 
> 

Grrr, I should have seen that -- sorry.  Try your luck with this instead:

diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
index 11b76b3..0694254 100644
--- a/drivers/net/bonding/bond_sysfs.c
+++ b/drivers/net/bonding/bond_sysfs.c
@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d,
 	struct slave *slave;
 	struct bonding *bond = to_bond(d);
 
-	write_lock_bh(&bond->lock);
+	rtnl_lock();
+	read_lock_bh(&bond->lock);
+	write_lock_bh(&bond->curr_slave_lock);
+
 	if (!USES_PRIMARY(bond->params.mode)) {
 		printk(KERN_INFO DRV_NAME
 		       ": %s: Unable to set primary slave; %s is in mode %d\n",
@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d,
 		}
 	}
 out:
-	write_unlock_bh(&bond->lock);
-
+	write_unlock_bh(&bond->curr_slave_lock);
+	read_unlock_bh(&bond->lock);
 	rtnl_unlock();
 
 	return count;
@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct device *d,
 	struct bonding *bond = to_bond(d);
 
 	rtnl_lock();
-	write_lock_bh(&bond->lock);
+	read_lock_bh(&bond->lock);
+	write_lock_bh(&bond->curr_slave_lock);
 
 	if (!USES_PRIMARY(bond->params.mode)) {
 		printk(KERN_INFO DRV_NAME
@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct device *d,
 		}
 	}
 out:
-	write_unlock_bh(&bond->lock);
+	write_unlock_bh(&bond->curr_slave_lock);
+	read_unlock_bh(&bond->lock);
 	rtnl_unlock();
 
 	return count;

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 18:26         ` Andy Gospodarek
@ 2007-12-14 18:57           ` Krzysztof Oledzki
  2007-12-14 22:03             ` Andy Gospodarek
  2007-12-14 22:47             ` Andy Gospodarek
  0 siblings, 2 replies; 24+ messages in thread
From: Krzysztof Oledzki @ 2007-12-14 18:57 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev

[-- Attachment #1: Type: TEXT/PLAIN, Size: 59892 bytes --]



On Fri, 14 Dec 2007, Andy Gospodarek wrote:

> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
>>
>>
>> On Wed, 12 Dec 2007, Jay Vosburgh wrote:
>>
>>> Herbert Xu <herbert@gondor.apana.org.au> wrote:
>>>
>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>> drivers/net/bonding/bond_sysfs.c
>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>> +++ a/drivers/net/bonding/bond_sysfs.c
>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>>>>> out:
>>>>>       write_unlock_bh(&bond->lock);
>>>>>
>>>>> -       rtnl_unlock();
>>>>> -
>>>>
>>>> Looking at the changeset that added this perhaps the intention
>>>> is to hold the lock? If so we should add an rtnl_lock to the start
>>>> of the function.
>>>
>>> 	Yes, this function needs to hold locks, and more than just
>>> what's there now.  I believe the following should be correct; I haven't
>>> tested it, though (I'm supposedly on vacation right now).
>>>
>>> 	The following change should be correct for the
>>> bonding_store_primary case discussed in this thread, and also corrects
>>> the bonding_store_active case which performs similar functions.
>>>
>>> 	The bond_change_active_slave and bond_select_active_slave
>>> functions both require rtnl, bond->lock for read and curr_slave_lock for
>>> write_bh, and no other locks.  This is so that the lower level
>>> mode-specific functions can release locks down to just rtnl in order to
>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
>>>
>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>>>
>>> diff --git a/drivers/net/bonding/bond_sysfs.c
>>> b/drivers/net/bonding/bond_sysfs.c
>>> index 11b76b3..28a2d80 100644
>>> --- a/drivers/net/bonding/bond_sysfs.c
>>> +++ b/drivers/net/bonding/bond_sysfs.c
>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
>>> *d,
>>> 	struct slave *slave;
>>> 	struct bonding *bond = to_bond(d);
>>>
>>> -	write_lock_bh(&bond->lock);
>>> +	rtnl_lock();
>>> +	read_lock(&bond->lock);
>>> +	write_lock_bh(&bond->curr_slave_lock);
>>> +
>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>> 		printk(KERN_INFO DRV_NAME
>>> 		       ": %s: Unable to set primary slave; %s is in mode
>>> 		       %d\n",
>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
>>> *d,
>>> 		}
>>> 	}
>>> out:
>>> -	write_unlock_bh(&bond->lock);
>>> -
>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>> +	read_unlock(&bond->lock);
>>> 	rtnl_unlock();
>>>
>>> 	return count;
>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
>>> device *d,
>>> 	struct bonding *bond = to_bond(d);
>>>
>>> 	rtnl_lock();
>>> -	write_lock_bh(&bond->lock);
>>> +	read_lock(&bond->lock);
>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>
>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>> 		printk(KERN_INFO DRV_NAME
>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
>>> device *d,
>>> 		}
>>> 	}
>>> out:
>>> -	write_unlock_bh(&bond->lock);
>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>> +	read_unlock(&bond->lock);
>>> 	rtnl_unlock();
>>>
>>> 	return count;
>>
>> Vanilla 2.6.24-rc5 plus this patch:
>>
>> =========================================================
>> [ INFO: possible irq lock inversion dependency detected ]
>> 2.6.24-rc5 #1
>> ---------------------------------------------------------
>> events/0/9 just changed the state of lock:
>>  (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
>> but this lock took another, soft-read-irq-unsafe lock in the past:
>>  (&bond->lock){-.--}
>>
>> and interrupts could create inverse lock ordering between them.
>>
>>
>
> Grrr, I should have seen that -- sorry.  Try your luck with this instead:
<CUT>

No luck.

bonding: bond0: setting mode to active-backup (1).
bonding: bond0: Setting MII monitoring interval to 100.
ADDRCONF(NETDEV_UP): bond0: link is not ready
bonding: bond0: Adding slave eth0.
e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow 
Control: RX/TX
bonding: bond0: making interface eth0 the new active one.
bonding: bond0: first active interface up!
bonding: bond0: enslaving eth0 as an active interface with an up link.
bonding: bond0: Adding slave eth1.
ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready

=========================================================
[ INFO: possible irq lock inversion dependency detected ]
2.6.24-rc5 #1
---------------------------------------------------------
events/0/9 just changed the state of lock:
  (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
but this lock took another, soft-read-irq-unsafe lock in the past:
  (&bond->lock){-.--}

and interrupts could create inverse lock ordering between them.


other info that might help us debug this:
4 locks held by events/0/9:
  #0:  (events){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6
  #1:  ((linkwatch_work).work){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6
  #2:  (rtnl_mutex){--..}, at: [<c03abd50>] linkwatch_event+0x5/0x22
  #3:  (&ndev->lock){-.-+}, at: [<c0411b61>] mld_ifc_timer_expire+0x17/0x1fb

the first lock's dependencies:
-> (&mc->mca_lock){-+..} ops: 10 {
    initial-use  at:
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c014289c>] __lock_acquire+0x4ba/0xc07
                         [<c0109ef2>] save_stack_trace+0x20/0x3a
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0439d62>] _spin_lock_bh+0x3b/0x64
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                         [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                         [<c0401834>] ipv6_add_dev+0x21c/0x24b
                         [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                         [<c05c5b40>] addrconf_init+0x13/0x193
                         [<c0199f63>] proc_net_fops_create+0x10/0x21
                         [<c0419b38>] ip6_flowlabel_init+0x1e/0x20
                         [<c05c5a20>] inet6_init+0x1f0/0x2ad
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    in-softirq-W at:
                         [<c0142822>] __lock_acquire+0x440/0xc07
                         [<c013fe0a>] get_lock_stats+0xd/0x2e
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c0439d62>] _spin_lock_bh+0x3b/0x64
                         [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                         [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c012df52>] run_timer_softirq+0xfa/0x15d
                         [<c012a8a6>] __do_softirq+0x56/0xdb
                         [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                         [<c012a8b8>] __do_softirq+0x68/0xdb
                         [<c012a961>] do_softirq+0x36/0x51
                         [<c012ae4a>] local_bh_enable_ip+0xad/0xed
                         [<c03bf107>] rt_run_flush+0x64/0x8b
                         [<c03e9296>] fib_netdev_event+0x61/0x65
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a2ae5>] netdev_state_change+0x18/0x29
                         [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e
                         [<c03abd68>] linkwatch_event+0x1d/0x22
                         [<c0133cab>] run_workqueue+0xdb/0x1b6
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c03abd4b>] linkwatch_event+0x0/0x22
                         [<c01346cb>] worker_thread+0x0/0x85
                         [<c0134744>] worker_thread+0x79/0x85
                         [<c0137179>] autoremove_wake_function+0x0/0x35
                         [<c01370c2>] kthread+0x38/0x5e
                         [<c013708a>] kthread+0x0/0x5e
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-W at:
                         [<c01417ee>] find_usage_backwards+0xbb/0xe2
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c014286a>] __lock_acquire+0x488/0xc07
                         [<c0109ef2>] save_stack_trace+0x20/0x3a
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0439d62>] _spin_lock_bh+0x3b/0x64
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c04120d6>] igmp6_group_added+0x18/0x11d
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                         [<c0412452>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                         [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                         [<c0401834>] ipv6_add_dev+0x21c/0x24b
                         [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                         [<c05c5b40>] addrconf_init+0x13/0x193
                         [<c0199f63>] proc_net_fops_create+0x10/0x21
                         [<c0419b38>] ip6_flowlabel_init+0x1e/0x20
                         [<c05c5a20>] inet6_init+0x1f0/0x2ad
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
  }
  ... key      at: [<c087e2d8>] __key.30798+0x0/0x8
  -> (_xmit_ETHER){-...} ops: 8 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0412114>] igmp6_group_added+0x56/0x11d
    [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
    [<c0410100>] igmp6_mc_seq_start+0xde/0x138
    [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
    [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
    [<c0401834>] ipv6_add_dev+0x21c/0x24b
    [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
    [<c0401e17>] addrconf_notify+0x60/0x7b7
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c0439066>] mutex_lock_nested+0x286/0x2ac
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f88>] register_netdevice_notifier+0x49/0x126
    [<c05c5bda>] addrconf_init+0xad/0x193
    [<c05c5b48>] addrconf_init+0x1b/0x193
    [<c05c5a20>] inet6_init+0x1f0/0x2ad
    [<c05a9499>] kernel_init+0x150/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c0104baf>] kernel_thread_helper+0x7/0x10
    [<ffffffff>] 0xffffffff

  -> (&bonding_netdev_xmit_lock_key){-...} ops: 6 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c0877804>] bonding_netdev_xmit_lock_key+0x0/0x8
   -> (&bond->lock){-.--} ops: 98 {
      initial-use  at:
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c014289c>] __lock_acquire+0x4ba/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c0439eee>] _read_lock_bh+0x3b/0x64
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                             [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                             [<c03aaa26>] rtnetlink_event+0x3f/0x42
                             [<c013ac20>] notifier_call_chain+0x2a/0x52
                             [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                             [<c03a31de>] register_netdevice+0x2a7/0x2e7
                             [<c02ed862>] bond_create+0x1f2/0x26a
                             [<c05bedcd>] bonding_init+0x761/0x7ea
                             [<c05be635>] e1000_init_module+0x45/0x7c
                             [<c05a9499>] kernel_init+0x150/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
      hardirq-on-W at:
                             [<c014286a>] __lock_acquire+0x488/0xc07
                             [<c012093c>] try_to_wake_up+0x2ce/0x2d8
                             [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                             [<c0439e25>] _write_lock_bh+0x3b/0x64
                             [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                             [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d
                             [<c03a1675>] dev_set_rx_mode+0x23/0x36
                             [<c03a3d50>] dev_open+0x5e/0x77
                             [<c03a2a1f>] dev_change_flags+0x9d/0x14b
                             [<c03a1823>] __dev_get_by_name+0x68/0x73
                             [<c03e3850>] devinet_ioctl+0x22b/0x536
                             [<c03a3b45>] dev_ioctl+0x46f/0x5b7
                             [<c0399c78>] sock_ioctl+0x167/0x18b
                             [<c0399b11>] sock_ioctl+0x0/0x18b
                             [<c01725f7>] do_ioctl+0x1f/0x62
                             [<c0172867>] vfs_ioctl+0x22d/0x23f
                             [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                             [<c01728ac>] sys_ioctl+0x33/0x4b
                             [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                             [<ffffffff>] 0xffffffff
      softirq-on-R at:
                             [<c0141986>] mark_lock+0x64/0x451
                             [<c013575e>] __kernel_text_address+0x5/0xe
                             [<c0104ee2>] dump_trace+0x83/0x8d
                             [<c0142889>] __lock_acquire+0x4a7/0xc07
                             [<c013fc76>] save_trace+0x37/0x89
                             [<c0133c57>] run_workqueue+0x87/0x1b6
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c0439f4d>] _read_lock+0x36/0x5f
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c0133cab>] run_workqueue+0xdb/0x1b6
                             [<c0133c57>] run_workqueue+0x87/0x1b6
                             [<c02eee44>] bond_mii_monitor+0x0/0x85
                             [<c01346cb>] worker_thread+0x0/0x85
                             [<c0134744>] worker_thread+0x79/0x85
                             [<c0137179>] autoremove_wake_function+0x0/0x35
                             [<c01370c2>] kthread+0x38/0x5e
                             [<c013708a>] kthread+0x0/0x5e
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
      hardirq-on-R at:
                             [<c013fe0a>] get_lock_stats+0xd/0x2e
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c0142844>] __lock_acquire+0x462/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c0439eee>] _read_lock_bh+0x3b/0x64
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                             [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                             [<c03aaa26>] rtnetlink_event+0x3f/0x42
                             [<c013ac20>] notifier_call_chain+0x2a/0x52
                             [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                             [<c03a31de>] register_netdevice+0x2a7/0x2e7
                             [<c02ed862>] bond_create+0x1f2/0x26a
                             [<c05bedcd>] bonding_init+0x761/0x7ea
                             [<c05be635>] e1000_init_module+0x45/0x7c
                             [<c05a9499>] kernel_init+0x150/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
    }
    ... key      at: [<c08777d0>] __key.32969+0x0/0x8
    -> (_xmit_ETHER){-...} ops: 8 {
       initial-use  at:
                               [<c014289c>] __lock_acquire+0x4ba/0xc07
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0439d62>] _spin_lock_bh+0x3b/0x64
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0412114>] igmp6_group_added+0x56/0x11d
                               [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                               [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                               [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                               [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                               [<c0401834>] ipv6_add_dev+0x21c/0x24b
                               [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                               [<c0401e17>] addrconf_notify+0x60/0x7b7
                               [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                               [<c0141dac>] mark_held_locks+0x39/0x53
                               [<c0439066>] mutex_lock_nested+0x286/0x2ac
                               [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                               [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                               [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                               [<c05c5bda>] addrconf_init+0xad/0x193
                               [<c05c5b48>] addrconf_init+0x1b/0x193
                               [<c05c5a20>] inet6_init+0x1f0/0x2ad
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
       hardirq-on-W at:
                               [<c0141986>] mark_lock+0x64/0x451
                               [<c014286a>] __lock_acquire+0x488/0xc07
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0439d62>] _spin_lock_bh+0x3b/0x64
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                               [<c0412114>] igmp6_group_added+0x56/0x11d
                               [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                               [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                               [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                               [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                               [<c0401834>] ipv6_add_dev+0x21c/0x24b
                               [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                               [<c0401e17>] addrconf_notify+0x60/0x7b7
                               [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                               [<c0141dac>] mark_held_locks+0x39/0x53
                               [<c0439066>] mutex_lock_nested+0x286/0x2ac
                               [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                               [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                               [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                               [<c05c5bda>] addrconf_init+0xad/0x193
                               [<c05c5b48>] addrconf_init+0x1b/0x193
                               [<c05c5a20>] inet6_init+0x1f0/0x2ad
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
     }
     ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
    ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
    [<c0109ef2>] save_stack_trace+0x20/0x3a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf
    [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65
    [<c02ee9af>] bond_select_active_slave+0x95/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

    -> (lweventlist_lock){.+..} ops: 10 {
       initial-use  at:
                               [<c0141986>] mark_lock+0x64/0x451
                               [<c014289c>] __lock_acquire+0x4ba/0xc07
                               [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3
                               [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3
                               [<c024a7de>] delay_tsc+0x25/0x3b
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                               [<c02e1c43>] e1000_probe+0xad1/0xbe8
                               [<c0257f3f>] pci_device_probe+0x36/0x57
                               [<c02d0e5f>] driver_probe_device+0xe1/0x15f
                               [<c043a2f9>] _spin_unlock+0x25/0x3b
                               [<c04375b2>] klist_next+0x58/0x6d
                               [<c02d0f6f>] __driver_attach+0x0/0x7f
                               [<c02d0fb8>] __driver_attach+0x49/0x7f
                               [<c02d0403>] bus_for_each_dev+0x36/0x58
                               [<c02d0cb7>] driver_attach+0x16/0x18
                               [<c02d0f6f>] __driver_attach+0x0/0x7f
                               [<c02d06fa>] bus_add_driver+0x6d/0x18d
                               [<c0258089>] __pci_register_driver+0x53/0x7f
                               [<c05be635>] e1000_init_module+0x45/0x7c
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
       in-softirq-W at:
                               [<c011d20a>] __wake_up_common+0x32/0x5c
                               [<c0142822>] __lock_acquire+0x440/0xc07
                               [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c02dff01>] e1000_watchdog+0x0/0x5c9
                               [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                               [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                               [<c03aeb42>] netif_carrier_on+0x16/0x27
                               [<c02e0156>] e1000_watchdog+0x255/0x5c9
                               [<c02dff01>] e1000_watchdog+0x0/0x5c9
                               [<c012df52>] run_timer_softirq+0xfa/0x15d
                               [<c012a8a6>] __do_softirq+0x56/0xdb
                               [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                               [<c012a8b8>] __do_softirq+0x68/0xdb
                               [<c012a961>] do_softirq+0x36/0x51
                               [<c012ab07>] irq_exit+0x43/0x4e
                               [<c0114122>] smp_apic_timer_interrupt+0x74/0x80
                               [<c0104a01>] apic_timer_interrupt+0x29/0x38
                               [<c0104a0b>] apic_timer_interrupt+0x33/0x38
                               [<c01600d8>] sys_swapon+0x29c/0x9aa
                               [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f
                               [<c0102447>] mwait_idle+0x0/0xf
                               [<c0102581>] cpu_idle+0x99/0xc6
                               [<c05a98c7>] start_kernel+0x2c7/0x2cf
                               [<c05a90e0>] unknown_bootoption+0x0/0x195
                               [<ffffffff>] 0xffffffff
     }
     ... key      at: [<c058a194>] lweventlist_lock+0x14/0x40
    ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abbbb>] linkwatch_fire_event+0x25/0x37
    [<c03aeb42>] netif_carrier_on+0x16/0x27
    [<c02ede2c>] bond_set_carrier+0x31/0x55
    [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

   ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c0143062>] lock_acquire+0x79/0x93
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c0439e25>] _write_lock_bh+0x3b/0x64
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c013fe35>] put_lock_stats+0xa/0x1e
    [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d
    [<c03a1675>] dev_set_rx_mode+0x23/0x36
    [<c03a3d50>] dev_open+0x5e/0x77
    [<c03a2a1f>] dev_change_flags+0x9d/0x14b
    [<c03a1823>] __dev_get_by_name+0x68/0x73
    [<c03e3850>] devinet_ioctl+0x22b/0x536
    [<c03a3b45>] dev_ioctl+0x46f/0x5b7
    [<c0399c78>] sock_ioctl+0x167/0x18b
    [<c0399b11>] sock_ioctl+0x0/0x18b
    [<c01725f7>] do_ioctl+0x1f/0x62
    [<c0172867>] vfs_ioctl+0x22d/0x23f
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c01728ac>] sys_ioctl+0x33/0x4b
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0412114>] igmp6_group_added+0x56/0x11d
    [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
    [<c0410100>] igmp6_mc_seq_start+0xde/0x138
    [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
    [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
    [<c0401834>] ipv6_add_dev+0x21c/0x24b
    [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
    [<c0401e17>] addrconf_notify+0x60/0x7b7
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c0439066>] mutex_lock_nested+0x286/0x2ac
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
    [<c03a3f88>] register_netdevice_notifier+0x49/0x126
    [<c05c5bda>] addrconf_init+0xad/0x193
    [<c05c5b48>] addrconf_init+0x1b/0x193
    [<c05c5a20>] inet6_init+0x1f0/0x2ad
    [<c05a9499>] kernel_init+0x150/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c0104baf>] kernel_thread_helper+0x7/0x10
    [<ffffffff>] 0xffffffff


the second lock's dependencies:
-> (&bond->lock){-.--} ops: 98 {
    initial-use  at:
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c014289c>] __lock_acquire+0x4ba/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c0439eee>] _read_lock_bh+0x3b/0x64
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                         [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                         [<c03aaa26>] rtnetlink_event+0x3f/0x42
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a31de>] register_netdevice+0x2a7/0x2e7
                         [<c02ed862>] bond_create+0x1f2/0x26a
                         [<c05bedcd>] bonding_init+0x761/0x7ea
                         [<c05be635>] e1000_init_module+0x45/0x7c
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-W at:
                         [<c014286a>] __lock_acquire+0x488/0xc07
                         [<c012093c>] try_to_wake_up+0x2ce/0x2d8
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c0439e25>] _write_lock_bh+0x3b/0x64
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c03a14db>] __dev_set_rx_mode+0x7b/0x7d
                         [<c03a1675>] dev_set_rx_mode+0x23/0x36
                         [<c03a3d50>] dev_open+0x5e/0x77
                         [<c03a2a1f>] dev_change_flags+0x9d/0x14b
                         [<c03a1823>] __dev_get_by_name+0x68/0x73
                         [<c03e3850>] devinet_ioctl+0x22b/0x536
                         [<c03a3b45>] dev_ioctl+0x46f/0x5b7
                         [<c0399c78>] sock_ioctl+0x167/0x18b
                         [<c0399b11>] sock_ioctl+0x0/0x18b
                         [<c01725f7>] do_ioctl+0x1f/0x62
                         [<c0172867>] vfs_ioctl+0x22d/0x23f
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c01728ac>] sys_ioctl+0x33/0x4b
                         [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                         [<ffffffff>] 0xffffffff
    softirq-on-R at:
                         [<c0141986>] mark_lock+0x64/0x451
                         [<c013575e>] __kernel_text_address+0x5/0xe
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c0142889>] __lock_acquire+0x4a7/0xc07
                         [<c013fc76>] save_trace+0x37/0x89
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c0439f4d>] _read_lock+0x36/0x5f
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c0133cab>] run_workqueue+0xdb/0x1b6
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c02eee44>] bond_mii_monitor+0x0/0x85
                         [<c01346cb>] worker_thread+0x0/0x85
                         [<c0134744>] worker_thread+0x79/0x85
                         [<c0137179>] autoremove_wake_function+0x0/0x35
                         [<c01370c2>] kthread+0x38/0x5e
                         [<c013708a>] kthread+0x0/0x5e
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-R at:
                         [<c013fe0a>] get_lock_stats+0xd/0x2e
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c0142844>] __lock_acquire+0x462/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c0439eee>] _read_lock_bh+0x3b/0x64
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c03aa3e7>] rtnl_fill_ifinfo+0x2bf/0x563
                         [<c03aa965>] rtmsg_ifinfo+0x5d/0xdf
                         [<c03aaa26>] rtnetlink_event+0x3f/0x42
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a31de>] register_netdevice+0x2a7/0x2e7
                         [<c02ed862>] bond_create+0x1f2/0x26a
                         [<c05bedcd>] bonding_init+0x761/0x7ea
                         [<c05be635>] e1000_init_module+0x45/0x7c
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
  }
  ... key      at: [<c08777d0>] __key.32969+0x0/0x8
  -> (_xmit_ETHER){-...} ops: 8 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0439d62>] _spin_lock_bh+0x3b/0x64
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c03a5aa1>] dev_mc_add+0x1a/0x6a
                           [<c0412114>] igmp6_group_added+0x56/0x11d
                           [<c04124a8>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0xde/0x138
                           [<c04124dd>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c0412205>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c0401834>] ipv6_add_dev+0x21c/0x24b
                           [<c040b07d>] 
ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401e17>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c0439066>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f4d>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f88>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
    [<c0109ef2>] save_stack_trace+0x20/0x3a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c0439d62>] _spin_lock_bh+0x3b/0x64
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c03a5aa1>] dev_mc_add+0x1a/0x6a
    [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf
    [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65
    [<c02ee9af>] bond_select_active_slave+0x95/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

  -> (lweventlist_lock){.+..} ops: 10 {
     initial-use  at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3
                           [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3
                           [<c024a7de>] delay_tsc+0x25/0x3b
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                           [<c02e1c43>] e1000_probe+0xad1/0xbe8
                           [<c0257f3f>] pci_device_probe+0x36/0x57
                           [<c02d0e5f>] driver_probe_device+0xe1/0x15f
                           [<c043a2f9>] _spin_unlock+0x25/0x3b
                           [<c04375b2>] klist_next+0x58/0x6d
                           [<c02d0f6f>] __driver_attach+0x0/0x7f
                           [<c02d0fb8>] __driver_attach+0x49/0x7f
                           [<c02d0403>] bus_for_each_dev+0x36/0x58
                           [<c02d0cb7>] driver_attach+0x16/0x18
                           [<c02d0f6f>] __driver_attach+0x0/0x7f
                           [<c02d06fa>] bus_add_driver+0x6d/0x18d
                           [<c0258089>] __pci_register_driver+0x53/0x7f
                           [<c05be635>] e1000_init_module+0x45/0x7c
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     in-softirq-W at:
                           [<c011d20a>] __wake_up_common+0x32/0x5c
                           [<c0142822>] __lock_acquire+0x440/0xc07
                           [<c043a3c5>] _spin_unlock_irqrestore+0x40/0x58
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c02dff01>] e1000_watchdog+0x0/0x5c9
                           [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abaf7>] linkwatch_add_event+0xd/0x2c
                           [<c03abbbb>] linkwatch_fire_event+0x25/0x37
                           [<c03aeb42>] netif_carrier_on+0x16/0x27
                           [<c02e0156>] e1000_watchdog+0x255/0x5c9
                           [<c02dff01>] e1000_watchdog+0x0/0x5c9
                           [<c012df52>] run_timer_softirq+0xfa/0x15d
                           [<c012a8a6>] __do_softirq+0x56/0xdb
                           [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                           [<c012a8b8>] __do_softirq+0x68/0xdb
                           [<c012a961>] do_softirq+0x36/0x51
                           [<c012ab07>] irq_exit+0x43/0x4e
                           [<c0114122>] smp_apic_timer_interrupt+0x74/0x80
                           [<c0104a01>] apic_timer_interrupt+0x29/0x38
                           [<c0104a0b>] apic_timer_interrupt+0x33/0x38
                           [<c01600d8>] sys_swapon+0x29c/0x9aa
                           [<c01021a6>] mwait_idle_with_hints+0x3b/0x3f
                           [<c0102447>] mwait_idle+0x0/0xf
                           [<c0102581>] cpu_idle+0x99/0xc6
                           [<c05a98c7>] start_kernel+0x2c7/0x2cf
                           [<c05a90e0>] unknown_bootoption+0x0/0x195
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c058a194>] lweventlist_lock+0x14/0x40
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c043a07f>] _spin_lock_irqsave+0x3f/0x6c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abaf7>] linkwatch_add_event+0xd/0x2c
    [<c03abbbb>] linkwatch_fire_event+0x25/0x37
    [<c03aeb42>] netif_carrier_on+0x16/0x27
    [<c02ede2c>] bond_set_carrier+0x31/0x55
    [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c0439084>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f6197>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fe9>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff


stack backtrace:
Pid: 9, comm: events/0 Not tainted 2.6.24-rc5 #1
  [<c0140b38>] print_irq_inversion_bug+0x108/0x112
  [<c014191d>] check_usage_forwards+0x3c/0x41
  [<c0141b09>] mark_lock+0x1e7/0x451
  [<c0142822>] __lock_acquire+0x440/0xc07
  [<c013fe0a>] get_lock_stats+0xd/0x2e
  [<c013fe35>] put_lock_stats+0xa/0x1e
  [<c0143062>] lock_acquire+0x79/0x93
  [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
  [<c0439d62>] _spin_lock_bh+0x3b/0x64
  [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
  [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
  [<c0411b4a>] mld_ifc_timer_expire+0x0/0x1fb
  [<c012df52>] run_timer_softirq+0xfa/0x15d
  [<c012a8a6>] __do_softirq+0x56/0xdb
  [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
  [<c012a8b8>] __do_softirq+0x68/0xdb
  [<c012a961>] do_softirq+0x36/0x51
  [<c012ae4a>] local_bh_enable_ip+0xad/0xed
  [<c03bf107>] rt_run_flush+0x64/0x8b
  [<c03e9296>] fib_netdev_event+0x61/0x65
  [<c013ac20>] notifier_call_chain+0x2a/0x52
  [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
  [<c03a2ae5>] netdev_state_change+0x18/0x29
  [<c03abd1d>] __linkwatch_run_queue+0x150/0x17e
  [<c03abd68>] linkwatch_event+0x1d/0x22
  [<c0133cab>] run_workqueue+0xdb/0x1b6
  [<c0133c57>] run_workqueue+0x87/0x1b6
  [<c03abd4b>] linkwatch_event+0x0/0x22
  [<c01346cb>] worker_thread+0x0/0x85
  [<c0134744>] worker_thread+0x79/0x85
  [<c0137179>] autoremove_wake_function+0x0/0x35
  [<c01370c2>] kthread+0x38/0x5e
  [<c013708a>] kthread+0x0/0x5e
  [<c0104baf>] kernel_thread_helper+0x7/0x10
  =======================
bonding: bond0: enslaving eth1 as a backup interface with a down link.
bonding: bond0: Setting eth0 as primary slave.
bond0: no IPv6 routers present

Best regards,

 				Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 18:57           ` Krzysztof Oledzki
@ 2007-12-14 22:03             ` Andy Gospodarek
  2007-12-14 22:11               ` Krzysztof Oledzki
  2007-12-18 19:52               ` Krzysztof Oledzki
  2007-12-14 22:47             ` Andy Gospodarek
  1 sibling, 2 replies; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-14 22:03 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: Andy Gospodarek, Jay Vosburgh, Herbert Xu, Andrew Morton,
	bugme-daemon, shemminger, davem, netdev

On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> 
> >On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
> >>
> >>
> >>On Wed, 12 Dec 2007, Jay Vosburgh wrote:
> >>
> >>>Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >>>
> >>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>drivers/net/bonding/bond_sysfs.c
> >>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>+++ a/drivers/net/bonding/bond_sysfs.c
> >>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >>>>>out:
> >>>>>      write_unlock_bh(&bond->lock);
> >>>>>
> >>>>>-       rtnl_unlock();
> >>>>>-
> >>>>
> >>>>Looking at the changeset that added this perhaps the intention
> >>>>is to hold the lock? If so we should add an rtnl_lock to the start
> >>>>of the function.
> >>>
> >>>	Yes, this function needs to hold locks, and more than just
> >>>what's there now.  I believe the following should be correct; I haven't
> >>>tested it, though (I'm supposedly on vacation right now).
> >>>
> >>>	The following change should be correct for the
> >>>bonding_store_primary case discussed in this thread, and also corrects
> >>>the bonding_store_active case which performs similar functions.
> >>>
> >>>	The bond_change_active_slave and bond_select_active_slave
> >>>functions both require rtnl, bond->lock for read and curr_slave_lock for
> >>>write_bh, and no other locks.  This is so that the lower level
> >>>mode-specific functions can release locks down to just rtnl in order to
> >>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
> >>>
> >>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
> >>>
> >>>diff --git a/drivers/net/bonding/bond_sysfs.c
> >>>b/drivers/net/bonding/bond_sysfs.c
> >>>index 11b76b3..28a2d80 100644
> >>>--- a/drivers/net/bonding/bond_sysfs.c
> >>>+++ b/drivers/net/bonding/bond_sysfs.c
> >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
> >>>*d,
> >>>	struct slave *slave;
> >>>	struct bonding *bond = to_bond(d);
> >>>
> >>>-	write_lock_bh(&bond->lock);
> >>>+	rtnl_lock();
> >>>+	read_lock(&bond->lock);
> >>>+	write_lock_bh(&bond->curr_slave_lock);
F
> >>>+
> >>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>		printk(KERN_INFO DRV_NAME
> >>>		       ": %s: Unable to set primary slave; %s is in mode
> >>>		       %d\n",
> >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
> >>>*d,
> >>>		}
> >>>	}
> >>>out:
> >>>-	write_unlock_bh(&bond->lock);
> >>>-
> >>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>+	read_unlock(&bond->lock);
> >>>	rtnl_unlock();
> >>>
> >>>	return count;
> >>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>device *d,
> >>>	struct bonding *bond = to_bond(d);
> >>>
> >>>	rtnl_lock();
> >>>-	write_lock_bh(&bond->lock);
> >>>+	read_lock(&bond->lock);
> >>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>
> >>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>		printk(KERN_INFO DRV_NAME
> >>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>device *d,
> >>>		}
> >>>	}
> >>>out:
> >>>-	write_unlock_bh(&bond->lock);
> >>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>+	read_unlock(&bond->lock);
> >>>	rtnl_unlock();
> >>>
> >>>	return count;
> >>
> >>Vanilla 2.6.24-rc5 plus this patch:
> >>
> >>=========================================================
> >>[ INFO: possible irq lock inversion dependency detected ]
> >>2.6.24-rc5 #1
> >>---------------------------------------------------------
> >>events/0/9 just changed the state of lock:
> >> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
> >>but this lock took another, soft-read-irq-unsafe lock in the past:
> >> (&bond->lock){-.--}
> >>
> >>and interrupts could create inverse lock ordering between them.
> >>
> >>
> >
> >Grrr, I should have seen that -- sorry.  Try your luck with this instead:
> <CUT>
> 
> No luck.
> 
> bonding: bond0: setting mode to active-backup (1).
> bonding: bond0: Setting MII monitoring interval to 100.
> ADDRCONF(NETDEV_UP): bond0: link is not ready
> bonding: bond0: Adding slave eth0.
> e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow 
> Control: RX/TX
> bonding: bond0: making interface eth0 the new active one.
> bonding: bond0: first active interface up!
> bonding: bond0: enslaving eth0 as an active interface with an up link.
> bonding: bond0: Adding slave eth1.
> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready

<SNIP>

> bonding: bond0: enslaving eth1 as a backup interface with a down link.
> bonding: bond0: Setting eth0 as primary slave.
> bond0: no IPv6 routers present
> 
 
Based on the console log, I'm guessing your initialization scripts use
sysfs to set eth0 as the primary interface for bond0?  Can you confirm?

If you did somehow use sysfs to set the primary device as eth0, I'm
guessing you never see this issue without that line or without this
patch.  Please confirm this as well.

Thanks,

-andy


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 22:03             ` Andy Gospodarek
@ 2007-12-14 22:11               ` Krzysztof Oledzki
  2007-12-14 22:27                 ` Andy Gospodarek
  2007-12-18 19:52               ` Krzysztof Oledzki
  1 sibling, 1 reply; 24+ messages in thread
From: Krzysztof Oledzki @ 2007-12-14 22:11 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev

[-- Attachment #1: Type: TEXT/PLAIN, Size: 5915 bytes --]



On Fri, 14 Dec 2007, Andy Gospodarek wrote:

> On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
>>
>>
>> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>>
>>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
>>>>
>>>>
>>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote:
>>>>
>>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote:
>>>>>
>>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>> drivers/net/bonding/bond_sysfs.c
>>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c
>>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>>>>>>> out:
>>>>>>>      write_unlock_bh(&bond->lock);
>>>>>>>
>>>>>>> -       rtnl_unlock();
>>>>>>> -
>>>>>>
>>>>>> Looking at the changeset that added this perhaps the intention
>>>>>> is to hold the lock? If so we should add an rtnl_lock to the start
>>>>>> of the function.
>>>>>
>>>>> 	Yes, this function needs to hold locks, and more than just
>>>>> what's there now.  I believe the following should be correct; I haven't
>>>>> tested it, though (I'm supposedly on vacation right now).
>>>>>
>>>>> 	The following change should be correct for the
>>>>> bonding_store_primary case discussed in this thread, and also corrects
>>>>> the bonding_store_active case which performs similar functions.
>>>>>
>>>>> 	The bond_change_active_slave and bond_select_active_slave
>>>>> functions both require rtnl, bond->lock for read and curr_slave_lock for
>>>>> write_bh, and no other locks.  This is so that the lower level
>>>>> mode-specific functions can release locks down to just rtnl in order to
>>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
>>>>>
>>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>>>>>
>>>>> diff --git a/drivers/net/bonding/bond_sysfs.c
>>>>> b/drivers/net/bonding/bond_sysfs.c
>>>>> index 11b76b3..28a2d80 100644
>>>>> --- a/drivers/net/bonding/bond_sysfs.c
>>>>> +++ b/drivers/net/bonding/bond_sysfs.c
>>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
>>>>> *d,
>>>>> 	struct slave *slave;
>>>>> 	struct bonding *bond = to_bond(d);
>>>>>
>>>>> -	write_lock_bh(&bond->lock);
>>>>> +	rtnl_lock();
>>>>> +	read_lock(&bond->lock);
>>>>> +	write_lock_bh(&bond->curr_slave_lock);
> F
>>>>> +
>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>> 		printk(KERN_INFO DRV_NAME
>>>>> 		       ": %s: Unable to set primary slave; %s is in mode
>>>>> 		       %d\n",
>>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
>>>>> *d,
>>>>> 		}
>>>>> 	}
>>>>> out:
>>>>> -	write_unlock_bh(&bond->lock);
>>>>> -
>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>> +	read_unlock(&bond->lock);
>>>>> 	rtnl_unlock();
>>>>>
>>>>> 	return count;
>>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>> device *d,
>>>>> 	struct bonding *bond = to_bond(d);
>>>>>
>>>>> 	rtnl_lock();
>>>>> -	write_lock_bh(&bond->lock);
>>>>> +	read_lock(&bond->lock);
>>>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>>>
>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>> 		printk(KERN_INFO DRV_NAME
>>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>> device *d,
>>>>> 		}
>>>>> 	}
>>>>> out:
>>>>> -	write_unlock_bh(&bond->lock);
>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>> +	read_unlock(&bond->lock);
>>>>> 	rtnl_unlock();
>>>>>
>>>>> 	return count;
>>>>
>>>> Vanilla 2.6.24-rc5 plus this patch:
>>>>
>>>> =========================================================
>>>> [ INFO: possible irq lock inversion dependency detected ]
>>>> 2.6.24-rc5 #1
>>>> ---------------------------------------------------------
>>>> events/0/9 just changed the state of lock:
>>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
>>>> but this lock took another, soft-read-irq-unsafe lock in the past:
>>>> (&bond->lock){-.--}
>>>>
>>>> and interrupts could create inverse lock ordering between them.
>>>>
>>>>
>>>
>>> Grrr, I should have seen that -- sorry.  Try your luck with this instead:
>> <CUT>
>>
>> No luck.
>>
>> bonding: bond0: setting mode to active-backup (1).
>> bonding: bond0: Setting MII monitoring interval to 100.
>> ADDRCONF(NETDEV_UP): bond0: link is not ready
>> bonding: bond0: Adding slave eth0.
>> e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow
>> Control: RX/TX
>> bonding: bond0: making interface eth0 the new active one.
>> bonding: bond0: first active interface up!
>> bonding: bond0: enslaving eth0 as an active interface with an up link.
>> bonding: bond0: Adding slave eth1.
>> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
>
> <SNIP>
>
>> bonding: bond0: enslaving eth1 as a backup interface with a down link.
>> bonding: bond0: Setting eth0 as primary slave.
>> bond0: no IPv6 routers present
>>
>
> Based on the console log, I'm guessing your initialization scripts use
> sysfs to set eth0 as the primary interface for bond0?  Can you confirm?

Yep, that's correct:

postup() {
         if [[ ${IFACE} == "bond0" ]] ; then
                 echo -n +eth0 > /sys/class/net/${IFACE}/bonding/slaves
                 echo -n +eth1 > /sys/class/net/${IFACE}/bonding/slaves
                 echo -n  eth0 > /sys/class/net/${IFACE}/bonding/primary
         fi
}

> If you did somehow use sysfs to set the primary device as eth0, I'm
> guessing you never see this issue without that line or without this
> patch.  Please confirm this as well.

Without this patch I get another error, of course. Unfortunately checking 
what happens without using sysfs to set the primary device as eth0, have 
to wait, as I am not able to test it before Monday.

Best regards,

 				Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 22:11               ` Krzysztof Oledzki
@ 2007-12-14 22:27                 ` Andy Gospodarek
  0 siblings, 0 replies; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-14 22:27 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: Andy Gospodarek, Jay Vosburgh, Herbert Xu, Andrew Morton,
	bugme-daemon, shemminger, davem, netdev

On Fri, Dec 14, 2007 at 11:11:15PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> 
> >On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
> >>
> >>
> >>On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> >>
> >>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
> >>>>
> >>>>
> >>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote:
> >>>>
> >>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >>>>>
> >>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>>>drivers/net/bonding/bond_sysfs.c
> >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c
> >>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >>>>>>>out:
> >>>>>>>     write_unlock_bh(&bond->lock);
> >>>>>>>
> >>>>>>>-       rtnl_unlock();
> >>>>>>>-
> >>>>>>
> >>>>>>Looking at the changeset that added this perhaps the intention
> >>>>>>is to hold the lock? If so we should add an rtnl_lock to the start
> >>>>>>of the function.
> >>>>>
> >>>>>	Yes, this function needs to hold locks, and more than just
> >>>>>what's there now.  I believe the following should be correct; I haven't
> >>>>>tested it, though (I'm supposedly on vacation right now).
> >>>>>
> >>>>>	The following change should be correct for the
> >>>>>bonding_store_primary case discussed in this thread, and also corrects
> >>>>>the bonding_store_active case which performs similar functions.
> >>>>>
> >>>>>	The bond_change_active_slave and bond_select_active_slave
> >>>>>functions both require rtnl, bond->lock for read and curr_slave_lock 
> >>>>>for
> >>>>>write_bh, and no other locks.  This is so that the lower level
> >>>>>mode-specific functions can release locks down to just rtnl in order to
> >>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
> >>>>>
> >>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
> >>>>>
> >>>>>diff --git a/drivers/net/bonding/bond_sysfs.c
> >>>>>b/drivers/net/bonding/bond_sysfs.c
> >>>>>index 11b76b3..28a2d80 100644
> >>>>>--- a/drivers/net/bonding/bond_sysfs.c
> >>>>>+++ b/drivers/net/bonding/bond_sysfs.c
> >>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct 
> >>>>>device
> >>>>>*d,
> >>>>>	struct slave *slave;
> >>>>>	struct bonding *bond = to_bond(d);
> >>>>>
> >>>>>-	write_lock_bh(&bond->lock);
> >>>>>+	rtnl_lock();
> >>>>>+	read_lock(&bond->lock);
> >>>>>+	write_lock_bh(&bond->curr_slave_lock);
> >F
> >>>>>+
> >>>>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>>>		printk(KERN_INFO DRV_NAME
> >>>>>		       ": %s: Unable to set primary slave; %s is in mode
> >>>>>		       %d\n",
> >>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct 
> >>>>>device
> >>>>>*d,
> >>>>>		}
> >>>>>	}
> >>>>>out:
> >>>>>-	write_unlock_bh(&bond->lock);
> >>>>>-
> >>>>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>>>+	read_unlock(&bond->lock);
> >>>>>	rtnl_unlock();
> >>>>>
> >>>>>	return count;
> >>>>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>>>device *d,
> >>>>>	struct bonding *bond = to_bond(d);
> >>>>>
> >>>>>	rtnl_lock();
> >>>>>-	write_lock_bh(&bond->lock);
> >>>>>+	read_lock(&bond->lock);
> >>>>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>>>
> >>>>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>>>		printk(KERN_INFO DRV_NAME
> >>>>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>>>device *d,
> >>>>>		}
> >>>>>	}
> >>>>>out:
> >>>>>-	write_unlock_bh(&bond->lock);
> >>>>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>>>+	read_unlock(&bond->lock);
> >>>>>	rtnl_unlock();
> >>>>>
> >>>>>	return count;
> >>>>
> >>>>Vanilla 2.6.24-rc5 plus this patch:
> >>>>
> >>>>=========================================================
> >>>>[ INFO: possible irq lock inversion dependency detected ]
> >>>>2.6.24-rc5 #1
> >>>>---------------------------------------------------------
> >>>>events/0/9 just changed the state of lock:
> >>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
> >>>>but this lock took another, soft-read-irq-unsafe lock in the past:
> >>>>(&bond->lock){-.--}
> >>>>
> >>>>and interrupts could create inverse lock ordering between them.
> >>>>
> >>>>
> >>>
> >>>Grrr, I should have seen that -- sorry.  Try your luck with this instead:
> >><CUT>
> >>
> >>No luck.
> >>
> >>bonding: bond0: setting mode to active-backup (1).
> >>bonding: bond0: Setting MII monitoring interval to 100.
> >>ADDRCONF(NETDEV_UP): bond0: link is not ready
> >>bonding: bond0: Adding slave eth0.
> >>e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow
> >>Control: RX/TX
> >>bonding: bond0: making interface eth0 the new active one.
> >>bonding: bond0: first active interface up!
> >>bonding: bond0: enslaving eth0 as an active interface with an up link.
> >>bonding: bond0: Adding slave eth1.
> >>ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
> >
> ><SNIP>
> >
> >>bonding: bond0: enslaving eth1 as a backup interface with a down link.
> >>bonding: bond0: Setting eth0 as primary slave.
> >>bond0: no IPv6 routers present
> >>
> >
> >Based on the console log, I'm guessing your initialization scripts use
> >sysfs to set eth0 as the primary interface for bond0?  Can you confirm?
> 
> Yep, that's correct:
> 
> postup() {
>         if [[ ${IFACE} == "bond0" ]] ; then
>                 echo -n +eth0 > /sys/class/net/${IFACE}/bonding/slaves
>                 echo -n +eth1 > /sys/class/net/${IFACE}/bonding/slaves
>                 echo -n  eth0 > /sys/class/net/${IFACE}/bonding/primary
>         fi
> }
> 

Good. Thanks for the confirmation.

> >If you did somehow use sysfs to set the primary device as eth0, I'm
> >guessing you never see this issue without that line or without this
> >patch.  Please confirm this as well.
> 
> Without this patch I get another error, of course. Unfortunately checking 

:-/


> what happens without using sysfs to set the primary device as eth0, have 
> to wait, as I am not able to test it before Monday.
> 

No problem, thanks for the feedback and quick response.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 18:57           ` Krzysztof Oledzki
  2007-12-14 22:03             ` Andy Gospodarek
@ 2007-12-14 22:47             ` Andy Gospodarek
  2007-12-15  4:10               ` Herbert Xu
  2007-12-18 19:53               ` Krzysztof Oledzki
  1 sibling, 2 replies; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-14 22:47 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: Andy Gospodarek, Jay Vosburgh, Herbert Xu, Andrew Morton,
	bugme-daemon, shemminger, davem, netdev

On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> 
> >On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
> >>
> >>
> >>On Wed, 12 Dec 2007, Jay Vosburgh wrote:
> >>
> >>>Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >>>
> >>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>drivers/net/bonding/bond_sysfs.c
> >>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>+++ a/drivers/net/bonding/bond_sysfs.c
> >>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >>>>>out:
> >>>>>      write_unlock_bh(&bond->lock);
> >>>>>
> >>>>>-       rtnl_unlock();
> >>>>>-
> >>>>
> >>>>Looking at the changeset that added this perhaps the intention
> >>>>is to hold the lock? If so we should add an rtnl_lock to the start
> >>>>of the function.
> >>>
> >>>	Yes, this function needs to hold locks, and more than just
> >>>what's there now.  I believe the following should be correct; I haven't
> >>>tested it, though (I'm supposedly on vacation right now).
> >>>
> >>>	The following change should be correct for the
> >>>bonding_store_primary case discussed in this thread, and also corrects
> >>>the bonding_store_active case which performs similar functions.
> >>>
> >>>	The bond_change_active_slave and bond_select_active_slave
> >>>functions both require rtnl, bond->lock for read and curr_slave_lock for
> >>>write_bh, and no other locks.  This is so that the lower level
> >>>mode-specific functions can release locks down to just rtnl in order to
> >>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
> >>>
> >>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
> >>>
> >>>diff --git a/drivers/net/bonding/bond_sysfs.c
> >>>b/drivers/net/bonding/bond_sysfs.c
> >>>index 11b76b3..28a2d80 100644
> >>>--- a/drivers/net/bonding/bond_sysfs.c
> >>>+++ b/drivers/net/bonding/bond_sysfs.c
> >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
> >>>*d,
> >>>	struct slave *slave;
> >>>	struct bonding *bond = to_bond(d);
> >>>
> >>>-	write_lock_bh(&bond->lock);
> >>>+	rtnl_lock();
> >>>+	read_lock(&bond->lock);
> >>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>+
> >>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>		printk(KERN_INFO DRV_NAME
> >>>		       ": %s: Unable to set primary slave; %s is in mode
> >>>		       %d\n",
> >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
> >>>*d,
> >>>		}
> >>>	}
> >>>out:
> >>>-	write_unlock_bh(&bond->lock);
> >>>-
> >>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>+	read_unlock(&bond->lock);
> >>>	rtnl_unlock();
> >>>
> >>>	return count;
> >>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>device *d,
> >>>	struct bonding *bond = to_bond(d);
> >>>
> >>>	rtnl_lock();
> >>>-	write_lock_bh(&bond->lock);
> >>>+	read_lock(&bond->lock);
> >>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>
> >>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>		printk(KERN_INFO DRV_NAME
> >>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>device *d,
> >>>		}
> >>>	}
> >>>out:
> >>>-	write_unlock_bh(&bond->lock);
> >>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>+	read_unlock(&bond->lock);
> >>>	rtnl_unlock();
> >>>
> >>>	return count;
> >>
> >>Vanilla 2.6.24-rc5 plus this patch:
> >>
> >>=========================================================
> >>[ INFO: possible irq lock inversion dependency detected ]
> >>2.6.24-rc5 #1
> >>---------------------------------------------------------
> >>events/0/9 just changed the state of lock:
> >> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
> >>but this lock took another, soft-read-irq-unsafe lock in the past:
> >> (&bond->lock){-.--}
> >>
> >>and interrupts could create inverse lock ordering between them.
> >>
> >>
> >
> >Grrr, I should have seen that -- sorry.  Try your luck with this instead:
> <CUT>
> 
> No luck.
> 


I'm guessing if we go back to using a write-lock for bond->lock this
will go back to working again, but I'm not totally convinced since there
are plenty of places where we used a read-lock with it.


diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
index 11b76b3..635b857 100644
--- a/drivers/net/bonding/bond_sysfs.c
+++ b/drivers/net/bonding/bond_sysfs.c
@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d,
 	struct slave *slave;
 	struct bonding *bond = to_bond(d);
 
+	rtnl_lock();
 	write_lock_bh(&bond->lock);
+	write_lock_bh(&bond->curr_slave_lock);
+
 	if (!USES_PRIMARY(bond->params.mode)) {
 		printk(KERN_INFO DRV_NAME
 		       ": %s: Unable to set primary slave; %s is in mode %d\n",
@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d,
 		}
 	}
 out:
+	write_unlock_bh(&bond->curr_slave_lock);
 	write_unlock_bh(&bond->lock);
-
 	rtnl_unlock();
 
 	return count;
@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct device *d,
 
 	rtnl_lock();
 	write_lock_bh(&bond->lock);
+	write_lock_bh(&bond->curr_slave_lock);
 
 	if (!USES_PRIMARY(bond->params.mode)) {
 		printk(KERN_INFO DRV_NAME
@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct device *d,
 		}
 	}
 out:
+	write_unlock_bh(&bond->curr_slave_lock);
 	write_unlock_bh(&bond->lock);
 	rtnl_unlock();
 

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 22:47             ` Andy Gospodarek
@ 2007-12-15  4:10               ` Herbert Xu
  2007-12-15 15:09                 ` Andy Gospodarek
  2007-12-18 19:53               ` Krzysztof Oledzki
  1 sibling, 1 reply; 24+ messages in thread
From: Herbert Xu @ 2007-12-15  4:10 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Krzysztof Oledzki, Jay Vosburgh, Andrew Morton, bugme-daemon,
	shemminger, davem, netdev

On Fri, Dec 14, 2007 at 05:47:22PM -0500, Andy Gospodarek wrote:
>
> I'm guessing if we go back to using a write-lock for bond->lock this
> will go back to working again, but I'm not totally convinced since there
> are plenty of places where we used a read-lock with it.

Sorry I forgot to cc you earlier Andy.

But to fix this you need make sure that all read locks on bond->lock
in process context disable BH.  This is because at least one write
lock can be taken from BH context.

You don't need to turn the read locks into write locks however.

This is also something that we can undo once the set_multicast
interface has been fixed to not take the tx lock.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-15  4:10               ` Herbert Xu
@ 2007-12-15 15:09                 ` Andy Gospodarek
  2007-12-16  2:27                   ` Herbert Xu
  0 siblings, 1 reply; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-15 15:09 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Krzysztof Oledzki, Jay Vosburgh, Andrew Morton, bugme-daemon,
	shemminger, davem, netdev

On Dec 14, 2007 11:10 PM, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Fri, Dec 14, 2007 at 05:47:22PM -0500, Andy Gospodarek wrote:
> >
> > I'm guessing if we go back to using a write-lock for bond->lock this
> > will go back to working again, but I'm not totally convinced since there
> > are plenty of places where we used a read-lock with it.
>
> Sorry I forgot to cc you earlier Andy.
>
> But to fix this you need make sure that all read locks on bond->lock
> in process context disable BH.  This is because at least one write
> lock can be taken from BH context.

I agree with you completely, Herbet, which is why I was surprised that
my first apparently did not resolve the issue.  I felt it should
have....

> You don't need to turn the read locks into write locks however.
>
> This is also something that we can undo once the set_multicast
> interface has been fixed to not take the tx lock.
>
> Cheers,
> --
> Visit Openswan at http://www.openswan.org/
> Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
>

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-15 15:09                 ` Andy Gospodarek
@ 2007-12-16  2:27                   ` Herbert Xu
  2007-12-16  3:17                     ` Andy Gospodarek
  0 siblings, 1 reply; 24+ messages in thread
From: Herbert Xu @ 2007-12-16  2:27 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: herbert, olel, fubar, akpm, bugme-daemon, shemminger, davem,
	netdev

Andy Gospodarek <andy@greyhouse.net> wrote:
>
> I agree with you completely, Herbet, which is why I was surprised that
> my first apparently did not resolve the issue.  I felt it should
> have....

Did it change all occurrences of read_lock(&bond->lock) to
read_lock_bh? If so I better look at the lockdep output again.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-16  2:27                   ` Herbert Xu
@ 2007-12-16  3:17                     ` Andy Gospodarek
  2007-12-16  3:23                       ` Herbert Xu
  0 siblings, 1 reply; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-16  3:17 UTC (permalink / raw)
  To: Herbert Xu; +Cc: olel, fubar, akpm, bugme-daemon, shemminger, davem, netdev

On Dec 15, 2007 9:27 PM, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> Andy Gospodarek <andy@greyhouse.net> wrote:
> >
> > I agree with you completely, Herbet, which is why I was surprised that
> > my first apparently did not resolve the issue.  I felt it should
> > have....
>
> Did it change all occurrences of read_lock(&bond->lock) to
> read_lock_bh? If so I better look at the lockdep output again.
>

Not all of them in the bonding code, but all two of them in the small patch.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-16  3:17                     ` Andy Gospodarek
@ 2007-12-16  3:23                       ` Herbert Xu
  0 siblings, 0 replies; 24+ messages in thread
From: Herbert Xu @ 2007-12-16  3:23 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: olel, fubar, akpm, bugme-daemon, shemminger, davem, netdev

On Sat, Dec 15, 2007 at 10:17:35PM -0500, Andy Gospodarek wrote:
>
> Not all of them in the bonding code, but all two of them in the small patch.

OK, we need to change all of the ones that may be called from
process context with BH on.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 22:03             ` Andy Gospodarek
  2007-12-14 22:11               ` Krzysztof Oledzki
@ 2007-12-18 19:52               ` Krzysztof Oledzki
  1 sibling, 0 replies; 24+ messages in thread
From: Krzysztof Oledzki @ 2007-12-18 19:52 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev

[-- Attachment #1: Type: TEXT/PLAIN, Size: 63719 bytes --]



On Fri, 14 Dec 2007, Andy Gospodarek wrote:

> On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
>>
>>
>> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>>
>>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
>>>>
>>>>
>>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote:
>>>>
>>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote:
>>>>>
>>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>> drivers/net/bonding/bond_sysfs.c
>>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c
>>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>>>>>>> out:
>>>>>>>      write_unlock_bh(&bond->lock);
>>>>>>>
>>>>>>> -       rtnl_unlock();
>>>>>>> -
>>>>>>
>>>>>> Looking at the changeset that added this perhaps the intention
>>>>>> is to hold the lock? If so we should add an rtnl_lock to the start
>>>>>> of the function.
>>>>>
>>>>> 	Yes, this function needs to hold locks, and more than just
>>>>> what's there now.  I believe the following should be correct; I haven't
>>>>> tested it, though (I'm supposedly on vacation right now).
>>>>>
>>>>> 	The following change should be correct for the
>>>>> bonding_store_primary case discussed in this thread, and also corrects
>>>>> the bonding_store_active case which performs similar functions.
>>>>>
>>>>> 	The bond_change_active_slave and bond_select_active_slave
>>>>> functions both require rtnl, bond->lock for read and curr_slave_lock for
>>>>> write_bh, and no other locks.  This is so that the lower level
>>>>> mode-specific functions can release locks down to just rtnl in order to
>>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
>>>>>
>>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>>>>>
>>>>> diff --git a/drivers/net/bonding/bond_sysfs.c
>>>>> b/drivers/net/bonding/bond_sysfs.c
>>>>> index 11b76b3..28a2d80 100644
>>>>> --- a/drivers/net/bonding/bond_sysfs.c
>>>>> +++ b/drivers/net/bonding/bond_sysfs.c
>>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
>>>>> *d,
>>>>> 	struct slave *slave;
>>>>> 	struct bonding *bond = to_bond(d);
>>>>>
>>>>> -	write_lock_bh(&bond->lock);
>>>>> +	rtnl_lock();
>>>>> +	read_lock(&bond->lock);
>>>>> +	write_lock_bh(&bond->curr_slave_lock);
> F
>>>>> +
>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>> 		printk(KERN_INFO DRV_NAME
>>>>> 		       ": %s: Unable to set primary slave; %s is in mode
>>>>> 		       %d\n",
>>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
>>>>> *d,
>>>>> 		}
>>>>> 	}
>>>>> out:
>>>>> -	write_unlock_bh(&bond->lock);
>>>>> -
>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>> +	read_unlock(&bond->lock);
>>>>> 	rtnl_unlock();
>>>>>
>>>>> 	return count;
>>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>> device *d,
>>>>> 	struct bonding *bond = to_bond(d);
>>>>>
>>>>> 	rtnl_lock();
>>>>> -	write_lock_bh(&bond->lock);
>>>>> +	read_lock(&bond->lock);
>>>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>>>
>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>> 		printk(KERN_INFO DRV_NAME
>>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>> device *d,
>>>>> 		}
>>>>> 	}
>>>>> out:
>>>>> -	write_unlock_bh(&bond->lock);
>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>> +	read_unlock(&bond->lock);
>>>>> 	rtnl_unlock();
>>>>>
>>>>> 	return count;
>>>>
>>>> Vanilla 2.6.24-rc5 plus this patch:
>>>>
>>>> =========================================================
>>>> [ INFO: possible irq lock inversion dependency detected ]
>>>> 2.6.24-rc5 #1
>>>> ---------------------------------------------------------
>>>> events/0/9 just changed the state of lock:
>>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
>>>> but this lock took another, soft-read-irq-unsafe lock in the past:
>>>> (&bond->lock){-.--}
>>>>
>>>> and interrupts could create inverse lock ordering between them.
>>>>
>>>>
>>>
>>> Grrr, I should have seen that -- sorry.  Try your luck with this instead:
>> <CUT>
>>
>> No luck.
>>
>> bonding: bond0: setting mode to active-backup (1).
>> bonding: bond0: Setting MII monitoring interval to 100.
>> ADDRCONF(NETDEV_UP): bond0: link is not ready
>> bonding: bond0: Adding slave eth0.
>> e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow
>> Control: RX/TX
>> bonding: bond0: making interface eth0 the new active one.
>> bonding: bond0: first active interface up!
>> bonding: bond0: enslaving eth0 as an active interface with an up link.
>> bonding: bond0: Adding slave eth1.
>> ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
>
> <SNIP>
>
>> bonding: bond0: enslaving eth1 as a backup interface with a down link.
>> bonding: bond0: Setting eth0 as primary slave.
>> bond0: no IPv6 routers present
>>
>
> Based on the console log, I'm guessing your initialization scripts use
> sysfs to set eth0 as the primary interface for bond0?  Can you confirm?
>
> If you did somehow use sysfs to set the primary device as eth0, I'm
> guessing you never see this issue without that line or without this
> patch.  Please confirm this as well.

Unpatched 2.6.24-rc5 with "echo -n eth0 > /sys/class/net/${IFACE}/bonding/primary"
removed, still has this issue. I also removed:
#               echo -n 1 > /sys/class/net/${IFACE}/bonding/mode
#               echo -n 100 > /sys/class/net/${IFACE}/bonding/miimon
but this did not help:

bonding: bond0: setting mode to active-backup (1).
bonding: bond0: Setting MII monitoring interval to 100.
ADDRCONF(NETDEV_UP): bond0: link is not ready
bonding: bond0: Adding slave eth0.
e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
bonding: bond0: making interface eth0 the new active one.
bonding: bond0: first active interface up!
bonding: bond0: enslaving eth0 as an active interface with an up link.
bonding: bond0: Adding slave eth1.
ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready

=========================================================
[ INFO: possible irq lock inversion dependency detected ]
2.6.24-rc5 #1
---------------------------------------------------------
events/0/9 just changed the state of lock:
  (&mc->mca_lock){-+..}, at: [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
but this lock took another, soft-read-irq-unsafe lock in the past:
  (&bond->lock){-.--}

and interrupts could create inverse lock ordering between them.


other info that might help us debug this:
4 locks held by events/0/9:
  #0:  (events){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6
  #1:  ((linkwatch_work).work){--..}, at: [<c0133c57>] run_workqueue+0x87/0x1b6
  #2:  (rtnl_mutex){--..}, at: [<c03abd28>] linkwatch_event+0x5/0x22
  #3:  (&ndev->lock){-.-+}, at: [<c0411b39>] mld_ifc_timer_expire+0x17/0x1fb

the first lock's dependencies:
-> (&mc->mca_lock){-+..} ops: 10 {
    initial-use  at:
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c014289c>] __lock_acquire+0x4ba/0xc07
                         [<c0109ef2>] save_stack_trace+0x20/0x3a
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c04120ae>] igmp6_group_added+0x18/0x11d
                         [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                         [<c04120ae>] igmp6_group_added+0x18/0x11d
                         [<c04120ae>] igmp6_group_added+0x18/0x11d
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                         [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                         [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                         [<c040180c>] ipv6_add_dev+0x21c/0x24b
                         [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                         [<c05c5b40>] addrconf_init+0x13/0x193
                         [<c0199f63>] proc_net_fops_create+0x10/0x21
                         [<c0419b10>] ip6_flowlabel_init+0x1e/0x20
                         [<c05c5a20>] inet6_init+0x1f0/0x2ad
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    in-softirq-W at:
                         [<c0142822>] __lock_acquire+0x440/0xc07
                         [<c011d686>] update_curr+0x52/0xc4
                         [<c013f175>] tick_sched_timer+0x129/0x165
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                         [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
                         [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                         [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb
                         [<c012df52>] run_timer_softirq+0xfa/0x15d
                         [<c012a8a6>] __do_softirq+0x56/0xdb
                         [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                         [<c012a8b8>] __do_softirq+0x68/0xdb
                         [<c012a961>] do_softirq+0x36/0x51
                         [<c012ae4a>] local_bh_enable_ip+0xad/0xed
                         [<c03bf0df>] rt_run_flush+0x64/0x8b
                         [<c03e926e>] fib_netdev_event+0x61/0x65
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a2abd>] netdev_state_change+0x18/0x29
                         [<c03abcf5>] __linkwatch_run_queue+0x150/0x17e
                         [<c03abd40>] linkwatch_event+0x1d/0x22
                         [<c0133cab>] run_workqueue+0xdb/0x1b6
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c03abd23>] linkwatch_event+0x0/0x22
                         [<c01346cb>] worker_thread+0x0/0x85
                         [<c0134744>] worker_thread+0x79/0x85
                         [<c0137179>] autoremove_wake_function+0x0/0x35
                         [<c01370c2>] kthread+0x38/0x5e
                         [<c013708a>] kthread+0x0/0x5e
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-W at:
                         [<c01417ee>] find_usage_backwards+0xbb/0xe2
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c014286a>] __lock_acquire+0x488/0xc07
                         [<c0109ef2>] save_stack_trace+0x20/0x3a
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c04120ae>] igmp6_group_added+0x18/0x11d
                         [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                         [<c04120ae>] igmp6_group_added+0x18/0x11d
                         [<c04120ae>] igmp6_group_added+0x18/0x11d
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                         [<c041242a>] ipv6_dev_mc_inc+0x24d/0x31c
                         [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                         [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                         [<c040180c>] ipv6_add_dev+0x21c/0x24b
                         [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                         [<c05c5b40>] addrconf_init+0x13/0x193
                         [<c0199f63>] proc_net_fops_create+0x10/0x21
                         [<c0419b10>] ip6_flowlabel_init+0x1e/0x20
                         [<c05c5a20>] inet6_init+0x1f0/0x2ad
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
  }
  ... key      at: [<c087e2d8>] __key.30798+0x0/0x8
  -> (_xmit_ETHER){-...} ops: 8 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c04120ec>] igmp6_group_added+0x56/0x11d
                           [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                           [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c040180c>] ipv6_add_dev+0x21c/0x24b
                           [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401def>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c043903e>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c04120ec>] igmp6_group_added+0x56/0x11d
                           [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                           [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c040180c>] ipv6_add_dev+0x21c/0x24b
                           [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401def>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c043903e>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c0439d3a>] _spin_lock_bh+0x3b/0x64
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c04120ec>] igmp6_group_added+0x56/0x11d
    [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
    [<c0410100>] igmp6_mc_seq_start+0x106/0x138
    [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
    [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
    [<c040180c>] ipv6_add_dev+0x21c/0x24b
    [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
    [<c0401def>] addrconf_notify+0x60/0x7b7
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c043903e>] mutex_lock_nested+0x286/0x2ac
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
    [<c03a3f25>] register_netdevice_notifier+0xe/0x126
    [<c03a3f25>] register_netdevice_notifier+0xe/0x126
    [<c03a3f60>] register_netdevice_notifier+0x49/0x126
    [<c05c5bda>] addrconf_init+0xad/0x193
    [<c05c5b48>] addrconf_init+0x1b/0x193
    [<c05c5a20>] inet6_init+0x1f0/0x2ad
    [<c05a9499>] kernel_init+0x150/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c0104baf>] kernel_thread_helper+0x7/0x10
    [<ffffffff>] 0xffffffff

  -> (&bonding_netdev_xmit_lock_key){-...} ops: 6 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c04120ec>] igmp6_group_added+0x56/0x11d
                           [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                           [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c040180c>] ipv6_add_dev+0x21c/0x24b
                           [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401def>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c043903e>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c04120ec>] igmp6_group_added+0x56/0x11d
                           [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                           [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c040180c>] ipv6_add_dev+0x21c/0x24b
                           [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401def>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c043903e>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c0877804>] bonding_netdev_xmit_lock_key+0x0/0x8
   -> (&bond->lock){-.--} ops: 99 {
      initial-use  at:
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c014289c>] __lock_acquire+0x4ba/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c0439ec6>] _read_lock_bh+0x3b/0x64
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563
                             [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf
                             [<c03aa9fe>] rtnetlink_event+0x3f/0x42
                             [<c013ac20>] notifier_call_chain+0x2a/0x52
                             [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                             [<c03a31b6>] register_netdevice+0x2a7/0x2e7
                             [<c02ed862>] bond_create+0x1f2/0x26a
                             [<c05bedcd>] bonding_init+0x761/0x7ea
                             [<c05be635>] e1000_init_module+0x45/0x7c
                             [<c05a9499>] kernel_init+0x150/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
      hardirq-on-W at:
                             [<c014286a>] __lock_acquire+0x488/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a3d5>] _spin_unlock_irq+0x20/0x41
                             [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                             [<c043a3e0>] _spin_unlock_irq+0x2b/0x41
                             [<c012208a>] finish_task_switch+0x50/0x8c
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                             [<c0439dfd>] _write_lock_bh+0x3b/0x64
                             [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                             [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c03a14b3>] __dev_set_rx_mode+0x7b/0x7d
                             [<c03a164d>] dev_set_rx_mode+0x23/0x36
                             [<c03a3d28>] dev_open+0x5e/0x77
                             [<c03a29f7>] dev_change_flags+0x9d/0x14b
                             [<c03a17fb>] __dev_get_by_name+0x68/0x73
                             [<c03e3828>] devinet_ioctl+0x22b/0x536
                             [<c03a3b1d>] dev_ioctl+0x46f/0x5b7
                             [<c0399c50>] sock_ioctl+0x167/0x18b
                             [<c0399ae9>] sock_ioctl+0x0/0x18b
                             [<c01725f7>] do_ioctl+0x1f/0x62
                             [<c0172867>] vfs_ioctl+0x22d/0x23f
                             [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                             [<c01728ac>] sys_ioctl+0x33/0x4b
                             [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                             [<ffffffff>] 0xffffffff
      softirq-on-R at:
                             [<c0141986>] mark_lock+0x64/0x451
                             [<c013575e>] __kernel_text_address+0x5/0xe
                             [<c0104ee2>] dump_trace+0x83/0x8d
                             [<c0142889>] __lock_acquire+0x4a7/0xc07
                             [<c013fc76>] save_trace+0x37/0x89
                             [<c0133c57>] run_workqueue+0x87/0x1b6
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c0439f25>] _read_lock+0x36/0x5f
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c02eee5d>] bond_mii_monitor+0x19/0x85
                             [<c0133cab>] run_workqueue+0xdb/0x1b6
                             [<c0133c57>] run_workqueue+0x87/0x1b6
                             [<c02eee44>] bond_mii_monitor+0x0/0x85
                             [<c01346cb>] worker_thread+0x0/0x85
                             [<c0134744>] worker_thread+0x79/0x85
                             [<c0137179>] autoremove_wake_function+0x0/0x35
                             [<c01370c2>] kthread+0x38/0x5e
                             [<c013708a>] kthread+0x0/0x5e
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
      hardirq-on-R at:
                             [<c013fe0a>] get_lock_stats+0xd/0x2e
                             [<c013fe35>] put_lock_stats+0xa/0x1e
                             [<c0142844>] __lock_acquire+0x462/0xc07
                             [<c0141dac>] mark_held_locks+0x39/0x53
                             [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
                             [<c0143062>] lock_acquire+0x79/0x93
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c0439ec6>] _read_lock_bh+0x3b/0x64
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c02edcc1>] bond_get_stats+0x28/0xd0
                             [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563
                             [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf
                             [<c03aa9fe>] rtnetlink_event+0x3f/0x42
                             [<c013ac20>] notifier_call_chain+0x2a/0x52
                             [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                             [<c03a31b6>] register_netdevice+0x2a7/0x2e7
                             [<c02ed862>] bond_create+0x1f2/0x26a
                             [<c05bedcd>] bonding_init+0x761/0x7ea
                             [<c05be635>] e1000_init_module+0x45/0x7c
                             [<c05a9499>] kernel_init+0x150/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c05a9349>] kernel_init+0x0/0x2b7
                             [<c0104baf>] kernel_thread_helper+0x7/0x10
                             [<ffffffff>] 0xffffffff
    }
    ... key      at: [<c08777d0>] __key.32969+0x0/0x8
    -> (_xmit_ETHER){-...} ops: 8 {
       initial-use  at:
                               [<c014289c>] __lock_acquire+0x4ba/0xc07
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03a5a79>] dev_mc_add+0x1a/0x6a
                               [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                               [<c03a5a79>] dev_mc_add+0x1a/0x6a
                               [<c03a5a79>] dev_mc_add+0x1a/0x6a
                               [<c04120ec>] igmp6_group_added+0x56/0x11d
                               [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                               [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                               [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                               [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                               [<c040180c>] ipv6_add_dev+0x21c/0x24b
                               [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                               [<c0401def>] addrconf_notify+0x60/0x7b7
                               [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                               [<c0141dac>] mark_held_locks+0x39/0x53
                               [<c043903e>] mutex_lock_nested+0x286/0x2ac
                               [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                               [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                               [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                               [<c05c5bda>] addrconf_init+0xad/0x193
                               [<c05c5b48>] addrconf_init+0x1b/0x193
                               [<c05c5a20>] inet6_init+0x1f0/0x2ad
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
       hardirq-on-W at:
                               [<c0141986>] mark_lock+0x64/0x451
                               [<c014286a>] __lock_acquire+0x488/0xc07
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03a5a79>] dev_mc_add+0x1a/0x6a
                               [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                               [<c03a5a79>] dev_mc_add+0x1a/0x6a
                               [<c03a5a79>] dev_mc_add+0x1a/0x6a
                               [<c04120ec>] igmp6_group_added+0x56/0x11d
                               [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                               [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                               [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                               [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                               [<c040180c>] ipv6_add_dev+0x21c/0x24b
                               [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                               [<c0401def>] addrconf_notify+0x60/0x7b7
                               [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                               [<c0141dac>] mark_held_locks+0x39/0x53
                               [<c043903e>] mutex_lock_nested+0x286/0x2ac
                               [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                               [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                               [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                               [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                               [<c05c5bda>] addrconf_init+0xad/0x193
                               [<c05c5b48>] addrconf_init+0x1b/0x193
                               [<c05c5a20>] inet6_init+0x1f0/0x2ad
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
     }
     ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
    ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
    [<c0109ef2>] save_stack_trace+0x20/0x3a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c0439d3a>] _spin_lock_bh+0x3b/0x64
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf
    [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65
    [<c02ee9af>] bond_select_active_slave+0x95/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

    -> (lweventlist_lock){.+..} ops: 10 {
       initial-use  at:
                               [<c0141986>] mark_lock+0x64/0x451
                               [<c014289c>] __lock_acquire+0x4ba/0xc07
                               [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3
                               [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3
                               [<c024a7de>] delay_tsc+0x25/0x3b
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03abacf>] linkwatch_add_event+0xd/0x2c
                               [<c043a057>] _spin_lock_irqsave+0x3f/0x6c
                               [<c03abacf>] linkwatch_add_event+0xd/0x2c
                               [<c03abacf>] linkwatch_add_event+0xd/0x2c
                               [<c03abb93>] linkwatch_fire_event+0x25/0x37
                               [<c02e1c43>] e1000_probe+0xad1/0xbe8
                               [<c0257f3f>] pci_device_probe+0x36/0x57
                               [<c02d0e5f>] driver_probe_device+0xe1/0x15f
                               [<c043a2d1>] _spin_unlock+0x25/0x3b
                               [<c043758a>] klist_next+0x58/0x6d
                               [<c02d0f6f>] __driver_attach+0x0/0x7f
                               [<c02d0fb8>] __driver_attach+0x49/0x7f
                               [<c02d0403>] bus_for_each_dev+0x36/0x58
                               [<c02d0cb7>] driver_attach+0x16/0x18
                               [<c02d0f6f>] __driver_attach+0x0/0x7f
                               [<c02d06fa>] bus_add_driver+0x6d/0x18d
                               [<c0258089>] __pci_register_driver+0x53/0x7f
                               [<c05be635>] e1000_init_module+0x45/0x7c
                               [<c05a9499>] kernel_init+0x150/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c05a9349>] kernel_init+0x0/0x2b7
                               [<c0104baf>] kernel_thread_helper+0x7/0x10
                               [<ffffffff>] 0xffffffff
       in-softirq-W at:
                               [<c011d20a>] __wake_up_common+0x32/0x5c
                               [<c0142822>] __lock_acquire+0x440/0xc07
                               [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
                               [<c0143062>] lock_acquire+0x79/0x93
                               [<c03abacf>] linkwatch_add_event+0xd/0x2c
                               [<c02dff01>] e1000_watchdog+0x0/0x5c9
                               [<c043a057>] _spin_lock_irqsave+0x3f/0x6c
                               [<c03abacf>] linkwatch_add_event+0xd/0x2c
                               [<c03abacf>] linkwatch_add_event+0xd/0x2c
                               [<c03abb93>] linkwatch_fire_event+0x25/0x37
                               [<c03aeb1a>] netif_carrier_on+0x16/0x27
                               [<c02e0156>] e1000_watchdog+0x255/0x5c9
                               [<c02dff01>] e1000_watchdog+0x0/0x5c9
                               [<c012df52>] run_timer_softirq+0xfa/0x15d
                               [<c012a8a6>] __do_softirq+0x56/0xdb
                               [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                               [<c012a8b8>] __do_softirq+0x68/0xdb
                               [<c012a961>] do_softirq+0x36/0x51
                               [<c012ae4a>] local_bh_enable_ip+0xad/0xed
                               [<c03bf0df>] rt_run_flush+0x64/0x8b
                               [<c03e7924>] ip_mc_inc_group+0x184/0x1c1
                               [<c03e79a2>] ip_mc_up+0x41/0x59
                               [<c03e32dc>] inetdev_event+0x257/0x465
                               [<c03aa8da>] rtnl_notify+0x3a/0x40
                               [<c03aa997>] rtmsg_ifinfo+0xb7/0xdf
                               [<c013ac20>] notifier_call_chain+0x2a/0x52
                               [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                               [<c03a3d3b>] dev_open+0x71/0x77
                               [<c02ef626>] bond_enslave+0x30f/0x884
                               [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                               [<c02f6161>] bonding_store_slaves+0x1a2/0x2fb
                               [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb
                               [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb
                               [<c02ce8d7>] dev_attr_store+0x27/0x2c
                               [<c019bcb9>] sysfs_write_file+0xad/0xe0
                               [<c019bc0c>] sysfs_write_file+0x0/0xe0
                               [<c0168ddc>] vfs_write+0x8a/0x10c
                               [<c0118566>] do_page_fault+0x0/0x54a
                               [<c0169361>] sys_write+0x41/0x67
                               [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                               [<ffffffff>] 0xffffffff
     }
     ... key      at: [<c058a194>] lweventlist_lock+0x14/0x40
    ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c043a057>] _spin_lock_irqsave+0x3f/0x6c
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c03abb93>] linkwatch_fire_event+0x25/0x37
    [<c03aeb1a>] netif_carrier_on+0x16/0x27
    [<c02ede2c>] bond_set_carrier+0x31/0x55
    [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

   ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c0143062>] lock_acquire+0x79/0x93
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c0439dfd>] _write_lock_bh+0x3b/0x64
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c02eda75>] bond_set_multicast_list+0x1d/0x241
    [<c013fe35>] put_lock_stats+0xa/0x1e
    [<c03a14b3>] __dev_set_rx_mode+0x7b/0x7d
    [<c03a164d>] dev_set_rx_mode+0x23/0x36
    [<c03a3d28>] dev_open+0x5e/0x77
    [<c03a29f7>] dev_change_flags+0x9d/0x14b
    [<c03a17fb>] __dev_get_by_name+0x68/0x73
    [<c03e3828>] devinet_ioctl+0x22b/0x536
    [<c03a3b1d>] dev_ioctl+0x46f/0x5b7
    [<c0399c50>] sock_ioctl+0x167/0x18b
    [<c0399ae9>] sock_ioctl+0x0/0x18b
    [<c01725f7>] do_ioctl+0x1f/0x62
    [<c0172867>] vfs_ioctl+0x22d/0x23f
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c01728ac>] sys_ioctl+0x33/0x4b
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c0439d3a>] _spin_lock_bh+0x3b/0x64
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c04120ec>] igmp6_group_added+0x56/0x11d
    [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
    [<c0410100>] igmp6_mc_seq_start+0x106/0x138
    [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
    [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
    [<c040180c>] ipv6_add_dev+0x21c/0x24b
    [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
    [<c0401def>] addrconf_notify+0x60/0x7b7
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0141dac>] mark_held_locks+0x39/0x53
    [<c043903e>] mutex_lock_nested+0x286/0x2ac
    [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
    [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
    [<c03a3f25>] register_netdevice_notifier+0xe/0x126
    [<c03a3f25>] register_netdevice_notifier+0xe/0x126
    [<c03a3f60>] register_netdevice_notifier+0x49/0x126
    [<c05c5bda>] addrconf_init+0xad/0x193
    [<c05c5b48>] addrconf_init+0x1b/0x193
    [<c05c5a20>] inet6_init+0x1f0/0x2ad
    [<c05a9499>] kernel_init+0x150/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c05a9349>] kernel_init+0x0/0x2b7
    [<c0104baf>] kernel_thread_helper+0x7/0x10
    [<ffffffff>] 0xffffffff


the second lock's dependencies:
-> (&bond->lock){-.--} ops: 99 {
    initial-use  at:
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c014289c>] __lock_acquire+0x4ba/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c0439ec6>] _read_lock_bh+0x3b/0x64
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563
                         [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf
                         [<c03aa9fe>] rtnetlink_event+0x3f/0x42
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a31b6>] register_netdevice+0x2a7/0x2e7
                         [<c02ed862>] bond_create+0x1f2/0x26a
                         [<c05bedcd>] bonding_init+0x761/0x7ea
                         [<c05be635>] e1000_init_module+0x45/0x7c
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-W at:
                         [<c014286a>] __lock_acquire+0x488/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a3d5>] _spin_unlock_irq+0x20/0x41
                         [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                         [<c043a3e0>] _spin_unlock_irq+0x2b/0x41
                         [<c012208a>] finish_task_switch+0x50/0x8c
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c0439dfd>] _write_lock_bh+0x3b/0x64
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c02eda75>] bond_set_multicast_list+0x1d/0x241
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c03a14b3>] __dev_set_rx_mode+0x7b/0x7d
                         [<c03a164d>] dev_set_rx_mode+0x23/0x36
                         [<c03a3d28>] dev_open+0x5e/0x77
                         [<c03a29f7>] dev_change_flags+0x9d/0x14b
                         [<c03a17fb>] __dev_get_by_name+0x68/0x73
                         [<c03e3828>] devinet_ioctl+0x22b/0x536
                         [<c03a3b1d>] dev_ioctl+0x46f/0x5b7
                         [<c0399c50>] sock_ioctl+0x167/0x18b
                         [<c0399ae9>] sock_ioctl+0x0/0x18b
                         [<c01725f7>] do_ioctl+0x1f/0x62
                         [<c0172867>] vfs_ioctl+0x22d/0x23f
                         [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                         [<c01728ac>] sys_ioctl+0x33/0x4b
                         [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                         [<ffffffff>] 0xffffffff
    softirq-on-R at:
                         [<c0141986>] mark_lock+0x64/0x451
                         [<c013575e>] __kernel_text_address+0x5/0xe
                         [<c0104ee2>] dump_trace+0x83/0x8d
                         [<c0142889>] __lock_acquire+0x4a7/0xc07
                         [<c013fc76>] save_trace+0x37/0x89
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c0439f25>] _read_lock+0x36/0x5f
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c02eee5d>] bond_mii_monitor+0x19/0x85
                         [<c0133cab>] run_workqueue+0xdb/0x1b6
                         [<c0133c57>] run_workqueue+0x87/0x1b6
                         [<c02eee44>] bond_mii_monitor+0x0/0x85
                         [<c01346cb>] worker_thread+0x0/0x85
                         [<c0134744>] worker_thread+0x79/0x85
                         [<c0137179>] autoremove_wake_function+0x0/0x35
                         [<c01370c2>] kthread+0x38/0x5e
                         [<c013708a>] kthread+0x0/0x5e
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
    hardirq-on-R at:
                         [<c013fe0a>] get_lock_stats+0xd/0x2e
                         [<c013fe35>] put_lock_stats+0xa/0x1e
                         [<c0142844>] __lock_acquire+0x462/0xc07
                         [<c0141dac>] mark_held_locks+0x39/0x53
                         [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
                         [<c0143062>] lock_acquire+0x79/0x93
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c0439ec6>] _read_lock_bh+0x3b/0x64
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c02edcc1>] bond_get_stats+0x28/0xd0
                         [<c03aa3bf>] rtnl_fill_ifinfo+0x2bf/0x563
                         [<c03aa93d>] rtmsg_ifinfo+0x5d/0xdf
                         [<c03aa9fe>] rtnetlink_event+0x3f/0x42
                         [<c013ac20>] notifier_call_chain+0x2a/0x52
                         [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                         [<c03a31b6>] register_netdevice+0x2a7/0x2e7
                         [<c02ed862>] bond_create+0x1f2/0x26a
                         [<c05bedcd>] bonding_init+0x761/0x7ea
                         [<c05be635>] e1000_init_module+0x45/0x7c
                         [<c05a9499>] kernel_init+0x150/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c05a9349>] kernel_init+0x0/0x2b7
                         [<c0104baf>] kernel_thread_helper+0x7/0x10
                         [<ffffffff>] 0xffffffff
  }
  ... key      at: [<c08777d0>] __key.32969+0x0/0x8
  -> (_xmit_ETHER){-...} ops: 8 {
     initial-use  at:
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c04120ec>] igmp6_group_added+0x56/0x11d
                           [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                           [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c040180c>] ipv6_add_dev+0x21c/0x24b
                           [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401def>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c043903e>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     hardirq-on-W at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014286a>] __lock_acquire+0x488/0xc07
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c0439d3a>] _spin_lock_bh+0x3b/0x64
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c03a5a79>] dev_mc_add+0x1a/0x6a
                           [<c04120ec>] igmp6_group_added+0x56/0x11d
                           [<c0412480>] ipv6_dev_mc_inc+0x2a3/0x31c
                           [<c0410100>] igmp6_mc_seq_start+0x106/0x138
                           [<c04124b5>] ipv6_dev_mc_inc+0x2d8/0x31c
                           [<c04121dd>] ipv6_dev_mc_inc+0x0/0x31c
                           [<c040180c>] ipv6_add_dev+0x21c/0x24b
                           [<c040b055>] ndisc_ifinfo_sysctl_change+0x0/0x1ef
                           [<c0401def>] addrconf_notify+0x60/0x7b7
                           [<c0142fa1>] __lock_acquire+0xbbf/0xc07
                           [<c0141dac>] mark_held_locks+0x39/0x53
                           [<c043903e>] mutex_lock_nested+0x286/0x2ac
                           [<c0141f9f>] trace_hardirqs_on+0x122/0x14c
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f25>] register_netdevice_notifier+0xe/0x126
                           [<c03a3f60>] register_netdevice_notifier+0x49/0x126
                           [<c05c5bda>] addrconf_init+0xad/0x193
                           [<c05c5b48>] addrconf_init+0x1b/0x193
                           [<c05c5a20>] inet6_init+0x1f0/0x2ad
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c087adc8>] netdev_xmit_lock_key+0x8/0x1c0
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
    [<c0109ef2>] save_stack_trace+0x20/0x3a
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c0439d3a>] _spin_lock_bh+0x3b/0x64
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c03a5a79>] dev_mc_add+0x1a/0x6a
    [<c02ee492>] bond_change_active_slave+0x1a9/0x3bf
    [<c02ec7c3>] bond_update_speed_duplex+0x26/0x65
    [<c02ee9af>] bond_select_active_slave+0x95/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff

  -> (lweventlist_lock){.+..} ops: 10 {
     initial-use  at:
                           [<c0141986>] mark_lock+0x64/0x451
                           [<c014289c>] __lock_acquire+0x4ba/0xc07
                           [<c02e365c>] e1000_read_phy_reg+0x1c7/0x1d3
                           [<c02e348b>] e1000_write_phy_reg+0xb9/0xc3
                           [<c024a7de>] delay_tsc+0x25/0x3b
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03abacf>] linkwatch_add_event+0xd/0x2c
                           [<c043a057>] _spin_lock_irqsave+0x3f/0x6c
                           [<c03abacf>] linkwatch_add_event+0xd/0x2c
                           [<c03abacf>] linkwatch_add_event+0xd/0x2c
                           [<c03abb93>] linkwatch_fire_event+0x25/0x37
                           [<c02e1c43>] e1000_probe+0xad1/0xbe8
                           [<c0257f3f>] pci_device_probe+0x36/0x57
                           [<c02d0e5f>] driver_probe_device+0xe1/0x15f
                           [<c043a2d1>] _spin_unlock+0x25/0x3b
                           [<c043758a>] klist_next+0x58/0x6d
                           [<c02d0f6f>] __driver_attach+0x0/0x7f
                           [<c02d0fb8>] __driver_attach+0x49/0x7f
                           [<c02d0403>] bus_for_each_dev+0x36/0x58
                           [<c02d0cb7>] driver_attach+0x16/0x18
                           [<c02d0f6f>] __driver_attach+0x0/0x7f
                           [<c02d06fa>] bus_add_driver+0x6d/0x18d
                           [<c0258089>] __pci_register_driver+0x53/0x7f
                           [<c05be635>] e1000_init_module+0x45/0x7c
                           [<c05a9499>] kernel_init+0x150/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c05a9349>] kernel_init+0x0/0x2b7
                           [<c0104baf>] kernel_thread_helper+0x7/0x10
                           [<ffffffff>] 0xffffffff
     in-softirq-W at:
                           [<c011d20a>] __wake_up_common+0x32/0x5c
                           [<c0142822>] __lock_acquire+0x440/0xc07
                           [<c043a39d>] _spin_unlock_irqrestore+0x40/0x58
                           [<c0143062>] lock_acquire+0x79/0x93
                           [<c03abacf>] linkwatch_add_event+0xd/0x2c
                           [<c02dff01>] e1000_watchdog+0x0/0x5c9
                           [<c043a057>] _spin_lock_irqsave+0x3f/0x6c
                           [<c03abacf>] linkwatch_add_event+0xd/0x2c
                           [<c03abacf>] linkwatch_add_event+0xd/0x2c
                           [<c03abb93>] linkwatch_fire_event+0x25/0x37
                           [<c03aeb1a>] netif_carrier_on+0x16/0x27
                           [<c02e0156>] e1000_watchdog+0x255/0x5c9
                           [<c02dff01>] e1000_watchdog+0x0/0x5c9
                           [<c012df52>] run_timer_softirq+0xfa/0x15d
                           [<c012a8a6>] __do_softirq+0x56/0xdb
                           [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
                           [<c012a8b8>] __do_softirq+0x68/0xdb
                           [<c012a961>] do_softirq+0x36/0x51
                           [<c012ae4a>] local_bh_enable_ip+0xad/0xed
                           [<c03bf0df>] rt_run_flush+0x64/0x8b
                           [<c03e7924>] ip_mc_inc_group+0x184/0x1c1
                           [<c03e79a2>] ip_mc_up+0x41/0x59
                           [<c03e32dc>] inetdev_event+0x257/0x465
                           [<c03aa8da>] rtnl_notify+0x3a/0x40
                           [<c03aa997>] rtmsg_ifinfo+0xb7/0xdf
                           [<c013ac20>] notifier_call_chain+0x2a/0x52
                           [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
                           [<c03a3d3b>] dev_open+0x71/0x77
                           [<c02ef626>] bond_enslave+0x30f/0x884
                           [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
                           [<c02f6161>] bonding_store_slaves+0x1a2/0x2fb
                           [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb
                           [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb
                           [<c02ce8d7>] dev_attr_store+0x27/0x2c
                           [<c019bcb9>] sysfs_write_file+0xad/0xe0
                           [<c019bc0c>] sysfs_write_file+0x0/0xe0
                           [<c0168ddc>] vfs_write+0x8a/0x10c
                           [<c0118566>] do_page_fault+0x0/0x54a
                           [<c0169361>] sys_write+0x41/0x67
                           [<c0103e92>] sysenter_past_esp+0x5f/0xa5
                           [<ffffffff>] 0xffffffff
   }
   ... key      at: [<c058a194>] lweventlist_lock+0x14/0x40
  ... acquired at:
    [<c0142dff>] __lock_acquire+0xa1d/0xc07
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c0142fa1>] __lock_acquire+0xbbf/0xc07
    [<c0143062>] lock_acquire+0x79/0x93
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c043a057>] _spin_lock_irqsave+0x3f/0x6c
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c03abacf>] linkwatch_add_event+0xd/0x2c
    [<c03abb93>] linkwatch_fire_event+0x25/0x37
    [<c03aeb1a>] netif_carrier_on+0x16/0x27
    [<c02ede2c>] bond_set_carrier+0x31/0x55
    [<c02ee9b6>] bond_select_active_slave+0x9c/0xcd
    [<c02ed22b>] bond_compute_features+0x45/0x84
    [<c02ef9be>] bond_enslave+0x6a7/0x884
    [<c043905c>] mutex_lock_nested+0x2a4/0x2ac
    [<c02f616d>] bonding_store_slaves+0x1ae/0x2fb
    [<c02f5fbf>] bonding_store_slaves+0x0/0x2fb
    [<c02ce8d7>] dev_attr_store+0x27/0x2c
    [<c019bcb9>] sysfs_write_file+0xad/0xe0
    [<c019bc0c>] sysfs_write_file+0x0/0xe0
    [<c0168ddc>] vfs_write+0x8a/0x10c
    [<c0118566>] do_page_fault+0x0/0x54a
    [<c0169361>] sys_write+0x41/0x67
    [<c0103e92>] sysenter_past_esp+0x5f/0xa5
    [<ffffffff>] 0xffffffff


stack backtrace:
Pid: 9, comm: events/0 Not tainted 2.6.24-rc5 #1
  [<c0140b38>] print_irq_inversion_bug+0x108/0x112
  [<c014191d>] check_usage_forwards+0x3c/0x41
  [<c0141b09>] mark_lock+0x1e7/0x451
  [<c0142822>] __lock_acquire+0x440/0xc07
  [<c011d686>] update_curr+0x52/0xc4
  [<c013f175>] tick_sched_timer+0x129/0x165
  [<c0143062>] lock_acquire+0x79/0x93
  [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb
  [<c0439d3a>] _spin_lock_bh+0x3b/0x64
  [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411c52>] mld_ifc_timer_expire+0x130/0x1fb
  [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb
  [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
  [<c0411b22>] mld_ifc_timer_expire+0x0/0x1fb
  [<c012df52>] run_timer_softirq+0xfa/0x15d
  [<c012a8a6>] __do_softirq+0x56/0xdb
  [<c0141f89>] trace_hardirqs_on+0x10c/0x14c
  [<c012a8b8>] __do_softirq+0x68/0xdb
  [<c012a961>] do_softirq+0x36/0x51
  [<c012ae4a>] local_bh_enable_ip+0xad/0xed
  [<c03bf0df>] rt_run_flush+0x64/0x8b
  [<c03e926e>] fib_netdev_event+0x61/0x65
  [<c013ac20>] notifier_call_chain+0x2a/0x52
  [<c013ac6a>] raw_notifier_call_chain+0x17/0x1a
  [<c03a2abd>] netdev_state_change+0x18/0x29
  [<c03abcf5>] __linkwatch_run_queue+0x150/0x17e
  [<c03abd40>] linkwatch_event+0x1d/0x22
  [<c0133cab>] run_workqueue+0xdb/0x1b6
  [<c0133c57>] run_workqueue+0x87/0x1b6
  [<c03abd23>] linkwatch_event+0x0/0x22
  [<c01346cb>] worker_thread+0x0/0x85
  [<c0134744>] worker_thread+0x79/0x85
  [<c0137179>] autoremove_wake_function+0x0/0x35
  [<c01370c2>] kthread+0x38/0x5e
  [<c013708a>] kthread+0x0/0x5e
  [<c0104baf>] kernel_thread_helper+0x7/0x10
  =======================
bonding: bond0: enslaving eth1 as a backup interface with a down link.
bond0: no IPv6 routers present



Best regards,

 				Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-14 22:47             ` Andy Gospodarek
  2007-12-15  4:10               ` Herbert Xu
@ 2007-12-18 19:53               ` Krzysztof Oledzki
  2007-12-19 14:42                 ` Andy Gospodarek
  1 sibling, 1 reply; 24+ messages in thread
From: Krzysztof Oledzki @ 2007-12-18 19:53 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev

[-- Attachment #1: Type: TEXT/PLAIN, Size: 5894 bytes --]



On Fri, 14 Dec 2007, Andy Gospodarek wrote:

> On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
>>
>>
>> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>>
>>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
>>>>
>>>>
>>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote:
>>>>
>>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote:
>>>>>
>>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>> drivers/net/bonding/bond_sysfs.c
>>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c
>>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>>>>>>> out:
>>>>>>>      write_unlock_bh(&bond->lock);
>>>>>>>
>>>>>>> -       rtnl_unlock();
>>>>>>> -
>>>>>>
>>>>>> Looking at the changeset that added this perhaps the intention
>>>>>> is to hold the lock? If so we should add an rtnl_lock to the start
>>>>>> of the function.
>>>>>
>>>>> 	Yes, this function needs to hold locks, and more than just
>>>>> what's there now.  I believe the following should be correct; I haven't
>>>>> tested it, though (I'm supposedly on vacation right now).
>>>>>
>>>>> 	The following change should be correct for the
>>>>> bonding_store_primary case discussed in this thread, and also corrects
>>>>> the bonding_store_active case which performs similar functions.
>>>>>
>>>>> 	The bond_change_active_slave and bond_select_active_slave
>>>>> functions both require rtnl, bond->lock for read and curr_slave_lock for
>>>>> write_bh, and no other locks.  This is so that the lower level
>>>>> mode-specific functions can release locks down to just rtnl in order to
>>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
>>>>>
>>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>>>>>
>>>>> diff --git a/drivers/net/bonding/bond_sysfs.c
>>>>> b/drivers/net/bonding/bond_sysfs.c
>>>>> index 11b76b3..28a2d80 100644
>>>>> --- a/drivers/net/bonding/bond_sysfs.c
>>>>> +++ b/drivers/net/bonding/bond_sysfs.c
>>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
>>>>> *d,
>>>>> 	struct slave *slave;
>>>>> 	struct bonding *bond = to_bond(d);
>>>>>
>>>>> -	write_lock_bh(&bond->lock);
>>>>> +	rtnl_lock();
>>>>> +	read_lock(&bond->lock);
>>>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>>> +
>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>> 		printk(KERN_INFO DRV_NAME
>>>>> 		       ": %s: Unable to set primary slave; %s is in mode
>>>>> 		       %d\n",
>>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
>>>>> *d,
>>>>> 		}
>>>>> 	}
>>>>> out:
>>>>> -	write_unlock_bh(&bond->lock);
>>>>> -
>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>> +	read_unlock(&bond->lock);
>>>>> 	rtnl_unlock();
>>>>>
>>>>> 	return count;
>>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>> device *d,
>>>>> 	struct bonding *bond = to_bond(d);
>>>>>
>>>>> 	rtnl_lock();
>>>>> -	write_lock_bh(&bond->lock);
>>>>> +	read_lock(&bond->lock);
>>>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>>>
>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>> 		printk(KERN_INFO DRV_NAME
>>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>> device *d,
>>>>> 		}
>>>>> 	}
>>>>> out:
>>>>> -	write_unlock_bh(&bond->lock);
>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>> +	read_unlock(&bond->lock);
>>>>> 	rtnl_unlock();
>>>>>
>>>>> 	return count;
>>>>
>>>> Vanilla 2.6.24-rc5 plus this patch:
>>>>
>>>> =========================================================
>>>> [ INFO: possible irq lock inversion dependency detected ]
>>>> 2.6.24-rc5 #1
>>>> ---------------------------------------------------------
>>>> events/0/9 just changed the state of lock:
>>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
>>>> but this lock took another, soft-read-irq-unsafe lock in the past:
>>>> (&bond->lock){-.--}
>>>>
>>>> and interrupts could create inverse lock ordering between them.
>>>>
>>>>
>>>
>>> Grrr, I should have seen that -- sorry.  Try your luck with this instead:
>> <CUT>
>>
>> No luck.
>>
>
>
> I'm guessing if we go back to using a write-lock for bond->lock this
> will go back to working again, but I'm not totally convinced since there
> are plenty of places where we used a read-lock with it.

Should I check this patch or rather, based on a future discussion, wait 
for another version?

>
> diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c
> index 11b76b3..635b857 100644
> --- a/drivers/net/bonding/bond_sysfs.c
> +++ b/drivers/net/bonding/bond_sysfs.c
> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device *d,
> 	struct slave *slave;
> 	struct bonding *bond = to_bond(d);
>
> +	rtnl_lock();
> 	write_lock_bh(&bond->lock);
> +	write_lock_bh(&bond->curr_slave_lock);
> +
> 	if (!USES_PRIMARY(bond->params.mode)) {
> 		printk(KERN_INFO DRV_NAME
> 		       ": %s: Unable to set primary slave; %s is in mode %d\n",
> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device *d,
> 		}
> 	}
> out:
> +	write_unlock_bh(&bond->curr_slave_lock);
> 	write_unlock_bh(&bond->lock);
> -
> 	rtnl_unlock();
>
> 	return count;
> @@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct device *d,
>
> 	rtnl_lock();
> 	write_lock_bh(&bond->lock);
> +	write_lock_bh(&bond->curr_slave_lock);
>
> 	if (!USES_PRIMARY(bond->params.mode)) {
> 		printk(KERN_INFO DRV_NAME
> @@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct device *d,
> 		}
> 	}
> out:
> +	write_unlock_bh(&bond->curr_slave_lock);
> 	write_unlock_bh(&bond->lock);
> 	rtnl_unlock();
>


Best regards,

 					Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-18 19:53               ` Krzysztof Oledzki
@ 2007-12-19 14:42                 ` Andy Gospodarek
  2008-01-07 17:57                   ` Krzysztof Oledzki
  0 siblings, 1 reply; 24+ messages in thread
From: Andy Gospodarek @ 2007-12-19 14:42 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: Andy Gospodarek, Jay Vosburgh, Herbert Xu, Andrew Morton,
	bugme-daemon, shemminger, davem, netdev

On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> 
> >On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
> >>
> >>
> >>On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> >>
> >>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
> >>>>
> >>>>
> >>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote:
> >>>>
> >>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >>>>>
> >>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>>>drivers/net/bonding/bond_sysfs.c
> >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c
> >>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >>>>>>>out:
> >>>>>>>     write_unlock_bh(&bond->lock);
> >>>>>>>
> >>>>>>>-       rtnl_unlock();
> >>>>>>>-
> >>>>>>
> >>>>>>Looking at the changeset that added this perhaps the intention
> >>>>>>is to hold the lock? If so we should add an rtnl_lock to the start
> >>>>>>of the function.
> >>>>>
> >>>>>	Yes, this function needs to hold locks, and more than just
> >>>>>what's there now.  I believe the following should be correct; I haven't
> >>>>>tested it, though (I'm supposedly on vacation right now).
> >>>>>
> >>>>>	The following change should be correct for the
> >>>>>bonding_store_primary case discussed in this thread, and also corrects
> >>>>>the bonding_store_active case which performs similar functions.
> >>>>>
> >>>>>	The bond_change_active_slave and bond_select_active_slave
> >>>>>functions both require rtnl, bond->lock for read and curr_slave_lock 
> >>>>>for
> >>>>>write_bh, and no other locks.  This is so that the lower level
> >>>>>mode-specific functions can release locks down to just rtnl in order to
> >>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
> >>>>>
> >>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
> >>>>>
> >>>>>diff --git a/drivers/net/bonding/bond_sysfs.c
> >>>>>b/drivers/net/bonding/bond_sysfs.c
> >>>>>index 11b76b3..28a2d80 100644
> >>>>>--- a/drivers/net/bonding/bond_sysfs.c
> >>>>>+++ b/drivers/net/bonding/bond_sysfs.c
> >>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct 
> >>>>>device
> >>>>>*d,
> >>>>>	struct slave *slave;
> >>>>>	struct bonding *bond = to_bond(d);
> >>>>>
> >>>>>-	write_lock_bh(&bond->lock);
> >>>>>+	rtnl_lock();
> >>>>>+	read_lock(&bond->lock);
> >>>>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>>>+
> >>>>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>>>		printk(KERN_INFO DRV_NAME
> >>>>>		       ": %s: Unable to set primary slave; %s is in mode
> >>>>>		       %d\n",
> >>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct 
> >>>>>device
> >>>>>*d,
> >>>>>		}
> >>>>>	}
> >>>>>out:
> >>>>>-	write_unlock_bh(&bond->lock);
> >>>>>-
> >>>>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>>>+	read_unlock(&bond->lock);
> >>>>>	rtnl_unlock();
> >>>>>
> >>>>>	return count;
> >>>>>@@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>>>device *d,
> >>>>>	struct bonding *bond = to_bond(d);
> >>>>>
> >>>>>	rtnl_lock();
> >>>>>-	write_lock_bh(&bond->lock);
> >>>>>+	read_lock(&bond->lock);
> >>>>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>>>
> >>>>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>>>		printk(KERN_INFO DRV_NAME
> >>>>>@@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
> >>>>>device *d,
> >>>>>		}
> >>>>>	}
> >>>>>out:
> >>>>>-	write_unlock_bh(&bond->lock);
> >>>>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>>>+	read_unlock(&bond->lock);
> >>>>>	rtnl_unlock();
> >>>>>
> >>>>>	return count;
> >>>>
> >>>>Vanilla 2.6.24-rc5 plus this patch:
> >>>>
> >>>>=========================================================
> >>>>[ INFO: possible irq lock inversion dependency detected ]
> >>>>2.6.24-rc5 #1
> >>>>---------------------------------------------------------
> >>>>events/0/9 just changed the state of lock:
> >>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
> >>>>but this lock took another, soft-read-irq-unsafe lock in the past:
> >>>>(&bond->lock){-.--}
> >>>>
> >>>>and interrupts could create inverse lock ordering between them.
> >>>>
> >>>>
> >>>
> >>>Grrr, I should have seen that -- sorry.  Try your luck with this instead:
> >><CUT>
> >>
> >>No luck.
> >>
> >
> >
> >I'm guessing if we go back to using a write-lock for bond->lock this
> >will go back to working again, but I'm not totally convinced since there
> >are plenty of places where we used a read-lock with it.
> 
> Should I check this patch or rather, based on a future discussion, wait 
> for another version?
> 
> >
> >diff --git a/drivers/net/bonding/bond_sysfs.c 
> >b/drivers/net/bonding/bond_sysfs.c
> >index 11b76b3..635b857 100644
> >--- a/drivers/net/bonding/bond_sysfs.c
> >+++ b/drivers/net/bonding/bond_sysfs.c
> >@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device 
> >*d,
> >	struct slave *slave;
> >	struct bonding *bond = to_bond(d);
> >
> >+	rtnl_lock();
> >	write_lock_bh(&bond->lock);
> >+	write_lock_bh(&bond->curr_slave_lock);
> >+
> >	if (!USES_PRIMARY(bond->params.mode)) {
> >		printk(KERN_INFO DRV_NAME
> >		       ": %s: Unable to set primary slave; %s is in mode 
> >		       %d\n",
> >@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device 
> >*d,
> >		}
> >	}
> >out:
> >+	write_unlock_bh(&bond->curr_slave_lock);
> >	write_unlock_bh(&bond->lock);
> >-
> >	rtnl_unlock();
> >
> >	return count;
> >@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct 
> >device *d,
> >
> >	rtnl_lock();
> >	write_lock_bh(&bond->lock);
> >+	write_lock_bh(&bond->curr_slave_lock);
> >
> >	if (!USES_PRIMARY(bond->params.mode)) {
> >		printk(KERN_INFO DRV_NAME
> >@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct 
> >device *d,
> >		}
> >	}
> >out:
> >+	write_unlock_bh(&bond->curr_slave_lock);
> >	write_unlock_bh(&bond->lock);
> >	rtnl_unlock();
> >
> 
> 
> Best regards,
> 
> 					Krzysztof Olędzki

For now, I prefer Jay's original patch -- with the read_locks (rather
than read/write_lock_bh) and the added rtnl_lock.  There is still a
lockdep issue that we need to sort-out, but this patch is needed first.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2007-12-19 14:42                 ` Andy Gospodarek
@ 2008-01-07 17:57                   ` Krzysztof Oledzki
  2008-01-07 20:26                     ` Andy Gospodarek
  0 siblings, 1 reply; 24+ messages in thread
From: Krzysztof Oledzki @ 2008-01-07 17:57 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev

[-- Attachment #1: Type: TEXT/PLAIN, Size: 6817 bytes --]



On Wed, 19 Dec 2007, Andy Gospodarek wrote:

> On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote:
>>
>>
>> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>>
>>> On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
>>>>
>>>>
>>>> On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>>>>
>>>>> On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
>>>>>>
>>>>>>
>>>>>> On Wed, 12 Dec 2007, Jay Vosburgh wrote:
>>>>>>
>>>>>>> Herbert Xu <herbert@gondor.apana.org.au> wrote:
>>>>>>>
>>>>>>>>> diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>>>> drivers/net/bonding/bond_sysfs.c
>>>>>>>>> --- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>>>>>>>>> +++ a/drivers/net/bonding/bond_sysfs.c
>>>>>>>>> @@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>>>>>>>>> out:
>>>>>>>>>     write_unlock_bh(&bond->lock);
>>>>>>>>>
>>>>>>>>> -       rtnl_unlock();
>>>>>>>>> -
>>>>>>>>
>>>>>>>> Looking at the changeset that added this perhaps the intention
>>>>>>>> is to hold the lock? If so we should add an rtnl_lock to the start
>>>>>>>> of the function.
>>>>>>>
>>>>>>> 	Yes, this function needs to hold locks, and more than just
>>>>>>> what's there now.  I believe the following should be correct; I haven't
>>>>>>> tested it, though (I'm supposedly on vacation right now).
>>>>>>>
>>>>>>> 	The following change should be correct for the
>>>>>>> bonding_store_primary case discussed in this thread, and also corrects
>>>>>>> the bonding_store_active case which performs similar functions.
>>>>>>>
>>>>>>> 	The bond_change_active_slave and bond_select_active_slave
>>>>>>> functions both require rtnl, bond->lock for read and curr_slave_lock
>>>>>>> for
>>>>>>> write_bh, and no other locks.  This is so that the lower level
>>>>>>> mode-specific functions can release locks down to just rtnl in order to
>>>>>>> call, e.g., dev_set_mac_address with the locks it expects (rtnl only).
>>>>>>>
>>>>>>> Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>>>>>>>
>>>>>>> diff --git a/drivers/net/bonding/bond_sysfs.c
>>>>>>> b/drivers/net/bonding/bond_sysfs.c
>>>>>>> index 11b76b3..28a2d80 100644
>>>>>>> --- a/drivers/net/bonding/bond_sysfs.c
>>>>>>> +++ b/drivers/net/bonding/bond_sysfs.c
>>>>>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct
>>>>>>> device
>>>>>>> *d,
>>>>>>> 	struct slave *slave;
>>>>>>> 	struct bonding *bond = to_bond(d);
>>>>>>>
>>>>>>> -	write_lock_bh(&bond->lock);
>>>>>>> +	rtnl_lock();
>>>>>>> +	read_lock(&bond->lock);
>>>>>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>>>>> +
>>>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>>>> 		printk(KERN_INFO DRV_NAME
>>>>>>> 		       ": %s: Unable to set primary slave; %s is in mode
>>>>>>> 		       %d\n",
>>>>>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct
>>>>>>> device
>>>>>>> *d,
>>>>>>> 		}
>>>>>>> 	}
>>>>>>> out:
>>>>>>> -	write_unlock_bh(&bond->lock);
>>>>>>> -
>>>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>>>> +	read_unlock(&bond->lock);
>>>>>>> 	rtnl_unlock();
>>>>>>>
>>>>>>> 	return count;
>>>>>>> @@ -1190,7 +1193,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>>>> device *d,
>>>>>>> 	struct bonding *bond = to_bond(d);
>>>>>>>
>>>>>>> 	rtnl_lock();
>>>>>>> -	write_lock_bh(&bond->lock);
>>>>>>> +	read_lock(&bond->lock);
>>>>>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>>>>>
>>>>>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>>>>>> 		printk(KERN_INFO DRV_NAME
>>>>>>> @@ -1247,7 +1251,8 @@ static ssize_t bonding_store_active_slave(struct
>>>>>>> device *d,
>>>>>>> 		}
>>>>>>> 	}
>>>>>>> out:
>>>>>>> -	write_unlock_bh(&bond->lock);
>>>>>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>>>>>> +	read_unlock(&bond->lock);
>>>>>>> 	rtnl_unlock();
>>>>>>>
>>>>>>> 	return count;
>>>>>>
>>>>>> Vanilla 2.6.24-rc5 plus this patch:
>>>>>>
>>>>>> =========================================================
>>>>>> [ INFO: possible irq lock inversion dependency detected ]
>>>>>> 2.6.24-rc5 #1
>>>>>> ---------------------------------------------------------
>>>>>> events/0/9 just changed the state of lock:
>>>>>> (&mc->mca_lock){-+..}, at: [<c0411c7a>] mld_ifc_timer_expire+0x130/0x1fb
>>>>>> but this lock took another, soft-read-irq-unsafe lock in the past:
>>>>>> (&bond->lock){-.--}
>>>>>>
>>>>>> and interrupts could create inverse lock ordering between them.
>>>>>>
>>>>>>
>>>>>
>>>>> Grrr, I should have seen that -- sorry.  Try your luck with this instead:
>>>> <CUT>
>>>>
>>>> No luck.
>>>>
>>>
>>>
>>> I'm guessing if we go back to using a write-lock for bond->lock this
>>> will go back to working again, but I'm not totally convinced since there
>>> are plenty of places where we used a read-lock with it.
>>
>> Should I check this patch or rather, based on a future discussion, wait
>> for another version?
>>
>>>
>>> diff --git a/drivers/net/bonding/bond_sysfs.c
>>> b/drivers/net/bonding/bond_sysfs.c
>>> index 11b76b3..635b857 100644
>>> --- a/drivers/net/bonding/bond_sysfs.c
>>> +++ b/drivers/net/bonding/bond_sysfs.c
>>> @@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
>>> *d,
>>> 	struct slave *slave;
>>> 	struct bonding *bond = to_bond(d);
>>>
>>> +	rtnl_lock();
>>> 	write_lock_bh(&bond->lock);
>>> +	write_lock_bh(&bond->curr_slave_lock);
>>> +
>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>> 		printk(KERN_INFO DRV_NAME
>>> 		       ": %s: Unable to set primary slave; %s is in mode
>>> 		       %d\n",
>>> @@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
>>> *d,
>>> 		}
>>> 	}
>>> out:
>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>> 	write_unlock_bh(&bond->lock);
>>> -
>>> 	rtnl_unlock();
>>>
>>> 	return count;
>>> @@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct
>>> device *d,
>>>
>>> 	rtnl_lock();
>>> 	write_lock_bh(&bond->lock);
>>> +	write_lock_bh(&bond->curr_slave_lock);
>>>
>>> 	if (!USES_PRIMARY(bond->params.mode)) {
>>> 		printk(KERN_INFO DRV_NAME
>>> @@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct
>>> device *d,
>>> 		}
>>> 	}
>>> out:
>>> +	write_unlock_bh(&bond->curr_slave_lock);
>>> 	write_unlock_bh(&bond->lock);
>>> 	rtnl_unlock();
>>>
>>
>>
>> Best regards,
>>
>> 					Krzysztof Olędzki
>
> For now, I prefer Jay's original patch -- with the read_locks (rather
> than read/write_lock_bh) and the added rtnl_lock.  There is still a
> lockdep issue that we need to sort-out, but this patch is needed first.

This bug has not been fixed yet as it still exists in 2.6.24-rc7. Any 
chances to cure it before 2.6.24-final?

Best regards,

 				Krzysztof Olędzki

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2008-01-07 17:57                   ` Krzysztof Oledzki
@ 2008-01-07 20:26                     ` Andy Gospodarek
  2008-01-07 20:40                       ` Jay Vosburgh
  0 siblings, 1 reply; 24+ messages in thread
From: Andy Gospodarek @ 2008-01-07 20:26 UTC (permalink / raw)
  To: Krzysztof Oledzki
  Cc: Jay Vosburgh, Herbert Xu, Andrew Morton, bugme-daemon, shemminger,
	davem, netdev

On Mon, Jan 07, 2008 at 06:57:25PM +0100, Krzysztof Oledzki wrote:
> 
> 
> On Wed, 19 Dec 2007, Andy Gospodarek wrote:
> 
> >On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote:
> >>
> >>
> >>On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> >>
> >>>On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
> >>>>
> >>>>
> >>>>On Fri, 14 Dec 2007, Andy Gospodarek wrote:
> >>>>
> >>>>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
> >>>>>>
> >>>>>>
> >>>>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote:
> >>>>>>
> >>>>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >>>>>>>
> >>>>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>>>>>drivers/net/bonding/bond_sysfs.c
> >>>>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
> >>>>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c
> >>>>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
> >>>>>>>>>out:
> >>>>>>>>>    write_unlock_bh(&bond->lock);
> >>>>>>>>>
> >>>>>>>>>-       rtnl_unlock();
> >>>>>>>>>-
> >>>>>>>>
> >>>>>>>>Looking at the changeset that added this perhaps the intention
> >>>>>>>>is to hold the lock? If so we should add an rtnl_lock to the start
> >>>>>>>>of the function.
> >>>>>>>
> >>>>>>>	Yes, this function needs to hold locks, and more than just
> >>>>>>>what's there now.  I believe the following should be correct; I 
> >>>>>>>haven't
> >>>>>>>tested it, though (I'm supposedly on vacation right now).
> >>>>>>>
> >>>>>>>	The following change should be correct for the
> >>>>>>>bonding_store_primary case discussed in this thread, and also 
> >>>>>>>corrects
> >>>>>>>the bonding_store_active case which performs similar functions.
> >>>>>>>
> >>>>>>>	The bond_change_active_slave and bond_select_active_slave
> >>>>>>>functions both require rtnl, bond->lock for read and curr_slave_lock
> >>>>>>>for
> >>>>>>>write_bh, and no other locks.  This is so that the lower level
> >>>>>>>mode-specific functions can release locks down to just rtnl in order 
> >>>>>>>to
> >>>>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl 
> >>>>>>>only).
> >>>>>>>
> >>>>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
> >>>>>>>
> >>>>>>>diff --git a/drivers/net/bonding/bond_sysfs.c
> >>>>>>>b/drivers/net/bonding/bond_sysfs.c
> >>>>>>>index 11b76b3..28a2d80 100644
> >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c
> >>>>>>>+++ b/drivers/net/bonding/bond_sysfs.c
> >>>>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct
> >>>>>>>device
> >>>>>>>*d,
> >>>>>>>	struct slave *slave;
> >>>>>>>	struct bonding *bond = to_bond(d);
> >>>>>>>
> >>>>>>>-	write_lock_bh(&bond->lock);
> >>>>>>>+	rtnl_lock();
> >>>>>>>+	read_lock(&bond->lock);
> >>>>>>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>>>>>+
> >>>>>>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>>>>>		printk(KERN_INFO DRV_NAME
> >>>>>>>		       ": %s: Unable to set primary slave; %s is in 
> >>>>>>>		       mode
> >>>>>>>		       %d\n",
> >>>>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct
> >>>>>>>device
> >>>>>>>*d,
> >>>>>>>		}
> >>>>>>>	}
> >>>>>>>out:
> >>>>>>>-	write_unlock_bh(&bond->lock);
> >>>>>>>-
> >>>>>>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>>>>>+	read_unlock(&bond->lock);
> >>>>>>>	rtnl_unlock();
> >>>>>>>
> >>>>>>>	return count;
> >>>>>>>@@ -1190,7 +1193,8 @@ static ssize_t 
> >>>>>>>bonding_store_active_slave(struct
> >>>>>>>device *d,
> >>>>>>>	struct bonding *bond = to_bond(d);
> >>>>>>>
> >>>>>>>	rtnl_lock();
> >>>>>>>-	write_lock_bh(&bond->lock);
> >>>>>>>+	read_lock(&bond->lock);
> >>>>>>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>>>>>
> >>>>>>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>>>>>		printk(KERN_INFO DRV_NAME
> >>>>>>>@@ -1247,7 +1251,8 @@ static ssize_t 
> >>>>>>>bonding_store_active_slave(struct
> >>>>>>>device *d,
> >>>>>>>		}
> >>>>>>>	}
> >>>>>>>out:
> >>>>>>>-	write_unlock_bh(&bond->lock);
> >>>>>>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>>>>>+	read_unlock(&bond->lock);
> >>>>>>>	rtnl_unlock();
> >>>>>>>
> >>>>>>>	return count;
> >>>>>>
> >>>>>>Vanilla 2.6.24-rc5 plus this patch:
> >>>>>>
> >>>>>>=========================================================
> >>>>>>[ INFO: possible irq lock inversion dependency detected ]
> >>>>>>2.6.24-rc5 #1
> >>>>>>---------------------------------------------------------
> >>>>>>events/0/9 just changed the state of lock:
> >>>>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] 
> >>>>>>mld_ifc_timer_expire+0x130/0x1fb
> >>>>>>but this lock took another, soft-read-irq-unsafe lock in the past:
> >>>>>>(&bond->lock){-.--}
> >>>>>>
> >>>>>>and interrupts could create inverse lock ordering between them.
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>Grrr, I should have seen that -- sorry.  Try your luck with this 
> >>>>>instead:
> >>>><CUT>
> >>>>
> >>>>No luck.
> >>>>
> >>>
> >>>
> >>>I'm guessing if we go back to using a write-lock for bond->lock this
> >>>will go back to working again, but I'm not totally convinced since there
> >>>are plenty of places where we used a read-lock with it.
> >>
> >>Should I check this patch or rather, based on a future discussion, wait
> >>for another version?
> >>
> >>>
> >>>diff --git a/drivers/net/bonding/bond_sysfs.c
> >>>b/drivers/net/bonding/bond_sysfs.c
> >>>index 11b76b3..635b857 100644
> >>>--- a/drivers/net/bonding/bond_sysfs.c
> >>>+++ b/drivers/net/bonding/bond_sysfs.c
> >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
> >>>*d,
> >>>	struct slave *slave;
> >>>	struct bonding *bond = to_bond(d);
> >>>
> >>>+	rtnl_lock();
> >>>	write_lock_bh(&bond->lock);
> >>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>+
> >>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>		printk(KERN_INFO DRV_NAME
> >>>		       ": %s: Unable to set primary slave; %s is in mode
> >>>		       %d\n",
> >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
> >>>*d,
> >>>		}
> >>>	}
> >>>out:
> >>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>	write_unlock_bh(&bond->lock);
> >>>-
> >>>	rtnl_unlock();
> >>>
> >>>	return count;
> >>>@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct
> >>>device *d,
> >>>
> >>>	rtnl_lock();
> >>>	write_lock_bh(&bond->lock);
> >>>+	write_lock_bh(&bond->curr_slave_lock);
> >>>
> >>>	if (!USES_PRIMARY(bond->params.mode)) {
> >>>		printk(KERN_INFO DRV_NAME
> >>>@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct
> >>>device *d,
> >>>		}
> >>>	}
> >>>out:
> >>>+	write_unlock_bh(&bond->curr_slave_lock);
> >>>	write_unlock_bh(&bond->lock);
> >>>	rtnl_unlock();
> >>>
> >>
> >>
> >>Best regards,
> >>
> >>					Krzysztof Olędzki
> >
> >For now, I prefer Jay's original patch -- with the read_locks (rather
> >than read/write_lock_bh) and the added rtnl_lock.  There is still a
> >lockdep issue that we need to sort-out, but this patch is needed first.
> 
> This bug has not been fixed yet as it still exists in 2.6.24-rc7. Any 
> chances to cure it before 2.6.24-final?
> 
> Best regards,
> 
> 				Krzysztof Olędzki

Krzysztof,

I doubt the lockdep issue will be fixed, but the patch Jay posted and I
acked needs to be included in 2.6.24.

I played around with the locking when setting the multicast list and I
can make the lockdep issue go away, but I need to be sure that it's OK
to switch it to a read-lock from a write-lock (and I don't really think
it is).

-andy


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055)
  2008-01-07 20:26                     ` Andy Gospodarek
@ 2008-01-07 20:40                       ` Jay Vosburgh
  0 siblings, 0 replies; 24+ messages in thread
From: Jay Vosburgh @ 2008-01-07 20:40 UTC (permalink / raw)
  To: Andy Gospodarek
  Cc: Krzysztof Oledzki, Herbert Xu, Andrew Morton, bugme-daemon,
	shemminger, davem, netdev

Andy Gospodarek <andy@greyhouse.net> wrote:

>On Mon, Jan 07, 2008 at 06:57:25PM +0100, Krzysztof Oledzki wrote:
>> 
>> 
>> On Wed, 19 Dec 2007, Andy Gospodarek wrote:
>> 
>> >On Tue, Dec 18, 2007 at 08:53:39PM +0100, Krzysztof Oledzki wrote:
>> >>
>> >>
>> >>On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>> >>
>> >>>On Fri, Dec 14, 2007 at 07:57:42PM +0100, Krzysztof Oledzki wrote:
>> >>>>
>> >>>>
>> >>>>On Fri, 14 Dec 2007, Andy Gospodarek wrote:
>> >>>>
>> >>>>>On Fri, Dec 14, 2007 at 05:14:57PM +0100, Krzysztof Oledzki wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>>On Wed, 12 Dec 2007, Jay Vosburgh wrote:
>> >>>>>>
>> >>>>>>>Herbert Xu <herbert@gondor.apana.org.au> wrote:
>> >>>>>>>
>> >>>>>>>>>diff -puN drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>> >>>>>>>>>drivers/net/bonding/bond_sysfs.c
>> >>>>>>>>>--- a/drivers/net/bonding/bond_sysfs.c~bonding-locking-fix
>> >>>>>>>>>+++ a/drivers/net/bonding/bond_sysfs.c
>> >>>>>>>>>@@ -1111,8 +1111,6 @@ static ssize_t bonding_store_primary(str
>> >>>>>>>>>out:
>> >>>>>>>>>    write_unlock_bh(&bond->lock);
>> >>>>>>>>>
>> >>>>>>>>>-       rtnl_unlock();
>> >>>>>>>>>-
>> >>>>>>>>
>> >>>>>>>>Looking at the changeset that added this perhaps the intention
>> >>>>>>>>is to hold the lock? If so we should add an rtnl_lock to the start
>> >>>>>>>>of the function.
>> >>>>>>>
>> >>>>>>>	Yes, this function needs to hold locks, and more than just
>> >>>>>>>what's there now.  I believe the following should be correct; I 
>> >>>>>>>haven't
>> >>>>>>>tested it, though (I'm supposedly on vacation right now).
>> >>>>>>>
>> >>>>>>>	The following change should be correct for the
>> >>>>>>>bonding_store_primary case discussed in this thread, and also 
>> >>>>>>>corrects
>> >>>>>>>the bonding_store_active case which performs similar functions.
>> >>>>>>>
>> >>>>>>>	The bond_change_active_slave and bond_select_active_slave
>> >>>>>>>functions both require rtnl, bond->lock for read and curr_slave_lock
>> >>>>>>>for
>> >>>>>>>write_bh, and no other locks.  This is so that the lower level
>> >>>>>>>mode-specific functions can release locks down to just rtnl in order 
>> >>>>>>>to
>> >>>>>>>call, e.g., dev_set_mac_address with the locks it expects (rtnl 
>> >>>>>>>only).
>> >>>>>>>
>> >>>>>>>Signed-off-by: Jay Vosburgh <fubar@us.ibm.com>
>> >>>>>>>
>> >>>>>>>diff --git a/drivers/net/bonding/bond_sysfs.c
>> >>>>>>>b/drivers/net/bonding/bond_sysfs.c
>> >>>>>>>index 11b76b3..28a2d80 100644
>> >>>>>>>--- a/drivers/net/bonding/bond_sysfs.c
>> >>>>>>>+++ b/drivers/net/bonding/bond_sysfs.c
>> >>>>>>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct
>> >>>>>>>device
>> >>>>>>>*d,
>> >>>>>>>	struct slave *slave;
>> >>>>>>>	struct bonding *bond = to_bond(d);
>> >>>>>>>
>> >>>>>>>-	write_lock_bh(&bond->lock);
>> >>>>>>>+	rtnl_lock();
>> >>>>>>>+	read_lock(&bond->lock);
>> >>>>>>>+	write_lock_bh(&bond->curr_slave_lock);
>> >>>>>>>+
>> >>>>>>>	if (!USES_PRIMARY(bond->params.mode)) {
>> >>>>>>>		printk(KERN_INFO DRV_NAME
>> >>>>>>>		       ": %s: Unable to set primary slave; %s is in 
>> >>>>>>>		       mode
>> >>>>>>>		       %d\n",
>> >>>>>>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct
>> >>>>>>>device
>> >>>>>>>*d,
>> >>>>>>>		}
>> >>>>>>>	}
>> >>>>>>>out:
>> >>>>>>>-	write_unlock_bh(&bond->lock);
>> >>>>>>>-
>> >>>>>>>+	write_unlock_bh(&bond->curr_slave_lock);
>> >>>>>>>+	read_unlock(&bond->lock);
>> >>>>>>>	rtnl_unlock();
>> >>>>>>>
>> >>>>>>>	return count;
>> >>>>>>>@@ -1190,7 +1193,8 @@ static ssize_t 
>> >>>>>>>bonding_store_active_slave(struct
>> >>>>>>>device *d,
>> >>>>>>>	struct bonding *bond = to_bond(d);
>> >>>>>>>
>> >>>>>>>	rtnl_lock();
>> >>>>>>>-	write_lock_bh(&bond->lock);
>> >>>>>>>+	read_lock(&bond->lock);
>> >>>>>>>+	write_lock_bh(&bond->curr_slave_lock);
>> >>>>>>>
>> >>>>>>>	if (!USES_PRIMARY(bond->params.mode)) {
>> >>>>>>>		printk(KERN_INFO DRV_NAME
>> >>>>>>>@@ -1247,7 +1251,8 @@ static ssize_t 
>> >>>>>>>bonding_store_active_slave(struct
>> >>>>>>>device *d,
>> >>>>>>>		}
>> >>>>>>>	}
>> >>>>>>>out:
>> >>>>>>>-	write_unlock_bh(&bond->lock);
>> >>>>>>>+	write_unlock_bh(&bond->curr_slave_lock);
>> >>>>>>>+	read_unlock(&bond->lock);
>> >>>>>>>	rtnl_unlock();
>> >>>>>>>
>> >>>>>>>	return count;
>> >>>>>>
>> >>>>>>Vanilla 2.6.24-rc5 plus this patch:
>> >>>>>>
>> >>>>>>=========================================================
>> >>>>>>[ INFO: possible irq lock inversion dependency detected ]
>> >>>>>>2.6.24-rc5 #1
>> >>>>>>---------------------------------------------------------
>> >>>>>>events/0/9 just changed the state of lock:
>> >>>>>>(&mc->mca_lock){-+..}, at: [<c0411c7a>] 
>> >>>>>>mld_ifc_timer_expire+0x130/0x1fb
>> >>>>>>but this lock took another, soft-read-irq-unsafe lock in the past:
>> >>>>>>(&bond->lock){-.--}
>> >>>>>>
>> >>>>>>and interrupts could create inverse lock ordering between them.
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>Grrr, I should have seen that -- sorry.  Try your luck with this 
>> >>>>>instead:
>> >>>><CUT>
>> >>>>
>> >>>>No luck.
>> >>>>
>> >>>
>> >>>
>> >>>I'm guessing if we go back to using a write-lock for bond->lock this
>> >>>will go back to working again, but I'm not totally convinced since there
>> >>>are plenty of places where we used a read-lock with it.
>> >>
>> >>Should I check this patch or rather, based on a future discussion, wait
>> >>for another version?
>> >>
>> >>>
>> >>>diff --git a/drivers/net/bonding/bond_sysfs.c
>> >>>b/drivers/net/bonding/bond_sysfs.c
>> >>>index 11b76b3..635b857 100644
>> >>>--- a/drivers/net/bonding/bond_sysfs.c
>> >>>+++ b/drivers/net/bonding/bond_sysfs.c
>> >>>@@ -1075,7 +1075,10 @@ static ssize_t bonding_store_primary(struct device
>> >>>*d,
>> >>>	struct slave *slave;
>> >>>	struct bonding *bond = to_bond(d);
>> >>>
>> >>>+	rtnl_lock();
>> >>>	write_lock_bh(&bond->lock);
>> >>>+	write_lock_bh(&bond->curr_slave_lock);
>> >>>+
>> >>>	if (!USES_PRIMARY(bond->params.mode)) {
>> >>>		printk(KERN_INFO DRV_NAME
>> >>>		       ": %s: Unable to set primary slave; %s is in mode
>> >>>		       %d\n",
>> >>>@@ -1109,8 +1112,8 @@ static ssize_t bonding_store_primary(struct device
>> >>>*d,
>> >>>		}
>> >>>	}
>> >>>out:
>> >>>+	write_unlock_bh(&bond->curr_slave_lock);
>> >>>	write_unlock_bh(&bond->lock);
>> >>>-
>> >>>	rtnl_unlock();
>> >>>
>> >>>	return count;
>> >>>@@ -1191,6 +1194,7 @@ static ssize_t bonding_store_active_slave(struct
>> >>>device *d,
>> >>>
>> >>>	rtnl_lock();
>> >>>	write_lock_bh(&bond->lock);
>> >>>+	write_lock_bh(&bond->curr_slave_lock);
>> >>>
>> >>>	if (!USES_PRIMARY(bond->params.mode)) {
>> >>>		printk(KERN_INFO DRV_NAME
>> >>>@@ -1247,6 +1251,7 @@ static ssize_t bonding_store_active_slave(struct
>> >>>device *d,
>> >>>		}
>> >>>	}
>> >>>out:
>> >>>+	write_unlock_bh(&bond->curr_slave_lock);
>> >>>	write_unlock_bh(&bond->lock);
>> >>>	rtnl_unlock();
>> >>>
>> >>
>> >>
>> >>Best regards,
>> >>
>> >>					Krzysztof Olędzki
>> >
>> >For now, I prefer Jay's original patch -- with the read_locks (rather
>> >than read/write_lock_bh) and the added rtnl_lock.  There is still a
>> >lockdep issue that we need to sort-out, but this patch is needed first.
>> 
>> This bug has not been fixed yet as it still exists in 2.6.24-rc7. Any 
>> chances to cure it before 2.6.24-final?
>> 
>> Best regards,
>> 
>> 				Krzysztof Olędzki
>
>Krzysztof,
>
>I doubt the lockdep issue will be fixed, but the patch Jay posted and I
>acked needs to be included in 2.6.24.

	I'm (finally) back from vacation and am working on the lock
problem right now; there are a couple of other changes that need to go
in (in addition to what was posted previously).  One is a spurious RTNL
warning, the other is a similar 'wrong lock' type of problem that arises
during module unload.

	I should have a patch set for this posted in a couple of hours.

>I played around with the locking when setting the multicast list and I
>can make the lockdep issue go away, but I need to be sure that it's OK
>to switch it to a read-lock from a write-lock (and I don't really think
>it is).

	I haven't looked at the lockdep problem yet.  If you want to be
brave and post your working patch for the lockdep thing, I might be able
to crush your hopes that it's ok.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2008-01-07 20:40 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <bug-9543-10286@http.bugzilla.kernel.org/>
2007-12-11 11:46 ` [Bugme-new] [Bug 9543] New: RTNL: assertion failed at net/ipv6/addrconf.c (2164)/RTNL: assertion failed at net/ipv4/devinet.c (1055) Andrew Morton
2007-12-11 15:04   ` Krzysztof Oledzki
2007-12-11 20:30     ` Andrew Morton
2007-12-12 13:31   ` Herbert Xu
2007-12-12 17:46     ` Jay Vosburgh
2007-12-12 19:07       ` Andy Gospodarek
2007-12-14 16:14       ` Krzysztof Oledzki
2007-12-14 18:26         ` Andy Gospodarek
2007-12-14 18:57           ` Krzysztof Oledzki
2007-12-14 22:03             ` Andy Gospodarek
2007-12-14 22:11               ` Krzysztof Oledzki
2007-12-14 22:27                 ` Andy Gospodarek
2007-12-18 19:52               ` Krzysztof Oledzki
2007-12-14 22:47             ` Andy Gospodarek
2007-12-15  4:10               ` Herbert Xu
2007-12-15 15:09                 ` Andy Gospodarek
2007-12-16  2:27                   ` Herbert Xu
2007-12-16  3:17                     ` Andy Gospodarek
2007-12-16  3:23                       ` Herbert Xu
2007-12-18 19:53               ` Krzysztof Oledzki
2007-12-19 14:42                 ` Andy Gospodarek
2008-01-07 17:57                   ` Krzysztof Oledzki
2008-01-07 20:26                     ` Andy Gospodarek
2008-01-07 20:40                       ` Jay Vosburgh

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).