public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH] LOCKDEP: fix mismatched lockdep_depth/curr_chain_hash
@ 2007-10-05  4:03 Gregory Haskins
  2007-10-05  9:31 ` [PATCH] lockdep: " Peter Zijlstra
  0 siblings, 1 reply; 10+ messages in thread
From: Gregory Haskins @ 2007-10-05  4:03 UTC (permalink / raw)
  To: mingo; +Cc: linux-kernel, linux-rt-users, ghaskins

Hi Ingo,
  I am seeing a problem on the latest -rt where lockdep completely overwhelms
  the system to the point that it grinds to a halt on large (8-way+) systems.
  The problem seems to be that the class->locks_before and locks_after grow
  unbounded (I have observed over 1M+ entries in them) so a lock_acquire call
  can take over 10 seconds to finish resolving.  Related to this seems to be
  that lockdep appears to see a chain-hash miss over and over for what I would
  assume should be an established graph (for instance, in
  double_lock_balance() in an rt_overload condition).  Turning off
  PROVE_LOCKING (statically, or by setting debug_locks=0 dynamically restores
  the system to normal behavior.

  I took some time tonight to study lockdep (it is quite an impressive body of
  code!), and came up with the following "fix".  It does improve things
  significantly by addressing what I believe is the issue with the
  cache-misses (though it would appear there are still a few more issues
  there that need addressing as some boots are still very lethargic).  I use 
  the term "fix" loosely since I am not confident that I fully understand the
  intention of your logic here so I can't say for sure if it was really
  broken, or if I have made it worse ;)

  Could you comment on what I have done here, or offer any advice on what to
  look for elsewhere?  I based the patch on pure linux-2.6.git since I see the
  same issue (by visual inspection, that is) there as well.

  Thanks in advance!
  -Greg   

------

LOCKDEP: fix mismatched lockdep_depth/curr_chain_hash

It is possible for the current->curr_chain_key to become inconsistent with the
current index if the chain fails to validate.  The end result is that future
lock_acquire() operations may inadvertently fail to find a hit in the cache
resulting in a new node being added to the graph for every acquire.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
---

 kernel/lockdep.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/kernel/lockdep.c b/kernel/lockdep.c
index 734da57..efb0d7e 100644
--- a/kernel/lockdep.c
+++ b/kernel/lockdep.c
@@ -2450,11 +2450,11 @@ static int __lock_acquire(struct lockdep_map *lock, unsigned int subclass,
 		chain_head = 1;
 	}
 	chain_key = iterate_chain_key(chain_key, id);
-	curr->curr_chain_key = chain_key;
 
 	if (!validate_chain(curr, lock, hlock, chain_head))
 		return 0;
 
+	curr->curr_chain_key = chain_key;
 	curr->lockdep_depth++;
 	check_chain_key(curr);
 #ifdef CONFIG_DEBUG_LOCKDEP


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-05  4:03 [PATCH] LOCKDEP: fix mismatched lockdep_depth/curr_chain_hash Gregory Haskins
@ 2007-10-05  9:31 ` Peter Zijlstra
  2007-10-08 17:24   ` [stable] " Greg KH
  0 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2007-10-05  9:31 UTC (permalink / raw)
  To: Gregory Haskins, stable; +Cc: mingo, linux-kernel


Stable team,

please consider this patch for the next 22-stable.

---
Subject: lockdep: fix mismatched lockdep_depth/curr_chain_hash
From: Gregory Haskins <ghaskins@novell.com>

It is possible for the current->curr_chain_key to become inconsistent with the
current index if the chain fails to validate.  The end result is that future
lock_acquire() operations may inadvertently fail to find a hit in the cache
resulting in a new node being added to the graph for every acquire.

[ peterz: this might explain some of the lockdep is so _slow_ complaints. ]
[ mingo: this does not impact the correctness of validation, but may slow
  down future operations significantly, if the chain gets very long. ]

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 kernel/lockdep.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6/kernel/lockdep.c
===================================================================
--- linux-2.6.orig/kernel/lockdep.c
+++ linux-2.6/kernel/lockdep.c
@@ -2166,7 +2166,6 @@ out_calc_hash:
 	}
 #endif
 	chain_key = iterate_chain_key(chain_key, id);
-	curr->curr_chain_key = chain_key;
 
 	/*
 	 * Trylock needs to maintain the stack of held locks, but it
@@ -2215,6 +2214,7 @@ out_calc_hash:
 		if (unlikely(!debug_locks))
 			return 0;
 
+	curr->curr_chain_key = chain_key;
 	curr->lockdep_depth++;
 	check_chain_key(curr);
 #ifdef CONFIG_DEBUG_LOCKDEP



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-05  9:31 ` [PATCH] lockdep: " Peter Zijlstra
@ 2007-10-08 17:24   ` Greg KH
  2007-10-08 17:36     ` Peter Zijlstra
  0 siblings, 1 reply; 10+ messages in thread
From: Greg KH @ 2007-10-08 17:24 UTC (permalink / raw)
  To: Peter Zijlstra; +Cc: Gregory Haskins, stable, mingo, linux-kernel

On Fri, Oct 05, 2007 at 11:31:26AM +0200, Peter Zijlstra wrote:
> 
> Stable team,
> 
> please consider this patch for the next 22-stable.

I don't see this patch in Linus's upstream tree.  We need it there to be
able to accept it for -stable.  Or is this just a bugfix of other things
that are already in his tree?

thanks,

greg k-h


> 
> ---
> Subject: lockdep: fix mismatched lockdep_depth/curr_chain_hash
> From: Gregory Haskins <ghaskins@novell.com>
> 
> It is possible for the current->curr_chain_key to become inconsistent with the
> current index if the chain fails to validate.  The end result is that future
> lock_acquire() operations may inadvertently fail to find a hit in the cache
> resulting in a new node being added to the graph for every acquire.
> 
> [ peterz: this might explain some of the lockdep is so _slow_ complaints. ]
> [ mingo: this does not impact the correctness of validation, but may slow
>   down future operations significantly, if the chain gets very long. ]
> 
> Signed-off-by: Gregory Haskins <ghaskins@novell.com>
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Signed-off-by: Ingo Molnar <mingo@elte.hu>
> ---
>  kernel/lockdep.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> Index: linux-2.6/kernel/lockdep.c
> ===================================================================
> --- linux-2.6.orig/kernel/lockdep.c
> +++ linux-2.6/kernel/lockdep.c
> @@ -2166,7 +2166,6 @@ out_calc_hash:
>  	}
>  #endif
>  	chain_key = iterate_chain_key(chain_key, id);
> -	curr->curr_chain_key = chain_key;
>  
>  	/*
>  	 * Trylock needs to maintain the stack of held locks, but it
> @@ -2215,6 +2214,7 @@ out_calc_hash:
>  		if (unlikely(!debug_locks))
>  			return 0;
>  
> +	curr->curr_chain_key = chain_key;
>  	curr->lockdep_depth++;
>  	check_chain_key(curr);
>  #ifdef CONFIG_DEBUG_LOCKDEP
> 
> 
> _______________________________________________
> stable mailing list
> stable@linux.kernel.org
> http://linux.kernel.org/mailman/listinfo/stable

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-08 17:24   ` [stable] " Greg KH
@ 2007-10-08 17:36     ` Peter Zijlstra
  2007-10-08 17:39       ` Greg KH
  0 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2007-10-08 17:36 UTC (permalink / raw)
  To: Greg KH; +Cc: Gregory Haskins, stable, mingo, linux-kernel


On Mon, 2007-10-08 at 10:24 -0700, Greg KH wrote:
> On Fri, Oct 05, 2007 at 11:31:26AM +0200, Peter Zijlstra wrote:
> > 
> > Stable team,
> > 
> > please consider this patch for the next 22-stable.
> 
> I don't see this patch in Linus's upstream tree.  We need it there to be
> able to accept it for -stable.  Or is this just a bugfix of other things
> that are already in his tree?

I send Linus a similar patch, haven´t seem him pick it up yet.
I´ll notify you when and if he picks it up.




^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-08 17:36     ` Peter Zijlstra
@ 2007-10-08 17:39       ` Greg KH
  2007-10-25 17:48         ` Peter Zijlstra
  0 siblings, 1 reply; 10+ messages in thread
From: Greg KH @ 2007-10-08 17:39 UTC (permalink / raw)
  To: Peter Zijlstra; +Cc: Gregory Haskins, stable, mingo, linux-kernel

On Mon, Oct 08, 2007 at 07:36:10PM +0200, Peter Zijlstra wrote:
> 
> On Mon, 2007-10-08 at 10:24 -0700, Greg KH wrote:
> > On Fri, Oct 05, 2007 at 11:31:26AM +0200, Peter Zijlstra wrote:
> > > 
> > > Stable team,
> > > 
> > > please consider this patch for the next 22-stable.
> > 
> > I don't see this patch in Linus's upstream tree.  We need it there to be
> > able to accept it for -stable.  Or is this just a bugfix of other things
> > that are already in his tree?
> 
> I send Linus a similar patch, haven??t seem him pick it up yet.
> I??ll notify you when and if he picks it up.

Great, that would be great for us -stable monkeys...

greg k-h

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-08 17:39       ` Greg KH
@ 2007-10-25 17:48         ` Peter Zijlstra
  2007-10-25 18:55           ` Chuck Ebbert
  0 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2007-10-25 17:48 UTC (permalink / raw)
  To: Greg KH; +Cc: Gregory Haskins, stable, mingo, linux-kernel

On Mon, 2007-10-08 at 10:39 -0700, Greg KH wrote:
> On Mon, Oct 08, 2007 at 07:36:10PM +0200, Peter Zijlstra wrote:
> > 
> > On Mon, 2007-10-08 at 10:24 -0700, Greg KH wrote:
> > > On Fri, Oct 05, 2007 at 11:31:26AM +0200, Peter Zijlstra wrote:
> > > > 
> > > > Stable team,
> > > > 
> > > > please consider this patch for the next 22-stable.
> > > 
> > > I don't see this patch in Linus's upstream tree.  We need it there to be
> > > able to accept it for -stable.  Or is this just a bugfix of other things
> > > that are already in his tree?
> > 
> > I send Linus a similar patch, haven??t seem him pick it up yet.
> > I??ll notify you when and if he picks it up.
> 
> Great, that would be great for us -stable monkeys...

3aa416b07f0adf01c090baab26fb70c35ec17623



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-25 17:48         ` Peter Zijlstra
@ 2007-10-25 18:55           ` Chuck Ebbert
  2007-10-31 14:37             ` Greg KH
  0 siblings, 1 reply; 10+ messages in thread
From: Chuck Ebbert @ 2007-10-25 18:55 UTC (permalink / raw)
  To: Peter Zijlstra; +Cc: Greg KH, Gregory Haskins, stable, mingo, linux-kernel

On 10/25/2007 01:48 PM, Peter Zijlstra wrote:
> On Mon, 2007-10-08 at 10:39 -0700, Greg KH wrote:
>> On Mon, Oct 08, 2007 at 07:36:10PM +0200, Peter Zijlstra wrote:
>>> On Mon, 2007-10-08 at 10:24 -0700, Greg KH wrote:
>>>> On Fri, Oct 05, 2007 at 11:31:26AM +0200, Peter Zijlstra wrote:
>>>>> Stable team,
>>>>>
>>>>> please consider this patch for the next 22-stable.
>>>> I don't see this patch in Linus's upstream tree.  We need it there to be
>>>> able to accept it for -stable.  Or is this just a bugfix of other things
>>>> that are already in his tree?
>>> I send Linus a similar patch, haven??t seem him pick it up yet.
>>> I??ll notify you when and if he picks it up.
>> Great, that would be great for us -stable monkeys...
> 
> 3aa416b07f0adf01c090baab26fb70c35ec17623
> 
> 

And for both 2.6.22 and 2.6.23...


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-25 18:55           ` Chuck Ebbert
@ 2007-10-31 14:37             ` Greg KH
  0 siblings, 0 replies; 10+ messages in thread
From: Greg KH @ 2007-10-31 14:37 UTC (permalink / raw)
  To: Chuck Ebbert; +Cc: Peter Zijlstra, Gregory Haskins, stable, mingo, linux-kernel

On Thu, Oct 25, 2007 at 02:55:05PM -0400, Chuck Ebbert wrote:
> On 10/25/2007 01:48 PM, Peter Zijlstra wrote:
> > On Mon, 2007-10-08 at 10:39 -0700, Greg KH wrote:
> >> On Mon, Oct 08, 2007 at 07:36:10PM +0200, Peter Zijlstra wrote:
> >>> On Mon, 2007-10-08 at 10:24 -0700, Greg KH wrote:
> >>>> On Fri, Oct 05, 2007 at 11:31:26AM +0200, Peter Zijlstra wrote:
> >>>>> Stable team,
> >>>>>
> >>>>> please consider this patch for the next 22-stable.
> >>>> I don't see this patch in Linus's upstream tree.  We need it there to be
> >>>> able to accept it for -stable.  Or is this just a bugfix of other things
> >>>> that are already in his tree?
> >>> I send Linus a similar patch, haven??t seem him pick it up yet.
> >>> I??ll notify you when and if he picks it up.
> >> Great, that would be great for us -stable monkeys...
> > 
> > 3aa416b07f0adf01c090baab26fb70c35ec17623
> > 
> > 
> 
> And for both 2.6.22 and 2.6.23...

It does not apply to 2.6.22 at all, so unless someone sends us a
backported version, I'll not apply it there.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
@ 2007-10-31 15:03 Gregory Haskins
  0 siblings, 0 replies; 10+ messages in thread
From: Gregory Haskins @ 2007-10-31 15:03 UTC (permalink / raw)
  To: greg, cebbert; +Cc: mingo, peterz, stable, linux-kernel

>>> Greg KH <greg@kroah.com> 10/31/07 10:37 AM >>>
>It does not apply to 2.6.22 at all, so unless someone sends us a
>backported version, I'll not apply it there.

Ill take care of this for you, Greg.

-Greg


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [stable] [PATCH] lockdep: fix mismatched lockdep_depth/curr_chain_hash
  2007-10-31 15:44 Gregory Haskins
@ 2007-10-31 16:24 ` Greg KH
  0 siblings, 0 replies; 10+ messages in thread
From: Greg KH @ 2007-10-31 16:24 UTC (permalink / raw)
  To: Gregory Haskins; +Cc: Peter Zijlstra, linux-kernel, Chuck Ebbert, mingo, stable

On Wed, Oct 31, 2007 at 11:44:05AM -0400, Gregory Haskins wrote:
> Hi Greg,
>    Here is the backported version of the patch.  I applied it on top of
>    2.6.22.10.  Let me know if you have any issues.

Looks good, thanks.  I've added it to the -stable queue and will add it
to the current 2.6.22-stable release.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2007-10-31 16:26 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-10-05  4:03 [PATCH] LOCKDEP: fix mismatched lockdep_depth/curr_chain_hash Gregory Haskins
2007-10-05  9:31 ` [PATCH] lockdep: " Peter Zijlstra
2007-10-08 17:24   ` [stable] " Greg KH
2007-10-08 17:36     ` Peter Zijlstra
2007-10-08 17:39       ` Greg KH
2007-10-25 17:48         ` Peter Zijlstra
2007-10-25 18:55           ` Chuck Ebbert
2007-10-31 14:37             ` Greg KH
  -- strict thread matches above, loose matches on Subject: below --
2007-10-31 15:03 Gregory Haskins
2007-10-31 15:44 Gregory Haskins
2007-10-31 16:24 ` [stable] " Greg KH

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox