[PATCH] lockdep: increase static allocations

[PATCH] lockdep: increase static allocations

[Sasha Levin]

Fuzzing a recent kernel with a large configuration hits the static
allocation limits and disables lockdep.

This patch doubles the limits.

Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
---
 kernel/locking/lockdep_internals.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
index 4f560cf..51c4b24 100644
--- a/kernel/locking/lockdep_internals.h
+++ b/kernel/locking/lockdep_internals.h
@@ -54,9 +54,9 @@ enum {
  * table (if it's not there yet), and we check it for lock order
  * conflicts and deadlocks.
  */
-#define MAX_LOCKDEP_ENTRIES	16384UL
+#define MAX_LOCKDEP_ENTRIES	32768UL
 
-#define MAX_LOCKDEP_CHAINS_BITS	15
+#define MAX_LOCKDEP_CHAINS_BITS	16
 #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
 
 #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
@@ -65,7 +65,7 @@ enum {
  * Stack-trace: tightly packed array of stack backtrace
  * addresses. Protected by the hash_lock.
  */
-#define MAX_STACK_TRACE_ENTRIES	262144UL
+#define MAX_STACK_TRACE_ENTRIES	524288UL
 
 extern struct list_head all_lock_classes;
 extern struct lock_chain lock_chains[];
-- 
1.8.3.2

Re: [PATCH] lockdep: increase static allocations

[Andi Kleen]

Sasha Levin <sasha.levin@oracle.com> writes:

> Fuzzing a recent kernel with a large configuration hits the static
> allocation limits and disables lockdep.

Doesn't that use a lot more memory?  I thought lockdep preallocates.

Doubling may be too aggressive.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only

Re: [PATCH] lockdep: increase static allocations

[Sasha Levin]

On 01/08/2014 02:51 PM, Andi Kleen wrote:
> Sasha Levin <sasha.levin@oracle.com> writes:
>
>> Fuzzing a recent kernel with a large configuration hits the static
>> allocation limits and disables lockdep.
>
> Doesn't that use a lot more memory?  I thought lockdep preallocates.
>
> Doubling may be too aggressive.

The patch adds about 4MB of memory usage, I didn't think it's too much for something
that is only enabled during debugging.

If this is an issue, can I suggest making these values configurable in the .config
and just let users pick whatever they want?


Thanks,
Sasha

Re: [PATCH] lockdep: increase static allocations

[Andi Kleen]

On Wed, Jan 08, 2014 at 03:10:55PM -0500, Sasha Levin wrote:
> On 01/08/2014 02:51 PM, Andi Kleen wrote:
> >Sasha Levin <sasha.levin@oracle.com> writes:
> >
> >>Fuzzing a recent kernel with a large configuration hits the static
> >>allocation limits and disables lockdep.
> >
> >Doesn't that use a lot more memory?  I thought lockdep preallocates.
> >
> >Doubling may be too aggressive.
> 
> The patch adds about 4MB of memory usage, I didn't think it's too much for something
> that is only enabled during debugging.

Wasting 4MB is an issue.

Linus' first Linux system had less total memory than that.

> 
> If this is an issue, can I suggest making these values configurable in the .config
> and just let users pick whatever they want?

Better allocate it at boot time, using a boot parameter or somesuch.

-Andi

Re: [PATCH] lockdep: increase static allocations

[Peter Zijlstra]

On Thu, Jan 09, 2014 at 12:23:22AM +0100, Andi Kleen wrote:
> On Wed, Jan 08, 2014 at 03:10:55PM -0500, Sasha Levin wrote:
> > On 01/08/2014 02:51 PM, Andi Kleen wrote:
> > >Sasha Levin <sasha.levin@oracle.com> writes:
> > >
> > >>Fuzzing a recent kernel with a large configuration hits the static
> > >>allocation limits and disables lockdep.
> > >
> > >Doesn't that use a lot more memory?  I thought lockdep preallocates.
> > >
> > >Doubling may be too aggressive.
> > 
> > The patch adds about 4MB of memory usage, I didn't think it's too much for something
> > that is only enabled during debugging.
> 
> Wasting 4MB is an issue.
> 
> Linus' first Linux system had less total memory than that.

Meh.. if someone from the embedded people care we can add
CONFIG_BASE_SMALL option, until then I couldn't care less about 4m.

> > 
> > If this is an issue, can I suggest making these values configurable in the .config
> > and just let users pick whatever they want?
> 
> Better allocate it at boot time, using a boot parameter or somesuch.

wrongbot andi is at it again.. no that's far too late.

Re: [PATCH] lockdep: increase static allocations

[Sasha Levin]

On 01/08/2014 02:21 PM, Sasha Levin wrote:
> Fuzzing a recent kernel with a large configuration hits the static
> allocation limits and disables lockdep.
>
> This patch doubles the limits.
>
> Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
> ---
>   kernel/locking/lockdep_internals.h | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
> index 4f560cf..51c4b24 100644
> --- a/kernel/locking/lockdep_internals.h
> +++ b/kernel/locking/lockdep_internals.h
> @@ -54,9 +54,9 @@ enum {
>    * table (if it's not there yet), and we check it for lock order
>    * conflicts and deadlocks.
>    */
> -#define MAX_LOCKDEP_ENTRIES	16384UL
> +#define MAX_LOCKDEP_ENTRIES	32768UL
>
> -#define MAX_LOCKDEP_CHAINS_BITS	15
> +#define MAX_LOCKDEP_CHAINS_BITS	16
>   #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
>
>   #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
> @@ -65,7 +65,7 @@ enum {
>    * Stack-trace: tightly packed array of stack backtrace
>    * addresses. Protected by the hash_lock.
>    */
> -#define MAX_STACK_TRACE_ENTRIES	262144UL
> +#define MAX_STACK_TRACE_ENTRIES	524288UL
>
>   extern struct list_head all_lock_classes;
>   extern struct lock_chain lock_chains[];
>


Can someone pick the patch up please? PeterZ even (seemed to) acked it.


Thanks,
Sasha

Re: [PATCH] lockdep: increase static allocations

[Mike Galbraith]

On Fri, 2014-02-28 at 14:32 -0500, Sasha Levin wrote: 
> On 01/08/2014 02:21 PM, Sasha Levin wrote:
> > Fuzzing a recent kernel with a large configuration hits the static
> > allocation limits and disables lockdep.
> >
> > This patch doubles the limits.
> >
> > Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
> > ---
> >   kernel/locking/lockdep_internals.h | 6 +++---
> >   1 file changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
> > index 4f560cf..51c4b24 100644
> > --- a/kernel/locking/lockdep_internals.h
> > +++ b/kernel/locking/lockdep_internals.h
> > @@ -54,9 +54,9 @@ enum {
> >    * table (if it's not there yet), and we check it for lock order
> >    * conflicts and deadlocks.
> >    */
> > -#define MAX_LOCKDEP_ENTRIES	16384UL
> > +#define MAX_LOCKDEP_ENTRIES	32768UL
> >
> > -#define MAX_LOCKDEP_CHAINS_BITS	15
> > +#define MAX_LOCKDEP_CHAINS_BITS	16
> >   #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
> >
> >   #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
> > @@ -65,7 +65,7 @@ enum {
> >    * Stack-trace: tightly packed array of stack backtrace
> >    * addresses. Protected by the hash_lock.
> >    */
> > -#define MAX_STACK_TRACE_ENTRIES	262144UL
> > +#define MAX_STACK_TRACE_ENTRIES	524288UL
> >
> >   extern struct list_head all_lock_classes;
> >   extern struct lock_chain lock_chains[];
> >
> 
> 
> Can someone pick the patch up please? PeterZ even (seemed to) acked it.

I have to do the (exact) same to rt trees, else lockdep routinely gets
in a snit, takes it's cool toys and goes home.

-Mike

Re: [PATCH] lockdep: increase static allocations

[Sasha Levin]

On 02/28/2014 02:32 PM, Sasha Levin wrote:
> On 01/08/2014 02:21 PM, Sasha Levin wrote:
>> Fuzzing a recent kernel with a large configuration hits the static
>> allocation limits and disables lockdep.
>>
>> This patch doubles the limits.
>>
>> Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
>> ---
>>   kernel/locking/lockdep_internals.h | 6 +++---
>>   1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
>> index 4f560cf..51c4b24 100644
>> --- a/kernel/locking/lockdep_internals.h
>> +++ b/kernel/locking/lockdep_internals.h
>> @@ -54,9 +54,9 @@ enum {
>>    * table (if it's not there yet), and we check it for lock order
>>    * conflicts and deadlocks.
>>    */
>> -#define MAX_LOCKDEP_ENTRIES    16384UL
>> +#define MAX_LOCKDEP_ENTRIES    32768UL
>>
>> -#define MAX_LOCKDEP_CHAINS_BITS    15
>> +#define MAX_LOCKDEP_CHAINS_BITS    16
>>   #define MAX_LOCKDEP_CHAINS    (1UL << MAX_LOCKDEP_CHAINS_BITS)
>>
>>   #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
>> @@ -65,7 +65,7 @@ enum {
>>    * Stack-trace: tightly packed array of stack backtrace
>>    * addresses. Protected by the hash_lock.
>>    */
>> -#define MAX_STACK_TRACE_ENTRIES    262144UL
>> +#define MAX_STACK_TRACE_ENTRIES    524288UL
>>
>>   extern struct list_head all_lock_classes;
>>   extern struct lock_chain lock_chains[];
>>
> 
> 
> Can someone pick the patch up please? PeterZ even (seemed to) acked it.

Pretty please? A few people reported that they do what the patch does
to get lockdep working right for them, and there were no objections to
the patch.


Thanks,
Sasha

Re: [PATCH] lockdep: increase static allocations

[Peter Zijlstra]

> 
> Pretty please? A few people reported that they do what the patch does
> to get lockdep working right for them, and there were no objections to
> the patch.

Thanks for the reminder!, got it queued.

[tip:locking/core] lockdep: Increase static allocations

[tip-bot for Sasha Levin]

Commit-ID:  1413c03893332366e5b4d1e26f942ada25f3e82a
Gitweb:     http://git.kernel.org/tip/1413c03893332366e5b4d1e26f942ada25f3e82a
Author:     Sasha Levin <sasha.levin@oracle.com>
AuthorDate: Wed, 8 Jan 2014 14:21:46 -0500
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 18 Apr 2014 14:20:50 +0200

lockdep: Increase static allocations

Fuzzing a recent kernel with a large configuration hits the static
allocation limits and disables lockdep.

This patch doubles the limits.

Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1389208906-24338-1-git-send-email-sasha.levin@oracle.com
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 kernel/locking/lockdep_internals.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
index 4f560cf..51c4b24 100644
--- a/kernel/locking/lockdep_internals.h
+++ b/kernel/locking/lockdep_internals.h
@@ -54,9 +54,9 @@ enum {
  * table (if it's not there yet), and we check it for lock order
  * conflicts and deadlocks.
  */
-#define MAX_LOCKDEP_ENTRIES	16384UL
+#define MAX_LOCKDEP_ENTRIES	32768UL
 
-#define MAX_LOCKDEP_CHAINS_BITS	15
+#define MAX_LOCKDEP_CHAINS_BITS	16
 #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
 
 #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
@@ -65,7 +65,7 @@ enum {
  * Stack-trace: tightly packed array of stack backtrace
  * addresses. Protected by the hash_lock.
  */
-#define MAX_STACK_TRACE_ENTRIES	262144UL
+#define MAX_STACK_TRACE_ENTRIES	524288UL
 
 extern struct list_head all_lock_classes;
 extern struct lock_chain lock_chains[];

Re: [PATCH] lockdep: increase static allocations

[James Hogan]

Hi Peter,

On 9 January 2014 at 08:56, Peter Zijlstra <peterz@infradead.org> wrote:
> On Thu, Jan 09, 2014 at 12:23:22AM +0100, Andi Kleen wrote:
>> On Wed, Jan 08, 2014 at 03:10:55PM -0500, Sasha Levin wrote:
>> > On 01/08/2014 02:51 PM, Andi Kleen wrote:
>> > >Sasha Levin <sasha.levin@oracle.com> writes:
>> > >
>> > >>Fuzzing a recent kernel with a large configuration hits the static
>> > >>allocation limits and disables lockdep.
>> > >
>> > >Doesn't that use a lot more memory?  I thought lockdep preallocates.
>> > >
>> > >Doubling may be too aggressive.
>> >
>> > The patch adds about 4MB of memory usage, I didn't think it's too much for something
>> > that is only enabled during debugging.
>>
>> Wasting 4MB is an issue.
>>
>> Linus' first Linux system had less total memory than that.
>
> Meh.. if someone from the embedded people care we can add
> CONFIG_BASE_SMALL option, until then I couldn't care less about 4m.

FYI, this could also be a problem for systems with a 16MB DMA zone for
certain devices, since 4M more static kernel data can easily consume
what is left of that zone, even on a system with much more memory. Is
that likely to be a problem on PCs? (especially since PROVE_RCU is
tied to PROVE_LOCKING in linux-next)

(I hit this on MIPS Malta after Paul McKenney's patch in linux-next to
tie PROVE_RCU to PROVE_LOCKING, but luckily the devices in question
actually had 32-bit DMA masks, so the MIPS arch code shouldn't have
been using the __GFP_DMA flag in the first place).

Cheers
James

>
>> >
>> > If this is an issue, can I suggest making these values configurable in the .config
>> > and just let users pick whatever they want?
>>
>> Better allocate it at boot time, using a boot parameter or somesuch.
>
> wrongbot andi is at it again.. no that's far too late.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/



-- 
James Hogan