public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3 0/3] lockdep: minor config and documentation fixes
@ 2024-10-24 18:36 Carlos Llamas
  2024-10-24 18:36 ` [PATCH v3 1/3] lockdep: fix upper limit for LOCKDEP_*_BITS configs Carlos Llamas
                   ` (3 more replies)
  0 siblings, 4 replies; 11+ messages in thread
From: Carlos Llamas @ 2024-10-24 18:36 UTC (permalink / raw)
  To: Boqun Feng, Andrew Morton
  Cc: linux-kernel, kernel-team, Carlos Llamas, Huang Ying,
	J. R. Okajima, Peter Zijlstra, Ingo Molnar, Waiman Long,
	Will Deacon

These are some minor follow-up patches that came up during conversation
at: https://lore.kernel.org/all/30795.1620913191@jrobl/

This v3 is based on master as commit e0ba72e3a442 ("lockdep: upper limit
LOCKDEP_CHAINS_BITS") has now landed.

Cc: Huang Ying <ying.huang@intel.com>
Cc: J. R. Okajima <hooanon05g@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>

v3: rebased on master and collected tags
v2: https://lore.kernel.org/all/20240807143922.919604-1-cmllamas@google.com/
v1: https://lore.kernel.org/all/20240806010128.402852-1-cmllamas@google.com/

Carlos Llamas (3):
  lockdep: fix upper limit for LOCKDEP_*_BITS configs
  lockdep: clarify size for LOCKDEP_*_BITS configs
  lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation

 kernel/locking/lockdep_internals.h |  3 ++-
 lib/Kconfig.debug                  | 18 +++++++++---------
 2 files changed, 11 insertions(+), 10 deletions(-)

-- 
2.47.0.163.g1226f6d8fa-goog


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 1/3] lockdep: fix upper limit for LOCKDEP_*_BITS configs
  2024-10-24 18:36 [PATCH v3 0/3] lockdep: minor config and documentation fixes Carlos Llamas
@ 2024-10-24 18:36 ` Carlos Llamas
  2024-12-24 18:53   ` [tip: locking/core] lockdep: Fix " tip-bot2 for Carlos Llamas
  2024-10-24 18:36 ` [PATCH v3 2/3] lockdep: clarify size " Carlos Llamas
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 11+ messages in thread
From: Carlos Llamas @ 2024-10-24 18:36 UTC (permalink / raw)
  To: Boqun Feng, Andrew Morton
  Cc: linux-kernel, kernel-team, Carlos Llamas, J. R. Okajima,
	Peter Zijlstra, Ingo Molnar, Waiman Long, Will Deacon

Lockdep has a set of configs used to determine the size of the static
arrays that it uses. However, the upper limit that was initially setup
for these configs is too high (30 bit shift). This equates to several
GiB of static memory for individual symbols. Using such high values
leads to linker errors:

  $ make defconfig
  $ ./scripts/config -e PROVE_LOCKING --set-val LOCKDEP_BITS 30
  $ make olddefconfig all
  [...]
  ld: kernel image bigger than KERNEL_IMAGE_SIZE
  ld: section .bss VMA wraps around address space

Adjust the upper limits to the maximum values that avoid these issues.
The need for anything more, likely points to a problem elsewhere. Note
that LOCKDEP_CHAINS_BITS was intentionally left out as its upper limit
had a different symptom and has already been fixed [1].

Reported-by: J. R. Okajima <hooanon05g@gmail.com>
Closes: https://lore.kernel.org/all/30795.1620913191@jrobl/ [1]
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Acked-by: Waiman Long <longman@redhat.com>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
---
 lib/Kconfig.debug | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7312ae7c3cc5..c8b1db37abe6 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1524,7 +1524,7 @@ config LOCKDEP_SMALL
 config LOCKDEP_BITS
 	int "Bitsize for MAX_LOCKDEP_ENTRIES"
 	depends on LOCKDEP && !LOCKDEP_SMALL
-	range 10 30
+	range 10 24
 	default 15
 	help
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_ENTRIES too low!" message.
@@ -1540,7 +1540,7 @@ config LOCKDEP_CHAINS_BITS
 config LOCKDEP_STACK_TRACE_BITS
 	int "Bitsize for MAX_STACK_TRACE_ENTRIES"
 	depends on LOCKDEP && !LOCKDEP_SMALL
-	range 10 30
+	range 10 26
 	default 19
 	help
 	  Try increasing this value if you hit "BUG: MAX_STACK_TRACE_ENTRIES too low!" message.
@@ -1548,7 +1548,7 @@ config LOCKDEP_STACK_TRACE_BITS
 config LOCKDEP_STACK_TRACE_HASH_BITS
 	int "Bitsize for STACK_TRACE_HASH_SIZE"
 	depends on LOCKDEP && !LOCKDEP_SMALL
-	range 10 30
+	range 10 26
 	default 14
 	help
 	  Try increasing this value if you need large STACK_TRACE_HASH_SIZE.
@@ -1556,7 +1556,7 @@ config LOCKDEP_STACK_TRACE_HASH_BITS
 config LOCKDEP_CIRCULAR_QUEUE_BITS
 	int "Bitsize for elements in circular_queue struct"
 	depends on LOCKDEP
-	range 10 30
+	range 10 26
 	default 12
 	help
 	  Try increasing this value if you hit "lockdep bfs error:-1" warning due to __cq_enqueue() failure.
-- 
2.47.0.163.g1226f6d8fa-goog


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v3 2/3] lockdep: clarify size for LOCKDEP_*_BITS configs
  2024-10-24 18:36 [PATCH v3 0/3] lockdep: minor config and documentation fixes Carlos Llamas
  2024-10-24 18:36 ` [PATCH v3 1/3] lockdep: fix upper limit for LOCKDEP_*_BITS configs Carlos Llamas
@ 2024-10-24 18:36 ` Carlos Llamas
  2024-10-24 19:00   ` Waiman Long
  2024-12-24 18:53   ` [tip: locking/core] lockdep: Clarify " tip-bot2 for Carlos Llamas
  2024-10-24 18:36 ` [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation Carlos Llamas
  2024-10-28 17:34 ` [PATCH v3 0/3] lockdep: minor config and documentation fixes Boqun Feng
  3 siblings, 2 replies; 11+ messages in thread
From: Carlos Llamas @ 2024-10-24 18:36 UTC (permalink / raw)
  To: Boqun Feng, Andrew Morton
  Cc: linux-kernel, kernel-team, Carlos Llamas, Peter Zijlstra,
	Ingo Molnar, Waiman Long, Will Deacon

The LOCKDEP_*_BITS configs control the size of internal structures used
by lockdep. The size is calculated as a power of two of the configured
value (e.g. 16 => 64KB). Update these descriptions to more accurately
reflect this, as "Bitsize" can be misleading.

Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
---
 lib/Kconfig.debug | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c8b1db37abe6..5a769cbf4bb2 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1522,7 +1522,7 @@ config LOCKDEP_SMALL
 	bool
 
 config LOCKDEP_BITS
-	int "Bitsize for MAX_LOCKDEP_ENTRIES"
+	int "Size for MAX_LOCKDEP_ENTRIES (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 24
 	default 15
@@ -1530,7 +1530,7 @@ config LOCKDEP_BITS
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_ENTRIES too low!" message.
 
 config LOCKDEP_CHAINS_BITS
-	int "Bitsize for MAX_LOCKDEP_CHAINS"
+	int "Size for MAX_LOCKDEP_CHAINS (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 21
 	default 16
@@ -1538,7 +1538,7 @@ config LOCKDEP_CHAINS_BITS
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message.
 
 config LOCKDEP_STACK_TRACE_BITS
-	int "Bitsize for MAX_STACK_TRACE_ENTRIES"
+	int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 26
 	default 19
@@ -1546,7 +1546,7 @@ config LOCKDEP_STACK_TRACE_BITS
 	  Try increasing this value if you hit "BUG: MAX_STACK_TRACE_ENTRIES too low!" message.
 
 config LOCKDEP_STACK_TRACE_HASH_BITS
-	int "Bitsize for STACK_TRACE_HASH_SIZE"
+	int "Size for STACK_TRACE_HASH_SIZE (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 26
 	default 14
@@ -1554,7 +1554,7 @@ config LOCKDEP_STACK_TRACE_HASH_BITS
 	  Try increasing this value if you need large STACK_TRACE_HASH_SIZE.
 
 config LOCKDEP_CIRCULAR_QUEUE_BITS
-	int "Bitsize for elements in circular_queue struct"
+	int "Size for elements in circular_queue struct (as Nth power of 2)"
 	depends on LOCKDEP
 	range 10 26
 	default 12
-- 
2.47.0.163.g1226f6d8fa-goog


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation
  2024-10-24 18:36 [PATCH v3 0/3] lockdep: minor config and documentation fixes Carlos Llamas
  2024-10-24 18:36 ` [PATCH v3 1/3] lockdep: fix upper limit for LOCKDEP_*_BITS configs Carlos Llamas
  2024-10-24 18:36 ` [PATCH v3 2/3] lockdep: clarify size " Carlos Llamas
@ 2024-10-24 18:36 ` Carlos Llamas
  2024-10-24 19:01   ` Waiman Long
                     ` (2 more replies)
  2024-10-28 17:34 ` [PATCH v3 0/3] lockdep: minor config and documentation fixes Boqun Feng
  3 siblings, 3 replies; 11+ messages in thread
From: Carlos Llamas @ 2024-10-24 18:36 UTC (permalink / raw)
  To: Boqun Feng, Andrew Morton, Peter Zijlstra, Ingo Molnar,
	Will Deacon
  Cc: linux-kernel, kernel-team, Carlos Llamas, Waiman Long, Huang Ying,
	J. R. Okajima

Define a macro AVG_LOCKDEP_CHAIN_DEPTH to document the magic number '5'
used in the calculation of MAX_LOCKDEP_CHAIN_HLOCKS. The number
represents the estimated average depth (number of locks held) of a lock
chain. The calculation of MAX_LOCKDEP_CHAIN_HLOCKS was first added in
commit 443cd507ce7f ("lockdep: add lock_class information to lock_chain
and output it").

Suggested-by: Waiman Long <longman@redhat.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: J. R. Okajima <hooanon05g@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Will Deacon <will@kernel.org>
Acked-by: Waiman Long <longman@redhat.com>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
---
v3: collect tags
v2: switched the comment for a macro as suggested by Waiman Long.

 kernel/locking/lockdep_internals.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
index bbe9000260d0..20f9ef58d3d0 100644
--- a/kernel/locking/lockdep_internals.h
+++ b/kernel/locking/lockdep_internals.h
@@ -119,7 +119,8 @@ static const unsigned long LOCKF_USED_IN_IRQ_READ =
 
 #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
 
-#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
+#define AVG_LOCKDEP_CHAIN_DEPTH		5
+#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)
 
 extern struct lock_chain lock_chains[];
 
-- 
2.47.0.163.g1226f6d8fa-goog


^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 2/3] lockdep: clarify size for LOCKDEP_*_BITS configs
  2024-10-24 18:36 ` [PATCH v3 2/3] lockdep: clarify size " Carlos Llamas
@ 2024-10-24 19:00   ` Waiman Long
  2024-12-24 18:53   ` [tip: locking/core] lockdep: Clarify " tip-bot2 for Carlos Llamas
  1 sibling, 0 replies; 11+ messages in thread
From: Waiman Long @ 2024-10-24 19:00 UTC (permalink / raw)
  To: Carlos Llamas, Boqun Feng, Andrew Morton
  Cc: linux-kernel, kernel-team, Peter Zijlstra, Ingo Molnar,
	Will Deacon

On 10/24/24 2:36 PM, Carlos Llamas wrote:
> The LOCKDEP_*_BITS configs control the size of internal structures used
> by lockdep. The size is calculated as a power of two of the configured
> value (e.g. 16 => 64KB). Update these descriptions to more accurately
> reflect this, as "Bitsize" can be misleading.
>
> Suggested-by: Andrew Morton <akpm@linux-foundation.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Boqun Feng <boqun.feng@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Waiman Long <longman@redhat.com>
> Cc: Will Deacon <will@kernel.org>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
> ---
>   lib/Kconfig.debug | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index c8b1db37abe6..5a769cbf4bb2 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1522,7 +1522,7 @@ config LOCKDEP_SMALL
>   	bool
>   
>   config LOCKDEP_BITS
> -	int "Bitsize for MAX_LOCKDEP_ENTRIES"
> +	int "Size for MAX_LOCKDEP_ENTRIES (as Nth power of 2)"
>   	depends on LOCKDEP && !LOCKDEP_SMALL
>   	range 10 24
>   	default 15
> @@ -1530,7 +1530,7 @@ config LOCKDEP_BITS
>   	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_ENTRIES too low!" message.
>   
>   config LOCKDEP_CHAINS_BITS
> -	int "Bitsize for MAX_LOCKDEP_CHAINS"
> +	int "Size for MAX_LOCKDEP_CHAINS (as Nth power of 2)"
>   	depends on LOCKDEP && !LOCKDEP_SMALL
>   	range 10 21
>   	default 16
> @@ -1538,7 +1538,7 @@ config LOCKDEP_CHAINS_BITS
>   	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message.
>   
>   config LOCKDEP_STACK_TRACE_BITS
> -	int "Bitsize for MAX_STACK_TRACE_ENTRIES"
> +	int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)"
>   	depends on LOCKDEP && !LOCKDEP_SMALL
>   	range 10 26
>   	default 19
> @@ -1546,7 +1546,7 @@ config LOCKDEP_STACK_TRACE_BITS
>   	  Try increasing this value if you hit "BUG: MAX_STACK_TRACE_ENTRIES too low!" message.
>   
>   config LOCKDEP_STACK_TRACE_HASH_BITS
> -	int "Bitsize for STACK_TRACE_HASH_SIZE"
> +	int "Size for STACK_TRACE_HASH_SIZE (as Nth power of 2)"
>   	depends on LOCKDEP && !LOCKDEP_SMALL
>   	range 10 26
>   	default 14
> @@ -1554,7 +1554,7 @@ config LOCKDEP_STACK_TRACE_HASH_BITS
>   	  Try increasing this value if you need large STACK_TRACE_HASH_SIZE.
>   
>   config LOCKDEP_CIRCULAR_QUEUE_BITS
> -	int "Bitsize for elements in circular_queue struct"
> +	int "Size for elements in circular_queue struct (as Nth power of 2)"
>   	depends on LOCKDEP
>   	range 10 26
>   	default 12
Acked-by: Waiman Long <longman@redhat.com>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation
  2024-10-24 18:36 ` [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation Carlos Llamas
@ 2024-10-24 19:01   ` Waiman Long
  2024-10-25  3:01   ` Huang, Ying
  2024-12-24 18:53   ` [tip: locking/core] lockdep: Document " tip-bot2 for Carlos Llamas
  2 siblings, 0 replies; 11+ messages in thread
From: Waiman Long @ 2024-10-24 19:01 UTC (permalink / raw)
  To: Carlos Llamas, Boqun Feng, Andrew Morton, Peter Zijlstra,
	Ingo Molnar, Will Deacon
  Cc: linux-kernel, kernel-team, Huang Ying, J. R. Okajima

On 10/24/24 2:36 PM, Carlos Llamas wrote:
> Define a macro AVG_LOCKDEP_CHAIN_DEPTH to document the magic number '5'
> used in the calculation of MAX_LOCKDEP_CHAIN_HLOCKS. The number
> represents the estimated average depth (number of locks held) of a lock
> chain. The calculation of MAX_LOCKDEP_CHAIN_HLOCKS was first added in
> commit 443cd507ce7f ("lockdep: add lock_class information to lock_chain
> and output it").
>
> Suggested-by: Waiman Long <longman@redhat.com>
> Cc: Huang Ying <ying.huang@intel.com>
> Cc: J. R. Okajima <hooanon05g@gmail.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Boqun Feng <boqun.feng@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Will Deacon <will@kernel.org>
> Acked-by: Waiman Long <longman@redhat.com>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>
> ---
> v3: collect tags
> v2: switched the comment for a macro as suggested by Waiman Long.
>
>   kernel/locking/lockdep_internals.h | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
> index bbe9000260d0..20f9ef58d3d0 100644
> --- a/kernel/locking/lockdep_internals.h
> +++ b/kernel/locking/lockdep_internals.h
> @@ -119,7 +119,8 @@ static const unsigned long LOCKF_USED_IN_IRQ_READ =
>   
>   #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
>   
> -#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
> +#define AVG_LOCKDEP_CHAIN_DEPTH		5
> +#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)
>   
>   extern struct lock_chain lock_chains[];
>   
Acked-by: Waiman Long <longman@redhat.com>


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation
  2024-10-24 18:36 ` [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation Carlos Llamas
  2024-10-24 19:01   ` Waiman Long
@ 2024-10-25  3:01   ` Huang, Ying
  2024-12-24 18:53   ` [tip: locking/core] lockdep: Document " tip-bot2 for Carlos Llamas
  2 siblings, 0 replies; 11+ messages in thread
From: Huang, Ying @ 2024-10-25  3:01 UTC (permalink / raw)
  To: Carlos Llamas
  Cc: Boqun Feng, Andrew Morton, Peter Zijlstra, Ingo Molnar,
	Will Deacon, linux-kernel, kernel-team, Waiman Long,
	J. R. Okajima

Carlos Llamas <cmllamas@google.com> writes:

> Define a macro AVG_LOCKDEP_CHAIN_DEPTH to document the magic number '5'
> used in the calculation of MAX_LOCKDEP_CHAIN_HLOCKS. The number
> represents the estimated average depth (number of locks held) of a lock
> chain. The calculation of MAX_LOCKDEP_CHAIN_HLOCKS was first added in
> commit 443cd507ce7f ("lockdep: add lock_class information to lock_chain
> and output it").
>
> Suggested-by: Waiman Long <longman@redhat.com>
> Cc: Huang Ying <ying.huang@intel.com>
> Cc: J. R. Okajima <hooanon05g@gmail.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Boqun Feng <boqun.feng@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Will Deacon <will@kernel.org>
> Acked-by: Waiman Long <longman@redhat.com>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>

LGTM, Thanks!  Feel free to add

Acked-by: "Huang, Ying" <ying.huang@intel.com>

in the future versions.

> ---
> v3: collect tags
> v2: switched the comment for a macro as suggested by Waiman Long.
>
>  kernel/locking/lockdep_internals.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
> index bbe9000260d0..20f9ef58d3d0 100644
> --- a/kernel/locking/lockdep_internals.h
> +++ b/kernel/locking/lockdep_internals.h
> @@ -119,7 +119,8 @@ static const unsigned long LOCKF_USED_IN_IRQ_READ =
>  
>  #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
>  
> -#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
> +#define AVG_LOCKDEP_CHAIN_DEPTH		5
> +#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)
>  
>  extern struct lock_chain lock_chains[];

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 0/3] lockdep: minor config and documentation fixes
  2024-10-24 18:36 [PATCH v3 0/3] lockdep: minor config and documentation fixes Carlos Llamas
                   ` (2 preceding siblings ...)
  2024-10-24 18:36 ` [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation Carlos Llamas
@ 2024-10-28 17:34 ` Boqun Feng
  3 siblings, 0 replies; 11+ messages in thread
From: Boqun Feng @ 2024-10-28 17:34 UTC (permalink / raw)
  To: Carlos Llamas
  Cc: Andrew Morton, linux-kernel, kernel-team, Huang Ying,
	J. R. Okajima, Peter Zijlstra, Ingo Molnar, Waiman Long,
	Will Deacon

On Thu, Oct 24, 2024 at 06:36:25PM +0000, Carlos Llamas wrote:
> These are some minor follow-up patches that came up during conversation
> at: https://lore.kernel.org/all/30795.1620913191@jrobl/
> 
> This v3 is based on master as commit e0ba72e3a442 ("lockdep: upper limit
> LOCKDEP_CHAINS_BITS") has now landed.
> 
> Cc: Huang Ying <ying.huang@intel.com>
> Cc: J. R. Okajima <hooanon05g@gmail.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Boqun Feng <boqun.feng@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Waiman Long <longman@redhat.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> 

Queued into lockdep-for-tip for future tests and reviews, if all goes
well, this will be merged in tip during v6.13 and in mainline during
v6.14 merge window.

Thanks!

Regards,
Boqun

> v3: rebased on master and collected tags
> v2: https://lore.kernel.org/all/20240807143922.919604-1-cmllamas@google.com/
> v1: https://lore.kernel.org/all/20240806010128.402852-1-cmllamas@google.com/
> 
> Carlos Llamas (3):
>   lockdep: fix upper limit for LOCKDEP_*_BITS configs
>   lockdep: clarify size for LOCKDEP_*_BITS configs
>   lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation
> 
>  kernel/locking/lockdep_internals.h |  3 ++-
>  lib/Kconfig.debug                  | 18 +++++++++---------
>  2 files changed, 11 insertions(+), 10 deletions(-)
> 
> -- 
> 2.47.0.163.g1226f6d8fa-goog
> 

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [tip: locking/core] lockdep: Document MAX_LOCKDEP_CHAIN_HLOCKS calculation
  2024-10-24 18:36 ` [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation Carlos Llamas
  2024-10-24 19:01   ` Waiman Long
  2024-10-25  3:01   ` Huang, Ying
@ 2024-12-24 18:53   ` tip-bot2 for Carlos Llamas
  2 siblings, 0 replies; 11+ messages in thread
From: tip-bot2 for Carlos Llamas @ 2024-12-24 18:53 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Waiman Long, Huang Ying, J. R. Okajima, Peter Zijlstra,
	Boqun Feng, Ingo Molnar, Will Deacon, Carlos Llamas, x86,
	linux-kernel

The following commit has been merged into the locking/core branch of tip:

Commit-ID:     bd7b5ae26618ad2bd6f6264e2cb6c5815d323e75
Gitweb:        https://git.kernel.org/tip/bd7b5ae26618ad2bd6f6264e2cb6c5815d323e75
Author:        Carlos Llamas <cmllamas@google.com>
AuthorDate:    Thu, 24 Oct 2024 18:36:28 
Committer:     Boqun Feng <boqun.feng@gmail.com>
CommitterDate: Sun, 15 Dec 2024 11:49:35 -08:00

lockdep: Document MAX_LOCKDEP_CHAIN_HLOCKS calculation

Define a macro AVG_LOCKDEP_CHAIN_DEPTH to document the magic number '5'
used in the calculation of MAX_LOCKDEP_CHAIN_HLOCKS. The number
represents the estimated average depth (number of locks held) of a lock
chain. The calculation of MAX_LOCKDEP_CHAIN_HLOCKS was first added in
commit 443cd507ce7f ("lockdep: add lock_class information to lock_chain
and output it").

Suggested-by: Waiman Long <longman@redhat.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: J. R. Okajima <hooanon05g@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Will Deacon <will@kernel.org>
Acked-by: Waiman Long <longman@redhat.com>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Acked-by: "Huang, Ying" <ying.huang@intel.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20241024183631.643450-4-cmllamas@google.com
---
 kernel/locking/lockdep_internals.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
index bbe9000..20f9ef5 100644
--- a/kernel/locking/lockdep_internals.h
+++ b/kernel/locking/lockdep_internals.h
@@ -119,7 +119,8 @@ static const unsigned long LOCKF_USED_IN_IRQ_READ =
 
 #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
 
-#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
+#define AVG_LOCKDEP_CHAIN_DEPTH		5
+#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)
 
 extern struct lock_chain lock_chains[];
 

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [tip: locking/core] lockdep: Clarify size for LOCKDEP_*_BITS configs
  2024-10-24 18:36 ` [PATCH v3 2/3] lockdep: clarify size " Carlos Llamas
  2024-10-24 19:00   ` Waiman Long
@ 2024-12-24 18:53   ` tip-bot2 for Carlos Llamas
  1 sibling, 0 replies; 11+ messages in thread
From: tip-bot2 for Carlos Llamas @ 2024-12-24 18:53 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Andrew Morton, Peter Zijlstra, Boqun Feng, Ingo Molnar,
	Waiman Long, Will Deacon, Carlos Llamas, x86, linux-kernel

The following commit has been merged into the locking/core branch of tip:

Commit-ID:     88a79e88a97cb9309bb48a472be2bf1316d40adc
Gitweb:        https://git.kernel.org/tip/88a79e88a97cb9309bb48a472be2bf1316d40adc
Author:        Carlos Llamas <cmllamas@google.com>
AuthorDate:    Thu, 24 Oct 2024 18:36:27 
Committer:     Boqun Feng <boqun.feng@gmail.com>
CommitterDate: Sun, 15 Dec 2024 11:49:35 -08:00

lockdep: Clarify size for LOCKDEP_*_BITS configs

The LOCKDEP_*_BITS configs control the size of internal structures used
by lockdep. The size is calculated as a power of two of the configured
value (e.g. 16 => 64KB). Update these descriptions to more accurately
reflect this, as "Bitsize" can be misleading.

Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Acked-by: Waiman Long <longman@redhat.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20241024183631.643450-3-cmllamas@google.com
---
 lib/Kconfig.debug | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7635b36..cf2a41d 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1502,7 +1502,7 @@ config LOCKDEP_SMALL
 	bool
 
 config LOCKDEP_BITS
-	int "Bitsize for MAX_LOCKDEP_ENTRIES"
+	int "Size for MAX_LOCKDEP_ENTRIES (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 24
 	default 15
@@ -1510,7 +1510,7 @@ config LOCKDEP_BITS
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_ENTRIES too low!" message.
 
 config LOCKDEP_CHAINS_BITS
-	int "Bitsize for MAX_LOCKDEP_CHAINS"
+	int "Size for MAX_LOCKDEP_CHAINS (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 21
 	default 16
@@ -1518,7 +1518,7 @@ config LOCKDEP_CHAINS_BITS
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message.
 
 config LOCKDEP_STACK_TRACE_BITS
-	int "Bitsize for MAX_STACK_TRACE_ENTRIES"
+	int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 26
 	default 19
@@ -1526,7 +1526,7 @@ config LOCKDEP_STACK_TRACE_BITS
 	  Try increasing this value if you hit "BUG: MAX_STACK_TRACE_ENTRIES too low!" message.
 
 config LOCKDEP_STACK_TRACE_HASH_BITS
-	int "Bitsize for STACK_TRACE_HASH_SIZE"
+	int "Size for STACK_TRACE_HASH_SIZE (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
 	range 10 26
 	default 14
@@ -1534,7 +1534,7 @@ config LOCKDEP_STACK_TRACE_HASH_BITS
 	  Try increasing this value if you need large STACK_TRACE_HASH_SIZE.
 
 config LOCKDEP_CIRCULAR_QUEUE_BITS
-	int "Bitsize for elements in circular_queue struct"
+	int "Size for elements in circular_queue struct (as Nth power of 2)"
 	depends on LOCKDEP
 	range 10 26
 	default 12

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [tip: locking/core] lockdep: Fix upper limit for LOCKDEP_*_BITS configs
  2024-10-24 18:36 ` [PATCH v3 1/3] lockdep: fix upper limit for LOCKDEP_*_BITS configs Carlos Llamas
@ 2024-12-24 18:53   ` tip-bot2 for Carlos Llamas
  0 siblings, 0 replies; 11+ messages in thread
From: tip-bot2 for Carlos Llamas @ 2024-12-24 18:53 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: J. R. Okajima, Peter Zijlstra, Boqun Feng, Ingo Molnar,
	Waiman Long, Will Deacon, Carlos Llamas, x86, linux-kernel

The following commit has been merged into the locking/core branch of tip:

Commit-ID:     e638072e61726cae363d48812815197a2a0e097f
Gitweb:        https://git.kernel.org/tip/e638072e61726cae363d48812815197a2a0e097f
Author:        Carlos Llamas <cmllamas@google.com>
AuthorDate:    Thu, 24 Oct 2024 18:36:26 
Committer:     Boqun Feng <boqun.feng@gmail.com>
CommitterDate: Sun, 15 Dec 2024 11:49:35 -08:00

lockdep: Fix upper limit for LOCKDEP_*_BITS configs

Lockdep has a set of configs used to determine the size of the static
arrays that it uses. However, the upper limit that was initially setup
for these configs is too high (30 bit shift). This equates to several
GiB of static memory for individual symbols. Using such high values
leads to linker errors:

  $ make defconfig
  $ ./scripts/config -e PROVE_LOCKING --set-val LOCKDEP_BITS 30
  $ make olddefconfig all
  [...]
  ld: kernel image bigger than KERNEL_IMAGE_SIZE
  ld: section .bss VMA wraps around address space

Adjust the upper limits to the maximum values that avoid these issues.
The need for anything more, likely points to a problem elsewhere. Note
that LOCKDEP_CHAINS_BITS was intentionally left out as its upper limit
had a different symptom and has already been fixed [1].

Reported-by: J. R. Okajima <hooanon05g@gmail.com>
Closes: https://lore.kernel.org/all/30795.1620913191@jrobl/ [1]
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Acked-by: Waiman Long <longman@redhat.com>
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20241024183631.643450-2-cmllamas@google.com
---
 lib/Kconfig.debug | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 49a3819..7635b36 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1504,7 +1504,7 @@ config LOCKDEP_SMALL
 config LOCKDEP_BITS
 	int "Bitsize for MAX_LOCKDEP_ENTRIES"
 	depends on LOCKDEP && !LOCKDEP_SMALL
-	range 10 30
+	range 10 24
 	default 15
 	help
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_ENTRIES too low!" message.
@@ -1520,7 +1520,7 @@ config LOCKDEP_CHAINS_BITS
 config LOCKDEP_STACK_TRACE_BITS
 	int "Bitsize for MAX_STACK_TRACE_ENTRIES"
 	depends on LOCKDEP && !LOCKDEP_SMALL
-	range 10 30
+	range 10 26
 	default 19
 	help
 	  Try increasing this value if you hit "BUG: MAX_STACK_TRACE_ENTRIES too low!" message.
@@ -1528,7 +1528,7 @@ config LOCKDEP_STACK_TRACE_BITS
 config LOCKDEP_STACK_TRACE_HASH_BITS
 	int "Bitsize for STACK_TRACE_HASH_SIZE"
 	depends on LOCKDEP && !LOCKDEP_SMALL
-	range 10 30
+	range 10 26
 	default 14
 	help
 	  Try increasing this value if you need large STACK_TRACE_HASH_SIZE.
@@ -1536,7 +1536,7 @@ config LOCKDEP_STACK_TRACE_HASH_BITS
 config LOCKDEP_CIRCULAR_QUEUE_BITS
 	int "Bitsize for elements in circular_queue struct"
 	depends on LOCKDEP
-	range 10 30
+	range 10 26
 	default 12
 	help
 	  Try increasing this value if you hit "lockdep bfs error:-1" warning due to __cq_enqueue() failure.

^ permalink raw reply related	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2024-12-24 18:53 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-24 18:36 [PATCH v3 0/3] lockdep: minor config and documentation fixes Carlos Llamas
2024-10-24 18:36 ` [PATCH v3 1/3] lockdep: fix upper limit for LOCKDEP_*_BITS configs Carlos Llamas
2024-12-24 18:53   ` [tip: locking/core] lockdep: Fix " tip-bot2 for Carlos Llamas
2024-10-24 18:36 ` [PATCH v3 2/3] lockdep: clarify size " Carlos Llamas
2024-10-24 19:00   ` Waiman Long
2024-12-24 18:53   ` [tip: locking/core] lockdep: Clarify " tip-bot2 for Carlos Llamas
2024-10-24 18:36 ` [PATCH v3 3/3] lockdep: document MAX_LOCKDEP_CHAIN_HLOCKS calculation Carlos Llamas
2024-10-24 19:01   ` Waiman Long
2024-10-25  3:01   ` Huang, Ying
2024-12-24 18:53   ` [tip: locking/core] lockdep: Document " tip-bot2 for Carlos Llamas
2024-10-28 17:34 ` [PATCH v3 0/3] lockdep: minor config and documentation fixes Boqun Feng

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox