linux-arch.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v9 00/39] dyndbg: fix classmaps API for subsystems, query extensions, and selftests
@ 2026-09-09  0:54 Jim Cromie via B4 Relay
  2026-09-09  0:54 ` [PATCH v9 01/39] selftests/dyndbg: Add kselftest script to verify dynamic-debug Jim Cromie via B4 Relay
                   ` (38 more replies)
  0 siblings, 39 replies; 42+ messages in thread
From: Jim Cromie via B4 Relay @ 2026-09-09  0:54 UTC (permalink / raw)
  To: Jason Baron, Shuah Khan, Andrew Morton, Maarten Lankhorst,
	Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
	Arnd Bergmann, Luis Chamberlain, Petr Pavlu, Daniel Gomez,
	Sami Tolvanen, Aaron Tomlin, Jonathan Corbet, Greg Kroah-Hartman,
	Nathan Chancellor, Nicolas Schier, Shuah Khan, Randy Dunlap,
	Rafael J. Wysocki, Pavel Machek, Len Brown
  Cc: linux-kernel, linux-kselftest, dri-devel, linux-arch,
	linux-modules, linux-doc, linux-kbuild, linux-pm, Jim Cromie,
	Louis Chauvet, Philipp Hahn, kernel test robot

The drm subsystem has *lots* of debug statements, in 11 categories:

 $> ack '\w*_dbg' drivers/gpu/drm/ | wc
    5532   29318  553806
 $> ack 'DRM_DEBUG\w*' drivers/gpu/drm/ | wc
    2208   12856  212035

All of these are bit-tests on __drm_debug, exposed to users as
/sys/module/drm/parameters/debug. Many of these are done often;
vblank is done ~100/sec for some displays. Over the uptime of many
boxes, this is a lot of cpu cycles on bits that are almost always off.

Dynamic-debug excels at replacing such tests with NOOPs (via static
keys). Classmaps was devised to bring that 0-off-cost to drm's
categories.

Classmaps-v1 went into the kernel in Sept 2022, in 2 chunks:
  b7b4eebdba7b..6ea3bf466ac6    # core dyndbg changes
  0406faf25fb1..ee7d633f2dfb    # drm adoption

Sadly DRM-CI found a regression during init with drm.debug=<initval>;
the static-keys underneath the drm-dbgs in drm.ko got enabled, but
those in drivers & helpers did not.

So in Feb 2023, it got pulled:
commit bb2ff6c27bc9 ("drm: Disable dynamic debug as broken")

Root Problem:

DECLARE_DYNDBG_CLASSMAP defined the classmap, but its repeated use in
both core and drivers violated a K&R rule "define once, refer
afterwards". This flaw resulted in a regression; with drm.debug=0xFF
boot arg, drm-core got enabled, but drivers/helpers did not.

This patchset replaces DECLARE_DYNDBG_CLASSMAP with:
- DYNAMIC_DEBUG_CLASSMAP_DEFINE (invoked once in the exporting module)
- DYNAMIC_DEBUG_CLASSMAP_USE (invoked repeatedly in drivers & helpers)

_DEFINE exports the classmap it creates (in drm.ko), and other modules
_USE the classmap. The _USE adds a record referencing the _DEFINEd (&
exported) classmap in a 2nd __dyndbg_class_users section.

At modprobe, dyndbg scans the new section after __dyndbg_class_maps,
follows the linkage to the _DEFINEr module, finds the (optional)
kernel-param controlling the classmap, examines its drm.debug=<initval>,
and applies it to the module being initialized.

To recapitulate the multi-module problem wo DRM involvement, we add:

- tools/testing/selftests/dynamic_debug/dyndbg_selftest.sh:
  Alters pr_debugs in test modules, counts results, checks against
  expectations, and exercises the control grammar.

- lib/test_dynamic_debug.c & test_dynamic_debug_submod.c:
  Builds parent & _submod modules with _DEFINE and _USE inside #if/#else
  blocks, reproducing the 2-module scenario under selftests.

Series Breakdown (40 Patches):

0. Subsystem separation (v9):
   - Decoupled DRM driver opt-in patches to route separately through
     dri-devel / drm-misc; this series contains the 40 dyndbg core
     patches for -mm.

1. Selftest added first:
   - Added tools/testing/selftests/dynamic_debug/dyndbg_selftest.sh
     early (patch 2) to establish baseline verification across
     subsequent patches.
   - Checksums dyndbg's control state and "$1" to catch all output and
     state changes.
   - Includes "I know" mode (K=1) so checksum differences report output
     drift without failing the script, while strict mode (K=0) enforces
     exact baseline conformance.

2. Linker script consolidation & 32-bit fixes:
   - Factored BOUNDED_SECTION* into
     include/asm-generic/bounded_sections.lds.h.
   - Added dyndbg output sections to scripts/module.lds.S to fix lost
     sections.
   - Fixed 8-byte section alignment omission causing i386 NULL pointer
     derefs.

3. Disambiguate Builtin Module Names:
   - master uses KBUILD_MODNAME, which is not unique for builtin modules
     so we have 4 unrelated "main" modules.
   - v8 introduced "pathname-for-builtins" to help
     but sashiko had issues
   - v9 refines this into clean "subsystem-name" (KBUILD_DD_MODNAME),
     starts with full path, strip drivers/ arch/*/ and some leaves.
     yields: init, kernel/*, gpu/drm, etc
   - ie: consolidates fragmented modules (278 -> 211).
   - "kernel/power" gets makefile override to just "power" (maybe undo?)

4. Validation hardening & API:
   - Added compile-time validation for classmap parameters and offset
     bounds.
   - Hardened modprobe error detection for unknown classes and ID
     conflicts.
   - Promoted DYNAMIC_DEBUG_CLASSMAP_PARAM to API.
   - Reduced class parameter storage to u32.

5. Parser & grammar extensions:
   - Treat comma as a token separator.
   - Added multi-query splitting via '@'.
   - Bumped max tokens per command from 9 to 15.
   - Added hyphen-agnostic matching for module names
     (kvm-intel == kvm_intel).
   - Dropped class "protection" special-casing per maintainer feedback.

Testing:
- Tested locally using virtme-ng on x86_64 across 8 configs, including
  KASAN with zero KMEMLEAK warnings.
- dyndbg_selftest.sh passes 100% (99/99 GOLDEN_RECORDS hits).

Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
---
Changes in v9:
- Decouple DRM core setup and driver opt-in commits to route separately
  through dri-devel / drm-misc, keeping this foundation series 100%
  focused on core dynamic_debug infrastructure for -mm.
- Incorporate review findings and automated test fixes prompted by
  sashiko:
  . Rework Kbuild naming heuristic in scripts/Makefile.lib to assign
    clean subsystem-scoped names to built-ins and shared helpers
    (e.g. [init], [power], [coco/sev], [mmc/host]), eliminating
    multi-token whitespace in column 2 of dynamic_debug/control.
  . Fix missing DYNAMIC_DEBUG_CLASSMAP_USE_() stub under
    !CONFIG_DYNAMIC_DEBUG.
  . Make ddebug_add_module() non-fatal on failure during
    dynamic_debug_init().
  . Fix off-by-one assertions in __DYNAMIC_DEBUG_CLASSMAP_CHECK and
    ddebug_add_module().
  . Prevent loop wrapping lockup in test_dynamic_debug:do_bulk() on
    UINT_MAX and add cond_resched() to bulk print loops.
  . Fix 4-argument signature on DYNAMIC_DEBUG_CLASSMAP_PARAM_REF
    fallback macro stub.
  . Assign map->controlling_param to enforce classmap protection on
    parameterized classes.
  . Update ddebug_proc_start() seeking to return EPILOGUE_TOKEN
    when n == 0.
  . Free temporary buffer in reset_stats handler and iterate across
    possible CPUs for dynamic debug call counter.
  . Selftests: sanitize "$K", drop duplicate declarations, fix
    unquoted error output, align LACK_DD_BUILTIN filters to labels,
    add K=2 silent mode, and prime params with +p in
    FT_comma_terminators.
- Link to v8: https://lore.kernel.org/r/20260905-dd-cmap-part2-clean-v8-0-a4cc0674f6fd@gmail.com

Changes in v8:
- Unified 44-patch series (incorporating follow-on compile-time checks,
  comma-token delimiters, '@' multi-query separator, and inheritance
  tests).
- Rebased onto upstream v7.3-rc1.
- Passing on dyndbg_selftest.sh under KASAN + KMEMLEAK.
- Link to v7: https://lore.kernel.org/r/20260721-dd-maint-2-v7-0-010fbe73b311@gmail.com

Changes in v6..v7:
- Add compile-time validation for classmap parameters and offset
bounds (__DYNAMIC_DEBUG_CLASSMAP_CHECK).
- Harden modprobe-time error detection for unknown class names and
  class ID reservation conflicts.
- Promote DYNAMIC_DEBUG_CLASSMAP_PARAM to public API.
- Shrink class parameter storage to u32.
- Hoist classmap filtering up to ddebug_add_module().

Changes in v4..v5:
- Tighten function signatures (ddebug_apply_class_bitmap,
  param_set_dyndbg_classes).
- Replace classmap linked-list with vector / array-slice.
- Add for_subvec() loop helper and restructure _ddebug_info substructs.
- Move mod_name down from struct ddebug_table to _ddebug_info.

Changes in v2..v3:
- Refactor BOUNDED_SECTION* macros from
  include/asm-generic/vmlinux.lds.h into
  include/asm-generic/bounded_sections.lds.h.
- Add dyndbg output sections to scripts/module.lds.S to fix lost
  sections in loadable modules.
- Fix 8-byte section alignment omission causing i386 NULL pointer deref.
- Move dyndbg_selftest.sh to the front of the series for bisectability.

Changes in v12 (DRM combined series):
- Refactor vmlinux.lds.h and add bounded_sections.lds.h and
  dyndbg.lds.h.
- Refine DYNAMIC_DEBUG_CLASSMAP_USE*() with extern'd classmap linkages
  and compile-time offset checks.
- Include patch from Philipp Hahn <phahn-oss@avm.de> (Ignore additional
  arguments from pr_fmt).
- Link to v12: https://lore.kernel.org/lkml/20260326175533.1402867-1-jim.cromie@gmail.com/

Changes in v11 (DRM combined series):
- Rebase on drm-misc-next for DRM-CI testing.
- Fix 32-bit truncation error in drm_buddy.
- Fix drm_printer_debug_fn message spew causing test timeouts.
- Verify on DRM-CI Pipeline #1622778 (621 KUnit tests, 370 i915-CML
  tests passed, 0 failures).
- Link to v11: https://lore.kernel.org/lkml/20260313132103.2529746-1-jim.cromie@gmail.com/

Changes in v10 (DRM combined series):
- Initial replacement of DECLARE_DYNDBG_CLASSMAP with
  DYNAMIC_DEBUG_CLASSMAP_DEFINE and DYNAMIC_DEBUG_CLASSMAP_USE.
- Add tools/testing/selftests/dynamic_debug/ and
  test_dynamic_debug_submod.ko.
- Drop class "protection" special-casing per Jason Baron's feedback.
- Link to v10: https://lore.kernel.org/lkml/20250125064619.8305-1-jim.cromie@gmail.com/

---
Jim Cromie (38):
      selftests/dyndbg: Add kselftest script to verify dynamic-debug
      vmlinux.lds.h: refactor BOUNDED_SECTION_* macros into bounded_sections.lds.h
      vmlinux.lds.h: drop unused HEADERED_SECTION* macros
      vmlinux.lds.h: Fix ALIGN(8) omission causing NULL ptr on i386
      vmlinux.lds.h: remove redundant ALIGN(8) directives
      dyndbg.lds.S: fix lost dyndbg sections in modules
      dyndbg: factor ddebug_match_desc out from ddebug_change
      dyndbg: add stub macro for DECLARE_DYNDBG_CLASSMAP
      dyndbg: reword "class unknown," to "class:_UNKNOWN_"
      dyndbg-API: remove DD_CLASS_TYPE_(DISJOINT|LEVEL)_NAMES and code
      dyndbg: drop NUM_TYPE_ARGS
      dyndbg: bump num-tokens in a query-cmd from 9 to 15
      dyndbg: reduce verbose/debug clutter
      lib/parser: add match_wildcard_hyphen() for agnostic matching
      kbuild, dyndbg: Clean up builtin module-name ambiguities
      dyndbg: refactor param_set_dyndbg_classes and below
      dyndbg: tighten fn-sig of ddebug_apply_class_bitmap
      dyndbg: replace classmap list with an array-slice
      dyndbg: macrofy a 2-index for-loop pattern
      dyndbg: reduce class param storage to u32
      dyndbg,module: make proper substructs in _ddebug_info
      dyndbg: move mod_name down from struct ddebug_table to _ddebug_info
      dyndbg: hoist classmap-filter-by-modname up to ddebug_add_module
      dyndbg-API: replace DECLARE_DYNDBG_CLASSMAP
      selftests/dyndbg: enable FT_classmap_inheritance
      dyndbg: detect class_id reservation conflicts
      dyndbg: check DYNAMIC_DEBUG_CLASSMAP_{DEFINE,USE_} args at compile-time
      dyndbg-test: add do_bulk testpoint, rename do_prints to do_classes
      dyndbg-API: promote DYNAMIC_DEBUG_CLASSMAP_PARAM to API
      dyndbg: control-parser: treat comma as a token separator
      selftests: enable comma-terminator tests
      dyndbg: split multi-query strings with @
      dyndbg: resolve "protection" of class'd pr_debug
      dyndbg: harden classmap and descriptor validation
      docs/dyndbg: add classmap info to howto
      dyndbg: add epilogue to dynamic_debug/control file
      dyndbg: add +c flag to count advantage of classmaps for DRM
      dyndbg: add DEBUG-biased fallback stubs for _dynamic_func_call_cls

Philipp Hahn (1):
      dyndbg: Ignore additional arguments from pr_fmt

 Documentation/admin-guide/dynamic-debug-howto.rst  | 197 ++++-
 MAINTAINERS                                        |   2 +
 drivers/gpu/drm/drm_print.c                        |   4 +-
 include/asm-generic/bounded_sections.lds.h         |  32 +
 include/asm-generic/dyndbg.lds.h                   |  22 +
 include/asm-generic/vmlinux.lds.h                  |  68 +-
 include/drm/drm_print.h                            |   2 +-
 include/linux/dynamic_debug.h                      | 368 ++++++--
 include/linux/parser.h                             |   1 +
 kernel/module/main.c                               |  15 +-
 kernel/power/Makefile                              |   2 +
 lib/Kconfig.debug                                  |  24 +-
 lib/Makefile                                       |   3 +
 lib/dynamic_debug.c                                | 945 ++++++++++++++-------
 lib/parser.c                                       |  58 +-
 lib/test_dynamic_debug.c                           | 275 ++++--
 lib/test_dynamic_debug_submod.c                    |  21 +
 scripts/Makefile.lib                               |  26 +
 scripts/module.lds.S                               |   2 +
 tools/testing/selftests/dynamic_debug/Makefile     |  10 +
 tools/testing/selftests/dynamic_debug/config       |   8 +
 .../selftests/dynamic_debug/dyndbg_selftest.sh     | 846 ++++++++++++++++++
 .../dynamic_debug/syslog_hash_validation.sh        | 393 +++++++++
 23 files changed, 2777 insertions(+), 547 deletions(-)
---
base-commit: 136ebbeb1c6040f2739ac4a9e0f704395faaf64f
change-id: 20260901-dd-cmap-part2-clean-369ec194e4af

Best regards,
-- 
Jim Cromie <jim.cromie@gmail.com>



^ permalink raw reply	[flat|nested] 42+ messages in thread

end of thread, other threads:[~2026-09-11 18:21 UTC | newest]

Thread overview: 42+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-09  0:54 [PATCH v9 00/39] dyndbg: fix classmaps API for subsystems, query extensions, and selftests Jim Cromie via B4 Relay
2026-09-09  0:54 ` [PATCH v9 01/39] selftests/dyndbg: Add kselftest script to verify dynamic-debug Jim Cromie via B4 Relay
2026-09-09  0:54 ` [PATCH v9 02/39] vmlinux.lds.h: refactor BOUNDED_SECTION_* macros into bounded_sections.lds.h Jim Cromie via B4 Relay
2026-09-09  0:54 ` [PATCH v9 03/39] vmlinux.lds.h: drop unused HEADERED_SECTION* macros Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 04/39] vmlinux.lds.h: Fix ALIGN(8) omission causing NULL ptr on i386 Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 05/39] vmlinux.lds.h: remove redundant ALIGN(8) directives Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 06/39] dyndbg.lds.S: fix lost dyndbg sections in modules Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 07/39] dyndbg: factor ddebug_match_desc out from ddebug_change Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 08/39] dyndbg: add stub macro for DECLARE_DYNDBG_CLASSMAP Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 09/39] dyndbg: reword "class unknown," to "class:_UNKNOWN_" Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 10/39] dyndbg-API: remove DD_CLASS_TYPE_(DISJOINT|LEVEL)_NAMES and code Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 11/39] dyndbg: drop NUM_TYPE_ARGS Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 12/39] dyndbg: bump num-tokens in a query-cmd from 9 to 15 Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 13/39] dyndbg: reduce verbose/debug clutter Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 14/39] lib/parser: add match_wildcard_hyphen() for agnostic matching Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 15/39] kbuild, dyndbg: Clean up builtin module-name ambiguities Jim Cromie via B4 Relay
2026-09-09 15:50   ` Nicolas Schier
2026-09-11 18:20     ` jim.cromie
2026-09-09  0:55 ` [PATCH v9 16/39] dyndbg: refactor param_set_dyndbg_classes and below Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 17/39] dyndbg: tighten fn-sig of ddebug_apply_class_bitmap Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 18/39] dyndbg: replace classmap list with an array-slice Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 19/39] dyndbg: macrofy a 2-index for-loop pattern Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 20/39] dyndbg: reduce class param storage to u32 Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 21/39] dyndbg,module: make proper substructs in _ddebug_info Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 22/39] dyndbg: move mod_name down from struct ddebug_table to _ddebug_info Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 23/39] dyndbg: hoist classmap-filter-by-modname up to ddebug_add_module Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 24/39] dyndbg-API: replace DECLARE_DYNDBG_CLASSMAP Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 25/39] selftests/dyndbg: enable FT_classmap_inheritance Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 26/39] dyndbg: detect class_id reservation conflicts Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 27/39] dyndbg: check DYNAMIC_DEBUG_CLASSMAP_{DEFINE,USE_} args at compile-time Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 28/39] dyndbg-test: add do_bulk testpoint, rename do_prints to do_classes Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 29/39] dyndbg-API: promote DYNAMIC_DEBUG_CLASSMAP_PARAM to API Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 30/39] dyndbg: control-parser: treat comma as a token separator Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 31/39] selftests: enable comma-terminator tests Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 32/39] dyndbg: split multi-query strings with @ Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 33/39] dyndbg: resolve "protection" of class'd pr_debug Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 34/39] dyndbg: harden classmap and descriptor validation Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 35/39] docs/dyndbg: add classmap info to howto Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 36/39] dyndbg: Ignore additional arguments from pr_fmt Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 37/39] dyndbg: add epilogue to dynamic_debug/control file Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 38/39] dyndbg: add +c flag to count advantage of classmaps for DRM Jim Cromie via B4 Relay
2026-09-09  0:55 ` [PATCH v9 39/39] dyndbg: add DEBUG-biased fallback stubs for _dynamic_func_call_cls Jim Cromie via B4 Relay

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).