AMD-GFX Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v8 20/35] dyndbg-doc: add classmap info to howto
@ 2024-04-29 19:39 Jim Cromie
  2024-04-29 19:39 ` [PATCH v8 21/35] dyndbg: treat comma as a token separator Jim Cromie
                   ` (14 more replies)
  0 siblings, 15 replies; 18+ messages in thread
From: Jim Cromie @ 2024-04-29 19:39 UTC (permalink / raw)
  To: jbaron, gregkh, linux-kernel, dri-devel, amd-gfx, intel-gvt-dev,
	intel-gfx
  Cc: ukaszb, linux-doc, daniel.vetter, tvrtko.ursulin, jani.nikula,
	ville.syrjala, seanpaul, robdclark, groeck, yanivt, bleung,
	Jim Cromie

Describe the 3 API macros providing dynamic_debug's classmaps

DYNDBG_CLASSMAP_DEFINE - create, exports a module's classmap
DYNDBG_CLASSMAP_USE    - refer to exported map
DYNDBG_CLASSMAP_PARAM  - bind control param to the classmap
DYNDBG_CLASSMAP_PARAM_REF + use module's storage - __drm_debug

cc: linux-doc@vger.kernel.org
Signed-off-by: Jim Cromie <jim.cromie@gmail.com>
---
v5 adjustments per Randy Dunlap
v7 checkpatch fixes
v8 more
---
 .../admin-guide/dynamic-debug-howto.rst       | 63 ++++++++++++++++++-
 1 file changed, 62 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/dynamic-debug-howto.rst b/Documentation/admin-guide/dynamic-debug-howto.rst
index 6a8ce5a34382..742eb4230c6e 100644
--- a/Documentation/admin-guide/dynamic-debug-howto.rst
+++ b/Documentation/admin-guide/dynamic-debug-howto.rst
@@ -225,7 +225,6 @@ the ``p`` flag has meaning, other flags are ignored.
 Note the regexp ``^[-+=][fslmpt_]+$`` matches a flags specification.
 To clear all flags at once, use ``=_`` or ``-fslmpt``.
 
-
 Debug messages during Boot Process
 ==================================
 
@@ -375,3 +374,65 @@ just a shortcut for ``print_hex_dump(KERN_DEBUG)``.
 For ``print_hex_dump_debug()``/``print_hex_dump_bytes()``, format string is
 its ``prefix_str`` argument, if it is constant string; or ``hexdump``
 in case ``prefix_str`` is built dynamically.
+
+Dynamic Debug classmaps
+=======================
+
+Dyndbg allows selection/grouping of *prdbg* callsites using structural
+info: module, file, function, line.  Classmaps allow authors to add
+their own domain-oriented groupings using class-names.  Classmaps are
+exported, so they referencable from other modules.
+
+  # enable classes individually
+  :#> ddcmd class DRM_UT_CORE +p
+  :#> ddcmd class DRM_UT_KMS +p
+  # or more selectively
+  :#> ddcmd class DRM_UT_CORE module drm +p
+
+The "class FOO" syntax protects class'd prdbgs from generic overwrite::
+
+  # IOW this doesn't wipe any DRM.debug settings
+  :#> ddcmd -p
+
+To support the DRM.debug parameter, DYNDBG_CLASSMAP_PARAM* updates all
+classes in a classmap, mapping param-bits 0..N onto the classes:
+DRM_UT_<*> for the DRM use-case.
+
+Dynamic Debug Classmap API
+==========================
+
+DYNDBG_CLASSMAP_DEFINE - modules use this to create classmaps, naming
+each of the classes (stringified enum-symbols: "DRM_UT_<*>"), and
+type, and mapping the class-names to consecutive _class_ids.
+
+By doing so, modules tell dyndbg that they have prdbgs with those
+class_ids, and they authorize dyndbg to accept "class FOO" for the
+module defining the classmap, and its contained classnames.
+
+DYNDBG_CLASSMAP_USE - drm drivers invoke this to ref the CLASSMAP that
+drm DEFINEs.  This shares the classmap definition, and authorizes
+dyndbg to apply changes to the user module's class'd pr_debugs.  It
+also tells dyndbg how to initialize the user's prdbgs at modprobe,
+based upon the current setting of the parent's controlling param.
+
+There are 2 types of classmaps:
+
+ DD_CLASS_TYPE_DISJOINT_BITS: classes are independent, like DRM.debug
+ DD_CLASS_TYPE_LEVEL_NUM: classes are relative, ordered (V3 > V2)
+
+DYNDBG_CLASSMAP_PARAM - modelled after module_param_cb, it refers to a
+DEFINEd classmap, and associates it to the param's data-store.  This
+state is then applied to DEFINEr and USEr modules when they're modprobed.
+
+This interface also enforces the DD_CLASS_TYPE_LEVEL_NUM relation
+amongst the contained classnames; all classes are independent in the
+control parser itself.
+
+Modules or module-groups (drm & drivers) can define multiple
+classmaps, as long as they share the limited 0..62 per-module-group
+_class_id range, without overlap.
+
+``#define DEBUG`` will enable all pr_debugs in scope, including any
+class'd ones.  This won't be reflected in the PARAM readback value,
+but the class'd pr_debug callsites can be forced off by toggling the
+classmap-kparam all-on then all-off.
-- 
2.44.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread
* [PATCH v8 00/35] fix CONFIG_DRM_USE_DYNAMIC_DEBUG=y regression
@ 2024-04-29 19:31 Jim Cromie
  2024-04-29 19:31 ` [PATCH v8 21/35] dyndbg: treat comma as a token separator Jim Cromie
  0 siblings, 1 reply; 18+ messages in thread
From: Jim Cromie @ 2024-04-29 19:31 UTC (permalink / raw)
  To: jbaron, gregkh, linux-kernel, dri-devel, amd-gfx, intel-gvt-dev,
	intel-gfx
  Cc: ukaszb, linux-doc, daniel.vetter, tvrtko.ursulin, jani.nikula,
	ville.syrjala, seanpaul, robdclark, groeck, yanivt, bleung,
	Jim Cromie

hi Greg, Jason, DRM-folk,

This patchset fixes the CONFIG_DRM_USE_DYNAMIC_DEBUG=y regression,
Fixes: bb2ff6c27bc9 ("drm: Disable dynamic debug as broken")

this is v8.
Its also here:
https://github.com/jimc/linux/tree/dd-classmap-fix-8a

v7 had at least 2 problems:

 https://lore.kernel.org/lkml/20231101002609.3533731-1-jim.cromie@gmail.com/
 https://patchwork.freedesktop.org/series/125066/

1. missing __align(8) in METATDATA macro, giving too much placement
freedom to linker, caused weird segvs following non-ptr vals, but for
builtin modules only. found by lkp-test.

2. the main patch changed both the dyndbg API, and the drm/drivers.
This was a flag-day annoyance, and not practical.  Fix by preserving
old API macro until "later", and splitting the patch and set into 2
sequential subsets.  removal can wait.
 
What was broken ?

Booting a modular kernel with drm.debug=0x1ff, this enabled pr_debugs
only in drm itself, not the yet-to-be loaded driver + helpers.  I had
tested with scripts doing lots of modprobes with dyndbg=<> options
permuting.  I didn't notice that I didn't really test without them.

The deeper cause was my design error, a violation of the K&R rule:
"define once, refer many times".

DECLARE_DYNDBG_CLASSMAP defined the classmap, and was used everywhere,
re-declaring the same static classmap repeatedly. Jani Nikula actually
picked up on this at the time, but didn't scream loudly enough for
anyone to notice, I know I didn't get it then.  One patchset across 2
trees didn't help either.

The revised classmap API "splits" it to def & ref.

DYNDBG_CLASSMAP_DEFINE fixes & updates the busted macro, EXPORTing the
classmap instead.  It gets invoked once per subsystem, by the
parent/builtin, drm.ko for DRM.

DYNDBG_CLASSMAP_USE in drivers and helpers refer to the classmap by
name, which links the 2 modules (like __drm_debug already does).

These 2 tell dyndbg to map "class FOO" to the defined FOO_ID, which
allows it to make those changes via >control.

DYNDBG_CLASSMAP_PARAM*, defines the controlling kparam, and binds it
to both the _var, and the _DEFINEd classmap.  So drm uses this to bind
the classmap to __drm_debug.

It provides the common control-point for the sub-system; it is applied
to the classmaps during modprobe of both _DEFINEr and USErs.  It also
enforces the relative nature of LEVEL classmaps, ie V3>V2.

DECLARE_DYNDBG_CLASSMAP is preserved to decouple the DRM patches.

A new struct and elf section contain the _USEs; on modprobe, these are
scanned similarly to the _DEFINEs, but follow the references to their
defining modules, find the kparam wired to the classmap, and apply its
classmap settings to the USEr.  This action is what V1 missed, which
is why drivers failed to enable debug during modprobe.

In order to recapitulate the regression scenario without involving
DRM, the patchset (v6 I think) adds test_dynamic_debug_submod, which
is a duplicate of its parent; _submod.c #defines _SUBMOD, and then
includes parent.

This puts _DEFINE and _USE close together in the same file, for
obviousness.  It also guarantees that the submod always has the same
complement of debug()s, giving consistent output from both when
classmaps are working properly, as tested when changing callsites via
both param and >control.

To provide a turn-key selftest, the patchset also adds
./tools/testing/selftests/dynamic_debug/dyndbg_selftest.sh, pilfered
from a debug-to-trace patchset I and Lukasz Bartozik have been working
out.  It starts with basic_tests, then to test 2 new parsing
delimiters, which simplify testing of the classmap functionality.

It works nicely from virtme-ng:

[jimc@frodo vx]$ vrun_ -- ./tools/testing/selftests/dynamic_debug/dyndbg_selftest.sh
doing: vng --verbose --name v6.9-rc5-34-g2f1ace6e1c68 \
       --user root --cwd ../.. \
       -a dynamic_debug.verbose=2 -p 4 \
       -- ./tools/testing/selftests/dynamic_debug/dyndbg_selftest.sh
virtme: waiting for virtiofsd to start
...
[    3.546739] ip (260) used greatest stack depth: 12368 bytes left
[    3.609288] virtme-init: starting script
test_dynamic_debug_submod not there
test_dynamic_debug not there
# BASIC_TESTS 
...
# Done on: Fri Apr 26 20:45:08 MDT 2024
[    4.765751] virtme-init: script returned {0}
Powering off.
[    4.805790] ACPI: PM: Preparing to enter system sleep state S5
[    4.806223] kvm: exiting hardware virtualization
[    4.806564] reboot: Power down
[jimc@frodo vx]$ 


I've been running the kernel on my x86 desktop & laptop, booting with
drm.debug=0x1f, then turning it all-off after sleep 15.

a few highlights from a bare-metal boot:

here modprobe amdgpu; dyndbg applies last bit/class/category, and finishes init,
then drm and amdgpu start logging as they execute
...
[    9.019696] gandalf kernel: dyndbg: query 0: "class DRM_UT_ATOMIC +p" mod:amdgpu
[    9.019704] gandalf kernel: dyndbg: class-ref: amdgpu.DRM_UT_ATOMIC  module:amdgpu nd:4754 nc:0 nu:1
[    9.020012] gandalf kernel: dyndbg: processed 1 queries, with 21 matches, 0 errs
[    9.020017] gandalf kernel: dyndbg: bit_4: 21 matches on class: DRM_UT_ATOMIC -> 0x1f
[    9.020021] gandalf kernel: dyndbg: applied bitmap: 0x1f to: 0x0 for amdgpu
[    9.020026] gandalf kernel: dyndbg: attach-client-module:  module:amdgpu nd:4754 nc:0 nu:1
[    9.020031] gandalf kernel: dyndbg: 4754 debug prints in module amdgpu
[    9.055065] gandalf kernel: [drm] amdgpu kernel modesetting enabled.
[    9.055138] gandalf kernel: [drm:amdgpu_acpi_detect [amdgpu]] No matching acpi device found for AMD3000
[    9.055564] gandalf kernel: amdgpu: Virtual CRAT table created for CPU
[    9.055585] gandalf kernel: amdgpu: Topology: Add CPU node
[    9.055752] gandalf kernel: amdgpu 0000:0c:00.0: enabling device (0006 -> 0007)
[    9.055821] gandalf kernel: [drm] initializing kernel modesetting (NAVI10 0x1002:0x731F 0x148C:0x2398 0xC1).
[    9.055835] gandalf kernel: [drm] register mmio base: 0xFCB00000
[    9.055839] gandalf kernel: [drm] register mmio size: 524288
[    9.059148] gandalf kernel: [drm:amdgpu_discovery_set_ip_blocks [amdgpu]] number of dies: 1
[    9.059387] gandalf kernel: [drm:amdgpu_discovery_set_ip_blocks [amdgpu]] number of hardware IPs on die0: 39
[    9.059623] gandalf kernel: [drm:amdgpu_discovery_set_ip_blocks [amdgpu]] ATHUB(35) #0 v2.0.0:
[    9.059856] gandalf kernel: [drm:amdgpu_discovery_set_ip_blocks [amdgpu]]         0x00000c00
[    9.060096] gandalf kernel: [drm:amdgpu_discovery_set_ip_blocks [amdgpu]]         0x02408c00
[    9.060328] gandalf kernel: [drm:amdgpu_discovery_set_ip_blocks [amdgpu]] set register base offset for ATHUB

a startup script, after sleep 15, turns off the logging:

  echo 0 > /sys/module/drm/parameters/debug

heres 1st 2 bits/classes/categories being turned off:

[   29.105991] gandalf kernel: [drm:amdgpu_ih_process [amdgpu]] amdgpu_ih_process: rptr 90752, wptr 90784
[   29.118086] gandalf kernel: dyndbg: bits:0x0 > *.debug
[   29.118096] gandalf kernel: dyndbg: apply bitmap: 0x0 to: 0x1f for '*'
[   29.118102] gandalf kernel: dyndbg: query 0: "class DRM_UT_CORE -p" mod:*
[   29.118122] gandalf kernel: dyndbg: good-class: drm.DRM_UT_CORE  module:drm nd:338 nc:1 nu:0
[   29.119548] gandalf kernel: dyndbg: class-ref: drm_kms_helper.DRM_UT_CORE  module:drm_kms_helper nd:93 
nc:0 nu:1
[   29.119552] gandalf kernel: dyndbg: class-ref: drm_display_helper.DRM_UT_CORE  module:drm_display_helper nd:151 nc:0 nu:1
[   29.119737] gandalf kernel: dyndbg: class-ref: amdgpu.DRM_UT_CORE  module:amdgpu nd:4754 nc:0 nu:1
[   29.122181] gandalf kernel: [drm:amdgpu_ih_process [amdgpu]] amdgpu_ih_process: rptr 90784, wptr 90816
[   29.127687] gandalf kernel: dyndbg: processed 1 queries, with 466 matches, 0 errs
[   29.127690] gandalf kernel: dyndbg: bit_0: 466 matches on class: DRM_UT_CORE -> 0x0
[   29.127692] gandalf kernel: dyndbg: query 0: "class DRM_UT_DRIVER -p" mod:*
[   29.127696] gandalf kernel: dyndbg: good-class: drm.DRM_UT_DRIVER  module:drm nd:338 nc:1 nu:0
[   29.127699] gandalf kernel: dyndbg: class-ref: drm_kms_helper.DRM_UT_DRIVER  module:drm_kms_helper nd:93 nc:0 nu:1
[   29.127701] gandalf kernel: dyndbg: class-ref: drm_display_helper.DRM_UT_DRIVER  module:drm_display_helper nd:151 nc:0 nu:1
[   29.127885] gandalf kernel: dyndbg: class-ref: amdgpu.DRM_UT_DRIVER  module:amdgpu nd:4754 nc:0 nu:1
[   29.152925] gandalf kernel: dyndbg: processed 1 queries, with 1384 matches, 0 errs


The resulting journal is ~14.6k lines, written in the 1st 15 (29)
seconds of startup.  I'm unsure what the 15/29 discrepancy might
indicate/betray, besides a lot of logging work.  sleep 15 is not the
best stopwatch.

recent spins thru lkp-test have also been SUCCESS-ful.

Jim Cromie (35):

old-bugs & cleanups:
  dyndbg: fix old BUG_ON in >control parser
  docs/dyndbg: update examples \012 to \n
  test-dyndbg: fixup CLASSMAP usage error
  dyndbg: reword "class unknown," to "class:_UNKNOWN_"
  dyndbg: make ddebug_class_param union members same size
  dyndbg: replace classmap list with a vector
  dyndbg: ddebug_apply_class_bitmap - add module arg, select on it
  dyndbg: split param_set_dyndbg_classes to _module & wrapper fns
  dyndbg: drop NUM_TYPE_ARRAY
  dyndbg: reduce verbose/debug clutter
  dyndbg: silence debugs with no-change updates
  dyndbg: tighten ddebug_class_name() 1st arg type
  dyndbg: tighten fn-sig of ddebug_apply_class_bitmap
  dyndbg: reduce verbose=3 messages in ddebug_add_module
  dyndbg-API: remove DD_CLASS_TYPE_(DISJOINT|LEVEL)_NAMES and code

core fix & selftests:
  dyndbg-API: fix DECLARE_DYNDBG_CLASSMAP
  selftests-dyndbg: add tools/testing/selftests/dynamic_debug/*
  selftests-dyndbg: exit 127 if no facility
  dyndbg-API: promote DYNDBG_CLASSMAP_PARAM to API
  dyndbg-doc: add classmap info to howto
  dyndbg: treat comma as a token separator
  selftests-dyndbg: add comma_terminator_tests
  dyndbg: split multi-query strings with %
  selftests-dyndbg: test_percent_splitting multi-cmds on module classes
  docs/dyndbg: explain new delimiters: comma, percent
  selftests-dyndbg: add test_mod_submod
  selftests-dyndbg: test dyndbg-to-tracefs
  dyndbg-doc: explain flags parse 1st
  dyndbg: add __counted_by annotations

DRM parts
  drm+drivers: adapt to use DYNDBG_CLASSMAP_{DEFINE,USE}
  drm-dyndbg: adapt to use DYNDBG_CLASSMAP_PARAM
  drm: use correct ccflags-y spelling
  drm-drivers: DRM_CLASSMAP_USE in 2nd batch of drivers, helpers
  drm: restore CONFIG_DRM_USE_DYNAMIC_DEBUG un-BROKEN
  drm-print: workaround compiler meh

 .../admin-guide/dynamic-debug-howto.rst       |  99 ++-
 MAINTAINERS                                   |   3 +-
 drivers/gpu/drm/Kconfig                       |   3 +-
 drivers/gpu/drm/Makefile                      |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c       |  12 +-
 drivers/gpu/drm/display/drm_dp_helper.c       |  12 +-
 drivers/gpu/drm/drm_crtc_helper.c             |  12 +-
 drivers/gpu/drm/drm_gem_shmem_helper.c        |   2 +
 drivers/gpu/drm/drm_print.c                   |  38 +-
 drivers/gpu/drm/gud/gud_drv.c                 |   2 +
 drivers/gpu/drm/i915/i915_params.c            |  12 +-
 drivers/gpu/drm/mgag200/mgag200_drv.c         |   2 +
 drivers/gpu/drm/nouveau/nouveau_drm.c         |  12 +-
 drivers/gpu/drm/qxl/qxl_drv.c                 |   2 +
 drivers/gpu/drm/radeon/radeon_drv.c           |   2 +
 drivers/gpu/drm/udl/udl_main.c                |   2 +
 drivers/gpu/drm/vkms/vkms_drv.c               |   2 +
 drivers/gpu/drm/vmwgfx/vmwgfx_drv.c           |   2 +
 include/asm-generic/vmlinux.lds.h             |   1 +
 include/drm/drm_print.h                       |  10 +
 include/linux/dynamic_debug.h                 | 130 ++-
 kernel/module/main.c                          |   3 +
 lib/Kconfig.debug                             |  24 +-
 lib/Makefile                                  |   3 +
 lib/dynamic_debug.c                           | 443 ++++++----
 lib/test_dynamic_debug.c                      | 131 +--
 lib/test_dynamic_debug_submod.c               |  17 +
 tools/testing/selftests/Makefile              |   1 +
 .../testing/selftests/dynamic_debug/Makefile  |   9 +
 tools/testing/selftests/dynamic_debug/config  |   2 +
 .../dynamic_debug/dyndbg_selftest.sh          | 765 ++++++++++++++++++
 31 files changed, 1398 insertions(+), 363 deletions(-)
 create mode 100644 lib/test_dynamic_debug_submod.c
 create mode 100644 tools/testing/selftests/dynamic_debug/Makefile
 create mode 100644 tools/testing/selftests/dynamic_debug/config
 create mode 100755 tools/testing/selftests/dynamic_debug/dyndbg_selftest.sh

-- 
2.44.0


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2024-05-01  8:03 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-29 19:39 [PATCH v8 20/35] dyndbg-doc: add classmap info to howto Jim Cromie
2024-04-29 19:39 ` [PATCH v8 21/35] dyndbg: treat comma as a token separator Jim Cromie
2024-04-29 19:39 ` [PATCH v8 22/35] selftests-dyndbg: add comma_terminator_tests Jim Cromie
2024-04-29 19:39 ` [PATCH v8 23/35] dyndbg: split multi-query strings with % Jim Cromie
2024-04-29 19:39 ` [PATCH v8 24/35] selftests-dyndbg: test_percent_splitting multi-cmds on module classes Jim Cromie
2024-04-29 19:39 ` [PATCH v8 25/35] docs/dyndbg: explain new delimiters: comma, percent Jim Cromie
2024-04-29 19:39 ` [PATCH v8 26/35] selftests-dyndbg: add test_mod_submod Jim Cromie
2024-04-29 19:39 ` [PATCH v8 27/35] selftests-dyndbg: test dyndbg-to-tracefs Jim Cromie
2024-04-29 19:39 ` [PATCH v8 28/35] dyndbg-doc: explain flags parse 1st Jim Cromie
2024-04-29 19:39 ` [PATCH v8 29/35] dyndbg: add __counted_by annotations Jim Cromie
2024-04-30 18:49   ` jim.cromie
2024-04-29 19:39 ` [PATCH v8 30/35] drm+drivers: adapt to use DYNDBG_CLASSMAP_{DEFINE, USE} Jim Cromie
2024-04-29 19:39 ` [PATCH v8 31/35] drm-dyndbg: adapt to use DYNDBG_CLASSMAP_PARAM Jim Cromie
2024-04-29 19:39 ` [PATCH v8 32/35] drm: use correct ccflags-y spelling Jim Cromie
2024-04-29 19:39 ` [PATCH v8 33/35] drm-drivers: DRM_CLASSMAP_USE in 2nd batch of drivers, helpers Jim Cromie
2024-04-29 19:39 ` [PATCH v8 34/35] drm: restore CONFIG_DRM_USE_DYNAMIC_DEBUG un-BROKEN Jim Cromie
2024-04-29 19:39 ` [PATCH v8 35/35] drm-print: workaround compiler meh Jim Cromie
  -- strict thread matches above, loose matches on Subject: below --
2024-04-29 19:31 [PATCH v8 00/35] fix CONFIG_DRM_USE_DYNAMIC_DEBUG=y regression Jim Cromie
2024-04-29 19:31 ` [PATCH v8 21/35] dyndbg: treat comma as a token separator Jim Cromie

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox