[PATCH v11 00/28] Add gdb python scripts as kernel debugging helpers

[PATCH v11 00/28] Add gdb python scripts as kernel debugging helpers

[Jan Kiszka]

Hi Andrew,

here comes the requested update of the series.

Changes since v10:
 - rebase over recent Linus master (rc6+)
 - fix stuck gdb pagination during modprobe related symbol reloading

See http://lkml.indiana.edu/hypermail/linux/kernel/1210.0/01598.html for
the original description and

    git://git.kiszka.org/linux.git queues/gdb-scripts

for the latest version.

On the gdb upstream front there is good and bad news again. The good
news is that the regression reported in the last round was fixed now
[1]. The bad news is that upcoming 7.9 has a new regression that is even
causing a crash of the target kernel [2]. So the recommendation is still
to stick with gdb 7.7.

Jan

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=17642
[2] https://sourceware.org/bugzilla/show_bug.cgi?id=17837

CC: Daniel Thompson <daniel.thompson@linaro.org>
CC: Daniel Wagner <daniel.wagner@bmw-carit.de>
CC: "David S. Miller" <davem@davemloft.net>
CC: Fenghua Yu <fenghua.yu@intel.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
CC: Kay Sievers <kay@vrfy.org>
CC: linux-doc@vger.kernel.org
CC: linux-ia64@vger.kernel.org
Cc: linux-kbuild@vger.kernel.org
CC: linux-kbuild@vger.kernel.org
Cc: Michal Marek <mmarek@suse.cz>
CC: Michal Marek <mmarek@suse.cz>
CC: Pantelis Koukousoulas <pktoss@gmail.com>
CC: Rob Landley <rob@landley.net>
CC: Rusty Russell <rusty@rustcorp.com.au>
CC: sparclinux@vger.kernel.org
CC: Tony Luck <tony.luck@intel.com>

Daniel Thompson (1):
  scripts/gdb: Ignore byte-compiled python files

Daniel Wagner (1):
  scripts/gdb: Use a generator instead of iterator for task list

Jan Kiszka (25):
  scripts/gdb: Add infrastructure
  scripts/gdb: Add cache for type objects
  scripts/gdb: Add container_of helper and convenience function
  scripts/gdb: Add module iteration class
  scripts/gdb: Add lx-symbols command
  module: Do not inline do_init_module
  scripts/gdb: Add automatic symbol reloading on module insertion
  scripts/gdb: Add internal helper and convenience function to look up a
    module
  scripts/gdb: Add get_target_endianness helper
  scripts/gdb: Add read_u16/32/64 helpers
  scripts/gdb: Add lx-dmesg command
  scripts/gdb: Add task iteration class
  scripts/gdb: Add helper and convenience function to look up tasks
  scripts/gdb: Add is_target_arch helper
  scripts/gdb: Add internal helper and convenience function to retrieve
    thread_info
  scripts/gdb: Add get_gdbserver_type helper
  scripts/gdb: Add internal helper and convenience function for per-cpu
    lookup
  scripts/gdb: Add lx_current convenience function
  scripts/gdb: Add class to iterate over CPU masks
  scripts/gdb: Add lx-lsmod command
  scripts/gdb: Add basic documentation
  scripts/gdb: Convert ModuleList to generator function
  scripts/gdb: Convert CpuList to generator function
  scripts/gdb: Define maintainer
  scripts/gdb: Disable pagination while printing from breakpoint handler

Pantelis Koukousoulas (1):
  scripts/gdb: Port to python3 / gdb7.7

 .gitignore                             |   1 +
 Documentation/gdb-kernel-debugging.txt | 160 +++++++++++++++++++++++++++++
 MAINTAINERS                            |   5 +
 Makefile                               |   6 +-
 kernel/module.c                        |   9 +-
 lib/Kconfig.debug                      |  11 ++
 scripts/Makefile                       |   3 +-
 scripts/gdb/Makefile                   |   1 +
 scripts/gdb/linux/.gitignore           |   2 +
 scripts/gdb/linux/Makefile             |  11 ++
 scripts/gdb/linux/__init__.py          |   0
 scripts/gdb/linux/cpus.py              | 135 +++++++++++++++++++++++++
 scripts/gdb/linux/dmesg.py             |  65 ++++++++++++
 scripts/gdb/linux/modules.py           | 103 +++++++++++++++++++
 scripts/gdb/linux/symbols.py           | 177 +++++++++++++++++++++++++++++++++
 scripts/gdb/linux/tasks.py             | 100 +++++++++++++++++++
 scripts/gdb/linux/utils.py             | 156 +++++++++++++++++++++++++++++
 scripts/gdb/vmlinux-gdb.py             |  30 ++++++
 18 files changed, 971 insertions(+), 4 deletions(-)
 create mode 100644 Documentation/gdb-kernel-debugging.txt
 create mode 100644 scripts/gdb/Makefile
 create mode 100644 scripts/gdb/linux/.gitignore
 create mode 100644 scripts/gdb/linux/Makefile
 create mode 100644 scripts/gdb/linux/__init__.py
 create mode 100644 scripts/gdb/linux/cpus.py
 create mode 100644 scripts/gdb/linux/dmesg.py
 create mode 100644 scripts/gdb/linux/modules.py
 create mode 100644 scripts/gdb/linux/symbols.py
 create mode 100644 scripts/gdb/linux/tasks.py
 create mode 100644 scripts/gdb/linux/utils.py
 create mode 100644 scripts/gdb/vmlinux-gdb.py

-- 
2.1.4

[PATCH v11 01/28] scripts/gdb: Add infrastructure

[Jan Kiszka]

This provides the basic infrastructure to load kernel-specific python
helper scripts when debugging the kernel in gdb.

The loading mechanism is based on gdb loading for <objfile>-gdb.py when
opening <objfile>. Therefore, this places a corresponding link to the
main helper script into the output directory that contains vmlinux.

The main scripts will pull in submodules containing Linux specific gdb
commands and functions. To avoid polluting the source directory with
compiled python modules, we link to them from the object directory.

Due to gdb.parse_and_eval and string redirection for gdb.execute, we
depend on gdb >= 7.2.

This feature is enabled via CONFIG_GDB_SCRIPTS.

CC: Michal Marek <mmarek@suse.cz>
CC: linux-kbuild@vger.kernel.org
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 Makefile                      |  5 ++++-
 lib/Kconfig.debug             | 11 +++++++++++
 scripts/Makefile              |  3 ++-
 scripts/gdb/Makefile          |  1 +
 scripts/gdb/linux/Makefile    | 11 +++++++++++
 scripts/gdb/linux/__init__.py |  0
 scripts/gdb/vmlinux-gdb.py    | 23 +++++++++++++++++++++++
 7 files changed, 52 insertions(+), 2 deletions(-)
 create mode 100644 scripts/gdb/Makefile
 create mode 100644 scripts/gdb/linux/Makefile
 create mode 100644 scripts/gdb/linux/__init__.py
 create mode 100644 scripts/gdb/vmlinux-gdb.py

diff --git a/Makefile b/Makefile
index 95a0e82..6881ddf 100644
--- a/Makefile
+++ b/Makefile
@@ -922,6 +922,9 @@ endif
 ifdef CONFIG_BUILD_DOCSRC
 	$(Q)$(MAKE) $(build)=Documentation
 endif
+ifdef CONFIG_GDB_SCRIPTS
+	$(Q)ln -fsn `cd $(srctree) && /bin/pwd`/scripts/gdb/vmlinux-gdb.py
+endif
 	+$(call if_changed,link-vmlinux)
 
 # The actual objects are generated when descending,
@@ -1176,7 +1179,7 @@ MRPROPER_FILES += .config .config.old .version .old_version $(version_h) \
 		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
 		  signing_key.priv signing_key.x509 x509.genkey		\
 		  extra_certificates signing_key.x509.keyid		\
-		  signing_key.x509.signer
+		  signing_key.x509.signer vmlinux-gdb.py
 
 # clean - Delete most, but leave enough to build external modules
 #
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 5f2ce61..74984af 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -167,6 +167,17 @@ config DEBUG_INFO_DWARF4
 	  But it significantly improves the success of resolving
 	  variables in gdb on optimized code.
 
+config GDB_SCRIPTS
+	bool "Provide GDB scripts for kernel debugging"
+	select DEBUG_INFO
+	help
+	  This creates the required links to GDB helper scripts in the
+	  build directory. If you load vmlinux into gdb, the helper
+	  scripts will be automatically imported by gdb as well, and
+	  additional functions are available to analyze a Linux kernel
+	  instance. See Documentation/gdb-kernel-debugging.txt for further
+	  details.
+
 config ENABLE_WARN_DEPRECATED
 	bool "Enable __deprecated logic"
 	default y
diff --git a/scripts/Makefile b/scripts/Makefile
index 72902b5..7413b1b 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -36,6 +36,7 @@ subdir-$(CONFIG_MODVERSIONS) += genksyms
 subdir-y                     += mod
 subdir-$(CONFIG_SECURITY_SELINUX) += selinux
 subdir-$(CONFIG_DTC)         += dtc
+subdir-$(CONFIG_GDB_SCRIPTS) += gdb
 
 # Let clean descend into subdirs
-subdir-	+= basic kconfig package
+subdir-	+= basic kconfig package gdb
diff --git a/scripts/gdb/Makefile b/scripts/gdb/Makefile
new file mode 100644
index 0000000..62f5f65
--- /dev/null
+++ b/scripts/gdb/Makefile
@@ -0,0 +1 @@
+subdir-y := linux
diff --git a/scripts/gdb/linux/Makefile b/scripts/gdb/linux/Makefile
new file mode 100644
index 0000000..dcb63ab
--- /dev/null
+++ b/scripts/gdb/linux/Makefile
@@ -0,0 +1,11 @@
+always := gdb-scripts
+
+SRCTREE := $(shell cd $(srctree) && /bin/pwd)
+
+$(obj)/gdb-scripts:
+ifneq ($(KBUILD_SRC),)
+	$(Q)ln -fsn $(SRCTREE)/$(obj)/*.py $(objtree)/$(obj)
+endif
+	@:
+
+clean-files := *.pyc $(if $(KBUILD_SRC),*.py)
diff --git a/scripts/gdb/linux/__init__.py b/scripts/gdb/linux/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/scripts/gdb/vmlinux-gdb.py b/scripts/gdb/vmlinux-gdb.py
new file mode 100644
index 0000000..c1d90ce
--- /dev/null
+++ b/scripts/gdb/vmlinux-gdb.py
@@ -0,0 +1,23 @@
+#
+# gdb helper commands and functions for Linux kernel debugging
+#
+#  loader module
+#
+# Copyright (c) Siemens AG, 2012, 2013
+#
+# Authors:
+#  Jan Kiszka <jan.kiszka@siemens.com>
+#
+# This work is licensed under the terms of the GNU GPL version 2.
+#
+
+import os
+
+sys.path.insert(0, os.path.dirname(__file__) + "/scripts/gdb")
+
+try:
+    gdb.parse_and_eval("0")
+    gdb.execute("", to_string=True)
+except:
+    gdb.write("NOTE: gdb 7.2 or later required for Linux helper scripts to "
+              "work.\n")
-- 
2.1.4

Re: [PATCH v11 01/28] scripts/gdb: Add infrastructure

[Michal Marek]

On 2015-01-29 07:46, Jan Kiszka wrote:
> --- a/scripts/Makefile
> +++ b/scripts/Makefile
> @@ -36,6 +36,7 @@ subdir-$(CONFIG_MODVERSIONS) += genksyms
>  subdir-y                     += mod
>  subdir-$(CONFIG_SECURITY_SELINUX) += selinux
>  subdir-$(CONFIG_DTC)         += dtc
> +subdir-$(CONFIG_GDB_SCRIPTS) += gdb
>  
>  # Let clean descend into subdirs
> -subdir-	+= basic kconfig package
> +subdir-	+= basic kconfig package gdb

The second assignment is superfluous. basic, kconfig and package are
listed there, because they are built explicitly by the toplevel Makefile
and not during the recursion.

Michal

Re: [PATCH v11 01/28] scripts/gdb: Add infrastructure

[Jan Kiszka]

On 2015-01-29 14:23, Michal Marek wrote:
> On 2015-01-29 07:46, Jan Kiszka wrote:
>> --- a/scripts/Makefile
>> +++ b/scripts/Makefile
>> @@ -36,6 +36,7 @@ subdir-$(CONFIG_MODVERSIONS) += genksyms
>>  subdir-y                     += mod
>>  subdir-$(CONFIG_SECURITY_SELINUX) += selinux
>>  subdir-$(CONFIG_DTC)         += dtc
>> +subdir-$(CONFIG_GDB_SCRIPTS) += gdb
>>  
>>  # Let clean descend into subdirs
>> -subdir-	+= basic kconfig package
>> +subdir-	+= basic kconfig package gdb
> 
> The second assignment is superfluous. basic, kconfig and package are
> listed there, because they are built explicitly by the toplevel Makefile
> and not during the recursion.

Left-over from the pre-CONFIG_GDB_SCRIPTS times. Update will follow.

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

[PATCH v11 23/28] scripts/gdb: Ignore byte-compiled python files

[Jan Kiszka]

From: Daniel Thompson <daniel.thompson@linaro.org>

Using the gdb scripts leaves byte-compiled python files in the scripts/
directory. These should be ignored by git and cleaned up by mrproper.

Signed-off-by: Daniel Thompson <daniel.thompson@linaro.org>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 .gitignore                   | 1 +
 Makefile                     | 3 ++-
 scripts/gdb/linux/.gitignore | 2 ++
 3 files changed, 5 insertions(+), 1 deletion(-)
 create mode 100644 scripts/gdb/linux/.gitignore

diff --git a/.gitignore b/.gitignore
index ce57b79..ecb89e3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -43,6 +43,7 @@ Module.symvers
 /TAGS
 /linux
 /vmlinux
+/vmlinux-gdb.py
 /vmlinuz
 /System.map
 /Module.markers
diff --git a/Makefile b/Makefile
index 6881ddf..ee7394e 100644
--- a/Makefile
+++ b/Makefile
@@ -1179,7 +1179,8 @@ MRPROPER_FILES += .config .config.old .version .old_version $(version_h) \
 		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
 		  signing_key.priv signing_key.x509 x509.genkey		\
 		  extra_certificates signing_key.x509.keyid		\
-		  signing_key.x509.signer vmlinux-gdb.py
+		  signing_key.x509.signer vmlinux-gdb.py		\
+		  scripts/gdb/linux/*.py[co]
 
 # clean - Delete most, but leave enough to build external modules
 #
diff --git a/scripts/gdb/linux/.gitignore b/scripts/gdb/linux/.gitignore
new file mode 100644
index 0000000..52e4e61
--- /dev/null
+++ b/scripts/gdb/linux/.gitignore
@@ -0,0 +1,2 @@
+*.pyc
+*.pyo
-- 
2.1.4

Re: [PATCH v11 23/28] scripts/gdb: Ignore byte-compiled python files

[Michal Marek]

On 2015-01-29 07:46, Jan Kiszka wrote:
> --- a/Makefile
> +++ b/Makefile
> @@ -1179,7 +1179,8 @@ MRPROPER_FILES += .config .config.old .version .old_version $(version_h) \
>  		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
>  		  signing_key.priv signing_key.x509 x509.genkey		\
>  		  extra_certificates signing_key.x509.keyid		\
> -		  signing_key.x509.signer vmlinux-gdb.py
> +		  signing_key.x509.signer vmlinux-gdb.py		\
> +		  scripts/gdb/linux/*.py[co]

In 01/28, you already have

+clean-files := *.pyc $(if $(KBUILD_SRC),*.py)

Maybe just add *.pyo there and you don't need to add it to the toplevel
Makefile.

Michal

Re: [PATCH v11 23/28] scripts/gdb: Ignore byte-compiled python files

[Jan Kiszka]

On 2015-01-29 14:15, Michal Marek wrote:
> On 2015-01-29 07:46, Jan Kiszka wrote:
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -1179,7 +1179,8 @@ MRPROPER_FILES += .config .config.old .version .old_version $(version_h) \
>>  		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
>>  		  signing_key.priv signing_key.x509 x509.genkey		\
>>  		  extra_certificates signing_key.x509.keyid		\
>> -		  signing_key.x509.signer vmlinux-gdb.py
>> +		  signing_key.x509.signer vmlinux-gdb.py		\
>> +		  scripts/gdb/linux/*.py[co]
> 
> In 01/28, you already have
> 
> +clean-files := *.pyc $(if $(KBUILD_SRC),*.py)
> 
> Maybe just add *.pyo there and you don't need to add it to the toplevel
> Makefile.

Based on a quick test, you seem right. Daniel, anything we miss?

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v11 23/28] scripts/gdb: Ignore byte-compiled python files

[Daniel Thompson]

On 29/01/15 13:35, Jan Kiszka wrote:
> On 2015-01-29 14:15, Michal Marek wrote:
>> On 2015-01-29 07:46, Jan Kiszka wrote:
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -1179,7 +1179,8 @@ MRPROPER_FILES += .config .config.old .version .old_version $(version_h) \
>>>  		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
>>>  		  signing_key.priv signing_key.x509 x509.genkey		\
>>>  		  extra_certificates signing_key.x509.keyid		\
>>> -		  signing_key.x509.signer vmlinux-gdb.py
>>> +		  signing_key.x509.signer vmlinux-gdb.py		\
>>> +		  scripts/gdb/linux/*.py[co]
>>
>> In 01/28, you already have
>>
>> +clean-files := *.pyc $(if $(KBUILD_SRC),*.py)
>>
>> Maybe just add *.pyo there and you don't need to add it to the toplevel
>> Makefile.
> 
> Based on a quick test, you seem right. Daniel, anything we miss?

No. Deleting it with the clean rule makes sense.

Of course the .gitignore parts of my patch will still be needed in some
form or other.


Daniel.

Re: [PATCH v11 23/28] scripts/gdb: Ignore byte-compiled python files

[Jan Kiszka]

On 2015-01-29 15:34, Daniel Thompson wrote:
> On 29/01/15 13:35, Jan Kiszka wrote:
>> On 2015-01-29 14:15, Michal Marek wrote:
>>> On 2015-01-29 07:46, Jan Kiszka wrote:
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -1179,7 +1179,8 @@ MRPROPER_FILES += .config .config.old .version .old_version $(version_h) \
>>>>  		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
>>>>  		  signing_key.priv signing_key.x509 x509.genkey		\
>>>>  		  extra_certificates signing_key.x509.keyid		\
>>>> -		  signing_key.x509.signer vmlinux-gdb.py
>>>> +		  signing_key.x509.signer vmlinux-gdb.py		\
>>>> +		  scripts/gdb/linux/*.py[co]
>>>
>>> In 01/28, you already have
>>>
>>> +clean-files := *.pyc $(if $(KBUILD_SRC),*.py)
>>>
>>> Maybe just add *.pyo there and you don't need to add it to the toplevel
>>> Makefile.
>>
>> Based on a quick test, you seem right. Daniel, anything we miss?
> 
> No. Deleting it with the clean rule makes sense.
> 
> Of course the .gitignore parts of my patch will still be needed in some
> form or other.

Sure. Can you provide an updated version, one that augments the
clean-files rule instead? Then I'll sign-off and include it in a
potential follow-up round.

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux