[PATCH] Add hard/soft lockup debugger entry points

[PATCH] Add hard/soft lockup debugger entry points

[Jeff Merkey]

This patch adds an export which can be set by system debuggers to direct
the hard lockup and soft lockup detector to trigger an INT3 exception
and enter a debugger if one is active.  It is assumed that if someone
sets this variable, then an int3 handler of some sort will be actively
loaded or registered via the notify die handler chain.

This addition is extremely useful for debugging hard and soft lockups
real time and quickly from a console debugger.

Signed-off-by: Jeff Merkey <linux.mdb@gmail.com>
---
 kernel/watchdog.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index b3ace6e..f927722 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -108,6 +108,9 @@ static DEFINE_PER_CPU(struct perf_event *, watchdog_ev);
 #endif
 static unsigned long soft_lockup_nmi_warn;
 
+int debug_watchdog_lockups;
+EXPORT_SYMBOL_GPL(debug_watchdog_lockups);
+
 /* boot commands */
 /*
  * Should we panic when a soft-lockup or hard-lockup occurs:
@@ -358,6 +361,9 @@ static void watchdog_overflow_callback(struct perf_event *event,
 		else
 			dump_stack();
 
+		if (debug_watchdog_lockups)
+			__asm__ __volatile__ ("int $0x03");
+
 		/*
 		 * Perform all-CPU dump only once to avoid multiple hardlockups
 		 * generating interleaving traces
@@ -478,6 +484,9 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
 		else
 			dump_stack();
 
+		if (debug_watchdog_lockups)
+			__asm__ __volatile__ ("int $0x03");
+
 		if (softlockup_all_cpu_backtrace) {
 			/* Avoid generating two back traces for current
 			 * given that one is already made above
-- 
1.8.3.1

Re: [PATCH] Add hard/soft lockup debugger entry points

[Chris Metcalf]

On 1/25/2016 9:36 PM, Jeff Merkey wrote:
> This patch adds an export which can be set by system debuggers to direct
> the hard lockup and soft lockup detector to trigger an INT3 exception
> and enter a debugger if one is active.  It is assumed that if someone
> sets this variable, then an int3 handler of some sort will be actively
> loaded or registered via the notify die handler chain.
>
> This addition is extremely useful for debugging hard and soft lockups
> real time and quickly from a console debugger.
>
> Signed-off-by: Jeff Merkey<linux.mdb@gmail.com>
> ---
>   kernel/watchdog.c | 9 +++++++++
>   1 file changed, 9 insertions(+)

Introducing Intel-specific assembly into generic code won't work. You would
need to add a framework to allow platforms to specify a debugging instruction
or maybe leverage some existing framework like arch_kgdb_breakpoint.

-- 
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com

Re: [PATCH] Add hard/soft lockup debugger entry points

[Jeff Merkey]

On 1/25/16, Chris Metcalf <cmetcalf@ezchip.com> wrote:
> On 1/25/2016 9:36 PM, Jeff Merkey wrote:
>> This patch adds an export which can be set by system debuggers to direct
>> the hard lockup and soft lockup detector to trigger an INT3 exception
>> and enter a debugger if one is active.  It is assumed that if someone
>> sets this variable, then an int3 handler of some sort will be actively
>> loaded or registered via the notify die handler chain.
>>
>> This addition is extremely useful for debugging hard and soft lockups
>> real time and quickly from a console debugger.
>>
>> Signed-off-by: Jeff Merkey<linux.mdb@gmail.com>
>> ---
>>   kernel/watchdog.c | 9 +++++++++
>>   1 file changed, 9 insertions(+)
>
> Introducing Intel-specific assembly into generic code won't work. You would
> need to add a framework to allow platforms to specify a debugging
> instruction
> or maybe leverage some existing framework like arch_kgdb_breakpoint.
>
> --
> Chris Metcalf, EZChip Semiconductor
> http://www.ezchip.com
>
>

Yep.  I'll start on v2.

Jeff

Re: [PATCH] Add hard/soft lockup debugger entry points

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

Hi Jeff,

[auto build test ERROR on v4.5-rc1]
[also build test ERROR on next-20160125]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Jeff-Merkey/Add-hard-soft-lockup-debugger-entry-points/20160126-103801
config: powerpc-defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=powerpc 

All errors (new ones prefixed by >>):

   {standard input}: Assembler messages:
>> {standard input}:550: Error: unrecognized opcode: `int'
   {standard input}:1121: Error: unrecognized opcode: `int'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 21912 bytes --]

Re: [PATCH] Add hard/soft lockup debugger entry points

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

Hi Jeff,

[auto build test ERROR on v4.5-rc1]
[also build test ERROR on next-20160125]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Jeff-Merkey/Add-hard-soft-lockup-debugger-entry-points/20160126-103801
config: arm64-defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=arm64 

All errors (new ones prefixed by >>):

   /tmp/cc3NhccY.s: Assembler messages:
>> /tmp/cc3NhccY.s:1030: Error: unknown mnemonic `int' -- `int $0x03'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 20865 bytes --]

Re: [PATCH] Add hard/soft lockup debugger entry points

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

Hi Jeff,

[auto build test ERROR on v4.5-rc1]
[also build test ERROR on next-20160125]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Jeff-Merkey/Add-hard-soft-lockup-debugger-entry-points/20160126-103801
config: sparc64-defconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=sparc64 

All errors (new ones prefixed by >>):

   {standard input}: Assembler messages:
>> {standard input}:335: Error: Unknown opcode: `int'

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 16661 bytes --]

Re: [PATCH] Add hard/soft lockup debugger entry points

[Jeff Merkey]

On 1/25/16, kbuild test robot <lkp@intel.com> wrote:
> Hi Jeff,
>
> [auto build test ERROR on v4.5-rc1]
> [also build test ERROR on next-20160125]
> [if your patch is applied to the wrong git tree, please drop us a note to
> help improving the system]
>
> url:
> https://github.com/0day-ci/linux/commits/Jeff-Merkey/Add-hard-soft-lockup-debugger-entry-points/20160126-103801
> config: sparc64-defconfig (attached as .config)
> reproduce:
>         wget
> https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross
> -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         make.cross ARCH=sparc64
>
> All errors (new ones prefixed by >>):
>
>    {standard input}: Assembler messages:
>>> {standard input}:335: Error: Unknown opcode: `int'
>
> ---
> 0-DAY kernel test infrastructure                Open Source Technology
> Center
> https://lists.01.org/pipermail/kbuild-all                   Intel
> Corporation
>

I am resubmitting this with the correct arch_hw_bp.  Wow, That test
robot is very efficient.

Jeff