[PATCH for-6.0 0/2] document gdbstub debugging of multicore machines

[PATCH for-6.0 0/2] document gdbstub debugging of multicore machines

[Peter Maydell]

The way gdb and our gdbstub handle multicore and multicluster
machines is not very obvious.  This patchset adds some documentation
of how to do it. In particular it gives the necessary runes
for how to get gdb to work with machines which have multiple
clusters of heterogenous CPUs (that's the sifive_u board, some
of the MPS2/MPS3 boards, and the Xilinx one).

I don't think there's any way for the stub to tell gdb
"I have 2 inferiors, attach to both of them", unfortunately,
so the user has to manually connect to them. (We should talk
to the gdb folks to confirm that and perhaps suggest protocol
enhancements. For 6.0, at least let's document the current state
of affairs.)

(We noticed the lack of docs as part of
https://bugs.launchpad.net/qemu/+bug/1921092)

thanks
-- PMM

Peter Maydell (2):
  docs/system/gdb.rst: Add some more heading structure
  docs/system/gdb.rst: Document how to debug multicore machines

 docs/system/gdb.rst | 63 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 62 insertions(+), 1 deletion(-)

-- 
2.20.1

[PATCH for-6.0 1/2] docs/system/gdb.rst: Add some more heading structure

[Peter Maydell]

We're about to add a new section to gdb.rst. In
preparation, add some more headings so it isn't just
one huge run-on section.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/gdb.rst | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/docs/system/gdb.rst b/docs/system/gdb.rst
index 72b1e68f4ef..0bb1bedf1be 100644
--- a/docs/system/gdb.rst
+++ b/docs/system/gdb.rst
@@ -45,7 +45,11 @@ Here are some useful tips in order to use gdb on system code:
 3. Use ``set architecture i8086`` to dump 16 bit code. Then use
    ``x/10i $cs*16+$eip`` to dump the code at the PC position.
 
-Advanced debugging options:
+Advanced debugging options
+==========================
+
+Changing single-stepping behaviour
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The default single stepping behavior is step with the IRQs and timer
 service routines off. It is set this way because when gdb executes a
@@ -88,6 +92,8 @@ three commands you can query and set the single step behavior:
       sending: "qemu.sstep=0x5"
       received: "OK"
 
+Examining physical memory
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Another feature that QEMU gdbstub provides is to toggle the memory GDB
 works with, by default GDB will show the current process memory respecting
-- 
2.20.1

[PATCH for-6.0 2/2] docs/system/gdb.rst: Document how to debug multicore machines

[Peter Maydell]

Document how multicore machines appear to GDB when debugged
via the debug stub. This is particularly non-intuitive for
the "multiple heterogenous clusters" case, but unfortunately
as far as I know there is no way with the remote protocol
for the stub to tell gdb "I have 2 inferiors, please connect
to both", so the user must set it all up manually.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 docs/system/gdb.rst | 55 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/docs/system/gdb.rst b/docs/system/gdb.rst
index 0bb1bedf1be..144d083df31 100644
--- a/docs/system/gdb.rst
+++ b/docs/system/gdb.rst
@@ -45,6 +45,61 @@ Here are some useful tips in order to use gdb on system code:
 3. Use ``set architecture i8086`` to dump 16 bit code. Then use
    ``x/10i $cs*16+$eip`` to dump the code at the PC position.
 
+Debugging multicore machines
+============================
+
+GDB's abstraction for debugging targets with multiple possible
+parallel flows of execution is a two layer one: it supports multiple
+"inferiors", each of which can have multiple "threads". When the QEMU
+machine has more than one CPU, QEMU exposes each CPU cluster as a
+separate "inferior", where each CPU within the cluster is a separate
+"thread". Most QEMU machine types have identical CPUs, so there is a
+single cluster which has all the CPUs in it.  A few machine types are
+heterogenous and have multiple clusters: for example the ``sifive_u``
+machine has a cluster with one E51 core and a second cluster with four
+U54 cores. Here the E51 is the only thread in the first inferior, and
+the U54 cores are all threads in the second inferior.
+
+When you connect gdb to the gdbstub, it will automatically
+connect to the first inferior; you can display the CPUs in this
+cluster using the gdb ``info thread`` command, and switch between
+them using gdb's usual thread-management commands.
+
+For multi-cluster machines, unfortunately gdb does not by default
+handle multiple inferiors, and so you have to explicitly connect
+to them. First, you must connect with the ``extended-remote``
+protocol, not ``remote``::
+
+    (gdb) target extended-remote localhost:1234
+
+Once connected, gdb will have a single inferior, for the
+first cluster. You need to create inferiors for the other
+clusters and attach to them, like this::
+
+  (gdb) add-inferior
+  Added inferior 2
+  (gdb) inferior 2
+  [Switching to inferior 2 [<null>] (<noexec>)]
+  (gdb) attach 2
+  Attaching to process 2
+  warning: No executable has been specified and target does not support
+  determining executable automatically.  Try using the "file" command.
+  0x00000000 in ?? ()
+
+Once you've done this, ``info threads`` will show CPUs in
+all the clusters you have attached to::
+
+  (gdb) info threads
+    Id   Target Id         Frame
+    1.1  Thread 1.1 (cortex-m33-arm-cpu cpu [running]) 0x00000000 in ?? ()
+  * 2.1  Thread 2.2 (cortex-m33-arm-cpu cpu [halted ]) 0x00000000 in ?? ()
+
+You probably also want to set gdb to ``schedule-multiple`` mode,
+so that when you tell gdb to ``continue`` it resumes all CPUs,
+not just those in the cluster you are currently working on::
+
+  (gdb) set schedule-multiple on
+
 Advanced debugging options
 ==========================
 
-- 
2.20.1

Re: [PATCH for-6.0 0/2] document gdbstub debugging of multicore machines

[Philippe Mathieu-Daudé]

On 3/25/21 6:50 PM, Peter Maydell wrote:

> Peter Maydell (2):
>   docs/system/gdb.rst: Add some more heading structure
>   docs/system/gdb.rst: Document how to debug multicore machines
> 
>  docs/system/gdb.rst | 63 ++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 62 insertions(+), 1 deletion(-)
> 

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH for-6.0 0/2] document gdbstub debugging of multicore machines

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> The way gdb and our gdbstub handle multicore and multicluster
> machines is not very obvious.  This patchset adds some documentation
> of how to do it. In particular it gives the necessary runes
> for how to get gdb to work with machines which have multiple
> clusters of heterogenous CPUs (that's the sifive_u board, some
> of the MPS2/MPS3 boards, and the Xilinx one).
>
> I don't think there's any way for the stub to tell gdb
> "I have 2 inferiors, attach to both of them", unfortunately,
> so the user has to manually connect to them. (We should talk
> to the gdb folks to confirm that and perhaps suggest protocol
> enhancements. For 6.0, at least let's document the current state
> of affairs.)

Queued to for-6.0/fixes-for-rc2, thanks.

-- 
Alex Bennée