Re: Inquiring about Debugging Platform Drivers using Crash Utility for Kernel Coredump

[Omar Sandoval <osandov@osandov.com>]

On Tue, Jun 20, 2023 at 01:47:10PM +0300, Shenhar, Talel wrote:
> Dear Linux Kernel Community,
> 
> I hope this message finds you well.
> 
> I'd like to use crash utility for postmortem of my kernel coredump analysis.
> 
> I was able to collect coredump and able to use various operation from within
> the crash utility such as irq -s,  log, files and others.
> 
> I am using: crash-arm64 version: 7.3.0, gdb version: 7.6, kernel version
> 4.19.
> 
> My specific interest lies in debugging drivers internal state, e.g. platform
> drivers.
> 
> For some hands-on experience with crash utility I'd like to start by
> iterating over all the platform drivers and print their names,
> 
> However, I am finding it challenging to get started with this process and I
> am uncertain of the best approach to achieve this. I have scoured various
> resources for insights, but the information related to this specific usage
> seems to be scattered and not exhaustive.
> 
> Given the collective expertise on this mailing list, I thought it would be
> the best place to seek guidance. Specifically, I would appreciate it if you
> could provide:
> 
> Any relevant documentation, guides, or tutorials to debug platform drivers
> using the crash utility for kernel coredump analysis.
> Some simple examples of using the crash utility to debug platform drivers,
> if possible.
> Any important points or common pitfalls to keep in mind while performing
> this kind of analysis.
> Any other tips, best practices, or recommendations to effectively debug
> platform drivers using the crash utility would also be greatly appreciated.
> 
> Thank you for your time and assistance. I look forward to hearing from you.

Hi, Talel,

The only thing I have to add to Stephen's excellent answer is my attempt
at getting the information you requested with drgn. I'm not very
familiar with platform drivers, so I basically read the code for
platform_driver_register() and translated the relevant parts to drgn.
Something like this should get you started:

------------------------------------------------------------------------
from drgn import NULL, container_of
from drgn.helpers.linux.list import list_for_each_entry


# This was directly translated from the bus_to_subsys() function in
# drivers/base/bus.c of the Linux kernel. We should probably add it as a
# drgn helper.
def bus_to_subsys(bus):
    for sp in list_for_each_entry(
        "struct subsys_private",
        prog["bus_kset"].list.address_of_(),
        "subsys.kobj.entry",
    ):
        if sp.bus == bus:
            return sp
    return NULL(bus.prog_, "struct subsys_private *")


# Platform drivers are registered to the struct bus_type
# platform_bus_type in drivers/base/platform.c. The struct
# subsys_private has a kset containing a list of drivers.
sp = bus_to_subsys(prog["platform_bus_type"].address_of_())
for priv in list_for_each_entry(
    "struct driver_private", sp.drivers_kset.list.address_of_(), "kobj.entry"
):
    # This is a struct device_driver *.
    driver = priv.driver
    # To get the struct platform_driver *, do:
    # platform_driver = container_of(driver, "struct platform_driver", "driver")
    print(driver.name.string_().decode())
------------------------------------------------------------------------

(I also pushed this script to the contrib directory of the drgn
repository:
https://github.com/osandov/drgn/blob/main/contrib/platform_drivers.py)

On my ARM64 QEMU VM, this prints:

------------------------------------------------------------------------
sbsa-uart
alarmtimer
simple-pm-bus
pci-host-generic
of_fixed_factor_clk
of_fixed_clk
gpio-clk
------------------------------------------------------------------------

Hopefully this helps!