Re: [PATCH v3 0/8] Introduce debugfs support in IOMMU

[Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>]

Hi,

I hope you're doing well.  Just a friendly reminder to kindly review the
patches at your earliest convenience. Your feedback is greatly appreciated.

Thanks
Dheeraj

On 2/6/2025 11:29 AM, Dheeraj Kumar Srivastava wrote:
> From: root <root@BLRDHSRIVAS.amd.com>
> 
> Introducing debugfs support in AMD/IOMMU driver that will allow
> userspace to dump below IOMMU information
> 1) MMIO and Capability register per IOMMU
> 2) Command buffer
> 3) Device table entry
> 4) Interrupt remapping table entry
> 
> Analyzing contents of IOMMU data structures helps in understanding IOMMU
> capabilities and behavior and debug issues faster.
> 
> 1. MMIO and Capability registers - Add support to dump MMIO and Capability
>     registers per IOMMU.
> 
>     Example:
>     a. Write MMIO register offset to dump it
>        $ echo 0x18 > /sys/kernel/debug/iommu/amd/iommu00/mmio
>        $ cat /sys/kernel/debug/iommu/amd/iommu00/mmio
> 
>        Output:
>        $ 0x0003f48d
> 
>      b. Write capability register offset to dump it
>         $ echo 0x10 > /sys/kernel/debug/iommu/amd/iommu00/capability
>         $ cat /sys/kernel/debug/iommu/amd/iommu00/capability
> 
>         Output:
>         $ 0x00203040
> 
> 2. Command buffer - Add support to dump per IOMMU command buffer.
> 
>     Example:
>     a. cat /sys/kernel/debug/iommu/amd/iommu00/cmdbuf
> 
>     Output:
>     CMD Buffer Head Offset:339 Tail Offset:339
>       0: 00835001100000010000990000000000
>       1: 0000000030000005fffff0037fffffff
>       2: 00835001100000010000990100000000
>       3: 0000000030000005fffff0037fffffff
>       4: 00835001100000010000990200000000
>     ...................................
>     ...................................
>     ...................................
> 
> 3. Device table - Add support to dump device table per IOMMU.
> 
>     Example:
>     a. Write device id to dump device table entry for that device
>        $ echo 0000:01:00.0 > /sys/kernel/debug/iommu/amd/devid
>        $ cat /sys/kernel/debug/iommu/amd/devid
> 
>        Output:
>        0000:01:00.0
> 
>        Dump the device table entry for the input given
>        $ cat /sys/kernel/debug/iommu/amd/devtbl
> 
>        Output:
>        DeviceId             QWORD[3]         QWORD[2]         QWORD[1]         QWORD[0] iommu
>        0000:01:00.0 0000000000000000 20000001373b8013 0000000000000038 6000000114d7b603 iommu3
> 
>      b. Write device id to dump device table entry for that device
>        $ echo 01:00.0 > /sys/kernel/debug/iommu/amd/devid
>        $ cat /sys/kernel/debug/iommu/amd/devid
> 
>        Output:
>        0000:01:00.0
> 
>        Dump the device table entry for the input given
>        $ cat /sys/kernel/debug/iommu/amd/devtbl
> 
>        Output:
>        DeviceId             QWORD[3]         QWORD[2]         QWORD[1]         QWORD[0] iommu
>        0000:01:00.0 0000000000000000 20000001373b8013 0000000000000038 6000000114d7b603 iommu3
> 
> 4. Interrupt remapping table - Add support to dump IRT table valid entries in
>     "iommu_irqtbl" file. This supports user input to dump IRT entry for a
>     specific pci device.
> 
>     Example:
>     a. Write device id to dump device table entry for that device
>        $ echo 0000:01:00.0 > /sys/kernel/debug/iommu/amd/devid
>        $ cat /sys/kernel/debug/iommu/amd/devid
> 
>        Output:
>        0000:01:00.0
> 
>        Dump the device table entry for the input given
>        $ cat /sys/kernel/debug/iommu/amd/irtbl
> 
>        Output:
>        DeviceId 0000:01:00.0
>        IRT[0000] 00000000000000200000000000000241
>        IRT[0001] 00000000000000200000000000000841
>        IRT[0002] 00000000000000200000000000002041
>        IRT[0003] 00000000000000200000000000008041
>        IRT[0004] 00000000000000200000000000020041
>        ..........................................
>        ..........................................
>        ..........................................
> 
>     b. Write device id to dump device table entry for that device
>        $ echo 01:00.0 > /sys/kernel/debug/iommu/amd/devid
>        $ cat /sys/kernel/debug/iommu/amd/devid
> 
>        Output:
>        0000:01:00.0
> 
>        Dump the device table entry for the input given
>        $ cat /sys/kernel/debug/iommu/amd/irttbl
> 
>        Output:
>        Device 0000:01:00.0
>        IRT[0000] 00000000000000200000000000000241
>        IRT[0001] 00000000000000200000000000000841
>        IRT[0002] 00000000000000200000000000002041
>        IRT[0003] 00000000000000200000000000008041
>        IRT[0004] 00000000000000200000000000020041
>        ..........................................
>        ..........................................
>        ..........................................
> 
> Open question:[Suggestion from Bjorn Helgaas] Instead of using device-id input with
> 	      "/sys/kernel/debug/iommu/amd/devid", can we have something like other drivers
> 	      have "/sys/kernel/debug/iommu/amd/<bdf>/".
> 
> 	      Each "/sys/kernel/debug/iommu/amd/<bdf>/" will have "devtbl" and "irqtbl" files
>                for device table and interrupt remapping table dump file.
> 
> 
> Changes since v2:
> -> Patch 2/8 and 3/8: Use single file mmio and capability for input and output registers
>     offset and registers dump respctively.
> -> Patch 8/8: Correct path for files "irqtbl" and "devtbl" in Documentation
> 
> Changes since v1:
> -> Patch 2/8 and 3/8: Use kstrtou32_from_user() instead of memdup_user_nul() --> kstrtou32()
> -> Patch 4/8: Dump command buffer head and tail offset instead of head and tail pointer registers.
> -> Patch 8/8: Fix bot reported warning on v1 upstream.
> 
> Dheeraj Kumar Srivastava (8):
>    iommu/amd: Refactor AMD IOMMU debugfs initial setup
>    iommu/amd: Add debugfs support to dump IOMMU MMIO registers
>    iommu/amd: Add debugfs support to dump IOMMU Capability registers
>    iommu/amd: Add debugfs support to dump IOMMU command buffer
>    iommu/amd: Add support for device id user input
>    iommu/amd: Add debugfs support to dump device table
>    iommu/amd: Add debugfs support to dump IRT Table
>    iommu/amd: Add documentation for AMD IOMMU debugfs support
> 
>   Documentation/ABI/testing/debugfs-amd-iommu | 114 +++++++
>   drivers/iommu/amd/amd_iommu.h               |   4 +-
>   drivers/iommu/amd/amd_iommu_types.h         |   7 +
>   drivers/iommu/amd/debugfs.c                 | 350 +++++++++++++++++++-
>   drivers/iommu/amd/init.c                    |   5 +-
>   drivers/iommu/amd/iommu.c                   |   7 -
>   6 files changed, 466 insertions(+), 21 deletions(-)
>   create mode 100644 Documentation/ABI/testing/debugfs-amd-iommu
>