Re: [PATCH v3 0/8] Introduce debugfs support in IOMMU

Dheeraj Kumar Srivastava
2025-03-07T08:17:18Z

Hi,

I hope you're doing well. Just a friendly reminder to kindly review the
patches at your earliest convenience. Your feedback is greatly appreciated.

Thanks
Dheeraj

On 2/6/2025 11:29 AM, Dheeraj Kumar Srivastava wrote:
> From: root <root@BLRDHSRIVAS.amd.com>
>
> Introducing debugfs support in AMD/IOMMU driver that will allow
> userspace to dump below IOMMU information
> 1) MMIO and Capability register per IOMMU
> 2) Command buffer
> 3) Device table entry
> 4) Interrupt remapping table entry
>
> Analyzing contents of IOMMU data structures helps in understanding IOMMU
> capabilities and behavior and debug issues faster.
>
> 1. MMIO and Capability registers - Add support to dump MMIO and Capability
> registers per IOMMU.
>
> Example:
> a. Write MMIO register offset to dump it
> $ echo 0x18 > /sys/kernel/debug/iommu/amd/iommu00/mmio
> $ cat /sys/kernel/debug/iommu/amd/iommu00/mmio
>
> Output:
> $ 0x0003f48d
>
> b. Write capability register offset to dump it
> $ echo 0x10 > /sys/kernel/debug/iommu/amd/iommu00/capability
> $ cat /sys/kernel/debug/iommu/amd/iommu00/capability
>
> Output:
> $ 0x00203040
>
> 2. Command buffer - Add support to dump per IOMMU command buffer.
>
> Example:
> a. cat /sys/kernel/debug/iommu/amd/iommu00/cmdbuf
>
> Output:
> CMD Buffer Head Offset:339 Tail Offset:339
> 0: 00835001100000010000990000000000
> 1: 0000000030000005fffff0037fffffff
> 2: 00835001100000010000990100000000
> 3: 0000000030000005fffff0037fffffff
> 4: 00835001100000010000990200000000
> ...................................
> ...................................
> ...................................
>
> 3. Device table - Add support to dump device table per IOMMU.
>
> Example:
> a. Write device id to dump device table entry for that device
> $ echo 0000:01:00.0 > /sys/kernel/debug/iommu/amd/devid
> $ cat /sys/kernel/debug/iommu/amd/devid
>
> Output:
> 0000:01:00.0
>
> Dump the device table entry for the input given
> $ cat /sys/kernel/debug/iommu/amd/devtbl
>
> Output:
> DeviceId QWORD[3] QWORD[2] QWORD[1] QWORD[0] iommu
> 0000:01:00.0 0000000000000000 20000001373b8013 0000000000000038 6000000114d7b603 iommu3
>
> b. Write device id to dump device table entry for that device
> $ echo 01:00.0 > /sys/kernel/debug/iommu/amd/devid
> $ cat /sys/kernel/debug/iommu/amd/devid
>
> Output:
> 0000:01:00.0
>
> Dump the device table entry for the input given
> $ cat /sys/kernel/debug/iommu/amd/devtbl
>
> Output:
> DeviceId QWORD[3] QWORD[2] QWORD[1] QWORD[0] iommu
> 0000:01:00.0 0000000000000000 20000001373b8013 0000000000000038 6000000114d7b603 iommu3
>
> 4. Interrupt remapping table - Add support to dump IRT table valid entries in
> "iommu_irqtbl" file. This supports user input to dump IRT entry for a
> specific pci device.
>
> Example:
> a. Write device id to dump device table entry for that device
> $ echo 0000:01:00.0 > /sys/kernel/debug/iommu/amd/devid
> $ cat /sys/kernel/debug/iommu/amd/devid
>
> Output:
> 0000:01:00.0
>
> Dump the device table entry for the input given
> $ cat /sys/kernel/debug/iommu/amd/irtbl
>
> Output:
> DeviceId 0000:01:00.0
> IRT[0000] 00000000000000200000000000000241
> IRT[0001] 00000000000000200000000000000841
> IRT[0002] 00000000000000200000000000002041
> IRT[0003] 00000000000000200000000000008041
> IRT[0004] 00000000000000200000000000020041
> ..........................................
> ..........................................
> ..........................................
>
> b. Write device id to dump device table entry for that device
> $ echo 01:00.0 > /sys/kernel/debug/iommu/amd/devid
> $ cat /sys/kernel/debug/iommu/amd/devid
>
> Output:
> 0000:01:00.0
>
> Dump the device table entry for the input given
> $ cat /sys/kernel/debug/iommu/amd/irttbl
>
> Output:
> Device 0000:01:00.0
> IRT[0000] 00000000000000200000000000000241
> IRT[0001] 00000000000000200000000000000841
> IRT[0002] 00000000000000200000000000002041
> IRT[0003] 00000000000000200000000000008041
> IRT[0004] 00000000000000200000000000020041
> ..........................................
> ..........................................
> ..........................................
>
> Open question:[Suggestion from Bjorn Helgaas] Instead of using device-id input with
> "/sys/kernel/debug/iommu/amd/devid", can we have something like other drivers
> have "/sys/kernel/debug/iommu/amd/<bdf>/".
>
> Each "/sys/kernel/debug/iommu/amd/<bdf>/" will have "devtbl" and "irqtbl" files
> for device table and interrupt remapping table dump file.
>
>
> Changes since v2:
> -> Patch 2/8 and 3/8: Use single file mmio and capability for input and output registers
> offset and registers dump respctively.
> -> Patch 8/8: Correct path for files "irqtbl" and "devtbl" in Documentation
>
> Changes since v1:
> -> Patch 2/8 and 3/8: Use kstrtou32_from_user() instead of memdup_user_nul() --> kstrtou32()
> -> Patch 4/8: Dump command buffer head and tail offset instead of head and tail pointer registers.
> -> Patch 8/8: Fix bot reported warning on v1 upstream.
>
> Dheeraj Kumar Srivastava (8):
> iommu/amd: Refactor AMD IOMMU debugfs initial setup
> iommu/amd: Add debugfs support to dump IOMMU MMIO registers
> iommu/amd: Add debugfs support to dump IOMMU Capability registers
> iommu/amd: Add debugfs support to dump IOMMU command buffer
> iommu/amd: Add support for device id user input
> iommu/amd: Add debugfs support to dump device table
> iommu/amd: Add debugfs support to dump IRT Table
> iommu/amd: Add documentation for AMD IOMMU debugfs support
>
> Documentation/ABI/testing/debugfs-amd-iommu | 114 +++++++
> drivers/iommu/amd/amd_iommu.h | 4 +-
> drivers/iommu/amd/amd_iommu_types.h | 7 +
> drivers/iommu/amd/debugfs.c | 350 +++++++++++++++++++-
> drivers/iommu/amd/init.c | 5 +-
> drivers/iommu/amd/iommu.c | 7 -
> 6 files changed, 466 insertions(+), 21 deletions(-)
> create mode 100644 Documentation/ABI/testing/debugfs-amd-iommu
>